Control device, measurement device, terminal, server, meal measurement system, control method, and program

Abstract

To improve the accuracy of meal measurement more than before.SOLUTION: The control device includes an information acquisition unit that acquires measurement target information indicating presence of a measurement target person and acquires meal determination information which is information regarding a meal using a captured image or sensor information, a meal determination unit that outputs a determination result indicating that a meal is being taken when a composite condition in which a plurality of conditions are combined is satisfied based on the measurement target information and the meal determination information, and a control instruction unit that outputs a control signal for executing measurement of the meal when the determination result indicates that the meal is being taken.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The disclosed technology relates to dietary measurement technology. [Background technology]

[0002] Dietary measurement technology is a technology for measuring the diet of a subject. Patent Document 1 discloses a technique ("dietary advice system") for measuring dietary intake using a video captured by a camera of a subject eating a meal. In the "Dietary Advice System" of Patent Document 1, "the photographing device can be configured to be activated by detecting that food has been placed on the dining table or that the subject has sat down, etc." However, Patent Document 1 does not describe the specific configuration of the detection means that detects "that food has been placed on the dining table" or "that the subject has sat down" in order to activate the photographing device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2017-054163 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the "dietary advice system" of Patent Document 1, even if it is possible to detect that food has been placed on the dining table, there are cases where simply having food on the table does not mean that a meal is being served. Also, there are cases where simply having a subject sit down does not mean that a meal is being served. In other words, conventional technology has the problem that even when a person is not actually eating, it performs meal-related measurements and outputs the measurement results as if the person is eating, so there is room for improvement in the accuracy of meal measurements.

[0005] The present disclosure is intended to solve the above-mentioned problems and aims to improve the accuracy of meal measurement compared to conventional methods. [Means for solving the problem]

[0006] The control device of the present disclosure includes an information acquisition unit that acquires information about the subject to be measured that indicates the presence of the subject based on a captured image, and acquires information for meal determination, which is information about meals, using the captured image or sensor information; a meal determination unit that outputs a determination result indicating that the subject is eating if a composite condition combining multiple conditions is met based on the information about the subject to be measured and the information for meal determination; and a control instruction unit that outputs a control signal to perform meal measurement if the determination result indicates that the subject is eating. [Effects of the Invention]

[0007] According to the present disclosure, it is possible to achieve an effect of improving the accuracy of meal measurement compared to conventional methods. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example configuration of a control device 10 according to a first embodiment of the present disclosure. [Figure 2] FIG. 2 is a flowchart showing an example of processing by the control device 10 according to the first embodiment of the present disclosure. [Figure 3] FIG. 3 is a diagram showing a configuration example of a diet measurement system 1 (1A) including a control device 10 (10A) according to the first embodiment of the present disclosure. [Figure 4] Figure 4 shows an example of the internal configuration of the decision rule storage unit 140, where Figure 4A is an example of the configuration when it includes a decision rule memory unit 141, Figure 4B is an example of the configuration when it includes a prediction model unit 142, and Figure 4C is an example of the configuration when it includes a learning model unit 143. [Figure 5] FIG. 5 is a flowchart showing an example of the processing of the information acquisition unit 110 (110A). [Figure 6]FIG. 6 is a flowchart showing an example of the processing of the meal determining section 130 (130A). [Figure 7] FIG. 7 is a diagram showing a configuration example of a diet measurement system 1 (1B) including a control device 10 (10B) according to the second embodiment of the present disclosure. [Figure 8] FIG. 8 is a flowchart showing an example of processing by the control device 10 (10B) according to the second embodiment of the present disclosure. [Figure 9] FIG. 9 is a flowchart showing an example of the processing of the information acquisition unit 110 (110B). [Figure 10] FIG. 10 is a flowchart showing an example of the processing of the meal determining section 130 (130B). [Figure 11] FIG. 11 is a flowchart showing an example of the processing of the control instruction unit 140 (140B). [Figure 12] FIG. 12 is a diagram showing a configuration example of a diet measurement system 1 (1C) including a control device 10 (10C) according to the third embodiment of the present disclosure. [Figure 13] FIG. 13 is a flowchart showing a first example of the processing of the meal determining section 130 (130C). [Figure 14] FIG. 14 is a diagram showing an example of the internal configuration when the composite judgment unit 1310 (1310C) in the meal judgment unit 130 (130C) is composed of a first composite judgment unit 1320-1, a second composite judgment unit 1320-2, a third composite judgment unit 1320-3, and a fourth composite judgment unit 1320-4. [Figure 15] FIG. 15 is a flowchart showing a first example of the processing in the meal determining section 130 (130C) in the case where the configuration shown in FIG. 14 is used. [Figure 16] FIG. 16 is a flowchart showing an example of the process of the first composite determining section 1320-1. [Figure 17] FIG. 17 is a flowchart showing a detailed example of the processing in the first composite determining section 1320-1. [Figure 18] FIG. 18 is a flowchart showing an example of the process of the second composite determining section 1320-2. [Figure 19] FIG. 19 is a flowchart showing a detailed example of the processing in the second composite determining section 1320-2. [Figure 20] FIG. 20 is a flowchart showing an example of the process of the third composite determining section 1320-3. [Figure 21] FIG. 21 is a flowchart showing a detailed example of the processing in the third composite determining section 1320-3. [Figure 22] FIG. 22 is a flowchart showing an example of the process of the fourth composite determining section 1320-4. [Figure 23] FIG. 23 is a flowchart showing a detailed example of the processing in the fourth composite determining section 1320-4. [Figure 24] FIG. 24 is a flowchart showing a second example of the processing by the meal determining section 130 (130C). [Figure 25] FIG. 25 is a flowchart showing a second example of the processing in the meal determining section 130 (130C) in the case where the configuration shown in FIG. 14 is used. [Figure 26] FIG. 26 is a diagram showing a configuration example of a diet measurement system 1 (1D) including a control device 10 (10D) and a measurement device 15 (15D) according to the fourth embodiment of the present disclosure. [Figure 27] FIG. 27 is a flowchart showing an example of processing by the measuring device 15 (15D) according to the fourth embodiment of the present disclosure. [Figure 28] FIG. 28 is a flowchart showing an example of the processing of the control instruction unit 150 (150D). [Figure 29] FIG. 29 is a diagram showing a configuration example of a diet measurement system 1 (1E) including a control device 10 (10E) and a measurement device 15 (15E) according to the fifth embodiment of the present disclosure. [Figure 30] FIG. 30 is a sequence diagram showing an example of processing of the diet measurement system 1 (1E) including the control device 10 (10E) and the measurement device 15 (15E) according to the fifth embodiment of the present disclosure. [Figure 31] FIG. 31 is a diagram illustrating a first example of a hardware configuration for realizing the functions according to the configuration of the present disclosure. [Figure 32] FIG. 32 is a diagram illustrating a second example of a hardware configuration for realizing the functions according to the configuration of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0009] In order to explain the present disclosure in more detail, embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0010] Embodiment 1 In the first embodiment, a configuration example having a basic configuration for controlling meal measurement will be described.

[0011] An example configuration of a control device according to a first embodiment of the present disclosure will be described. FIG. 1 is a diagram illustrating an example configuration of a control device 10 according to a first embodiment of the present disclosure. The control device 10 controls a measuring device that measures food. The control device 10 estimates whether the subject is eating or not eating and outputs a control signal to start or end measurement. The control device 10 shown in FIG. 1 includes an information acquisition unit 110, a meal determination unit 130, and a control instruction unit 150.

[0012] The information acquisition unit 110 acquires information for controlling the measurement device. The information acquiring section 110 acquires information about the subject to be measured that indicates the presence of the subject to be measured, and also acquires information for meal determination, which is information about meals, using the captured image or sensor information. The measurement subject information is information indicating the presence of a measurement subject. The information on the subject to be measured is, for example, information on the subject to be measured that can be obtained from a captured image, such as information indicating the presence or absence of the subject to be measured, an image or information indicating the area of ​​the subject to be measured in the captured image, identification information on the subject to be measured obtained using registered information (see the embodiment described below for registered information), or data combining two or more of these. Alternatively, the information on the subject to be measured may be information on the subject to be measured that can be obtained using sensor information from a human presence sensor, for example. Furthermore, the measurement subject information may be information including the direction of the measurement subject. When there are multiple measurement subjects, the measurement subject information indicates the presence of each measurement subject and includes the direction of each measurement subject. The meal determination information is information related to meals that can be obtained from at least one of the captured image and sensor information, and is, for example, information indicating at least one of an image of food in the captured image, an image including the subject's eating behavior in the captured image, an image of eating utensils used by the subject during eating, odor, temperature, humidity, or sound. The meal determination information includes an image of food in the captured image, an image including the subject's eating behavior in the captured image, or an image of eating utensils used by the subject during eating, and an image or information indicating an area in the captured image. The captured image may include, for example, not only visible light images but also infrared light images such as thermography images. This improves the accuracy of food determination, as described below. The information acquiring section 110 may acquire the measurement subject information and the meal determination information that are generated externally, or may acquire the measurement subject information and the meal determination information by detecting them. When detecting information about the subject to be measured and information for dietary assessment, the information acquisition unit 110 performs an analysis process on the captured image to detect the area of ​​the subject to be measured and acquire information about the subject to be measured, and performs an analysis process based on the captured image or sensor information to detect information about the diet and acquire information for dietary assessment.

[0013] The meal determining section 130 determines whether the person is eating or not eating based on the information acquired by the information acquiring section 110 . The meal determination unit 130 outputs a determination result indicating that a meal has occurred when a composite condition, which is a combination of multiple conditions, is satisfied based on the measurement object information and the meal determination information. In this embodiment, it is sufficient that a composite condition consisting of at least one set of conditions is satisfied. Furthermore, as will be shown in the embodiment described later, the meal determination unit 130 may be configured to set multiple composite conditions and output a determination result indicating that a meal has occurred when one or more of the composite conditions are satisfied.

[0014] The control command unit 150 controls the start, continuation, or end of meal measurement. If the determination result indicates a meal, the control instruction unit 150 outputs a control signal to execute a meal measurement. Specifically, the control instruction unit 150 acquires the determination result from the meal determination unit 130, and if the determination result indicates a meal, outputs a control signal to instruct the start of meal measurement or the continuation of meal measurement.

[0015] Here, when the captured image includes a plurality of measurement subjects, the information acquisition section 110, the meal determination section 130, and the control instruction section 150 function as follows. The information acquiring section 110 acquires, for each subject to be measured, information on the subject to be measured that indicates the area in which the subject to be measured is located. The meal determining section 130 outputs a determination result indicating that each measurement subject is eating. The control command unit 150 outputs a control signal to execute measurement of the meal for each subject.

[0016] In addition to the above components, the control device 10 also includes a control unit (not shown), a storage unit (not shown), and a communication unit (not shown). A control unit (not shown) controls the entire control device 10 and each of its components. The control unit (not shown) starts up the control device 10 in accordance with, for example, an external command. The control unit (not shown) also controls the state of the control device 10 (operating state = start-up, shutdown, sleep, etc.). A storage unit (not shown) stores each piece of data used by the control device 10. The storage unit (not shown) stores, for example, output (output data) from each component in the control device 10, and outputs data requested by each component to the component that made the request. A communication unit (not shown) communicates with an external device. For example, communication is performed between the control device 10 and a peripheral device (for example, a measurement device or a terminal device including a measurement device). For example, if the control device 10 and the measurement device or the terminal device are not connected by wire, the communication unit (not shown) has a function of communicating between the control device 10 and the measurement device or the terminal device. The control unit (not shown), the storage unit (not shown), and the communication unit (not shown) are the same in the embodiments described below.

[0017] A processing example of the control device according to the first embodiment of the present disclosure will be described. FIG. 2 is a flowchart showing an example of processing by the control device 10 according to the first embodiment of the present disclosure. The process shown in FIG. 2 is a control method performed by the control device 10. For example, when a control start condition is met, such as when a state in which meal measurement by the measuring device is possible, the control device 10 shown in FIG. 1 starts the process shown in FIG. 2 ("Start").

[0018] The control device 10 then executes information acquisition processing (step ST1100). In the information acquisition process, the information acquisition unit 110 of the control device 10 acquires information about the subject to be measured that indicates the presence of the subject to be measured based on the captured image, and also acquires information for meal determination, which is information about meals, using the captured image or sensor information. The information acquiring section 110 outputs the measurement object information to the meal determining section 130. In addition, the information acquiring section 110 outputs meal determination information to the meal determining section 130.

[0019] The control device 10 then executes a meal determination process (step ST1200). In the meal determination process, the meal determination unit 130 of the control device 10 outputs a determination result indicating that a meal has occurred if a composite condition, which is a combination of multiple conditions, is satisfied based on the measurement object information and the meal determination information. Specifically, when the meal determination unit 130 receives the measurement subject information and the meal determination information from the meal determination unit 130, it uses the measurement subject information and the meal determination information to determine whether a composite condition, which is a combination of multiple pre-set conditions, is met. If the composite condition is met, the meal determination unit 130 outputs a determination result indicating a meal to the control command unit 150. The meal determination unit 130 may output information indicating a meal or a non-meal as the determination result.

[0020] Next, the control device 10 executes a control command process (step ST1300). In the control command processing, the control command unit 150 of the control device 10 outputs a control signal for executing measurement of a meal when the determination result indicates a meal. Specifically, when control instruction unit 150 receives the determination result from meal determination unit 130, if the determination result indicates a meal, it outputs a control signal to start meal measurement if meal measurement is not being performed, or continues meal measurement if meal measurement is being performed.If the determination result indicates a non-meal, control instruction unit 150 continues the state in which meal measurement is not being performed if meal measurement is not being performed, or outputs a control signal to stop (end) meal measurement if meal measurement is being performed.

[0021] Next, the control device 10 executes an end determination process (step ST1400 "End?"). In the termination determination process, a control unit (not shown) of the control device 10 determines whether to terminate the processing of the control device 10. The control unit (not shown) determines whether to terminate the processing of the control device 10 in accordance with, for example, an external termination command or an execution program. When the control unit (not shown) determines not to end the processing of the control device 10 ("NO" in step ST1400), the process proceeds to step ST1100, and the process is repeated from step ST1100. When the control unit (not shown) determines that the processing of the control device 10 is to be ended (step ST1400 "YES"), the control device 10 ends the processing ("end").

[0022] Next, a configuration example of a system including the control device according to the first embodiment will be described. FIG. 3 is a diagram showing a configuration example of a diet measurement system 1 (1A) including a control device 10 (10A) according to the first embodiment of the present disclosure. The meal measurement system 1 (1A) is composed of devices related to meal measurement that measure meals. The diet measurement system 1A shown in FIG. 3 includes a control device 10 (10A), a measurement device 15 (15A), and an information source device 30 (30A).

[0023] The information source device 30A acquires information used in the processing of the control device 10A and outputs it to the control device 10A. In the description, the information source device 30A is separate from the control device 10A or the measuring device 15A, but it may be configured as an integrated device included in either one of them. The information source device 30A includes an imaging device 31. The imaging device 31 captures an image of the place where the meal is to be eaten, and outputs the captured image. The imaging device 31 is, for example, a camera placed on a table where meals are eaten. In this case, the imaging device 31 may be configured so that the height of the imaging position or the imaging direction can be changed. Alternatively, the imaging device 31 may be, for example, a camera installed on the ceiling above a table (dining table). In this case, the imaging device 31 may be configured to be able to change the shooting direction. The imaging device 31 may be capable of outputting images other than visible light images (visible light images) as long as it can output captured images that enable the information acquisition unit 110 or the meal determination unit 130 to function. For example, the imaging device 31 may be a visible light imaging device that captures visible light, and may also include an infrared light imaging device, such as a thermography camera, that captures infrared light. In this case, it is possible to identify food even when it is not possible to identify it using a visible light image. Specifically, for example, if white soup, rice, or cold tofu is placed in a white dish, the visible light image may indicate that no food is in the dish. Even in such a case, it is possible to detect the temperature of the soup or rice using thermography and determine that food is present. The soup and rice are displayed in warm colors, and the cold tofu is displayed in warm colors, making it clear that there is food in the dish.

[0024] The measurement device 15A measures meals and outputs the measurement results of the meal measurements or meal information based on the measurement results. For example, the measurement device 15A outputs information about food and drink for each meal.

[0025] The control device 10A is a device that controls the measuring device 15A. The control device 10A estimates whether the subject is eating or not eating based on a composite condition that combines multiple conditions, and causes the measuring device 15A to perform measurement if the subject is eating. The control device 10A shown in FIG. 3 is different from the configuration shown in FIG. 1 in that an example of the internal configuration of the information acquisition unit 110 and a determination rule storage unit 140 are added. The control device 10A includes an information acquisition unit 110 (110A), a meal determination unit 130 (130A), a determination rule storage unit 140 (140A), and a control command unit 150 (150A).

[0026] The information acquiring section 110A acquires information on the subject to be measured that indicates the presence of the subject to be measured based on the photographed image, and also acquires information for meal determination, which is information on meals, using the photographed image. The information acquiring section 110A shown in FIG. 3 includes a measurement object information acquiring section 1110 (1110A) and a meal determination information acquiring section 1120 (1120A).

[0027] The measurement subject information acquiring unit 1110A acquires measurement subject information indicating the presence of the measurement subject based on the captured image. The measurement subject information acquiring unit 1110A may acquire the measurement subject information indicating the presence of the measurement subject as a result of the measurement subject being detected using the captured image by an external device, or may acquire the measurement subject information indicating the presence of the measurement subject by receiving the captured image and detecting the measurement subject using the captured image.

[0028] The meal determination information acquisition unit 1120A acquires meal determination information, which is information related to meals, using a captured image. The meal determination information acquisition unit 1120A may acquire the meal determination information, which is information related to meals, as a result of an external device detecting information related to meals using the captured image, or may acquire the meal determination information by receiving a captured image or sensor information and detecting information related to meals using the captured image. As described above, the captured image may include not only a visible light image but also an infrared light image. In this case, it is possible to identify the food even when the food cannot be identified using a visible light image. Specifically, for example, if white soup, rice, or cold tofu is placed in a white dish, the visible light image may indicate that no food is in the dish. Even in such a case, the temperature of the soup or rice can be detected using thermography to determine that food is present. The soup and rice are displayed in warm colors, and the cold tofu is displayed in warm colors, making it clear that there is food in the dish.

[0029] Similar to the meal determination unit 130 already described, the meal determination unit 130A outputs a determination result indicating that the person is eating if a composite condition, which is a combination of multiple conditions, is met based on the information on the measurement target and the information for meal determination. When the meal determining unit 130A receives the measurement object information and the meal determination information, it outputs a determination result that determines whether the composite condition is satisfied based on the meal determination rules stored in the determination rule storage unit 140A.

[0030] The determination rule storage unit 140A stores rules for meal determination by the meal determination unit 130A. The determination rule storage unit 140A stores information on rules for meal determination in the form of, for example, a determination rule database, a prediction model, a learning model, or a combination thereof. Figure 4 shows an example of the internal configuration of the decision rule storage unit 140, where Figure 4A is an example of the configuration when it includes a decision rule memory unit 141, Figure 4B is an example of the configuration when it includes a prediction model unit 142, and Figure 4C is an example of the configuration when it includes a learning model unit 143. The determination rule storage unit 140 in the present disclosure may have the configurations shown in FIGS. 4A, 4B, and 4C.

[0031] The determination rule storage unit 140 shown in FIG. 4A includes a determination rule storage unit 141. The determination rule storage unit 141 stores a determination rule database. The determination rule database stores composite determination rules. A composite determination rule is, for example, a composite condition made up of a combination of multiple conditions. Alternatively, a combination of multiple composite conditions.

[0032] The determination rule storage unit 140 shown in Fig. 4B is configured to include a prediction model unit 142. The prediction model unit 142 stores data that realizes a prediction model. The prediction model is a trained model that receives as input an image of a subject to be measured and information for meal determination as information about the subject to be measured, and that has been trained in advance to output determination result information indicating whether the subject is eating or not. In this case, the meal determination unit 130A inputs the information on the object to be measured and the information for meal determination into the trained model, and outputs a determination result indicating that it is a meal using the determination result information indicating whether it is a meal or not output by the trained model.

[0033] The decision rule storage unit 140 shown in FIG. 4A includes a learning model unit 143 . The learning model unit 143 holds data that realizes the learning model. The learning model is a model constructed by learning to receive as input an image of the subject to be measured and information for meal determination as measurement subject information, and to output determination result information indicating whether the subject is eating or not. The learning model may be a model that continues to be constructed as it learns. Since the learning model uses information that is likely to indicate eating, the accuracy of the determination result can be improved by constructing the model using this information. In this case, the meal determination unit 130A inputs the information on the object to be measured and the information for meal determination into the trained model, and outputs a determination result indicating that it is a meal using the determination result information indicating whether it is a meal or not output by the trained model.

[0034] The internal configuration of the determination rule storage unit 140 is the same in the embodiments described below.

[0035] Similar to the control command unit 150 already described, the control command unit 150A outputs a control signal to execute meal measurement when the result of the meal determination by the meal determination unit 130A indicates that a meal is being measured. Specifically, the control instruction unit 150A acquires the determination result from the meal determination unit 130A, and if the determination result indicates a meal, outputs a control signal to instruct the start of meal measurement or the continuation of meal measurement.

[0036] Here, when the captured image includes a plurality of measurement subjects, the information acquisition section 110A, the meal determination section 130A, and the control instruction section 150A function as follows. The information acquiring unit 110A acquires, for each subject to be measured, information on the subject to be measured that indicates the area in which the subject to be measured is located. The meal determining section 130A outputs a determination result indicating that each measurement subject is eating. The control command unit 150A outputs a control signal for executing meal measurement for each subject.

[0037] Next, a processing example in the control device according to the first embodiment of the present disclosure will be described. The difference is that the control device 10A can explicitly indicate that the source of the captured image used in the processing is the information source device 30A, and that the control destination is the measuring device 15A. However, since the processing is similar to that of the control device 10 already described, here we will explain the processing that was not explained in the explanation of the processing of the control device 10.

[0038] First, an example of processing by the information acquisition unit 110 (110A) will be described. FIG. 5 is a flowchart showing an example of the processing of the information acquisition unit 110 (110A). The information acquisition unit 110A starts the process shown in FIG. 5 (“Start”) when a control start condition is met, such as when the control device 10A is in operation, for example, when the power of the control device 10A is ON, or when the measuring device 15A is in a state where meal measurement is possible.

[0039] Next, the information acquiring section 110A executes the measurement target information acquiring process (step ST1110). In the measurement subject information acquisition process, the measurement subject information acquisition section 1110A of the information acquisition section 110A acquires measurement subject information indicating the presence of a measurement subject based on a captured image. The measurement subject information acquisition section 1110A acquires measurement subject information indicating the presence of a measurement subject as a result of the measurement subject being detected by an external device using the captured image. Alternatively, the measurement subject information acquisition section 1110A receives the captured image and detects the measurement subject using the captured image, thereby acquiring measurement subject information indicating the presence of the measurement subject.

[0040] The information acquisition section 110A then executes information acquisition processing for meal determination (step ST1120). In the meal determination information acquisition process, the meal determination information acquisition unit 1120A of the information acquisition unit 110A acquires meal determination information, which is information related to meals, using captured images. The meal determination information acquisition unit 1120A acquires meal determination information, which is information related to meals, as a result of information related to meals being detected using captured images by an external device. Alternatively, the meal determination information acquisition unit 1120A acquires meal determination information by accepting captured images and detecting information related to meals using the captured images.

[0041] The information acquiring section 110A then executes a process of outputting the measurement object information and the meal determination information (step ST1130). In this process, the measurable object information acquiring section 1110A of the information acquiring section 110A outputs the measurable object information to the meal determining section 130A. The meal determination information acquisition section 1120A of the information acquisition section 110A outputs the meal determination information to the meal determination section 130A.

[0042] After outputting the measurement object information and the meal determination information to the meal determination section 130A, the information acquisition section 110A ends the process ("end") and waits until the next process starts.

[0043] Here, an example of the processing performed by the meal determining unit 130 (130A) will be described. FIG. 6 is a flowchart showing an example of the processing of the meal determining section 130 (130A). The meal determination unit 130A starts the process shown in FIG. 6 when a control start condition is met, such as when the control device 10A is in operation, for example, when the control device 10A is powered on, or when the measuring device 15A is in a state where meal measurement is possible. ("Start")

[0044] The meal determining section 130A then executes a process for acquiring information on the measurement object and information for meal determination (step ST1210). In this process, the meal determining section 130A obtains measurement object information from the measurement object information obtaining section 1110A. Also, the meal determining section 130A obtains meal determination information from the meal determination information obtaining section 1120A.

[0045] The meal determining section 130A then executes meal determination processing using the measurement object information and the meal determination information (step ST1220). In this process, the meal determination unit 130A outputs a determination result indicating a meal when a composite condition, which is a combination of multiple conditions, is met based on the measurement subject information and the meal determination information. Specifically, the meal determination unit 130A uses the measurement subject information and the meal determination information to determine a meal when, in addition to the condition that the measurement subject is present, a meal-related condition, such as the presence of food or the measurement subject performing an eating motion, is met. If any of the conditions is not met, the meal determination unit 130A determines that the measurement subject is not eating (i.e., not eating).

[0046] The meal determining section 130A then executes a determination result output process (step ST1230). In the determination result output process, the meal determination unit 130A outputs the determination result of the meal determination process to the control instruction unit 150.

[0047] After outputting the determination result, the meal determination unit 130A ends the process ("end") and waits until the next process starts ("end").

[0048] This embodiment shows a configuration including the following. an information acquisition unit that "acquires information about a subject to be measured that indicates the presence of the subject to be measured, and acquires information for meal determination that is information about meals using photographed images or sensor information"; a meal determination unit that "outputs a determination result indicating that the subject is eating when a composite condition, which is a combination of a plurality of conditions, is satisfied based on the measurement object information and the meal determination information"; a control command unit that outputs a control signal to execute meal measurement when the determination result indicates that the subject is eating; A control device comprising: As a result, the present disclosure has the effect of providing a control device that enables the accuracy of meal measurement to be improved compared to conventional devices.

[0049] This embodiment shows a configuration including the following. A control method by a control device, an information acquisition step in which an information acquisition unit of the control device "acquires information about the subject to be measured that indicates the presence of the subject to be measured, and acquires information for meal determination, which is information about meal, using a captured image or sensor information"; a meal determination step in which the meal determination unit of the control device "outputs a determination result indicating that the subject has eaten if a composite condition, which is a combination of multiple conditions, is satisfied based on the measurement object information and the meal determination information"; a control command step in which a control command unit of the control device "outputs a control signal to execute measurement of a meal when the determination result indicates that the meal has been eaten"; Equipped with A control method comprising: As a result, the present disclosure makes it possible to improve the accuracy of meal measurement compared to the prior art. This brings about an effect of providing a control method.

[0050] This embodiment shows a configuration including the following. Computer, an information acquisition unit that "acquires information about a subject to be measured that indicates the presence of the subject to be measured, and acquires information for meal determination that is information about meals using photographed images or sensor information"; a meal determination unit that "outputs a determination result indicating that a meal has occurred when a composite condition, which is a combination of multiple conditions, is satisfied based on the measurement object information and the meal determination information"; a control command unit that outputs a control signal to execute measurement of a meal when the determination result indicates that the meal has been eaten; a control device comprising: A program characterized by operating as As a result, the present disclosure has the effect of providing a program that realizes a control device that improves the accuracy of meal measurement compared to conventional methods.

[0051] This embodiment further includes the following configuration. When the captured image includes a plurality of measurement subjects, The information acquisition unit "acquires, for each subject to be measured, information on the subject to be measured that indicates the area in which the subject to be measured is located." The meal determining unit "outputs a determination result indicating that each measurement subject is eating." The control command unit "outputs a control signal to execute measurement of the meal for each measurement subject." A control device characterized by: As a result, the present disclosure further has the effect of being able to provide a control device that makes it possible to improve the accuracy of meal measurement compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a control device, the above control method, or the above program.

[0052] This embodiment further includes the following configuration. The meal determination unit inputs information about the object to be measured and information for meal determination into a trained model, and outputs a determination result indicating that it is a meal using determination result information indicating whether it is a meal or not output by the trained model. A control device characterized by: As a result, the present disclosure further has the effect of being able to provide a control device that makes it possible to improve the accuracy of meal measurement compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a control device, the above control method, or the above program.

[0053] Embodiment 2 In the above-described first embodiment, a configuration example including a case where externally detected information is acquired and used for meal determination has been described. Also, a configuration example including a case where meal determination is performed using only one composite condition has been described. In the second embodiment, a configuration example including a detecting means will be described, and a configuration example in which a plurality of composite conditions are set will also be described. In the second embodiment, among the components of the second embodiment, the components and their functions that are similar to the components of the first embodiment already described are indicated by the same component names and the same symbols, and duplicate explanations of those components are omitted as appropriate.

[0054] Next, a configuration example of a control device according to a second embodiment of the present disclosure will be described. FIG. 7 is a diagram showing a configuration example of a diet measurement system 1 (1B) including a control device 10 (10B) according to the second embodiment of the present disclosure. The meal measurement system 1 (1B) is composed of devices related to meal measurement that measure meals. The diet measurement system 1B shown in FIG. 7 includes a control device 10 (10B), a measurement device 15 (15B), and an information source device 30 (30B). In the description, information source device 30B is assumed to be separate from control device 10A or measurement device 15A, but may be configured integrally with either of them. The information source device 30 (30B) shown in FIG. 7 includes an imaging device 31, an odor sensor 32, a temperature sensor 33, a humidity sensor , a sound collecting device 35, and a human sensor .

[0055] The imaging device 31 has the same configuration as the imaging device 31 already described. The odor sensor 32 measures the odor and outputs odor data that is a numerical representation of the odor as sensor information. The odor sensor 32 may be any sensor that can output sensor information as the odor measurement result, and may be realized using existing technology. The temperature sensor 33 measures the temperature and outputs temperature data indicating the temperature as sensor information. The humidity sensor 34 measures the humidity and outputs humidity data indicating the humidity as sensor information. The sound collection device 35 measures sound and outputs the measured sound as sensor information. The sound collection device 35 collects eating sounds, which are sounds related to eating. Specifically, for example, eating sounds are internal sounds such as chewing sounds and swallowing sounds. In this case, the sound collection device 35 may be configured to amplify and collect sounds, for example, from the throat of each subject. Furthermore, for example, meal sounds are external sounds such as conversations about eating made by the person being measured and the sound of eating utensils. Conversations include, for example, utterances such as "Itadakimasu" (thank you for the meal) and "It's time for dinner." Dish sounds include the sound of dishes hitting each other with cutlery. In this case, since meal conversations and dish sounds may be difficult to hear due to noise from the TV or air conditioning, for example, sound collection device 35 can mask other sounds to improve accuracy. Furthermore, for example, meal sounds are pre-meal sounds such as cooking sounds and meal preparation sounds. Cooking sounds are sounds made when cooking ingredients, including the sounds made by a microwave oven, for example. Meal preparation sounds include the sounds made when placing dishes on the table. In this case, if the eating area is close to the dining area, the sounds can be collected using a built-in microphone. On the other hand, if the eating area is far from the dining area, the sounds can be collected using a separate microphone. The human presence sensor 36 detects the presence or absence of a person. Specifically, the presence or absence of a person to be measured can be detected using known human detection technology. The human presence sensor 36 can be configured to replace the imaging device 31, or can be configured to be included together with the imaging device 31. This makes it possible to detect the presence of a person to be measured without keeping the imaging device 31 running all the time. In other words, the contents of the present disclosure can be realized at low cost and with low power consumption.

[0056] Similar to the measurement device 15A already described, the measurement device 15B measures meals and outputs the measurement results of the meal measurements or meal information based on the measurement results. For example, the measurement device 15B outputs information about food and drink for each meal. The measuring device 15B shown in FIG. 7 includes a meal measuring section 160 (160B) and a meal information output section 170 (170B). The meal measuring section 160B performs measurements related to meals and outputs the measurement results. The meal measuring section 160B performs measurements related to meals based on, for example, captured images. The meal information output unit 170B outputs the measurement results obtained by the meal measurement unit 160B to the outside. The meal information output unit 170B outputs the measurement results to a recording device for storage, for example. Alternatively, the information output unit 170B outputs the measurement results to an external server for storage, for example.

[0057] The control device 10B is a device that controls the measuring device 15B. The control device 10B estimates whether the subject is eating or not eating and outputs a control signal to the measuring device 15B to start or end measurement. The control device 10B shown in FIG. 7 includes an information acquisition unit 110 (110B), a meal determination unit 130 (130B), a determination rule storage unit 140 (140B), and a control instruction unit 150 (150B).

[0058] The information acquisition unit 110B "acquires measurement subject information indicating the presence of a measurement subject based on a photographed image, and also acquires meal determination information, which is information related to meals, using the photographed image or sensor information." The information acquiring section 110B shown in FIG. 7 includes a measurement object information acquiring section 1110 (1110B) and a meal determination information acquiring section 1120 (1120B). The measurement subject information acquiring section 1110B acquires measurement subject information indicating the presence of a measurement subject based on the captured image. The measurement object information acquisition unit 1110B shown in FIG. The measurement object detection unit 1111 receives a captured image, detects a measurement object using the captured image, and outputs an image showing the area of ​​the measurement object.

[0059] The meal determination information acquisition unit 1120B detects information related to meals using photographed images or sensor information acquired in time series, and acquires the meal determination information. The meal determination information acquisition unit 1120B of the information acquisition unit 110B acquires meal determination information, which is information related to meals, using captured images or sensor information. The meal determination information acquisition unit 1120B acquires the meal determination information by accepting the captured images or sensor information and detecting information related to meals using the captured images or sensor information. The information acquisition unit 1120B for meal determination shown in Figure 7 is configured to include a meal status detection unit (food, smell, temperature, humidity) 1121, a first meal action detection unit (hands, arms) 1122, a second meal action detection unit (mouth, pharynx) 1123, and a meal sound detection unit 1124.

[0060] The eating situation detection unit (food, smell, temperature, humidity) 1121 outputs eating situation information indicating a situation where the food is likely to be eaten, using a captured image or sensor information. In this case, the information for determining whether or not a meal is being eaten is meal status information indicating whether or not a meal is being eaten, and the meal status information indicates at least one of the following: whether or not food is being eaten, whether or not there is a change in smell, whether or not there is a change in temperature, or whether or not there is a change in humidity. The presence of food is information indicating that food is present when food is included in the area of ​​a table (dining table) in the captured image output by the imaging device 31. The presence of a change in odor is a detection result based on the sensor information output in time series by the odor sensor 32, and is information indicating that the odor has changed. The presence of a temperature change is a detection result based on sensor information output in time series by the temperature sensor, and is information indicating that the temperature has changed. The "change in humidity" is a detection result based on sensor information output in time series by the humidity sensor, and is information indicating that the humidity has changed.

[0061] The first eating movement detection section 1122 detects a first eating movement, which is a movement related to eating of the subject. The first eating action detection unit 1122 detects at least one of the hand movement or arm movement of the subject based on, for example, photographed images or sensor information acquired in time series, and further, when it detects an action such as picking up food or drink and bringing it to the mouth, it outputs an image including the first eating action as information for meal assessment. In this case, the eating determination information is an image including a first eating motion, which is a motion of the subject bringing food to his / her mouth, and the first eating motion is a motion based on at least one of a hand motion and an arm motion.

[0062] The second eating movement detection section 1123 detects a second eating movement, which is a movement related to eating of the subject. The second eating movement detection unit 1123 detects, for example, at least one of the movement of the mouth or the movement of the pharynx of the subject based on the captured images acquired in time series, and further, if it detects that the movement of the mouth is chewing food or that the movement of the pharynx is swallowing food or that the movement of the pharynx is swallowing food, it outputs information indicating the second eating movement as information for meal assessment. In this case, the eating determination information is an image including a second eating movement, which is a movement made by the subject after bringing food or drink to his / her mouth. The second eating movement is a movement based on at least one of a movement of the mouth and a movement of the pharynx.

[0063] The meal sound detection unit 1124 detects sounds related to eating. The meal sound detection unit 1124 acquires sound data from the sound collection device and analyzes the sound data to detect meal sounds, which are sounds related to eating. In this case, the meal determination information is meal sounds that are likely to be generated when eating. Specifically, for example, the eating sounds are internal sounds such as chewing sounds, swallowing sounds, etc. In this case, the eating determination information is one or more of the chewing sounds and the swallowing sounds. Furthermore, for example, meal sounds are external sounds such as conversations about eating made by the measurement subject and the sound of dishes. Conversations include, for example, utterances such as "Itadakimasu" (thank you for the meal) and "It's time for dinner." Dishes sounds include the sound of dishes hitting cutlery against each other. In this case, the meal determination information is one or more of conversations about eating and cooking sounds. Furthermore, for example, meal sounds are pre-meal sounds such as cooking sounds and meal preparation sounds. Cooking sounds are sounds made when cooking ingredients, including sounds made by a microwave oven. Meal preparation sounds include sounds made when placing dishes on a table. In this case, the meal determination information is one or more of dish sounds and meal preparation sounds. The eating sounds are at least one of chewing sounds, swallowing sounds, conversations about eating, tableware sounds, cooking sounds, and meal preparation sounds.

[0064] The meal determining unit 130B "outputs a determination result indicating that the subject is eating if a composite condition, which is a combination of a plurality of conditions, is satisfied based on the measurement object information and the meal determination information." The meal determining section 130B includes a composite determining section 1310 (1310B). The composite determination unit 1310B uses the measurement object information and the meal determination information to output a determination result indicating that a meal has occurred if the composite condition is met based on the determination rules stored in the determination rule storage unit 140A. The composite determination unit 1310B refers to the determination rule storage unit 140B or inputs information into the determination rule storage unit 140B, and outputs a determination result indicating that a meal has occurred if the composite condition is met.

[0065] The determination rule storage unit 140B stores rules for meal determination by the meal determination unit 130B, similar to the determination rule storage unit 140A already described. The determination rule storage unit 140B stores information on rules for meal determination in the form of, for example, a determination rule database, a prediction model, a learning model, or a combination thereof.

[0066] Control instruction unit 150B, like control instruction unit 150 already described, "outputs a control signal to execute meal measurement when the determination result indicates that a meal is being measured."

[0067] A processing example of the control device according to the second embodiment of the present disclosure will be described. FIG. 8 is a flowchart showing an example of processing by the control device 10 (10B) according to the second embodiment of the present disclosure. The process shown in FIG. 8 is a control method performed by the control device 10B. For example, the control device 10B shown in FIG. 7 starts the process shown in FIG. 8 when a control start condition is satisfied, such as when the control device 10A is in operation, for example, when the power of the control device 10A is ON, or when a state in which meal measurement by the measuring device 15B is possible is reached. ("Start")

[0068] The control device 10B then executes information acquisition processing (step ST2100). In the information acquisition process, the information acquisition unit 110B of the control device 10B acquires information about the subject to be measured that indicates the presence of the subject to be measured based on the captured image, and also acquires information for meal determination, which is information about meals, using the captured image or sensor information. The information acquiring section 110B outputs the measurement object information to the meal determining section 130B. In addition, the information acquiring section 110B outputs information for meal determination to the meal determining section 130B.

[0069] The control device 10B then executes a meal determination process (step ST2200). In the meal determination process, the meal determination unit 130B of the control device 10B outputs a determination result indicating that a meal has occurred when a composite condition, which is a combination of multiple conditions, is satisfied based on the measurement object information and the meal determination information. Specifically, when the meal determining unit 130B receives the measurement target information and the meal determination information from the meal determining unit 130B, the meal determining unit 130B uses the measurement target information and the meal determination information to determine whether a composite condition, which is a combination of multiple pre-set conditions, is met. If the composite condition is met, the meal determining unit 130B outputs a determination result indicating a meal to the control command unit 150B. The meal determining unit 130B may output information indicating a meal or a non-meal as the determination result.

[0070] Control device 10B then executes a control command process (step ST2300). In the control command processing, the control command unit 150B of the control device 10B outputs a control signal to execute measurement of a meal when the determination result indicates a meal. Specifically, when control instruction unit 150B receives the determination result from meal determination unit 130B, if the determination result indicates a meal, it outputs a control signal to start meal measurement if meal measurement is not being performed, or continues meal measurement if meal measurement is being performed.If the determination result indicates a non-meal, control instruction unit 150B continues the state in which meal measurement is not being performed if meal measurement is not being performed, or outputs a control signal to stop (end) meal measurement if meal measurement is being performed.

[0071] Control device 10B then executes an end determination process (step ST2400 "End?"). In the termination determination process, a control unit (not shown) of the control device 10B determines whether to terminate the processing of the control device 10B. The control unit (not shown) determines whether to terminate the processing of the control device 10B in accordance with, for example, an external termination command or an execution program. When the control unit (not shown) determines not to end the processing of the control device 10B ("NO" in step ST2400), the process proceeds to step ST2100, and the process is repeated from step ST2100. When the control unit (not shown) determines that the processing of the control device 10B is to be ended (step ST2400 "YES"), the control device 10B ends the processing ("end").

[0072] Here, a detailed example of the information acquisition process (step ST2100) will be described. FIG. 9 is a flowchart showing an example of the processing of the information acquisition unit 110 (110B). When information acquiring section 110B starts the process ("start"), it then executes the measurement object detection process (step ST2110). In the measurement subject detection process, the measurement subject information acquisition section 1110B of the information acquisition section 110B receives a captured image and detects the measurement subject using the captured image, thereby acquiring measurement subject information indicating the presence of the measurement subject. Specifically, the measurement object detection unit 1111 of the measurement object information acquisition unit 1110B acquires the captured image output by the imaging device 31B, analyzes the captured image, and detects the measurement object in the captured image. When the measurement object detection unit 1111 detects the measurement object in the captured image, it outputs the image of the measurement object in the captured image.

[0073] The information acquiring unit 110B then executes a process of determining whether a measurement target is present (step ST2111 "Is a measurement target present?"). In the measurable object presence determination process, the measurable object information acquisition section 1110B of the information acquisition section 110B determines the presence or absence of the measurable object based on the detection result in the measurable object detection section 1111. If it is determined that no measurement target is present ("NO" in step ST2111), the information acquiring unit 110B proceeds to the process of step ST2119. If it is determined that a measurement target is present ("YES" in step ST2111), the information acquiring unit 110B proceeds to the process of step ST2112.

[0074] Next, the information acquiring unit 110B executes the measurement object information output process (step ST2112). In the measurement subject information output process, the measurement subject information acquisition section 1110B of the information acquisition section 110B outputs measurement subject information including the image of the measurement subject in the captured image to the meal determination section 130B.

[0075] Next, the information acquiring section 110B executes the eating state detecting process (step ST2113). In the eating state detection process, the eating determination-oriented information acquisition section 1120B of the information acquisition section 110B detects and acquires eating state information as eating determination-oriented information. Specifically, the eating state detection unit 1121 in the eating determination information acquisition unit 1120B outputs eating state information indicating a state where eating is likely to occur, using a captured image or sensor information. The meal status information may include, for example, information such as "food is in the fridge," "dishes are in the fridge," "change in smell," "change in temperature," and "change in humidity." When the meal situation information is "presence of food" or "presence of cookware", the meal situation detection unit 1121 outputs meal situation information indicating "presence of food" or "presence of cookware" when food or cookware is included in the table (dining table) area in the captured image. When the eating status information indicates "there is a change in the smell," the eating status detection unit 1121 acquires the sensor information output in time series by the smell sensor 32, and when it detects that the smell has changed, it outputs eating status information indicating "there is a change in the smell." If the eating status information is "temperature has changed", the eating status detection unit 1121 acquires sensor information output in chronological order by the temperature sensor, and if it detects that the temperature has changed, it outputs eating status information indicating "temperature has changed". If the eating status information is "humidity has changed", the eating status detection unit 1121 acquires sensor information output in chronological order by the humidity sensor, and if it detects that the humidity has changed, it outputs eating status information indicating "humidity has changed".

[0076] Next, the information acquisition section 110B executes a first eating motion detection process (step ST2114). In the first eating motion detection process, the eating determination-oriented information acquisition section 1120B of the information acquisition section 110B detects and acquires first eating motion information as eating determination-oriented information. Specifically, the first eating motion detection unit 1122 in the meal determination information acquisition unit 1120B acquires captured images or sensor information in chronological order and performs an analysis process. If the first eating motion detection unit 1122 detects at least one of hand movement or arm movement of the subject as a result of the analysis process, it then performs a process to detect a hand movement or arm movement such as picking up food or drink and bringing it to the mouth. If the first eating motion detection unit 1122 detects a movement such as picking up food or drink and bringing it to the mouth, it outputs information indicating that the first eating motion has been performed or an image including the first eating motion as meal determination information.

[0077] Next, the information acquisition section 110B executes a second eating motion detection process (step ST2115). In the second eating motion detection process, the eating determination-oriented information acquisition section 1120B of the information acquisition section 110B detects and acquires second eating motion information as eating determination-oriented information. Specifically, the second eating movement detection unit 1123 in the eating assessment information acquisition unit 1120B acquires captured images or sensor information in chronological order and performs an analysis process. If the second eating movement detection unit 1123 detects at least one of the mouth movement or pharyngeal movement of the subject as a result of the analysis process, it then performs a process to detect an action such as chewing food or an action such as swallowing food from the mouth movement or an action such as swallowing food from the pharyngeal movement. If the second eating movement detection unit 1123 detects an action such as chewing food or an action such as swallowing food from the mouth movement or an action such as swallowing food or an action such as swallowing food from the pharyngeal movement, it outputs information indicating that the second eating movement has been performed or an image including the second eating movement as eating assessment information.

[0078] Information acquiring section 110B then executes eating sound detection processing (step ST2116). In the eating sound detection process, the eating determination information acquisition section 1120B of the information acquisition section 110B detects and acquires eating sound information as eating determination information. Specifically, the eating sound detection unit 1124 in the meal determination information acquisition unit 1120B acquires sound data from a sound collection device. The eating sound detection unit 1124 analyzes the sound data and performs processing to detect eating sounds that are sounds related to eating. When the eating sound detection unit 1124 detects eating sounds that are sounds related to eating, it outputs information indicating the eating sounds as meal determination information.

[0079] The information acquisition unit 110B then executes a process for determining whether or not to acquire information for meal determination (step ST2117 "Acquire information for meal determination?"). In the meal determination information acquisition determination process, the meal determination information acquisition unit 1120B of the information acquisition unit 110B determines whether meal determination information has been acquired. Specifically, the information acquisition unit for eating determination 1120B determines whether any of eating state information, first eating motion information, second eating motion information, or eating sound has been acquired. If the information acquisition unit 1120B for meal determination acquires any one of the meal status information, the first meal motion information, the second meal motion information, or the meal sound (step ST2117 “YES”), the information acquisition unit 110B proceeds to the processing of step ST2118. If the information acquisition unit 1120B for meal determination has not acquired any of the information ("NO" in step ST2117), the information acquisition unit 110B proceeds to the process of step ST2119.

[0080] The information acquisition section 110B then executes a meal determination information output process (step ST2118). In the meal determination information output process, the meal determination information acquisition unit 1120B of the information acquisition unit 110B outputs the acquired meal determination information to the meal determination unit 130B.

[0081] The information acquiring unit 110B executes an end determination process (step ST2119 "end?") following the process of step ST2111, "YES" in step ST2117, or the process of step ST2118. In the termination determination process, the information acquisition unit 110B checks whether the control device 10B will terminate the process, and if the control device 10B will terminate the process, it determines that the process of the information acquisition unit 110B will terminate. When the information acquiring unit 110B determines not to end the process ("NO" in step ST2119), it moves to the process of step ST2111 and repeats the process from the process of step ST2111. When it is determined that the process should be ended (step ST2119 "YES"), the information acquisition unit 110B ends the process ("end").

[0082] Next, a detailed example of the meal determination process (step ST2200) will be described. FIG. 10 is a flowchart showing an example of the processing of the meal determining section 130 (130B). When the meal determination unit 130B starts (“START”) the meal determination process (step ST2200), it then executes the measurement target information acquisition process (step ST2210). In the measurement object information acquisition process, the meal determining section 130B acquires the measurement object information output by the information acquiring section 110B. Specifically, when the measurable object information is output by the measurable object information acquiring section 1110B of the information acquiring section 110B, the composite determining section 1310B of the meal determining section 130B accepts the measurable object information.

[0083] The meal determination unit 130B then executes information acquisition processing for meal determination (step ST2211). In the meal determination information acquisition process, the meal determination unit 130B acquires the meal determination information output by the information acquisition unit 110B. Specifically, when meal determination information is output by the meal determination information acquisition unit 1120B of the information acquisition unit 110B, the composite determination unit 1310B of the meal determination unit 130B accepts the meal determination information.

[0084] The meal determining unit 130B then executes the meal determining process (step ST2212 "Meal?"). In the meal determination process, the composite determination section 1310B of the meal determination section 130B uses the measurement object information and the information for meal determination to refer to the determination rule storage section 140B, and outputs a determination result indicating that a meal has occurred if the composite condition is met. Alternatively, the composite determining unit 1310B uses the measurement object information and the meal determination information to input information to the determination rule storage unit 140B, and outputs a determination result indicating a meal if the composite condition is satisfied. Using the information on the subject to be measured and the information for meal determination, for example, if the conditions that the subject to be measured is present in the captured image and that food is present in the captured image are satisfied, it is determined that the composite condition is satisfied and a determination result indicating that a meal has occurred is output. Alternatively, using the information on the subject to be measured and the information for meal determination, for example, if the conditions that the subject to be measured is present in the captured image and that the subject to be measured has performed a first eating motion or a second eating motion are satisfied, it is determined that the composite condition is satisfied and a determination result indicating that a meal has occurred is output.

[0085] When the determination result output by the composite determination section 1310B indicates a meal, the meal determination section 130B determines that the meal is a meal (step ST2213). When the determination result output by the composite determination section 1310B indicates that the subject has not eaten, the meal determining section 130B determines that the subject has not eaten (step ST2214).

[0086] The meal determining unit 130B then executes a determination result output process (step ST2215). In the determination result output process, the meal determining unit 130B outputs the determination result indicating whether the subject is eating or not to the control command unit 150B.

[0087] The meal determining unit 130B then executes the end determination process (step ST2216 "End?"). In the end determination process, the meal determination unit 130B checks whether the control device 10B will end the process, and if the control device 10B will end the process, determines that the process of the meal determination unit 130B will end. When the meal determining unit 130B determines not to end the process ("NO" in step ST2216), the meal determining unit 130B proceeds to the process of step ST2210 and repeats the process from step ST2210. When determining that the process should be ended ("YES" in step ST2216), the meal determining unit 130B ends the process ("End").

[0088] Next, a detailed example of the control command process (step ST2400) will be described. FIG. 11 is a flowchart showing an example of the processing of the control instruction unit 140 (140B). When the control command unit 140B starts the control command process (step ST2300) (“START”), it then executes the meal determination process (step ST2310 “Meal?”). In the meal determination process, the control instruction unit 140B performs meal determination using the meal determination result output by the meal determination unit 130B.

[0089] When it is determined in step ST2310 that the subject is eating (step ST2310 "YES"), the control instruction unit 140B then executes a measurement-on-going determination process (step ST2311 "measurement ongoing?").

[0090] When the control command unit 140B determines in step ST2311 that measurement is in progress (step ST2311 "YES"), it then executes a measurement continuation process (step ST2312). In the measurement continuation process, the control command unit 140B outputs a control signal to the diet measurement unit 160B of the measuring device 15B to instruct it to continue measurement. Alternatively, it is not necessary to output a particular control signal to the diet measurement unit 160B of the measuring device 15B that is in the middle of measurement.

[0091] When it is determined in step ST2311 that measurement is not in progress (step ST2311 "NO"), the control command unit 140B then executes a measurement start command process (step ST2313). In the measurement start command process, the control command unit 140B outputs a control signal to command the diet measurement unit 160B of the measuring device 15B that is not in progress of measurement to start measurement.

[0092] If it is determined in step ST2310 that the subject is not eating (step ST2310 "NO"), the control command unit 140B then executes a measurement-on-going determination process (step ST2314 "measurement on?"). In the measurement in progress determination process, the control command unit 140B checks the operating state of the meal measurement unit 160B of the measurement device 15B and determines whether measurement is in progress.

[0093] When it is determined in step ST2314 that measurement is in progress ("YES" in step ST2314), the control command unit 140B then executes a measurement stop command process (step ST2315). In the measurement stop command process, the control command unit 140B outputs a control signal to the diet measurement unit 160B of the measuring device 15B that is currently performing measurement, to command the measurement unit 160B to end (stop) the measurement.

[0094] When the control command unit 140B determines in step ST2314 that measurement is not in progress (step ST2314 "NO"), it then executes a non-measurement continuation process (step ST2316). In the non-measurement continuation process, the control command unit 140B outputs a control signal to command the diet measurement unit 160B of the measuring device 15B that is not in measurement to continue the non-measurement state. Alternatively, the control command unit 140B does not issue a command to the diet measurement unit 160B of the measuring device 15B that is not in measurement.

[0095] The control command unit 140B ends the process ("end") when it has executed any one of the measurement continuation process, measurement start command process, measurement stop command process, and non-measurement continuation process.

[0096] This embodiment further includes the following configuration. The information for meal determination is meal situation information indicating a situation in which the meal is likely to occur. A control device characterized by: As a result, the present disclosure further has the effect of being able to provide a control device that makes it possible to improve the accuracy of meal measurement compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a control device, the above control method, or the above program.

[0097] This embodiment further includes the following configuration. The meal status information indicates at least one of the presence of food, whether there is a change in smell, whether there is a change in temperature, or whether there is a change in humidity. A control device characterized by: As a result, the present disclosure further has the effect of being able to provide a control device that makes it possible to improve the accuracy of meal measurement compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a control device, the above control method, or the above program.

[0098] This embodiment further includes the following configuration. The eating determination information is an image including a first eating action, which is an action of the subject bringing food or drink to his / her mouth. A control device characterized by: As a result, the present disclosure further has the effect of being able to provide a control device that makes it possible to improve the accuracy of meal measurement compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a control device, the above control method, or the above program.

[0099] This embodiment further includes the following configuration. The first eating motion is a motion based on at least one of a hand motion and an arm motion. A control device characterized by: As a result, the present disclosure further has the effect of being able to provide a control device that makes it possible to improve the accuracy of meal measurement compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a control device, the above control method, or the above program.

[0100] This embodiment further includes the following configuration. The eating determination information is an image including a second eating action, which is an action taken by the subject after bringing food or drink to his / her mouth. A control device characterized by: As a result, the present disclosure further has the effect of being able to provide a control device that makes it possible to improve the accuracy of meal measurement compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a control device, the above control method, or the above program.

[0101] This embodiment further includes the following configuration. The second eating movement is a movement based on at least one of a movement of the mouth and a movement of the pharynx. A control device characterized by: As a result, the present disclosure further has the effect of being able to provide a control device that makes it possible to improve the accuracy of meal measurement compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a control device, the above control method, or the above program.

[0102] This embodiment further includes the following configuration. The information for meal determination is eating sounds that are likely to occur when eating. A control device characterized by: As a result, the present disclosure further has the effect of being able to provide a control device that makes it possible to improve the accuracy of meal measurement compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a control device, the above control method, or the above program.

[0103] This embodiment further includes the following configuration. The eating sound is one or more of chewing sounds and swallowing sounds. A control device characterized by: As a result, the present disclosure further has the effect of being able to provide a control device that makes it possible to improve the accuracy of meal measurement compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a control device, the above control method, or the above program.

[0104] This embodiment further includes the following configuration. The eating sounds are one or more of conversations about eating or cooking sounds. A control device characterized by: As a result, the present disclosure further has the effect of being able to provide a control device that makes it possible to improve the accuracy of meal measurement compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a control device, the above control method, or the above program.

[0105] This embodiment further includes the following configuration. The eating sounds are one or more of the following: tableware sounds or meal preparation sounds. A control device characterized by: As a result, the present disclosure further has the effect of being able to provide a control device that makes it possible to improve the accuracy of meal measurement compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a control device, the above control method, or the above program.

[0106] This embodiment further includes the following configuration. the information acquisition unit detects information related to meals using photographed images or sensor information acquired in time series, and acquires the meal determination information; A control device characterized by: As a result, the present disclosure further has the effect of being able to provide a control device that makes it possible to improve the accuracy of meal measurement compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a control device, the above control method, or the above program.

[0107] Embodiment 3 In the above-described embodiment, a configuration example including a case where meal determination is performed under one composite condition has been described. In the third embodiment, a configuration example of a form in which meal determination is performed by combining a plurality of composite conditions will be described. In the third embodiment, among the components of the third embodiment, the components and their functions that are similar to the components of the first or second embodiment already described are indicated by the same component names and the same symbols, and redundant explanations of those components are omitted as appropriate.

[0108] Next, a configuration example of a control device according to a third embodiment of the present disclosure will be described. FIG. 12 is a diagram showing a configuration example of a diet measurement system 1 (1C) including a control device 10 (10C) according to the third embodiment of the present disclosure. The diet measurement system 1C shown in FIG. 12 includes a control device 10 (10C), a measurement device 15 (15C), and an information source device 30 (30C).

[0109] Like the information source device 30B already described, the information source device 30C includes an imaging device 31, an odor sensor 32, a temperature sensor 33, a humidity sensor , a sound collecting device 35, and a human sensor .

[0110] The measuring device 15C is configured to include a meal measuring section 160 (160C) and a meal information output section 170 (170C), similar to the measuring device 15B already described. The meal measuring section 160C performs measurements related to meals and outputs the measurement results, similar to the meal measuring section 160B already described. The meal measuring section 160C performs measurements related to meals based on, for example, captured images. The meal information output unit 170C outputs the measurement results obtained by the meal measurement unit 160C to the outside, similar to the meal information output unit 170B already described. The meal information output unit 170C outputs the measurement results to a recording device for storage, for example. Alternatively, the information output unit 170C outputs the measurement results to an external server for storage, for example.

[0111] The control device 10C estimates whether the subject is eating or not eating based on a plurality of combined conditions, and causes the measuring device 15A to perform measurement if the subject is eating. The control device 10C shown in FIG. 12 includes an information acquisition unit 110 (110C), a meal determination unit 130 (130C), a determination rule storage unit 140 (140C), and a control instruction unit 150 (150C).

[0112] The information acquisition unit 110C has the same configuration as the information acquisition unit 110B already described. The information acquiring section 110C includes a measurement object information acquiring section 1110 (1110C) and a meal determination information acquiring section 1120 (1120C). The measurement object information acquiring section 1110C is configured in the same manner as the measurement object information acquiring section 1110B already described. The meal determination information acquisition unit 1120C is configured in the same manner as the meal determination information acquisition unit 1120B already described.

[0113] The determination rule storage unit 140C has the same configuration as the determination rule storage unit 140 already described.

[0114] Similar to the control command unit 150 already described, the control command unit 150C outputs a control signal to execute meal measurement when the result of the meal determination by the meal determination unit 130C indicates a meal. Specifically, the control instruction unit 150C acquires the determination result by the meal determination unit 130C, and when the determination result indicates a meal, outputs a control signal to instruct the start of meal measurement or the continuation of meal measurement.

[0115] If a composite condition, which is a combination of a plurality of conditions, is satisfied based on the measurement object information and the meal determination information, the meal determination unit 130C outputs a determination result indicating that the subject is eating. The meal determining section 130C includes a composite determining section 1310 (1310C). When the composite determination unit 1310C receives the measurement object information and the meal determination information, it outputs a determination result that determines whether the composite condition is satisfied based on the meal determination rules stored in the determination rule storage unit 140C. Furthermore, the composite determination unit 1310B uses the measurement object information and the meal determination information to determine whether each of a plurality of composite conditions is satisfied, and outputs the determination result.

[0116] The composite judgment unit 1310C outputs a judgment result indicating that the person is eating if the information for meal judgment acquired by the information acquisition unit 110C includes at least one of ``meal situation information indicating a situation that is likely to be a meal,'' ``an image including a first meal action'' which is the action of the person being measured bringing food or drink to their mouth, ``an image including a second meal action'' which is the action of the person being measured after bringing food or drink to their mouth, or ``eating sounds'' which are likely to be made when eating. The meal determining unit 130C outputs a determination result indicating that the event is a meal. The control command unit 150C outputs a control signal to execute meal measurement.

[0117] The composite determination unit 1310C The meal determination information acquired by the information acquisition unit 110C is If the information includes two or more of the following: "eating situation information indicating a situation where the subject is likely to be eating," "an image including a first eating action, which is an action of the subject bringing food or drink to their mouth," "an image including a second eating action, which is an action of the subject bringing food or drink to their mouth," or "eating sounds that may be made when eating," the meal determination unit outputs a determination result indicating that the food has been eaten; The control command unit outputs a control signal for causing the measurement subject to measure the food he or she eats. The composite determining section 1310C includes an n-th composite determining section 1320-n (1≦n≦N, N≧2).

[0118] The nth composite determination unit 1320-n (1≦n≦N, N≧2) determines each composite condition. Specifically, the first composite determination unit 1320-1 (n=1) determines whether the first composite condition is satisfied. The second composite determination unit 1320-2 (n=2) determines whether the second composite condition is satisfied. Similarly, each of a preset number of composite conditions determines whether the composite condition is satisfied, and the Nth composite determination unit 1320-1 (n=N) determines whether the Nth composite condition is satisfied. With this configuration, it is possible to adjust the cases in which a meal is determined to have occurred, such as by setting multiple composite conditions in advance and determining that a meal has occurred when a preset number of composite conditions are met.By making more appropriate settings, the accuracy of meal measurement can be further improved. In this case, a registration unit (not shown) may be provided to accept the settings as appropriate.

[0119] A processing example of the control device according to the third embodiment of the present disclosure will be described. The processing of the control device according to the third embodiment differs from the processing in the embodiments already described, particularly in the processing of the meal determining section 130C. Therefore, a description of the overall processing flow in the control device according to the third embodiment will be omitted, and the processing of the meal determining section 130C will be described below.

[0120] FIG. 13 is a flowchart showing a first example of the processing of the meal determining section 130 (130C). When the meal determination process starts (“START”), the meal determination unit 130C A plurality of pre-set and registered composite determination processes are executed in parallel (step ST3220-n (1≧n≧N, N≧2)). Specifically, the meal determining section 130C executes, for example, a first combined determination process (step ST3220-1) (n=1), . . . , an Nth combined determination process (step ST3220-N) (n=N, N≧2) in parallel.

[0121] The meal determining unit 130C then executes the meal determining process (step ST3221 "Meal?"). In the meal determination process, the composite determination unit 1310C of the meal determination unit 130C determines whether or not an object is a meal by determining whether the number of determination results indicating a meal in the first composite determination process (step ST3220-1) (n=1), ..., the Nth composite determination process (step ST3220-N) (n=N, N≧2) exceeds a threshold value that has been set in advance and registered.

[0122] If the number of determination results indicating a meal exceeds the threshold, the meal determining unit 130C determines that the meal is a meal (step ST3222). If the number of determination results indicating a meal does not exceed the threshold, the meal determining section 130C determines that the meal is not a meal (non-meal) (step ST3223).

[0123] The meal determining unit 130C then executes a determination result output process (step ST3224). In the determination result output process, the meal determining unit 130C outputs the determination result indicating whether the subject is eating or not to the control instruction unit 150C.

[0124] The meal determining unit 130C then executes an end determination process (step ST3225 "End?"). In the end determination process, the meal determination unit 130C checks whether the control device 10C will end the process, and if the control device 10C will end the process, the meal determination unit 130C determines that the process is to end. When determining not to end the process (step ST3225 "NO"), the meal determining section 130C proceeds to the process of step ST3220 (step ST3220-1 to step ST3220-N) and repeats the process from the process of step ST3220 (step ST3220-1 to step ST3220-N). When determining that the process should be ended ("YES" in step ST3225), the meal determining unit 130C ends the process ("end").

[0125] Here, a more specific example configuration of the meal determining unit 130C will be described. FIG. 14 is a diagram showing an example of the internal configuration when the composite judgment unit 1310 (1310C) in the meal judgment unit 130 (130C) is made up of a first composite judgment unit 1320-1, a second composite judgment unit 1320-2, a third composite judgment unit 1320-3, and a fourth composite judgment unit 1320-4. FIG. 4 shows an example of a configuration for determining four compound conditions. The composite determination section 1310 (1310C) shown in FIG. 4 includes a first composite determination section 1320-1, a second composite determination section 1320-2, a third composite determination section 1320-3, and a fourth composite determination section 1320-4.

[0126] The first compound condition is a condition that combines the condition that a person to be measured is present and the condition that food is present. The first composite determination unit 1320-1 executes a first composite determination process for determining a first composite condition using the measurement target information and the meal determination information, and outputs a determination result. Specifically, the first composite determination unit 1320-1 determines whether the condition that the measurement target is present is met based on the measurement target information, and determines that food is present using the meal determination information if two or more of "food present," "cooking equipment present," "change in smell," "change in temperature," and "change in humidity" are included, and outputs a determination result indicating whether or not the person is eating.

[0127] The second combined condition is a condition that combines a condition that the subject is present and a condition that the first eating behavior has been performed. The second combined determination unit 1320-2 executes a second combined determination process for determining a second combined condition using the measurement subject information and the meal determination information, and outputs a determination result. The second combined determination unit 1320-2 determines whether the condition that the measurement subject is present is met based on the measurement subject information, and determines that a first eating action has occurred when a hand or arm movement indicates a first eating action, such as picking up food or drink and bringing it to a mouth, and outputs a determination result indicating whether or not the person is eating.

[0128] The third combined condition is a condition that combines a condition that the subject is present and a condition that the second eating behavior has been performed. The third composite determination unit 1320-3 executes a third composite determination process for determining a third composite condition using the measurement subject information and the meal determination information, and outputs a determination result. The third composite determination unit 1320-3 determines whether the condition that the measurement subject is present is met based on the measurement subject information, and determines that a second eating action has occurred using the meal determination information when a second eating action is indicated, in which the movement of the mouth indicates chewing food or the movement of the pharynx indicates swallowing food, and outputs a determination result indicating whether or not the subject is eating.

[0129] The fourth combined condition is a condition that combines the condition that the subject is present and the condition that eating sounds are occurring. The fourth combined determination unit 1320-4 executes a fourth combined determination process for determining a fourth combined condition using the measurement subject information and the meal determination information, and outputs a determination result. The fourth combined determination unit 1320-4 determines whether the condition that the measurement subject is present is met based on the measurement subject information, determines the condition that eating sounds are occurring using the meal determination information, and outputs a determination result indicating whether or not eating.

[0130] Next, the processing of the meal determining unit 130C in the configuration shown in FIG. 14 will be described. FIG. 15 is a flowchart showing a first example of the processing in the meal determining section 130 (130C) in the case where the configuration shown in FIG. 14 is used. When the meal determining unit 130C starts the process ("start"), it executes four composite determination processes in parallel. in particular, The first combined determination section 1320-1 of the meal determining section 130C executes a first combined determination process (step ST3220-1) (n=1) and outputs a determination result indicating a meal or a non-meal. The second combined determination section 1320-2 of the meal determining section 130C executes a second combined determination process (step ST3220-2) (n=2) and outputs a determination result indicating a meal or a non-meal. The third combined determination section 1320-3 of the meal determining section 130C executes a third combined determination process (step ST3220-3) (n=3) and outputs a determination result indicating a meal or a non-meal. The fourth combined determination section 1320-4 of the meal determining section 130C executes a fourth combined determination process (step ST3220-4) (n=4) and outputs a determination result indicating a meal or a non-meal.

[0131] The meal determining section 130C then executes a process of determining whether or not the event is a meal, using the determination results and the determination conditions from the composite determining sections (step ST3221 "meal?"). When a determination condition such as when one composite condition is satisfied is set in advance, the meal determination unit 130C determines that an event is a meal when any one of the determination results by each composite determination unit indicates that an event is a meal. When a determination condition such as two combined conditions being satisfied is set in advance, the meal determining unit 130C determines that a meal has occurred when two of the determination results from the combined determining units indicate that the event is a meal. When a determination condition such as three composite conditions being satisfied is set in advance, the meal determining unit 130C determines that a meal has occurred when three of the determination results from the composite determining units indicate that a meal has occurred. When judgment conditions are set in advance, such as when four composite conditions are met (or when all composite conditions are met), the meal judgment unit 130C judges that an event is a meal when all of the judgment results from each composite judgment unit indicate that an event is a meal.

[0132] If the determination condition is met in step ST3221 ("YES" in step ST3221), the meal determining unit 130C determines that the event is a meal (step ST3222). If the determination condition is not satisfied in step ST3221 ("NO" in step ST3221), the meal determining unit 130C determines that it is not a meal (non-meal) (step ST3223).

[0133] The meal determining unit 130C then executes a determination result output process (step ST3224). In the determination result output process, the meal determining unit 130C outputs the determination result indicating whether or not the subject is eating to the command control unit 150C.

[0134] After outputting the determination result, the meal determining unit 130C then executes an end determination process (step ST3225 "end?"). In the end determination process, the meal determination unit 130C checks whether the control device 10C will end the process, and if the control device 10C will end the process, determines that the process of the meal determination unit 130C will end. When it is determined not to end the process ("NO" in step ST3225), the meal determining unit 130C proceeds to the process of steps ST3220-1, ST3220-2, ST3220-3, and ST3220-4, and repeats the process from the process of steps ST3220-1, ST3220-2, ST3220-3, and ST3220-4. When determining that the process should be ended ("YES" in step ST3225), the meal determining unit 130C ends the process ("end").

[0135] Next, the first composite determination process will be described. FIG. 16 is a flowchart showing an example of the process of the first composite determining section 1320-1. When the first composite judgment unit 1320-1 starts the first composite judgment process (“start”), the first composite judgment unit 1320-1 then executes a process to judge whether the multiple conditions included in the first composite condition are met (step ST33211 “Is the object to be measured present + food present?”). When it is determined in step ST33211 that a plurality of conditions are satisfied (step ST33211 "YES"), the first complex determining unit 1320-1 determines that the event is a meal (step ST33212). When it is determined in step ST33211 that any one of the conditions is not satisfied (step ST33211 "NO"), the first complex determining unit 1320-1 determines that it is not a meal (non-meal) (step ST33213). The first composite judgment unit 1320-1 then executes a judgment result output process (step ST33214). In the judgment result output process, the first composite judgment unit 1320-1 outputs a judgment result indicating whether or not the subject has eaten. After executing the determination result output process, first composite determination section 1320-1 ends the process ("end").

[0136] Here, an example of the food presence determination process in the first composite determination unit 1320-1 will be described. FIG. 17 is a flowchart showing a detailed example of the processing in the first composite determining section 1320-1. The first composite judgment unit 1320-1 starts processing when it is determined that a person to be measured is present based on the measurement object information (“Start”). The first composite judgment unit 1320-1 then executes a dishware judgment process (step ST33221). In the dishware judgment process, the first composite judgment unit 1320-1 analyzes the captured images based on the meal judgment information to perform a process to detect dishware, thereby determining whether dishware is present. Alternatively, the first composite judgment unit 1320-1 analyzes the captured images in chronological order based on at least one of the measurement object information and the meal judgment information, and determines that dishware is present when a state changes from one in which dishware is absent to one in which dishware is present.

[0137] The first composite determination unit 1320-1 then executes a food determination process (step ST33222). In the food determination process, the first composite determination unit 1320-1 analyzes the captured image based on at least one of the meal determination information to perform a food detection process to determine whether food is included. Alternatively, the first composite determination unit 1320-1 analyzes the captured images in a time series and determines that food is included when a state where no food is present changes to a state where food is present.

[0138] The first composite determination unit 1320-1 then executes an odor change determination process (step ST33223). In the odor change determination process, the first composite determination unit 1320-1 analyzes time-series odor sensor information based on at least one of the meal determination information to determine an odor change.

[0139] The first composite determination unit 1320-1 then executes a temperature change determination process (step ST33224). In the temperature change determination process, the first composite determination unit 1320-1 analyzes time-series temperature sensor information based on at least one of the meal determination information to determine a temperature change.

[0140] The first composite judgment unit 1320-1 then uses the judgment results from the processing of steps ST33221 to ST33224 to execute a process to judge whether the item corresponds to two or more of the following: tableware, cooking, change in smell, or change in temperature (step ST33225 "Are there two or more types?").

[0141] If the determination processing in step ST33225 corresponds to two or more types ("YES" in step ST33225), the first complex determination unit 1320-1 determines that the event is a meal (step ST33226).

[0142] When the determination process in step ST33225 does not match two or more types (step ST33225 "NO"), the first complex determination unit 1320-1 determines that it is not a meal (non-meal) (step ST33227).

[0143] The first composite judgment unit 1320-1 executes a judgment result output process (step ST33228). In the judgment result output process, the first composite judgment unit 1320-1 outputs a judgment result indicating whether or not the person has eaten.

[0144] After outputting the determination result, first composite determination unit 1320-1 ends the process ("end").

[0145] Next, the second composite determination process will be described. FIG. 18 is a flowchart showing an example of the process of the second composite determining section 1320-2. When the second composite judgment unit 1320-2 starts the second composite judgment process (“start”), the second composite judgment unit 1320-2 then executes a process to judge whether multiple conditions included in the second composite condition are satisfied (step ST33231 “Is the measured object present + first eating action?”).

[0146] When it is determined in step ST33231 that a plurality of conditions are satisfied (step ST33231 "YES"), the second complex determining unit 1320-2 determines that the event is a meal (step ST33232).

[0147] When it is determined in step ST33231 that any one of the conditions is not satisfied (step ST33231 "NO"), the second composite determining unit 1320-2 determines that the user is not eating (not eating) (step ST33233).

[0148] The second composite judgment unit 1320-2 then executes a judgment result output process (step ST33234). In the judgment result output process, the second composite judgment unit 1320-2 outputs a judgment result indicating whether or not the subject has eaten.

[0149] After executing the determination result output process, the second composite determination unit 1320-2 ends the process ("end").

[0150] Here, a detailed example of the second eating movement determination process in the second composite determination section 1320-2 will be described. FIG. 19 is a flowchart showing a detailed example of the processing in the second composite determining section 1320-2. The second composite determination unit 1320-2 starts processing (“start”) when it is determined that the subject is present based on the subject information (in the case of first eating behavior determination).

[0151] The second combined determination unit 1320-2 then determines the gesture of holding the dishware (step ST33241). The second combined determination unit 1320-2 determines the gesture of holding the dishware based on at least one of the measurement object information and the meal determination information.

[0152] The second composite determination unit 1320-2 then determines the action of holding cutlery (step ST33242).

[0153] The second composite determination unit 1320-2 then determines the action of bringing the food to the mouth (step ST33243).

[0154] The second composite judgment unit 1320-2 then uses the judgment results from the processing of steps ST33241 to ST33243 to execute a process to judge whether the action applied to two or more of the following actions: holding tableware, holding cutlery, and bringing food to the mouth (step ST33244 "Are there two or more actions?").

[0155] If the determination processing in step ST33244 corresponds to two or more types ("YES" in step ST33244), the second complex determination unit 1320-2 determines that the event is a meal (step ST33245).

[0156] If the determination process in step ST33244 does not correspond to two or more types ("NO" in step ST33244), the second complex determination unit 1320-2 determines that the event is not a meal (non-meal) (step ST33246).

[0157] The second composite judgment unit 1320-2 executes a judgment result output process (step ST33247). In the judgment result output process, the second composite judgment unit 1320-2 outputs a judgment result indicating whether or not the subject has eaten. After outputting the determination result, second composite determination section 1320-2 ends the process ("end").

[0158] Next, the third composite determination process will be described. FIG. 20 is a flowchart showing an example of the process of the third composite determining section 1320-3. When the third composite determination unit 1320-3 starts the third composite determination process (“START”),

[0159] The third composite determining unit 1320-3 then executes a process of determining whether or not a plurality of conditions included in the third composite condition are satisfied (step ST33251 "Is a measurement target present+is a second eating behavior occurring?").

[0160] When it is determined in step ST33251 that a plurality of conditions are satisfied (step ST33251 "YES"), the third complex determining unit 1320-3 determines that the event is a meal (step ST33252).

[0161] When it is determined in step ST33251 that a plurality of conditions are satisfied (step ST33251 "NO"), the third complex determining unit 1320-3 determines that it is not a meal (non-meal) (step ST33253).

[0162] The third composite judgment unit 1320-3 then executes a judgment result output process (step ST33254). In the judgment result output process, the third composite judgment unit 1320-3 outputs a judgment result indicating whether or not the subject has eaten. After executing the determination result output process, the third composite determination unit 1320-3 ends the process ("end").

[0163] Here, an example of the third eating movement determination process in the third composite determining unit 1320-3 will be described. FIG. 21 is a flowchart showing a detailed example of the processing in the third composite determining section 1320-3. The third composite determining unit 1320-3 starts processing ("start") when it is determined that a person to be measured is present based on the information on the person to be measured.

[0164] The third composite determination unit 1320-3 then executes a mouth movement change determination process (step ST33261). In the mouth movement change determination process, the third composite determination unit 1320-3 analyzes time-series images based on at least one of the measurement object information and the meal determination information to determine and acquire a mouth movement change.

[0165] The third composite determining unit 1320-3 then executes a process of determining whether there is a change in the movement of the mouth (step ST33262 "Is there a change in the movement of the mouth?").

[0166] When it is determined that there is a change in the movement of the mouth ("YES" in step ST33262), the third complex determining unit 1320-3 determines that the subject is eating (step ST33263).

[0167] When it is determined that there is no change in the movement of the mouth ("NO" in step ST33262), the third complex determining unit 1320-3 determines that it is not a meal (non-meal) (step ST33264).

[0168] Furthermore, the third composite determination unit 1320-3 executes a pharynx movement change determination process (step ST33265). In the pharynx movement change determination process, the third composite determination unit 1320-3 analyzes time-series images based on at least one of the measurement object information and the meal determination information to determine and acquire a pharynx movement change.

[0169] The third composite determining unit 1320-3 executes a process of determining whether there is a change in the movement of the pharynx (step ST33266 "Is there a change in the movement of the pharynx?"). When it is determined in the determination process of step ST33266 that there is a change in the movement of the pharynx ("YES" in step ST33266), the third complex determination unit 1320-3 determines that the subject is eating (step ST33267). When it is determined in the determination process of step ST33266 that there is no change in the movement of the pharynx (step ST33266 "NO"), the third complex determining unit 1320-3 determines that there is no meal (step ST33268).

[0170] The third composite judgment unit 1320-3 executes a judgment result output process (step ST33269). In the judgment result output process, the third composite judgment unit 1320-3 outputs a judgment result indicating whether or not the subject has eaten. After outputting the determination result, the third composite determination unit 1320-3 ends the process ("end").

[0171] Next, the fourth composite determination process will be described. FIG. 22 is a flowchart showing an example of the process of the fourth composite determining section 1320-4. When the fourth composite judgment unit 1320-4 starts the fourth composite judgment process (“start”), the fourth composite judgment unit 1320-4 then executes a process to judge whether multiple conditions included in the fourth composite condition are satisfied (step ST33271 “Is a measured object present + eating sound?”).

[0172] When it is determined in step ST33271 that a plurality of conditions are satisfied (step ST33271 "YES"), the fourth complex determining unit 1320-4 determines that the event is a meal (step ST33272). When it is determined in step ST33271 that a plurality of conditions are satisfied (step ST33271 "NO"), the fourth complex determining unit 1320-4 determines that it is not a meal (non-meal) (step ST33273).

[0173] The fourth composite judgment unit 1320-4 then executes a judgment result output process (step ST33274). In the judgment result output process, the fourth composite judgment unit 1320-4 outputs a judgment result indicating whether or not the subject has eaten. After executing the determination result output process, the fourth composite determination unit 1320-4 ends the process ("end").

[0174] Here, an example of the fourth eating movement determination process in the fourth composite determining unit 1320-4 will be described. FIG. 23 is a flowchart showing a detailed example of the processing in the fourth composite determining section 1320-4. The fourth composite determining unit 1320-4 starts processing ("start") when it is determined that a person to be measured is present based on the information on the person to be measured.

[0175] The fourth composite determination unit 1320-4 then executes a meal utterance determination process (step ST33281). In the meal utterance determination process, the fourth composite determination unit 1320-4 performs image analysis or sound analysis based on at least one of the measurement object information and the meal determination information to determine a conversation related to a meal and acquire meal utterance information.

[0176] The fourth composite judgment unit 1320-4 then executes cooking sound judgment processing (step ST33282). In the cooking sound judgment processing, the fourth composite judgment unit 1320-4 performs sound analysis based on the meal judgment information to judge whether a conversation is about eating, and acquires cooking sound information.

[0177] The fourth compound judgment unit 1320-4 then executes a dishware sound judgment process (step ST33283). In the dishware sound judgment process, the fourth compound judgment unit 1320-4 performs sound analysis based on the meal judgment information to judge whether the conversation is related to eating, and acquires dish sound information.

[0178] The fourth compound judgment unit 1320-4 then executes a process of judging whether the sound corresponds to two or more of eating utterances, cooking sounds, and tableware sounds, using the judgment results from the processes of steps ST33281 to ST33283 (step ST33284 "Are there two or more types?").

[0179] If the determination processing in step ST33284 corresponds to two or more types ("YES" in step ST33284), the fourth composite determination unit 1320-4 determines that the person is eating (step ST33285).

[0180] If the determination process of step ST33284 does not correspond to two or more types ("NO" in step ST33284), the fourth composite determination unit 1320-4 determines that the person is not eating (not eating) (step ST33286).

[0181] The fourth composite judgment unit 1320-4 then executes a judgment result output process (step ST33287). In the judgment result output process, the fourth composite judgment unit 1320-4 outputs a judgment result indicating whether or not the subject has eaten. After outputting the determination result, the fourth composite determination unit 1320-4 ends the process ("end").

[0182] Next, a second example of the processing of the meal determination unit 130 (130C) will be described. In the above-mentioned process, the process of determining all pre-set composite conditions is performed before making a meal determination. However, here we will explain an example of a process in which multiple composite conditions are determined in a pre-set order, and once a determination condition is met, the determination of subsequent composite conditions can be omitted. FIG. 24 is a flowchart showing a second example of the processing by the meal determining section 130 (130C). When the meal determining unit 130C starts processing ("start"), it first initializes the composite determination number n (n=0).

[0183] The meal determining unit 130C then performs a process of incrementing the number n (step ST3230 "n=n+1").

[0184] The meal determining unit 130C then executes the nth ((1≦n≦N) (N≧2)) combined determination process (step ST3231 "nth combined determination (1≦n≦N) (N≧2)").

[0185] The meal determining unit 130C then executes a process of determining whether or not the event is a meal, using the determination result of the n-th combined determination process (step ST3232 "meal?").

[0186] When the determination result of the n-th combined determination process indicates a meal ("YES" in step ST3232), the meal determining unit 130C determines that it is a meal (step ST3233). When the determination result of the n-th combined determination process indicates that it is not a meal (non-meal) (step ST3232 "NO"), the meal determining unit 130C determines that it is not a meal (non-meal) (step ST3234).

[0187] Here, if the system is configured to make a final judgment based on the number of judgment results from the multiple composite judgments already described, the number of judgment results is counted up each time the judgment result of the nth composite judgment process indicates a meal, and the number of judgment results is compared with a threshold value to make a final judgment.

[0188] The meal determining section 130C then executes a determination result output process (step ST3235). In the determination result output process, the meal determining unit 130C outputs a determination result indicating whether the subject is eating or not eating.

[0189] After outputting the determination result, the meal determining unit 130C then executes an end determination process (step ST3236 "End?"). In the end determination process, the meal determination unit 130C checks whether the control device 10C will end the process, and if the control device 10C will end the process, determines that the process of the meal determination unit 130C will end. When determining not to end the process ("NO" in step ST3236), the meal determining unit 130C proceeds to the process of step ST3230 and repeats the process from step ST3230. When determining that the process should be ended ("YES" in step ST3236), the meal determining unit 130C ends the process ("end").

[0190] Next, a second example of the processing in the meal determining section 130C in the configuration shown in FIG. 14 will be described with reference to FIG. When the meal determining unit 130C starts processing (“start”), it first initializes (n=0) the compound determination identification number n (1≦n≦N) (N=4).

[0191] The meal determining unit 130C then performs a process of incrementing the number n (step ST3230 "n=n+1").

[0192] The meal determining unit 130C then executes a first combined determination process (step ST3231). In the first combined determination process, the first combined determination unit 1320-1 of the meal determining unit 130C determines whether a subject is present based on the measurement subject information, and determines whether food is present based on at least one of the measurement subject information and the meal determination information.

[0193] The meal determining section 130C then executes a process of determining whether or not it is a meal, using the determination result of the first combined determination process (step ST3232 "meal?"). When the determination result of the first combined determination process indicates a meal ("YES" at step ST3232), the meal determining unit 130C determines that the event is a meal (step ST3233). When the determination result of the first combined determination process indicates that the event is not a meal (non-meal) (step ST3232 "NO"), the meal determining unit 130C determines that the event is not a meal (non-meal) (step ST3234).

[0194] The meal determining section 130C then executes a determination result output process (step ST3235). In the determination result output process, the meal determining unit 130C outputs a determination result indicating whether the subject is eating or not eating.

[0195] After outputting the determination result, the meal determining unit 130C then executes an end determination process (step ST3236 "End?"). In the end determination process, the meal determination unit 130C determines whether the determination result does not indicate a meal and whether the identification number n of the composite determination has reached a preset number N (N=4). If the determination result indicates a meal, or if the meal determination unit 130C determines that the identification number n of the composite determination has reached a preset number N (N=4), the process ends. If the determination result does not indicate a meal and the number N (N=4) has not been reached, the meal determining section 130C does not end the process but repeats the process from step ST3230.

[0196] If the determination result of the first combined determination process is not a meal, the meal determination unit 130C then executes a second combined determination process. In the second combined determination process, the second combined determination unit 1320-2 of the meal determination unit 130C determines whether the subject is present based on the measurement subject information, and determines whether the first meal activity is based on at least one of the measurement subject information and the meal determination information.

[0197] If the determination result of the second combined determination process is not a meal, the meal determination unit 130C then executes a third combined determination process. In the third combined determination process, the third combined determination unit 1320-3 of the meal determination unit 130C determines whether the subject is present based on the measurement subject information, and determines whether the second meal activity is present based on at least one of the measurement subject information and the meal determination information.

[0198] If the determination result of the third combined determination process is not a meal, the meal determination unit 130C then executes a fourth combined determination process. In the fourth combined determination process, the fourth combined determination unit 1320-4 of the meal determination unit 130C determines whether a subject is present based on the measurement subject information, and determines whether eating sounds are occurring based on at least one of the measurement subject information and the meal determination information.

[0199] Next, a second example of the processing in the meal determining section 130 (130C) in the configuration shown in FIG. 14 will be described. FIG. 25 is a flowchart showing a second example of the processing in the meal determining section 130 (130C) in the case where the configuration shown in FIG. 14 is used. When the meal determining unit 130C starts the process ("start"), it executes four composite determination processes in parallel. in particular, The first combined determination section 1320-1 of the meal determining section 130C executes a first combined determination process (step ST3240-1) (n=1) and outputs a determination result indicating a meal or a non-meal. The second combined determination section 1320-2 of the meal determining section 130C executes the second combined determination process (step ST3240-2) (n=2) and outputs a determination result indicating a meal or a non-meal. The third combined determination section 1320-3 of the meal determining section 130C executes a third combined determination process (step ST3240-3) (n=3) and outputs a determination result indicating a meal or a non-meal. The fourth combined determination section 1320-4 of the meal determining section 130C executes a fourth combined determination process (step ST3240-4) (n=4) and outputs a determination result indicating a meal or a non-meal.

[0200] The meal determining section 130C then executes a process of determining whether or not the event is a meal, using the determination results and the determination conditions from the composite determining sections (step ST3241 "meal?"). When a determination condition such as when one composite condition is satisfied is set in advance, the meal determination unit 130C determines that an event is a meal when any one of the determination results by each composite determination unit indicates that an event is a meal. When a determination condition such as two combined conditions being satisfied is set in advance, the meal determining unit 130C determines that a meal has occurred when two of the determination results from the combined determining units indicate that the event is a meal. When a determination condition such as three composite conditions being satisfied is set in advance, the meal determining unit 130C determines that a meal has occurred when three of the determination results from the composite determining units indicate that a meal has occurred. When judgment conditions are set in advance, such as when four composite conditions are met (or when all composite conditions are met), the meal judgment unit 130C judges that an event is a meal when all of the judgment results from each composite judgment unit indicate that an event is a meal.

[0201] If the determination condition is met in step ST3241 ("YES" in step ST3241), the meal determining unit 130C determines that the event is a meal (step ST3242).

[0202] If the determination condition is not satisfied in step ST3241 (step ST3241 "NO"), the meal determining section 130C determines that it is not a meal (non-meal) (step ST3243).

[0203] The meal determining section 130C then executes a determination result output process (step ST3244). In the determination result output process, the meal determining unit 130C outputs the determination result indicating whether or not the subject is eating to the command control unit 150C.

[0204] After outputting the determination result, the meal determining unit 130C then executes an end determination process (step ST3245 "end?"). In the end determination process, the meal determination unit 130C determines whether a determination result indicating a meal has been output or whether the composite determination identification number n has reached a preset number N (1≦n≦N, 2≦N). When the meal determining unit 130C outputs a determination result indicating a meal, or when it determines that the identification number n of the composite determination has reached a preset number N, the meal determining unit 130C ends the process. If the meal determining section 130C outputs a determination result indicating no meal and if the number N has not been reached, the process does not end but repeats the process from step ST3240.

[0205] The control device 10C checks whether the processing is to be ended, and if the control device 10C is to end the processing, it determines that the processing of the meal determining unit 130C is to be ended. When it is determined not to end the process ("NO" in step ST3245), the meal determining unit 130C proceeds to the process of steps ST3240-1, ST3240-2, ST3240-3, and ST3240-4, and repeats the process from the process of steps ST3240-1, ST3240-2, ST3240-3, and ST3240-4. When determining that the process should be ended ("YES" in step ST3245), the meal determining unit 130C ends the process ("end").

[0206] This embodiment further includes the following configuration. The meal determination information acquired by the information acquisition unit is Meal status information indicating a situation where the meal may be occurring; an image including a first eating action, which is an action of the subject bringing food or drink to his / her mouth; an image including a second eating action, which is an action by the subject after bringing food or drink to his / her mouth; or The sounds that may occur when eating, If it contains at least one of the meal determination unit outputs a determination result indicating that the person is eating; The control command unit outputs a control signal to execute measurement of the meal. A control device characterized by: As a result, the present disclosure further has the effect of being able to provide a control device that makes it possible to improve the accuracy of meal measurement compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a control device, the above control method, or the above program.

[0207] This embodiment further includes the following configuration. The meal determination information acquired by the information acquisition unit is Meal status information indicating a situation where the meal may be occurring; an image including a first eating action, which is an action of the subject bringing food or drink to his / her mouth; an image including a second eating action, which is an action by the subject after bringing food or drink to his / her mouth; or The sounds that may occur when eating, If it contains two or more of the following: the meal determination unit outputs a determination result indicating that the person is eating; The control command unit outputs a control signal for causing the measurement subject to measure the food he or she eats. A control device characterized by: As a result, the present disclosure further has the effect of being able to provide a control device that makes it possible to improve the accuracy of meal measurement compared to conventional devices. Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a system including a control device, the above control method, or the above program.

[0208] Embodiment 4 In the above-described embodiment, the control device is a device separate from the measurement device. In the fourth embodiment, a configuration example in which the control device and the measurement device are integrated will be described. In the fourth embodiment, among the components of the fourth embodiment, the components and their functions that are similar to the components of the first, second, or third embodiment already described are indicated by the same component names and the same symbols, and redundant explanations of those components are omitted as appropriate.

[0209] A configuration example of a control device according to a fourth embodiment of the present disclosure will be described. FIG. 26 is a diagram showing a configuration example of a diet measurement system 1 (1D) including a control device 10 (10D) and a measurement device 15 (15D) according to the fourth embodiment of the present disclosure. A diet measurement system 1D shown in FIG. 26 includes a control device 10 (10D), a measurement device 15 (15D), and an information source device 30 (30D).

[0210] The information source device 30 (30D) is configured to include an imaging device 31, an odor sensor 32, a temperature sensor 33, a humidity sensor 34, a sound collection device 35, and a human presence sensor 36, similar to the information source device 30B already described.

[0211] The measuring device 15D is different from the measuring device 15B or the measuring device 15C already described, and is configured to include a control device 10D. The control device 10D may be configured in the same manner as any of the control devices 10 already described. The control device 10D shown in FIG. 26 has the same configuration as the control device 10C of the control devices 10 already described. The measuring device 15D shown in Figure 26 is configured to include an information acquisition unit 110 (110D), a meal determination unit 130 (130D), a determination rule storage unit 140 (140D), a control instruction unit 150 (150D), a meal measurement unit 160 (160D), and a meal information output unit 170 (170D).

[0212] The information acquisition unit 110D has the same configuration as any of the information acquisition units 110 already described. The information acquiring section 110D shown in FIG. 26 includes a measurement subject information acquiring section 1110 (1110D) and a meal determination information acquiring section 1120 (1120D).

[0213] The meal determination unit 130D is configured in the same manner as any of the meal determination units 130 already described. The meal determination unit 130D shown in Fig. 26 is configured to include a composite determination unit 1310 (1310D). The composite determination unit 1310D is configured to include an n-th composite determination unit 1320-n (1 ≤ n ≤ N, N ≥ 2).

[0214] The determination rule storage unit 140D has the same configuration as any of the determination rule storage units 140 already described. The control command unit 150D has the same configuration as any of the control command units 150D already described. The meal measurement unit 160D is configured in the same manner as any of the meal measurement units 160 already described, and starts or ends meal measurement, which is measurement related to meals, in accordance with a control signal output by the control command unit 150D in the control device 10D. The meal information output section 170D is configured in the same manner as any of the meal information output sections 170 already described, and outputs meal information based on the measurement results of the meal measurement by the meal measurement section 160D.

[0215] A processing example of the measurement device according to the fourth embodiment of the present disclosure will be described. FIG. 27 is a flowchart showing an example of processing by the measuring device 15 (15D) according to the fourth embodiment of the present disclosure. The process shown in FIG. 27 is a control method performed by the measuring device 15D. For example, the measuring device 15D shown in FIG. 26 starts the process shown in FIG. 27 when a control start condition is satisfied, such as when the measuring device 15D is in operation, for example, when the power of the measuring device 15D is ON, or when the measuring device 15D is in a state where it can measure food. ("Start")

[0216] Next, the measuring equipment 15D executes the information acquisition process (step ST4100). In the information acquisition process, the information acquisition unit 110D of the measuring device 15D acquires information about the subject to be measured that indicates the presence of the subject to be measured based on the captured image, and also acquires information for meal determination, which is information about meals, using the captured image or sensor information. The information acquiring section 110D outputs the measurement object information to the meal determining section 130D. In addition, the information acquiring section 110D outputs information for meal determination to the meal determining section 130D.

[0217] Next, the measuring device 15D executes a meal determination process (step ST4200). In the meal determination process, the meal determination unit 130D of the measuring device 15D outputs a determination result indicating that a meal has occurred when a composite condition, which is a combination of multiple conditions, is satisfied based on the measurement subject information and the meal determination information. Specifically, upon receiving the measurement target information and the meal determination information from the meal determination unit 130D, the meal determination unit 130D uses the measurement target information and the meal determination information to determine whether a composite condition, which is a combination of multiple pre-set conditions, is met. If the composite condition is met, the meal determination unit 130D outputs a determination result indicating a meal to the control command unit 150D. The meal determination unit 130D may output information indicating a meal or not as the determination result.

[0218] Next, the measuring device 15D executes a control command process (step ST4300). In the control command processing, when the determination result indicates a meal, the control command unit 150D of the measuring device 15D outputs a control signal to execute measurement of a meal. Specifically, when control instruction unit 150D receives the determination result from meal determination unit 130D, if the determination result indicates a meal, it outputs a control signal to start meal measurement if meal measurement is not being performed, or continues meal measurement if meal measurement is being performed.If the determination result indicates a non-meal, control instruction unit 150D continues the state in which meal measurement is not being performed if meal measurement is not being performed, or outputs a control signal to stop (end) meal measurement if meal measurement is being performed.

[0219] Next, the measuring device 15D executes a meal measurement process (step ST4400). In the meal measurement process, the meal measurement unit 160D of the measurement device 15D starts or ends meal measurement, which is measurement related to meals, in accordance with the control signal output by the control command unit 150D in the control device 10D.

[0220] Next, measuring equipment 15D executes an end determination process (step ST4500 "End?"). In the termination determination process, a control unit (not shown) of the measuring device 15D determines whether to terminate the processing of the measuring device 15D. The control unit (not shown) determines whether to terminate the processing of the measuring device 15D in accordance with, for example, an external termination command or an execution program. When the control unit (not shown) determines not to end the processing of the measuring device 15D ("NO" in step ST4500), the process proceeds to step ST4100, and the process is repeated from step ST4100. When the control unit (not shown) determines that the process of the measuring device 15D is to be ended ("YES" in step ST4500), the measuring device 15D ends the process ("end").

[0221] FIG. 28 is a flowchart showing an example of the processing of the control instruction unit 150 (150D). The control instruction unit 150D starts processing when it receives the determination result from the meal determination unit 130D ("START"). Control command unit 150D then executes a command type determination process (step ST4410). If the command type is to continue measurement or start measurement, control command unit 150D executes a process of issuing a command to perform measurement and bringing the device into a measurement state (step ST4411). If the command type is to stop measurement or not continue measurement, control command unit 150D issues a command to stop measurement and executes processing to bring about a measurement stop state (step ST4412). In addition, in the case of non-measurement continuation, the control command unit 150D does not need to issue a command since no measurement is being performed. Control command unit 150D then ends the process ("End").

[0222] This embodiment shows a configuration including the following. the control device; a meal measurement unit that starts or ends meal measurement, which is measurement related to meals, in accordance with a control signal output by the control command unit in the control device; a meal information output unit that outputs meal information based on the measurement results of the meal measurement; A measuring device equipped with: As a result, the present disclosure has the effect of providing a measurement device that enables the accuracy of meal measurement to be improved compared to conventional devices.

[0223] Embodiment 5. In the above-described fourth embodiment, the form of the measurement device integrated with the control device has been described. In the fifth embodiment, a configuration example in which the diet measurement system is configured with a terminal device and a server will be described. In the fifth embodiment, among the components of the fifth embodiment, those components and their functions that are similar to those of the components of the first, second, third, or fourth embodiment already described are indicated by the same component names and the same symbols, and redundant explanations of those components are omitted as appropriate.

[0224] A configuration example of a control device and a diet measurement system including the control device according to a fifth embodiment of the present disclosure will be described. FIG. 29 is a diagram showing a configuration example of a diet measurement system 1 (1E) including a control device 10 (10E) and a measurement device 15 (15E) according to the fifth embodiment of the present disclosure. The meal measurement system 1 (1E), like any of the meal measurement systems 1 already described, is configured to include a control device 10 (10E), a measurement device 15 (15E), and an information source device 30 (30E). The information source device 30E is configured in the same manner as any of the information source devices 30 already described, and in particular, the information source device 30E shown in Figure 29 is configured to include an imaging device 31, an odor sensor 32, a temperature sensor 33, a humidity sensor 34, a sound collection device 35, and a human presence sensor 36.

[0225] The dietary measurement system 1E is composed of a terminal device 2 (2E) on the side of the subject to be measured, and a server 3 (3E) on the side of providing the dietary measurement service. The diet measurement system 1E shown in FIG. 29 includes a terminal device 2E that functions as a measurement device 15E, and a server 3E that functions as a control device 10E. The terminal device 2E outputs the measurement results of the dietary measurement of the measurement subject to the server 3E, and receives dietary information that is the result of analysis by the server 3E using the measurement results of the dietary measurement.

[0226] The terminal device 2E is configured to include a configuration as the measuring device 15E and an output device (output unit) 190 (190E). The terminal device 2E may be configured to include the information source device 30E already described. The terminal device 2E includes a detection unit 120 (120E), a meal measurement unit 160 (160E), and a meal information output unit 170 (170E).

[0227] The detector 120E detects information related to meals using captured images or sensor information acquired in time series. The detection section 120E shown in FIG. 29 includes a measurement object information acquisition section 1110 (1110E) and a meal determination information acquisition section 1120 (1120E). The measurable object information acquisition unit 1110 (1110E) is configured to include a measurable object detection unit 1111, similar to the measurable object information acquisition unit 1110B of the measurable object information acquisition unit 1110 already described, and detects the image area of ​​the measurable object using the captured image and acquires measurable object information indicating the presence of the measurable object based on the captured image. The information acquisition unit for meal determination 1120 (1120E) is configured to include a meal status detection unit 1121, a first eating motion detection unit 1122, a second eating motion detection unit 1123, and a eating sound detection unit 1124, similar to the information acquisition unit for meal determination 1120 already described, particularly the information acquisition unit for meal determination 1120B, and detects information related to meals using captured images or sensor information acquired in chronological order, and acquires information for meal determination, which is information related to meals. Furthermore, the detection unit 120E "outputs to the server information on the subject to be measured indicating the presence of the subject to be measured when the subject to be measured is included in the photographed image based on the photographed image, and outputs to the server information for meal determination, which is information related to meals, using the photographed image or sensor information."

[0228] The meal measurement unit 160E is configured in the same manner as any of the meal measurement units 160 already described, and starts or ends meal measurement, which is measurement related to meals, in accordance with a control signal output by the control instruction unit 150E in the control device 10E. Specifically, when the meal measurement unit 160E receives a control signal output from the server based on the output measurement object information and meal determination information, it starts or ends meal measurement in accordance with the received control signal.

[0229] The meal information output unit 170E is configured in the same manner as any of the meal information output units 170 already described, and outputs meal information based on the measurement results of the meal measurement by the meal measurement unit 160E.

[0230] The server 3E receives the measurement results of the dietary measurement of the measurement subject from the terminal device, and outputs dietary information, which is the result of analysis using the measurement results of the dietary measurement, to the terminal device. The server 3E is configured to include the configuration of the control device 10E. The server 3E shown in Figure 29 is configured to include an information acquisition unit 110 (110E), a meal determination unit 130 (130E), a determination rule storage unit 140 (140E), a control instruction unit 150 (150E), and an analysis device (analysis unit) 180 (180E).

[0231] The information acquisition unit 110E "receives from the terminal device information on the subject to be measured that indicates the presence of the subject to be measured based on the captured image, and also receives from the terminal device information for meal determination, which is information on meals, using the captured image or sensor information." The information acquiring section 110E includes a measurement object information acquiring section 111 and a meal determination information acquiring section 112. The measurement target information acquisition unit 111 receives the measurement target information output by the terminal device 15E. The meal determination information acquisition unit 112 receives meal determination information output by the terminal device 15E.

[0232] Like any of the meal determination units 130 already described, the meal determination unit 130E is configured to "output a determination result indicating that a meal has occurred when a composite condition, which is a combination of multiple conditions, is satisfied based on the information on the object to be measured and the information for meal determination." The meal determining section 130E shown in FIG. 29 is configured similarly to the meal determining sections 130C and 130D of the meal determining section 130 already described, and includes a composite determining section 1310 (1310E). The composite determining section 1310E includes an n-th composite determining section 1320-n (1≦n≦N, N≧2).

[0233] Control command unit 150E, like any of the control command units 150 already described, is configured to "output a control signal to a terminal device to perform meal measurement if the judgment result indicates a meal."

[0234] A processing example of a meal measurement system including a control device according to a fifth embodiment of the present disclosure will be described. FIG. 30 is a sequence diagram showing an example of processing of the diet measurement system 1 (1E) including the control device 10 (10E) and the measurement device 15 (15E) according to the fifth embodiment of the present disclosure. In the diet measurement system 1E, first, the information source device 30E (terminal device 2E) executes a captured image output process (step ST5010). Next, the information source device 30E (terminal device 2E) executes a sensor information output process (step ST5020). 29 outputs odor sensor information, temperature sensor information, humidity sensor information, and sound data. If the information source device 30E is equipped with a human presence sensor 36, it also outputs sensor information output by the human presence sensor 36.

[0235] When the information source device 30E (terminal device 2E) executes the captured image output process, the measuring device 15E (terminal device 2E) accepts the captured image and executes the measurement object detection process (step ST5110). When the information source device 30E (terminal device 2E) executes the sensor information output process, the measuring device 15E (terminal device 2E) receives the sensor information and executes the measured object information output process (step ST5112).

[0236] The measuring device 15E (terminal device 2E) detects information related to meals using captured images or sensor information, and outputs meal determination information to the server 3E. Specifically, the detection unit 120E of the terminal device outputs to the server 3E information about the subject to be measured indicating the presence of the subject to be measured when the subject to be measured is included in the captured image based on the captured image, and also outputs to the server 3E information for meal determination, which is information about meals, using the captured image or sensor information. The measuring device 15E (terminal device 2E) then executes eating state detection processing (step ST5113). In the eating state detection processing, the eating state detection unit 1121 in the detection unit 120E of the measuring device 15E (terminal device 2E) outputs eating state information indicating a state that is likely to be a meal, using a captured image or sensor information.

[0237] The measuring device 15E (terminal device 2E) then executes a first eating motion detection process (step ST5114). In the first eating motion detection process, the first eating motion detection unit 1122 in the detection unit 120E of the measuring device 15E (terminal device 2E) acquires captured images or sensor information in chronological order and performs an analysis process. If the first eating motion detection unit 1122 detects at least one of hand movement and arm movement of the person to be measured as a result of the analysis process, it then performs a process to detect a hand movement or arm movement such as picking up food and bringing it to a mouth. If the first eating motion detection unit 1122 detects a movement such as picking up food and bringing it to a mouth, it outputs information indicating that the first eating motion has been performed or an image including the first eating motion as information for meal determination.

[0238] The measuring device 15E (terminal device 2E) then executes a second eating motion detection process (step ST5115). In the second eating motion detection process, the second eating motion detection unit 1123 in the detection unit 120E of the measuring device 15E (terminal device 2E) acquires captured images or sensor information in chronological order and performs an analysis process. If the second eating motion detection unit 1123 detects at least one of the mouth movement or pharyngeal movement of the subject as a result of the analysis process, it then performs a process to detect whether the mouth movement indicates chewing food or whether the pharyngeal movement indicates swallowing food. If the second eating motion detection unit 1123 detects whether the mouth movement indicates chewing food or whether the pharyngeal movement indicates swallowing food, it outputs information indicating that the second eating motion has been performed or an image including the second eating motion as information for meal determination.

[0239] The measuring device 15E (terminal device 2E) then executes a meal sound detection process (step ST5116). In the meal sound detection process, the meal sound detection unit 1124 in the detection unit 120E of the measuring device 15E (terminal device 2E) acquires sound data from the sound collection device. The meal sound detection unit 1124 analyzes the sound data and performs a process to detect meal sounds that are sounds related to eating. When the eating sound detection unit 1124 detects meal sounds that are sounds related to eating, it outputs information indicating the meal sounds as meal determination information.

[0240] The measuring device 15E (terminal device 2E) then executes a meal determination information output process (step ST5118). In the meal determination information output process, the meal determination information output unit 1120E in the detection unit 120E of the measuring device 15E (terminal device 2E) outputs meal determination information to the control device 10E (server 3E).

[0241] Upon receiving the measurement target information, the control device 10E (server 3E) executes a measurement target information acquisition process (step ST5308).

[0242] Upon receiving the information for meal determination, the control device 10E (server 3E) executes a process for obtaining information for meal determination (step ST5309).

[0243] The control device 10E (server 3E) executes the meal determination process (step ST5310). The control device 10E (server 3E) receives the determination result of the meal determination process and executes command processing according to whether the person is eating or measuring (step ST5311), (step ST5312), (step ST5313), (step ST5314), (step ST5315), and (step ST5316). This processing is similar to (step ST2312), (step ST2313), (step ST2315), and (step ST2316) already explained in FIG. 11. The control device 10E (server 3E) executes a control command process (step ST5320). In the control command process, the control command unit 150E of the control device 10E (server 3E) outputs a control signal to the measuring device 15E (terminal device 2E) in accordance with the determination result of the meal determination process.

[0244] Upon receiving the control signal output from the control device 10E (server 3E), the measuring device 15E (terminal device 2E) executes a command reception process (step ST5400). The measuring device 15E (terminal device 2E) then executes operation processing in accordance with the command. Specifically, the measuring device 15E (terminal device 2E) executes processing when the command is to start measurement (step ST5410), processing when the command is to start measurement (step ST5411), and processing when the command is to stop measurement (step ST5412). If the command is to not continue measurement, the measuring device 15E (terminal device 2E) does not perform the meal measurement operation. When the meal measurement unit 160E of the terminal device 2E receives a control signal output from the server based on the output information on the object to be measured and information for meal determination, it starts or ends meal measurement according to the received control signal.

[0245] This embodiment shows a configuration including the following. A terminal device that outputs a measurement result of a dietary measurement of a subject to a server and receives dietary information that is a result of analysis by the server using the measurement result of the dietary measurement, A detection unit that "outputs information about the subject to be measured indicating the presence of the subject to be measured to a server when the subject to be measured is present, and outputs information for meal determination, which is information about meals, to the server using the captured image or sensor information." A meal measurement unit that "when receiving a control signal output from the server based on the output measurement object information and meal determination information, starts or ends meal measurement according to the received control signal"; A terminal device comprising: As a result, the present disclosure has the effect of providing a terminal device that enables the accuracy of meal measurement to be improved compared to conventional devices.

[0246] This embodiment shows a configuration including the following. A server that receives a measurement result of a dietary measurement of a subject from a terminal device and outputs dietary information, which is a result of an analysis using the measurement result of the dietary measurement, to the terminal device, an information acquisition unit that "receives information about a subject to be measured that indicates the presence of the subject to be measured from a terminal device, and receives information for meal determination, which is information about meals, from the terminal device using captured images or sensor information"; a meal determination unit that "outputs a determination result indicating that the subject has eaten if a composite condition, which is a combination of multiple conditions, is satisfied based on the information on the measurement object and the information for meal determination"; a control command unit that "outputs a control signal to a terminal device to execute measurement of a meal when the determination result indicates a meal"; A server comprising: As a result, the present disclosure has the effect of providing a server that enables the accuracy of meal measurement to be improved compared to conventional methods.

[0247] This embodiment shows a configuration including the following. A dietary measurement system including a terminal device that outputs dietary measurement results for a subject to be measured to a server and receives dietary information that is a result of analysis by the server using the dietary measurement results, The server an information acquisition unit that "receives information about a subject to be measured that indicates the presence of the subject to be measured from a terminal device, and receives information for meal determination, which is information about meals, from the terminal device using captured images or sensor information"; a meal determination unit that "outputs a determination result indicating that the subject has eaten if a composite condition, which is a combination of multiple conditions, is satisfied based on the information on the measurement object and the information for meal determination"; a control command unit that "outputs a control signal to a terminal device to execute measurement of a meal when the determination result indicates a meal"; Equipped with The terminal device A detection unit that "outputs information about the subject to be measured indicating the presence of the subject to be measured to the server when the subject to be measured is present, and outputs information for meal determination, which is information about meals, to the server using a captured image or sensor information." A meal measurement unit that "when receiving a control signal output from the server based on the output measurement object information and meal determination information, starts or ends meal measurement according to the received control signal"; Equipped with A meal measurement system characterized by: As a result, the present disclosure has the effect of providing a meal measurement system that enables the accuracy of meal measurement to be improved compared to conventional systems.

[0248] This embodiment further includes the following configuration. A control method for a terminal device that outputs measurement results of dietary measurements on a subject to a server and receives dietary information that is a result of analysis by the server using the results of the dietary measurements, a detection step in which the detection unit of the terminal device "outputs, when the subject is present, information on the subject that indicates the presence of the subject to the server, and outputs information for meal determination, which is information on meals, to the server using the captured image or sensor information"; a meal measurement step in which the meal measurement unit of the terminal device "starts or ends meal measurement according to the received control signal when it receives a control signal output from the server based on the output measurement object information and meal determination information"; A control method comprising: As a result, the present disclosure has the effect of providing a control method that makes it possible to improve the accuracy of meal measurement compared to conventional methods.

[0249] This embodiment shows a configuration including the following. Computer, A terminal device that outputs measurement results of dietary measurements on a subject to a server and receives dietary information that is a result of analysis by the server using the results of the dietary measurements, "When the person to be measured is present, output information on the person to be measured indicating the presence of the person to be measured to the server, and a detection unit that outputs meal determination information, which is information about meals, to the server using the captured image or sensor information; a meal measurement unit that "when receiving a control signal output from the server based on the output measurement subject information and meal determination information, starts or ends measurement of the measurement subject's meal according to the received control signal"; a terminal device comprising: A program characterized by operating as As a result, the present disclosure has the effect of providing a program that realizes a terminal device that improves the accuracy of meal measurement compared to conventional devices.

[0250] Here, a hardware configuration for realizing the functions of the present disclosure will be described. FIG. 31 is a diagram illustrating a first example of a hardware configuration for realizing the functions according to the configuration of the present disclosure. FIG. 32 is a diagram illustrating a second example of a hardware configuration for realizing the functions according to the configuration of the present disclosure. The control devices 10, 10A, 10B, 10C, 10D, and 10E or the measuring devices 15 (15A, 15B, 15C, 15D, and 15E) of the present disclosure are realized by hardware such as that shown in FIG. 31 or FIG. 32, respectively.

[0251] Each of the control devices 10, 10A, 10B, 10C, 10D, and 10E, or the measuring device 15 (15A, 15B, 15C, 15D, and 15E), is composed of, for example, a processor 10001, a memory 10002, an input / output interface 10003, and a communication circuit 10004, as shown in FIG. The processor 10001 and the memory 10002 are, for example, installed in a computer. The memory 10002 stores the computer, information acquisition units 110, 110A, 110B, 110C, 110D, and 110E, meal determination units 130, 130A, 130B, 130C, 130D, and 130E, determination rule storage units 140, 140A, 140B, 140C, 140D, and 140E, a determination rule storage unit 141, a prediction model unit 142, and a 2, learning model unit 143, control command unit 150, 150A, 150B, 150C, 150D, 150E, meal measurement unit 160, 160B, 160C, 160D, 160E, meal information output unit 170, 170B, 170C, 170D, 170E, analysis device (analysis unit) 180, 180E, output device (output unit) 190, 190E, measured The devices store programs for functioning as measurement object information acquisition units 1110, 1110A, 1110B, 1110C, 1110D, 1110E, measurement object detection unit 1111, meal judgment information acquisition units 1120, 1120A, 1120B, 1120C, 1120D, 1120E, meal status detection unit (food, smell, temperature, humidity) 1121, first meal motion detection unit (hands, arms) 1122, second meal motion detection unit (mouth, pharynx) 1123, meal sound detection unit 1124, composite judgment units 1310, 1310B, 1310C, 1310D, 1310E, nth composite judgment unit (1≦n≦N, N≧2) 1320-1, ..., 1320-n, and a control unit not shown.The processor 10001 reads out and executes the programs stored in the memory 10002, whereby the information acquisition units 110, 110A, 110B, 110C, 110D, 110E, meal determination units 130, 130A, 130B, 130C, 130D, 130E, determination rule holding units 140, 140A, 140B, 140C, 140D, 140E, determination rule memory unit 141, prediction model unit 142, learning model unit 143, control command units 150, 150A, 150B, 150C, 150D, 150E, meal measurement units 160, 160B, 160C, 160D, 160E, meal information output units 170, 170B, 170C, 170D, 170E, analysis devices (analysis units) 180, 180E, Output devices (output units) 190, 190E, measurement object information acquisition units 1110, 1110A, 1110B, 1110C, 1110D, 1110E, measurement object detection unit 1111, meal determination information acquisition units 1120, 1120A, 1120B, 1120C, 1120D, 1120E, meal status detection unit (food, smell, temperature, humidity) 1121, The functions of a first eating motion detection unit (hand, arm) 1122, a second eating motion detection unit (mouth, pharynx) 1123, an eating sound detection unit 1124, composite determination units 1310, 1310B, 1310C, 1310D, 1310E, an nth composite determination unit (1≦n≦N, N≧2) 1320-1, ..., 1320-n, and a control unit not shown are realized. Furthermore, a storage unit (not shown) is realized by the memory 10002 or another memory (not shown). Furthermore, the communication circuit 10004 realizes a communication unit (not shown).

[0252] The processor 10001 is, for example, a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, a microcontroller, or a digital signal processor (DSP). Memory 10002 may be a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable Read Only Memory) or flash memory, or a magnetic disk such as a hard disk or flexible disk, or an optical disk such as a CD (Compact Disc) or DVD (Digital Versatile Disc), or a magneto-optical disk. The processor 10001 and the memory 10002 or the communication circuit 10004 are connected in a state where they can transmit data to each other. The processor 10001, the memory 10002, and the communication circuit 10004 are also connected in a state where they can transmit data to other hardware via the input / output interface 10003.

[0253] Or, the information acquisition units 110, 110A, 110B, 110C, 110D, 110E, meal determination units 130, 130A, 130B, 130C, 130D, 130E, and determination rule storage units 140, 140A, 140B, 140C in the control devices 10, 10A, 10B, 10C, 10D, 10E or the measurement devices 15 (15A, 15B, 15C, 15D, 15E) , 140D, 140E, judgment rule memory unit 141, prediction model unit 142, learning model unit 143, control command unit 150, 150A, 150B, 150C, 150D, 150E, meal measurement unit 160, 160B, 160C, 160D, 160E, meal information output unit 170, 170B, 170C, 170D, 170E, analysis device (analysis unit) 180, 180E, output device (output unit) 190, 190E, measurable object information acquisition units 1110, 1110A, 1110B, 1110C, 1110D, 1110E, measurable object detection unit 1111, meal determination information acquisition units 1120, 1120A, 1120B, 1120C, 1120D, 1120E, meal status detection unit (food, smell, temperature, humidity) 1121, first meal action detection unit (hand, arm) 1122, The functions of the second eating movement detection unit (mouth, pharynx) 1123, the eating sound detection unit 1124, the composite determination units 1310, 1310B, 1310C, 1310D, 1310E, the nth composite determination unit (1≦n≦N, N≧2) 1320-1, ..., 1320-n, and the control unit (not shown) may be realized by a dedicated processing circuit 20001, as shown in Figure 32.

[0254] The processing circuit 20001 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field-Programmable Gate Array), an SoC (System-on-a-Chip), or a system LSI (Large-Scale Integration). Furthermore, the memory 20002 or another memory not shown implements a storage unit not shown. Memory 20002 may be a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable Read Only Memory) or flash memory, or a magnetic disk such as a hard disk or flexible disk, or an optical disk such as a CD (Compact Disc) or DVD (Digital Versatile Disc), or a magneto-optical disk. Furthermore, the communication circuit 20004 realizes a communication unit (not shown). The processing circuit 20001 and the memory 20002 or the communication circuit 20004 are connected in a state where they can transmit data to each other. In addition, the processing circuit 20001, the memory 20002, and the communication circuit 20004 are connected in a state where they can transmit data to each other and to other hardware via the input / output interface 20003. The information acquisition units 110, 110A, 110B, 110C, 110D, and 110E, the meal determination units 130, 130A, 130B, 130C, 130D, and 130E, and the determination rule storage units 140, 140A, 140B, and 140E in the control devices 10, 10A, 10B, 10C, 10D, and 10E or the measurement devices 15 (15A, 15B, 15C, 15D, and 15E) C, 140D, 140E, a determination rule storage unit 141, a prediction model unit 142, a learning model unit 143, a control command unit 150, 150A, 150B, 150C, 150D, 150E, a meal measurement unit 160, 160B, 160C, 160D, 160E, a meal information output unit 170, 170B, 170C, 170D, 170E, an analysis device (analysis unit) 180, 180E, an output device ( output unit) 190, 190E, measurement object information acquisition units 1110, 1110A, 1110B, 1110C, 1110D, 1110E, measurement object detection unit 1111, meal determination information acquisition units 1120, 1120A, 1120B, 1120C, 1120D, 1120E, meal status detection unit (food, smell, temperature, humidity) 1121, first meal action detection unit (hand, arm) 11 22, the functions of the second eating action detection unit (mouth, pharynx) 1123, the eating sound detection unit 1124, the composite determination units 1310, 1310B, 1310C, 1310D, 1310E, the nth composite determination unit (1≦n≦N, N≧2) 1320-1, ..., 1320-n, and the control unit (not shown) may be realized by separate processing circuits, or may be realized together by a processing circuit.

[0255] Alternatively, the information acquisition units 110, 110A, 110B, 110C, 110D, 110E, meal determination units 130, 130A, 130B, 130C, 130D, 130E, and determination rule storage units 140, 140A, 140B, 140C, 140D, 140E in the control devices 10, 10A, 10B, 10C, 10D, 10E or the measurement devices 15 (15A, 15B, 15C, 15D, 15E) may be 40E, determination rule storage unit 141, prediction model unit 142, learning model unit 143, control command unit 150, 150A, 150B, 150C, 150D, 150E, meal measurement unit 160, 160B, 160C, 160D, 160E, meal information output unit 170, 170B, 170C, 170D, 170E, analysis device (analysis unit) 180, 180E, output device (output unit) 190, 190E, measurement target Elephant information acquisition units 1110, 1110A, 1110B, 1110C, 1110D, 1110E, measurement object detection unit 1111, meal determination information acquisition units 1120, 1120A, 1120B, 1120C, 1120D, 1120E, meal status detection unit (food, smell, temperature, humidity) 1121, first meal action detection unit (hands, arms) 1122, second meal action detection unit (mouth, pharynx) 1123, Some of the functions of the eating sound detection unit 1124, the composite judgment units 1310, 1310B, 1310C, 1310D, 1310E, the nth composite judgment unit (1≦n≦N, N≧2) 1320-1, ..., 1320-n, and a control unit (not shown) may be realized by the processor 10001 and memory 10002, and the remaining functions may be realized by the processing circuit 20001.

[0256] It should be noted that, within the scope of this disclosure, the embodiments may be freely combined, any component of each embodiment may be modified, or any component of each embodiment may be omitted.

[0257] The present disclosure can improve the accuracy of meal measurement compared to conventional methods, and is therefore suitable for use in, for example, a control device that controls a measurement device that performs meal measurement, a measurement device that includes such a control device, etc. [Explanation of symbols]

[0258] 1 (1A, 1B, 1C, 1D, 1E) Meal measurement system, 2 (2E) Terminal device, 3 (3E) Server, 10, 10A, 10B, 10C, 10D, 10E Control device, 15 (15A, 15B, 15C, 15D, 15E) Measuring device, 30 (30A, 30B, 30C, 30D, 30E) Information source device, 31 Imaging device, 32 Odor sensor, 33 Temperature sensor, 34 Humidity sensor, 35 Sound collection device, 36 Human presence sensor, 110, 110A, 110B, 110C, 110D, 110E Information acquisition unit, 130, 130A, 130B, 130C, 130D, 130E Meal determination unit, 140, 140A, 140B, 140C, 140D, 140E determination rule holding unit, 141 determination rule memory unit, 142 prediction model unit, 143 learning model unit, 150, 150A, 150B, 150C, 150D, 150E control command unit, 160, 160B, 160C, 160D, 160E meal measurement unit, 170, 170B, 170C, 170D, 170E meal information output unit, 180, 180E analysis device (analysis unit), 190, 190E output device (output unit), 1110, 1110A, 1110B, 1110C, 1110D, 1110E measurement object information acquisition unit, 1111 measurement object detection unit, 1120, 1120A, 1120B, 1120C, 1120D, 1120E meal judgment information acquisition unit, 1121 meal status detection unit (cooking, smell, temperature, humidity), 1122 First eating motion detection unit (hand, arm), 1123 second eating motion detection unit (mouth, pharynx), 1124 eating sound detection unit, 1310, 1310B, 1310C, 1310D, 1310E composite judgment units, 1320-1,...,1320-n nth composite judgment unit (1≦n≦N, N≧2), 10001 processor, 10002 memory, 10003 input / output interface, 10004 communication circuit, 20001 processing circuit, 20002 memory, 20003 input / output interface, 20004 communication circuit.

Claims

1. an information acquisition unit that acquires information about a subject to be measured that indicates the presence of the subject to be measured, and acquires information for meal determination that is information about meals using a captured image or sensor information; a meal determination unit that outputs a determination result indicating that the subject is eating when a composite condition, which is a combination of a plurality of conditions, is satisfied based on the measurement object information and the meal determination information; a control command unit that outputs a control signal to execute measurement of a meal when the determination result indicates that the person is eating; A control device comprising:

2. When a plurality of subjects are included, The information acquisition unit acquires, for each subject, subject information indicating an area in which the subject is present. The meal determining unit outputs a determination result indicating that each subject is eating. The control command unit outputs a control signal for executing measurement of the meal for each measurement subject.

2. The control device according to claim 1.

3. The information for meal determination is meal situation information indicating a situation in which the meal is likely to occur.

3. The control device according to claim 1 or 2.

4. The meal status information indicates at least one of the presence of food, whether there is a change in smell, whether there is a change in temperature, or whether there is a change in humidity.

4. The control device according to claim 3.

5. The information for meal determination is an image including a first eating action, which is an action of the measurement subject bringing food or drink to his / her mouth.

3. The control device according to claim 1 or 2.

6. The first eating motion is a motion based on at least one of a hand motion and an arm motion. The control device according to claim 5 .

7. The information for meal determination is an image including a second eating action, which is an action taken by the subject after bringing food or drink to his / her mouth.

3. The control device according to claim 1 or 2.

8. The second eating action is an action based on at least one of a movement of the mouth and a movement of the pharynx. The control device according to claim 7 .

9. The information for meal determination is eating sounds that are likely to occur when eating.

3. The control device according to claim 1 or 2.

10. The eating sound is one or more of a chewing sound or a swallowing sound.

10. The control device according to claim 9.

11. The eating sounds are one or more of a conversation about eating or a cooking sound.

10. The control device according to claim 9.

12. The eating sounds are one or more of tableware sounds and meal preparation sounds.

10. The control device according to claim 9.

13. The meal determination information acquired by the information acquisition unit is Meal status information indicating a situation where the meal may be occurring; an image including a first eating action, which is an action of the measurement subject bringing food or drink to his / her mouth; an image including a second eating action, which is an action by the subject after bringing food or drink to his / her mouth; or The sounds that may occur when eating, If it contains at least one of the meal determination unit outputs a determination result indicating that the person is eating; The control command unit outputs a control signal to execute measurement of the meal.

3. The control device according to claim 1 or 2.

14. The meal determination information acquired by the information acquisition unit is Meal status information indicating a situation where the meal may be occurring; an image including a first eating action, which is an action of the measurement subject bringing food or drink to his / her mouth; an image including a second eating action, which is an action by the subject after bringing food or drink to his / her mouth; or The sounds that may occur when eating, If it contains two or more of the meal determination unit outputs a determination result indicating that the person is eating; The control command unit outputs a control signal for causing the measurement subject to measure the food he or she eats.

3. The control device according to claim 1 or 2.

15. The meal determination unit inputs information about the object to be measured and information for meal determination into a trained model, and outputs a determination result indicating that the object is a meal using the determination result information indicating whether the object is a meal or not output by the trained model.

3. The control device according to claim 1 or 2.

16. the information acquisition unit detects information related to meals using photographed images or sensor information acquired in time series, and acquires the meal determination information; 3. The control device according to claim 1 or 2.

17. The control device according to claim 1 or 2; a meal measurement unit that starts or ends meal measurement, which is measurement related to a meal, in accordance with a control signal output by the control command unit in the control device; a meal information output unit that outputs meal information based on the measurement results of the meal measurement; A measuring device equipped with:

18. A terminal device that outputs a measurement result of a dietary measurement of a subject to a server and receives dietary information that is a result of analysis by the server using the measurement result of the dietary measurement, a detection unit that outputs, when the subject is present, information about the subject to be measured that indicates the presence of the subject to be measured to a server, and outputs information for meal determination, which is information about meals, to the server using the captured image or sensor information; a meal measurement unit that, when receiving a control signal output from the server based on the output measurement object information and meal determination information, starts or ends meal measurement in accordance with the received control signal; A terminal device comprising:

19. A server that receives a measurement result of a dietary measurement of a subject from a terminal device and outputs dietary information, which is a result of an analysis using the measurement result of the dietary measurement, to the terminal device, an information acquisition unit that receives, from a terminal device, information about a subject to be measured that indicates the presence of the subject to be measured, and receives, from the terminal device, information for meal determination that is information about meals using a captured image or sensor information; a meal determination unit that outputs a determination result indicating that the subject is eating when a composite condition, which is a combination of a plurality of conditions, is satisfied based on the measurement object information and the meal determination information; a control instruction unit that outputs a control signal to a terminal device to execute measurement of a meal when the determination result indicates that the meal has been eaten; A server comprising:

20. A dietary measurement system including a terminal device that outputs dietary measurement results for a subject to be measured to a server and receives dietary information that is a result of analysis by the server using the dietary measurement results, The server an information acquisition unit that receives, from a terminal device, information about a subject to be measured that indicates the presence of the subject to be measured, and receives, from the terminal device, information for meal determination that is information about meals using a captured image or sensor information; a meal determination unit that outputs a determination result indicating that the subject is eating when a composite condition, which is a combination of a plurality of conditions, is satisfied based on the measurement object information and the meal determination information; a control instruction unit that outputs a control signal to the terminal device to execute measurement of a meal when the determination result indicates that the meal has been eaten; Equipped with The terminal device a detection unit that outputs, when the subject is present, information about the subject to be measured that indicates the presence of the subject to be measured to the server, and outputs information for meal determination, which is information about meals, to the server using the captured image or sensor information; a meal measurement unit that, when receiving a control signal output from the server based on the output measurement object information and meal determination information, starts or ends meal measurement in accordance with the received control signal; Equipped with A meal measurement system characterized by:

21. A control method by a control device, an information acquisition step in which an information acquisition unit of the control device acquires information on the subject to be measured that indicates the presence of the subject to be measured, and acquires information for meal determination, which is information related to meals, using a captured image or sensor information; a meal determination step in which the meal determination unit of the control device outputs a determination result indicating that the subject has eaten when a composite condition, which is a combination of multiple conditions, is satisfied based on the measurement object information and the meal determination information; a control command step in which a control command unit of the control device outputs a control signal to execute measurement of a meal when the determination result indicates that the meal has been eaten; Equipped with A control method comprising:

22. A control method for a terminal device that outputs measurement results of dietary measurements on a subject to a server and receives dietary information that is a result of analysis by the server using the results of the dietary measurements, a detection step in which a detection unit of the terminal device outputs, when a subject to be measured is present, information on the subject to be measured indicating the presence of the subject to be measured to the server, and outputs information for meal determination, which is information on meals, to the server using the captured image or sensor information; a meal measurement step in which, when the meal measurement unit of the terminal device receives a control signal output from the server based on the output measurement object information and meal determination information, the meal measurement unit starts or ends meal measurement according to the received control signal; A control method comprising:

23. Computer, an information acquisition unit that acquires information about a subject to be measured that indicates the presence of the subject to be measured, and acquires information for meal determination that is information about meals using a captured image or sensor information; a meal determination unit that outputs a determination result indicating that the subject has eaten when a composite condition, which is a combination of a plurality of conditions, is satisfied based on the measurement object information and the meal determination information; a control command unit that outputs a control signal to execute measurement of a meal when the determination result indicates that the meal has been eaten; a control device comprising: A program characterized by operating as

24. Computer, A terminal device that outputs measurement results of dietary measurements on a subject to a server and receives dietary information that is a result of analysis by the server using the results of the dietary measurements, If the subject to be measured is present, output the subject information indicating the presence of the subject to be measured to the server; a detection unit that outputs meal determination information, which is information related to meals, to the server using captured images or sensor information; a diet measurement unit that, when receiving a control signal output from the server based on the output measurement subject information and meal determination information, starts or ends measurement of the measurement subject's meal according to the received control signal; a terminal device comprising: A program characterized by operating as

Images