Eating behavior analysis system and program

Abstract

To provide an eating behavior analysis system and program that automatically analyzes eating behavior of a user with small burden on the user and supports a proposal of more rational and healthier eating behavior.SOLUTION: In an eating behavior analysis system 1, a server includes a control unit including: an eating behavior estimation unit that identifies characteristic information related to cooking and eating from measurement information measured by a sensor such as an image sensor that measures measurement information related to cooking and eating for a refrigerator or cooking appliance, and uses the characteristic information to estimate eating behavior including cooking behavior indicating the content of cooking and eating behavior indicating the content of a user's meal; an eating behavior analysis unit that analyzes eating behavior, and calculates cooking characteristics indicating the burden of cooking and eating characteristics including health characteristics indicating the impact of eating on the user's health; and a proposal information creation unit that creates proposal information including recommended eating behavior advice based on the calculated eating characteristics. A terminal includes an output unit that outputs the created proposal information to at least the user.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a technique for analyzing a user's eating habits. [Background technology]

[0002] With the current trend of increasing health consciousness and attention being paid to food waste, there is a growing need to understand users' eating behavior, including food consumption. Here, eating behavior refers to behavior from cooking food to eating (ingesting food), and is composed of cooking behavior and eating behavior. Moreover, cooking behavior is composed of cooking steps and the foods used in the cooking, and eating behavior refers to the acquisition of food, that is, consumption behavior.

[0003] In order to understand eating behavior, it is considered to manage food. For this purpose, some refrigerators that store food can use sensors installed in the refrigerator to detect food inventory and food being taken in and out of the refrigerator, and can manage the food inventory in the refrigerator. For example, some refrigerators use a camera attached to the refrigerator to capture an image when the door is opened and detect the inventory and food being taken in and out from the image, while others use a weight sensor to obtain the weight of the food in the refrigerator and detect the consumption of the food. However, even if the taking in and taking out of the refrigerator is detected, it is not possible to infer how the food taken out of the refrigerator is cooked and served as a meal (dish).

[0004] In addition, some cooking appliances that perform cooking can detect the cooking status of the appliance using sensors installed in the main unit. For example, there are oven ranges and cooking heaters that use a camera attached to the heating chamber to capture images of the interior and use the images to detect what food is being cooked, and rice cookers that use a weight sensor to measure the weight of rice. However, even if a cooking appliance can detect the cooking status, it cannot obtain information on the intake of foods that are not cooked, making it difficult to analyze the user's eating behavior and support healthy eating habits.

[0005] In response to this, a system has been proposed that captures and analyzes photos of food as a way of acquiring and analyzing information about a user's eating habits, such as what they ate and when.

[0006] For example, Patent Document 1 describes an eating behavior improvement support system that includes "an eating behavior detection and analysis means for analyzing the eating behavior of a user" and "generates an inappropriate eating behavior suppression signal" when "it is determined that the user's eating behavior is inappropriate."

[0007] For example, Patent Document 2 describes a cooking content determination system and an intake content determination system that include "a first camera that photographs the cooking status of food in the kitchen and a second camera that photographs the cooking status of food in the dining room" and "determines the cooking content based on the image capture results of each camera." [Prior art documents] [Patent documents]

[0008] [Patent Document 1] JP 2011-107768 A [Patent Document 2] JP 2019-101736 A Summary of the Invention [Problem to be solved by the invention]

[0009] Conventional systems for detecting eating behavior often rely on the user to input information, such as by taking photos of their meals or inputting what they have eaten. This means that users have to enter information manually, which is time-consuming and can lead to users forgetting to enter information. As a result, the systems are unable to detect eating behavior accurately.

[0010] The problems with the devices described in Patent Documents 1 and 2 are described below. In the eating behavior improvement support system described in Patent Document 1, a system is presented in which an eating behavior detection and analysis means analyzes the meal contents based on a photograph taken, and an eating behavior suitability judgment means judges whether the eating behavior is appropriate or not. Images are used as input, and when the user "takes a photograph of the meal contents he or she is about to eat," the eating behavior is recorded, and the photograph of the meal is analyzed and processed to calculate information such as the calorie content. Therefore, it is necessary for the user to take a photograph of the meal before eating, which is time-consuming for the user to input, and if the user forgets to take a photograph and starts eating, an accurate analysis cannot be performed. Even if the type and amount of food can be determined from the photograph of the meal, it may be difficult to guess what ingredients are included in the meal, such as finely chopped vegetables or kneaded ingredients.

[0011] The cooking contents discrimination system and intake contents discrimination system described in Patent Document 2 present a system that discriminates cooking contents and dietary intake contents by installing cameras that capture the cooking situation in the kitchen and dining room of a house. Because the camera is installed on the ceiling of the house, there is a possibility that not only the cooking contents to be discriminated but also the user himself / herself will be unintentionally captured in the camera, and users may be reluctant to have the camera installed due to the risk of violating their privacy.

[0012] As described above, in Patent Documents 1 and 2, the analysis of eating behavior relies on the user's voluntary input, so the records may be inaccurate, and the user may be conscious of recording their eating behavior, which may make the system difficult to accept. For this reason, the system does not automatically and casually obtain daily eating behavior without the user being aware of it.

[0013] The present invention has been made in consideration of these circumstances, and its main objective is to obtain more accurate information about eating behavior and support more rational and healthier eating behavior, without causing the user any trouble or aversion, simply by performing a series of normal actions such as cooking and eating. [Means for solving the problem]

[0014] In order to solve the above problems, the present invention estimates a user's eating behavior based on characteristic information identified from measurement information from multiple cooking-related appliances, calculates eating characteristics for the eating behavior by analyzing the estimated eating behavior, and creates eating behavior advice based on the calculated eating characteristics and outputs it to the user and related parties.

[0015] A more specific aspect of the present invention is an eating behavior analysis system having a sensor that measures measurement information related to cooking and eating for each of a plurality of cooking-related appliances, an eating behavior estimation unit that identifies characteristic information related to the cooking and the meal from the measured measurement information and uses the characteristic information to estimate eating behavior including cooking behavior indicating the content of the cooking and eating behavior indicating the content of the user's meal, an eating behavior analysis unit that analyzes the eating behavior and calculates cooking characteristics indicating the cost of cooking and eating characteristics including health characteristics indicating the health impact of the meal on the user, a proposal information creation unit that creates proposal information including recommended eating behavior advice based on the calculated dietary characteristics, and an output unit that outputs the created proposal information to at least the user.

[0016] The eating behavior analysis system of the present invention is preferably composed of a terminal that can be realized by a computer, an eating behavior analysis device that can be realized by a server, and various cooking-related home appliances. The present invention also includes each device that constitutes the eating behavior analysis system and a subcombination that is a combination of parts of them. Furthermore, the present invention also includes a program for causing each of these devices to function as a computer and a storage medium that stores the program. Furthermore, the present invention also includes an eating behavior analysis method executed by the eating behavior analysis system, the subcombination, and each device. Effect of the Invention

[0017] According to the present invention, the eating behavior of a user can be automatically analyzed with little burden on the user, making it possible to support the suggestion of more rational and healthier eating behavior. [Brief description of the drawings]

[0018] [Figure 1] 1 is a schematic diagram of an example of an eating behavior analysis system 1 according to a first embodiment. [Diagram 2] 1 is an example of a block diagram of an eating behavior analysis system 1 according to a first embodiment. [Diagram 3] 2 is an example of a block diagram of a control unit 22 of the cooking-related home appliance of the first embodiment. [Figure 4] 2 is an example of a block diagram illustrating details of a server 10 according to the first embodiment. [Diagram 5] 2 is an example of a block diagram of a terminal 4 according to the first embodiment. [Figure 6] 1 is a diagram illustrating an example of user management information 1021 according to the first embodiment. [Figure 7] 13 is a diagram illustrating an example of measurement information 1022 according to the first embodiment. [Figure 8] 13 is a diagram showing an example of eating behavior information 1023 according to the first embodiment. [Figure 9] 1 is an example of a diagram showing food information 1024 in the first embodiment. [Figure 10] 1 is a diagram illustrating an example of cooking-related home appliance information 1025 according to the first embodiment. [Figure 11] 1 is an example of a diagram for explaining an eating behavior inference process for inferring an eating behavior 11 according to the first embodiment. FIG. [Figure 12] 1 is an example of a diagram showing an example of estimation of eating behavior and output processing after the eating behavior estimation processing in the first embodiment. FIG. [Figure 13] 1 is an example of a flowchart showing a process flow of an eating behavior analysis method according to a first embodiment. [Figure 14] 1 is a schematic diagram of an example of an eating behavior analysis system 1 according to a second embodiment. [Figure 15] FIG. 11 is an example of a block diagram of an eating behavior analysis system 1 according to a second embodiment. [Figure 16] 13 is an example of a diagram for explaining an eating behavior inference process for inferring an eating behavior 11 according to a second embodiment. FIG. [Figure 17] 13 is a schematic diagram of an example of an eating behavior analysis system 1 according to a third embodiment. [Figure 18]13 is a diagram illustrating an example of an estimation of an eating behavior and an eating behavior estimation process according to a third embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] An embodiment of the present invention will be described below. In this embodiment, a user's eating behavior is analyzed by targeting cooking and eating in an ordinary household. Here, the user is a person who consumes food, and cooking may be performed by the user himself or by another person such as a family member of the user.

[0020] To this end, in this embodiment, an eating behavior analysis system is provided that includes a sensor that measures measurement information related to cooking and eating for each of a plurality of cooking-related home appliances, an eating behavior estimation unit that identifies characteristic information related to the cooking and the meal from the measured measurement information and uses the characteristic information to estimate eating behavior including cooking behavior indicating the content of the cooking and eating behavior indicating the content of the user's meal, an eating behavior analysis unit that analyzes the eating behavior and calculates cooking characteristics that indicate the cost of cooking and eating characteristics that indicate the health impact of the meal on the user, a proposal information creation unit that creates proposal information including recommended eating behavior advice based on the calculated dietary characteristics, and an output unit that outputs the created proposal information to at least the user.

[0021] This concludes the description of the present embodiment, and examples that are specific examples of the present embodiment will now be described in detail with reference to the drawings. In each example, so-called energy saving, that is, energy efficiency in cooking, is used as the main example of a rationalization index for cooking behavior. However, cost effectiveness in terms of expenses and cooking effort (efficiency index) can also be used as a rationalization index. EXAMPLES

[0022] First, a first embodiment will be described with reference to Fig. 1 to Fig. 13. Fig. 1 is an example of a schematic diagram of an eating behavior analysis system 1 according to the first embodiment. Fig. 2 is an example of a block diagram of the eating behavior analysis system 1 according to the first embodiment.

[0023] [Overall configuration] First, the configuration of an eating behavior analysis system 1 according to a first embodiment will be described with reference to Figures 1 and 2. The eating behavior analysis system 1 according to the present embodiment is composed of a refrigerator 2, a cooking appliance 3, a server 10, and a terminal 4, which are connected to each other in a manner enabling information communication. The refrigerator 2 and the cooking appliance 3 are installed in a house 50. The refrigerator 2 is an example of a storage appliance, which also includes a wine cellar and a pantry. The cooking appliance 3 includes an oven range 31 and a cooking heater 32. The storage appliance and the cooking appliance can be collectively referred to as cooking-related appliances.

[0024] The server 10 is installed outside the house 50. The server 10 may mainly analyze eating behavior, or may be used by a manufacturer or distributor of cooking-related home appliances to manage them and their users 5, and may execute the analysis as one of its functions. The terminal 4 is used by the user 5. A plurality of terminals 4 may be provided. In this case, the plurality of terminals 4 are used by the user 5, as well as by related parties such as family members and family doctors. Furthermore, when the user 5 who eats the meal is different from the cook, the cook can also be considered as a related party.

[0025] Next, the details of each of these devices will be described. In FIG. 2, the refrigerator 2 is equipped with a communication unit 201, a storage unit 202, a control unit 203, an opening / closing sensor 204, and an image sensor 205. The opening / closing sensor 204 detects the opening / closing of the door of the refrigerator 2. The image sensor 205 acquires images of the ingredients 51 to be cooked in the refrigerator 2 and the ingredients 51 taken in and out of the refrigerator 2 by the user 5. The control unit 203 processes the measurement information, which is a log or an image showing the opening and closing of the door, to estimate the eating behavior of the user 5, such as the type and amount of the ingredients 51 taken in and out of the refrigerator 2. The eating behavior and the dietary characteristics as the analysis results, and eating behavior advice based on these can be stored in the storage unit 202 or transmitted to the server 10 outside the refrigerator 2 via the communication unit 201. The ingredients 51 are an example of food, and processed foods and purchased cooked foods (e.g., side dishes) can be analyzed.

[0026] The cooking appliances 3, ie, the microwave oven 31 and the cooking heater 32, have the same configuration, and the configuration will be described below using the microwave oven 31. The microwave oven 31 includes a communication unit 311, a memory unit 312, a control unit 313, an operation panel 314, and a temperature sensor 315. The operation panel 314 receives instructions from the user 5 to the microwave oven 31. The temperature sensor 315 measures the temperature of the food 52 in the microwave oven 31. By processing cooking-related information realized by such temperatures and the like using the control unit 313 or a timer built in, characteristic information indicating characteristics of cooking and eating is specified from the temperature of the food 52 and cooking time (measurement information). Then, cooking characteristics, which are an example of eating behavior, are estimated from the characteristic information, and the estimated information can be stored in the memory unit 312 or transmitted to the server 10 outside the microwave oven 31 via the communication unit 311. In addition, in the first embodiment, the cooking appliance 3 is shown to be a microwave oven 31 or a cooking heater 32, but any cooking appliance that performs heating and cooking, such as a rice cooker or a hot plate, may be used. As described above, in this embodiment, the temperature sensor 315 and other sensors are used to measure measurement information related to cooking and eating.

[0027] Here, the control unit 22 in the cooking-related appliance such as the refrigerator 2 or the cooking appliance 3 will be described. This control unit 22 is a representative configuration of the control unit 203 or the control unit 313. FIG. 3 is an example of a block diagram of the control unit 22 of the cooking-related appliance of the first embodiment. The control unit 22 executes estimation processing and the like based on the measurement information and the feature information. For this purpose, as shown in FIG. 2, the control unit 22 has a control signal creation unit 221, an eating behavior estimation unit 222, an eating behavior analysis unit 223, and a suggested information creation unit 224.

[0028] Here, the control signal creation unit 221 creates a control signal for controlling the operation of the cooking-related home appliances. This creation is performed according to input to the operation panel 314, control commands from the terminal 4, and pre-stored operation logic. Furthermore, the eating behavior estimation unit 222 estimates the eating behavior of the user 5 using characteristic information identified from the measurement information. This eating behavior indicates the details of the meals of the user 5, and includes individual behaviors that include cooking behavior and eating behavior. Furthermore, it includes the lifestyle patterns of the user 5 related to cooking and eating.

[0029] Cooking behavior refers to the history (time series data) of cooking contents, and includes, for example, cooking procedures, cooking time, and cooking menus. Eating behavior refers to the history (time series data) of the meal contents of the user 5. For example, it includes foods containing ingredients consumed by the user 5, and the nutritional value, etc. can be identified. Furthermore, it is desirable that the eating behavior also includes the corresponding meal times. The lifestyle pattern includes the cooking time periods and meal time periods of the user 5.

[0030] Furthermore, the eating behavior analysis unit 223 analyzes the estimated eating behavior to calculate the eating characteristics of the user 5. The eating characteristics indicate characteristics of the diet of the user 5, and include, for example, health characteristics and cooking characteristics. The health characteristics indicate the health effects of a meal on the user 5, and include, for example, nutrients obtained from the meal. Furthermore, the cooking characteristics indicate the burden of costs in cooking. The burden of costs includes, for example, the energy usage rate, costs incurred, effort, and environmental load in cooking.

[0031] In addition, the proposed information creation unit 224 uses the dietary characteristics to identify eating behavior advice recommended for the user 5 and creates proposed information including the same. The eating behavior advice includes cooking behavior advice and eating behavior advice. In addition, it is preferable that the proposed information includes the eating behavior advice recommended to the user 5.

[0032] The eating behavior estimation unit 222, the eating behavior analysis unit 223, and the proposed information creation unit 224 can be omitted. For example, when all of these are omitted, the communication unit 201 to the communication unit 311 output measurement information measured by sensors such as the opening / closing sensor 204, the image sensor 205, and the temperature sensor 315 to the server 10 described later. When the eating behavior analysis unit 223 and the proposed information creation unit 224 are omitted, the communication unit 201 to the communication unit 311 output the eating behavior estimated by the eating behavior estimation unit 222 to the server 10. When the proposed information creation unit 224 is omitted, the communication unit 201 to the communication unit 311 output the eating characteristics specified by the eating behavior analysis unit 223 to the server 10. When all of these are provided, the communication unit 201 to the communication unit 311 output the eating behavior advice specified by the proposed information creation unit 224 or the proposed information created by the proposed information creation unit 224 to the server 10. This concludes the description of the cooking-related home appliances, and we will return to FIG. 2 to continue the description.

[0033] The server 10 functions as an eating behavior analysis device and executes the main part of the eating behavior analysis process of this embodiment. For this purpose, the server 10 includes a communication unit 101, a storage unit 102, and a control unit 103. The communication unit 101 receives information output, i.e., transmitted, from the refrigerator 2 and the cooking appliance 3, and communicates suggested information, etc. to the terminal 4.

[0034] The control unit 103 is a main component that executes the main part of the eating behavior analysis process. Therefore, the control unit 103 can process the acquired information, store the processed result in the storage unit 102, or transmit it to the terminal 4 via the communication unit 101. For example, the server 10 creates suggested information for the user 5 by analyzing information received from the cooking-related appliances such as the refrigerator 2 and the cooking appliance 3 via the communication unit 101. This suggested information includes eating behavior advice recommended to the user 5. In addition, in the first embodiment, the eating behavior analysis process estimates the user 5's lifestyle patterns related to cooking and meals, such as the time and number of meals and cooking, and eating behavior including the contents of meals. These will be described later. The communication unit 101 also transmits the suggested information to the terminal 4. The communication unit 101 may also notify the user of the lifestyle patterns and eating behavior described above. As a result, the terminal 4 displays these. As a result, it is possible to notify the user 5 of the suggested information, or to execute control in the refrigerator 2 and the cooking appliance 3 according to the suggested information. In addition, the control according to the recommendation information is assumed to be, for example, increasing the output when time-saving cooking is required. It is also assumed to control the temperature of the refrigerator 2 according to eating behavior such as individual behavior and life patterns.

[0035] Here, the server 10 will be described in detail with the control unit 103 at the center. Fig. 4 is an example of a block diagram showing the details of the server 10 of the first embodiment. In Fig. 4, the server 10 has a communication unit 101, a storage unit 102, and a control unit 103 as described above. The control unit 103 has an eating behavior estimation unit 104, an eating behavior analysis unit 105, and a proposed information creation unit 106. These have the same functions as the eating behavior estimation unit 222, the eating behavior analysis unit 223, and the proposed information creation unit 224, respectively.

[0036] The storage unit 102 also stores user management information 1021, measurement information 1022, eating behavior information 1023, food information 1024, and cooking-related home appliance information 1025. These pieces of information will be described later.

[0037] Here, in the first embodiment, the server 10 is installed outside the house 50, but it does not need to be an independent server 10, and may be installed virtually on a cloud service or the like, or may be installed inside the house 50. In this way, the server 10 can be realized by a so-called computer that executes information processing. For this reason, the hardware configuration of the server 10 can be realized by having a processing device, a communication device, and a storage device, and connecting them via a communication path. Then, the processing device that can be realized by a so-called processor such as a CPU realizes the functions of the eating behavior estimation unit 222, the eating behavior analysis unit 223, and the proposed information creation unit 224 by calculation according to the program stored in the storage device. Here, these functions may be realized as one function of an integrated management program that manages cooking-related home appliances and other vacuum cleaners, or may be realized as a program that exclusively executes eating behavior analysis. In addition, the storage device may be a so-called storage medium, or may be realized as a database system in a separate housing from the server 10.

[0038] Next, the terminal 4 will be described. Fig. 5 is an example of a block diagram of the terminal 4 in the first embodiment. The terminal 4 is used by the user 5 and has a function of managing and operating cooking-related home appliances. For this purpose, the terminal 4 has a communication unit 41, an eating behavior estimation unit 42, an eating behavior analysis unit 43, a proposal information creation unit 44, a control command unit 45, an input unit 46, an output unit 47, and a memory unit 48.

[0039] Here, the communication unit 41 has a communication function including a short-range wireless communication function, and communicates with cooking-related appliances and the server 10. At this time, the communication is performed via a router or a network. Also, the eating behavior estimation unit 42, the eating behavior analysis unit 43, and the proposed information creation unit 44 have the same functions as the eating behavior estimation unit 104, the eating behavior analysis unit 105, and the proposed information creation unit 106, respectively.

[0040] Furthermore, the control command unit 45 creates a control command for controlling the cooking-related appliances. This control command is transmitted to the cooking-related appliances via the communication unit 41, and the cooking-related appliances execute operations according to the control command. This control command includes a control command according to the proposal information including the proposal information itself and a control command according to an operation from the user 5.

[0041] The input unit 46 also accepts various operations from the user 5. The output unit 47 also outputs various information such as the created proposal information. The input unit 46 and the output unit 47 can be integrated into one unit such as a touch panel. The storage unit 48 also stores information used for processing in each of the above-mentioned units. In this embodiment, user management information 481, measurement information 482, eating behavior information 483, food information 484, and cooking-related home appliance information 485 are stored. These will be described later together with user management information 1021, measurement information 1022, eating behavior information 1023, food information 1024, and cooking-related home appliance information 1025. It is preferable that the terminal 4 is provided with a camera and a microphone.

[0042] The terminal 4 is realized by a computer such as a smartphone, a tablet terminal, or a PC. Therefore, the hardware configuration of the terminal 4 can be realized by having a processing device, a communication device, and a storage device, which are connected to each other via a communication path. The processing device, which can be realized by a so-called processor such as a CPU, realizes the functions of the eating behavior estimation unit 42, the eating behavior analysis unit 43, the proposal information creation unit 44, and the control command unit 45 by calculation according to a program stored in the storage device. Here, these functions may be realized as one function of an integrated management program (app) that manages cooking-related home appliances and other appliances such as vacuum cleaners, or may be realized as a program that exclusively executes eating behavior analysis. The storage device may also be a so-called storage medium.

[0043] As described above, the terminal 4 and the server 10 can execute various processes according to a program that can be stored in a storage medium. Furthermore, the eating behavior estimation unit, the eating behavior analysis unit, and the proposal information creation unit of the cooking-related home appliance, the terminal 4, and the server 10 may be included in any one (one type) of devices, and may be omitted in other devices. Furthermore, these functions may be executed by the cooking-related home appliance, the terminal 4, and the server 10 in cooperation with each other, or may be executed by any one or two devices. This concludes the description of the configuration of this embodiment, and next, information used in this embodiment will be described.

[0044] [information] Next, information used in the first embodiment will be described. FIG. 6 is an example of a diagram showing user management information 1021 in the first embodiment. The user management information 1021 is information about a user, and as shown in FIG. 6, has the following items for each user: notification destination, address, age, home appliance, related party, and related party notification destination. The user can use an ID such as the name or a membership number that identifies the user 5. The notification destination indicates the destination of information notification to the terminal 4 of the user, and can use an email address or its identification information.

[0045] The address and age are examples of attributes of the user. The home appliances are information for identifying home appliances used by the user, and may be limited to cooking-related home appliances, which are the subject of this embodiment. The related parties, relationships, and notification recipients indicate related parties of the user and their notification recipients.

[0046] FIG. 7 is an example of a diagram showing the measurement information 1022 of the first embodiment. The measurement information 1022 is information measured according to the operation of the cooking-related appliances, and includes log data (history). For this reason, the measurement information 1022 has items of cooking-related appliances and measurement information for each user. Here, the cooking-related appliances are used by the user and identify the cooking-related appliances that are the subject of this embodiment. The measurement information indicates the contents measured from the corresponding cooking-related appliance. For this reason, the measurement information is information about food, including cooking and eating. The measurement information also includes log data indicating the contents of operation for each acquired time, images (including videos), sounds, and the like. Here, the images include images of the interior of the refrigerator taken by the camera of the refrigerator 2 or the terminal 4, and images showing the cooking process taken by the terminal 4.

[0047] 8 is an example of a diagram showing the eating behavior information 1023 of the first embodiment. The eating behavior information 1023 is information related to eating behavior. Therefore, the eating behavior information 1023 has items of date and time, characteristic information, eating behavior, dietary characteristics, life pattern, and eating behavior advice for each user. The date and time indicates the unit date and time when the eating behavior performed by the corresponding user was performed.

[0048] The characteristic information is characteristic of the cooking and eating of the user, and indicates foods (including ingredients) and cooking contents. This characteristic information is identified from the measurement information 1022. The eating behavior indicates the cooking and eating behavior of the user, and includes an eating behavior history and a life pattern, but at least one of them can be omitted. The eating behavior history is information indicating the chronological behavior history of the user related to cooking and eating. The eating behavior history is estimated based on the characteristic information identified from the acquired measurement information. Therefore, the eating behavior history is created according to the measurement information acquired in chronological order.

[0049] The eating behavior history includes cooking behavior and eating behavior. The cooking behavior includes cooking procedures, cooking time, and cooking menus for the meals consumed by the user, and preferably includes cooking-related appliances used for the cooking. The eating behavior indicates ingredients (including foods) consumed by the user and meal times. In this way, the eating behavior history uses dates and times to make characteristic information more specific and chronological.

[0050] Furthermore, the lifestyle pattern indicates the lifestyle pattern related to cooking and eating for the user, and includes cooking time periods and meal time periods. This is estimated by patterning the characteristic information and / or eating behavior history. Here, meal times may vary depending on the season, week, day, etc. For example, wake-up times may be later in winter and on holidays. It is desirable to take these variations into consideration when identifying the lifestyle pattern.

[0051] The dietary characteristics indicate characteristics of the diet calculated from the corresponding eating behavior, and include cooking characteristics indicating the burden cost of cooking and health characteristics for the user including nutrients.

[0052] The eating behavior advice indicates advice on the eating behavior of the user that is characterized from the eating characteristics. This eating behavior advice includes cooking behavior advice including recommended cooking procedures and eating behavior advice indicating recommended foods, menus, and eating methods. The proposed information includes the eating behavior advice, but may also include at least one of the eating behavior history, the eating characteristics, and the lifestyle pattern.

[0053] FIG. 9 is an example of a diagram showing food information 1024 in the first embodiment. This information indicates the attributes of a food. For this reason, as shown in FIG. 9, each food has the items of nutrients, price, and environmental impact. Here, nutrients indicate the nutrients contained in the food. This nutrient indicates the amount of nutrients per unit amount (e.g., weight). Furthermore, price indicates the price of the food. This price indicates the price per unit amount (e.g., weight). Furthermore, environmental impact indicates the environmental impact of the food, and may use the amount of CO2 emissions during transportation and production, the amount of chemicals in packaging, etc. Furthermore, food ingredients are also included in the food.

[0054] FIG. 10 is an example of a diagram showing cooking-related home appliance information 1025 in the first embodiment. The cooking-related home appliance information 1025 shows the specifications of the cooking-related home appliance. More specifically, each model of the cooking-related home appliance has a field for its power consumption. The power consumption shows the power consumption used for each function such as a menu. Note that the power consumption is one type of specification, and other fields showing the electricity bill and the environmental load may be provided. However, these may be specified on an external site such as an electric company based on the power consumption.

[0055] Moreover, the user management information 481, the measurement information 482, the eating behavior information 483, the food information 484, and the cooking-related home appliance information 485 are the same as the user management information 1021, the measurement information 1022, the eating behavior information 1023, the food information 1024, and the cooking-related home appliance information 1025. However, the user management information 481 and the measurement information 482 are limited to the user 5 of the terminal 4 or the family including the user 5. Also, the cooking-related home appliance may store the same information. Furthermore, these pieces of information may be stored in at least one of the cooking-related home appliance, the terminal 4, and the server 10. Also, each device may be configured to store a part of each piece of information. This concludes the description of the information used in the first embodiment.

[0056] [Processing flow] Next, the processing flow of the eating behavior analysis processing in the first embodiment will be described in two parts: an overview (concept) and details according to the flowchart.

[0057] [Eating behavior analysis method (overview)] First, an eating behavior estimation process of the eating behavior analysis method will be described. Fig. 11 is an example of a diagram for explaining the eating behavior estimation process for estimating an eating behavior 11 according to the first embodiment. In the eating behavior estimation process, an eating behavior is estimated from measurement information related to food.

[0058] First, the image sensor 205 of the refrigerator 2 can measure the type, quantity, and portion (size) of ingredients stored in the refrigerator 2 as measurement information. Furthermore, by using the opening / closing sensor 204, it is possible to measure information indicating ingredients put into and taken out of the refrigerator 2 by taking images when the door is opened and closed. Furthermore, the opening / closing sensor 204 can measure information indicating how the refrigerator 2 is used, such as the timing at which the door is opened and closed and the number of times it is opened and closed. Furthermore, if the refrigerator 2 is equipped with a temperature sensor and a weight sensor, it is also possible to measure the temperature and weight of ingredients stored in the refrigerator 2. In this manner, measurement information is measured. Then, the type and amount of ingredients in the refrigerator 2 can be identified as characteristic information based on this measurement information.

[0059] Furthermore, the temperature sensor 315 of the microwave oven 31 can measure the heating output of the microwave oven 31, the temperature of the food being cooked, and the temperature inside the oven as measurement information. Furthermore, the operation panel 314 can acquire the automatic menu set by the user 5, the selected cooking menu such as microwave, oven, grill, etc., and the cooking time from a timer provided inside the control unit as measurement information. Therefore, by combining the measurement information such as the output, cooking temperature, selected menu, time, etc., the cooking menu such as rice or meat and the cooking method such as grilling, boiling, or warming can be estimated as feature information.

[0060] Also, in the cooking heater 32, like the oven range 31, the cooking menu and the cooking method, such as baking, boiling, or frying, can be estimated as feature information from measurement information obtained by the temperature sensor 325, the operation panel 324, and the timer.

[0061] Then, by combining feature information such as the type and amount of ingredients detected by the refrigerator 2 and the cooking menu and cooking method estimated by the oven range 31 and cooking heater 32, it is possible to infer eating behavior 11, such as what was cooked and when, and what was eaten and when. That is, eating behavior can be inferred using feature information identified from measurement information. Note that the measurement information in FIG. 11 corresponds to the measurement information in FIG. 7. Also, the feature information in FIG. 11 corresponds to the feature information in FIG. 8. And eating behavior 11 in FIG. 11 corresponds to the eating behavior in FIG. 8, particularly the eating behavior history.

[0062] Next, an example of the estimated 11 and the processing after the eating behavior estimation process will be described. FIG. 12 is an example of a diagram showing an example of the estimation of eating behavior in the first embodiment and the output processing after the eating behavior estimation process. An example of the eating behavior 11 shown graphically is shown in the upper part of FIG. 12. In FIG. 12, the eating behavior 11 is composed of a life pattern 110 and its details including meal contents 111. The details are composed of cooking behavior and eating behavior for each time period, which is the eating behavior history, and the eating behavior includes the meal contents 111.

[0063] Among this eating behavior 11, at around 6:00 (left speech bubble), when frozen rice is taken out of refrigerator 2 and microwave heating is continuously measured in oven range 31, it is inferred that the eating behavior is that the frozen rice is heated and eaten. Similarly, it is inferred that tofu and miso are taken out of refrigerator 2 and a pot is heated (cooked by boiling) on ​​cooking heater 32 for about 10 to 15 minutes as cooking behavior, and it is inferred that tofu miso soup is eaten. Similarly, it is inferred that a fillet of fish is taken out and grilled on cooking heater 32, and grilled fish is cooked.

[0064] Furthermore, among the eating behaviors 11, if it is estimated that milk was taken out of the refrigerator 2 before 24:00 (the speech bubble on the right) and the cooking appliance 3 was not used immediately thereafter, then the eating behavior of drinking cold milk is inferred. If it is estimated that the cooking behavior of heating the milk in the oven range 31 was performed immediately after taking it out, then the eating behavior of drinking warm milk is inferred. As described above, the meal contents 111 that cannot be inferred from the refrigerator 2 or the cooking appliance 3 alone can be inferred by combining the information obtained from the refrigerator 2 and the information obtained from the cooking appliance 3.

[0065] Furthermore, in the eating behavior 11, the time of day when cooking and eating are performed, and cooking time periods and meal time periods are inferred from the operation timing of the refrigerator 2 and cooking appliance 3, and thus a lifestyle pattern 110 is inferred. By storing a history of cooking behavior and eating behavior such as the lifestyle pattern 110 and meal contents 111 in chronological order for each predetermined period such as every day, an eating behavior history 112 of when, what time, and what kind of eating behavior is performed is accumulated.

[0066] Therefore, by analyzing the eating behavior history 112, the terminal 4 held by the user 5 can display and recommend suggested information such as the eating behavior 11 of the user 5 and the eating behavior advice 114 in accordance with the eating behavior history 112. For example, if it is estimated that the user 5 is cooking meals in a way that wastes energy, an energy-saving cooking method that can reduce energy usage is recommended. If the user 5 boils water on a daily basis using a cooking heater to boil vegetables as a waste of energy, a method of boiling vegetables using microwave heating in an oven range that does not boil water is proposed. This can reduce the energy usage for the same boiled cooking. Also, for example, if grilled fish is cooked using the grill heating of an oven range, it is recommended that the grill heating of the cooking heater can be used to heat the fish at a closer distance. As a result, the energy usage for the same grilled fish cooking can be reduced. Furthermore, by appropriately informing the user of the maintenance method, etc., it is also possible to propose reducing energy consumption by maintaining the heating efficiency that may be lost due to the adhesion of dirt inside the oven.

[0067] Also, if the eating behavior 11 of the user 5 is determined to be unhealthy, healthy recipes and cooking methods are recommended. For example, if the user cooks a lot of fried foods on a cooking heater, other cooking methods that are not fried are suggested, or recipes and cooking methods for dishes similar to fried foods that do not use much oil on an oven range or cooking heater are suggested. This makes it possible to suggest healthy meals that reduce the amount of oil consumed by the user. For example, if the user often eats late at night, recipes for dishes that are easy to digest and low in calories can be suggested so that the user does not feel heavy after meals and have little adverse effects on health. For example, for a person who consumes a small amount of vegetables, vegetable-centered recipes and simple cooking methods can be suggested. Also, by analyzing the nutrients contained in the meal, recipes for dishes that contain a lot of specific nutrients, such as protein, calcium, iron, etc., can be suggested to a user who is thought to be deficient in those nutrients.

[0068] As described above, in the eating behavior analysis of this embodiment, the eating behavior 11 is estimated, and more energy-efficient cooking methods and healthy recipes and cooking methods can be proposed. Note that energy saving and health are merely examples, and one of them may be omitted. Furthermore, energy saving is an example of the burden cost related to cooking, and other factors such as environmental load, incurred expenses, and labor can be considered. Furthermore, in order to take energy saving and health into account, it is more preferable to analyze the eating behavior and use health characteristics that indicate the characteristics of the health of meals for the user 5 and cooking characteristics that indicate the burden cost of cooking. Details of these will be explained later.

[0069] [Eating behavior analysis method (details)] Next, a description will be given of details of the eating behavior analysis method of Example 1. Fig. 13 is an example of a flowchart showing a process flow of the eating behavior analysis method of Example 1.

[0070] This flowchart starts on the condition that the cooking-related home appliances of the eating behavior analysis system 1 are turned on. First, in step S1, the open / close sensor 204 checks whether the door of the refrigerator 2 is open or closed and waits. As a result, if the open / close sensor 204 detects that the door of the refrigerator 2 is open (YES), the process proceeds to step S3. If the open / close sensor 204 has not detected that the door of the refrigerator 2 is open (NO), the process continues to step S1.

[0071] In step S2, the image sensor 205 acquires ingredient information indicating ingredients that are taken in and out of the refrigerator 2. This ingredient information is an example of measurement information. Note that, although ingredient information is used in this embodiment, food information on a wide range of foods can be handled.

[0072] In step S3, the communication unit 201 of the refrigerator 2 transmits the acquired ingredient information to the server 10. Here, the ingredient information may be image information of ingredients stored in the refrigerator 2 or processed text information of the ingredients. For this purpose, it is preferable that the control unit 22 has a conversion unit or the like that converts the image information into text information. The conversion unit or the like may also acquire additional information such as the amount of the ingredient and the temperature of the ingredient in addition to the information on the type of ingredient.

[0073] Next, in step S4, the communication unit 101 of the server 10 receives the ingredient information. The eating behavior estimation unit 104 identifies characteristic information from the received ingredient information. The eating behavior estimation unit 104 saves the received ingredient information as measurement information 1022. The eating behavior estimation unit 104 saves the identified characteristic information in eating behavior information 1023. For example, when image information of ingredients is sent from the refrigerator 2, it is possible to process it and save it as text information. It is also possible to save ingredient information indicating ingredients stored inside the refrigerator 2 by saving it as characteristic information about ingredients removed from the refrigerator 2 or ingredients placed in the refrigerator 2.

[0074] Furthermore, the process from step S5 onwards branches depending on whether the removed ingredients are cooked by the cooking appliance 3. If the ingredients have been cooked (YES), the process from step S6 onwards is executed. If the ingredients have not been cooked (NO), the process from step S10 onwards is executed. This branch is executed, for example, depending on whether the ingredients have been cooked by the cooking appliance 3 immediately after they were removed from the refrigerator 2 (within a specified time).

[0075] In addition, in step S6, the operation panel 314 and the temperature sensor 315 of the cooking appliance 3 acquire cooking information for heating by the cooking appliance 3. This cooking information is a type of measurement information. Then, in step S7, the communication unit 311 transmits the acquired cooking information to the server 10. As a result, the communication unit 101 of the server 10 receives the cooking information.

[0076] In step S8, the eating behavior estimation unit 104 of the server 10 estimates the menu and cooking method, which are examples of characteristic information, based on the received cooking information and ingredient information. For example, the following process is performed to achieve this. First, the eating behavior estimation unit 104 estimates the cooking method as a cooking behavior based on a combination of cooking mode and heating time. Cooking food in a high-power microwave oven Cooking by "baking" food using high-temperature ovens or grills For example, cooking by "simmering" food using a low-temperature oven or low-power microwave.

[0077] The eating behavior inference unit 104 then infers an eating behavior that can also be expressed as a cooking menu (cooked foods) or eating and drinking information, based on the food ingredient information and cooking method received in step S4.

[0078] Then, in step S9, the eating behavior inference unit 104 registers these cooking behaviors and eating behaviors in the eating behavior information 1023.

[0079] Furthermore, in step S10 when no cooking information is received for a predetermined time like step S6, the eating behavior estimation unit 104 identifies non-heat cooking as the cooking method. This is because the lack of reception of cooking information indicates that the ingredients taken out of the refrigerator 2 are not heated. The eating behavior estimation unit 104 then estimates an eating behavior that combines the ingredient information received in step S4 and non-heat cooking, and registers this in the eating behavior information 1023. At this time, the eating behavior estimation unit 104 also identifies non-heat cooking as a cooking behavior, and registers this in the eating behavior information 1023.

[0080] As a result of the registration in steps S9 and S10, the eating behavior history is recorded in the eating behavior information 1023. This eating behavior history is associated with the date and time, so that it is possible to know when, what, and how the user 5 cooked and ate. In the first embodiment, the characteristic information is specified in steps S4 and S9, and the eating behavior is estimated using this. However, the eating behavior may be estimated directly from measurement information such as the meal information and cooking information.

[0081] In step S11, the eating behavior analysis unit 105 analyzes the registered eating behavior history. Specifically, from the eating behavior history, dietary characteristics indicating characteristics of the diet of the user 5 are calculated. The dietary characteristics include health characteristics and cooking characteristics. The health characteristics indicate the health effects of the diet on the user 5, and can use ingested nutrients. The cooking characteristics indicate the burden cost of cooking, and include at least one of the energy usage rate, incurred expenses, effort, and environmental impact due to cooking.

[0082] More preferably, in step S11, the eating behavior analysis unit 105 also identifies a lifestyle pattern using the eating behavior history and date and time. This lifestyle pattern can be identified by identifying an average eating behavior using an eating behavior history for a predetermined period, such as one week or one month. The above analysis results are registered in the eating behavior information 1023.

[0083] Then, the proposal information creating unit 106 creates proposal information based on the meal characteristics and provides it to related parties including the user 5. The proposal information may be created using a life pattern. An example will be described below. In step S12, the proposal information creating unit 106 analyzes the cooking characteristics included in the meal characteristics and judges whether there is waste (excess) in the burden cost. In this case, it is judged whether there is waste by comparing with a predetermined threshold. In the example of FIG. 13, the proposal information creating unit 106 uses the energy usage rate as the burden cost and judges whether energy is used wastefully in cooking. As a result, if there is waste (YES), the process proceeds to step S13. If there is no waste (NO), the process proceeds to step S14. In this judgment, the increase rate in a time series or the energy usage itself may be used. In addition, when the incurred cost or the environmental load is used as the burden cost, the price or the environmental load of the food information 1024 is used. Furthermore, in this case, it is desirable to further use the power consumption (electricity usage) in cooking of the cooking-related home appliances and the cooking-related home appliance information 1025.

[0084] In step S13, the proposed information creating unit 106 identifies an energy-saving cooking method as the eating behavior advice. Next, the proposed information creating unit 106 creates proposed information including the energy-saving cooking method. Then, the communication unit 101 outputs, that is, transmits, the proposed information to the terminal 4. As a result, the proposed information is displayed on the output unit of the terminal 4. Here, the terminal 4 to which the proposed information is sent may be the terminal 4 used by the user 5, may be a terminal used by related parties, or may be both of them. As a result, related parties including the user 5 can confirm the proposed information. Here, the energy-saving cooking method of the first embodiment includes the one shown in FIG. 12 described above. Furthermore, in step S13, the proposed information creating unit 106 may identify the terminal 4 that is the communication partner of the terminal 4's instruction to cooking-related home appliances or the short-range wireless communication as the output destination of the proposed information. As a result, the proposed information is output to the user 5. This is the same in step S15. At this time, the user 5 can also be identified by identifying the terminal 4.

[0085] In step S14, the proposal information creation unit 106 analyzes the health characteristics included in the dietary characteristics to determine whether the diet is unhealthy. This determination can be made by comparing the nutrients included in the health characteristics with predetermined reference information and determining whether the difference is a predetermined value or more. For example, it may be determined that the intake of lipids is excessive. If the result is that the diet is unhealthy (YES), the process proceeds to step S15. If the diet is healthy (NO), the process flow ends. Note that the increase rate of nutrients over time may be used in this determination.

[0086] In step S15, the proposed information creating unit 106 identifies a healthy recipe (including a cooking method) useful for maintaining health as the dietary behavior advice. The healthy recipe includes ingredients and cooking methods suitable for maintaining health. Next, the proposed information creating unit 106 creates proposed information including the healthy recipe. Then, the proposed information creating unit 106 outputs, that is, transmits, the proposed information to the terminal 4 using the communication unit 101. As a result, the proposed information is displayed on the output unit of the terminal 4. Here, the terminal 4 to which the proposed information is sent may be the terminal 4 used by the user 5, may be a terminal used by related parties, or may be both of them. As a result, related parties including the user 5 can confirm the proposed information. Here, the healthy recipes of the first embodiment include those shown in FIG. 12 above. Note that, in the first embodiment, the energy-saving cooking methods and the healthy recipes are output and displayed separately, but they may be output and displayed together. That is, steps S12 and S14 may be performed, and depending on the results, proposed information including the energy-saving cooking methods and / or the healthy recipes may be created and output. Furthermore, if both steps S12 and S14 are NO, suggested information to maintain the current eating habits may be created and output. Furthermore, the suggested information may include any item of the eating behavior information 1023 such as a lifestyle pattern or an eating behavior.

[0087] In this embodiment, the server 10 functioning as an eating behavior analysis device performs the estimation of eating behavior, the analysis, and the creation of the proposed information, but at least one of these may be performed by a cooking-related home appliance or the terminal 4. For example, the estimation of eating behavior is performed by a cooking-related home appliance and transmitted to the server 10, and the server 10 executes subsequent processing. At least one of the estimation of eating behavior, the analysis, and the proposed information may be performed by the terminal 4.

[0088] [effect] According to this embodiment, the refrigerator 2 and the cooking appliance 3 automatically detect the eating behavior 11 of the user 5 through the action of taking out ingredients 51 from the refrigerator 2 and cooking them with the cooking appliance 3 in daily life. This eliminates the need for the user 5 to take active actions such as registering the contents of meals and taking photos. In addition, it is possible to omit equipment such as installing a camera in the house 50, which may invade the user's privacy and may be aversive for the user 5, and by using the refrigerator 2 and the cooking appliance 3 in the house 50, the normal eating behavior 11 can be naturally estimated from daily life.

[0089] This makes it possible to prevent the user 5 from forgetting to register or making a mistake in registering, and to automatically and appropriately acquire eating behavior 11 in the house 50. Furthermore, eating behavior 11 is estimated using information on ingredients taken out of the refrigerator 2 and cooking information from the cooking appliance 3. This makes it possible to detect types of ingredients that are difficult to distinguish from photos of meals or cooked foods, and ingredients that are difficult to detect, and also estimate cooking methods that are difficult to distinguish from photos. This makes it possible to suggest cooking methods that are less costly, for example, energy-saving and healthy. EXAMPLES

[0090] Next, Example 2 will be described. In Example 1, the image sensor 205 is mounted on the refrigerator 2, and the cooking appliance 3 is not mounted with an image sensor. In contrast, in Example 2, the image sensor 316 and the weight sensor 318 are mounted on the oven range 31 of the cooking appliance 3. Also, an internal image sensor 326 is mounted inside the cooking heater 32, and an external image sensor 327 is installed on the top of the cooking heater 32. Therefore, although the internal configuration of the cooking appliance 3 is different, the configuration of the refrigerator 2, the information and flow chart used, and the effects obtained are the same, so that the description of the same contents as in Example 1 will be omitted, and only the differences from Example 1 will be described.

[0091] [Overall configuration] Fig. 14 is an example of a schematic diagram of an eating behavior analysis system 1 of Example 2. In Fig. 14, the eating behavior analysis system 1 of this example is configured with a refrigerator 2 installed in a house 50, and cooking appliances 3, a microwave oven 31 and a cooking heater 32, similar to Example 1. Since the refrigerator 2 is the same as Example 1, the configurations of the microwave oven 31 and the cooking heater 32, which are different from Example 1, will be described.

[0092] In addition to an operation panel 314 and a temperature sensor 315, the oven range 31 is equipped with an image sensor 316 at the top of the oven interior and a weight sensor 318 at the bottom of the oven interior. Therefore, for food 52 in the oven range 31, the temperature sensor 315 can detect the temperature of food 52, the image sensor 316 can detect an external image of food 52, and the weight sensor 318 can detect the weight of the food. From this, it is possible to estimate the type, temperature, and weight of the ingredients and food in food 52. The temperature and weight can be treated as measurement information, and the type of ingredients and food can be treated as characteristic information.

[0093] In addition to the operation panel 324, the cooking heater 32 is provided with an internal image sensor 326 inside the grill, which can obtain an external image of the ingredients inside the grill of the cooking heater 32. The cooking heater 32 is also provided with an external image sensor 327 capable of communicating with the cooking heater 32 as an external sensor of the cooking heater 32. By installing the external image sensor 327 above the cooking heater 32, an external image of the food 52 placed on the cooking heater 32 can be obtained. Therefore, the ingredients in the food 52, the type of food, the cooking contents, etc. can be inferred from the external image of the food 52. This image can be treated as measurement information. The ingredients, the type of food, the cooking contents, etc. can be treated as characteristic information.

[0094] Here, by using an image sensor with a limited field of view that captures images only above the cooking heater 32, it is possible to reduce the possibility of violating the privacy of the user 5 who is cooking by not capturing images, etc. In addition, by incorporating the external image sensor 327 into a range hood or a bulb of a ceiling light above the cooking heater 32, the effort required for installing the external image sensor 327 can be reduced.

[0095] An example of a block diagram of the above-mentioned eating behavior analysis system 1 is shown in Fig. 15. Compared to Fig. 2, an image sensor 316 and a weight sensor 318 are added to the oven range 31. Also, a cooking heater 32 is added. This cooking heater 32 is equipped with a communication unit 321, a storage unit 322, a control unit 323, an operation panel 324, a temperature sensor 325, a built-in image sensor 326, and an external image sensor 327. The functions of these are the same as those of the oven range 31 or are as described above.

[0096] [Processing flow] Next, an eating behavior analysis method according to the second embodiment, that is, a method of estimating eating behavior 11 using a refrigerator 2, a microwave oven 31, and a cooking heater 32, will be described. Fig. 16 is an example of a diagram for explaining an eating behavior estimation process for estimating eating behavior 11 according to the second embodiment. Since the type and amount of ingredients can be obtained from the refrigerator 2 in the same way as in the first embodiment, a description thereof will be omitted.

[0097] 16, the image sensor 316 of the oven range 31 can obtain external images of the ingredients 51 and food 52 stored inside the oven range 31, and the types of the ingredients 51 and food 52. The weight sensor 318 can obtain the weights of the ingredients 51 and food 52. Furthermore, as in the first embodiment, the temperature sensor 315 obtains the heating output and the temperature of the food, the operation panel 314 obtains the menu selected by the user 5, and the cooking time obtained by the timer can be combined to estimate the cooking menu and cooking method. In other words, characteristic information related to cooking is identified from measurement information measured by various sensors.

[0098] Also, in the cooking heater 32, the built-in image sensor 326 and the external image sensor 327 can be used to obtain external images of the ingredients 51 and the food 52 and the types of the ingredients 51 and the food 52. As in the first embodiment, the temperature sensor 325 obtains the heating output and the temperature of the food, and the operation panel 324 obtains the menu selected by the user 5, and the cooking time obtained by the timer can be combined to estimate the cooking menu and cooking method.

[0099] By combining information from the image sensor 316 of the oven range 31 and the built-in image sensor 326 and external image sensor 327 of the cooking heater 32 with information from the type and amount of ingredients obtained by the refrigerator 2, the type and amount of ingredients and cooking dishes can be identified with greater accuracy.

[0100] The process flow of the second embodiment is shown in the flowchart of Fig. 13, similarly to the first embodiment. That is, the refrigerator 2 acquires measurement information of ingredients 51 put into and taken out of the refrigerator 2 and transmits it to the server 10, and the cooking information of food 52 acquired by the cooking appliance 3 is transmitted to the server 10. Then, the server 10 uses the measurement information to estimate eating behavior 11, analyzes the information, and outputs proposal information showing appropriate recommendations to the terminal 4.

[0101] Also in this embodiment, the server 10 functioning as an eating behavior analysis device performs the estimation of eating behavior, the analysis, and the creation of the proposed information, but at least one of these may be performed by a cooking-related home appliance or the terminal 4. For example, the estimation of eating behavior is performed by a cooking-related home appliance and transmitted to the server 10, and the server 10 executes subsequent processing. Also, at least one of the estimation of eating behavior, the analysis, and the proposed information may be performed by the terminal 4.

[0102] [effect] An image sensor is mounted on the cooking appliance 3 such as the oven range 31 and the cooking heater 32 of the eating behavior analysis system 1 of the second embodiment. Therefore, the cooking appliance 3 alone can acquire the types of ingredients 51 and food 52. Therefore, the eating behavior 11 can be estimated not only for ingredients taken out of the refrigerator 2, but also for ingredients that have just been purchased and not stored in the refrigerator, or ingredients that have been stored at room temperature, when cooking. For example, the type of ingredients can be detected even when cooking using rice, root vegetables, canned foods, etc. that have been stored at room temperature. Therefore, it can be estimated that cooked rice is included in a meal by detecting that rice has been cooked in a rice cooker or that cooked cooked rice stored at room temperature is heated in the oven range 31.

[0103] In addition, by acquiring information on the food 52 as an image, the accuracy of detecting the cooking method can be improved compared to when the cooking appliance 3 is not equipped with an image sensor, compared to when the cooking method is estimated based only on the heating output and heating time. For example, meat and potatoes and curry have similar ingredients such as meat, potatoes, and carrots, and the cooking operation of boiling is similar, so they cannot be accurately distinguished from the ingredients taken out of the refrigerator 2 and the cooking method acquired from the cooking appliance 3. However, by installing an image sensor in the cooking appliance 3, it is possible to determine whether the food is meat and potatoes or curry from the color and shape of the food 52. Therefore, by acquiring images from both the refrigerator 2 and the cooking appliance 3, it is possible to more accurately estimate the eating behavior indicating the content of the meal.

[0104] However, the image sensor mounted on the cooking appliance 3 can only measure the food 52 that is to be cooked by the cooking appliance 3. Therefore, it is not possible to identify food ingredients 51 that are not to be cooked by heating. For example, the cooking appliance 3 cannot detect cold drinks that are taken out of the refrigerator 2 and drunk as is, or food ingredients such as yogurt, natto, and ice cream that are eaten without being cooked by heating. Therefore, in the second embodiment, the meal contents 111 can be detected most accurately by detecting the types of food ingredients 51 and food ingredients 52 by both the refrigerator 2 and the cooking appliance 3.

[0105] Moreover, since the oven range 31 is equipped with a weight sensor 318, it is possible to measure the weight of the food 52. Therefore, it is possible to determine the weight of the food 52 more accurately than estimating the amounts of the food ingredients 51 and the food ingredients 52 using the image sensor 205 of the refrigerator 2 or the image sensor 316 of the oven range 31. Furthermore, by identifying the presence or absence of a container and the type of container as characteristic information using the image sensor 316, it is possible to determine the weight of the food ingredients 51 excluding the weight of the container more accurately. Therefore, by using the oven range 31 equipped with the image sensor 316 and the weight sensor 318, it is possible to more accurately estimate the amount of food included in the eating behavior of the eating behavior 11.

[0106] In addition, since the image sensor mounted on the cooking appliance 3 measures the food 52, it cannot determine whether the ingredients 51 contained in the food 52 have been chopped or processed. Therefore, by combining information on the ingredients 51 taken out of the refrigerator 2 with information on the food 52 being cooked by the cooking appliance 3, it is possible to accurately measure the ingredient information (measurement information) of the ingredients 51 contained therein.

[0107] In addition, in the second embodiment, an example was shown in which the external image sensor 327 was installed on the cooking heater 32. However, it is also possible to obtain information on ingredients and cooked foods by combining an external image sensor with various cooking appliances such as a rice cooker, a bread maker, a food processor, a blender, a hot plate, etc. Also, the camera used in the terminal 4 can be used as the image sensor.

[0108] As described above, by estimating eating behavior 11 more accurately using information on ingredients taken out of refrigerator 2 and cooking information using cooking appliance 3, it is possible to suggest a lower-stress or healthier eating lifestyle. EXAMPLES

[0109] In the eating behavior analysis systems 1 of the above-mentioned first and second embodiments, the eating behavior analysis systems 1 are configured with a refrigerator 2, a cooking appliance 3, a terminal 4, and a server 10. In contrast, the eating behavior analysis system 1 of the third embodiment differs in that a human presence sensor 501 is added to the above. Since the rest of the configurations and flow charts of the refrigerator 2 and the cooking appliance 3, and the effects obtained are the same, descriptions of the same contents as those of the first and second embodiments will be omitted, and only the part of the human presence sensor 501 that is different only in the third embodiment will be described.

[0110] [Overall configuration] Fig. 17 is an example of a schematic diagram of an eating behavior analysis system 1 of Example 3. As shown in Fig. 17, the eating behavior analysis system 1 of this example is composed of a refrigerator 2 installed in a house 50, a microwave oven 31 which is a cooking appliance 3, a human presence sensor 501K, a human presence sensor 501D, a terminal 4, and a server 10. Here, the human presence sensor 501K is installed on the ceiling of a kitchen 50K and the human presence sensor 501D is installed on the ceiling of a dining room 50D which are present in the house 50. Here, the human presence sensors 501K and 501D do not have to be installed on the ceiling, but may be installed on other surfaces such as a wall or floor, or on furniture such as a kitchen counter or dining table, or may be incorporated into other home appliances.

[0111] Also, since the human presence sensor 501K and the human presence sensor 501D are assumed to be sensors that detect the presence or absence of a person using radio waves such as infrared rays, individual sensors are provided for each. However, a sensor that detects the position of a person in the house 50 using radio waves such as millimeter waves or microwaves may be used to simultaneously detect the presence of the user 5 in both the kitchen and the dining room using a single sensor.

[0112] Also, when the user 5 is in the kitchen 50K, the motion sensor 501K detects that the user 5 is cooking, and the motion sensor 501D detects that the user 5 is in the dining room 50D, that is, that the user 5 is eating. These detection results can be used as measurement information. Therefore, it is possible to more accurately estimate whether the user 5 is cooking or eating in the lifestyle pattern 110 of the eating behavior 11.

[0113] [Processing flow] Next, the eating behavior estimation process of the third embodiment will be described with reference to Fig. 18. Fig. 18 is an example of a diagram for explaining an example of estimating eating behavior and the eating behavior estimation process of the third embodiment. The eating behavior analysis system 1 of this embodiment is provided with a human presence sensor 501K and a human presence sensor 501D. Therefore, by combining the measurement information acquired from the refrigerator 2 and the cooking appliance 3 with the measurement information acquired from the human presence sensor 501K and the human presence sensor 501D, it is possible to more accurately estimate eating behavior such as cooking behavior, such as cooking time and meal time, and eating behavior.

[0114] By taking out ingredients from the refrigerator 2 and cooking them with the cooking appliance 3, the eating behavior estimation unit 104 can estimate the cooking time period to some extent from the operating time periods of the refrigerator 2 and the cooking appliance 3. However, by adding sensing information from the human presence sensor 501K to this, it is possible to estimate the cooking time for a time period when neither the refrigerator 2 nor the cooking appliance 3 is being used but cooking is being performed.

[0115] For example, the work of peeling or cutting fruit stored at room temperature cannot be detected by the refrigerator 2 and the cooking appliance 3. However, by detecting a person working in the kitchen as measurement information using the human presence sensor 501K, the eating behavior estimation unit 104 can estimate this as a cooking time period. However, since there is a possibility that the user 5 is not cooking even if he or she is in the kitchen, the eating behavior estimation unit 104 can more accurately estimate the cooking time period by combining information from the refrigerator 2 and the cooking appliance 3.

[0116] Furthermore, when it is detected that a meal has been cooked using the refrigerator 2 and the cooking appliance 3, the eating behavior estimation unit 104 can estimate the meal time period assuming that the user 5 has eaten after that. However, by adding the sensing information of the human presence sensor 501D to the above, the eating behavior estimation unit 104 can estimate the meal time period more accurately. For example, when it is detected that the user 5 has moved to the dining room after cooking using the cooking appliance 3, the eating behavior estimation unit 104 can estimate that the user 5 has started eating. Furthermore, by detecting that the user 5 has moved from the dining room to the kitchen and thus detecting that the meal has ended, the eating behavior estimation unit 104 can estimate the meal time period from the start of the meal to the end of the meal. However, since the presence of the user 5 in the dining room does not necessarily mean that the user 5 is eating, the eating behavior estimation unit 104 can improve the accuracy of the estimation of the eating behavior by combining the information of the refrigerator 2 and the cooking appliance 3.

[0117] Also in this embodiment, the server 10 functioning as an eating behavior analysis device performs the estimation of eating behavior, the analysis, and the creation of the proposed information, but at least one of these may be performed by a cooking-related home appliance or the terminal 4. For example, the estimation of eating behavior is performed by a cooking-related home appliance and transmitted to the server 10, which then executes subsequent processing. Also, at least one of the estimation of eating behavior, the analysis, and the proposed information may be performed by the terminal 4.

[0118] [effect] The effect obtained in the third embodiment is the same as the effect obtained in the first and second embodiments, and by adding the human presence sensor 501D and the human presence sensor 501K to the refrigerator 2 and the cooking appliance 3, the eating behavior 11 of the person in the house 50 can be detected with higher accuracy. Therefore, it is possible to suggest healthier meals.

[0119] For example, regarding meal times, in addition to information on what you eat, when you eat, the contents of the meal, and the time of day you eat, it will be possible to obtain the time it takes to eat. Therefore, depending on whether the meal takes a long or short time compared to the contents and amount of food, advice such as "It would be better for your health if you eat more slowly" can be suggested. It is also possible to make suggestions about healthy eating, such as suggesting recipes for meals that can be eaten at meal times and are easy to digest.

[0120] In addition, since cooking time and cooking method can be detected with higher accuracy, advice can be given to reduce cooking effort and energy consumption by eliminating unnecessary movements in the kitchen. For example, the following suggestions can be made for cooking that can be done with both the oven range 31 and the cooking heater 32 among the cooking appliances 3. For example, it is possible to estimate eating behavior 11 including cooking behavior and eating behavior, such as suggesting which cooking appliance to use to reduce energy consumption, or suggesting reducing the effort of carrying food on a large plate for someone who frequently goes back and forth between the kitchen and dining room. This makes it possible to make eating behavior recommendations that lead to energy consumption reduction and health support in daily life.

[0121] This concludes the description of the embodiments of the present invention, but the present invention is not limited to the above-mentioned embodiments. For example, the burden cost of a specific cooking can be at least one of the energy usage rate, the cost incurred, the effort, and the environmental impact of cooking. When two or more burden costs are used, a combined cost of these is used. In this case, it is preferable that the judgment in step S12 is based on the result of a predetermined calculation such as the addition of each burden cost. Furthermore, although each embodiment uses ingredient information, food information of food containing ingredients may also be used. [Explanation of symbols]

[0122] 1: Eating behavior analysis system, 10: Server, 101: Communication unit, 102: Memory unit, 103: Control unit, 104: Eating behavior estimation unit, 105: Eating behavior analysis unit, 106: Proposal information creation unit, 11: Eating behavior, 110: Lifestyle pattern, 111: Meal content, 112: Eating behavior history, 114: Eating behavior advice, 2: Refrigerator, 201: Communication unit, 202: Memory unit, 203: Control unit, 204: Opening / closing sensor, 205: Image sensor, 3: Cooking appliance, 31: Oven range, 311: Communication unit, 3 12: memory unit, 313: control unit, 314: operation panel, 315: temperature sensor, 316: image sensor, 318: weight sensor, 32: cooking heater, 321: communication unit, 322: memory unit, 323: control unit, 324: operation panel, 325: temperature sensor, 326: built-in image sensor, 327: external image sensor, 4: terminal, 5: user, 50: residence, 50K: kitchen, 50D: dining room, 51: food, 52: cooking food, 501K: human presence sensor, 501D: human presence sensor

Claims

1. A sensor that measures measurement information related to cooking and eating for each of a plurality of cooking-related home appliances; an eating behavior estimation unit that identifies characteristic information related to the cooking and the meal from the measured measurement information, and estimates an eating behavior including a cooking behavior indicating the content of the cooking and an eating behavior indicating the content of the meal of the user using the characteristic information; an eating behavior analysis unit that analyzes the eating behavior and calculates cooking characteristics indicating a burden cost of the cooking and eating characteristics including health characteristics indicating an impact of the meal on the user's health; a proposal information creation unit that creates proposal information including recommended eating behavior advice based on the calculated dietary characteristics; An eating behavior analysis system having an output unit that outputs the created proposal information to at least the user.

2. The eating behavior analysis system according to claim 1 , In the eating behavior analysis system, the burdened cost is at least one of the energy usage rate, the cost incurred, the effort, and the environmental burden due to cooking.

3. The eating behavior analysis system according to claim 2, The eating behavior advice includes cooking behavior advice indicating recommended cooking behavior and eating behavior advice indicating recommended eating behavior.

4. The eating behavior analysis system according to claim 3, The burdened cost includes the energy usage rate, The suggested information creation unit creates suggested information including an energy-saving cooking method for reducing energy consumption in cooking as the cooking behavior advice.

5. The eating behavior analysis system according to claim 3, The proposed information creation unit creates proposed information including healthy recipes that include ingredients and cooking methods suitable for maintaining the user's health as the cooking behavior advice.

6. The eating behavior analysis system according to claim 1 , The cooking-related home appliances include food storage home appliances and food cooking home appliances, The storage appliance is an eating behavior analysis system that is equipped with either an imaging unit that captures images of its own interior, or an imaging unit that can communicate with the cooking appliance and capture images of the objects to be cooked, or a weight sensor that can measure the weight of the food inside the appliance.

7. a computer that receives measurement information related to cooking and eating for each of a plurality of cooking-related home appliances measured by a plurality of sensors; an eating behavior estimation unit that identifies characteristic information related to the cooking and the meal from the measured measurement information, and estimates an eating behavior including a cooking behavior indicating the content of the cooking and an eating behavior indicating the content of the meal of the user using the characteristic information; an eating behavior analysis unit that analyzes the eating behavior and calculates cooking characteristics indicating a burden cost of the cooking and eating characteristics including health characteristics indicating an impact of the meal on the user's health; a proposal information creation unit that creates proposal information including recommended eating behavior advice based on the calculated dietary characteristics; A program that functions as an output unit that outputs the created proposal information to at least the user.

8. The program according to claim 7, The cost burden is at least one of the energy usage rate, the cost incurred, the effort, and the environmental impact due to cooking.

9. The program according to claim 8, The program, in which the eating behavior advice includes cooking behavior advice indicating recommended cooking behavior and eating behavior advice indicating recommended eating behavior.

10. The program according to claim 9, The burdened cost includes the energy usage rate, The proposed information creation unit is a program that creates proposed information including an energy-saving cooking method for reducing energy consumption in cooking as the cooking behavior advice.

11. The program according to claim 9, The proposed information creation unit is a program that creates proposed information including healthy recipes that include ingredients and cooking methods suitable for maintaining the user's health as the cooking behavior advice.

12. The program according to claim 7, The cooking-related home appliances include food storage home appliances and food cooking home appliances, The storage appliance is a program that includes either an imaging unit that captures images of its own interior, or an imaging unit that can communicate with the cooking appliance and capture images of the food being cooked, or a weight sensor that can measure the weight of food inside the appliance.

Images