Presentation device, learning device, presentation method, and presentation program

Abstract

A presentation device according to an aspect of the present disclosure includes: a tolerance acquisition unit that acquires a deviation tolerance estimated for each cooking step based on reference data indicating a cooking process, and the deviation tolerance indicates a degree to which an actual measurement value of the parameter measured during cooking is permitted with respect to a value of the parameter indicating the cooking process; and a presentation unit that presents the deviation tolerance for each cooking step.

Description

Technical Field

[0001] This invention relates to a presentation device, a learning device, a presentation method, and a presentation program. Background Technology

[0002] Cooking assistance systems based on sensor information are known. In conventional cooking assistance systems, heating temperature and heating time are automatically controlled based on sensor information obtained from the heating appliance.

[0003] Citation List

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Publication No. 2016-051526

[0006] Patent Document 2: Japanese Patent Publication No. 2010-192274 Summary of the Invention

[0007] Technical issues

[0008] Traditional cooking aids aim to automate the cooking process. Therefore, they do not provide assistance to prompt the cook to perform appropriate cooking actions based on the cooking situation. On the other hand, recently, with the popularity of internet communities such as COOKPAD (registered trademark), an environment has been created where cooks can enjoy their own cooking. Cooks can recreate dishes prepared by another person by using recipe information posted on social networking sites.

[0009] Recipe information includes details such as heating power (heating temperature) and heating time. However, even when these conditions are accurately reproduced, it is difficult to replicate the final dish. This is believed to be due to inconsistencies in the quantity, characteristics, cutting, and mixing of food ingredients. Providing specialized cooking utensils, food-cutting kits, and automatic heating control based on matching conditions improves accuracy. However, this approach resembles that of a food factory, and the enjoyment of cooking is diminished.

[0010] Therefore, this disclosure proposes a presentation device, learning device, presentation method, and presentation procedure that can improve the reproduction of dish completion.

[0011] Solution to the problem

[0012] To address the aforementioned problems, a presentation apparatus according to embodiments of the present disclosure includes: a tolerance acquisition unit for acquiring a deviation tolerance, the deviation tolerance being estimated for each cooking step based on reference data indicating the cooking process, and the deviation tolerance indicating the degree to which the actual measured value of a parameter measured during cooking is permissible relative to the value of the parameter indicating the cooking process; and a presentation unit for presenting the deviation tolerance for each cooking step.

[0013] To address the aforementioned problems, a learning apparatus according to an embodiment of this disclosure includes: a learning unit that learns a hidden Markov model to output the variance of the values ​​of a cooking process parameter as a deviation tolerance used by a presentation device; the presentation device includes: a tolerance acquisition unit that acquires the deviation tolerance, the deviation tolerance being estimated for each cooking step based on reference data indicative of the cooking process, and the deviation tolerance indicating the degree to which the actual measured value of a parameter measured during cooking is permissible relative to the value of the parameter indicative of the cooking process; and a presentation unit that presents the deviation tolerance for each cooking step. Attached Figure Description

[0014] [ Figure 1 ] Figure 1 This is a diagram showing an overview of the cooking assistance system.

[0015] [ Figure 2 ] Figure 2 This is a functional block diagram of a cooking assistance system.

[0016] [ Figure 3 ] Figure 3 This is a functional block diagram of the cooking data modeling unit.

[0017] [ Figure 4 ] Figure 4 This is a diagram illustrating the processing flow in recording mode.

[0018] [ Figure 5 ] Figure 5 This is a diagram illustrating an example of a visualization of the target state of an object to be cooked.

[0019] [ Figure 6 ] Figure 6 This is a graph showing time series data.

[0020] [ Figure 7 ] Figure 7 This is a diagram illustrating an example of learning from a Hidden Markov Model.

[0021] [ Figure 8 ] Figure 8 This is a diagram illustrating the learning process of a Hidden Markov Model.

[0022] [ Figure 9 ] Figure 9 This is a diagram illustrating an example of deviation tolerance.

[0023] [ Figure 10 ] Figure 10 This is a diagram illustrating another example of deviation tolerance.

[0024] [ Figure 11 ] Figure 11 This is a flowchart illustrating the difference correction process.

[0025] [ Figure 12 ] Figure 12 This is a block diagram illustrating an example of a computer hardware configuration according to this disclosure. Detailed Implementation

[0026] In the following, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Note that the presentation apparatus, learning apparatus, presentation method, and presentation program according to the present disclosure are not limited to these embodiments. Furthermore, in the following embodiments, the same parts are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0027] Implementation Method

[0028] 1. Preface

[0029] Conventionally, a method has been proposed to visualize in real-time time-series information (TTW information) related to the heating temperature, heating time, and weight of the object to be cooked in order to assist in manual cooking. In such methods, the TTW information may differ from the profile information of a cook (e.g., a professional cook) used as a reference, for example, depending on the moisture content and temperature of the ingredients at the time of input, the temperature of the cooking chamber, etc. Such deviation from the reference is permissible for the purpose of achieving the same result, but there are important points (key points) that minimize the deviation from the reference in order to achieve the same level of completion. However, in conventional methods, because the key points cannot be properly distinguished, cooks attempting to reproduce a dish performed by another person (e.g., a reference cook) become anxious.

[0030] By simply presenting data related to the cooking reproduction as a reference (i.e., reference data indicating the cooking process to be reproduced), the cook cannot visually determine which element of the data (e.g., temperature, weight, and heat) should be emphasized at what time, or how much deviation from the reference data is permissible.

[0031] Therefore, this disclosure proposes a mechanism for estimating the tolerance of deviation from reference data (deviation tolerance). Specifically, in this disclosure, time-series information (e.g., TTW information) recorded in multiple ideal cooking processes is modeled using a Hidden Markov Model.

[0032] Here, the tolerance for deviation is information estimated for each cooking step based on reference data indicating the cooking process to be reproduced, and it is information indicating the permissible degree of the actual measured value of a parameter measured during cooking relative to the value of the parameter indicating the cooking process. Additionally, for example, the parameter measured during cooking is information detected by sensors or the like while the cook, who plans to reproduce the dish by referring to the reference data, is cooking. Furthermore, the tolerance for deviation is information indicating the variance of the parameter values ​​output by a Hidden Markov Model (HMM), in which the characteristic patterns of the reference data are modeled, for each state corresponding to the cooking process.

[0033] By using a Hidden Markov Model (HMM), it is possible to configure a model that absorbs the time distortion of each reference data point. Since such a model has a mean and variance within each state, a signal with low variance in each state is used as an indicator to determine whether the difference from the reference data should be reduced, and conversely, a signal with high variance is used as an indicator to determine whether the difference from the reference data is permissible.

[0034] In this disclosure, for example, because variance information and reference data are displayed on a monitor in the kitchen, cooks can easily grasp key points that match the reference data while cooking. Therefore, cooks can improve the reproducibility of their dishes.

[0035] 2. Overview

[0036] Next, an overview of this disclosure will be described. This disclosure estimates the tolerance for deviations from reference data. Furthermore, this disclosure relates to technology related to cooking assistance systems, which are systems that assist cooking in environments such as kitchens. This disclosure applies to cooking assistance systems using heating appliances such as IH ovens. The key points of this disclosure are as follows.

[0037] 1. Multiple pre-recorded reference data (cooking data) are applied to a Hidden Markov Model, and the time-series signal of the cooking steps is transformed into a discrete-state sequence. Each cooking step is represented by multiple states, and each state includes the mean and variance modeled in that state. Hereinafter, this discrete-state sequence is referred to as the "cooking model".

[0038] 2. Present information to the cook in real time when it can be compared with reference data indicating the cooking process of the dish being cooked.

[0039] 3. In addition to displaying time-series plots of the cooking data to be compared and the reference data, the cooking model also displays the mean and variance for each state. The cooking data to be compared can be real-time data during cooking or recorded time-series data.

[0040] 4. By displaying the mean and variance for each state, the degree of deviation can be visually identified, indicating how much the cooking data being compared differs from the reference data. For example, a large variance in a state representing the moment when the temperature drops as ingredients are added during cooking indicates a high tolerance for deviation in that state.

[0041] 5. On the other hand, when the variance of the displayed state is small, the difference between the cooking data to be compared and the reference data should be reduced, and this difference should be highlighted as a key point when reproducing the cooking.

[0042] 3. Effects

[0043] According to this disclosure, by presenting a tolerance for deviation, feedback can be provided to the cook at the current point in time during cooking regarding the tolerance for matching with reference data. Therefore, even if there is a deviation between the cooking data and the reference data during cooking, this deviation is tolerated even with a large variance, and it can be recognized that it does not affect the reproduction of the dish. On the other hand, even when there is a deviation between the cooking data and the reference data during cooking, but the variance is small, it can be recognized that the current point is a critical point affecting the reproduction of the dish, and it is necessary to consciously match the current point with the reference data. Therefore, it is possible to alleviate the anxiety experienced by cooks who are unaware of whether their methods are flawed when reproducing the dish, and to improve the degree of dish reproduction.

[0044] 4. Configuration of the cooking assistance system

[0045] Figure 1 This is a diagram showing an overview of the cooking assistance system (CS). Figure 2 This is a functional block diagram of the cooking assistance system (CS).

[0046] A Cooking Assist System (CS) is a smart kitchen that assists in cooking through collaboration between cooking appliances with built-in sensors and an information terminal. For example, a CS assists with cooking using a heating appliance with a power rating of KW. Figure 1 In the example, the induction heating (IH) heater HT and the pot PT are used as heating cookware KW.

[0047] The cooking assistance system CS generates cooking data for each dish using sensor data acquired from the sensor device SE. Cooking data is a summary of the cooking process performed by the cook US from start to finish. For example, cooking data includes time-series information (TTW information) related to the heating temperature, heating time, and weight of the object being cooked CO. Information related to the heating power (KW) of the heating cooker is also included in the cooking data associated with the heating temperature. Cooking data may also include motion images (cooking motion images) captured by the camera device sensor CM, showing the cooking process.

[0048] The web server SV stores a large amount of recorded cooking data. The cook at the time of recording could be the cook US or another cook (e.g., a professional cook). The cooking assistance system CS retrieves the recorded cooking data from the web server SV as reference data RF. The cook US performs cooking while referring to the reference data RF. Therefore, the recorded dish can be reproduced.

[0049] The cooking assistance system (CS) generates reproduction data (RP) based on the cooking profile of the dish being cooked. The cooking profile refers to the time-series information representing the operational history of the object being cooked (CO) and its state changes during the cooking process. The operational history of kitchen appliances such as the IH heater (HT) or pot (PT) is also included in the cooking profile as an indirect operational history of the object being cooked (CO). Furthermore, state changes of cooking appliances such as the IH heater (HT) or pot (PT) also affect the state changes of the object being cooked (CO) and are therefore included in the cooking profile.

[0050] Reproduction data (RP) is cooking data generated using sensor data acquired while the cook (US) is cooking, referencing reference data (RF). The cooking assistance system (CS) always presents the reproduced cooking data (RP) to the cook (US) in a form comparable to the reference data (RF) during the cooking period. This prompts the cook (US) to perform appropriate cooking actions based on the cooking conditions. An example of the presentation will be described later.

[0051] In addition, cooking assistance systems (CS) include, for example, IH heaters (HT), pots (PT), displays (DP), camera sensors (CM), microphones (MC), lighting (LT), web servers (SV), network routers (RT), input UI devices (IND), and information processing devices (IP).

[0052] The camera sensor CM captures an image of the object CO to be cooked in the pot PT. The microphone MC detects sound during cooking. The detected sound includes sounds produced by cooking and the voice of the cook US. The display DP presents various information (auxiliary information) for cooking assistance based on cooking data via video and audio. The input user interface (UI) device IND provides user input information from the cook US to the information processing device IP.

[0053] The input UI device IND includes the complete user interface of the cook US corresponding to the input device. Examples of input UI devices IND include, for example, information terminals such as smartphones, touch panels included in displays DP, etc. Furthermore, voice input via microphone MC, gestures input via camera sensor CM, etc., can also be included in the input information when using the input UI device IND. Therefore, microphone MC and camera sensor CM are used as the input UI device IND.

[0054] A display device (DP) includes all output UI devices that present information to the cook (US). For example, in addition to fixed displays such as LCD monitors or projectors, wearable displays such as augmented reality (AR) glasses can also be used as a display device (DP). A display device (DP) can also include non-visual information presentation devices, such as speakers or haptic devices.

[0055] The information processing unit IP controls the entire cooking assistance system CS holistically based on various types of information detected by the cooking assistance system CS. For example, the information processing unit IP includes a processor PR, a temperature sensor TS, a weight sensor WS, and a storage device ST. All components of the information processing unit IP may be contained within the IH heater HT, or part or all of the information processing unit IP may be mounted on an external device that communicates with the IH heater HT.

[0056] The temperature sensor TS measures the temperature of the object to be cooked (CO) in the pot PT or the pot PT. The weight sensor WS measures the weight of the object to be cooked (CO) in the pot PT. For example, a radiation thermometer capable of non-contact measurement with high response is used as the temperature sensor TS. A load sensor, etc., is used as the weight sensor WS. Figure 1 In the example, the temperature sensor TS is included in the IH heater HT, which serves as the heating control unit. However, the temperature sensor TS could also be included in the pot PT, which serves as the heating medium.

[0057] Temperature sensor TS and weight sensor WS, together with camera sensor CM and microphone MC, constitute sensor device SE for detecting various types of information in cooking assistance system CS. The state of the object to be cooked CO is detected based on sensor data acquired by sensor device SE. In addition to the aforementioned sensors, sensor device SE may also include other sensors capable of detecting the state of the object to be cooked CO. Sensor data acquired by sensor device SE is supplied to information processing device IP in real time.

[0058] The processor PR generates cooking data based on sensor data acquired from the sensor device SE. The processor PR uses the cooking data to generate auxiliary information and supplies this information to the display DP. The auxiliary information includes visual or auditory information that enables the cook US to identify the difference between the reproduced data RP and the reference data RF. The processor PR presents the reproduced data RP to the cook US in real time in a form that can be compared with the reference data RF, indicating a cooking profile of the dish being cooked.

[0059] The processor PR can adjust the lighting LT to control the shooting conditions of the camera sensor CM. The lighting LT is part of the collaborative device LD that controls the cooking environment. The collaborative device LD may include other devices such as air conditioning. The processor PR controls the operation of the collaborative device LD based on sensor data.

[0060] In addition, the processor PR communicates with external devices, web server SV, etc., via network router RT. The processor PR controls the operation of each device in the cooking assistance system CS based on external information, sensor data, and user input information acquired through communication.

[0061] The cooking data modeling unit UN corresponds to an apparatus for implementing information processing according to embodiments of the present disclosure (hereinafter referred to as "information processing according to embodiments"), and adds new functionality to the processor PR. That is, the cooking data modeling unit UN can be understood as part of the processor PR.

[0062] Figure 3 This is a functional block diagram of the cooking data modeling unit (UN). For example... Figure 3 As shown, the cooking data modeling unit UN may include a learning device 100 and a presentation device 200. Figure 3 At least a portion of the configuration shown is implemented by a predetermined program executed by the processor PR.

[0063] First, the learning device 100 will be described. For example... Figure 3 As shown, the learning device 100 includes a data acquisition unit 101, a learning unit 102, and a transmission unit 103.

[0064] The data acquisition unit 101 acquires reference data RF indicating the cooking process to be reproduced. For example, the data acquisition unit 101 acquires the reference data RF from the storage device ST.

[0065] The reference data RF includes time-series information (TTW information) related to the heating temperature, heating time, and weight of the object to be cooked (CO). Additionally, the reference data RF may also include time-series information on the set heating power of the heater HT. Furthermore, the reference data RF may include a cooking step ID for identifying the cooking step, and the cooking step ID can be used to indicate time-series transitions in the cooking process.

[0066] As mentioned above, the reference data RF is cooking data in which cooking processes by cook US or other cooks (e.g., professional cooks) are recorded, and serves as auxiliary information for cook US in cooking. The cooking process is represented by parameters such as the heating temperature, heating time, and weight of the object to be cooked (CO), and the set heating power of the heater HT. Therefore, the reference data RF can be considered as time-series data representing the values ​​of the parameters of the cooking process. Details of the reference data RF will be described later.

[0067] Learning unit 102 learns a model for estimating the tolerance of deviation. Specifically, learning unit 102 performs learning to model the feature patterns of reference data RF using a hidden Markov model. That is, learning unit 102 uses reference data RF as learning data to generate a hidden Markov model that estimates the mean and variance of the values ​​of the parameters representing the cooking process at each state.

[0068] For example, when information indicating a transition in a cooking step is associated with reference data RF, the learning unit 102 learns a Hidden Markov Model to assign states to each cooking step identified by a cooking step ID. Furthermore, when the number of states is greater than the number of cooking steps, the learning unit 102 can learn a Hidden Markov Model to assign multiple states to any cooking step.

[0069] The sending unit 103 sends the mean and variance of the output (estimated) by the hidden Markov model to the presentation device 200.

[0070] Subsequently, refer to Figure 3 Describe the presentation device 200. (e.g., ...) Figure 3 As shown, the presentation device 200 includes a tolerance acquisition unit 201, a generation unit 202, and a presentation unit 203.

[0071] The tolerance acquisition unit 201 acquires the average value and variance sent by the sending unit 103, and sends the acquired average value and variance to the generation unit 202.

[0072] The generation unit 202 generates information indicating the tolerance of deviation based on the mean and variance as auxiliary information for cooking assistance. For example, the generation unit 202 can also generate range information (tolerance range) indicating the variance status based on the mean and variance as information indicating the tolerance of deviation. For example, the presentation unit 203 can generate a navigation screen that compares the reproduced data RP with the reference data RF and displays the range information indicating the variance status in an overlay manner on the navigation screen.

[0073] Presentation unit 203 presents information indicating the tolerance of deviation for each cooking step. For example, presentation unit 203 can present the range information of the indicated variance state in a state superimposed on the reference data. In addition, presentation unit 203 can present the reproduction data RP of the cooking profile indicating the dish being cooked in real time in a format that can be compared with the reference data RF. For example, presentation unit 203 can display the navigation screen generated by generation unit 202 on display DP.

[0074] 5. Processing Flow

[0075] The information processing according to the implementation method can be broadly divided into two types. One is processing that only records cooking without presenting auxiliary information (recording mode). The other is processing that records cooking while presenting auxiliary information to the cook (US) (reproduction mode). The processing flow of each mode will be described below.

[0076] 5-1. Recording Mode

[0077] Figure 4 This is a diagram illustrating the processing flow in recording mode.

[0078] <Step S1: Recipe Setup>

[0079] The cook (US) sets recipe information as cooking conditions via the input UI device (IND). For example, the recipe information includes some or all of the following.

[0080] (i) Types and names of dishes

[0081] (ii) The types, quantities, cutting methods, and mixing methods of the ingredients to be used.

[0082] (iii) Cooking utensils to be used

[0083] (iv) Cooking steps (time and quantity of ingredients to be entered, heating power setting and heating time)

[0084] (v) The amount completed.

[0085] For example, when a cook selects a favorite recipe from a menu displayed on the input UI device IND, the recipe information is automatically set. Cooks can fine-tune individual information through the app.

[0086] <Step S2: The Beginning of Cooking>

[0087] The processor PR detects the start of cooking based on trigger events triggered by the cook US's operations. Upon detection of cooking start, the processor PR executes initialization processing. This initialization processing, along with recipe information, is registered in the web server SV. In the following flow, the process from sensor data input (step S3) to data recording (step S7) is a real-time, repetitive loop. Depending on the specific processing system, the execution timing, parallelization methods, etc., will vary.

[0088] <Step S3: Sensor Data Input>

[0089] The processor PR receives sensor data from the sensor device SE that detects the cooking status. Typically, the sensor frame rates differ and are not necessarily synchronized. All received data carries a timestamp based on a clock device managed by the processor PR. While the specific use of the clock device is unrestricted, the timestamps of all sensor data are time-managed to allow for comparison.

[0090] <Step S4: UI Data Input>

[0091] The processor PR receives user input information from the input UI device IND.

[0092] <Step S5: Identification of Cooking Steps>

[0093] The processor PR identifies the current cooking step based on recipe information and input data up to the present (sensor data, user input). In other words, the processor PR specifies which stage of the cooking process is being performed. Real-time execution of this process is optional in recording mode, and the recorded cooking data can be analyzed and processed offline after cooking is complete.

[0094] <Step S6: Screen Display Update>

[0095] The processor (PR) uses information processed in previous stages to present information related to the current cooking state on the display (DP). The specific presentation method will be described later.

[0096] <Step S7: Data Recording>

[0097] The processor PR records all the data required in playback mode in an appropriate format and generates cooking data. It is assumed that a non-volatile storage device ST is the conventional recording destination, but the recording destination is not limited to this. Any storage device accessible by the processor PR can be used as the recording destination. Recording is not necessarily performed, and data can be streamed to external devices.

[0098] <Step S8: End of Cooking>

[0099] The processor PR detects the end of cooking based on trigger events from the cook US's operations. When the end of cooking is detected, the processor PR executes termination processing. This termination processing, along with the recipe information, is registered in the web server SV.

[0100] 5-2. Reproduction Mode

[0101] In the following text, the differences in operation relative to the recording mode will be focused on describing the content of each process in the playback mode. Since steps S2, S3, S4, S7, and S8 are the same as those in the recording mode, their description will be omitted.

[0102] <Step S1: Recipe Setup>

[0103] The cook, US, uses the recorded cooking data (reference data RF) specified by the input UI device IND as a reference. The reference data RF shows the cooking process to be reproduced by the cook, US. Using the reference data RF as a target, the cook, US, performs cooking such that the data obtained through cooking performed by the cook (reproduced data RP) approximates the reference data RF.

[0104] The processor PR reads the specified reference data RF. The processor PR sets the recipe information included in the reference data RF as the cooking conditions. The cook US can start cooking under the same conditions, or change the conditions via the input UI device IND, similar to the recording mode.

[0105] <Step S5: Identification of Cooking Steps>

[0106] The processor PR identifies the current cooking step based on recipe information and input data up to the present (sensor data, user input). In other words, the processor PR specifies which stage of the cooking process is being performed. This process is required in reproduction mode. In particular, it is important to specify the start and end times of each cooking step with high accuracy based on the cooking start time.

[0107] <Step S6: Screen Display Update>

[0108] In reproduction mode, the processor (PR) presents information related to the current cooking state on the display (DP) in a format that can be compared with the reference data (RF). The specific presentation method will be described later.

[0109] 6. The target state of the object to be cooked

[0110] Figure 5 This is an example diagram visualizing the target state of the object CO to be cooked. The start / end time of each cooking step is specified against reference data RF, based on cooking step identification by the processor PR. Therefore, the target state of a cooking step can be presented based on sensor data near the end of that step.

[0111] The target state indicates the conditions at which a cooking step is completed and the process moves to the next cooking step. The target state is defined by any one of the time / temperature / weight (TTW) values ​​that can be measured by the cooking assistance system (CS). The TTW value is the value of the target parameter, which defines the target state.

[0112] Here, Figure 5 An example of a navigation screen displayed on a monitor DP is shown.

[0113] exist Figure 5 In area 151, a small rectangular region in the upper left corner of the navigation screen, the number indicating that the cooking step being performed is the 12th of 22 cooking steps is displayed. In the horizontally elongated area 152 to the right of area 151, text indicating the content of the cooking task is displayed. Based on the text displayed in area 152, the cook can confirm that the task, as the 12th cooking step, involves mixing the ingredients.

[0114] Below region 152 are regions 153-1 to 153-3. Regions 153-1 to 153-3 are display areas set to display the weight, time, and temperature, which define the target state, respectively.

[0115] exist Figure 5 In the example, area 153-1 shows a weight difference of 5.9 g between the weight set to define the target state and the current weight. This weight difference decreases over time due to the evaporation of moisture accompanying heating. The cook will mix the ingredients until the displayed weight difference is 0 g. Once the weight difference reaches 0 g, the target state is determined to have been achieved, and cooking step 13 begins.

[0116] When the cooking step in progress involves inputting ingredients, the ingredient information is displayed in area 154 at the bottom of the screen. In area 155 to the right of area 154, information is displayed indicating that the ingredients to be added in the 13th cooking step are chopped tomatoes and room temperature water.

[0117] By checking such a navigation screen, cooks can easily proceed through each cooking step to reproduce the cooking process.

[0118] according to Figure 5 For example, if the target state is defined by the elapsed time since the start of the cooking step (when the cooking step is being performed), the processor PR will start the next cooking step when the elapsed time since the start of the cooking step reaches the time defined as the target state.

[0119] When the target state is defined by the temperature of the object to be heated or the heating medium, the processor PR starts the next cooking step when the temperature measured by the temperature sensor TS reaches the temperature defined as the target state.

[0120] When the target state is defined by the weight of the object to be heated, the processor PR starts the next cooking step when the weight measured by the weight sensor WS reaches the weight defined as the target state.

[0121] In this way, the processor PR instructs the value of the target parameter, maintains the execution of the cooking steps until the target parameter value is reached, and guides the cook to the next cooking step when the target parameter value is reached. Through such guidance, even if the cook US does not fully grasp the cooking method, the cook US can intuitively understand what type of cooking step should be executed at what time, thereby improving the reproducibility of the dish.

[0122] Here, as mentioned above, while there is an acceptable deviation from the reference data in order to achieve the goal of making the dish as identical as possible, there are also important points (i.e., key points) where the difference from the reference data should be minimized to achieve the same result. However, since guiding solely by the value of the target parameter cannot properly distinguish these key points, the cook (US) still experiences anxiety in improving the reproducibility of the dish. Specifically, there is room for improvement in reducing the anxiety experienced by the cook due to uncertainty about the validity of their methods when reproducing the dish. Cooking under anxiety does not necessarily improve the reproducibility of the dish.

[0123] In view of such problems, the inventors of this disclosure have introduced a new cooking data modeling unit UN (an additional function of the processor PR) that implements information processing according to the embodiments. The following description focuses on the cooking data modeling unit UN.

[0124] 7. Learning Data

[0125] The learning unit 102 of the modeling unit UN uses the time series data included in the reference data RF to learn the hidden Markov model, and describes the time series data used as the learning data. Figure 6 This is a graph showing time series data.

[0126] Figure 6 The entire cooking process, from the start to the end of cooking, is shown. Figure 6 The horizontal axis indicates the time from the start of cooking. For example, as time-series data, the heating power P (set heating power) of the IH heater HT, the temperature T measured by the temperature sensor TS, and the weight W measured by the weight sensor WS are recorded in time series. For example, the heating power of the IH heater HT is represented by levels from 1 to 7. The higher the level, the greater the set value of the heating power.

[0127] In addition, according to Figure 6 For example, "#1", "#2", "#3", "#4", and "#5" are assigned as cooking step IDs to time series data to identify cooking steps. For instance, a processor PR can assign cooking step IDs by recognizing cooking steps.

[0128] For example, the start / end time of each cooking step is specified based on sensor data and known information stored by the cooking assistance system (CS). For instance, the boundaries of cooking steps identified by the processor (PR) are characterized at the timing of the operations performed. These operations do not always determine the boundaries of cooking steps, but are often effective demarcation criteria.

[0129] Switch heating power

[0130] Input the ingredients into the pot PT

[0131] Remove the ingredients from the pot (including removing any impurities).

[0132] Start / stop operation (e.g., stirring ingredients and shaking the pot)

[0133] Place / remove the pot itself, pot lid, etc.

[0134] The processor PR analyzes the sensed time-series data and the recipe information input by the user to specify the timing for performing each operation. The processor PR interprets these timings as boundaries of steps and estimates the start / end timing of each step.

[0135] For example, the timing for switching heating power can be accurately specified based on changes in the heating power setting of the cooking oven HT. By capturing discontinuous changes in weight W, the timing for all processes, such as the input and removal of ingredients, and the placement and removal of the pot and lid, can be estimated. The tendency for the rate of change of temperature T to become discontinuous when the amount of ingredients to be input is large and the temperature difference between the ingredients and the food in the pot is also a factor in determining the material. When the weight W continuously changes discontinuously, it is possible to estimate that an operation involving human intervention (e.g., stirring) has been performed. Therefore, as... Figure 6 As shown, the processor PR is able to associate cooking step IDs with each cooking step by separating the cooking work for each cooking step.

[0136] according to Figure 6 For example, the cooking step IDs change in the order of "#1" → "#2" → "#3" → "#4" → "#5" according to the time sequence. Therefore, the information of the time sequence order of the cooking step IDs corresponds to the information indicating the transition of the cooking steps.

[0137] Note that cooking step IDs can be manually associated with time series data. However, when using time series data as training data, cooking step IDs are not necessarily included. However, as... Figure 6 As shown, by using cooking step IDs in time-series data to indicate transitions in cooking steps, learning unit 102 is able to learn a Hidden Markov Model to enforce state assignment for each cooking step. Therefore, learning unit 102 is able to generate a Hidden Markov Model that reliably estimates the mean and variance of each cooking step.

[0138] 8. Learning Methods

[0139] Next, we will refer to Figure 7 Describe the learning methods for Hidden Markov Models. Figure 7 This is a diagram illustrating an example of learning using a Hidden Markov Model (HMM). First, the HMM deals with time series data labeled with discrete times. The data format can be, for example, real numbers, integers, or categorical data, as long as a probability distribution can be defined.

[0140] Therefore, learning unit 102 will Figure 6 The time series data shown is converted into time series data labeled with discrete time. For example, learning unit 102 can convert time series data into signal sequence data including N-dimensional × S-step (the number of cooking steps S). Figure 7 An example of signal sequence data is shown.

[0141] Specifically, Figure 7(a) illustrates an example of how the learning unit 102 converts reference data RF (time series data) into signal sequence data SQ, which includes: the heating power P of the IH heater HT, the temperature T measured by the temperature sensor TS, the weight W measured by the weight sensor WS, and four elements (four dimensions) of the cooking step ID, as well as five steps (number of cooking steps: 5). For example, with reference data RF prepared for M samples, the learning unit 102 converts each of the reference data RF to obtain M signal sequence data SQ.

[0142] Note that the number S of cooking steps used for each reference data RF can be different. An advantage of using a Hidden Markov Model is that, regardless of the number of steps, time with similar sensing is assigned to the same state.

[0143] In this state, learning unit 102 learns a Hidden Markov Model by using signal sequence data SQ as learning data. As described above, in this disclosure, since a cooking model that models the feature patterns of time series data is generated, a left-to-right form that does not transition to states in past times is used. Here, in Figure 7 (b) shows the configuration of the cooking model of the Hidden Markov Model. Figure 7 (b) shows the general configuration of a left-to-right hidden Markov model, known as a state transition diagram. In the state transition diagram, circles indicate states, and arrows between circles indicate state transitions. Furthermore, Figure 7 (b) shows an example of an 8-state, 8-transition cycle hidden Markov model.

[0144] according to Figure 7 In example (b), to generate a cooking model using a Hidden Markov Model, features are extracted for each element (heating power P, temperature T, weight W) included in the signal sequence data SQ, and state partitioning is performed in the feature sequence. Then, based on the cooking step ID, the cooking step ID is associated with each state (state 1, state 2, state 3, ...) obtained as a result of the state partitioning. In other words, states are assigned to cooking step IDs.

[0145] As an example, state 1 is assigned to the cooking step identified by cooking step ID "#1", and states 2 and 3 are assigned to the cooking step identified by cooking step ID "#2". Additionally, states 4 and 5 are assigned to the cooking step identified by cooking step ID "#3", states 6 and 7 are assigned to the cooking step identified by cooking step ID "#4", and state 8 is assigned to the cooking step identified by cooking step ID "#5".

[0146] Note that although the number of states needs to be pre-set to generate the cooking model, as in the example above, when the number of states is greater than the number of cooking steps S, multiple states are assigned to a single cooking step. The number of states is a meta-parameter given by humans.

[0147] In the learning of a Hidden Markov Model (HMM), the model's parameters are optimized to best generate a given observation sequence. The observation sequence used here is the training data, and the signal sequence data SQ is provided as the training data. Therefore, optimizing the model parameters means learning the HMM.

[0148] use Figure 7 In example (b), given the state j at time t-1, the probability called transition probability a is given. ij The model parameters, the transition probability a ij This represents the probability that the state transitions to i at time t. Additionally, the initial state probability π is also given. i The output distribution (the probability of observation generation in each state) p is used as the model parameters. Here, in this disclosure, the output distribution p is defined as a Gaussian distribution, and the mean μ and variance σ in each state are further defined as follows: 2 Defined as model parameters, used to determine the output distribution p.

[0149] In such an example, learning a Hidden Markov Model refers to estimating the model parameters λ (transition probabilities a). ij Initial state probability π i Output distribution p, mean μ, variance σ 2 The goal is to maximize the likelihood of the given learning data. In other words, maximizing the likelihood estimate is to obtain the model parameters with the maximum likelihood.

[0150] In the case of learning a Hidden Markov Model, connection learning is used. Therefore, the iterative steps of the Baum-Welch algorithm are executed to estimate the model parameters.

[0151] In the Baum-Welch algorithm, the expected value is estimated based on the initial value of the model parameter λ, and this initial value is used to estimate the observed value x indicated by the signal sequence data SQ. S Perform maximum likelihood estimation (S = number of cooking steps). Define a step as the step that ultimately optimizes the model parameter λ to match the result and obtain a new value. Repeat this step until the value of the model parameter λ converges. The conditions for determining convergence can be arbitrarily determined, for example, such that the value of the model parameter λ does not change within 10 or more steps.

[0152] Here, with the signal sequence data SQ given as the learning data, the mean and variance of the parameter values ​​are estimated using the Baum-Welch algorithm for each parameter representing the cooking process. Specifically, for each of the heating power P, temperature T, and weight W, the mean and variance are estimated for each cooking step.

[0153] 9. Learning Process

[0154] Figure 8 This diagram illustrates the learning process of a Hidden Markov Model. First, the processor PR executes the process of recording reference data RF and storing the recorded reference data RF in the storage device ST (step S801). The method for recording the reference data RF is as described in [reference]. Figure 4 As described.

[0155] Next, the data acquisition unit 101 acquires (samples) the reference data RF from the storage device ST (step S802). For example, the data acquisition unit 101 may sample the reference data RF every n (milliseconds) (n is any number).

[0156] Learning unit 102 converts the acquired reference data RF into signal sequence data SQ (step S803). For example, as Figure 6 As shown, when the cooking step ID is associated with the reference data RF, the learning unit 102 can convert the reference data RF into signal sequence data SQ of the number S of cooking steps identified according to the cooking step ID.

[0157] Learning unit 102 generates a cooking model by using a left-to-right Hidden Markov Model as a predetermined Hidden Markov Model (step S804). Specifically, learning unit 102 performs learning (learning for estimating model parameters λ) by providing signal sequence data SQ to the Hidden Markov Model. Therefore, a cooking model that models the feature patterns of reference data RF can be obtained.

[0158] Finally, learning unit 102 stores the model parameters λ in storage device ST (step S805). Note that although in Figure 8 Although not shown in the diagram, the sending unit 103 can also obtain the model parameters λ estimated (output) by the hidden Markov model from the storage device ST and send the model parameters λ to the presentation device 200. The sent model parameters λ are obtained by the tolerance acquisition unit 201.

[0159] 10. Example of presenting deviation tolerance

[0160] Figure 9This is a diagram illustrating an example of the presentation of the deviation tolerance. When the deviation tolerance (i.e., the mean and variance estimated through learning by the hidden Markov model) is obtained by the tolerance acquisition unit 201, the generation unit 202 generates a screen RFG, on which the information indicating the deviation tolerance is displayed together with the reference data RF.

[0161] As described above, based on the Hidden Markov Model, the mean and variance of each of the heating power P, temperature T, and weight W are estimated for each cooking step. Therefore, the generation unit 202 can generate a separate frame RFG for each of the heating power P, temperature T, and weight W. Specifically, the generation unit 202 can generate frames RFG1, RFG2, and RFG3 separately. The mean and variance corresponding to temperature T are displayed on frame RFG1 together with the reference data RF for temperature T; the mean and variance corresponding to weight W are displayed on frame RFG2 together with the reference data RF for weight W; and the mean and variance corresponding to heating power P are displayed on frame RFG3 together with the reference data RF for heating power P.

[0162] In this state, the presentation unit 203 can cause the display DP to display screens RFG1, RFG2, and RFG3 at the same time. On the other hand, the presentation unit 203 can cause the display DP to display only screens RFG1, RFG2, and RFG3 selected by the cook US. Figure 9 An example is shown where the presentation unit 203 displays screen RFG1 on the display DP according to the selection of the cook US.

[0163] exist Figure 9 In the example, it is assumed that in the learning process of the Hidden Markov Model, the mean and variance in state 1 are estimated by assigning state 1 to the cooking step identified by cooking step ID "#1" through the Hidden Markov Model. In this case, the generation unit 202 generates a variance image IM, which is an image of the state in which the variance is represented by circular regions, and the generated variance image IM is superimposed and displayed to correspond to the cooking step ID "#1" in the reference data RF of heating power P.

[0164] Furthermore, assuming that in the learning process of the Hidden Markov Model, since state 2 is assigned to the cooking step identified by cooking step ID "#2", the mean and variance in state 2 are estimated by the Hidden Markov Model. In this case, the generation unit 202 generates a variance image IM, which is an image of the state in which the variance is represented by circular regions, and overlays and displays the generated variance image IM to correspond to the cooking step ID "#2" in the reference data RF of the heating power P.

[0165] Furthermore, assuming that in the learning process of the Hidden Markov Model, since state 3 is assigned to the cooking step identified by cooking step ID "#3", the mean and variance in state 3 are estimated by the Hidden Markov Model. In this case, the generation unit 202 generates a variance image IM, which is an image of the state in which the variance is represented by circular regions, and overlays and displays the generated variance image IM to correspond to the cooking step ID "#3" in the reference data RF of the heating power P.

[0166] Furthermore, assuming that in the learning process of the Hidden Markov Model, since state 4 is assigned to the cooking step identified by cooking step ID "#4", the mean and variance in state 4 are estimated by the Hidden Markov Model. In this case, the generation unit 202 generates a variance image IM, which is an image of the state in which the variance is represented by circular regions, and overlays and displays the generated variance image IM to correspond to the cooking step ID "#4" in the reference data RF of the heating power P.

[0167] Furthermore, assuming that in the learning process of the Hidden Markov Model, since state 5 is assigned to the cooking step identified by cooking step ID "#5", the mean and variance in state 5 are estimated by the Hidden Markov Model. In this case, the generation unit 202 generates a variance image IM, which is an image of the state in which the variance is represented by circular regions, and overlays and displays the generated variance image IM to correspond to the cooking step ID "#5" in the reference data RF of the heating power P.

[0168] Figure 9 An example of screen RFG1 corresponding to the example above is shown. According to... Figure 9 The example shown in the image RFG1 has 13 circular variance images IM superimposed and displayed on the reference data RF of the heating power P. In such an example, the variance is estimated to decrease at points where the temperature T needs to be strictly adjusted with the reference data RF (e.g., the area of ​​the variance image IM is less than a predetermined threshold).

[0169] Therefore, for example, generation unit 202 can extract the variance image IM from the 13 variance images IM as important points (i.e., key points CP) that require strict adjustment of temperature T, and display the extracted variance image IM along with the key points CP. Thus, the cook US is able to recognize the need for more careful work when approaching the key points CP while referring to the reproduction data RP generated, for example, based on the cooking profile of the dish he / she is cooking. Therefore, the cook US is able to improve the reproduction accuracy of the dish.

[0170] Although Figure 9Although not shown in the diagram, the reproduction data RP corresponding to temperature T can also be displayed on screen RFG1. The reproduction data RP is real-time information and is transmitted via... Figure 4 The method described in the document is used to obtain it.

[0171] 11. Another example of deviation tolerance

[0172] Figure 10 This is another example of a presentation illustrating the tolerance for deviation. Generation unit 202 can generate a visualization VD, which allows the cook (US) to visually compare the reproduced data RP and the reference data RF. The difference (deviation) between the reproduced data RP and the reference data RF is displayed in real-time on the visualization VD. Figure 10 An example is shown of presenting auxiliary information using a time-series graph (GP) on a visualization screen (VD).

[0173] In this example, the sensor data is directly visualized as a time-series plot (GP). Time is set on the horizontal axis, and values ​​of time-series data such as heating power P, temperature T, and weight W are set on the vertical axis. As a simple example, Figure 10 A graph illustrating the change in pot bottom temperature over time, measured by temperature sensor TS, is schematically shown. The horizontal axis represents time t, and the vertical axis represents pot bottom temperature T. The origin t0 of the time axis is the start time of each cooking step. Therefore, the visualization VD can be a magnified and displayed portion of the image RFG1. For example, the presentation unit 203 can reproduce data RP close to... Figure 9 The timing of the key point CP in the display causes the display DP to focus on the visualization VD near the key point CP.

[0174] Reference data (RF) and reproduction data (RP) can be displayed with an origin aligned to each cooking step (the start time of the cooking step). Therefore, visualization allows for comparison of temperature values ​​with the time axis origin aligned to each cooking step. The dashed line is an indicator of the current time. The current temperature difference relative to the reference data (RF) is displayed as "-5°C". The graph is updated in real-time over time, and the line indicating the reproduction data (RP) extends to the right. Additionally, a variance image (IM) and its corresponding mean (AV) are also displayed.

[0175] The cook, US, can control the cooking process while observing the curve, thus reducing the temperature difference with the target reference data, RF.

[0176] Figure 9 and Figure 10An example is shown where generation unit 202 generates a variance image IM and presentation unit 203 displays the variance image IM, which is an image in which circular regions represent the range (tolerance range) of the variance state. However, the range (tolerance range) of the variance state is not necessarily represented by circular regions. For example, line segments indicating the upper limit of variance and line segments indicating the lower limit of variance can be used to represent the range of the variance state.

[0177] Here, the presentation unit 203 can further cause the display DP to display information corresponding to the following determination result: whether the difference between the value of the parameter representing the cooking process included in the reference data RF and the actual measured value of the parameter measured during cooking falls within the range of the indicated variance state (deviation tolerance range). For example, if the difference does not fall within the range, the presentation unit 203 presents a method to bring the difference into the range. On the other hand, if the difference falls within the range, the presentation unit 203 presents that cooking is proceeding normally. (Refer to...) Figure 11 Describe in detail the processes that facilitate the correction of differences.

[0178] Figure 11 This is a flowchart illustrating the difference correction process. First, the presentation unit 203 determines whether the difference between the parameter value indicated by the reference data RF and the actual measured value of the parameter indicated by the reproducible data RP falls within the range of the variance state indicated by the variance image IM, which serves as the key point CP (step S1101). The parameter here can be any one of the heating power P, temperature T, and weight W.

[0179] If the difference does not fall within the range (step S1101: No), the presentation unit 203 specifies a method for correcting the difference (i.e., a method for bringing the difference within the range) (step S1102). For example, if the actual measured value of the parameter indicated by the reproduction data RP is lower than the parameter value indicated by the reference data RF and does not fall within the range, the presentation unit 203 may specify increasing the actual measured value of the parameter as a correction method. Figure 10 For example, the rendering unit 203 specifies the operation of increasing the temperature so that the temperature T falls within the variance range indicated by the variance image IM as the correction method.

[0180] Furthermore, if the actual measured value of a parameter indicated by the reproducible data RP is higher than the parameter value indicated by the reference data RF but not within the range, the presentation unit 203 can specify a method to reduce the actual measured value of the parameter as a correction method. Figure 10 For example, the rendering unit 203 specifies the operation of lowering the set temperature so that the temperature T falls within the variance range indicated by the variance image IM as the correction method.

[0181] As described above, when a correction method for the difference is specified, the presentation unit 203 causes the display DP to show the specified correction method (step S1103). For example, the presentation unit 203 may display the correction method and an alarm message for warning of a large difference that is out of range in the currently displayed visualization screen VD. The alarm message may be voice or text.

[0182] On the other hand, if the difference falls within the range (step S1101: Yes), the presentation unit 203 presents that cooking is proceeding normally (step S1104). For example, the presentation unit 203 may present a statement that enhances the cook's (US) mood, such as "That's it!".

[0183] As described above, based on the processing of correction of the facilitation difference through the presentation unit 203, the cook US can be appropriately guided, thereby improving the reproduction degree of the dish's completion. Note that, although Figure 11 An example is shown where the rendering unit 203 performs correction processing on the variance image IM corresponding to the key point CP, but correction processing can also be performed on all variance images IM, regardless of the key point CP.

[0184] Furthermore, when the range of the indicated variance state is represented by a line segment of the upper limit of the indicated variance and a line segment of the lower limit of the indicated variance, in step S1101, the presentation unit 203 can determine whether the difference falls within the area between the line segment of the upper limit of the indicated variance and the line segment of the lower limit of the indicated variance.

[0185] 12. Hardware Configuration

[0186] Reference Figure 12 Examples of computer hardware configurations corresponding to the presentation device 200 and learning device 100 according to this disclosure are provided. Figure 12 This is a block diagram illustrating an example of a computer hardware configuration according to this disclosure. Note that... Figure 12 Examples of hardware configurations for a computer according to this disclosure are shown, and the configurations are not necessarily limited to... Figure 12 The configuration shown.

[0187] like Figure 12 As shown, computer 1000 includes a central processing unit (CPU) 1100, random access memory (RAM) 1200, read-only memory (ROM) 1300, hard disk drive (HDD) 1400, communication interface 1500, and input / output interface 1600. Each unit of computer 1000 is connected via bus 1050.

[0188] The CPU 1100 operates based on programs stored in ROM 1300 or HDD 1400 and controls each unit. For example, the CPU 1100 develops programs stored in ROM 1300 or HDD 1400 in RAM 1200 and executes processing corresponding to various programs.

[0189] ROM 1300 stores boot programs such as the Basic Input / Output System (BIOS) executed by CPU 1100 when computer 1000 starts up, and programs that depend on the hardware of computer 1000.

[0190] HDD 1400 is a computer-readable recording medium that non-transitorily records a program executed by CPU 1100, data used by the program, etc. Specifically, HDD 1400 records program data 1450. Program data 1450 is an example of an information processing program for implementing the information processing method according to embodiments of the present disclosure and data used by the information processing program.

[0191] Communication interface 1500 is an interface for connecting computer 1000 to an external network 1550 (e.g., the Internet). For example, CPU 1100 receives data from another device or sends data generated by CPU 1100 to another device via communication interface 1500.

[0192] Input / output interface 1600 is an interface for connecting input / output device 1650 and computer 1000. For example, CPU 1100 receives data from input devices such as keyboard and mouse via input / output interface 1600. Additionally, CPU 1100 sends data to output devices such as display devices, speakers, or printers via input / output interface 1600. Furthermore, input / output interface 1600 can be used as a media interface for reading programs recorded on a predetermined recording medium (medium). For example, the medium is an optical recording medium such as a digital multifunction disc (DVD) or phase-change rewritable disc (PD), a magneto-optical recording medium such as a magneto-optical disc (MO), magnetic tape, magnetic recording media, semiconductor memory, etc.

[0193] For example, when computer 1000 is used as a presentation device 200 according to the present disclosure, the CPU 1100 of computer 1000 executes an information processing program loaded on RAM 1200 to implement the... Figure 3 Each process shown executes various processing functions. That is, the CPU 1100, RAM 1200, etc., cooperate with software (a presentation program loaded on RAM 1200) to implement the presentation method of the presentation apparatus 200 according to this disclosure.

[0194] Furthermore, when the computer 1000 is used as a learning device 100 according to this disclosure, the CPU 1100 of the computer 1000 executes an information processing program loaded on the RAM 1200 to implement the... Figure 3 Each process shown executes various processing functions. That is, the CPU 1100, RAM 1200, etc., cooperate with software (a learning program loaded on RAM 1200) to implement the learning method of the learning device 100 according to this disclosure.

[0195] 13. Summary

[0196] Although embodiments of the present disclosure have been described above, the technical scope of the present disclosure is not limited to the embodiments described above, and various modifications can be made without departing from the spirit of the present disclosure. Furthermore, components of the embodiments and modifications can be appropriately combined.

[0197] Furthermore, the effects of the embodiments described in this specification are merely illustrative and not limiting, and may provide other effects.

[0198] Note that this disclosure may also have the following configurations.

[0199] (1) A presentation device, comprising:

[0200] A tolerance acquisition unit for acquiring a deviation tolerance, wherein the deviation tolerance is estimated for each cooking step based on reference data indicative of the cooking process, and the deviation tolerance indicates the degree to which the actual measured value of a parameter measured during cooking is permissible relative to the value of the parameter indicative of the cooking process; and

[0201] The presentation unit presents the tolerance for deviation for each cooking step.

[0202] (2) The presentation device according to (1), wherein the presentation unit presents the deviation tolerance in a state superimposed on the reference data.

[0203] (3) The presentation device according to (2), wherein,

[0204] The deviation tolerance is the variance of the parameter values ​​output by a hidden Markov model, which models the feature patterns of the reference data, for each state corresponding to the cooking step.

[0205] The presentation unit presents variance information indicating the state of variance in a superimposed state for each cooking method included in the reference data.

[0206] (4) According to the presentation device described in (3), wherein the presentation unit presents the variance image as the variance information in a superimposed state, and the state of the variance is represented by a circular region in the variance image.

[0207] (5) The presentation device according to any one of (1) to (4), wherein the presentation unit presents in real time reproduction data of a cooking profile indicating a dish being cooked in a format that can be compared with the reference data.

[0208] (6) The presentation device according to (1), wherein the presentation unit presents information based on the following determination result: whether the difference between the value of the parameter representing the cooking process included in the reference data and the actual measured value of the parameter measured during cooking falls within the range indicated by the deviation tolerance.

[0209] (7) The presentation device according to (6), wherein when the difference does not fall within the range, the presentation unit presents a method for bringing the difference into the range, and when the difference falls within the range, the presentation unit presents that the current cooking is proceeding normally.

[0210] (8) A learning device comprising a learning unit that learns a hidden Markov model to output the variance of values ​​representing parameters of a cooking process as a tolerance for deviation used by a presentation device, the presentation device comprising:

[0211] A tolerance acquisition unit for acquiring the deviation tolerance, the deviation tolerance being estimated for each cooking step based on reference data indicative of the cooking process, and the deviation tolerance indicating the degree to which the actual measured value of the parameter measured during cooking is permissible relative to the value of the parameter indicative of the cooking process; and

[0212] The presentation unit presents the tolerance for deviation for each cooking step.

[0213] (9) The learning device according to (8) further includes a data acquisition unit for acquiring the reference data, wherein,

[0214] The learning unit performs learning to model the feature patterns of the reference data using the hidden Markov model.

[0215] (10) The learning device according to (8) or (9), wherein,

[0216] The reference data includes time-series data representing the values ​​of parameters of the cooking process, and

[0217] The learning unit generates a hidden Markov model by using the time series data as learning data, and the hidden Markov model outputs the mean and variance of the parameters in each state.

[0218] (11) The learning apparatus according to any one of (8) to (10), wherein, when information indicating the transition of the cooking step is associated with the time series data, the learning unit learns the hidden Markov model to assign a state to each cooking step.

[0219] (12) The learning device according to (11), wherein, when the number of states is set to be greater than the number of cooking steps, the learning unit learns the hidden Markov model to assign multiple states to any one of the cooking steps.

[0220] (13) A presentation method performed by a presentation device, the presentation method comprising:

[0221] A tolerance acquisition step for obtaining a deviation tolerance, wherein the deviation tolerance is estimated for each cooking step based on reference data indicative of the cooking process, and the deviation tolerance indicates the degree to which the actual measured value of a parameter measured during cooking is permissible relative to the value of the parameter indicative of the cooking process; and

[0222] The presentation step presents the tolerance for deviation for each cooking step.

[0223] (14) A presentation procedure for causing a presentation device to perform the following process:

[0224] A tolerance acquisition process for obtaining a deviation tolerance, wherein the deviation tolerance is estimated for each cooking step based on reference data indicative of the cooking process, and the deviation tolerance indicates the degree to which the actual measured value of a parameter measured during cooking is permissible relative to the value of the parameter indicative of the cooking process; and

[0225] The presentation process presents the tolerance for deviation for each cooking step.

[0226] List of reference numerals

[0227] UN Cooking Data Modeling Unit

[0228] 100 Learning Devices

[0229] 101 Data Acquisition Unit

[0230] 102 Learning Units

[0231] 103 Transmitting Unit

[0232] 200 presentation devices

[0233] 201 Tolerance Acquisition Unit

[0234] 202 Generation Unit

[0235] 203 presentation units

Claims

1. A presentation device, comprising: A tolerance acquisition unit for acquiring a deviation tolerance, the deviation tolerance being estimated for each cooking step based on reference data indicating the cooking process, and the deviation tolerance indicating the degree to which the actual measured value of a parameter measured during cooking is permissible relative to the value of the parameter indicating the cooking process; as well as The presentation unit presents the tolerance for deviation for each cooking step.

2. The presentation device according to claim 1, wherein, The presentation unit presents the deviation tolerance in a state superimposed on the reference data.

3. The presentation device according to claim 2, wherein, The deviation tolerance is the variance of the parameter values ​​output by a hidden Markov model, which models the feature patterns of the reference data, for each state corresponding to the cooking step. The presentation unit presents variance information indicating the state of the variance in an overlay state for each cooking process included in the reference data.

4. The presentation device according to claim 3, wherein, The presentation unit presents the variance image as the variance information in a superimposed state, and the state of the variance is represented by a circular area in the variance image.

5. The presentation device according to claim 1, wherein, The presentation unit presents in real time reproduced data of the cooking profile indicating the dish being cooked in a format that allows comparison with the reference data.

6. The presentation device according to claim 1, wherein, The presentation unit presents information based on the following determination: whether the difference between the value of the parameter representing the cooking process included in the reference data and the actual measured value of the parameter measured during cooking falls within the range indicated by the deviation tolerance.

7. The presentation device according to claim 6, wherein, When the difference does not fall within the range, the presentation unit presents a method for bringing the difference into the range, and when the difference falls within the range, the presentation unit presents that the current cooking is proceeding normally.

8. A learning device comprising a learning unit that learns a hidden Markov model to output the variance of values ​​representing parameters of a cooking process as a tolerance for deviation used by a presentation device, the presentation device comprising: A tolerance acquisition unit for acquiring the deviation tolerance, the deviation tolerance being estimated for each cooking step based on reference data indicative of the cooking process, and the deviation tolerance indicating the degree to which the actual measured value of the parameter measured during cooking is permissible relative to the value of the parameter indicative of the cooking process; as well as The presentation unit presents the tolerance for deviation for each cooking step.

9. The learning device according to claim 8, further comprising a data acquisition unit for acquiring the reference data, wherein, The learning unit performs learning to model the feature patterns of the reference data using the hidden Markov model.

10. The learning device according to claim 8, wherein, The reference data includes time-series data representing the values ​​of parameters of the cooking process, and The learning unit generates a hidden Markov model by using the time series data as learning data, and the hidden Markov model outputs the mean and variance of the parameters in each state.

11. The learning device according to claim 10, wherein, When information indicating transitions in cooking steps is associated with the time-series data, the learning unit learns the Hidden Markov Model to assign a state to each cooking step.

12. The learning device according to claim 11, wherein, When the number of states is set to be greater than the number of cooking steps, the learning unit learns the Hidden Markov Model to assign multiple states to any one of the cooking steps.

13. A presentation method performed by a presentation device, the presentation method comprising: The tolerance acquisition step for obtaining the deviation tolerance is estimated for each cooking step based on reference data indicating the cooking process, and the deviation tolerance indicates the degree to which the actual measured value of a parameter measured during cooking is permissible relative to the value of the parameter indicating the cooking process; as well as The presentation step presents the tolerance for deviation for each cooking step.

14. A presentation procedure for causing a presentation device to perform the following process: A tolerance acquisition process for obtaining a deviation tolerance, wherein the deviation tolerance is estimated for each cooking step based on reference data indicative of the cooking process, and the deviation tolerance indicates the degree to which the actual measured value of a parameter measured during cooking is permissible relative to the value of the parameter indicative of the cooking process; and The presentation process presents the tolerance for deviation for each cooking step.

Images