Model training method, oil temperature monitoring method, cooking control method and system

Abstract

The present disclosure provides a model training method, an oil temperature monitoring method, a cooking control method and system, which comprises: acquiring a plurality of target sample cooking images; identifying target sample cooking features associated with cooking in each target sample cooking image; based on a pre-constructed mapping relationship between different cooking features and infrared temperature measurement reflectivity, controlling the infrared temperature measurement device to work at a matched infrared temperature measurement reflectivity, and collecting sample temperature sequence features of an oil surface in the cooking device under different preset time windows; taking each group of target sample cooking features as input and the corresponding sample temperature sequence features as output, and training an oil temperature prediction model. In the present disclosure, by acquiring sample temperature sequence features under multiple time windows, the trend or characteristics of the dynamic change of the oil temperature of the oil surface are effectively extracted, ensuring the accuracy and reliability of the prediction results of the oil temperature prediction model, so as to improve the cooking experience of the user.

Description

Technical Field

[0001] This disclosure relates to the field of smart home appliance technology, and in particular to a model training method, an oil temperature monitoring method, a cooking control method and system. Background Technology

[0002] In cooking scenarios, by measuring the oil temperature in the cooking equipment and using smart recipes, users can be guided to cook more nutritiously and scientifically, resulting in more delicious dishes.

[0003] Currently, the oil temperature in cooking equipment is mainly obtained through the following two methods: (1) By placing a non-contact infrared sensor above the cooking area, the received infrared radiation is converted into temperature information to monitor the oil temperature; (2) Oil temperature is monitored by placing a thermocouple temperature probe in the center of the bottom of the cooking equipment to achieve pressure contact.

[0004] However, regardless of whether it is a non-contact or contact temperature measurement method, the accuracy of temperature measurement will vary with the reflectivity of the object being measured, resulting in low accuracy of oil temperature acquisition. In addition, the aforementioned contact temperature measurement method is also easily affected by the material and thickness of the cooking equipment, which can easily lead to inaccurate oil temperature monitoring. Summary of the Invention

[0005] The technical problem to be solved by this disclosure is to overcome the shortcomings of existing cooking equipment oil temperature monitoring methods, which all have low oil temperature monitoring accuracy, and to provide a solution.

[0006] This disclosure solves the above-mentioned technical problems through the following technical solution:

[0007] This disclosure provides a training method for an oil temperature prediction model. The training method includes:

[0008] Acquire several cooking images of the target sample;

[0009] Identify the target sample cooking features associated with cooking in each target sample cooking image;

[0010] The target sample cooking features include target sample equipment features that characterize the equipment parameters of the cooking equipment, and / or target sample oil usage features that characterize the oil usage in the cooking equipment.

[0011] Based on the pre-constructed mapping relationship between different cooking features and corresponding infrared thermometric reflectance, and the cooking features of the target sample, the infrared thermometric device set in the cooking scene is controlled to work with the matching infrared thermometric reflectance, so as to collect the sample temperature sequence features of the oil surface in the cooking device under different preset time windows.

[0012] The target sample cooking features of each group are used as input, and the corresponding sample temperature sequence features are used as output to train an oil temperature prediction model.

[0013] The oil temperature prediction model is used to output the predicted oil temperature sequence of the oil surface in the cooking device within a set time window.

[0014] Preferably, the step of acquiring the sample temperature sequence characteristics of the oil surface in the cooking device under different preset time windows includes:

[0015] Several temperature values ​​of the oil surface in the cooking device were collected under each preset time window;

[0016] Several temperature values ​​are processed using a preset calculation method to obtain the sample temperature sequence features under the corresponding preset time window;

[0017] The preset calculation method includes variance, average, average difference, maximum and minimum values, range, or a preset function constructed that is related to the temperature of the oil surface.

[0018] Preferably, before the step of identifying the target sample cooking features associated with cooking in each of the target sample cooking images, the method further includes:

[0019] Collect several sets of sample data;

[0020] Each set of sample data includes a first sample cooking image and a first sample cooking feature corresponding to the identified first sample cooking image;

[0021] The first sample cooking features include first sample equipment features characterizing the equipment parameters of the cooking equipment, and / or first sample oil usage features characterizing the oil usage in the cooking equipment;

[0022] The cooking image of the first sample in each group is used as input, and the corresponding cooking feature of the first sample is used as output to train a cooking feature recognition model.

[0023] The step of identifying the target sample cooking features associated with cooking in each target sample cooking image includes:

[0024] The target sample cooking image is input into the cooking feature recognition model to output the corresponding target sample cooking features;

[0025] or,

[0026] The step of identifying the target sample cooking features associated with cooking in each target sample cooking image includes:

[0027] Each target sample cooking image is labeled to obtain the corresponding sample cooking features.

[0028] Preferably, the step of training a cooking feature recognition model by taking the first sample cooking image of each group as input and the corresponding first sample cooking feature as output includes:

[0029] The first sample cooking image of each group is used as input, and the corresponding first sample cooking feature is used as output. The model is trained using a preset feature extraction network and a preset recognition and classification network to obtain the cooking feature recognition model.

[0030] And / or,

[0031] The step of training an oil temperature prediction model by taking the cooking features of the target samples in each group as input and the corresponding sample temperature sequence features as output includes:

[0032] The target sample cooking features of each group are used as input, and the corresponding sample temperature sequence features are used as output. The model is trained using a pre-defined supervised learning model to obtain the oil temperature prediction model.

[0033] The pre-defined supervised learning model includes KNN (Knowledge Neighbors), decision tree, Naive Bayes, support vector machine algorithm, or neural network.

[0034] This disclosure also provides a method for monitoring the oil temperature of a cooking device, the oil temperature monitoring method being implemented based on the training method of the oil temperature prediction model described above;

[0035] The oil temperature monitoring method includes:

[0036] Acquire the actual cooking image at the current acquisition moment;

[0037] Identify the actual cooking features in the actual cooking images;

[0038] The actual cooking characteristics include actual equipment characteristics that characterize the equipment parameters of the cooking equipment, and / or actual oil usage characteristics that characterize the oil usage in the cooking equipment;

[0039] The actual cooking characteristics are input into the oil temperature prediction model to output a predicted oil temperature sequence of the oil surface in the cooking device under a set future time window.

[0040] This disclosure also provides a cooking control method, which is based on the oil temperature monitoring method of the cooking equipment described above;

[0041] The cooking control method includes:

[0042] Based on the obtained predicted oil temperature sequence, a corresponding cooking control strategy is generated;

[0043] Cooking control is performed according to the cooking control strategy described above.

[0044] Preferably, the cooking control strategy includes at least one of the following: display of oil temperature change trends, warning signal for excessively high oil temperature, and cooking parameter adjustment method combined with intelligent recipe.

[0045] This disclosure also provides a training system for an oil temperature prediction model, the training system comprising:

[0046] The target sample image acquisition module is used to acquire several cooking images of target samples;

[0047] The target sample feature recognition module is used to identify the target sample cooking features associated with cooking in each target sample cooking image;

[0048] The target sample cooking features include target sample equipment features characterizing the equipment parameters of the cooking equipment, and / or target sample oil usage features characterizing the oil usage in the cooking equipment;

[0049] The sample temperature feature acquisition module is used to control the infrared thermometer set in the cooking scene to work with a matching infrared thermometer reflectance based on the pre-constructed mapping relationship between different cooking features and corresponding infrared thermometric reflectance, as well as the cooking features of the target sample, so as to collect the sample temperature sequence features of the oil surface in the cooking device under different preset time windows.

[0050] The oil temperature prediction model training module is used to train the oil temperature prediction model by taking the cooking features of the target samples of each group as input and the corresponding sample temperature sequence features as output.

[0051] The oil temperature prediction model is used to output the predicted oil temperature sequence of the oil surface in the cooking device within a set time window.

[0052] Preferably, the training system for the oil temperature prediction model further includes:

[0053] The sample data acquisition module is used to collect several sets of sample data;

[0054] Each set of sample data includes a first sample cooking image and a first sample cooking feature corresponding to the identified first sample cooking image;

[0055] The first sample cooking features include first sample equipment features characterizing the equipment parameters of the cooking equipment, and / or first sample oil usage features characterizing the oil usage in the cooking equipment;

[0056] The feature recognition model acquisition module is used to take the first sample cooking image of each group as input and the corresponding first sample cooking feature as output to train a cooking feature recognition model.

[0057] or,

[0058] The target sample feature recognition module is used to input the target sample cooking image into the cooking feature recognition model to output the corresponding target sample cooking features;

[0059] Preferably, the target sample feature recognition module is used to annotate each target sample cooking image to obtain the corresponding sample cooking features.

[0060] Preferably, the feature recognition model acquisition module is used to take the first sample cooking image of each group as input and the corresponding first sample cooking features as output, and train it using a preset feature extraction network and a preset recognition classification network to obtain the cooking feature recognition model.

[0061] Preferably, the oil temperature prediction model training module is used to take the cooking features of the target samples of each group as input and the corresponding sample temperature sequence features as output, and train the model using a preset supervised learning model to obtain the oil temperature prediction model.

[0062] The pre-defined supervised learning model includes KNN, decision tree, Naive Bayes, support vector machine algorithm, or neural network.

[0063] This disclosure also provides an oil temperature monitoring system for a cooking device, which is implemented based on the training system of the above-mentioned oil temperature prediction model;

[0064] The oil temperature monitoring system includes:

[0065] The actual image acquisition module is used to acquire the actual cooking image at the current acquisition moment;

[0066] The actual cooking feature recognition module is used to identify the actual cooking features in the actual cooking image;

[0067] The actual cooking characteristics include actual equipment characteristics that characterize the equipment parameters of the cooking equipment, and / or actual oil usage characteristics that characterize the oil usage in the cooking equipment;

[0068] The oil temperature prediction sequence acquisition module is used to input the actual cooking characteristics into the oil temperature prediction model to output the predicted oil temperature sequence of the oil surface in the cooking device under a set time window in the future.

[0069] This disclosure also provides a cooking control system, which is based on the oil temperature monitoring system of the cooking equipment described above;

[0070] The cooking control system includes:

[0071] The cooking control strategy generation module is used to generate corresponding cooking control strategies based on the obtained predicted oil temperature sequence.

[0072] A cooking control module is used to perform cooking control according to the cooking control strategy.

[0073] Preferably, the cooking control strategy includes at least one of the following: display of oil temperature change trends, warning signal for excessively high oil temperature, and cooking parameter adjustment method combined with intelligent recipe.

[0074] This disclosure also provides an intelligent cooking system, which includes the cooking control system described above.

[0075] This disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described oil temperature prediction model training method; or, implements the above-described cooking device oil temperature monitoring method; or, implements the above-described cooking control method.

[0076] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method for training an oil temperature prediction model; or, implements the above-described method for monitoring the oil temperature of a cooking device; or, implements the above-described cooking control method.

[0077] Based on common knowledge in the field, the preferred conditions described can be combined arbitrarily to obtain the preferred embodiments of this disclosure.

[0078] The positive and progressive effects of this disclosure are as follows:

[0079] In this disclosure, a cooking feature recognition model is pre-constructed to identify the equipment features (including but not limited to its location, size, material, and whether it is covered with a lid) of the cooking equipment and the oil usage features (including but not limited to color and amount of oil) of the cooking equipment. An oil temperature prediction model is also constructed to output the predicted oil temperature sequence of the oil surface in the cooking equipment under a set time window. By combining multi-dimensional data for comprehensive analysis, for any cooking image in an actual cooking scenario, the real-time oil temperature sequence of the oil surface in the cooking equipment under a set time window of 1 second, 2 seconds, 5 seconds, etc. can be directly, automatically, timely and with high precision output. This enables accurate and reliable monitoring of the oil temperature in the cooking equipment during dynamic cooking.

[0080] In addition, based on accurate monitoring of the oil temperature inside the cooking equipment, and in conjunction with intelligent recipes, the system can automatically, timely, and reasonably control and adjust the parameters of the cooking equipment to obtain healthy and convenient dishes. It can also predict the subsequent cooking trend and promptly remind the user before the temperature gets too high, ensuring safety during the cooking process and thus effectively improving the user's cooking experience. At the same time, it also enhances the overall product performance of the intelligent cooking system. Attached Figure Description

[0081] Figure 1 This is a flowchart of the training method for the oil temperature prediction model in Embodiment 1 of this disclosure.

[0082] Figure 2 This is a first flowchart of the training method for the oil temperature prediction model in Embodiment 2 of this disclosure.

[0083] Figure 3 This is a second flowchart of the training method for the oil temperature prediction model in Embodiment 3 of this disclosure.

[0084] Figure 4 This is a flowchart of the oil temperature monitoring method for the cooking equipment according to Embodiment 4 of this disclosure.

[0085] Figure 5 This is a flowchart of the cooking control method of Embodiment 5 of this disclosure.

[0086] Figure 6 This is a schematic diagram of the modules of the training system for the oil temperature prediction model of Embodiment 6 of this disclosure.

[0087] Figure 7 This is a schematic diagram of the training system for the oil temperature prediction model of Embodiment 7 of this disclosure.

[0088] Figure 8 This is a schematic diagram of the oil temperature monitoring system of the cooking equipment in Embodiment 8 of this disclosure.

[0089] Figure 9 This is a schematic diagram of the modules of the cooking control system of Embodiment 9 of this disclosure.

[0090] Figure 10 This is a schematic diagram of the structure of the electronic device according to Embodiment 10 of this disclosure. Detailed Implementation

[0091] The present disclosure is further illustrated below by way of embodiments, but the present disclosure is not limited to the scope of the embodiments described herein.

[0092] Example 1

[0093] The application scenario of this embodiment is a cooking scenario, in which infrared temperature measuring devices and cameras are set up. The cameras are used to collect cooking images from the corresponding collection angle of the cooking scenario, and the infrared temperature measuring devices are used to detect the oil surface temperature in the cooking equipment and the ambient temperature of the cooking scenario at a specific sampling frequency.

[0094] Specifically, infrared temperature measurement devices include, but are limited to, infrared temperature sensors, such as non-contact infrared temperature measurement modules that sample at 4 Hz to 8 Hz to simultaneously collect and output cooking environment temperature and oil surface temperature. The infrared temperature measurement device can detect surfaces with a viewing angle of 5° to 20°, and one or more infrared temperature measurement modules can be installed. The camera includes an image recognition module, and one or more image recognition modules with a resolution of 720p can be installed.

[0095] Preferably, the infrared temperature measuring device and the camera are installed at the same preset location, such as on the range hood and directly above the burner. Of course, the location, method, number, and model of the infrared temperature measuring device and camera can be redefined and adjusted according to the actual needs of the scenario.

[0096] like Figure 1 As shown, the training method for the oil temperature prediction model in this embodiment includes:

[0097] S101. Obtain several cooking images of the target samples;

[0098] S102. Identify the cooking features of the target sample associated with cooking in each target sample cooking image;

[0099] Among them, the target sample cooking features include target sample equipment features that characterize the equipment parameters of the cooking equipment, and / or target sample oil usage features that characterize the oil usage in the cooking equipment;

[0100] Specifically, the characteristics that characterize the parameters of cooking equipment include information such as the location, size, material, and whether a lid is on the cooking equipment; the characteristics that characterize the oil used in cooking equipment include color and amount of oil.

[0101] S103. Based on the pre-constructed mapping relationship between different cooking features and corresponding infrared thermometric reflectance, and the cooking features of the target sample, control the infrared thermometric device set in the cooking scene to work with the matching infrared thermometric reflectance.

[0102] Based on the interaction between several historical cooking characteristics and the infrared thermometric reflectance of infrared thermometers, a one-to-one correspondence between different cooking characteristics and infrared thermometric reflectance is established in advance to ensure the accuracy of sample oil temperature data collection, thereby ensuring the reliability of the oil temperature prediction model.

[0103] S104. Collect the sample temperature sequence characteristics of the oil surface in the cooking equipment under different preset time windows;

[0104] The preset time windows include 1 second, 2 seconds, 5 seconds, 10 seconds, 15 seconds, 30 seconds, and 60 seconds, which means that the sample temperature sequence features corresponding to multiple preset time windows are obtained.

[0105] S105. Take the cooking features of the target samples in each group as input and the corresponding sample temperature sequence features as output to train the oil temperature prediction model.

[0106] The oil temperature prediction model is used to output the predicted oil temperature sequence of the oil surface in the cooking equipment within a set time window.

[0107] In this embodiment, by identifying the equipment characteristics of cooking equipment in the image (including but not limited to its location, size, material, and whether it is covered with a lid), and the oil usage characteristics (including but not limited to color and amount of oil), the infrared thermometric reflectance of the infrared thermometric device is controlled based on the correspondence between cooking characteristics and infrared thermometric reflectance, ensuring the accuracy of obtaining the sample temperature sequence characteristics of the oil surface. Furthermore, by acquiring sample temperature sequence characteristics under multiple time windows, the trend or characteristics of the dynamic changes in oil temperature are effectively extracted, thereby ensuring the accuracy and reliability of the oil temperature prediction model's prediction results.

[0108] The above-described implementation scheme in this embodiment avoids the fact that the accuracy of existing non-contact or contact temperature measurement methods varies with the reflectivity of the object being measured. Specifically, it obtains the characteristics of the cooking equipment and the oil used by image recognition information, infers the accurate reflectivity of the object being measured, and then analyzes the temperature information.

[0109] In addition, it avoids the shortcomings of most current oil temperature monitoring methods, which rely on fixed formulas such as linear fitting. These methods are not very accurate for monitoring dynamically changing oil temperatures and cannot better predict the subsequent cooking oil temperature, thus failing to provide users with a better cooking experience. Specifically, by using a constructed oil temperature prediction model, it continuously, dynamically, and accurately monitors the changes in oil temperature in the cooking equipment, ensuring the accuracy of oil temperature acquisition and providing users with a good cooking experience.

[0110] Example 2

[0111] The training method for the oil temperature prediction model in this embodiment is a further improvement on Embodiment 1, specifically:

[0112] In a feasible solution, such as Figure 2 As shown, step S104 includes:

[0113] S1041. Collect several temperature values ​​of the oil surface in the cooking device under each preset time window;

[0114] S1042. Several temperature values ​​are processed using a preset calculation method to obtain the sample temperature sequence features under the corresponding preset time window.

[0115] The preset calculation methods include variance, mean, mean difference, maximum and minimum values, range, or a preset function relating to the oil surface temperature. Of course, other processing methods can also be used, as long as the corresponding sample temperature sequence features can be obtained, and these features can accurately characterize the oil surface temperature in the cooking equipment within the corresponding time window.

[0116] By processing several temperature values ​​collected within a preset time window (such as 15 seconds or 30 seconds), the sample temperature sequence characteristics under the corresponding 15-second time window and the sample temperature sequence characteristics under the corresponding 30-second time window can be obtained. Alternatively, several temperature values ​​collected within a continuous 45-second time window can be processed to obtain the sample temperature sequence characteristics under the corresponding 45-second time window.

[0117] In this scheme, a multi-time-window cooking oil temperature monitoring method is used to calculate feature values ​​and make predictions under multiple time windows, which ensures that the sample temperature sequence features are acquired in a timely and accurate manner, so as to achieve the effect of timely and accurate prediction of real-time oil temperature during dynamic cooking.

[0118] In one feasible embodiment, the method further includes the following steps prior to step S102:

[0119] S1001. Collect several sets of sample data;

[0120] Each set of sample data includes a first sample cooking image and a first sample cooking feature corresponding to the identified first sample cooking image;

[0121] The first sample cooking features include first sample equipment features characterizing the equipment parameters of the cooking equipment, and / or first sample oil usage features characterizing the oil usage in the cooking equipment;

[0122] S1002. Take the first sample cooking image of each group as input and the corresponding first sample cooking feature as output to train a cooking feature recognition model.

[0123] Step S102 includes:

[0124] S1021. Input the target sample cooking image into the cooking feature recognition model to output the corresponding target sample cooking features;

[0125] In this solution, a specialized cooking feature recognition model is pre-built to directly, promptly, and with high precision output the corresponding cooking features for any input target sample cooking image. This ensures the accuracy, reliability, and efficiency of the oil temperature prediction model training, thereby guaranteeing the precision and rationality of cooking control in actual cooking scenarios.

[0126] In one feasible embodiment, step S102 includes:

[0127] Each target sample cooking image is labeled to obtain the corresponding sample cooking features.

[0128] In this scheme, each target sample cooking image is processed by annotation, and corresponding label data is generated based on the annotation results to serve as the sample cooking features of each sample cooking image. This method does not require model training, but can still provide the necessary training data for the oil temperature prediction model.

[0129] In a feasible solution, such as Figure 3 As shown, step S1002 includes:

[0130] S10021. Take the first sample cooking image of each group as input and the corresponding first sample cooking features as output, and train it using a preset feature extraction network and a preset recognition and classification network to obtain a cooking feature recognition model.

[0131] Of course, other types of network models can be used to train cooking feature recognition models according to the actual needs of the scenario, which will not be elaborated here.

[0132] In this solution, a cooking feature recognition model is constructed by using a preset feature extraction network and a preset recognition and classification network, as well as a first sample cooking image and the matched first sample cooking features. Based on this cooking feature recognition model, the model outputs cooking equipment features such as the location, material, and whether there is a lid. It can also output oil features such as the color of the oil surface and the amount of oil, thus ensuring the accuracy and reliability of the cooking feature recognition model.

[0133] In one feasible embodiment, step S105 includes:

[0134] S1051. Take the cooking features of the target samples in each group as input and the corresponding sample temperature sequence features as output, and train the model using a pre-defined supervised learning model to obtain the oil temperature prediction model.

[0135] The pre-defined supervised learning models include KNN, decision tree, Naive Bayes, support vector machine algorithm, or neural network.

[0136] Of course, other types of network models can be used to train cooking feature recognition models according to the actual needs of the scenario, which will not be elaborated here.

[0137] In this scheme, an oil temperature prediction model is constructed by pre-setting a supervised learning model, as well as the cooking features of the target sample and the temperature sequence features of the matched sample, so as to ensure the accuracy and reliability of the prediction results of the oil temperature prediction model.

[0138] Example 3

[0139] The oil temperature monitoring method of the cooking equipment in this embodiment is implemented based on the training method of the oil temperature prediction model in the above embodiment.

[0140] like Figure 4 As shown, oil temperature monitoring methods include:

[0141] S201. Obtain the actual cooking image at the current acquisition time;

[0142] Actual cooking images from the perspective of the cooking scene are captured using a camera or image recognition module.

[0143] S202. Identify the actual cooking features in the actual cooking images;

[0144] Among them, actual cooking characteristics include actual equipment characteristics that characterize the equipment parameters of the cooking equipment, and / or actual oil usage characteristics that characterize the oil usage in the cooking equipment;

[0145] Specifically, the actual cooking image is input into the constructed cooking feature recognition model to directly output the corresponding actual cooking features; of course, the actual cooking features can also be obtained through annotation processing; preferably, the actual cooking features are obtained by inputting the cooking feature recognition model. Of course, the specific method used to obtain the actual cooking features in a real-world scenario can be determined based on actual needs.

[0146] S203. Input the actual cooking characteristics into the oil temperature prediction model to output the predicted oil temperature sequence of the oil surface in the cooking equipment under a set time window in the future.

[0147] The predicted oil temperature sequence within a future set time window allows users to know the real-time and dynamic changes in oil temperature, so as to better understand the subsequent cooking oil temperature status and facilitate timely intervention.

[0148] In this disclosure, a cooking feature recognition model is pre-constructed to identify the equipment characteristics (including but not limited to its location, size, material, and whether it is covered with a lid) of the cooking equipment and the oil usage characteristics (including but not limited to color and amount of oil) of the cooking equipment. An oil temperature prediction model is also constructed to output a predicted oil temperature sequence of the oil surface in the cooking equipment within a set time window. This combines multi-dimensional data for comprehensive analysis, enabling the automatic, timely, and highly accurate output of the oil temperature sequence of the oil surface in the cooking equipment within a set time window (1 second, 2 seconds, 5 seconds, etc.) for any cooking image in a real-world cooking scenario. This achieves accurate and reliable monitoring of the oil temperature in the cooking equipment during dynamic cooking, resulting in real-time oil temperature monitoring. This assists in subsequent cooking control and ensures a positive user cooking experience.

[0149] Example 4

[0150] The cooking control method in this embodiment is based on the oil temperature monitoring method of the cooking equipment described in the above embodiment.

[0151] like Figure 5 As shown, the cooking control method in this embodiment includes:

[0152] S301. Generate a corresponding cooking control strategy based on the obtained predicted oil temperature sequence;

[0153] S302. Perform cooking control according to the cooking control strategy.

[0154] In this solution, the predicted oil temperature sequence is monitored and analyzed in a timely manner to determine the matching cooking control strategy and execute the corresponding cooking adjustments in a timely manner. This avoids poor cooking results caused by abnormalities such as excessive oil temperature, and ensures timely and effective control of the entire cooking process.

[0155] In a feasible solution, the cooking control strategy includes displaying the trend of oil temperature changes, providing warning signals for excessively high oil temperatures, and adjusting cooking parameters in conjunction with smart recipes.

[0156] Specifically, based on the monitored and predicted oil temperature sequence, the system displays preset trends in oil temperature changes over different time windows in the future to the user interface, allowing users to anticipate and adjust the heat accordingly. Furthermore, it can generate timely alerts to users if the oil temperature becomes too high a certain time window after the current data collection point, and automatically reduce or shut down the cooking equipment to prevent unsafe situations, thus ensuring the safety of the cooking process. It can also combine intelligent recipes with the predicted real-time oil temperature sequence data to automatically control the cooking parameters of the equipment, maintaining a correct and healthy cooking curve to obtain healthy and convenient dishes. This meets the user's actual cooking needs, effectively improving the user's cooking experience and enhancing the overall performance of the intelligent cooking system.

[0157] Example 5

[0158] The application scenario of this embodiment is a cooking scenario, in which infrared temperature measuring devices and cameras are set up. The cameras are used to collect cooking images from the corresponding collection angle of the cooking scenario, and the infrared temperature measuring devices are used to detect the oil surface temperature in the cooking equipment and the ambient temperature of the cooking scenario at a specific sampling frequency.

[0159] Specifically, infrared temperature measurement devices include, but are limited to, infrared temperature sensors, such as non-contact infrared temperature measurement modules that sample at 4 Hz to 8 Hz to simultaneously collect and output cooking environment temperature and oil surface temperature. The infrared temperature measurement device can detect surfaces with a viewing angle of 5° to 20°, and one or more infrared temperature measurement modules can be installed. The camera includes an image recognition module, and one or more image recognition modules with a resolution of 720p can be installed.

[0160] Preferably, the infrared temperature measuring device and the camera are installed at the same preset location, such as on the range hood and directly above the burner. Of course, the location, method, number, and model of the infrared temperature measuring device and camera can be redefined and adjusted according to the actual needs of the scenario.

[0161] like Figure 6 As shown, the training system for the oil temperature prediction model in this embodiment includes:

[0162] Target sample image acquisition module 1 is used to acquire several cooking images of target samples;

[0163] The target sample feature recognition module 2 is used to identify the target sample cooking features associated with cooking in each target sample cooking image;

[0164] Among them, the target sample cooking features include target sample equipment features that characterize the equipment parameters of the cooking equipment, and / or target sample oil usage features that characterize the oil usage in the cooking equipment;

[0165] Specifically, the characteristics that characterize the parameters of cooking equipment include information such as the location, size, material, and whether a lid is on the cooking equipment; the characteristics that characterize the oil used in cooking equipment include color and amount of oil.

[0166] The sample temperature feature acquisition module 3 is used to control the infrared temperature measuring device set in the cooking scene to work with the matching infrared temperature measuring reflectance based on the pre-constructed mapping relationship between different cooking features and the corresponding infrared temperature reflectance, as well as the target sample cooking features, so as to collect the sample temperature sequence features of the oil surface in the cooking device under different preset time windows.

[0167] Based on the interaction between several historical cooking characteristics and the infrared thermometric reflectance of infrared thermometers, a one-to-one correspondence between different cooking characteristics and infrared thermometric reflectance is established in advance to ensure the accuracy of sample oil temperature data collection, thereby ensuring the reliability of the oil temperature prediction model.

[0168] The preset time windows include 1 second, 2 seconds, 5 seconds, 10 seconds, 15 seconds, 30 seconds, and 60 seconds, which means that the sample temperature sequence features corresponding to multiple preset time windows are obtained.

[0169] Oil temperature prediction model training module 4 is used to train an oil temperature prediction model by taking the cooking features of the target samples of each group as input and the corresponding sample temperature sequence features as output.

[0170] The oil temperature prediction model is used to output the predicted oil temperature sequence of the oil surface in the cooking equipment within a set time window.

[0171] In this embodiment, by identifying the equipment characteristics of cooking equipment in the image (including but not limited to its location, size, material, and whether it is covered with a lid), and the oil usage characteristics (including but not limited to color and amount of oil), the infrared thermometric reflectance of the infrared thermometric device is controlled based on the correspondence between cooking characteristics and infrared thermometric reflectance, ensuring the accuracy of obtaining the sample temperature sequence characteristics of the oil surface. Furthermore, by acquiring sample temperature sequence characteristics under multiple time windows, the trend or characteristics of the dynamic changes in oil temperature are effectively extracted, thereby ensuring the accuracy and reliability of the oil temperature prediction model's prediction results.

[0172] The above-described implementation scheme in this embodiment avoids the fact that the accuracy of existing non-contact or contact temperature measurement methods varies with the reflectivity of the object being measured. Specifically, it obtains the characteristics of the cooking equipment and the oil used by image recognition information, infers the accurate reflectivity of the object being measured, and then analyzes the temperature information.

[0173] In addition, it avoids the shortcomings of most current oil temperature monitoring methods, which rely on fixed formulas such as linear fitting. These methods are not very accurate for monitoring dynamically changing oil temperatures and cannot better predict the subsequent cooking oil temperature, thus failing to provide users with a better cooking experience. Specifically, by using a constructed oil temperature prediction model, it continuously, dynamically, and accurately monitors the changes in oil temperature in the cooking equipment, ensuring the accuracy of oil temperature acquisition and providing users with a good cooking experience.

[0174] Example 6

[0175] The training system for the oil temperature prediction model in this embodiment is a further improvement on Embodiment 5, specifically:

[0176] In a feasible solution, such as Figure 7 As shown, the sample temperature feature acquisition module 3 includes:

[0177] Temperature acquisition unit 5 is used to acquire several temperature values ​​of the oil surface in the cooking device under each preset time window;

[0178] The sample temperature feature acquisition unit 6 is used to process several temperature values ​​using a preset calculation method to obtain the sample temperature sequence features under the corresponding preset time window.

[0179] The preset calculation methods include variance, mean, mean difference, maximum and minimum values, range, or a preset function relating to the oil surface temperature. Of course, other processing methods can also be used, as long as the corresponding sample temperature sequence features can be obtained, and these features can accurately characterize the oil surface temperature in the cooking equipment within the corresponding time window.

[0180] By processing several temperature values ​​collected within a preset time window (such as 15 seconds or 30 seconds), the sample temperature sequence characteristics under the corresponding 15-second time window and the sample temperature sequence characteristics under the corresponding 30-second time window can be obtained. Alternatively, several temperature values ​​collected within a continuous 45-second time window can be processed to obtain the sample temperature sequence characteristics under the corresponding 45-second time window.

[0181] In this scheme, a multi-time-window cooking oil temperature monitoring method is used to calculate feature values ​​and make predictions under multiple time windows, which ensures that the sample temperature sequence features are acquired in a timely and accurate manner, so as to achieve the effect of timely and accurate prediction of real-time oil temperature during dynamic cooking.

[0182] In one feasible approach, the training system for the oil temperature prediction model also includes:

[0183] Sample data acquisition module 7 is used to collect several sets of sample data;

[0184] Each set of sample data includes a first sample cooking image and a first sample cooking feature corresponding to the identified first sample cooking image;

[0185] The first sample cooking features include first sample equipment features characterizing the equipment parameters of the cooking equipment, and / or first sample oil usage features characterizing the oil usage in the cooking equipment;

[0186] The feature recognition model acquisition module 8 is used to take the first sample cooking image of each group as input and the corresponding first sample cooking feature as output to train a cooking feature recognition model.

[0187] The target sample feature recognition module 2 is used to input the target sample cooking image into the cooking feature recognition model to output the corresponding target sample cooking features;

[0188] In this solution, a specialized cooking feature recognition model is pre-built to directly, promptly, and with high precision output the corresponding cooking features for any input target sample cooking image. This ensures the accuracy, reliability, and efficiency of the oil temperature prediction model training, thereby guaranteeing the precision and rationality of cooking control in actual cooking scenarios.

[0189] In one feasible solution, the target sample feature recognition module 2 is used to annotate each target sample cooking image to obtain the corresponding sample cooking features.

[0190] In this scheme, each target sample cooking image is processed by annotation, and corresponding label data is generated based on the annotation results to serve as the sample cooking features of each sample cooking image. This method does not require model training, but can still provide the necessary training data for the oil temperature prediction model.

[0191] In one feasible scheme, the feature recognition model acquisition module 8 is used to take the first sample cooking image of each group as input and the corresponding first sample cooking features as output, and train it using a preset feature extraction network and a preset recognition classification network to obtain a cooking feature recognition model.

[0192] Of course, other types of network models can be used to train cooking feature recognition models according to the actual needs of the scenario, which will not be elaborated here.

[0193] In this solution, a cooking feature recognition model is constructed by using a preset feature extraction network and a preset recognition and classification network, as well as a first sample cooking image and the matched first sample cooking features. Based on this cooking feature recognition model, the model outputs cooking equipment features such as the location, material, and whether there is a lid. It can also output oil features such as the color of the oil surface and the amount of oil, thus ensuring the accuracy and reliability of the cooking feature recognition model.

[0194] In one feasible solution, the oil temperature prediction model training module 4 is used to take the cooking features of each group of target samples as input and the corresponding sample temperature sequence features as output, and train the model using a pre-set supervised learning model to obtain the oil temperature prediction model.

[0195] The pre-defined supervised learning models include KNN, decision tree, Naive Bayes, support vector machine algorithm, or neural network.

[0196] Of course, other types of network models can be used to train cooking feature recognition models according to the actual needs of the scenario, which will not be elaborated here.

[0197] In this scheme, an oil temperature prediction model is constructed by pre-setting a supervised learning model, as well as the cooking features of the target sample and the temperature sequence features of the matched sample, so as to ensure the accuracy and reliability of the prediction results of the oil temperature prediction model.

[0198] Example 7

[0199] The oil temperature monitoring system of the cooking equipment in this embodiment is implemented based on the training system of the oil temperature prediction model in the above embodiment.

[0200] like Figure 8 As shown, the oil temperature monitoring system in this embodiment includes:

[0201] Actual image acquisition module 9 is used to acquire the actual cooking image at the current acquisition moment;

[0202] Actual cooking images from the perspective of the cooking scene are captured using a camera or image recognition module.

[0203] The actual cooking feature recognition module 10 is used to identify actual cooking features in actual cooking images;

[0204] Among them, actual cooking characteristics include actual equipment characteristics that characterize the equipment parameters of the cooking equipment, and / or actual oil usage characteristics that characterize the oil usage in the cooking equipment;

[0205] Specifically, the actual cooking image is input into the constructed cooking feature recognition model to directly output the corresponding actual cooking features; of course, the actual cooking features can also be obtained through annotation processing; preferably, the actual cooking features are obtained by inputting the cooking feature recognition model. Of course, the specific method used to obtain the actual cooking features in a real-world scenario can be determined based on actual needs.

[0206] The oil temperature prediction sequence acquisition module 11 is used to input actual cooking characteristics into the oil temperature prediction model to output the predicted oil temperature sequence of the oil surface in the cooking device under a set time window in the future.

[0207] The predicted oil temperature sequence within a future set time window allows users to know the real-time and dynamic changes in oil temperature, so as to better understand the subsequent cooking oil temperature status and facilitate timely intervention.

[0208] In this disclosure, a cooking feature recognition model is pre-constructed to identify the equipment characteristics (including but not limited to its location, size, material, and whether it is covered with a lid) of the cooking equipment and the oil usage characteristics (including but not limited to color and amount of oil) of the cooking equipment. An oil temperature prediction model is also constructed to output a predicted oil temperature sequence of the oil surface in the cooking equipment within a set time window. This combines multi-dimensional data for comprehensive analysis, enabling the automatic, timely, and highly accurate output of the oil temperature sequence of the oil surface in the cooking equipment within a set time window (1 second, 2 seconds, 5 seconds, etc.) for any cooking image in a real-world cooking scenario. This achieves accurate and reliable monitoring of the oil temperature in the cooking equipment during dynamic cooking, resulting in real-time oil temperature monitoring. This assists in subsequent cooking control and ensures a positive user cooking experience.

[0209] Example 8

[0210] The cooking control system in this embodiment is based on the oil temperature monitoring system of the cooking equipment described in the above embodiment.

[0211] like Figure 9 As shown, the cooking control system of this embodiment includes:

[0212] The cooking control strategy generation module 11 is used to generate a corresponding cooking control strategy based on the acquired predicted oil temperature sequence.

[0213] The cooking control module 12 is used to control the cooking process according to the cooking control strategy.

[0214] In this solution, the predicted oil temperature sequence is monitored and analyzed in a timely manner to determine the matching cooking control strategy and execute the corresponding cooking adjustments in a timely manner. This avoids poor cooking results caused by abnormalities such as excessive oil temperature, and ensures timely and effective control of the entire cooking process.

[0215] In a feasible solution, the cooking control strategy includes displaying the trend of oil temperature changes, providing warning signals for excessively high oil temperatures, and adjusting cooking parameters in conjunction with smart recipes.

[0216] In this embodiment, based on the monitored and predicted oil temperature sequence, the user interface is shown the preset trend of oil temperature changes at different time windows in the future, so that the user can know in advance and adjust the heat in time. Furthermore, if the oil temperature is too high a certain time window after the current acquisition time, a prompt message is generated for the user, and the cooking equipment is automatically reduced or turned off to prevent unsafe scenarios, thus ensuring the safety of the cooking process in a timely and effective manner. The intelligent recipe can also be combined with the predicted real-time oil temperature sequence data to automatically control the cooking parameters of the cooking equipment to maintain a correct and healthy cooking curve, resulting in healthy and convenient dishes. This meets the user's actual cooking needs, effectively improving the user's cooking experience and enhancing the overall performance of the intelligent cooking system.

[0217] Example 9

[0218] The intelligent cooking system in this embodiment includes the cooking control system in embodiment 8.

[0219] Intelligent cooking systems include, but are not limited to, kitchen cooking systems such as range hoods and cooktops.

[0220] The intelligent cooking system in this embodiment integrates the aforementioned cooking control system, which can perform timely analysis based on the monitored and predicted oil temperature sequence to determine the matching cooking control strategy and execute corresponding cooking adjustments in a timely manner. This avoids poor cooking results caused by abnormalities such as excessive oil temperature, ensuring timely and effective control of the entire cooking process and improving the overall product performance of the intelligent cooking system.

[0221] Example 10

[0222] Figure 10 This is a schematic diagram of the structure of an electronic device according to Embodiment 5 of this disclosure. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the methods described in the above embodiments. Figure 10 The electronic device 30 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0223] like Figure 10As shown, the electronic device 30 can be represented in the form of a general computing device, such as a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including memory 32 and processor 31).

[0224] Bus 33 includes a data bus, an address bus, and a control bus.

[0225] The memory 32 may include volatile memory, such as random access memory (RAM) 321 and / or cache memory 322, and may further include read-only memory (ROM) 323.

[0226] The memory 32 may also include a program / utility 325 having a set (at least one) of program modules 324, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0227] The processor 31 performs various functional applications and data processing, such as the methods described in the above embodiments of this disclosure, by running computer programs stored in the memory 32.

[0228] Electronic device 30 can also communicate with one or more external devices 34 (e.g., keyboard, pointing device, etc.). This communication can be performed via input / output (I / O) interface 35. Furthermore, the model-generating device 30 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 36. Figure 10 As shown, network adapter 36 communicates with other modules of the model-generated device 30 via bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the model-generated device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0229] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0230] Example 11

[0231] This embodiment provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps in the method described above.

[0232] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0233] In possible implementations, this disclosure can also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the methods described above.

[0234] The program code for executing this disclosure can be written using any combination of one or more programming languages. The program code can be executed entirely on a user device, partially on a user device, as a standalone software package, partially on a user device and partially on a remote device, or entirely on a remote device.

[0235] While specific embodiments of this disclosure have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of this disclosure is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of this disclosure, but all such changes and modifications fall within the scope of protection of this disclosure.

Claims

1. A training method for an oil temperature prediction model, characterized in that, The training method comprises: acquiring a plurality of target sample cooking images; identifying target sample cooking features associated with cooking in each of the target sample cooking images; wherein the target sample cooking features comprise target sample device features representing device parameters of a cooking device, and / or target sample oil features representing oil usage in the cooking device; based on a pre-constructed mapping relationship between different cooking features and corresponding infrared temperature measurement reflectivity, and the target sample cooking features, controlling an infrared temperature measurement device arranged in a cooking scene to work at a matching infrared temperature measurement reflectivity to collect sample temperature sequence features of an oil surface in the cooking device under different preset time windows; training an oil temperature prediction model by taking each group of the target sample cooking features as input and corresponding sample temperature sequence features as output; wherein the oil temperature prediction model is used to output a predicted oil temperature sequence of the oil surface in the cooking device under a set time window; the step of collecting sample temperature sequence features of an oil surface in the cooking device under different preset time windows comprises: collecting a plurality of temperature values of the oil surface in the cooking device under each of the preset time windows; processing the plurality of temperature values using a pre-designed calculation method to obtain the sample temperature sequence features under the corresponding preset time window; wherein the pre-designed calculation method comprises a variance value, an average value, an average difference, a maximum value, a range, or a pre-set function constructed in association with the temperature of the oil surface.

2. The method of claim 1, wherein, Before the step of identifying target sample cooking features associated with cooking in each of the target sample cooking images, the method further comprises: collecting a plurality of sample data; wherein each group of sample data comprises a first sample cooking image and a first sample cooking feature identified from the first sample cooking image; the first sample cooking feature comprises a first sample device feature representing device parameters of the cooking device, and / or a first sample oil feature representing oil usage in the cooking device; training a cooking feature recognition model by taking each group of the first sample cooking image as input and corresponding first sample cooking feature as output; the step of identifying target sample cooking features associated with cooking in each of the target sample cooking images comprises: inputting the target sample cooking image into the cooking feature recognition model to output corresponding target sample cooking features; or, the step of identifying target sample cooking features associated with cooking in each of the target sample cooking images comprises: annotating each of the target sample cooking images to obtain corresponding sample cooking features.

3. The method of claim 2, wherein the oil temperature prediction model is trained based on the oil temperature data and the engine operation data. the step of training a cooking feature recognition model by taking each group of the first sample cooking image as input and corresponding first sample cooking feature as output comprises: training the cooking feature recognition model by taking each group of the first sample cooking image as input and corresponding first sample cooking feature as output using a pre-set feature extraction network and a pre-set recognition and classification network; and / or, The step of training the oil temperature prediction model by taking the target sample cooking features of each group as input and the corresponding sample temperature sequence features as output comprises: training the oil temperature prediction model by taking the target sample cooking features of each group as input and the corresponding sample temperature sequence features as output, and using a preset supervised learning model. The preset supervised learning model comprises KNN, decision tree, naive Bayes, support vector machine algorithm or neural network.

4. An oil temperature monitoring method of a cooking apparatus, characterized by, The oil temperature monitoring method is implemented based on the training method of the oil temperature prediction model in any one of claims 1-3. The oil temperature monitoring method comprises: acquiring an actual cooking image at a current acquisition time; identifying actual cooking features in the actual cooking image; The actual cooking features comprise actual device features representing device parameters of the cooking device and / or actual oil use features representing oil use in the cooking device. The actual cooking features are input into the oil temperature prediction model to output a predicted oil temperature sequence of an oil surface in the cooking device in a future set time window.

5. A cooking control method characterized by, The cooking control method is implemented based on the oil temperature monitoring method of the cooking device in claim 4. The cooking control method comprises: generating a corresponding cooking control strategy according to the acquired predicted oil temperature sequence; and controlling cooking according to the cooking control strategy.

6. The cooking control method according to claim 5, wherein The cooking control strategy comprises at least one of a display result of an oil temperature change trend, an oil temperature overhigh early warning signal and a cooking parameter adjustment mode combined with an intelligent menu.

7. A training system for an oil temperature prediction model, characterized in that, The training system is used to implement the training method of the oil temperature prediction model in any one of claims 1-3, and comprises: a target sample image acquisition module configured to acquire a plurality of target sample cooking images; a target sample feature identification module configured to identify target sample cooking features associated with cooking in each of the target sample cooking images; The target sample cooking features comprise target sample device features representing device parameters of a cooking device and / or target sample oil use features representing oil use in the cooking device. A sample temperature feature acquisition module is configured to control an infrared temperature measurement device arranged in a cooking scene to work at a matched infrared temperature measurement reflectivity based on a pre-constructed mapping relationship between different cooking features and corresponding infrared temperature measurement reflectivities and the target sample cooking features, so as to acquire sample temperature sequence features of an oil surface in the cooking device in different preset time windows. An oil temperature prediction model training module is configured to train an oil temperature prediction model by taking the target sample cooking features of each group as input and the corresponding sample temperature sequence features as output. The oil temperature prediction model is used to output a predicted oil temperature sequence of an oil surface in the cooking device in a set time window. The sample temperature feature acquisition module comprises: a temperature acquisition unit configured to acquire a plurality of temperature values of the oil surface in the cooking device in each of the preset time windows. The sample temperature feature acquisition unit is configured to process the temperature values in a pre-designed calculation manner to obtain the sample temperature sequence feature in the preset time window. The pre-designed calculation manner includes a variance value, an average value, an average difference, a maximum value, a range, or a preset function associated with the temperature of the oil surface.

8. An oil temperature monitoring system of a cooking apparatus, characterized by, The oil temperature monitoring system is implemented based on the training system of the oil temperature prediction model in claim 7. The oil temperature monitoring system includes: An actual image acquisition module configured to acquire an actual cooking image at a current acquisition time; An actual cooking feature identification module configured to identify an actual cooking feature in the actual cooking image; The actual cooking feature includes an actual device feature representing a device parameter of the cooking device and / or an actual oil use feature representing an oil use condition in the cooking device. A predicted oil temperature sequence acquisition module configured to input the actual cooking feature into the oil temperature prediction model to output a predicted oil temperature sequence of an oil surface in the cooking device in a future set time window.

9. A cooking control system characterized by, The cooking control system is implemented based on the oil temperature monitoring system of the cooking device in claim 8. The cooking control system includes: A cooking control strategy generation module configured to generate a corresponding cooking control strategy according to the acquired predicted oil temperature sequence; A cooking control module configured to perform cooking control according to the cooking control strategy.

10. An intelligent cooking system, characterized by, The intelligent cooking system includes the cooking control system in claim 9.

11. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the training method of the oil temperature prediction model in any one of claims 1 to 3; or, implement the oil temperature monitoring method of the cooking device in claim 4; or, implement the cooking control method in claim 5 or 6.

12. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the training method of the oil temperature prediction model in any one of claims 1 to 3; or, implement the oil temperature monitoring method of the cooking device in claim 4; or, implement the cooking control method in claim 5 or 6.

Images