Automatic cooking method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application relate to an automatic cooking method and device, electronic equipment and storage medium, the method comprising: obtaining a first food image of a food material to be cooked; using a preset food material detection model to detect the type of food material in the first food image and determine the attributes of the food material; generating an initial cooking scheme based on the attributes of the food material; obtaining dietary habit data of a target user; using a preset dietary habit model to analyze the dietary habit data and determine the dietary habit type of the target user; adjusting the parameters of the initial cooking scheme based on the dietary habit type to obtain a target cooking scheme that matches the dietary habit type; and controlling a cooking device to cook the food material based on the target cooking scheme. Embodiments of the present application automatically detect the type of food material to be cooked and automatically generate a cooking scheme that matches the user's dietary habits, which is more suitable for the user's individual needs and improves cooking efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent household appliances, and in particular to an automatic cooking method and device, an electronic device, and a computer readable storage medium. BACKGROUND

[0002] With the improvement of people's living standards, the demand for food cooking diversity and personalization is increasing. In the field of cooking, intelligent cooking equipment such as steam ovens and air fryers are important equipment in the home kitchen, and the degree of intelligence directly affects the user's cooking experience. However, traditional cooking equipment, although providing automatic cooking functions, often lacks the ability to intelligently adjust according to the user's personal eating habits, and the flexibility and pertinence of the cooking method are poor, requiring user manual intervention in the cooking process, and the cooking efficiency is low. SUMMARY

[0003] In view of this, in order to solve the above-mentioned part or all technical problems, the embodiments of the present application provide an automatic cooking method, device, electronic device and computer readable storage medium.

[0004] In a first aspect, the embodiments of the present application provide an automatic cooking method, which comprises: acquiring a first food material image photographed for food material to be cooked; using a preset food material detection model to detect the type of food material for the first food material image, and determining the attribute of the food material; generating an initial cooking scheme based on the attribute of the food material; acquiring eating habit data of a target user; using a preset eating habit model to analyze the eating habit data and determine the eating habit type of the target user; adjusting the parameters of the initial cooking scheme based on the eating habit type to obtain a target cooking scheme matching the eating habit type; and controlling the cooking equipment to cook the food material based on the target cooking scheme.

[0005] In one possible implementation, based on the target cooking scheme, the cooking equipment is controlled to cook the food material, which comprises: acquiring current environmental condition information in the cooking equipment during the cooking process, and determining a first difference between expected environmental condition information corresponding to the target cooking scheme and the current environmental condition information; and / or, acquiring a second food material image photographed for the food material during the cooking process, detecting the state of the food material for the second food material image to obtain current food material state information; determining a second difference between expected food material state information corresponding to the target cooking scheme and the current food material state information; adjusting the parameters of the target cooking scheme based on the first difference and / or the second difference, and controlling the cooking equipment to cook the food material based on the adjusted target cooking scheme.

[0006] In a possible implementation, after the cooking device is controlled to cook the food material based on the target cooking scheme, the method further includes: in response to the end of the cooking process, obtaining evaluation information of the cooked food material; adjusting parameters of the target cooking scheme based on the evaluation information, and storing the adjusted target cooking scheme.

[0007] In a possible implementation, the obtaining of the evaluation information of the cooked food material includes: in response to the end of the cooking process, obtaining a food material image of the food material after the cooking process as a third food material image; performing food material state detection on the third food material image to obtain finished food material state information; determining a difference between the expected food material state information corresponding to the parameters of the target cooking scheme and the finished food material state information; generating first evaluation information of the cooked food material based on the difference; and / or, obtaining second evaluation information input by feedback on the food material after the cooking process; generating the evaluation information of the cooked food material based on the first evaluation information and / or the second evaluation information.

[0008] In a possible implementation, the eating habit model is a deep reinforcement model; after the evaluation information of the cooked food material is obtained, the method further includes: performing reward and punishment score calculation on the evaluation information based on a preset reward and punishment function to obtain a reward and punishment score representing a difference between an eating habit type output by the eating habit model and an actual eating habit of the target user; and adjusting parameters of the eating habit model based on the reward and punishment score and according to a deep reinforcement learning algorithm.

[0009] In a possible implementation, after the initial cooking scheme is generated based on the attributes of the food material, the method further includes: obtaining attributes of currently stored food materials from a food material storage device; generating a to-be-determined cooking scheme based on the attributes of the food material to be cooked and the attributes of the currently stored food materials, and outputting the to-be-determined cooking scheme; and in response to receiving confirmation information confirming the to-be-determined cooking scheme, updating the initial cooking scheme based on the to-be-determined cooking scheme.

[0010] In a possible implementation, after the parameters of the initial cooking scheme are adjusted based on the eating habit type to obtain the target cooking scheme matching the eating habit type, the method further includes: obtaining body state information of the target user from a user body state detection device; and adjusting the parameters of the target cooking scheme based on the body state information, so that the adjusted target cooking scheme matches the body state information.

[0011] In a second aspect, the embodiments of the present application provide an automatic cooking device, comprising: a first obtaining module configured to obtain a first food material image photographed for a food material to be cooked; a detecting module configured to perform food material type detection on the first food material image by using a preset food material detection model to determine an attribute of the food material; a first generating module configured to generate an initial cooking scheme based on the attribute of the food material; a second obtaining module configured to obtain dietary habit data of a target user; an analyzing module configured to analyze the dietary habit data by using a preset dietary habit model to determine a dietary habit type of the target user; a first adjusting module configured to adjust parameters of the initial cooking scheme based on the dietary habit type to obtain a target cooking scheme matching the dietary habit type; and a control module configured to control a cooking device to cook the food material based on the target cooking scheme.

[0012] In a third aspect, the embodiments of the present application provide an electronic device, comprising: a memory configured to store a computer program; and a processor configured to execute the computer program stored in the memory, and when the computer program is executed, any embodiment of the automatic cooking method of the first aspect of the present application is implemented.

[0013] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method of any embodiment of the automatic cooking method of the first aspect described above is implemented.

[0014] In a fifth aspect, the embodiments of the present application provide a computer program, the computer program comprising computer readable code which, when run on a device, causes a processor in the device to implement the method of any embodiment of the automatic cooking method of the first aspect described above.

[0015] The automatic cooking method, device, electronic device and computer readable storage medium provided by the embodiments of the present application, by using the food material detection model to perform food material type detection on the photographed first food material image to determine the attribute of the food material, generating the initial cooking scheme based on the attribute of the food material, then using the dietary habit model to analyze the dietary habit data of the target user to determine the dietary habit type of the target user, adjusting the parameters of the initial cooking scheme based on the dietary habit type to obtain the target cooking scheme matching the dietary habit, and finally controlling the cooking device to cook the food material based on the target cooking scheme. The embodiments of the present application realize automatic detection of the food material type to be cooked, and automatically generate the cooking scheme matching the dietary habit of the user, so that the cooked dishes are more suitable for the personalized needs of the user, the number of times of user intervention in the cooking process is reduced, and the cooking efficiency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the application and, together with the description, further serve to explain the principles of the application.

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, those skilled in the art can obtain other drawings from these drawings without any creative effort.

[0018] One or more embodiments are illustrated by way of example in the drawings that are not necessarily drawn to scale, and which will be described in detail herein, with the understanding that these embodiments and / or examples represent non-limiting examples or aspects of the embodiments illustrated and described herein, as defined by the appended claims. Like reference numbers in the figures indicate like components, unless otherwise specified. The figures in the drawings are not necessarily to scale, and the sizes of some features have been exaggerated relative to others for clarity.

[0019] Figure 1 A flowchart of an automatic cooking method provided for an embodiment of the present application;

[0020] Figure 2 An exemplary structural diagram of a food material detection model provided for an embodiment of the present application;

[0021] Figure 3 A flowchart of another automatic cooking method provided for an embodiment of the present application;

[0022] Figure 4 A flowchart of yet another automatic cooking method provided for an embodiment of the present application;

[0023] Figure 5 A flowchart of yet another automatic cooking method provided for an embodiment of the present application;

[0024] Figure 6 A flowchart of yet another automatic cooking method provided for an embodiment of the present application;

[0025] Figure 7 A flowchart of yet another automatic cooking method provided for an embodiment of the present application;

[0026] Figure 8 A flowchart of yet another automatic cooking method provided for an embodiment of the present application;

[0027] Figure 9 A structural diagram of an automatic cooking device provided for an embodiment of the present application;

[0028] Figure 10 A structural diagram of an electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0029] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only part of the embodiments of the present application and are not intended to limit the present application in any way. It should be noted that the relative arrangement of the components and steps, numerical expressions, and numerical values set forth in these embodiments are not limitations on the scope of the present application, unless otherwise specifically stated.

[0030] Those skilled in the art can understand that the terms "first", "second", etc. in the embodiments of the present application are only used to distinguish different steps, devices or modules, and do not represent any specific technical meaning, nor represent the logical order between them.

[0031] It should also be understood that in the present embodiments, "a plurality of" can mean two or more, and "at least one" can mean one, two or more.

[0032] It should also be understood that for any component, data or structure mentioned in the embodiments of the present application, unless specifically limited or contrary indications are given in the context, it can be understood as one or more in general.

[0033] In addition, the term "and / or" in the present application is only a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can represent the existence of A alone, the existence of A and B together, and the existence of B alone. In addition, the character " / " in the present application generally represents an "or" relationship between the associated objects before and after it.

[0034] It should also be understood that the description of various embodiments of the present application focuses on the differences between the embodiments, and the same or similar parts can be referred to each other, and for the sake of brevity, will not be repeated.

[0035] The following description of at least one example embodiment is merely illustrative in nature and is in no way limiting on the application or its use.

[0036] Techniques, circuitry, and devices known to those of ordinary skill in the relevant art can not be discussed in detail herein, but should be considered part of the specification as appropriate.

[0037] It should be noted that similar reference numbers and letters in the following drawings represent similar items, so once an item is defined in one drawing, it does not need to be discussed further in subsequent drawings.

[0038] It should be noted that the embodiments and features in the present application can be combined with each other in the case of no conflict. In order to facilitate the understanding of the embodiments of the present application, the present application will be described in detail below with reference to the drawings and in combination with the embodiments. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0039] In order to solve the technical problem that the prior art cannot provide personalized cooking schemes for the user's eating habits, the present application provides an automatic cooking method, which can combine food material automatic detection and user eating habits to efficiently and specifically provide personalized cooking schemes for the user, thereby improving the cooking efficiency.

[0040] Figure 1 A flowchart of an automatic cooking method provided by the embodiments of the present application is shown. The method can be applied to various types of intelligent cooking devices such as steam ovens, air fryers, etc., can be executed by a processor included in the intelligent cooking device, and can also be executed by one or more electronic devices such as smartphones, notebook computers, desktop computers, servers, etc. in communication connection with the intelligent cooking device. In addition, the execution subject of the method can be hardware or software. When the execution subject is hardware, the execution subject can be one or more of the above-mentioned electronic devices. For example, a single electronic device can execute the method, or multiple electronic devices can cooperate with each other to execute the method. When the execution subject is software, the method can be implemented as multiple software or software modules, or as a single software or software module. No specific limitation is made herein.

[0041] As shown in the method, the method specifically includes: Figure 1

[0042] Step 101: obtaining a first food material image photographed for food material to be cooked.

[0043] In the present embodiment, the food material to be cooked is the food material placed in the cooking device, or the food material to be placed in the cooking device. For example, a camera can be provided on the cooking device, and after the food material is placed in the cooking device, the camera will photograph the food material to obtain the first food material image. For another example, the user can photograph the food material to be cooked by using a mobile phone or other device, and send the photographed first food material image to the cooking device or other electronic device for executing the method.

[0044] Step 102: using a preset food material detection model to detect the type of the first food material image, and determining the attribute of the food material.

[0045] ​In the present embodiment, the food material detection model can be a target detection model constructed based on various structures of neural networks, such as YOLO, RPN (Region Proposal Networks), etc. The food material detection model can be obtained by pre-training through a machine learning method, i.e., using sample food material images and labeled food material positions, categories, freshness, etc. attributes, adjusting the parameters of the initial model, stopping training when the initial model meets the training end condition, and obtaining the food material detection model. The food material detection model can determine the positions, sizes, types, freshness, etc. of various food materials from the input food material images, which can be used as the attributes of the food materials.

[0046] As an example, the food material detection model can be obtained based on a YOLO model. As shown in FIG. 1, the YOLO model includes: Figure 2

[0047] A preprocessing module, the main functions of which include data augmentation, adaptive anchor box calculation, and adaptive image scaling, etc.

[0048] A backbone network for converting the input image to a channel dimension, expanding the number of channels, and improving the running efficiency of the model. It can also include a lightweight attention module CBAM for adaptive feature refinement.

[0049] A neck network for realizing data bottom layer and high layer information fusion, making full use of the bottom layer features of the target, and obtaining multi-scale fusion features.

[0050] A head network including a dense prediction unit and a sparse prediction unit, wherein the dense prediction obtains the target frame meeting the requirements through positive and negative sample matching strategy and loss calculation, and the sparse prediction obtains the best target frame by suppressing redundant frames through NMS (Non-maximum Suppression). In the loss calculation, the confidence loss is also given different weights in different prediction units (such as including detection frame prediction unit, category prediction unit, etc.). In addition, a small target detection head (tiny) can be added to improve the detection accuracy of small targets.

[0051] Step 103, generating an initial cooking scheme based on the attributes of the food materials.

[0052] ​In this embodiment, the properties of various food materials can be matched with the preset cooking scheme library, and the cooking scheme with the highest matching degree with the detected food material properties can be selected as the initial cooking scheme. As an example, the matching process can be similarity calculation between the food material types, the amount of each type of food material, and the like included in the identified properties, and the food material types, the amount, and the like corresponding to each cooking scheme in the cooking scheme library, to determine the cooking scheme with the greatest similarity as the initial cooking scheme.

[0053] In step 104, the dietary habit data of the target user is obtained.

[0054] In this embodiment, the target user is the user to which the current cooking operation is directed. Generally, the correspondence between the cooking operation and the target user can be established by inputting the account information of the target user. The dietary habit data of the target user can be preset, and the electronic device executing the method can obtain the dietary habit data corresponding to the target user from a storage area. Alternatively, the dietary habit data can also be input by the target user on site. As an example, the dietary habit data can include but is not limited to at least one of the following: food type preference, taste, intake amount, eating time, etc.

[0055] In step 105, the dietary habit data is analyzed using a preset dietary habit model to determine the dietary habit type of the target user.

[0056] In this embodiment, the dietary habit model can be a software module of various types representing the correspondence between the dietary habit data and the dietary habit type. For example, the dietary habit model can be a neural network model trained in advance by a machine learning method, or a table representing the correspondence between the dietary habit data and the dietary habit type statistically in advance.

[0057] As an example, the dietary habit data can include information such as the type of food material, the amount, the cooking time, and the cooking frequency recorded by the target user at each cooking time in advance, and can also include information such as the preference of the target user for the type of food material and the preference of the target user for the taste. The electronic device can preprocess the information such as data deduplication, data enhancement, and data normalization to obtain data that meets the model processing standard. Then, the dietary habit model can be used to extract features from the input data, classify the features, and obtain the dietary habit type.

[0058] In step 106, the parameters of the initial cooking scheme are adjusted based on the dietary habit type to obtain a target cooking scheme matching the dietary habit type.

[0059] In the embodiment, the correspondence between the eating habit type and the parameter of the cooking scheme can be established in advance, and the correspondence can be represented in the form of a table or a calculation formula. As an example, if the eating habit type is to prefer soft food, the time parameter of the initial cooking scheme can be adjusted to increase the cooking time; if the eating habit type is to prefer hot dishes, the heating temperature can be increased.

[0060] Optionally, the cooking device can include a device (for example, including a discharge box, a seasoning outlet driven by a controlled motor, etc.) for automatically adding cooking ingredients (for example, water, soy sauce, vinegar, salt, etc.). According to the eating habit type, the addition amount of various ingredients of the initial cooking scheme can be adjusted. For example, if the eating habit type is to prefer spicy or salty, the motor can be automatically controlled to drive the seasoning outlet to add a corresponding amount of chili or salt to the food being cooked.

[0061] In step 107, the cooking device is controlled to cook the food material based on the target cooking scheme.

[0062] In the embodiment, the electronic device executing the method can control the heating device, the stirring device, the ventilation device, the ingredient adding device, etc. of the cooking device according to the control parameter corresponding to the target cooking scheme, perform corresponding actions according to the set cooking time, cooking temperature, etc., and complete the cooking process.

[0063] The automatic cooking method provided in the embodiment can detect the type of the food material by using the food material detection model to detect the food material type of the first food material image, determine the attribute of the food material, generate an initial cooking scheme based on the attribute of the food material, analyze the eating habit data of the target user by using the eating habit model, determine the eating habit type of the target user, adjust the parameters of the initial cooking scheme based on the eating habit type, obtain a target cooking scheme matched with the eating habit, and finally control the cooking device to cook the food material based on the target cooking scheme. The embodiment realizes automatic detection of the type of the food material to be cooked, and automatically generates a cooking scheme matched with the eating habit of the user in combination with the eating habit of the user, so that the cooked dish is more suitable for the personalized needs of the user, the number of times of user intervention in the cooking process is reduced, and the cooking efficiency is improved.

[0064] In some optional implementation manners of the embodiment, as shown in Figure 3 Step 107 includes:

[0065] In step 1071, current environmental condition information in the cooking device during the cooking process is obtained, and a first difference between the expected environmental condition information corresponding to the target cooking scheme and the current environmental condition information is determined.

[0066] The current environment condition information can be collected by various sensors in the cooking device. For example, the current environment condition information can include at least one of temperature, humidity, air volume, and the like. The expected environment condition information is the set environment condition information corresponding to the target cooking scheme. Different cooking stages can correspond to different expected environment condition information, so that the difference between the current environment condition information and the corresponding expected environment condition information can be detected in real time in different cooking stages. The difference can include the difference between state information in different dimensions.

[0067] In step 1072, a second food material image of the food material during cooking is acquired, and food material state detection is performed on the second food material image to obtain current food material state information.

[0068] The second food material image is a food material image collected by a camera on the cooking device in real time. The current food material state information can include state information in various dimensions representing the color, luster, and line shape of the food material. The food material state detection can be performed by a pre-trained food material state detection model. That is, food material images in different states are collected in advance as sample images, and the food material states corresponding to the sample images are labeled. The food material state detection model can include multiple convolution layers, pooling layers, fully connected layers, and the like. The color, shape, luster, and the like of the input food material image are extracted, and the features are classified by a classifier to obtain predicted food material state information. By comparing the error between the labeled food material state information and the predicted food material state information, the parameters of the food material state detection model are adjusted. When a training end condition is met, the training is ended, and the food material state detection model is obtained.

[0069] The trained food material state detection model can extract the color, shape, luster, and the like of the second food material image, classify the features, and obtain the current food material state information.

[0070] In step 1073, a second difference between the expected food material state information corresponding to the target cooking scheme and the current food material state information is determined.

[0071] The expected food material state information can be preset state information corresponding to the target cooking scheme, i.e., the ideal finished food material state. The difference between the expected food material state information and the current food material state information can include the difference between state information in different dimensions.

[0072] In step 1074, based on the first difference and / or the second difference, the parameters of the target cooking scheme are adjusted, and the cooking device is controlled to cook the food material based on the adjusted target cooking scheme.

[0073] The step is performed by using at least one of the first difference and the second difference. When the first difference is used, the steps 1072-1073 can not be performed. When the second difference is used, the step 1071 can not be performed.

[0074] The differences in different dimensions have preset corresponding relationships with a parameter of the target cooking scheme, so that the parameter can be adjusted to change the difference. For example, when the first difference includes a temperature difference, the temperature of the cooking device can be adjusted. When the second difference indicates that the food material state is uncooked, the cooking time can be extended to reduce the second difference.

[0075] The embodiment realizes automatic adjustment of the cooking scheme during the cooking process by detecting the current environmental condition information and the food material state information in the cooking device in real time during the cooking process, so that the difference between the food material state after cooking and the ideal state can be reduced, the number of manual interventions in the cooking process can be reduced, and the cooking quality and efficiency can be improved.

[0076] In some optional implementation manners of the embodiment, as shown in Figure 4 After the step 107, the method further includes:

[0077] In response to the end of the cooking process, the evaluation information of the cooked food material is obtained.

[0078] The evaluation information can be information input by the user or information generated after automatic detection of the food material.

[0079] Based on the evaluation information, the parameter of the target cooking scheme is adjusted, and the adjusted target cooking scheme is stored.

[0080] The evaluation information can represent the difference in different dimensions (such as color, luster, and maturity) between the state of the cooked food material and the ideal state. According to the corresponding relationship between the difference in different dimensions and the parameter of the cooking scheme, the parameter of the target cooking scheme can be adjusted, and the adjusted target cooking scheme can be recorded. For example, the evaluation information indicates that the maturity of the cooked food material is low, and then the cooking time parameter of the target cooking scheme can be adjusted to achieve the ideal state of the food material when the target cooking scheme is executed next time. The stored target cooking scheme can be used as the initial cooking scheme for the next execution to improve the cooking quality next time.

[0081] The embodiment realizes the storage of the personalized cooking scheme for the target user by obtaining the evaluation information after the end of the cooking and adjusting the parameter of the executed target cooking scheme based on the evaluation information and storing the adjusted target cooking scheme, which helps to provide the cooking efficiency for the target user and improve the cooking quality.

[0082] In some optional implementation manners of the embodiment, as shown in Figure 5As shown, step 108 includes:

[0083] Step 1081, in response to the end of the cooking process, obtaining a food material image of the food material after cooking as a third food material image.

[0084] The third food material image can be an image captured by a camera on the cooking device, or an image captured by the user using a mobile device.

[0085] Step 1082, detecting the state of the food material in the third food material image to obtain finished food material state information.

[0086] Step 1083, determining the difference between the expected food material state information corresponding to the parameters of the target cooking scheme and the finished food material state information.

[0087] The method of detecting the state of the food material in the third food material image and determining the difference between the food material state information can be the same as steps 1072 and 1073 described above, and will not be repeated here.

[0088] Step 1084, generating first evaluation information of the cooked food material based on the difference.

[0089] Specifically, the difference can be directly determined as the first evaluation information, or the color, luster, maturity, etc. of the finished food material can be determined based on the difference, and these properties can be used as the first evaluation information.

[0090] Step 1085, obtaining second evaluation information input by the user for feedback on the food material after cooking.

[0091] The second evaluation information can be information input by the user to the cooking device through a mobile device or other device, or information input by the user operating the control panel of the cooking device.

[0092] Step 1086, generating evaluation information of the cooked food material based on the first evaluation information and / or the second evaluation information.

[0093] This step uses at least one of the first evaluation information and the second evaluation information, i.e. when using the first evaluation information, step 1085 above can not be executed, and when using the second evaluation information, steps 1081-1084 above can not be executed.

[0094] When using the first evaluation information and the second evaluation information, the two types of evaluation information can be combined to generate the evaluation information used in step 109. When combining, information of the same dimension can be selected from the first evaluation information and the second evaluation information as information included in the generated evaluation information according to a predetermined priority, or if the evaluation information of a certain dimension is a numerical value, the average value can be taken as the information included in the evaluation information.

[0095] The embodiment provides automatically generated evaluation information of cooked food materials and artificially generated evaluation information, realizes accurate and comprehensive evaluation of the cooked food materials, and thus a target cooking scheme adjusted according to the evaluation information can be more matched with the eating habits of a target user.

[0096] In some optional implementation of the embodiment, as shown in Figure 6 The eating habits model is a deep reinforcement learning model. As an example, the eating habits model is a DQN (Deep Q Network) model.

[0097] After the step 108, the method further includes:

[0098] In step 110, a reward and punishment score of the evaluation information is calculated based on a preset reward and punishment function, to obtain a reward and punishment score representing a difference between the eating habits type output by the eating habits model and the actual eating habits of the target user.

[0099] The information of each dimension (for example, the dimensions of taste, color, and luster) included in the evaluation information can correspond to a weight respectively, and the reward and punishment score can be calculated according to the weights of the information included in the evaluation information. The greater the reward and punishment score is, the higher the matching degree between the eating habits type output by the eating habits model and the actual eating habits is, and the smaller the reward and punishment score is, the lower the matching degree between the eating habits type output by the eating habits model and the actual eating habits is.

[0100] In step 111, the parameters of the eating habits model are adjusted according to a deep reinforcement learning algorithm based on the reward and punishment score.

[0101] Deep reinforcement learning combines the feature extraction capability of deep learning and the decision-making capability of reinforcement learning, can directly make an optimal decision output according to the input multi-dimensional data, is an end-to-end decision control system, and learns the optimal decision by taking the evaluation information input from the outside world as an input to obtain the experience of failure or success to update the parameters of the decision network.

[0102] The embodiment trains the eating habits model by using the deep reinforcement learning algorithm, can take the evaluation information fed back for the cooked food materials as an index for model training, and makes the trained eating habits model more targeted to analyze the input eating habits data, and improves the prediction accuracy of the eating habits model.

[0103] In some optional implementations of the embodiment, as shown in FIG. 1C, after step 103, the method further includes: Figure 7

[0104] Step 112, obtaining the attribute of the currently stored food material from the food material storage device.

[0105] The food material storage device can be a refrigerator, a freezer, or other types of devices for storing food materials. The food material storage device can be communicatively connected with the electronic device that executes the method. The food material storage device can identify the type of the stored food material through a camera, RFID, or other ways, or the user can input the type of the stored food material.

[0106] Step 113, generating the to-be-determined cooking scheme based on the attribute of the food material to be cooked and the attribute of the currently stored food material, and outputting the to-be-determined cooking scheme.

[0107] Specifically, the type, quantity, and other attributes of the food material to be cooked can be compared with the type, quantity, and other attributes of the currently stored food material. The food material to be cooked and the stored food material are integrated, and a cooking scheme that matches the integrated food material is determined from the cooking scheme library as the to-be-determined cooking scheme.

[0108] The to-be-determined cooking scheme can be output by being displayed on the display screen of the cooking device, or by being pushed to the mobile phone or other device used by the target user. The target user can view the to-be-determined cooking scheme.

[0109] Step 114, in response to receiving the confirmation information that confirms the to-be-determined cooking scheme, updating the initial cooking scheme based on the to-be-determined cooking scheme.

[0110] When the target user confirms the to-be-determined cooking scheme by operating the cooking device or the mobile phone or other device, the current to-be-determined cooking scheme can be used as the updated initial cooking scheme.

[0111] It should be noted that after the user confirms the cooking scheme, the cooking device can be in a standby state. After the user prepares the food material and further performs the start cooking operation, the execution subject of the method can continue to perform the subsequent steps 104-107 to control the cooking device to perform the cooking operation.

[0112] The embodiment can generate a more intelligent cooking scheme by communicating with the food material storage device, provide the user with more rich cooking scheme suggestions, and thus improve the flexibility of cooking and the matching degree with the eating habits of the target user.

[0113] In some optional implementations of the embodiment, as shown in FIG. 1C, after step 103, the method further includes: Figure 8 ​As shown, after step 106, the method further comprises:

[0114] Step 115, obtaining the body state information of the target user from the user body state detection device.

[0115] The user body state detection device can be a smart bracelet, a body scale, a sphygmomanometer, or other devices for detecting the body state of the target user. The execution subject of the method can be communicatively connected to the user body state detection device to obtain the recorded body state information.

[0116] Step 116, adjusting the parameters of the target cooking scheme based on the body state information, so that the adjusted target cooking scheme matches the body state information.

[0117] Based on the adjusted target cooking scheme, step 107 can be performed to make the cooked food match the body state of the target user.

[0118] For example, if the body state information indicates that the target user's body state is suitable for a list diet, the parameters of the target cooking scheme are adjusted so that the automatic ingredient adding device adds a small amount of salt.

[0119] By communicating with the user body state detection device, the embodiment can automatically adjust the target cooking scheme according to the user's body state, thereby providing more personalized cooking schemes for the user and reducing the user's health risks caused by diet.

[0120] Figure 9 A structural schematic diagram of an automatic cooking device provided by the embodiment of the present application is shown. Specifically, the structural schematic diagram comprises:

[0121] The first obtaining module 901 is configured to obtain a first food material image of the food material to be cooked;

[0122] The detection module 902 is configured to use a preset food material detection model to detect the type of food material in the first food material image and determine the attribute of the food material.

[0123] The first generating module 903 is configured to generate an initial cooking scheme based on the attribute of the food material.

[0124] The second obtaining module 904 is configured to obtain the dietary habit data of the target user.

[0125] The analysis module 905 is configured to use a preset dietary habit model to analyze the dietary habit data and determine the dietary habit type of the target user.

[0126] The first adjustment module 906 is configured to adjust the parameters of the initial cooking scheme based on the dietary habit type to obtain a target cooking scheme that matches the dietary habit type.

[0127] The control module 907 is configured to control the cooking device to cook the food material based on the target cooking scheme.

[0128] In some optional implementations of the embodiment, the control module includes: a first acquisition unit configured to acquire current environmental condition information in the cooking device during the cooking process, and determine a first difference between the expected environmental condition information corresponding to the target cooking scheme and the current environmental condition information; and / or, a second acquisition unit configured to acquire a second food material image of the food material during the cooking process, perform food material state detection on the second food material image to obtain current food material state information; a first determination unit configured to determine a second difference between the expected food material state information corresponding to the target cooking scheme and the current food material state information; and an adjustment unit configured to adjust the parameters of the target cooking scheme based on the first difference and / or the second difference, and control the cooking device to cook the food material based on the adjusted target cooking scheme.

[0129] In some optional implementations of the embodiment, the apparatus further includes: a third acquisition module configured to acquire evaluation information of the cooked food material in response to the end of the cooking process; and a second adjustment module configured to adjust the parameters of the target cooking scheme based on the evaluation information, and store the adjusted target cooking scheme.

[0130] In some optional implementations of the embodiment, the third acquisition module includes: a third acquisition unit configured to acquire a food material image of the food material after the end of the cooking process as a third food material image in response to the end of the cooking process; a detection unit configured to perform food material state detection on the third food material image to obtain finished food material state information; a second determination unit configured to determine a difference between the expected food material state information corresponding to the parameters of the target cooking scheme and the finished food material state information; a first generation unit configured to generate first evaluation information of the cooked food material based on the difference; and / or, a fourth acquisition unit configured to acquire second evaluation information input by feedback on the food material after the end of the cooking process; and a second generation unit configured to generate evaluation information of the cooked food material based on the first evaluation information and / or the second evaluation information.

[0131] In some optional implementations of the embodiment, the eating habit model is a deep reinforcement model; the apparatus further includes: a calculation module configured to perform a reward and punishment score calculation on the evaluation information based on a preset reward and punishment function to obtain a reward and punishment score representing a difference between the eating habit type output by the eating habit model and the actual eating habit of the target user; and a third adjustment module configured to adjust the parameters of the eating habit model based on the reward and punishment score according to a deep reinforcement learning algorithm.

[0132] In some optional implementation of the embodiment, the apparatus further includes a fourth obtaining module configured to obtain the attribute of the currently stored food material from the food material storage device; a second generating module configured to generate the to-be-determined cooking scheme based on the attribute of the food material to be cooked and the attribute of the currently stored food material, and output the to-be-determined cooking scheme; and an updating module configured to update the initial cooking scheme based on the to-be-determined cooking scheme in response to receiving the confirmation information after the to-be-determined cooking scheme is confirmed.

[0133] In some optional implementation of the embodiment, the apparatus further includes a fifth obtaining module configured to obtain the body state information of the target user from the user body state detection device; and a fourth adjusting module configured to adjust the parameter of the target cooking scheme based on the body state information, so that the adjusted target cooking scheme matches the body state information.

[0134] The automatic cooking apparatus provided in the embodiment can be the automatic cooking apparatus as shown in Figure 9 The automatic cooking apparatus can execute all steps of the above automatic cooking methods, and thus achieve the technical effects of the above automatic cooking methods. For brevity, details are not described herein.

[0135] Figure 10 A structural schematic diagram of an electronic device provided in the embodiment of the present application is shown in Figure 10 The electronic device 1000 shown in the embodiment includes at least one processor 1001, a memory 1002, at least one network interface 1004 and other user interfaces 1003. The various components in the electronic device 1000 are coupled together through a bus system 1005. It can be understood that the bus system 1005 is used to realize the connection and communication between the components. The bus system 1005 includes a data bus, a power bus, a control bus and a status signal bus, but for the purpose of clear illustration, all the buses are marked as the bus system 1005 in the Figure 10

[0136] The user interface 1003 can include a display, a keyboard or a clicking device (for example, a mouse, a trackball, a touchpad or a touch screen, etc.).

[0137] ​It is to be understood that the memory 1002 in embodiments of the present application can be a volatile memory or a nonvolatile memory, or can include both volatile and nonvolatile memory. Among them, the nonvolatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM) used as an external cache. By way of example, and not limitation, many forms of RAM can be used, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous Dynamic RAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synch link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 1002 described herein is intended to include, without being limited to, these and any other suitable types of memory.

[0138] In some embodiments, the memory 1002 stores the following elements, executable units or data structures, or a subset of them, or an extended set of them: an operating system 10021 and an application program 10022.

[0139] Among them, the operating system 10021 contains various system programs, such as framework layer, core library layer, driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application program 10022 contains various application programs, such as Media Player, Browser, etc., for implementing various application services. The program for implementing the method of the embodiments of the present application can be contained in the application program 10022.

[0140] In the present embodiment, by calling the program or instruction stored in the memory 1002, specifically, the program or instruction stored in the application program 10022, the processor 1001 is used to execute the method steps provided by each method embodiment, for example, including:

[0141] Obtaining a first food material image photographed for food material to be cooked; performing food material type detection on the first food material image by using a preset food material detection model to determine an attribute of the food material; generating an initial cooking scheme based on the attribute of the food material; obtaining dietary habit data of a target user; performing analysis on the dietary habit data by using a preset dietary habit model to determine a dietary habit type of the target user; adjusting parameters of the initial cooking scheme based on the dietary habit type to obtain a target cooking scheme matched with the dietary habit type; and controlling a cooking device to cook the food material based on the target cooking scheme.

[0142] The method disclosed in the embodiments of the present application can be applied to the processor 1001 or implemented by the processor 1001. The processor 1001 can be an integrated circuit chip having a signal processing capability. In implementation process, the steps of the method disclosed above can be completed by hardware integrated logic circuit or software form of instruction in the processor 1001. The processor 1001 can be a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The methods, steps and logical block diagrams disclosed in the embodiments of the present application can be implemented or executed by the processor 1001. The general purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the methods disclosed in conjunction with the embodiments of the present application can be directly embodied in hardware code of the processor, or executed by a combination of hardware and software modules in the processor. The software module can reside in a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable read-only memory (PROM), an electrically programmable read-only memory (EPROM), a electrically erasable programmable read-only memory (EEPROM), registers, or other forms of storage. The storage medium is located in the storage 1002, and the processor 1001 reads information in the storage 1002 and combines the hardware to complete the steps of the methods described above.

[0143] It can be understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing units can be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSP Devices, DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general purpose processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described above, or a combination thereof.

[0144] For software implementation, the techniques described herein can be implemented with a technology described above. Software codes can be stored in memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0145] The electronic device provided by the embodiments can be an electronic device as shown in Figure 10 The electronic device provided by the embodiments can be an electronic device as shown in

[0146] The embodiments of the present application also provide a storage medium (computer readable storage medium). The storage medium stores one or more programs. The storage medium can include a volatile memory, such as a random access memory; the storage medium can also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk or a solid state disk; the storage medium can also include a combination of the above kinds of memories.

[0147] When the one or more programs in the storage medium are executed by the one or more processors, the automatic cooking method described above executed on the side of the electronic device can be implemented.

[0148] The processor described above is used to execute the program stored in the memory, so as to implement the steps of the automatic cooking method executed on the side of the electronic device as follows:

[0149] acquire a first food material image photographed for food material to be cooked; perform food material type detection on the first food material image by using a preset food material detection model, to determine an attribute of the food material; generate an initial cooking scheme based on the attribute of the food material; acquire dietary habit data of a target user; analyze the dietary habit data by using a preset dietary habit model, to determine a dietary habit type of the target user; adjust parameters of the initial cooking scheme based on the dietary habit type, to obtain a target cooking scheme matching the dietary habit type; and control a cooking device to cook the food material based on the target cooking scheme.

[0150] Those skilled in the art should further appreciate that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be embodied in electronic hardware, computer software, or a combination of both, where the disclosure is not limited to any specific combination of hardware and software. The various examples have been described in general terms above in connection with the illustrative examples. To the extent there are any steps described in the above in connection with the examples, those steps can be embodied in hardware, software, or a combination of both, where appropriate. The disclosure is not limited to any particular implementation of the examples described above.

[0151] The circuit or algorithm steps described in connection with the embodiments disclosed herein can be embodied directly in hardware, in software executed by a processor, or in a combination of the two. Software modules can reside in Random Access Memory (RAM), non-volatile memory (e.g., Flash memory), read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0152] It should be understood that the terms used herein are for the purpose of describing particular example embodiments only and are not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms "comprises", "comprising", "includes", "including" and the like are to be construed to be inclusive (i.e., to include both instances of open ended terms and instances of terms limiting to just the enumerated instances) unless otherwise indicated herein with the singular, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms "comprises", "comprising", "includes", "including" and the like are to be construed to be inclusive (i.e., to include both instances of open ended terms and instances of terms limiting to just the enumerated instances) unless otherwise indicated herein. The steps, processes, and operations described herein are not to be construed as necessarily requiring their performance in the particular order in which they are described unless otherwise indicated. It is also to be understood that additional or alternative steps can be employed.

[0153] The foregoing detailed description of the application has been presented for purposes of illustration and description. Various modifications and changes can be made to these embodiments without departing from the spirit and scope of the application. It is intended that the scope of the application should not be limited by the particular representative embodiments described above.

Claims

1. An automatic cooking method, characterized in that, The method includes: Acquire the first image of the ingredients to be cooked; Using a preset food detection model, the first food image is subjected to food type detection to determine the attributes of the food. Based on the properties of the ingredients, an initial cooking plan is generated; Obtain dietary habit data from target users; Using a pre-defined dietary habit model, the dietary habit data is analyzed to determine the dietary habit type of the target user; Based on the dietary habit type, the parameters of the initial cooking plan are adjusted to obtain a target cooking plan that matches the dietary habit type; Based on the target cooking scheme, control the cooking equipment to cook the ingredients; In response to the end of the cooking process, obtain evaluation information on the cooked ingredients; Based on the evaluation information, the parameters of the target cooking program are adjusted, and the adjusted target cooking program is stored. The dietary habit model is a deep reinforcement model; after obtaining the evaluation information of the cooked ingredients, the method further includes: Based on a preset reward and punishment function, the evaluation information is used to calculate reward and punishment scores to obtain a reward and punishment score that represents the difference between the dietary habit type output by the dietary habit model and the actual dietary habits of the target user. Based on the reward and punishment scores, the parameters of the dietary habit model are adjusted using a deep reinforcement learning algorithm.

2. The method according to claim 1, characterized in that, The step of controlling the cooking equipment to cook the ingredients based on the target cooking scheme includes: Acquire current environmental condition information within the cooking equipment during the cooking process, and determine a first difference between the desired environmental condition information corresponding to the target cooking scheme and the current environmental condition information; and / or, A second food image is captured during the cooking process, and the food state is detected in the second food image to obtain the current food state information; Determine a second difference between the desired ingredient state information corresponding to the target cooking scheme and the current ingredient state information; Based on the first difference and / or the second difference, the parameters of the target cooking scheme are adjusted, and the cooking equipment is controlled to cook the ingredients based on the adjusted target cooking scheme.

3. The method according to claim 1, characterized in that, The acquisition of evaluation information on cooked ingredients includes: In response to the end of the cooking process, an image of the ingredients taken after cooking is obtained as a third ingredient image; The third ingredient image is subjected to ingredient state detection to obtain finished ingredient state information; Determine the difference between the expected ingredient state information and the finished ingredient state information corresponding to the parameters of the target cooking scheme; Based on the aforementioned differences, a first assessment of the cooked ingredients is generated; and / or, Obtain the second evaluation information input as feedback on the ingredients after the cooking process is completed; Based on the first evaluation information and / or the second evaluation information, evaluation information for the cooked ingredients is generated.

4. The method according to claim 1, characterized in that, After generating the initial cooking plan based on the properties of the ingredients, the method further includes: Obtain the attributes of the food currently stored from the food storage equipment; Based on the properties of the ingredients to be cooked and the properties of the currently stored ingredients, a cooking plan to be determined is generated and output. In response to receiving confirmation information after confirming the cooking plan to be determined, the initial cooking plan is updated based on the cooking plan to be determined.

5. The method according to claim 1, characterized in that, After adjusting the parameters of the initial cooking plan based on the dietary habit type to obtain a target cooking plan that matches the dietary habit type, the method further includes: Obtain the target user's physical condition information from the user's physical condition detection device; Based on the body condition information, the parameters of the target cooking program are adjusted so that the adjusted target cooking program matches the body condition information.

6. An automatic cooking device, characterized in that, The device includes: The first acquisition module is used to acquire a first image of the ingredients to be cooked. The detection module is used to detect the type of the food in the first food image using a preset food detection model, and to determine the attributes of the food. The first generation module is used to generate an initial cooking plan based on the properties of the ingredients; The second acquisition module is used to acquire dietary habit data of the target user; The analysis module is used to analyze the dietary habit data using a preset dietary habit model to determine the dietary habit type of the target user; The first adjustment module is used to adjust the parameters of the initial cooking plan based on the dietary habit type to obtain a target cooking plan that matches the dietary habit type. The control module is used to control the cooking equipment to cook the ingredients based on the target cooking scheme; The device also includes: The third acquisition module is used to acquire evaluation information on the cooked ingredients in response to the end of the cooking process; The second adjustment module is used to adjust the parameters of the target cooking program based on the evaluation information, and to store the adjusted target cooking program. The dietary habit model is a deep reinforcement model; the device also includes: The calculation module is used to calculate reward and punishment scores on the evaluation information based on a preset reward and punishment function, so as to obtain a reward and punishment score representing the difference between the dietary habit type output by the dietary habit model and the actual dietary habits of the target user; The third adjustment module is used to adjust the parameters of the dietary habit model based on the reward and punishment scores and according to a deep reinforcement learning algorithm.

7. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor is configured to execute a computer program stored in the memory, wherein when the computer program is executed, it implements the automatic cooking method according to any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the automatic cooking method according to any one of claims 1-5.

Images