Automatic cooking method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention relates to an automatic cooking method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a first food material image shot by a to-be-cooked food material; performing food material type detection on the first food material image by using a preset food material detection model, and determining attributes of food materials; generating an initial cooking scheme based on the attributes of the food materials; obtaining dietary habit data of the target user; analyzing the dietary habit data by using a preset dietary habit model, and determining a dietary habit type of the target user; based on the dietary habit type, adjusting parameters of the initial cooking scheme to obtain a target cooking scheme matched with the dietary habit type; and based on the target cooking scheme, controlling the cooking equipment to cook the food materials. According to the embodiment of the invention, the method achieves the automatic detection of the type of the to-be-cooked food material, automatically generates the cooking scheme matched with the diet habit of the user, better meets the personalized demands of the user, and improves the cooking efficiency.

Description

Technical Field

[0001] The present application relates to the technical field of smart home appliances, and in particular to an automatic cooking method, device, electronic device and computer-readable storage medium. Background Art

[0002] As people's living standards improve, the demand for diversity and personalization of food cooking is increasing. In the field of cooking, smart cooking equipment such as steam ovens and air fryers are important equipment in home kitchens, and their intelligence directly affects the user's cooking experience. However, although traditional cooking equipment provides automatic cooking functions, it often lacks the ability to make intelligent adjustments based on the user's personal eating habits. The flexibility and pertinence of cooking methods are poor, and users are required to manually intervene in the cooking process, resulting in low cooking efficiency. Summary of the invention

[0003] In view of this, in order to solve some or all of the above-mentioned technical problems, the embodiments of the present application provide an automatic cooking method, device, electronic device and computer-readable storage medium.

[0004] In a first aspect, an embodiment of the present application provides an automatic cooking method, the method comprising: acquiring a first ingredient image taken of an ingredient to be cooked; using a preset ingredient detection model to perform ingredient type detection on the first ingredient image to determine the attributes of the ingredient; generating an initial cooking plan based on the attributes of the ingredient; acquiring dietary habit data of a target user; using a preset dietary habit model to analyze the dietary habit data to determine the dietary habit type of the target user; based on the dietary habit type, adjusting parameters of the initial cooking plan to obtain a target cooking plan that matches the dietary habit type; and based on the target cooking plan, controlling a cooking device to cook the ingredients.

[0005] In one possible embodiment, controlling a cooking device to cook ingredients based on a target cooking scheme includes: obtaining current environmental condition information in the cooking device during the cooking process, and determining a first difference between expected environmental condition information corresponding to the target cooking scheme and current environmental condition information; and / or obtaining a second ingredient image taken of the ingredient during the cooking process, performing ingredient status detection on the second ingredient image, and obtaining current ingredient status information; determining a second difference between expected ingredient status information corresponding to the target cooking scheme and current ingredient status information; and adjusting parameters of the target cooking scheme based on the first difference and / or the second difference, and controlling the cooking device to cook the ingredients based on the adjusted target cooking scheme.

[0006] In one possible embodiment, after controlling the cooking device to cook the ingredients based on the target cooking plan, the method further includes: in response to the end of the cooking process, obtaining evaluation information of the cooked ingredients; based on the evaluation information, adjusting parameters of the target cooking plan, and storing the adjusted target cooking plan.

[0007] In one possible embodiment, obtaining evaluation information of the cooked ingredients includes: in response to the end of the cooking process, obtaining an ingredient image taken of the ingredients after cooking as a third ingredient image; performing ingredient status detection on the third ingredient image to obtain finished ingredient status information; determining the difference between the expected ingredient status information and the finished ingredient status information corresponding to the parameters of the target cooking plan; based on the difference, generating first evaluation information of the cooked ingredients; and / or obtaining second evaluation information input for feedback on the ingredients after cooking; and generating evaluation information of the cooked ingredients based on the first evaluation information and / or the second evaluation information.

[0008] In one possible implementation, the dietary habit model is a deep reinforcement model; after obtaining evaluation information on the cooked ingredients, the method further includes: calculating a reward or punishment score for the evaluation information based on a preset reward or punishment function, to obtain a reward or punishment score representing the difference between the dietary habit type output by the dietary habit model and the actual dietary habits of the target user; and adjusting the parameters of the dietary habit model according to a deep reinforcement learning algorithm based on the reward or punishment score.

[0009] In one possible embodiment, after generating the initial cooking plan based on the attributes of the ingredients, the method further includes: acquiring the attributes of the currently stored ingredients from the ingredient storage device; generating a cooking plan to be determined based on the attributes of the ingredients to be cooked and the attributes of the currently stored ingredients, and outputting the cooking plan to be determined; in response to receiving confirmation information after confirming the cooking plan to be determined, updating the initial cooking plan based on the cooking plan to be determined.

[0010] In one possible implementation, after adjusting the parameters of the initial cooking plan based on the dietary habit type to obtain a target cooking plan that matches the dietary habit type, the method further includes: acquiring physical state information of the target user from a user physical state detection device; and adjusting the parameters of the target cooking plan based on the physical state information so that the adjusted target cooking plan matches the physical state information.

[0011] In a second aspect, an embodiment of the present application provides an automatic cooking device, which includes: a first acquisition module, used to acquire a first food image taken of food to be cooked; a detection module, used to use a preset food detection model to detect the food type of the first food image, and determine the attributes of the food; a first generation module, used to generate an initial cooking plan based on the attributes of the food; a second acquisition module, used to acquire the dietary habit data of a target user; an analysis module, used to analyze the dietary habit data using a preset dietary habit model, and determine the dietary habit type of the target user; a first adjustment module, used to adjust the parameters of the initial cooking plan based on the dietary habit type, and obtain a target cooking plan that matches the dietary habit type; and a control module, used to control the cooking device to cook the food based on the target cooking plan.

[0012] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory for storing a computer program; a processor for executing the computer program stored in the memory, and when the computer program is executed, the method of any embodiment of the automatic cooking method of the first aspect of the present application is implemented.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method of any embodiment of the automatic cooking method of the first aspect described above is implemented.

[0014] In a fifth aspect, an embodiment of the present application provides a computer program, which includes a computer readable code. When the computer readable code is executed on a device, the processor in the device implements a method as in any embodiment of the automatic cooking method of the first aspect described above.

[0015] The automatic cooking method, device, electronic device and computer-readable storage medium provided in the embodiments of the present application detect the type of food in the first food image taken by using the food detection model to determine the properties of the food; based on the properties of the food, an initial cooking plan is generated, and then the dietary habit model is used to analyze the dietary habit data of the target user to determine the dietary habit type of the target user, and based on the dietary habit type, the parameters of the initial cooking plan are adjusted to obtain a target cooking plan that matches the dietary habits, and finally, based on the target cooking plan, the cooking device is controlled to cook the food. The embodiments of the present application realize the automatic detection of the type of food to be cooked, and at the same time, automatically generate a cooking plan that matches the dietary habits of the user in combination with the dietary habits of the user, so that the cooked dishes are more adapted to the personalized needs of the user, the number of times the user intervenes in the cooking process is reduced, and the cooking efficiency is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0019] Figure 1 A schematic diagram of a flow chart of an automatic cooking method provided in an embodiment of the present application;

[0020] Figure 2 An exemplary structural diagram of a food detection model provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of another automatic cooking method provided in an embodiment of the present application;

[0022] Figure 4 A schematic diagram of a flow chart of another automatic cooking method provided in an embodiment of the present application;

[0023] Figure 5 A schematic diagram of a flow chart of another automatic cooking method provided in an embodiment of the present application;

[0024] Figure 6 A schematic diagram of a flow chart of another automatic cooking method provided in an embodiment of the present application;

[0025] Figure 7 A schematic diagram of a flow chart of another automatic cooking method provided in an embodiment of the present application;

[0026] Figure 8 A schematic diagram of a flow chart of another automatic cooking method provided in an embodiment of the present application;

[0027] Fig. 9 A schematic diagram of the structure of an automatic cooking device provided in an embodiment of the present application;

[0028] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. It should be noted that unless otherwise specifically stated, the relative arrangement of the components and steps, the numerical expressions and the numerical values ​​described in these embodiments do not limit the scope of the present application.

[0030] Those skilled in the art will understand that the terms "first" and "second" in the embodiments of the present application are only used to distinguish between different steps, devices, modules and other objects, and do not represent any specific technical meanings, nor do they indicate the logical order between them.

[0031] It should also be understood that in this embodiment, “plurality” may refer to two or more than two, and “at least one” may refer to one, two or more than two.

[0032] It should also be understood that any component, data or structure mentioned in the embodiments of the present application can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0033] In addition, the term "and / or" in this application is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this application generally indicates that the associated objects before and after are in an "or" relationship.

[0034] It should also be understood that the description of the various embodiments in this application focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0035] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present application, its application, or uses.

[0036] Technologies, circuits, and devices known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the above-mentioned technologies, circuits, and devices should be considered as part of the specification.

[0037] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0038] It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other. To facilitate the understanding of the embodiments of the present application, the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0039] In order to solve the technical problem that the prior art cannot provide personalized cooking plans based on the user's eating habits, the present application provides an automatic cooking method that can combine automatic food detection and user eating habits to efficiently and specifically provide users with personalized cooking plans, thereby improving cooking efficiency.

[0040] Figure 1 A flow chart of an automatic cooking method provided for an embodiment of the present application. This method can be applied to various types of intelligent cooking devices such as steam ovens and air fryers, and can be executed by a processor included in the intelligent cooking device, or by one or more electronic devices such as a smart phone, a laptop computer, a desktop computer, a server, etc. that are connected to the intelligent cooking device for communication. In addition, the execution subject of this method can be hardware or software. When the above-mentioned execution subject is hardware, the execution subject can be one or more of the above-mentioned electronic devices. For example, a single electronic device can execute this method, or multiple electronic devices can cooperate with each other to execute this method. When the above-mentioned execution subject is software, this method can be implemented as multiple software or software modules, or as a single software or software module. No specific limitation is made here.

[0041] like Figure 1 As shown, the method specifically includes:

[0042] Step 101: Acquire a first food image taken of food to be cooked.

[0043] In this embodiment, the ingredients to be cooked are the ingredients put into the cooking device, or the ingredients to be put into the cooking device. For example, the cooking device may be provided with a camera, and after the ingredients are put into the cooking device, the camera will shoot the ingredients to obtain a first ingredient image. For another example, the user may shoot the ingredients to be cooked through a mobile phone or other device, and send the shot first ingredient image to the cooking device or other electronic device used to execute the method.

[0044] Step 102: Utilize a preset food detection model to perform food type detection on the first food image to determine the properties of the food.

[0045] In this embodiment, the food detection model can be a target detection model constructed based on neural networks of various structures, such as YOLO, RPN (Region Proposal Networks), etc. The food detection model can be trained in advance by a machine learning method, that is, the parameters of the initial model are adjusted using sample food images and annotated food location, type, freshness and other attribute information, and the training is stopped when the initial model meets the training end conditions to obtain the food detection model. The food detection model can determine the location, size, type, freshness and other information of various ingredients from the input food image, and this information can be used as the attributes of the food.

[0046] As an example, the food detection model can be trained based on the YOLO model. Figure 2 As shown, the YOLO model includes:

[0047] Preprocessing module, whose main functions include data enhancement, adaptive anchor box calculation and adaptive image scaling.

[0048] The backbone network is used to convert the input image into channel dimensions, expand the number of channels, and improve the model operation efficiency. It can also include a lightweight attention module CBAM for adaptive feature refinement.

[0049] Neck network: It is used to realize the fusion of bottom-level and high-level information of data, make full use of the underlying features of the target, and obtain multi-scale fusion features.

[0050] Head network (head): includes dense prediction units and sparse prediction units. Dense prediction obtains target frames that meet the requirements through positive and negative sample matching strategies and loss calculations, and sparse prediction obtains the best target frame by suppressing redundant frames through NMS (Non-maximum Suppression) algorithm. The confidence loss in the loss calculation is also given different weights in different prediction units (such as detection box prediction units, category prediction units, etc.). In addition, a small target detection head (tiny) can be added to improve the detection accuracy of small targets.

[0051] Step 103: Generate an initial cooking plan based on the properties of the ingredients.

[0052] In this embodiment, the properties of various ingredients can be matched with a preset cooking plan library, and the cooking plan with the highest matching degree with the detected ingredient properties can be selected as the above-mentioned initial cooking plan. As an example, the matching process can be to calculate similarity between the ingredient type, the amount of each type of ingredient, and other information included in the identified attributes and the ingredient type, amount, and other information corresponding to each cooking plan in the cooking plan library, and determine the cooking plan with the greatest similarity as the initial cooking plan.

[0053] Step 104, obtaining the target user's dietary habit data.

[0054] In this embodiment, the target user is the user targeted by the current cooking operation. Generally, the corresponding relationship between the cooking operation and the target user can be established by inputting the target user's account information. The target user's dietary habit data can be preset, and the electronic device executing this method can obtain the dietary habit data corresponding to the target user from the storage area. Alternatively, the dietary habit data can also be input on-site by the target user. As an example, the dietary habit data can include but is not limited to at least one of the following: food type preference, taste, intake, eating time, etc.

[0055] Step 105 , using a preset eating habit model, analyzing the eating habit data to determine the eating habit type of the target user.

[0056] In this embodiment, the dietary habit model may be various types of software modules that represent the correspondence between dietary habit data and dietary habit types. For example, the dietary habit model may be a neural network model pre-trained by a machine learning method, or a table that represents the correspondence between dietary habit data and dietary habit types pre-stated.

[0057] As an example, dietary habit data may include information such as the type of ingredients, amount, cooking time, and frequency of cooking that is recorded in advance for each cooking session of the target user, and may also include information such as the target user's preference for ingredient types and taste preferences. The electronic device may perform preprocessing such as data deduplication, data enhancement, and data normalization on this information to obtain data that meets the model processing standards. Then, the dietary habit model may be used to extract features from the input data, classify the features, and obtain dietary habit types.

[0058] Step 106: Based on the dietary habit type, the parameters of the initial cooking plan are adjusted to obtain a target cooking plan that matches the dietary habit type.

[0059] In this embodiment, the correspondence between the dietary habit type and the parameters of the cooking plan can be established in advance, and the correspondence can be expressed in the form of a table or a calculation formula. For example, if the dietary habit type prefers softer food, the time parameter of the initial cooking plan can be adjusted to increase the cooking time; if the dietary habit type prefers hot dishes, the heating temperature can be increased.

[0060] Optionally, the cooking device may include a device (e.g., including a discharging box, a seasoning outlet driven by a controlled motor, etc.) for automatically adding cooking ingredients (e.g., water, soy sauce, vinegar, salt, etc.), and the amount of various ingredients added to the initial cooking plan may be adjusted according to the dietary habit type. For example, if the dietary habit type is spicy or salty, the motor may be automatically controlled to rotate, driving the seasoning outlet to add a corresponding amount of chili or salt to the food being cooked.

[0061] Step 107: Based on the target cooking plan, control the cooking equipment to cook the ingredients.

[0062] In this embodiment, the electronic device executing the present method can control the heating device, stir-frying device, ventilation device, ingredient adding device, etc. of the cooking equipment according to the control parameters corresponding to the target cooking plan, and perform corresponding actions according to the set cooking time, cooking temperature and other conditions to complete the cooking process.

[0063] The automatic cooking method provided in the embodiment of the present application uses an ingredient detection model to detect the ingredient type of the first ingredient image taken to determine the attribute of the ingredient; based on the attribute of the ingredient, an initial cooking plan is generated, and then the dietary habit model is used to analyze the dietary habit data of the target user to determine the dietary habit type of the target user, and based on the dietary habit type, the parameters of the initial cooking plan are adjusted to obtain a target cooking plan that matches the dietary habit, and finally, based on the target cooking plan, the cooking device is controlled to cook the ingredients. The embodiment of the present application realizes the automatic detection of the type of ingredients to be cooked, and at the same time, automatically generates a cooking plan that matches the dietary habit of the user in combination with the dietary habit of the user, so that the cooked dishes are more adapted to the personalized needs of the user, reducing the number of times the user intervenes in the cooking process and improving the cooking efficiency.

[0064] In some optional implementations of this embodiment, such as Figure 3 As shown, step 107 includes:

[0065] Step 1071, obtaining current environmental condition information in the cooking device during the cooking process, and determining a first difference between the expected environmental condition information corresponding to the target cooking scheme and the current environmental condition information.

[0066] Among them, the current environmental condition information can be collected by various sensors in the cooking device. For example, the current environmental condition information may include at least one of the following: temperature, humidity, air volume and other information. The above-mentioned expected environmental condition information is the set environmental condition information corresponding to the target cooking plan. Different cooking stages may correspond to different expected environmental condition information, so that in different cooking stages, the difference between the current environmental condition information and the corresponding expected environmental condition information can be detected in real time, and the difference may include the difference between the state information of different dimensions.

[0067] Step 1072, obtaining a second food image taken of the food during the cooking process, performing food status detection on the second food image, and obtaining current food status information.

[0068] The second food image is the food image captured in real time by the camera on the cooking device. The current food state information may include state information of various dimensions such as the color, gloss, and shape of the food. Food state detection may be performed by a pre-trained food state detection model. That is, food images of different states are collected in advance as sample images, and the food states corresponding to the sample images are annotated. The food state detection model may include multiple convolutional layers, pooling layers, fully connected layers and other structures, extract the color, shape, gloss and other features of the input food image, and classify the features by a classifier to obtain predicted food state information. By comparing the error between the annotated food state information and the predicted food state information, the parameters of the food state detection model are adjusted, and when the training end condition is met, the training is terminated to obtain the food state detection model.

[0069] The trained food state detection model can extract the color, shape, gloss and other features of the second food image, classify the features, and obtain the current food state information.

[0070] Step 1073: Determine a second difference between the expected food state information corresponding to the target cooking scheme and the current food state information.

[0071] The expected food state information may be preset state information corresponding to the target cooking scheme, that is, an ideal finished food state. The difference between the expected food state information and the current food state information may include the difference between state information in different dimensions.

[0072] Step 1074: adjust parameters of the target cooking scheme based on the first difference and / or the second difference, and control the cooking device to cook the ingredients based on the adjusted target cooking scheme.

[0073] This step is performed using at least one of the first difference and the second difference, that is, when the first difference is used, the above steps 1072-1073 may not be performed, and when the second difference is used, the above step 1071 may not be performed.

[0074] The differences in different dimensions have a preset correspondence with a parameter of the target cooking scheme, so that the parameter can be adjusted to change the difference. For example, when the first difference includes a temperature difference, the temperature of the cooking device can be adjusted. When the second difference indicates that the food is undercooked, the cooking time can be extended to reduce the second difference.

[0075] This embodiment realizes that the cooking plan can be automatically adjusted in a targeted manner during the cooking process by detecting the current environmental condition information and food status information in the cooking equipment in real time during the cooking process, thereby reducing the difference between the food status after cooking and the ideal status, reducing the number of times of manual intervention in the cooking process, and improving cooking quality and efficiency.

[0076] In some optional implementations of this embodiment, such as Figure 4 As shown, after step 107, the method further includes:

[0077] Step 108: in response to the cooking process being finished, obtaining evaluation information of the cooked ingredients.

[0078] The evaluation information may be information input by the user or information automatically generated after food ingredients are tested.

[0079] Step 109: adjusting parameters of the target cooking scheme based on the evaluation information, and storing the adjusted target cooking scheme.

[0080] The evaluation information can indicate the difference in different dimensions (e.g., color, gloss, and degree of maturity) between the state of the cooked ingredients and the ideal state. According to the correspondence between the differences in different dimensions and the parameters of the cooking plan, the parameters of the target cooking plan can be adjusted, and the adjusted target cooking plan can be recorded. For example, if the evaluation information indicates that the maturity of the cooked ingredients is low, the cooking time parameters of the target cooking plan can be adjusted to achieve the ideal state of the ingredients when the target cooking plan is executed next time. The stored target cooking plan can be used as the initial cooking plan for the next execution to improve the cooking quality next time.

[0081] This embodiment obtains evaluation information after cooking is completed, adjusts parameters of the executed target cooking plan based on the evaluation information and stores it, thereby storing personalized cooking plans for target users, which helps to improve cooking efficiency and cooking quality for target users.

[0082] In some optional implementations of this embodiment, such as Figure 5As shown, step 108 includes:

[0083] Step 1081, in response to the cooking process being completed, obtaining an image of the food after cooking as a third food image.

[0084] The third food image may be an image taken by a camera on the cooking device, or an image taken by a user using a mobile phone to log in to the device.

[0085] Step 1082: Perform food status detection on the third food image to obtain finished food status information.

[0086] Step 1083, determining the difference between the expected food state information corresponding to the parameters of the target cooking scheme and the finished food state information.

[0087] Among them, the method of performing food status detection on the third food image and determining the difference between food status information may be the same as the above-mentioned step 1072 and step 1073, which will not be repeated here.

[0088] Step 1084: Generate first evaluation information of the cooked ingredients based on the differences.

[0089] Specifically, the difference may be directly determined as the first evaluation information, or the color, gloss, maturity and other attributes of the finished food material may be judged based on the difference and these attributes may be used as the first evaluation information.

[0090] Step 1085, obtaining second evaluation information input for feedback on the ingredients after cooking.

[0091] The second evaluation information may be information input by a user into the cooking device via a mobile phone or other device, or may be information input by a user by operating a control panel of the cooking device.

[0092] Step 1086: Generate evaluation information of the cooked ingredients based on the first evaluation information and / or the second evaluation information.

[0093] This step is performed using at least one of the first evaluation information and the second evaluation information, that is, when the first evaluation information is used, the above step 1085 may not be performed, and when the second evaluation information is used, the above steps 1081 to 1084 may not be performed.

[0094] When the first evaluation information and the second evaluation information are used, the two evaluation information may be merged to generate the evaluation information used in the above step 109. When merging, information of the same dimension may be selected from the first evaluation information and the second evaluation information according to a preset priority as the information included in the generated evaluation information, or if the evaluation information of a certain dimension is a numerical value, the average value may be taken as the information included in the evaluation information.

[0095] This embodiment provides automatic generation of evaluation information after cooking ingredients and manually generated evaluation information, thereby achieving accurate and comprehensive evaluation of the cooked ingredients, so that the target cooking plan adjusted according to the evaluation information can better match the dietary habits of the target user.

[0096] In some optional implementations of this embodiment, such as Figure 6 As shown, the eating habit model is a deep reinforcement learning model. As an example, the eating habit model is a DQN (Deep Q Network) model.

[0097] After the above step 108, the method further includes:

[0098] Step 110, based on a preset reward and punishment function, a reward and punishment score is calculated for the evaluation information to obtain a reward and punishment score representing the difference between the dietary habit type output by the dietary habit model and the actual dietary habit of the target user.

[0099] Among them, the information of each dimension (such as taste, color, gloss, etc.) included in the evaluation information can correspond to a weight respectively, and the information included in the evaluation information can be calculated according to the weight to obtain the reward and punishment score. The larger the reward and punishment score, the higher the degree of match between the eating habit type output by the eating habit model and the actual eating habits, and the smaller the reward and punishment score, the lower the degree of match between the eating habit type output by the eating habit model and the actual eating habits.

[0100] Step 111, based on the reward and punishment scores, adjust the parameters of the eating habit model according to the deep reinforcement learning algorithm.

[0101] Deep reinforcement learning combines the feature extraction capability of deep learning and the decision-making capability of reinforcement learning. It can directly make the best decision output based on the input multi-dimensional data. It is an end-to-end decision control system that continuously interacts with the outside world in real time, and uses the evaluation information input from the outside world as input to obtain the experience of failure or success to update the parameters of the decision network, thereby learning the best decision. When training the eating habit model based on the evaluation information, the input eating habit data can be used as the information of the state space, the predicted eating habit type can be used as the information of the action space, and the maximization of the reward and punishment score can be used as the optimization target of the model parameters. The back propagation method is used to adjust the parameters of the eating habit model.

[0102] This embodiment uses a deep reinforcement learning algorithm to train the eating habit model, and can use the evaluation information of the feedback on the cooked ingredients as an indicator for model training, so that the trained eating habit model can analyze the input eating habit data more specifically, thereby improving the prediction accuracy of the eating habit model.

[0103] In some optional implementations of this embodiment, such as Figure 7 As shown, after step 103, the method further includes:

[0104] Step 112, obtaining the properties of the currently stored food from the food storage device.

[0105] The food storage device may be a refrigerator, freezer or other type of device for storing food. The food storage device may be connected to the electronic device for executing the method in communication, and the food storage device may identify the type of food stored by means of a camera, RFID (Radio Frequency Identification), etc., or the type of food stored may be input by a user.

[0106] Step 113: generating a cooking scheme to be determined based on the properties of the ingredients to be cooked and the properties of the currently stored ingredients, and outputting the cooking scheme to be determined.

[0107] Specifically, the type, quantity and other attributes of the ingredients to be cooked can be compared with the type, quantity and other attributes of the currently stored ingredients, and the ingredients to be cooked and the stored ingredients can be integrated to determine a cooking plan that matches the integrated ingredients from the cooking plan library as the cooking plan to be determined.

[0108] The cooking plan to be determined may be displayed on a display screen of the cooking device, or may be output by pushing to a mobile phone or other device used by the target user, and the target user may view the cooking plan to be determined.

[0109] Step 114 , in response to receiving confirmation information after confirming the cooking scheme to be determined, updating the initial cooking scheme based on the cooking scheme to be determined.

[0110] When the target user confirms the above-mentioned to-be-determined cooking plan by operating a cooking device or a mobile phone or other device, the current to-be-determined cooking plan can be used as an updated initial cooking plan.

[0111] It should be noted that after the user confirms the cooking plan, the cooking device can be in a standby state. After the user prepares the ingredients and further starts the cooking operation, the executor of this method can continue to execute subsequent steps 104-107 to control the cooking device to perform the cooking operation.

[0112] By communicating with the food storage device, this embodiment can realize more intelligent generation of cooking plans and provide users with more abundant cooking plan suggestions, thereby improving cooking flexibility and matching with the target user's eating habits.

[0113] In some optional implementations of this embodiment, such as Figure 8As shown, after step 106, the method further includes:

[0114] Step 115: Acquire the target user's physical state information from the user's physical state detection device.

[0115] The user's physical state detection device may be a smart bracelet, a weight scale, a blood pressure monitor, or other devices used by the target user to detect the target user's physical state. The execution subject of this method may communicate with the user's physical state detection device to obtain the recorded physical state information.

[0116] Step 116, based on the body state information, adjusting the parameters of the target cooking scheme so that the adjusted target cooking scheme matches the body state information.

[0117] Based on the adjusted target cooking plan, step 107 may be performed to match the cooked ingredients with the physical condition of the target user.

[0118] As an example, if the above-mentioned physical condition information indicates that the physical condition of the target user is suitable for the diet, the parameters of the target cooking plan are adjusted so that the automatic ingredient adding device adds a small amount of salt.

[0119] This embodiment can automatically adjust the target cooking plan according to the user's physical condition by communicating with the user's physical condition detection device, thereby further providing the user with a more personalized cooking plan and reducing the user's health risks caused by diet.

[0120] Fig. 9 This is a schematic diagram of the structure of an automatic cooking device provided in an embodiment of the present application. Specifically comprising:

[0121] A first acquisition module 901 is used to acquire a first food image taken of the food to be cooked;

[0122] A detection module 902 is used to perform food type detection on the first food image using a preset food detection model to determine the properties of the food;

[0123] A first generating module 903, for generating an initial cooking plan based on the properties of the ingredients;

[0124] The second acquisition module 904 is used to acquire the dietary habit data of the target user;

[0125] The analysis module 905 is used to analyze the dietary habit data using a preset dietary habit model to determine the dietary habit type of the target user;

[0126] A first adjustment module 906 is used to adjust the parameters of the initial cooking plan based on the dietary habit type to obtain a target cooking plan that matches the dietary habit type;

[0127] The control module 907 is used to control the cooking equipment to cook the ingredients based on the target cooking plan.

[0128] In some optional implementations of the present embodiment, the control module includes: a first acquisition unit, used to acquire current environmental condition information in the cooking device during the cooking process, and determine a first difference between the expected environmental condition information corresponding to the target cooking scheme and the current environmental condition information; and / or, a second acquisition unit, used to acquire a second food image taken of the food during the cooking process, perform food state detection on the second food image, and obtain current food state information; a first determination unit, used to determine a second difference between the expected food state information corresponding to the target cooking scheme and the current food state information; an adjustment unit, used to adjust parameters of the target cooking scheme based on the first difference and / or the second difference, and control the cooking device to cook the food based on the adjusted target cooking scheme.

[0129] In some optional implementations of this embodiment, the device also includes: a third acquisition module, used to obtain evaluation information of the cooked ingredients in response to the end of the cooking process; a second adjustment module, used to adjust the parameters of the target cooking plan based on the evaluation information, and store the adjusted target cooking plan.

[0130] In some optional implementations of the present embodiment, the third acquisition module includes: a third acquisition unit, which is used to acquire, in response to the end of the cooking process, an image of the food taken of the food after cooking as a third food image; a detection unit, which is used to perform food status detection on the third food image to obtain finished food status information; a second determination unit, which is used to determine the difference between the expected food status information corresponding to the parameters of the target cooking plan and the finished food status information; a first generation unit, which is used to generate first evaluation information of the cooked food based on the difference; and / or a fourth acquisition unit, which is used to acquire second evaluation information input for feedback on the food after cooking; a second generation unit, which is used to generate evaluation information of the cooked food based on the first evaluation information and / or the second evaluation information.

[0131] In some optional implementations of this embodiment, the eating habit model is a deep reinforcement model; the device also includes: a calculation module, which is used to calculate the reward and punishment scores of the evaluation information based on a preset reward and punishment function, and obtain the reward and punishment scores representing the difference between the eating habit type output by the eating habit model and the actual eating habits of the target user; a third adjustment module, which is used to adjust the parameters of the eating habit model based on the reward and punishment scores according to the deep reinforcement learning algorithm.

[0132] In some optional implementations of this embodiment, the device also includes: a fourth acquisition module, used to acquire the properties of currently stored ingredients from the ingredient storage device; a second generation module, used to generate a cooking plan to be determined based on the properties of the ingredients to be cooked and the properties of the currently stored ingredients, and output the cooking plan to be determined; an update module, used to update the initial cooking plan based on the cooking plan to be determined in response to receiving confirmation information after confirming the cooking plan to be determined.

[0133] In some optional implementations of this embodiment, the device also includes: a fifth acquisition module, used to obtain the physical state information of the target user from the user physical state detection device; a fourth adjustment module, used to adjust the parameters of the target cooking plan based on the physical state information so that the adjusted target cooking plan matches the physical state information.

[0134] The automatic cooking device provided in this embodiment can be as follows Fig. 9 The automatic cooking device shown in can execute all the steps of the above automatic cooking methods, thereby achieving the technical effects of the above automatic cooking methods. Please refer to the above related description for details. For the sake of brevity, it will not be repeated here.

[0135] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, Fig.10 The electronic device 1000 shown includes: at least one processor 1001, a memory 1002, at least one network interface 1004 and other user interfaces 1003. The various components in the electronic device 1000 are coupled together via a bus system 1005. It is understood that the bus system 1005 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 1005 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 1005 is not described in detail. Fig.10 Various buses are labeled as bus system 1005 .

[0136] The user interface 1003 may include a display, a keyboard, or a pointing device (eg, a mouse, a trackball, a touch pad, or a touch screen).

[0137] It can be understood that the memory 1002 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus random access memory (DRRAM). The memory 1002 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0138] In some implementations, the memory 1002 stores the following elements, executable units or data structures, or a subset thereof, or an extended set thereof: an operating system 10021 and an application program 10022 .

[0139] The operating system 10021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application 10022 includes various application programs, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. The program for implementing the method of the embodiment of the present application can be included in the application 10022.

[0140] In this embodiment, by calling the program or instruction stored in the memory 1002, specifically, the program or instruction stored in the application 10022, the processor 1001 is used to execute the method steps provided by each method embodiment, for example, including:

[0141] Acquire a first food image photographed of food to be cooked; use a preset food detection model to detect the food type of the first food image to determine the properties of the food; generate an initial cooking plan based on the properties of the food; acquire dietary habit data of a target user; use a preset dietary habit model to analyze the dietary habit data to determine the dietary habit type of the target user; based on the dietary habit type, adjust parameters of the initial cooking plan to obtain a target cooking plan that matches the dietary habit type; based on the target cooking plan, control a cooking device to cook the food.

[0142] The method disclosed in the above embodiment of the present application can be applied to the processor 1001, or implemented by the processor 1001. The processor 1001 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 1001. The above processor 1001 can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software units in the decoding processor can be executed. The software unit can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1002, and the processor 1001 reads the information in the memory 1002 and completes the steps of the above method in combination with its hardware.

[0143] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPDevice, DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the above functions of the present application, or a combination thereof.

[0144] For software implementation, the technology described above in this article can be implemented by a unit that performs the functions described above in this article. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0145] The electronic device provided in this embodiment may be Fig.10 The electronic device shown in can execute all the steps of the automatic cooking methods described above, thereby achieving the technical effects of the automatic cooking methods described above. Please refer to the above related description for details. For the sake of brevity, it will not be repeated here.

[0146] The embodiment of the present application also provides a storage medium (computer-readable storage medium). The storage medium here stores one or more programs. The storage medium may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk or a solid-state drive; the memory may also include a combination of the above-mentioned types of memory.

[0147] When one or more programs in the storage medium can be executed by one or more processors, the automatic cooking method executed on the electronic device side can be implemented.

[0148] The processor is used to execute the program stored in the memory to implement the following steps of the automatic cooking method performed on the electronic device side:

[0149] Acquire a first food image photographed of food to be cooked; use a preset food detection model to detect the food type of the first food image to determine the properties of the food; generate an initial cooking plan based on the properties of the food; acquire dietary habit data of a target user; use a preset dietary habit model to analyze the dietary habit data to determine the dietary habit type of the target user; based on the dietary habit type, adjust parameters of the initial cooking plan to obtain a target cooking plan that matches the dietary habit type; based on the target cooking plan, control a cooking device to cook the food.

[0150] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different circuits to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0151] The steps of the circuits or algorithms described in conjunction with the embodiments disclosed herein may be implemented using hardware, software modules executed by a processor, or a combination of the two. The software modules may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0152] It should be understood that the terms used in the text are only for the purpose of describing specific example embodiments, and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used in the text may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The steps, processes, and operations described in the text are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0153] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. An automatic cooking method, characterized in that: The method comprises: Acquire a first food image captured of the food to be cooked; Using a preset food detection model, perform food type detection on the first food image to determine the attributes of the food; Based on the properties of the ingredients, generating an initial cooking plan; Obtain target users’ eating habits data; Analyze the dietary habit data using a preset dietary habit model to determine the dietary habit type of the target user; Based on the dietary habit type, adjusting the parameters of the initial cooking plan to obtain a target cooking plan that matches the dietary habit type; Based on the target cooking scheme, the cooking device is controlled to cook the ingredients.

2. The method according to claim 1, characterized in that The controlling the cooking device to cook the ingredients based on the target cooking scheme includes: Acquiring current environmental condition information in the cooking device during the cooking process, and determining a first difference between the expected environmental condition information corresponding to the target cooking scheme and the current environmental condition information; and / or, Acquire a second food image taken of the food during the cooking process, perform food state detection on the second food image, and obtain current food state information; determining a second difference between the expected food state information corresponding to the target cooking scheme and the current food state information; Based on the first difference and / or the second difference, the parameters of the target cooking scheme are adjusted, and the cooking device is controlled to cook the ingredients based on the adjusted target cooking scheme.

3. The method according to claim 1, characterized in that After controlling the cooking device to cook the food based on the target cooking scheme, the method further includes: In response to the cooking process being completed, obtaining evaluation information of the cooked food; Based on the evaluation information, parameters of the target cooking scheme are adjusted, and the adjusted target cooking scheme is stored.

4. The method according to claim 3, characterized in that The obtaining of evaluation information of the cooked food includes: In response to the cooking process being completed, acquiring a food image taken of the food after the cooking is completed as a third food image; Performing food state detection on the third food image to obtain finished food state information; Determine the difference between the expected food state information corresponding to the parameters of the target cooking scheme and the finished food state information; Based on the difference, generating first evaluation information of the cooked food; and / or, Acquiring second evaluation information input for providing feedback on the food after cooking is completed; Based on the first evaluation information and / or the second evaluation information, evaluation information of the cooked food is generated.

5. The method according to claim 3, characterized in that: The dietary habit model is a deep reinforcement model; After obtaining the evaluation information of the cooked food, the method further includes: Based on a preset reward and punishment function, a reward and punishment score is calculated for the evaluation information to obtain a reward and punishment score representing the difference between the dietary habit type output by the dietary habit model and the actual dietary habit of the target user; Based on the reward and punishment scores, the parameters of the eating habit model are adjusted according to a deep reinforcement learning algorithm.

6. The method according to claim 1, characterized in that After generating an initial cooking plan based on the properties of the ingredients, the method further includes: Obtaining properties of currently stored food from the food storage device; generating a cooking scheme to be determined based on the properties of the ingredients to be cooked and the properties of the currently stored ingredients, and outputting the cooking scheme to be determined; In response to receiving confirmation information after confirming the cooking scheme to be determined, the initial cooking scheme is updated based on the cooking scheme to be determined.

7. The method according to claim 1, characterized in that After adjusting the parameters of the initial cooking scheme based on the eating habit type to obtain a target cooking scheme matching the eating habit type, the method further includes: Acquire the physical state information of the target user from a user physical state detection device; Based on the body state information, the parameters of the target cooking scheme are adjusted so that the adjusted target cooking scheme matches the body state information.

8. An automatic cooking device, characterized in that: The device comprises: A first acquisition module, used to acquire a first food image taken of the food to be cooked; A detection module, configured to use a preset food detection model to perform food type detection on the first food image to determine the properties of the food; A first generating module, used for generating an initial cooking plan based on the properties of the ingredients; The second acquisition module is used to acquire the dietary habit data of the target user; An analysis module, used to analyze the dietary habit data using a preset dietary habit model to determine the dietary habit type of the target user; A first adjustment module, configured to adjust parameters of the initial cooking scheme based on the dietary habit type to obtain a target cooking scheme matching the dietary habit type; The control module is used to control the cooking equipment to cook the ingredients based on the target cooking plan.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor is used to execute the computer program stored in the memory, and when the computer program is executed, the automatic cooking method described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the automatic cooking method described in any one of claims 1 to 7 is implemented.

Images