Cooking prompting method and device based on menu and electronic equipment

Abstract

The embodiment of the invention discloses a cooking prompting method and device based on a menu and electronic equipment. The method comprises the following steps: determining a currently selected menu to be cooked, obtaining first image information of a cooking area in the cooking equipment, and identifying the volume of each target object in the first image information; calculating a first weight of the target object based on the volume, and optimizing the first weight based on a weight difference between the second weight and the third weight; and after the first weight is optimized, querying a weight interval of each target object in the to-be-cooked menu based on the current cooking step, and generating first prompt information according to a comparison result of the first weight and the weight interval. According to the embodiment of the invention, the user is prompted to adjust the shares of the food materials and / or the seasonings by generating the first prompt information, so that the adding accuracy of the food materials and the seasonings in the cooking equipment is improved, the taste of dishes is improved, and the finally cooked dishes are ensured to conform to the menu.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to intelligent recognition technology, and more particularly to a recipe-based cooking prompt method, device, and electronic device. Background Art

[0002] Existing cooking devices can only provide the function of heating the cooking device according to the selected temperature mode, and the selection of the specific temperature mode and temperature depends on the user's experience. For users with less cooking experience, the users generally cook according to the recipe. However, since most users who cook according to the recipe lack cooking experience, they may make mistakes and put the ingredients or seasonings in the cooking device in a quantity that does not meet the recipe requirements, which ultimately makes the taste of the cooked dish not meet the requirements. Therefore, there is a need for a method for prompting the cooking status in the cooking device when cooking according to the recipe. Summary of the invention

[0003] In order to solve the above problems, one or more embodiments of this specification describe a recipe-based cooking reminder method, device, and electronic device.

[0004] According to a first aspect, a recipe-based cooking prompt method is provided, the method comprising:

[0005] Determine the currently selected recipe to be cooked, obtain first image information of a cooking area in a cooking device, and identify the volume of each target object in the first image information, wherein the target object includes ingredients and seasonings;

[0006] Calculating a first weight of the target object based on the volume, and optimizing the first weight based on a weight difference between a second weight and a third weight, wherein the second weight is detected by a weight sensor in the cooking device, and the third weight is the sum of the first weights;

[0007] After the first weight optimization is completed, the weight range of each target object is queried in the recipe to be cooked based on the current cooking step, and a first prompt information is generated according to the comparison result of the first weight and the weight range. The first prompt information is used to characterize the target object that needs to be adjusted in weight in the cooking equipment and the weight to be adjusted.

[0008] Preferably, the acquiring first image information of the cooking area in the cooking device and identifying the volume of each target object in the first image information includes:

[0009] Continuously acquiring first image information of a cooking area in a cooking device within a preset time period, and performing feature recognition on each of the first image information;

[0010] After integrating the image features of each of the first image information, the object features of each target object are determined, and the volume of each of the target objects is calculated according to the object features.

[0011] Preferably, the optimizing the first weight based on the weight difference between the second weight and the third weight comprises:

[0012] calculating a weight difference between the second weight and the third weight;

[0013] When the weight difference is less than a preset difference, the first weight is not optimized;

[0014] When the weight difference is not less than a preset difference, a weight ratio of each target object is calculated based on each first weight, an estimated weight of each target object is calculated based on the weight ratio and the second weight, the estimated weight is used as the optimized first weight, feedback information is generated based on the first weight before and after optimization, and the feedback information is sent to a preset background terminal.

[0015] Preferably, the method further comprises:

[0016] When the current cooking step is a heating cooking step, recording a weight change curve of the food based on the weight sensor;

[0017] Obtaining an estimated cooking curve of the heating and cooking step, and comparing the estimated cooking curve with a weight change curve to determine the maturity stage of the food;

[0018] After the cooking stage reaches the target stage, a second prompt message is generated and the cooking device is stopped, wherein the second prompt message is used to indicate that cooking is completed.

[0019] Preferably, the method further comprises:

[0020] Acquiring second image information of a processing area in the cooking device;

[0021] When the second image information identifies food and detects food processing instructions, a processing mode corresponding to the food processing instructions is executed.

[0022] Preferably, after identifying the volume of each target object in the image information, the method further includes:

[0023] Determine a cooking combination of ingredients and seasonings, and query a preset database according to the cooking combination to obtain a seasoning absorption ratio;

[0024] After calculating the first weight of the target object based on the volume, the method further includes:

[0025] The first weight corresponding to the seasoning is adjusted based on the seasoning absorption ratio.

[0026] Preferably, the step of querying a preset database according to the cooking combination to obtain the seasoning absorption ratio includes:

[0027] The preset database is queried according to the cooking combination and the food processing method to obtain the seasoning absorption ratio.

[0028] According to a second aspect, a cooking prompt device based on a recipe is provided, the device comprising:

[0029] a determination module, configured to determine a currently selected recipe to be cooked, obtain first image information of a cooking area in a cooking device, and identify the volume of each target object in the first image information, wherein the target objects include ingredients and seasonings;

[0030] a calculation module, configured to calculate a first weight of the target object based on the volume, and optimize the first weight based on a weight difference between a second weight and a third weight, wherein the second weight is detected by a weight sensor in the cooking device, and the third weight is a sum of the first weights;

[0031] A generation module is used to query the weight range of each target object in the recipe to be cooked based on the current cooking step after the first weight optimization is completed, and generate first prompt information according to the comparison result of the first weight and the weight range, wherein the first prompt information is used to characterize the target object that needs to be adjusted in weight in the cooking equipment and the weight to be adjusted.

[0032] According to a third aspect, there is provided an electronic device, comprising a processor and a memory;

[0033] The processor is connected to the memory;

[0034] The memory is used to store executable program code;

[0035] The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the steps of the method provided in the first aspect or any possible implementation manner of the first aspect.

[0036] According to a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and the computer-readable storage medium stores instructions, which, when the instructions are executed on a computer or a processor, cause the computer or the processor to execute the method provided in the first aspect or any possible implementation of the first aspect.

[0037] The method provided in the embodiments of this specification can determine the types and first weights of ingredients and seasonings through image recognition during the cooking process, and after optimizing the first weight according to the second weight, compare the first weight with the weight ranges of each target object corresponding to the current cooking step to generate a first prompt message to prompt the user to adjust the ingredients and / or seasoning portions, thereby improving the accuracy of adding ingredients and seasonings in the cooking equipment, improving the taste of the dishes, and ensuring that the final cooked dishes conform to the recipe. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A schematic diagram of the system architecture of a recipe-based cooking reminder method provided for one embodiment of the present specification.

[0040] Figure 2 It is a flowchart of a recipe-based cooking prompt method in one embodiment of the present specification.

[0041] Figure 3 It is a structural schematic diagram of a recipe-based cooking reminder device in one embodiment of the present specification.

[0042] Figure 4 It is a structural diagram of an electronic device in one embodiment of this specification. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0044] In the following introduction, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. The following introduction provides multiple embodiments of the present application, and different embodiments can be replaced or combined, so the present application can also be considered to include all possible combinations of the same and / or different embodiments recorded. Therefore, if one embodiment includes features A, B, and C, and another embodiment includes features B and D, then the present application should also be considered to include embodiments containing one or more of all other possible combinations of A, B, C, and D, although the embodiment may not be clearly recorded in the following text.

[0045] The following description provides examples and does not limit the scope, applicability or examples set forth in the claims. Changes may be made to the functions and arrangements of the elements described without departing from the scope of the present application. Various processes or components may be appropriately omitted, substituted or added to each example. For example, the described method may be performed in an order different from the order described, and various steps may be added, omitted or combined. In addition, features described in some examples may be combined in other examples.

[0046] See also Figure 1 , Figure 1 A schematic diagram of the system architecture of a recipe-based cooking reminder method provided in an embodiment of this specification is shown.

[0047] like Figure 1 As shown, the system architecture of the recipe-based cooking prompt method may at least include an oven, a range hood and a meat slicer. The oven includes a temperature sensor, a weight sensor, a processor, a recipe database storage module, and a Bluetooth communication module. The range hood includes a visual sensor, a range hood control module, a Bluetooth communication module, and an image recognition database. The meat slicer includes a motor, a meat slicer control module, and a Bluetooth communication module. The range hood is connected to the oven to suck the smoke generated in the oven. The visual sensor set in the range hood can collect the first image information in the oven. The processor can obtain the first image information through the Bluetooth communication module, identify the target object by querying the image recognition database, calculate the first weight, and optimize the first weight according to the second weight of the weight sensor. The processor can also obtain the recipe according to the recipe database storage module, compare the first weight with the weight interval corresponding to the recipe, and generate the first prompt information. When the food needs to be processed by slicing and dicing, the processor can also drive the motor of the meat slicer through the Bluetooth communication module to perform corresponding processing, and when the range hood is connected to the meat slicer, the visual sensor can also obtain the image information at the meat slicer to ensure the safety of the meat cutting process.

[0048] In addition, it should be noted that Figure 1 What is shown is only one system provided by the present invention. In actual applications, other systems may also be selected. For example, a meat slicer may not be needed. For example, the oven may be replaced with a steamer. For example, for an oven that does not produce smoke, the modules in the range hood may be merged into the oven. For example, the image recognition database and recipe database storage modules may be in the range hood as well as in the oven, and so on.

[0049] See also Figure 2 , Figure 2 : is a flow chart of a recipe-based cooking reminder method provided in an embodiment of the present application. In the embodiment of the present application, the method includes:

[0050] S201. Determine a currently selected recipe to be cooked, obtain first image information of a cooking area in a cooking device, and identify the volume of each target object in the first image information.

[0051] Wherein, the target objects include food materials and seasonings.

[0052] The execution subject of the present application may be a processor of a cooking device.

[0053] In an embodiment of the present specification, when a user requires the cooking device to control the cooking process according to a recipe, the user can select a corresponding recipe by interacting with the cooking device. The cooking device can determine the recipe to be cooked currently selected by the user according to the user's selection instruction, and trigger the control process of the cooking process. A visual sensor can be provided in the cooking device, and the visual sensor can collect first image information of a pre-set cooking area. Through image recognition processing of the first image information, each target object placed by the user in the cooking area can be identified, and the volume of each target object can be determined respectively. Among them, the seasoning can include vinegar, oil, black pepper, etc. Before obtaining the first image information, it is also necessary to determine whether the visual sensor is blocked. If it is blocked, it is necessary to generate a warning message and stop collecting the first image information.

[0054] As an example, the process of identifying the volume of the target object according to the first image information may be:

[0055] 1. Image preprocessing. Preprocess the original first image information for subsequent processing. Preprocessing may include resizing, cropping, denoising, etc.

[0056] 2. Food detection. The object detection algorithm is used to identify the ingredients and seasonings in the preprocessed first image information by identifying the specific object bounding box, and the ingredients and seasonings are classified. The object detection algorithm can specifically use Faster R-CNN, etc. The neural network model is trained according to the selected algorithm and pre-labeled high-quality image data, and then the trained neural network model can be directly used for detection. For the neural network model for identifying seasonings, since seasonings are more difficult to identify, when training the model, in addition to using sample images containing only seasonings, sample images of smears or sprinkled on ingredients are also required to improve the accuracy of seasoning recognition.

[0057] 3. Ingredient segmentation. After the ingredients and seasonings are identified through the trained neural network model, the GrabCut algorithm can be used to segment the ingredients and seasonings from the image background through feature extraction, and determine the boundaries, shapes, sizes and other information of the ingredients and seasonings.

[0058] 4. Size measurement. More than one visual sensor may be provided to obtain first image information from different angles, and a three-dimensional model of the food is reconstructed through the multi-view information to determine the size. In addition, if the visual sensor is a depth camera that can provide depth information, the three-dimensional size of the food may also be determined directly in combination with the depth information.

[0059] 5. Volume calculation: Perform geometric calculations based on the size data of the ingredients and seasonings obtained in the previous steps to determine the volume of the ingredients and seasonings.

[0060] In one possible implementation manner, the acquiring first image information of a cooking area in a cooking device and identifying the volume of each target object in the first image information includes:

[0061] Continuously acquiring first image information of a cooking area in a cooking device within a preset time period, and performing feature recognition on each of the first image information;

[0062] After integrating the image features of each of the first image information, the object features of each target object are determined, and the volume of each of the target objects is calculated according to the object features.

[0063] In the embodiment of the present specification, if the cooking device is a frying pan, oven, etc., cooking may generate smoke and other fumes during the cooking process, and these fumes may block the food or seasoning, resulting in the first image information obtained not being able to fully include the full picture of the food or seasoning, thereby affecting the accuracy of the image recognition result. Therefore, the processor will continue to obtain the first image information within a relatively short preset time. Since the equipment such as the range hood in the cooking device will continue to suck the smoke, the position of the smoke will continue to change. By continuously obtaining multiple first image information, the position distribution of the smoke in each first image information is different, that is, the position of the food or seasoning blocked is also different. In this way, by integrating the image features of each first image information, all the object features of the target object can be integrated, and then the volume is calculated based on the object features, so as to avoid the incomplete object features obtained based on a single image information due to the occlusion of smoke, thereby causing volume estimation errors.

[0064] S202: Calculate a first weight of the target object based on the volume, and optimize the first weight based on a weight difference between the second weight and the third weight.

[0065] The second weight is detected by a weight sensor in the cooking device, and the third weight is the sum of the first weights.

[0066] In the embodiment of the present specification, when the volume is determined in the aforementioned step, the type of each target object has been identified according to the corresponding characteristic data, and the type refers not only to food or seasoning, but also to the specific type of food and seasoning. The density corresponding to each type of object is known, and a database can be pre-set, determined by querying the database, and then the first weight of each target object can be calculated by volume and density. There may still be errors in the image recognition method. In order to verify the accuracy of the first weight, the second weight detected by the weight sensor is also obtained. Since the second weight detects the total weight of all food and seasoning, when comparing, the sum of all first weights, that is, the third weight, should be used to compare with the second weight. Considering the error, the second weight should be basically the same as the third weight. If the difference between the two is large, it means that the recognition of the first weight is not accurate enough. Considering that the second weight is directly weighed, its accuracy is guaranteed, but the second weight cannot reflect the weight of a single target object, so at this time, the first weight will be optimized by the second weight to avoid using the first weight with a large error for subsequent processing. There are many ways to optimize the first weight according to the second weight, for example, the ratio between the first weights can be determined, and then the second weight can be divided according to the ratio, and the divided value can be used as the optimized first weight. It can also be calculated that the difference between the second weight and the third weight is divided equally according to the number of target objects and then added to each first weight, and other implementation methods can also be used.

[0067] In one embodiment, the optimizing the first weight based on the weight difference between the second weight and the third weight includes:

[0068] calculating a weight difference between the second weight and the third weight;

[0069] When the weight difference is less than a preset difference, the first weight is not optimized;

[0070] When the weight difference is not less than a preset difference, a weight ratio of each target object is calculated based on each first weight, an estimated weight of each target object is calculated based on the weight ratio and the second weight, the estimated weight is used as the optimized first weight, feedback information is generated based on the first weight before and after optimization, and the feedback information is sent to a preset background terminal.

[0071] In the embodiment of this specification, the processor will calculate the weight difference between the second weight and the third weight. If the weight difference is less than the preset difference, it means that the difference between the two is not large and is within an acceptable error range. At this time, the first weight will not be optimized. If the weight difference is not less than the preset difference, it means that the difference between the two is relatively obvious, and the first weight is not accurate enough, which will affect the accuracy of the subsequent calculation results. At this time, the weight ratio between each first weight will be determined, and the second weight will be allocated with the weight ratio to obtain the estimated weight of each target object, and the estimated weight will be used as the first weight after optimization. The first weight obtained in this way will be more accurate than before optimization, and it is not easy to cause a large deviation in the subsequent calculation results due to a large error. In addition, the processor will also generate feedback information based on the first weight before and after optimization, and send the feedback information to the corresponding background terminal to provide feedback on abnormal data to the R&D enterprise, so that the R&D enterprise can further develop and optimize the image recognition model based on the feedback information to improve the recognition accuracy of the model for this type of food and / or seasoning.

[0072] S203: After the first weight is optimized, the weight range of each target object is queried in the recipe to be cooked based on the current cooking step, and a first prompt message is generated according to a comparison result between the first weight and the weight range.

[0073] The first prompt information is used to represent the target object that needs to be adjusted in weight in the cooking device and the weight to be adjusted.

[0074] In the embodiment of the present specification, the corresponding weight interval is set in advance for each target object in each cooking step in the recipe to be cooked. The weight interval is a weight range set considering the error. As long as it is within this range, it is considered that the taste of the cooked food will not have obvious deviation. After the first weight is optimized, the corresponding weight interval is obtained according to the current cooking step, and the first weight and the weight interval are compared to determine whether the first weight of the target object exceeds the weight interval, and a first prompt message is generated. The user is prompted according to the first prompt message, so that the user is prompted in time after the amount of ingredients or seasonings has problems, so as to avoid the user finding problems with the taste of the dish when the cooking is completed and tasted. The first prompt message can represent the target object that needs to adjust the weight, that is, the target object whose first weight exceeds the corresponding weight interval, and the weight to be adjusted is displayed according to the difference between the first weight and the weight interval. If all the first weights are within the weight interval, the first prompt message will indicate that there is no target object that needs to be adjusted.

[0075] When there is a target object that needs to adjust the weight, after the adjustment is completed, the third image information is obtained for secondary recognition to determine whether the type and / or weight of the re-added ingredients or seasonings meet the requirements. If not, the first prompt message will continue to be sent to prompt the user to continue adjusting until the requirements are met before proceeding to the next step of cooking.

[0076] In addition, the current cooking step may not be the first step, that is, after some ingredients have been preliminarily cooked, new ingredients and / or seasonings need to be added to this step for cooking. In this case, the identification of ingredients and the comparison of weights will become more complicated and need to be adjusted accordingly. For example, the current image information and weight information of the cooking area may be recorded before the user adds ingredients or seasonings, and then the image information and weight information are obtained again after the user has added the ingredients for comparison, so as to determine whether the first weight of the newly added ingredients and seasonings in the current cooking step meets the weight range. For example, the first weight of all ingredients and seasonings in the cooking area can also be identified, but the weight of ingredients and seasonings not added in the current cooking step needs to be adjusted according to the estimated cooking curve pre-stored in the processor, so as to take into account the evaporation of water or the change of properties of the ingredients after heating, and avoid weight estimation errors. The estimated cooking curve can be a weight change curve of the ingredients during the cooking and heating process constructed in advance based on test data.

[0077] In one embodiment, the method further comprises:

[0078] When the current cooking step is a heating cooking step, recording a weight change curve of the food based on the weight sensor;

[0079] Obtaining an estimated cooking curve of the heating and cooking step, and comparing the estimated cooking curve with a weight change curve to determine the maturity stage of the food;

[0080] After the cooking stage reaches the target stage, a second prompt message is generated and the cooking device is stopped, wherein the second prompt message is used to indicate that cooking is completed.

[0081] In the embodiment of the present specification, if the current cooking step is a heating step, the processor will also continuously obtain the weight information of the ingredients through the weight sensor to record and generate the weight change curve of the ingredients. In addition, the cooking recipe will pre-store the estimated cooking curve of the ingredients in the heating cooking step, that is, the theoretical weight change curve of the ingredients under the specified heating conditions, and the curve can also be marked with the maturity stage of the ingredients at different weights. By matching the two curves, the current maturity stage of the ingredients can be determined. If the maturity stage of the ingredients reaches the target stage, it is considered that the ingredients are cooked. At this time, a second prompt message will be generated to inform the user that cooking is complete, and the cooking equipment will be stopped at the same time to avoid overcooking.

[0082] In one embodiment, the method further comprises:

[0083] Acquiring second image information of a processing area in the cooking device;

[0084] When the second image information identifies food and detects food processing instructions, a processing mode corresponding to the food processing instructions is executed.

[0085] In the embodiments of the present specification, the cooking device may be an integrated food processor, which may include a processing area corresponding to food processing equipment such as a meat slicer in addition to the cooking area. The processor will also obtain second image information of the processing area. If the food is identified in the second image information and the food processing instruction is detected, it means that the user needs to perform processing such as shredding and slicing on the food, and the corresponding processing mode will be selected and executed according to the food processing instruction. In addition, in order to ensure the safety of the processing process, the user's hands and other parts may be identified by infrared sensors and the like. If the hand is identified to be close to the processing area during the processing, the processing operation may be stopped.

[0086] In one possible implementation manner, after identifying the volume of each target object in the image information, the method further includes:

[0087] Determine a cooking combination of ingredients and seasonings, and query a preset database according to the cooking combination to obtain a seasoning absorption ratio;

[0088] After calculating the first weight of the target object based on the volume, the method further includes:

[0089] The first weight corresponding to the seasoning is adjusted based on the seasoning absorption ratio.

[0090] In the embodiment of the present specification, under a specific cooking combination (such as meat and salt), part of the seasoning is absorbed and infiltrated into the food after contacting the food. This part of the seasoning cannot be identified only by image recognition, which will lead to inaccurate weight recognition results. Therefore, a database is pre-stored in the processor, and the cooking combinations with seasoning absorption are set in the database, and the seasoning absorption ratio corresponding to each cooking combination is determined based on pre-test data. After determining the cooking combination composed of the target object, a query will be made in the database. If the seasoning absorption ratio can be queried, the first weight of the seasoning will be adjusted according to the seasoning absorption ratio to ensure the accuracy of the first weight.

[0091] In one possible implementation, querying a preset database according to the cooking combination to obtain the seasoning absorption ratio includes:

[0092] The preset database is queried according to the cooking combination and the food processing method to obtain the seasoning absorption ratio.

[0093] In the embodiments of this specification, the seasoning absorption ratio of the same type of food under different food processing methods (for example, meat is cut into chunks and slices) is also different. Therefore, when setting up the database, different seasoning absorption ratios should be mapped and set according to different cooking combinations and different food processing methods, rather than just setting a fixed seasoning absorption ratio according to the cooking combination. This can further ensure that the final first weight of the seasoning is accurate.

[0094] The following will be combined with the attached Figure 3 , a cooking reminder device based on recipes provided in an embodiment of the present application is introduced in detail. It should be noted that the attached Figure 3 The cooking reminder device based on the recipe is used to implement the present application Figure 2 For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the present application. Figure 2 The embodiment shown.

[0095] See also Figure 3 , Figure 3 Schematic diagram of a cooking reminder device based on a recipe provided in an embodiment of the present application. Figure 3 As shown, the device comprises:

[0096] The determination module 301 is used to determine the currently selected recipe to be cooked, obtain first image information of a cooking area in a cooking device, and identify the volume of each target object in the first image information, wherein the target object includes ingredients and seasonings;

[0097] a calculation module 302, configured to calculate a first weight of the target object based on the volume, and optimize the first weight based on a weight difference between a second weight and a third weight, wherein the second weight is detected by a weight sensor in the cooking device, and the third weight is a sum of the first weights;

[0098] The generation module 303 is used to query the weight range of each target object in the recipe to be cooked based on the current cooking step after the first weight optimization is completed, and generate a first prompt information according to the comparison result of the first weight and the weight range, wherein the first prompt information is used to characterize the target object that needs to be adjusted in weight in the cooking equipment and the weight to be adjusted.

[0099] In one possible implementation, the determination module 301 is specifically configured to:

[0100] Continuously acquiring first image information of a cooking area in a cooking device within a preset time period, and performing feature recognition on each of the first image information;

[0101] After integrating the image features of each of the first image information, the object features of each target object are determined, and the volume of each of the target objects is calculated according to the object features.

[0102] In one possible implementation, the calculation module 302 is specifically used for:

[0103] calculating a weight difference between the second weight and the third weight;

[0104] When the weight difference is less than a preset difference, the first weight is not optimized;

[0105] When the weight difference is not less than a preset difference, a weight ratio of each target object is calculated based on each first weight, an estimated weight of each target object is calculated based on the weight ratio and the second weight, the estimated weight is used as the optimized first weight, feedback information is generated based on the first weight before and after optimization, and the feedback information is sent to a preset background terminal.

[0106] In one embodiment, the device further comprises:

[0107] A recording module, configured to record a weight change curve of the food material based on the weight sensor when the current cooking step is a heating cooking step;

[0108] A comparison module, used for obtaining an estimated cooking curve of the heating and cooking step, and comparing the estimated cooking curve with a weight change curve to determine the maturity stage of the food;

[0109] The first control module is used to generate a second prompt message and stop the cooking device after the cooking stage reaches a target stage, wherein the second prompt message is used to indicate that cooking is completed.

[0110] In one embodiment, the device further comprises:

[0111] An acquisition module, used for acquiring second image information of a processing area in the cooking device;

[0112] The second control module is used to execute a processing mode corresponding to the food processing instruction when the second image information recognizes food and detects the food processing instruction.

[0113] In one possible implementation, the determination module 301 is further specifically configured to:

[0114] Determine a cooking combination of ingredients and seasonings, and query a preset database according to the cooking combination to obtain a seasoning absorption ratio;

[0115] The calculation module 302 is further specifically used for:

[0116] The first weight corresponding to the seasoning is adjusted based on the seasoning absorption ratio.

[0117] In one possible implementation, the determination module 301 is further specifically configured to:

[0118] The preset database is queried according to the cooking combination and the food processing method to obtain the seasoning absorption ratio.

[0119] Those skilled in the art can clearly understand that the technical solutions of the embodiments of the present application can be implemented with the help of software and / or hardware. The "unit" and "module" in this specification refer to software and / or hardware that can independently complete or cooperate with other components to complete specific functions, where the hardware can be, for example, a field programmable gate array (Field-Programmable Gate Array, FPGA), an integrated circuit (Integrated Circuit, IC), etc.

[0120] Each processing unit and / or module of the embodiments of the present application may be implemented by an analog circuit that implements the functions described in the embodiments of the present application, or may be implemented by software that executes the functions described in the embodiments of the present application.

[0121] See also Figure 4 , which shows a schematic diagram of the structure of an electronic device involved in an embodiment of the present application, the electronic device can be used to implement Figure 2 The method in the embodiment shown. Figure 4As shown, the electronic device 400 may include: at least one processor 401 , at least one network interface 404 , a user interface 403 , a memory 405 , and at least one communication bus 402 .

[0122] The communication bus 402 is used to realize the connection and communication between these components.

[0123] The user interface 403 may include a display screen (Display) and a camera (Camera), and the optional user interface 403 may also include a standard wired interface and a wireless interface.

[0124] The network interface 404 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0125] Among them, the processor 401 may include one or more processing cores. The processor 401 uses various interfaces and lines to connect various parts within the entire electronic device 400, and executes various functions and processes data of the electronic device 400 by running or executing instructions, programs, code sets or instruction sets stored in the memory 405, and calling data stored in the memory 405. Optionally, the processor 401 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 401 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics central processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 401, and it can be implemented separately through a chip.

[0126] Among them, the memory 405 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 405 includes a non-transitory computer-readable storage medium. The memory 405 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 405 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 405 may optionally be at least one storage device located away from the aforementioned processor 401. As Figure 4 As shown, the memory 405 as a computer storage medium may include an operating system, a network communication module, a user interface module, and program instructions.

[0127] exist Figure 4 In the electronic device 400 shown, the user interface 403 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 401 can be used to call the recipe-based cooking reminder application stored in the memory 405, and specifically perform the following operations:

[0128] Determine the currently selected recipe to be cooked, obtain first image information of a cooking area in a cooking device, and identify the volume of each target object in the first image information, wherein the target object includes ingredients and seasonings;

[0129] Calculating a first weight of the target object based on the volume, and optimizing the first weight based on a weight difference between a second weight and a third weight, wherein the second weight is detected by a weight sensor in the cooking device, and the third weight is the sum of the first weights;

[0130] After the first weight optimization is completed, the weight range of each target object is queried in the recipe to be cooked based on the current cooking step, and a first prompt information is generated according to the comparison result of the first weight and the weight range. The first prompt information is used to characterize the target object that needs to be adjusted in weight in the cooking equipment and the weight to be adjusted.

[0131] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0132] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0133] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0134] In the several embodiments provided in the present application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0135] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0136] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0137] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program code.

[0138] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by entering a program to instruct the relevant hardware. The program may be stored in a computer-readable memory, and the memory may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0139] The above is only an exemplary embodiment of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the specification and practicing the disclosure here, those skilled in the art will easily think of the implementation scheme of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field not recorded in the present disclosure. The description and examples are regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A cooking reminder method based on recipes, characterized in that: The method comprises: Determine the currently selected recipe to be cooked, obtain first image information of a cooking area in a cooking device, and identify the volume of each target object in the first image information, wherein the target object includes ingredients and seasonings; Calculating a first weight of the target object based on the volume, and optimizing the first weight based on a weight difference between a second weight and a third weight, wherein the second weight is detected by a weight sensor in the cooking device, and the third weight is the sum of the first weights; After the first weight optimization is completed, the weight range of each target object is queried in the recipe to be cooked based on the current cooking step, and a first prompt information is generated according to the comparison result of the first weight and the weight range. The first prompt information is used to characterize the target object that needs to be adjusted in weight in the cooking equipment and the weight to be adjusted.

2. The method according to claim 1, characterized in that The step of acquiring first image information of a cooking area in the cooking device and identifying the volume of each target object in the first image information includes: Continuously acquiring first image information of a cooking area in a cooking device within a preset time period, and performing feature recognition on each of the first image information; After integrating the image features of each of the first image information, the object features of each target object are determined, and the volume of each of the target objects is calculated according to the object features.

3. The method according to claim 1, characterized in that The optimizing the first weight based on the weight difference between the second weight and the third weight includes: calculating a weight difference between the second weight and the third weight; When the weight difference is less than a preset difference, the first weight is not optimized; When the weight difference is not less than a preset difference, a weight ratio of each target object is calculated based on each first weight, an estimated weight of each target object is calculated based on the weight ratio and the second weight, the estimated weight is used as the optimized first weight, feedback information is generated based on the first weight before and after optimization, and the feedback information is sent to a preset background terminal.

4. The method according to claim 1, characterized in that: The method further comprises: When the current cooking step is a heating cooking step, recording a weight change curve of the food based on the weight sensor; Obtaining an estimated cooking curve of the heating and cooking step, and comparing the estimated cooking curve with a weight change curve to determine the maturity stage of the food; After the cooking stage reaches the target stage, a second prompt message is generated and the cooking device is stopped, wherein the second prompt message is used to indicate that cooking is completed.

5. The method according to claim 1, characterized in that The method further comprises: Acquiring second image information of a processing area in the cooking device; When the second image information identifies food and detects food processing instructions, a processing mode corresponding to the food processing instructions is executed.

6. The method according to claim 1, characterized in that After identifying the volume of each target object in the image information, the method further includes: Determine a cooking combination of ingredients and seasonings, and query a preset database according to the cooking combination to obtain a seasoning absorption ratio; After calculating the first weight of the target object based on the volume, the method further includes: The first weight corresponding to the seasoning is adjusted based on the seasoning absorption ratio.

7. The method according to claim 6, characterized in that The step of querying a preset database according to the cooking combination to obtain the seasoning absorption ratio includes: The preset database is queried according to the cooking combination and the food processing method to obtain the seasoning absorption ratio.

8. A cooking reminder device based on recipes, characterized in that: The device comprises: a determination module, configured to determine a currently selected recipe to be cooked, obtain first image information of a cooking area in a cooking device, and identify the volume of each target object in the first image information, wherein the target objects include ingredients and seasonings; a calculation module, configured to calculate a first weight of the target object based on the volume, and optimize the first weight based on a weight difference between a second weight and a third weight, wherein the second weight is detected by a weight sensor in the cooking device, and the third weight is a sum of the first weights; A generation module is used to query the weight range of each target object in the recipe to be cooked based on the current cooking step after the first weight optimization is completed, and generate first prompt information according to the comparison result of the first weight and the weight range, wherein the first prompt information is used to characterize the target object that needs to be adjusted in weight in the cooking equipment and the weight to be adjusted.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, wherein the computer-readable storage medium has instructions stored therein, and when the instructions are executed on a computer or a processor, the computer or the processor executes the steps of the method according to any one of claims 1 to 7.

Images