Cooking auxiliary processing method and device and electronic equipment
Through the cooking auxiliary method combined with image recognition and sensor, the problem of inaccurate material preparation during the cooking process is solved, ensuring the improvement of the taste of the dishes, and achieving standardization of the cooking process and improving user experience.
Patent Information
- Application Number
- CN202411236422.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2024-11-26
AI Technical Summary
During the cooking process, it is difficult for users to prepare ingredients and seasonings accurately, resulting in poor taste of dishes, and the existing smart recipe guidance is insufficient to solve this problem.
Identify the type and weight of cooking materials through image acquisition equipment, compare with recipe requirements, generate adjustment information to ensure that the material preparation complies with specifications, and optimize the cooking process with weight sensors and temperature sensors.
Ensure the accuracy of the cooking materials and the standardization of the cooking process, and improve the taste and cooking experience of the final dish.
Smart Images

Figure CN119027939A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of intelligent kitchen appliances, and in particular to a cooking auxiliary processing method, device and electronic equipment. Background Art
[0002] With the improvement of people's living standards and the development of science and technology, people's demand for the intelligence of daily household appliances has also increased. As an indispensable household appliance in daily life, the degree of intelligence of kitchen appliances has a great impact on people's quality of life. In addition, with the improvement of people's healthy eating concept, people prefer to cook at home. However, food cooking is a relatively complicated process, which may cause great inconvenience due to inability to cook or lack of cooking experience.
[0003] In order to solve the above problems, there are related technologies in the prior art that can generate intelligent recipes. Based on the guidance of the intelligent recipes, the amount of ingredients and seasonings required for various dishes can be determined, and people can cook smoothly according to the intelligent recipes. However, in actual operation, people are often unable to prepare ingredients and seasonings completely according to the guidance, resulting in unsatisfactory dishes in the end. Summary of the invention
[0004] The purpose of the present invention includes, for example, providing a cooking auxiliary processing method, device and electronic device, which can ensure that the user's cooking materials accurately meet the regulations and ensure the taste of the final dish.
[0005] The embodiments of the present invention can be implemented as follows:
[0006] In a first aspect, the present invention provides a cooking auxiliary processing method, the method comprising:
[0007] Acquiring images of cooking materials in each target container by an image acquisition device, wherein the cooking materials include ingredients and seasonings;
[0008] Recognizing the image of each cooking material to determine the type of each cooking material;
[0009] Obtaining the weight of each of the cooking materials;
[0010] Comparing the type of each cooking ingredient with the type of cooking ingredient required in the target recipe determined based on the user's selection to obtain a type comparison result, and comparing the weight of each cooking ingredient with the weight of the corresponding cooking ingredient in the target recipe to obtain a weight comparison result;
[0011] Based on the type comparison result and the weight comparison result, it is determined whether to generate adjustment information; if the adjustment information is generated, the adjustment information is fed back to the user.
[0012] In an optional embodiment, the step of obtaining the weight of each cooking material comprises:
[0013] For each of the cooking materials, identifying the image of the cooking material to obtain a first weight;
[0014] obtaining a second weight of the cooking material based on a detection result of the weight sensor;
[0015] The weight of the cooking material is obtained according to the first weight and the second weight.
[0016] In an optional embodiment, the step of obtaining the weight of the cooking material according to the first weight and the second weight includes:
[0017] Calculating a difference between the first weight and the second weight, and detecting whether a ratio between the difference and the second weight is less than or equal to a preset threshold;
[0018] If it is less than or equal to the preset threshold, taking the first weight as the weight of the cooking material;
[0019] If it is greater than the preset threshold, the weight of the cooking material is calculated based on the first weight and the second weight according to a preset formula.
[0020] In an optional embodiment, the step of obtaining the weight of each cooking material comprises:
[0021] For each image of the cooking material, identifying the image to obtain shape features of the cooking material;
[0022] Obtaining the volume of the cooking material according to the shape feature;
[0023] The density of the cooking material is obtained based on the type of the cooking material, and the weight of the cooking material is calculated according to the density and the volume.
[0024] In an optional embodiment, the step of determining the type of each cooking material includes:
[0025] For each image of the cooking material, importing the image into a pre-trained type recognition model for processing, wherein the type recognition model is pre-trained based on a plurality of cooking material samples;
[0026] The texture information and color information of the cooking material in the image are obtained based on the type recognition model, and the type of the cooking material is determined based on the texture information and the color information.
[0027] In an optional embodiment, the method further comprises:
[0028] capturing, by the image capturing device, an image of cooking materials in the target cooking device during the cooking process;
[0029] The image of the cooking material in the target cooking device is recognized, and whether the cooking process is completed is determined based on the recognition result. If it is determined that the cooking process is completed, a prompt message is issued.
[0030] In an optional embodiment, the recognition result includes a color recognition result, and the step of judging whether the cooking process is completed according to the obtained recognition result includes:
[0031] Obtaining pre-stored standard color information, wherein the standard color information includes color information corresponding to a plurality of cooking stages in a cooking process;
[0032] The color recognition result is compared with the standard color information to determine whether the color recognition result is consistent with the color information of the mature stage in the multiple cooking stages. If they are consistent, it is determined that the cooking process has been completed.
[0033] In an optional embodiment, the method further comprises:
[0034] The real-time temperature value of the target cooking device during the cooking process is collected by a temperature sensor;
[0035] Comparing the real-time temperature value with pre-stored standard temperature information to determine whether the real-time temperature value matches the standard temperature value of the corresponding cooking stage, wherein the standard temperature information includes standard temperature values corresponding to a plurality of cooking stages during the cooking process;
[0036] If the real-time temperature value does not match the standard temperature value of the corresponding cooking stage, the gear position of the cooking heating device is adjusted.
[0037] In a second aspect, the present invention provides a cooking auxiliary processing device, the device comprising:
[0038] A collection module, used for collecting images of cooking materials in each target container through an image collection device, wherein the cooking materials include ingredients and seasonings;
[0039] A recognition module, used to recognize the image of each cooking material and determine the type of each cooking material;
[0040] An obtaining module, used for obtaining the weight of each cooking material;
[0041] a comparison module, configured to compare the type of each cooking material with the type of cooking material required in the target recipe determined based on the user's selection to obtain a type comparison result, and to compare the weight of each cooking material with the weight of the corresponding cooking material in the target recipe to obtain a weight comparison result;
[0042] A judgment module is used to judge whether to generate adjustment information based on the type comparison result and the weight comparison result, and if the adjustment information is generated, feed back the adjustment information to the user.
[0043] In a third aspect, the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, and when the processor executes the computer program, the steps of the method described in any one of the aforementioned implementation modes are implemented.
[0044] The beneficial effects of the embodiments of the present invention include, for example:
[0045] The present invention provides a cooking auxiliary processing method, device and electronic device, which acquire images of cooking materials in each target container through an image acquisition device, identify the images of each cooking material to determine the type of each cooking material, and obtain the weight of each cooking material. The type of each cooking material is compared with the type of cooking material required in the target recipe determined based on the user's selection to obtain a type comparison result, and the weight of each cooking material is compared with the weight of the corresponding cooking material in the target recipe to obtain a weight comparison result. Based on the type comparison result and the weight comparison result, it is determined whether to generate adjustment information, and if the adjustment information is generated, the adjustment information is fed back to the user. This solution can compare the cooking materials actually prepared by the user with the cooking materials required by the recipe, so as to ensure that the cooking materials prepared by the user meet the regulations and guarantee the taste of the final dish. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.
[0047] Figure 1 A flowchart of a cooking auxiliary processing method provided by an embodiment of the present invention;
[0048] Figure 2 for Figure 1 A flowchart of the sub-steps included in S12;
[0049] Figure 3 for Figure 1Another flow chart of the sub-steps included in S12;
[0050] Figure 4 for Figure 1 Another flow chart of the sub-steps included in S12;
[0051] Figure 5 for Figure 4 A flowchart of the sub-steps included in S125B;
[0052] Figure 6 A flow chart of a temperature adjustment method in a cooking auxiliary processing method provided by an embodiment of the present invention;
[0053] Figure 7 A flowchart of a method for determining maturity in a cooking auxiliary processing method provided by an embodiment of the present invention;
[0054] Figure 8 for Figure 7 A flowchart of the sub-steps included in S32;
[0055] Fig. 9 A functional module block diagram of a cooking auxiliary processing device provided by an embodiment of the present invention;
[0056] Fig.10 This is a structural block diagram of an electronic device provided by an embodiment of the present invention.
[0057] Icons: 10 - cooking auxiliary processing device; 101 - acquisition module; 102 - identification module; 103 - acquisition module; 104 - comparison module; 105 - judgment module; 20 - electronic device; 201 - processor; 202 - memory. DETAILED DESCRIPTION
[0058] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.
[0059] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0060] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0061] In the description of the present invention, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. appear to indicate an orientation or position relationship, they are based on the orientation or position relationship shown in the accompanying drawings, or are the orientation or position relationship in which the product of the invention is usually placed when used. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.
[0062] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.
[0063] See also Figure 1 , is a flow chart of a cooking assistance processing method provided by an embodiment of the present invention, the cooking assistance processing method can be implemented by a cooking assistance processing device, the cooking assistance processing device can be implemented by software and / or hardware, and can be configured in an electronic device, the electronic device can be a controller or a terminal device such as a smart phone, a tablet computer, etc. The detailed steps of the cooking assistance processing method are described as follows.
[0064] S11, acquiring images of cooking materials in each target container through an image acquisition device, wherein the cooking materials include ingredients and seasonings.
[0065] S12, identifying the image of each cooking ingredient, determining the type of each cooking ingredient, and obtaining the weight of each cooking ingredient.
[0066] S13, comparing the type of each cooking ingredient with the type of cooking ingredient required in the target recipe determined based on the user's selection to obtain a type comparison result, and comparing the weight of each cooking ingredient with the weight of the corresponding cooking ingredient in the target recipe to obtain a weight comparison result.
[0067] S14, judging whether to generate adjustment information based on the type comparison result and the weight comparison result, and if adjustment information is generated, feeding back the adjustment information to the user.
[0068] In this embodiment, the image acquisition device may be a camera, a video camera, or the like, and the image acquisition device is disposed above the stove and can be used to capture images of items on the stove. For example, the image acquisition device may be disposed on a range hood, or the image acquisition device may be disposed on an independent bracket, as long as it can capture images of items on the stove.
[0069] When the user wants to cook, he can select the desired recipe based on his needs. For example, the stove includes a touch panel, which can display multiple recipes. The user can select from the multiple recipes, and the target recipe can be determined based on the user's selection. Alternatively, multiple recipes can be displayed on the terminal device held by the user, and the target recipe can be determined based on the user's operation on the terminal device.
[0070] In this embodiment, for each recipe, cooking information related to each recipe is pre-stored, including information such as the type of cooking materials (including ingredients and seasonings) required, the weight of each cooking material, the cooking time required, the cooking temperature required, etc. The cooking information of the recipe can be stored in a storage module, which can be located in a stove, a range hood, a terminal device, or a backend server.
[0071] After determining the target recipe, cooking information related to the target recipe can be obtained. The cooking information related to the target recipe is displayed for the user to understand. For example, it can be displayed through a touch panel on the cooker or through a terminal device held by the user.
[0072] During implementation, the user can prepare the cooking materials according to the types and weights of the cooking materials required in the cooking information, and place the prepared cooking materials in the target container on the stove. Generally, a dish often requires multiple cooking materials, and the weight required for each cooking material is different. Therefore, during implementation, each cooking material can be placed separately, that is, each cooking material is placed in a target container.
[0073] After placement is completed and before the cooking process is started, the image acquisition device can acquire images of the cooking materials in each target container, including images of each ingredient and each seasoning. The image acquisition device can communicate with a controller or a terminal device, for example, by communicating with the controller via a Bluetooth module, or by communicating with the terminal device via wireless communication. The image acquisition device sends the acquired images to the controller or the terminal device.
[0074] By performing recognition processing on the image of each cooking material, the type of each cooking material can be determined, and the weight of each cooking material can be obtained.
[0075] The target recipe has requirements for the types of cooking materials required and the weight of each cooking material, so the types of cooking materials actually identified can be compared with the types of cooking materials required to obtain a type comparison result. The type comparison result can be, for example, whether the types of cooking materials actually identified are completely consistent with the types of cooking materials required. If they are not completely consistent, the specific missing types or redundant types can be determined.
[0076] In addition, the weight of the identified cooking material is compared with the weight of the required cooking material to obtain a weight comparison result. The weight comparison here mainly compares the weight of the identified cooking material with the weight of the required cooking material of the same type. For example, if the required cooking material includes cabbage and the identified cooking material also includes cabbage, the weight of the identified cabbage is compared with the weight of cabbage required in the target recipe.
[0077] The weight comparison result obtained can be, for example, weight that is heavier, weight that is lighter, a specific amount that is heavier, a specific amount that is lighter, etc.
[0078] Based on the obtained type comparison results and weight comparison results, it is determined whether adjustment information needs to be generated. For example, the type comparison result is detected to indicate whether the identified cooking materials are completely consistent with the types of the required cooking materials, and the weight comparison result is detected to indicate whether the difference in weight between each cooking material and the required cooking materials is within a preset range. If both conditions are met, there is no need to generate adjustment information, and the user can be prompted to continue the subsequent cooking process. If any of the two conditions is not met, adjustment information needs to be generated to prompt the user to make adjustments. Adjustment information includes the need to add a certain cooking material, the need to remove a certain cooking material, the need to increase the amount of a certain cooking material, the need to reduce the amount of a certain cooking material, etc.
[0079] After the user adjusts the cooking materials based on the adjustment information, the above-mentioned image recognition processing process may be performed again until the above-mentioned two conditions are met, and then the user may be prompted to continue the subsequent cooking process.
[0080] The cooking assistance processing method provided in this embodiment obtains the type and weight of the cooking materials actually prepared by the user through image recognition, and compares the type and weight of the cooking materials actually recognized with the type and weight of the cooking materials required for the target recipe selected by the user to ensure that the user prepares cooking materials that meet the regulations, thereby ensuring the taste of the final dish.
[0081] In this embodiment, please refer to Figure 2 The above step of identifying the image of each cooking material and determining the type of each cooking material may include the following sub-steps:
[0082] S121, for each cooking ingredient image, import the image into a pre-trained category recognition model for processing, where the category recognition model is pre-trained based on a plurality of cooking ingredient samples.
[0083] S122, obtaining texture information and color information of the cooking material in the image based on the type recognition model, and determining the type of the cooking material based on the texture information and the color information.
[0084] In this embodiment, the type recognition model is obtained by training based on a plurality of cooking material samples in advance, for example, the type recognition model can be obtained by training based on the constructed neural network model. The plurality of cooking material samples are image samples of a plurality of cooking materials.
[0085] Considering that images of different ingredients and different seasonings often have differences in texture information, and may also have differences in color information. Therefore, the category recognition model mainly learns the texture information and color information of different cooking material samples, classifies and judges based on the texture information and color information, and outputs the classification results. In this way, combining texture information and color information can achieve more accurate category recognition, avoiding the defect that it is difficult to accurately determine the category based on texture information or color information alone.
[0086] In this embodiment, the pre-trained type recognition model is saved. In the actual application stage, after collecting images of various cooking materials in the target container, the type recognition model can be directly called to quickly and accurately determine the types of various cooking materials.
[0087] In this embodiment, the weight of each cooking material can be obtained based on image recognition. Figure 3 In a possible implementation, the weight of the cooking material can be obtained by:
[0088] S123A: for each image of the cooking material, recognize the image to obtain shape features of the cooking material.
[0089] S124A, obtaining the volume of the cooking material according to the shape feature.
[0090] S125A, obtaining the density of the cooking material based on the type of the cooking material, and calculating the weight of the cooking material according to the density and the volume.
[0091] In this embodiment, after the image of the cooking material is collected, the area where the cooking material is located can be extracted, and then the shape characteristics of the cooking material can be determined. Since the collected image is generally a two-dimensional image, the shape characteristics of the cooking material determined are the shape characteristics of the surface of the cooking material. The shape characteristics may include size, shape, etc.
[0092] Since the cooking material is placed in the target container, and the target container has a certain shape, for example, the shape of the target container may be a bowl. Therefore, the size of the cross section of the target container is different at different depths. After obtaining the shape characteristics of the cooking material placed in the target container, the approximate height of the surface of the cooking material in the target container can be determined. In this way, the height of the cooking material, the shape and size of the surface of the cooking material can be obtained. The volume of the cooking material can be estimated by combining the shape of the target container, the height of the cooking material, and the size of the surface of the cooking material.
[0093] After knowing the type of each cooking material, the density of the cooking material can be obtained. In this way, the weight of the cooking material can be calculated according to the volume and density of the cooking material.
[0094] In this embodiment, the above method based on image recognition can realize the identification of the type of cooking ingredients and the determination of the weight of cooking ingredients.
[0095] In addition, considering that the weight determined based solely on image recognition may be biased, in order to ensure the accuracy of the obtained weight, please refer to Figure 4 In another possible implementation, the weight of the cooking material may be obtained in the following manner.
[0096] S123B: For each cooking material, recognize the image of the cooking material to obtain a first weight.
[0097] S124B, obtaining a second weight of the cooking material based on the detection result of the weight sensor.
[0098] S125B, obtaining the weight of the cooking material according to the first weight and the second weight.
[0099] In this implementation, on the one hand, the weight of the cooking material can be obtained by recognizing the image of the cooking material. For the sake of distinction, it is named the first weight here. The implementation method thereof can refer to the above implementation method and will not be described in detail here.
[0100] On the other hand, in this embodiment, a weight sensor may be provided at the bottom of each target container, and the weight sensor may collect and obtain the weight of the cooking material placed in the target container, which is named as the second weight. The weight sensor may communicate with the controller or the terminal device to send the detected second weight to the controller or the terminal device.
[0101] Considering that the target container itself has a certain weight, after the weight sensor obtains the detection result, the second weight of the cooking material can be obtained by subtracting the weight of the target container itself from the weight included in the detection result.
[0102] The first weight and the second weight obtained can be combined to finally determine the weight of the cooking material, thus avoiding the problem of inaccurate weight obtained based solely on the detection result of the weight sensor or based solely on image recognition.
[0103] Specifically, see Figure 5 In this embodiment, the weight of the cooking material can be obtained according to the first weight and the second weight in the following manner:
[0104] S1251B, calculate the difference between the first weight and the second weight, and detect whether the ratio between the difference and the second weight is less than or equal to a preset threshold. If it is less than or equal to the preset threshold, execute the following S1252B; if it is greater than the preset threshold, execute the following S1253B.
[0105] S1252B, taking the first weight as the weight of the cooking material.
[0106] S1253B, calculating the weight of the cooking material based on the first weight and the second weight according to a preset formula.
[0107] The first weight of the cooking material obtained based on image recognition is compared with the second weight obtained by the weight sensor to obtain the difference between the two. If the ratio between the difference and the second weight is less than or equal to the preset threshold, it indicates that the first weight obtained by image recognition is more accurate, and the first weight can be directly used as the final weight of the cooking material. If the ratio is greater than the preset threshold, it indicates that the first weight obtained by image recognition may be inaccurate and cannot be used as the standard. Similarly, the second weight detected by the weight sensor cannot be used as the standard.
[0108] Therefore, the weight of the cooking material can be calculated based on the first weight and the second weight according to a preset formula. In this embodiment, the first weight is recorded as t1, the second weight is recorded as t2, and the weight of the cooking material is calculated based on the first weight t1 and the second weight t2 according to the following formula:
[0109]
[0110] Assume that the first weight detected in the above manner is t1=10g, the second weight is t2=13g, and the preset threshold is 20%. The difference between the first weight and the second weight is calculated to be 3g, and the ratio between the difference and the second weight is 23%. That is, the obtained ratio 23% is greater than the preset threshold 20%. Therefore, the weight of the cooking material can be calculated by combining the first weight and the second weight. Substituting the first weight and the second weight into the above preset formula, the weight of the cooking material is calculated to be 11.5g.
[0111] In this embodiment, when the ratio is greater than a preset threshold, it indicates that the first weight obtained by the image recognition method may be inaccurate. In this case, a prompt message can be sent to the background server to prompt the developer to adjust the image recognition-related algorithms and image recognition-related models to improve subsequent recognition accuracy.
[0112] After obtaining the weight of the cooking material, the weight of the corresponding cooking material in the target recipe is compared. If the specification is met, the user can be prompted to continue the subsequent cooking process.
[0113] In this embodiment, the cooking information related to the target recipe also includes addition information such as the order and time period of adding each cooking material. After starting the cooking process, the user can place each cooking material in the target cooking device in a specified order and time period according to the prompt addition information. During this process, the image acquisition device can also capture a series of images of the target cooking device area. Based on the captured series of images, it can be detected whether the user adds the cooking materials in the specified order and time period. If the user fails to add the cooking materials in time, a reminder message can be issued in time to remind the user.
[0114] The cooking of a dish may involve multiple cooking stages, and the temperatures required for each cooking stage may be different. Therefore, the cooking heating device needs to be controlled at different gears to provide different heating temperatures in different cooking stages.
[0115] Based on this, in this embodiment, a temperature sensor is provided outside the target cooking device, and the temperature sensor can be used to detect the temperature value of the target cooking device, which can be understood as the temperature value provided to the cooking material contained in the target cooking device. The temperature sensor can communicate with a controller or a terminal device, etc., to send the detected temperature value to the controller or the terminal device, etc.
[0116] See also Figure 6 The cooking assistance method provided in this embodiment may further include the following steps:
[0117] S21, collecting the real-time temperature value of the target cooking device during the cooking process through a temperature sensor.
[0118] S22, comparing the real-time temperature value with pre-stored standard temperature information to determine whether the real-time temperature value matches the standard temperature value of the corresponding cooking stage, the standard temperature information including standard temperature values corresponding to multiple cooking stages during the cooking process.
[0119] S23, if the real-time temperature value does not match the standard temperature value of the corresponding cooking stage, adjust the gear of the cooking heating device.
[0120] In this embodiment, for the target recipe selected by the user, the cooking information related to the target recipe also includes standard temperature information, which includes the cooking stages required to complete the dish corresponding to the target recipe and the standard temperature value of each cooking stage. Each cooking stage is arranged in sequence on the time axis, and each cooking stage lasts for a certain period of time.
[0121] After the cooking process starts, a temperature sensor is set outside the target cooking device to collect the real-time temperature value of the target cooking device at a preset time interval. For each real-time temperature value, the corresponding cooking stage can be determined according to the timestamp of the real-time temperature value, and the real-time temperature value is matched with the standard temperature value of the corresponding cooking stage in the cooking information to detect whether it can match the corresponding standard temperature value.
[0122] For example, if the difference between the real-time temperature value and the standard temperature value is within a preset range, it can be determined that the real-time temperature value can match the standard temperature value. If the difference between the real-time temperature value and the standard temperature value exceeds the preset range, it can be determined that the real-time temperature value cannot match the standard temperature value.
[0123] In this embodiment, when the real-time temperature value cannot match the standard temperature value, the gear of the cooking and heating device can be adjusted to adjust the temperature of the target cooking device, thereby adjusting the temperature provided to the cooking material.
[0124] As the cooking process progresses, it is considered that some less experienced users may not be able to judge for themselves whether the cooking process is complete, that is, whether the dish is done. Therefore, please refer to Figure 7 The cooking auxiliary processing method provided in this embodiment may also include the following steps:
[0125] S31, capturing images of cooking materials in a target cooking device during a cooking process through an image capturing device.
[0126] S32, identifying the image of the cooking material in the target cooking device, and judging whether the cooking process is completed according to the obtained recognition result, and issuing a prompt message if it is judged that the cooking process is completed.
[0127] In this embodiment, during the cooking process, the image acquisition device can continuously acquire images of the cooking materials in the target cooking device. The images of the cooking materials are recognized, and here the images of the ingredients in the cooking materials are mainly recognized. Considering that the colors of most ingredients change significantly when they are ripe, the colors of the ingredients in the image are mainly recognized here. Based on the obtained recognition results, it is determined whether the cooking process is completed, that is, whether the ingredients are ripe. If it is determined that the entire cooking process has been completed, a prompt message can be sent to the user to prompt the user to turn off the cooking heating device, etc.
[0128] In this embodiment, the recognition result obtained includes a color recognition result. Figure 8 , when judging whether the cooking process is completed according to the recognition result, it can be achieved in the following ways:
[0129] S321, obtaining pre-stored standard color information, where the standard color information includes color information corresponding to multiple cooking stages during the cooking process.
[0130] S322, comparing the color recognition result with the standard color information to determine whether the color recognition result is consistent with the color information of the mature stage in the multiple cooking stages. If they are consistent, it is determined that the cooking process is completed.
[0131] In this embodiment, the cooking information related to the target recipe also includes standard color information, which includes color information of cooking materials at each cooking stage during the cooking process. The standard color information can be in the form of a color card to intuitively display the color changes of the cooking materials at each cooking stage during the cooking process. The multiple cooking stages include a mature stage, which is the last stage of the cooking process.
[0132] The color recognition result of the cooking material in the target cooking device is compared with the color information corresponding to the maturity stage. If the two are consistent, it indicates that the ingredients in the cooking material are mature and it can be determined that the cooking process has been completed.
[0133] In this embodiment, by automatically determining whether a dish is ready through image acquisition and recognition, some users with less cooking experience can be assisted in cooking and the cooking experience can be improved.
[0134] Based on the same inventive concept, please refer to Fig. 9 , shows a schematic diagram of the functional modules of the cooking auxiliary processing device 10 provided in an embodiment of the present invention. This embodiment can divide the functional modules of the cooking auxiliary processing device 10 according to the above-mentioned method embodiment. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present invention is schematic and is only a logical functional division. There may be other division methods in actual implementation.
[0135] For example, when each functional module is divided into corresponding functional modules, Fig. 9The cooking auxiliary processing device 10 shown is only a schematic diagram of the device. The cooking auxiliary processing device 10 may include a collection module 101, a recognition module 102, an acquisition module 103, a comparison module 104 and a judgment module 105. The functions of each functional module of the cooking auxiliary processing device 10 are described in detail below.
[0136] The acquisition module 101 is used to acquire images of cooking materials in each target container through an image acquisition device, wherein the cooking materials include ingredients and seasonings;
[0137] The recognition module 102 is used to recognize the image of each cooking material and determine the type of each cooking material;
[0138] An obtaining module 103 is used to obtain the weight of each cooking material;
[0139] The comparison module 104 is used to compare the type of each cooking material with the type of cooking material required in the target recipe determined based on the user's selection to obtain a type comparison result, and compare the weight of each cooking material with the weight of the corresponding cooking material in the target recipe to obtain a weight comparison result;
[0140] The judgment module 105 is used to judge whether to generate adjustment information based on the type comparison result and the weight comparison result, and if the adjustment information is generated, the adjustment information is fed back to the user.
[0141] In the solution provided in this embodiment, the types and weights of cooking materials actually prepared by the user are obtained through image recognition, and are compared with the types and weights of cooking materials required for the target recipe selected by the user, thereby ensuring that the cooking materials actually prepared meet the regulations and ensuring the taste of the final dish.
[0142] In a possible implementation, the obtaining module 103 is used to obtain the weight of each cooking material in the following manner:
[0143] For each cooking material, recognizing the image of the cooking material to obtain a first weight;
[0144] obtaining a second weight of the cooking material based on a detection result of the weight sensor;
[0145] The weight of the cooking material is obtained according to the first weight and the second weight.
[0146] In a possible implementation, the obtaining module 103 is used to obtain the weight of the cooking material according to the first weight and the second weight in the following manner:
[0147] Calculating a difference between the first weight and the second weight, and detecting whether a ratio between the difference and the second weight is less than or equal to a preset threshold;
[0148] If it is less than or equal to the preset threshold, the first weight is used as the weight of the cooking material;
[0149] If it is greater than a preset threshold, the weight of the cooking material is calculated based on the first weight and the second weight according to a preset formula.
[0150] In a possible implementation, the obtaining module 103 is used to obtain the weight of each cooking material in the following manner:
[0151] For each image of the cooking material, the image is recognized to obtain shape features of the cooking material;
[0152] Obtaining the volume of the cooking material according to the shape characteristics;
[0153] The density of the cooking material is obtained based on the type of the cooking material, and the weight of the cooking material is calculated according to the density and the volume.
[0154] In a possible implementation, the identification module 102 may be used to determine the type of each cooking material in the following manner:
[0155] For each cooking material image, the image is imported into a pre-trained type recognition model for processing, where the type recognition model is pre-trained based on a plurality of cooking material samples;
[0156] The texture information and color information of the cooking material in the image are obtained based on the type recognition model, and the type of the cooking material is determined based on the texture information and the color information.
[0157] In a possible implementation, the identification module 102 may also be used to:
[0158] capturing, by an image capturing device, an image of cooking materials in a target cooking device during a cooking process;
[0159] The image of the cooking material in the target cooking device is recognized, and whether the cooking process is completed is determined based on the obtained recognition result. If it is determined that the cooking process is completed, a prompt message is issued.
[0160] In a possible implementation, the recognition result includes a color recognition result, and the recognition module 102 may be used to determine whether the cooking process is completed in the following manner:
[0161] Obtaining pre-stored standard color information, the standard color information including color information corresponding to multiple cooking stages during the cooking process;
[0162] The color recognition result is compared with the standard color information to determine whether the color recognition result is consistent with the color information of the mature stage in the multiple cooking stages. If they are consistent, it is determined that the cooking process has been completed.
[0163] In a possible implementation, the identification module 102 may also be used to:
[0164] The real-time temperature value of the target cooking device during the cooking process is collected by a temperature sensor;
[0165] Comparing the real-time temperature value with the pre-stored standard temperature information to determine whether the real-time temperature value matches the standard temperature value of the corresponding cooking stage, the standard temperature information including the standard temperature values corresponding to the multiple cooking stages during the cooking process;
[0166] If the real-time temperature value does not match the standard temperature value of the corresponding cooking stage, adjust the gear of the cooking heating device.
[0167] See also Fig.10 An embodiment of the present invention further provides an electronic device 20, which may be a controller in a cooker or a terminal device, etc. The electronic device 20 includes a memory 202 and a processor 201. The memory 202 stores a computer program that can be run on the processor 201. The above steps are implemented when the processor 201 executes the computer program.
[0168] The memory 202 and the processor 201 in the electronic device 20 communicate via a communication bus and a communication interface. The communication bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus may be divided into an address bus, a data bus, a control bus, etc.
[0169] The memory 202 may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory 202 may also be at least one storage device located away from the processor 201.
[0170] The above-mentioned processor 201 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
[0171] An embodiment of the present invention further provides a computer-readable medium having a non-volatile program code executable by a processor, wherein the program code enables the processor to execute any of the above methods.
[0172] When the embodiments of the present invention are specifically implemented, reference may be made to the above embodiments, and the corresponding technical effects are achieved.
[0173] It is understood that the embodiments described in the present invention can be implemented by hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions of the present invention or a combination thereof.
[0174] For software implementation, the technology of the present invention can be implemented by a unit that performs the functions of the present invention. The software code can be stored in the memory 202 and executed by the processor 201. The memory 202 can be implemented in the processor 201 or outside the processor 201.
[0175] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0176] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0177] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.
[0178] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0179] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0180] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, ROM, RAM, disk or optical disk and other media that can store program codes.
[0181] It should be noted that, in the present invention, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0182] The above are only specific embodiments of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined in the present invention can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown in the present invention, but should conform to the widest range consistent with the principles and novel features of the present invention.
Claims
1. A cooking auxiliary processing method, characterized in that: The method comprises: Acquiring images of cooking materials in each target container by an image acquisition device, wherein the cooking materials include ingredients and seasonings; Recognizing the image of each cooking material, determining the type of each cooking material, and obtaining the weight of each cooking material; Comparing the type of each cooking ingredient with the type of cooking ingredient required in the target recipe determined based on the user's selection to obtain a type comparison result, and comparing the weight of each cooking ingredient with the weight of the corresponding cooking ingredient in the target recipe to obtain a weight comparison result; Based on the type comparison result and the weight comparison result, it is determined whether to generate adjustment information; if the adjustment information is generated, the adjustment information is fed back to the user.
2. The cooking auxiliary processing method according to claim 1, characterized in that: The step of obtaining the weight of each cooking material comprises: For each of the cooking materials, identifying the image of the cooking material to obtain a first weight; obtaining a second weight of the cooking material based on a detection result of the weight sensor; The weight of the cooking material is obtained according to the first weight and the second weight.
3. The cooking auxiliary processing method according to claim 2, characterized in that: The step of obtaining the weight of the cooking material according to the first weight and the second weight comprises: Calculating a difference between the first weight and the second weight, and detecting whether a ratio between the difference and the second weight is less than or equal to a preset threshold; If it is less than or equal to the preset threshold, taking the first weight as the weight of the cooking material; If it is greater than the preset threshold, the weight of the cooking material is calculated based on the first weight and the second weight according to a preset formula.
4. The cooking auxiliary processing method according to claim 1, characterized in that: The step of obtaining the weight of each cooking material comprises: For each image of the cooking material, identifying the image to obtain shape features of the cooking material; Obtaining the volume of the cooking material according to the shape feature; The density of the cooking material is obtained based on the type of the cooking material, and the weight of the cooking material is calculated according to the density and the volume.
5. The cooking auxiliary processing method according to claim 1, characterized in that: The step of determining the type of each cooking material comprises: For each image of the cooking material, importing the image into a pre-trained type recognition model for processing, wherein the type recognition model is pre-trained based on a plurality of cooking material samples; The texture information and color information of the cooking material in the image are obtained based on the type recognition model, and the type of the cooking material is determined based on the texture information and the color information.
6. The cooking auxiliary processing method according to any one of claims 1 to 5, characterized in that: The method further comprises: capturing, by the image capturing device, an image of cooking materials in the target cooking device during the cooking process; The image of the cooking material in the target cooking device is recognized, and whether the cooking process is completed is determined based on the recognition result. If it is determined that the cooking process is completed, a prompt message is issued.
7. The cooking auxiliary processing method according to claim 6, characterized in that: The recognition result includes a color recognition result, and the step of judging whether the cooking process is completed according to the obtained recognition result includes: Obtaining pre-stored standard color information, wherein the standard color information includes color information corresponding to a plurality of cooking stages in a cooking process; The color recognition result is compared with the standard color information to determine whether the color recognition result is consistent with the color information of the mature stage in the multiple cooking stages. If they are consistent, it is determined that the cooking process has been completed.
8. The cooking auxiliary processing method according to any one of claims 1 to 5, characterized in that: The method further comprises: The real-time temperature value of the target cooking device during the cooking process is collected by a temperature sensor; Comparing the real-time temperature value with pre-stored standard temperature information to determine whether the real-time temperature value matches the standard temperature value of the corresponding cooking stage, wherein the standard temperature information includes standard temperature values corresponding to a plurality of cooking stages during the cooking process; If the real-time temperature value does not match the standard temperature value of the corresponding cooking stage, the gear position of the cooking heating device is adjusted.
9. A cooking auxiliary processing device, characterized in that: The device comprises: A collection module, used for collecting images of cooking materials in each target container through an image collection device, wherein the cooking materials include ingredients and seasonings; A recognition module, used to recognize the image of each cooking material and determine the type of each cooking material; An obtaining module, used for obtaining the weight of each cooking material; a comparison module, configured to compare the type of each cooking material with the type of cooking material required in the target recipe determined based on the user's selection to obtain a type comparison result, and to compare the weight of each cooking material with the weight of the corresponding cooking material in the target recipe to obtain a weight comparison result; A judgment module is used to judge whether to generate adjustment information based on the type comparison result and the weight comparison result, and if the adjustment information is generated, feed back the adjustment information to the user.
10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.
