Cooking control method and device

Abstract

The present disclosure relates to a cooking control method and device, the method comprising: a mobile terminal acquires image information including at least part of the image of a cooking device and ingredients to be cooked through an image acquisition device; acquires cooking device information according to the image of the cooking device; acquires information of ingredients to be cooked according to the image of the ingredients to be cooked; adjusts a first cooking program corresponding to the ingredients to be cooked according to the identification information of the cooking device and / or the information of the ingredients to be cooked to obtain an adjusted second cooking program; controls the cooking device to cook the ingredients to be cooked according to the adjusted second cooking program, wherein the first cooking program includes a first cooking parameter, and the second cooking program includes a second cooking parameter. The present disclosure does not require a camera to be set inside the cooking device, does not require the camera to be cleaned, reduces costs, adjusts the cooking program through the cooking device and cooking ingredient information acquired through a single shot, and improves user interaction experience and operation convenience.

Description

Technical Field

[0001] The present disclosure relates to the field of smart life, and in particular to a cooking control method and device. Background Art

[0002] As the pace of life quickens, more and more people choose to use smart cooking that can achieve automatic cooking, so that they can easily make delicious meals even without cooking experience.

[0003] Currently, the automatic cooking of smart cooking equipment mainly relies on the camera installed inside it. The camera is used to identify the ingredients placed in the smart cooking equipment, and the corresponding recipes stored in the cloud are obtained based on the identification results of the ingredients, and then automatic cooking is further performed according to the recipes.

[0004] However, the camera installed inside the smart cooking device needs to be protected against high temperatures, and the camera is easily contaminated and difficult to clean, which affects the user experience. At the same time, the camera of the current cooking device cannot simultaneously obtain the image of the cooking device and the food to be cooked, so as to directly obtain the cooking device identification model and the food to be cooked information through the image information of the cooking device and the food to be cooked, and adjust the cooking program / parameters to further optimize the cooking process, resulting in a small correlation between the data of the cooking device and the food to be cooked, which cannot meet the personalized needs of users, and the user interaction experience is not strong, and the cooking convenience is not high. Summary of the invention

[0005] In order to solve the above technical problems, the present disclosure provides a cooking control method and device to improve the user experience.

[0006] In a first aspect, an embodiment of the present disclosure provides a cooking control method, which is applied to a cooking system, wherein the cooking system includes a cooking device and a mobile terminal, wherein the mobile terminal is provided with an image acquisition device, and the method includes: the mobile terminal acquires image information through the image acquisition device, wherein the image information includes at least a partial image of the cooking device and at least a partial image of the food to be cooked;

[0007] acquiring information of the cooking device according to at least a portion of the image of the cooking device;

[0008] Acquiring information of the food to be cooked according to at least a portion of the image of the food to be cooked;

[0009] According to the cooking device information and / or the information of the ingredients to be cooked, adjusting the first cooking program corresponding to the ingredients to be cooked to obtain an adjusted second cooking program;

[0010] Based on the data connection relationship established between the mobile terminal and the cooking device, the cooking device is controlled to cook the ingredients to be cooked according to the adjusted second cooking program, wherein the first cooking program includes first cooking parameters and the second cooking program includes second cooking parameters.

[0011] In some embodiments,

[0012] The food to be cooked is placed in the cooking device, and after the mobile terminal acquires the image information, the method further includes:

[0013] Inputting the image information into a pre-trained food detection model to obtain the name or code of the food to be cooked;

[0014] Determining the area of ​​the food to be cooked from the image information according to the name or code of the food to be cooked;

[0015] The area of ​​the food to be cooked in the image information is removed to obtain the cooking equipment area.

[0016] In some embodiments, determining the area of ​​the food to be cooked from the image information according to the name or code of the food to be cooked includes:

[0017] Determining the prior color information of the food to be cooked according to the name or code of the food to be cooked;

[0018] Based on the prior color information, the area of ​​the food to be cooked is determined from the image information.

[0019] In some embodiments, obtaining the cooking device information according to at least a portion of the image of the cooking device includes:

[0020] Obtaining device models / identifications / types of cooking devices in the cooking device area;

[0021] Determine the type of the cooking device according to the device model / identification / type.

[0022] In some embodiments, the plurality of device models / identifications / types are located at different positions of the cooking device, and obtaining the device models / identifications / types of the cooking devices in the cooking device area includes:

[0023] Determine at least one target area in the cooking equipment area, each of the target areas including an equipment model / identification / type;

[0024] Intercepting the device model / identification / category area corresponding to each target area in the image information, wherein the target area corresponds to the device model / identification / category area one by one;

[0025] Perform image recognition on at least one of the device model / identification / type regions to obtain the device model / identification / type.

[0026] In some embodiments, determining at least one target area in the cooking device area includes:

[0027] Performing binarization processing on the cooking equipment area to obtain a processed cooking equipment area;

[0028] Perform image recognition on the processed cooking equipment area to determine at least one target area in the cooking equipment area.

[0029] In some embodiments, adjusting the first cooking program corresponding to the food to be cooked according to the cooking device information and / or the food to be cooked information to obtain an adjusted second cooking program includes:

[0030] Determining information of the ingredients to be cooked according to the model information of the cooking device;

[0031] According to the information of the ingredients to be cooked and the model of the cooking device, obtaining a first cooking program corresponding to the ingredients to be cooked from a preset cooking program library;

[0032] The first cooking program is adjusted according to the information of the ingredients to be cooked to obtain an updated second cooking program.

[0033] In some embodiments, the information of the ingredients to be cooked includes at least the weight of the ingredients to be cooked, and determining the information of the ingredients to be cooked according to the model information of the cooking device includes:

[0034] Calculating the actual coverage area of ​​the food to be cooked according to the model information of the cooking device;

[0035] The weight of the food to be cooked is calculated according to the actual coverage area of ​​the food to be cooked and the name of the food to be cooked.

[0036] In some embodiments, adjusting the first cooking program corresponding to the ingredients to be cooked according to the cooking device information and / or the ingredients to be cooked information to obtain an adjusted second cooking program includes:

[0037] According to the position of the food to be cooked in the cooking device, the working mode of the heating tube and / or the air outlet in the cooking device is adjusted.

[0038] In a second aspect, an embodiment of the present disclosure provides a cooking control device, comprising:

[0039] An acquisition module, configured to acquire image information through the image acquisition device; wherein the image information includes at least a partial image of the cooking device and at least a partial image of the food to be cooked;

[0040] A first recognition module, configured to obtain information of the cooking device based on at least a portion of the image of the cooking device;

[0041] A second recognition module, configured to recognize information of the food to be cooked based on at least a portion of the image of the food to be cooked;

[0042] an adjustment module, configured to adjust the first cooking program corresponding to the ingredients to be cooked according to the cooking equipment information and / or the ingredients to be cooked information, so as to obtain an adjusted second cooking program;

[0043] A control module is used to control the cooking device to cook the ingredients to be cooked according to the adjusted second cooking program based on the data connection relationship established between the mobile terminal and the cooking device, wherein the first cooking program includes first cooking parameters and the second cooking program includes second cooking parameters.

[0044] The cooking control method and device provided by the disclosed embodiment uses a portable image acquisition device on a mobile terminal to acquire images of cooking equipment, and further uses image recognition to confirm the model of the cooking equipment to adjust and optimize the cooking program, thereby eliminating the need to set up a camera inside the cooking equipment, and the user does not need to clean the camera, thereby reducing costs and improving user experience. At the same time, the image acquisition device of the cooking equipment can simultaneously acquire images of the cooking equipment and the ingredients to be cooked, thereby directly acquiring the identification model of the cooking equipment and the information of the ingredients to be cooked through the image information of the cooking equipment and the ingredients to be cooked, and then adjusting the cooking program / parameters to further optimize the cooking process, thereby improving the data correlation between the cooking equipment and the ingredients to be cooked, meeting the personalized needs of users, and enhancing the user interaction experience and cooking convenience. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0047] Figure 1 A flow chart of a cooking control method provided by an embodiment of the present disclosure;

[0048] Figure 2 A schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0049] Figure 3 A flow chart of a cooking control method provided by another embodiment of the present disclosure;

[0050] Figure 4 A schematic diagram of an image of a cooking device provided in an embodiment of the present disclosure;

[0051] Figure 5 A schematic diagram of device model identification provided in an embodiment of the present disclosure;

[0052] Figure 6 A schematic diagram of device model identification provided in an embodiment of the present disclosure;

[0053] Figure 7 A schematic diagram of device model identification provided in an embodiment of the present disclosure;

[0054] Figure 8 A flow chart of a cooking control method provided by another embodiment of the present disclosure;

[0055] Fig. 9 A schematic diagram of the structure of a cooking control device provided in an embodiment of the present disclosure;

[0056] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0057] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0058] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0059] The disclosed embodiment provides a cooking control method, which is introduced below in conjunction with a specific embodiment.

[0060] Figure 1 A flow chart of a cooking control method provided by an embodiment of the present disclosure. The method can be applied to Figure 2The application scenario shown includes a server 21, a cooking device 22 and a mobile terminal 23, wherein the server 21 is in communication with the cooking device 22, and the server 21 is in communication with the mobile terminal 23. It is understandable that the cooking control method provided in the embodiment of the present disclosure can also be applied in other scenarios.

[0061] Combine the following Figure 2 The application scenario shown is Figure 1 The cooking control method shown in the figure is introduced, and the specific steps of the method are as follows:

[0062] S101. The mobile terminal acquires image information through the image acquisition device, wherein the image information includes at least a partial image of a cooking device and at least a partial image of ingredients to be cooked.

[0063] The mobile terminal includes a portable image acquisition device, wherein the mobile terminal may be a smart phone, a PDA, a tablet computer, etc., and the portable image acquisition device may be a camera included in the mobile terminal, etc.

[0064] The user can capture images of the cooking device by operating the portable image acquisition device on the mobile terminal, and the mobile terminal can store image information, which includes at least a partial image of the cooking device and at least a partial image of the food to be cooked. Furthermore, the server communicates with the mobile terminal, and the server acquires at least a partial image of the cooking device through the mobile terminal.

[0065] Figure 3 A schematic diagram of at least part of a cooking device provided by an embodiment of the present disclosure. Figure 3 As shown, at least a portion of the image of the cooking device may be an image of at least a portion of the cooking device body obtained by taking a bird's-eye view of the cooking device using an image acquisition device on the mobile terminal.

[0066] S102: Acquire the cooking device information according to at least a portion of the image of the cooking device.

[0067] Image recognition is performed on at least a portion of the obtained image of the cooking device to obtain identification / type / model information of the cooking device, wherein the identification information of the cooking device is used to identify the model of the cooking device in the image.

[0068] In some embodiments, the information of the cooking device may be characteristic information of the cooking device. The appearance images of the models on the market and the models to be on the market are collected from multiple angles in advance, and a model library is established. Preferably, the edge contour information of the entire cooking device (for example, the oval and circle of the rice cooker; the oval + internal small oval of the air fryer), the characteristic information of the panel (the number of silk screens on the panel, the silk screen sorting, the position and other information), the shape and color of the pot body of the cooking device, the nesting state of multiple circular rings at the bottom of the pot body, the honeycomb pattern shape of the bottom of the pot, and the grid shape inside the machine cover, etc., are all converted and extracted into a priori characteristic information sequences (including color, shape, circular nesting, specific form, etc.), and then associated with the specific model of the device.

[0069] Image recognition is performed based on the appearance and other features of the cooking equipment, and feature analysis is performed on the acquired image through a deep neural network to extract image features. Specifically, at least part of the image is input into a convolution layer to perform convolution feature extraction to obtain image features:

[0070] feature=∑W i X i +

[0071] Where W i is the convolution kernel, X i is a certain area sub-image of at least part of the image, and b is a weighting parameter. The image features are compared with the image data in the model library to calculate the similarity, and the model of a cooking device with the highest similarity is obtained as the recognition result.

[0072] In some embodiments, the models of multiple cooking devices with high similarity can also be used as recognition results, and then the prior features of at least part of the image are extracted and compared with the prior feature information sequence pre-set in the model library, and the models are verified and screened according to the appearance features of the cooking devices, and finally the model of the cooking device is determined and output.

[0073] S103: Acquire information of the food to be cooked according to at least a portion of the image of the food to be cooked.

[0074] Image recognition is performed on at least a portion of the image of the food to be cooked to determine relevant information of the food to be cooked.

[0075] Specifically, the information of the ingredients to be cooked includes the type and weight of the ingredients to be cooked, etc., which is not limited in the embodiments of the present disclosure.

[0076] S104: adjusting a first cooking program corresponding to the ingredients to be cooked according to the cooking equipment information and / or the ingredients to be cooked information to obtain an adjusted second cooking program.

[0077] S105. Based on the data connection relationship established between the mobile terminal and the cooking device, control the cooking device to cook the ingredients to be cooked according to the adjusted second cooking program, wherein the first cooking program includes first cooking parameters and the second cooking program includes second cooking parameters.

[0078] As in Figure 2 In the scenario shown, the mobile terminal and the cooking device can communicate data through the server. In other scenarios, the mobile terminal and the cooking device can also communicate data with each other, which is not limited in the embodiments of the present disclosure.

[0079] Different types of cooking equipment and different amounts of cooking ingredients correspond to different configurations and working methods. Therefore, for the cooking program corresponding to the same ingredients, different types of cooking equipment or different amounts of cooking ingredients produce different cooking effects. In order to achieve the best cooking effect for the ingredients, it is necessary to adjust the first cooking parameter in the default first cooking program according to the different cooking equipment and the information of the ingredients to be cooked, and obtain a second cooking program including the second cooking parameter to improve the cooking effect of the cooking device. For example, the working mode of the heating tube and / or the air outlet in the cooking device in the first cooking program can be adjusted.

[0080] In an embodiment of the present disclosure, at least partial images of a cooking device and ingredients to be cooked are acquired through a mobile terminal, wherein the mobile terminal comprises a portable image acquisition device; the model of the cooking device and information of the ingredients to be cooked are identified based on the at least partial images; a first cooking program corresponding to the ingredients to be cooked is adjusted based on the model of the cooking device and the information of the ingredients to be cooked to obtain an adjusted second cooking program; based on a data connection relationship established between the mobile terminal and the cooking device, the cooking device is controlled to cook the ingredients to be cooked according to the adjusted second cooking program, wherein the first cooking program comprises first cooking parameters, and the second cooking program comprises second cooking parameters, the image of the cooking device is acquired by using the portable image acquisition device on the mobile terminal, and the model of the cooking device is further confirmed by image recognition to adjust and optimize the cooking program, thereby eliminating the need to set a camera inside the cooking device and the user does not need to clean the camera, thereby reducing costs and improving user experience.

[0081] Figure 3 A flow chart of a cooking control method provided by another embodiment of the present disclosure is shown in FIG. Figure 3 As shown, the method includes the following steps:

[0082] S301. A mobile terminal obtains image information through the image obtaining device.

[0083] Specifically, the implementation process and principle of S301 and S101 are the same, and will not be repeated here.

[0084] S302: Input the image information into a pre-trained food detection model to obtain the name or code of the food to be cooked.

[0085] Ingredients to be cooked are placed in the cooking device, and the acquired image information is input into a pre-trained ingredient detection model to obtain the name or code of the ingredient to be cooked output by the ingredient detection model.

[0086] The trained food detection model may be a neural network model, which detects and locates food in the acquired image based on the neural network, and outputs the name or code of the food to be cooked. Optionally, the food detection model may also output a region frame of the food to be cooked.

[0087] S303: Determine the area of ​​the food to be cooked from the image information according to the name or code of the food to be cooked.

[0088] Specifically, according to the name or code of the ingredient to be cooked, the prior color information of the ingredient to be cooked is determined; based on the prior color information, the area of ​​the ingredient to be cooked is determined from the image information.

[0089] The prior color information of cooking ingredients is the pre-stored color information of each cooking ingredient. Specifically, the prior color information can be a color value (RGB) range. For example, the prior color information of chicken wings is (194, 71, 18) to (255, 243, 213); the prior color information of steak is (186, 135, 118) to (208, 165, 137); the prior color information of French fries is (200, 190, 160) to (250, 245, 230), and so on.

[0090] After obtaining the prior color information of the food to be cooked, the area in the image whose color value matches the prior color information can be determined as the food to be cooked area according to the prior color information. Optionally, the food to be cooked area can be a circumscribed rectangular area or a circumscribed circular area of ​​the area whose color value matches the prior color information.

[0091] S304: remove the area of ​​the food to be cooked in the image to obtain the cooking equipment area.

[0092] The area of ​​the food to be cooked in the image is removed to obtain an image of the cooking equipment area without food.

[0093] Optionally, since the ingredients to be cooked are placed in the cooking equipment before cooking, the area of ​​the ingredients to be cooked in the image is removed, and the cooking equipment area of ​​the image is obtained. The cooking equipment area includes the image of the cooking equipment, and the shape of the area of ​​the ingredients to be cooked is usually similar to the shape of the inner wall of the cooking equipment, and the coverage area is in a certain proportional relationship, which is the preset expansion coefficient here. Further, according to the minimum circumscribed rectangle or the minimum circumscribed circle of the area of ​​the ingredients to be cooked, expansion is performed according to the preset expansion coefficient. For example, taking the minimum circumscribed circle of the cooking ingredient area as an example, the radius of the minimum circumscribed circle of the cooking ingredient area is R0, and the preset expansion coefficient λ is 0.5 to 0.75, so that the radius of the suspected area of ​​the cooking equipment is between R0 and R1, where R1 = R0 / λ. A mask ring is generated according to the suspected area of ​​the cooking equipment, and the other parts of the original image except the suspected area of ​​the cooking equipment are shielded to obtain the cooking equipment area.

[0094] S305: Acquire the cooking device information according to at least a portion of the image of the cooking device.

[0095] Specifically, the image of the cooking device area may be recognized by referring to the method of performing image recognition on at least a portion of the obtained image of the cooking device in the above embodiment to obtain the identification / type / model information of the cooking device.

[0096] In some embodiments, if different models of equipment are produced in different colors, the cooking equipment identification / type / model information can also be obtained by identifying the cooking equipment based on its color.

[0097] In some embodiments, determining the area of ​​the food to be cooked from the image information can be achieved in the following way: after the neural network model is detected, the name of the food is obtained, and the prior color of the food can be determined according to the name of the food; at the same time, after the model of the cooking equipment is identified, the color of the cooking equipment is also known, that is, the theoretical foreground and background colors are obtained. Due to the difference in actual shooting light and mobile phone beautification algorithm during image acquisition, the actual color is different from the image color, and the watershed segmentation algorithm is used to perform the cutout processing of the area of ​​the food to be cooked.

[0098] In the watershed algorithm, the background (cooking equipment area) is set as the initial threshold of the geodesic, and the color floating threshold of the foreground (the area of ​​ingredients to be cooked) is used as the threshold of the dam. The segmentation iteration is started, and the dam area, which is the area of ​​ingredients to be cooked, is finally iterated as follows:

[0099]

[0100] S306: Adjust the first cooking program corresponding to the ingredients to be cooked according to the cooking equipment information and / or the information of the ingredients to be cooked to obtain an adjusted second cooking program.

[0101] S307. Based on the data connection relationship established between the mobile terminal and the cooking device, control the cooking device to cook the ingredients to be cooked according to the adjusted second cooking program, wherein the first cooking program includes first cooking parameters and the second cooking program includes second cooking parameters.

[0102] Specifically, the implementation process and principle of S305 to S307 are consistent with those of S103 to S105, and will not be repeated here.

[0103] The disclosed embodiment first removes the image of the food area to be cooked through the prior color information of the food to be cooked to obtain the cooking equipment area image, and further identifies the cooking equipment based on the cooking equipment area image, thereby reducing the amount of calculation for identifying the cooking equipment model and effectively improving the recognition efficiency.

[0104] In some embodiments, obtaining the cooking equipment information based on at least a partial image of the cooking equipment includes: obtaining the equipment model / identification / type of the cooking equipment in the cooking equipment area; and determining the model of the cooking equipment based on the equipment model / identification / type.

[0105] Due to objective factors of image acquisition, the image of the cooking equipment area may not be a complete image of the cooking equipment. To improve the accuracy of cooking equipment model recognition, the cooking equipment is specially marked and the equipment model / identification / type is set to assist in identifying the type of cooking equipment.

[0106] Specifically, the multiple device models / identifications / types are located at different positions of the cooking device. To avoid missing image acquisition, the same device model / identification / type is respectively set at different positions of the same cooking device, and each device model / identification / type is respectively located in a target area. That is, the cooking device area of ​​the image includes at least one target area, and the target area includes the device model / identification / type area.

[0107] Figure 4 This is a schematic diagram of a cooking device image provided by an embodiment of the present disclosure. Figure 4As shown, the equipment model / identification / type is respectively set at the positions of the four rectangular frames 41, 42, 43, and 44 in the upper, lower, left, and right of the cooking equipment area (i.e., the pot body). Even if the complete cooking equipment image is not captured during image acquisition, as long as any equipment model / identification / type is included in the image, the model of the cooking equipment can be identified according to the equipment model / identification / type. Taking the barcode-shaped equipment model / identification / type as an example, the barcode-shaped equipment model / identification / type is a group of vertical lines of varying lengths engraved at several specific positions on the pot body (multiple target areas in the cooking equipment area in the image), and different lengths correspond to different numbers. Figure 5 A schematic diagram of device model identification provided in an embodiment of the present disclosure, such as Figure 5 As shown, from left to right there are five white vertical lines and five black vertical lines, forming a barcode image, which is parsed into a string of binary codes, that is, the code of the machine model; and the physical size of the barcode is variable. The height and width of the barcode are adjusted according to the size and shape of the cooking equipment (such as the actual width and curvature of the pot edge), which is not limited in the embodiments of the present disclosure. Figure 6 , Figure 7 A schematic diagram of device model identification provided in an embodiment of the present disclosure, such as Figure 6 As shown, the black vertical line represents code 1, and the white vertical line represents code 0. Figure 6 The device model identification shown corresponds to binary code 001, representing model 001; Figure 7 The device model identification shown corresponds to the binary code 010, representing model 002.

[0108] In some embodiments, obtaining the equipment model / identification / type of the cooking equipment in the cooking equipment area includes: determining at least one target area in the cooking equipment area, each of the target areas including the equipment model / identification / type; intercepting the equipment model / identification / type area corresponding to each target area in the image information, the target area corresponding to the equipment model / identification / type area one-to-one; performing image recognition on at least one of the equipment model / identification / type areas to obtain the equipment model / identification / type.

[0109] Specifically, the determining of at least one target area in the cooking equipment area includes: performing binarization processing on the cooking equipment area to obtain a processed cooking equipment area; performing image recognition on the processed cooking equipment area to determine at least one target area in the cooking equipment area.

[0110] Optionally, after determining the cooking device area, the outer contour of the cooking device may be used as a reference, a certain number of pixels may be reduced toward the inside of the cooking device to form a circular ring or a square ring according to the shape of the cooking device, and then binarization may be performed.

[0111] Perform image recognition on the binarized cooking equipment area to obtain at least one target area in the cooking equipment area (i.e., at least one area suspected to contain a device model / identification / type). The original image is intercepted according to the position of the target area in the original image to obtain the device model / identification / type area.

[0112] Image recognition is performed on the device model / identification / type area to obtain the device model / identification / type. Specifically, the device model / identification / type area is grayed out and then binarized. Taking the barcode-shaped device model / identification / type as an example, a group of black and white vertical lines are finally obtained. The pixel width of the vertical lines is counted, and a binary code is output according to the combination of black and white pixel widths, thereby obtaining the device model of the cooking device.

[0113] The disclosed embodiment provides another implementation method for identifying the model of a cooking device. By identifying the device model / logo / type engraved on the cooking device, the model of the cooking device can still be identified when the cooking device is offline, thereby improving the flexibility and accuracy of the cooking control method.

[0114] Figure 8 A flow chart of a cooking control method provided by another embodiment of the present disclosure is shown in FIG. Figure 8 As shown, the method includes the following steps:

[0115] S801: The mobile terminal acquires image information through the image acquisition device.

[0116] S802: Input the image information into a pre-trained food detection model to obtain the name or code of the food to be cooked.

[0117] S803: Determine the area of ​​the food to be cooked from the image according to the name or code of the food to be cooked.

[0118] S804: Remove the area of ​​the food to be cooked in the image information to obtain the cooking equipment area.

[0119] S805: Acquire the cooking device information according to at least a portion of the image of the cooking device.

[0120] S806: Determine information about the ingredients to be cooked according to the model information of the cooking device.

[0121] Specifically, the actual coverage area of ​​the ingredients to be cooked is calculated according to the model information of the cooking device; and the weight of the ingredients to be cooked is calculated according to the actual coverage area of ​​the ingredients to be cooked and the names of the ingredients to be cooked.

[0122] After obtaining the model of the cooking device, the corresponding model information of the cooking device can be found in the preset model information library, such as the actual size of the cooking device. According to the pixel area of ​​the cooking device in the image and the actual size of the cooking device, combined with the pixel area of ​​the food to be cooked in the food area to be cooked, the actual coverage area of ​​the food to be cooked can be calculated.

[0123] After the actual coverage area of ​​the food to be cooked is obtained, the weight of the food to be cooked can be calculated according to the prior information of the food to be cooked.

[0124] Specifically, the Canny edge detection algorithm is used for the image of the cooking equipment area. First, the Sobel operator is used for gradient detection. The horizontal detection operator and the vertical detection operator are:

[0125]

[0126] The gradient amplitude is:

[0127]

[0128] The gradient direction is:

[0129] θ=actg(Gy / Gx)

[0130] The results of the gradient detection are further subjected to non-maximum suppression to connect lines with similar distances to obtain the contour lines of the cooking device. The contour lines of the cooking device are further fitted according to the shape of the cooking device and the relevant parameters for calculating the pixel area of ​​the cooking device are output, for example, the radius R1 of a circular pot and the side length W1 of a square pot are output after fitting.

[0131] Optionally, for circular cooking equipment, the least square method is used for circular fitting, and for square cooking equipment, four straight lines are fitted and then combined; wherein the circular fitting formula is:

[0132]

[0133] where x i ,y i is the coordinate of the pixel in the image, x c ,y c is the coordinate of the center pixel of the circle in the image, and R1 is the radius of the circle.

[0134] According to the model information and the pixel area of ​​the cooking equipment in the image, the proportional relationship between the image size and the actual size can be obtained:

[0135] PRrate=Dlength / (π*R1 2 )

[0136] or:

[0137] PRrate=Dlength / (W1 2 )

[0138] Where Dlength is the actual size of the cooking equipment, and PRrate is the ratio of the image size to the actual size.

[0139] The actual coverage area of ​​the food to be cooked is further calculated. For example, if the pixel length of the food to be cooked is foodw and the width is foodh, the actual length and width of the food to be cooked are calculated as follows:

[0140] foodrw=foodw*PRrate

[0141] foodrh=foodh*PRrate

[0142] The actual coverage area of ​​the food to be cooked is:

[0143] foodrw*foodrh

[0144] For the weight estimation of the food to be cooked, a relationship curve between the area and weight of the food to be cooked is generated in advance based on a large amount of experimental data, wherein the area of ​​the food to be cooked can be the pixel area of ​​the food to be cooked or the actual coverage area of ​​the food to be cooked, depending on the actual situation of the experimental data. According to the relationship curve, the weight of the food to be cooked can be determined.

[0145] In some embodiments, the relationship between the weight Weight of the French fries and its pixel area Area is as follows:

[0146] Weight = 0.001 Area 2 -0.91Area+234.99

[0147] S807: Acquire a first cooking program corresponding to the food to be cooked from a preset cooking program library according to the information of the food to be cooked and the model of the cooking device.

[0148] Specifically, first, according to the name of the ingredient to be cooked, all cooking programs for the corresponding ingredient are obtained from the preset cooking program library, recorded as cooking program set A, and then further filtered according to the model of the cooking equipment in the cooking program set A to obtain the first cooking program.

[0149] S808: Adjust the first cooking program corresponding to the ingredients to be cooked according to the cooking equipment information and / or the information of the ingredients to be cooked to obtain an adjusted second cooking program.

[0150] Specifically, the working mode of the heating tube and / or the air outlet in the cooking device is adjusted according to the position of the food to be cooked in the cooking device.

[0151] For example, when the air outlet and the heating tube are both directly above, the wind direction of the air outlet is adjusted according to the position of the food in the pot; for cooking equipment with multiple heating tubes, one or more heating tubes close to the position of the food to be cooked are determined to be in operation according to the relative positions of the food and the heating tubes.

[0152] In some embodiments, the temperature or duration parameters in the first cooking program may also be adjusted according to the size or weight of the food to be cooked.

[0153] Specifically, for ingredients whose surface is easily burnt, such as Western pastries, the cooking time in the first cooking program is increased according to the increase in the weight of the ingredients to be cooked; for meat, etc., the temperature in the first cooking program is increased. If the current temperature parameter has reached the maximum temperature of the cooking equipment, the cooking time in the first cooking program is increased.

[0154] After adjusting the first cooking program, an adjusted second cooking program is obtained, and the cooking device is further controlled to cook the cooking ingredients according to the second cooking program.

[0155] In some embodiments, if the first cooking program corresponding to the current cooking device and the ingredients to be cooked is not obtained, a machine model with a power close to the current cooking device MachineP is searched, for example, there are K similar machine models, whose powers are respectively recorded as {MachineP1, MachineP2, ..., MachinePk}; according to the similar machine models and the ingredients to be cooked, a search is performed in the preset cooking program library to obtain K related recipe process parameters, which are recorded as {NormalParam1, NormalParam2, ..., NormParamk}, and the recipe process parameters are further weighted averaged according to the power similarity between the similar machine models and the cooking device to generate the target process parameters:

[0156]

[0157] The disclosed embodiment obtains a second cooking program by adjusting the first cooking program according to information such as the area and weight of the food to be cooked, so that the cooking device can obtain a better cooking effect when processing the food to be cooked according to the second cooking program.

[0158] At the same time, when the first cooking program is not found, the embodiment of the present disclosure searches for similar devices according to the power of the cooking device, and generates target cooking parameters by weighted calculation of existing related cooking parameters according to power similarity, thereby maximizing the success rate of cooking and improving the intelligence of the user experience.

[0159] Fig. 9 Schematic diagram of the structure of the cooking control device provided in the embodiment of the present disclosure. The cooking control device may be the cooking device or mobile terminal as described in the above embodiment, or the cooking control device may be a component or assembly in the cooking device or mobile terminal. The cooking control device provided in the embodiment of the present disclosure may execute the processing flow provided in the cooking control method embodiment, such as Fig. 9 As shown, the cooking control device 90 includes an acquisition module 91, a first recognition module 92, a second recognition module 93, an adjustment module 94, and a control module 95; the acquisition module 91 is used to acquire image information through the image acquisition device, wherein the image information includes at least a partial image of the cooking device and at least a partial image of the food to be cooked; the first recognition module 92 is used to acquire the cooking device information based on at least a partial image of the cooking device; the second recognition module 93 is used to acquire the food information to be cooked based on at least a partial image of the food to be cooked; the adjustment module 94 is used to adjust the first cooking program corresponding to the food to be cooked according to the cooking device information and / or the food information to be cooked to obtain an adjusted second cooking program; the control module 95 is used to control the cooking device to cook the food to be cooked according to the adjusted second cooking program based on the data connection relationship established between the mobile terminal and the cooking device, wherein the first cooking program includes first cooking parameters, and the second cooking program includes second cooking parameters.

[0160] Optionally, the ingredients to be cooked are placed in the cooking equipment, and the cooking control device 90 also includes a processing module 96, which includes a detection unit 961, a first determination unit 962, and a removal unit 963; the detection unit 961 is used to input the image information into a pre-trained ingredient detection model to obtain the name or code of the ingredient to be cooked; the first determination unit 962 is used to determine the area of ​​the ingredients to be cooked from the image according to the name or code of the ingredient to be cooked; the removal unit 963 is used to remove the area of ​​the ingredients to be cooked in the image information to obtain the cooking equipment area.

[0161] Optionally, the first determination unit 962 is used to determine the prior color information of the ingredient to be cooked according to the name or code of the ingredient to be cooked; and determine the area of ​​the ingredient to be cooked from the image information based on the prior color information.

[0162] Optionally, the first identification module 92 includes a first acquisition unit 921 and a second determination unit 922; the acquisition unit 921 is used to acquire the equipment model / identification / type of the cooking equipment in the cooking equipment area; the second determination unit 922 is used to determine the type of the cooking equipment based on the equipment model / identification / type.

[0163] Optionally, multiple device models / identifications / types are located at different positions of the cooking equipment, and the first acquisition unit 921 is used to determine at least one target area in the cooking equipment area, each of the target areas including a device model / identification / type; the device model / identification / type area corresponding to each target area is intercepted in the image information, and the target area corresponds to the device model / identification / type area one-to-one; image recognition is performed on at least one of the device model / identification / type areas to obtain the device model / identification / type.

[0164] Optionally, the first acquisition unit 921 is used to perform binarization processing on the cooking equipment area to obtain a processed cooking equipment area; perform image recognition on the processed cooking equipment area to determine at least one target area in the cooking equipment area.

[0165] Optionally, the adjustment module 94 includes a third determination unit 941, a second acquisition unit 942, and an adjustment unit 943; the third determination unit 941 is used to determine the information of the ingredients to be cooked according to the model information of the cooking device; the second acquisition unit 942 is used to obtain a first cooking program corresponding to the ingredients to be cooked from a preset cooking program library according to the information of the ingredients to be cooked and the model of the cooking device; the adjustment unit 943 is used to adjust the first cooking program according to the information of the ingredients to be cooked to obtain an updated second cooking program.

[0166] Optionally, the information of the ingredients to be cooked includes at least the weight of the ingredients to be cooked, and the third determination unit 941 is further used to calculate the actual coverage area of ​​the ingredients to be cooked according to the model information of the cooking device; and calculate the weight of the ingredients to be cooked according to the actual coverage area of ​​the ingredients to be cooked and the name of the ingredients to be cooked.

[0167] Optionally, the adjustment unit 943 is used to adjust the working mode of the heating tube and / or the air outlet in the cooking device according to the position of the food to be cooked in the cooking device.

[0168] Fig. 9 The cooking control device of the illustrated embodiment can be used to execute the technical solution of the above-mentioned method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0169] Fig.10Schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. The electronic device may be a cooking device or a mobile terminal as described in the above embodiment. The electronic device provided in an embodiment of the present disclosure may execute the processing flow provided in an embodiment of the cooking control method, such as Fig.10 As shown, the electronic device 100 includes: a memory 101, a processor 102, a computer program and a communication interface 103; wherein the computer program is stored in the memory 101 and is configured so that the processor 102 executes the cooking control method as described above.

[0170] In addition, an embodiment of the present disclosure further provides a computer-readable storage medium on which a computer program is stored. The computer program is executed by a processor to implement the cooking control method described in the above embodiment.

[0171] In addition, an embodiment of the present disclosure also provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the cooking control method as described above is implemented.

[0172] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0173] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A cooking control method, applied to a cooking system, wherein the cooking system comprises a cooking device and a mobile terminal, wherein the mobile terminal is provided with an image acquisition device, characterized in that: The method comprises: The mobile terminal acquires image information through the image acquisition device, wherein the image information includes at least a partial image of the cooking device and at least a partial image of the food to be cooked; Inputting the image information into a pre-trained food detection model to obtain the name or code of the food to be cooked; According to the name or code of the food to be cooked, determining the area of ​​the food to be cooked from the image information, and placing the food to be cooked in the cooking device; acquiring information of the cooking device according to at least a portion of the image of the cooking device; Acquiring information of the food to be cooked according to at least a portion of the image of the food to be cooked; According to the cooking device information and / or the information of the ingredients to be cooked, adjusting the first cooking program corresponding to the ingredients to be cooked to obtain an adjusted second cooking program; Based on the data connection relationship established between the mobile terminal and the cooking device, controlling the cooking device to cook the food to be cooked according to the adjusted second cooking program, wherein the first cooking program includes a first cooking parameter, and the second cooking program includes a second cooking parameter; Determining the area of ​​the food to be cooked from the image information according to the name or code of the food to be cooked includes: determining the prior color information of the food to be cooked according to the name or code of the food to be cooked; determining the area of ​​the food to be cooked from the image information based on the prior color information; The method also includes: when the first cooking program is not acquired, determining multiple similar models corresponding to the cooking equipment and multiple recipe process parameters corresponding to the multiple similar models according to power similarity; performing weighted averaging on the multiple recipe process parameters according to the power similarity to generate target cooking parameters.

2. The method according to claim 1, characterized in that: After determining the area of ​​the food to be cooked from the image information according to the name or code of the food to be cooked, the method further includes: The area of ​​the food to be cooked in the image information is removed to obtain the cooking equipment area.

3. The method according to claim 1, characterized in that The acquiring the cooking device information according to at least a portion of the image of the cooking device comprises: Obtaining device models / identifications / types of cooking devices in the cooking device area; Determine the type of the cooking device according to the device model / identification / type.

4. The method according to claim 3, characterized in that The multiple device models / identifications / types are located at different positions of the cooking device, and obtaining the device models / identifications / types of the cooking devices in the cooking device area includes: Determine at least one target area in the cooking equipment area, each of the target areas including an equipment model / identification / type; Intercepting the device model / identification / category area corresponding to each target area in the image information, wherein the target area corresponds to the device model / identification / category area one by one; Image recognition is performed on at least one of the device model / identification / type regions to obtain the device model / identification / type.

5. The method according to claim 4, characterized in that The determining of at least one target area in the cooking device area comprises: Performing binarization processing on the cooking equipment area to obtain a processed cooking equipment area; Perform image recognition on the processed cooking equipment area to determine at least one target area in the cooking equipment area.

6. The method according to claim 1, characterized in that The step of adjusting the first cooking program corresponding to the food to be cooked according to the cooking device information and / or the food to be cooked information to obtain an adjusted second cooking program includes: Determining information of the ingredients to be cooked according to the model information of the cooking device; According to the information of the ingredients to be cooked and the model of the cooking device, obtaining a first cooking program corresponding to the ingredients to be cooked from a preset cooking program library; The first cooking program is adjusted according to the information of the ingredients to be cooked to obtain an updated second cooking program.

7. The method according to claim 6, characterized in that The information of the ingredients to be cooked includes at least the weight of the ingredients to be cooked, and the determining the information of the ingredients to be cooked according to the model information of the cooking device includes: Calculating the actual coverage area of ​​the food to be cooked according to the model information of the cooking device; The weight of the food to be cooked is calculated according to the actual coverage area of ​​the food to be cooked and the name of the food to be cooked.

8. The method according to claim 6, characterized in that The step of adjusting the first cooking program corresponding to the ingredients to be cooked according to the cooking device information and / or the ingredients to be cooked information to obtain an adjusted second cooking program includes: According to the position of the food to be cooked in the cooking device, the working mode of the heating tube and / or the air outlet in the cooking device is adjusted.

9. A cooking control device, wherein the cooking control device is a mobile terminal, or the cooking control device is a component or assembly in a mobile terminal, characterized in that: include: an acquisition module, configured to acquire image information through an image acquisition device, wherein the image information includes at least a partial image of the cooking device and at least a partial image of the food to be cooked; A first recognition module, configured to obtain information of the cooking device based on at least a portion of the image of the cooking device; The processing module comprises a first determining unit, wherein the first determining unit is used to determine the area of ​​the food to be cooked from the image according to the name or code of the food to be cooked, and the food to be cooked is placed in the cooking device; The processing module includes a detection unit, which is used to input the image information into a pre-trained food detection model to obtain the name or code of the food to be cooked; A second recognition module, configured to obtain information of the food to be cooked based on at least a portion of the image of the food to be cooked; an adjustment module, configured to adjust the first cooking program corresponding to the ingredients to be cooked according to the cooking equipment information and / or the ingredients to be cooked information, so as to obtain an adjusted second cooking program; a control module, configured to control the cooking device to cook the ingredients to be cooked according to the adjusted second cooking program based on the data connection relationship established between the mobile terminal and the cooking device, wherein the first cooking program includes a first cooking parameter and the second cooking program includes a second cooking parameter; The first determining unit is specifically configured to determine the prior color information of the food to be cooked according to the name or code of the food to be cooked; and determine the area of ​​the food to be cooked from the image information based on the prior color information; The cooking control device is also used for: when the first cooking program is not acquired, determining multiple similar models corresponding to the cooking equipment and multiple recipe process parameters corresponding to the multiple similar models according to power similarity; performing weighted averaging of the multiple recipe process parameters according to the power similarity to generate target cooking parameters.

Images