Cooking appliance control method and device, cooking appliance and storage medium

Abstract

The present application provides a cooking appliance control method and device, a cooking appliance and a storage medium. The control method comprises: in the process of cooking a food ingredient with a cooking appliance, generating at least one food ingredient image of the food ingredient; detecting the at least one food ingredient image, and determining at least one target image with pollutants in the image content; and restoring the at least one target image to a pollutant-free state.

Description

Cooking appliance control method, device, cooking appliance, and storage medium

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on July 24, 2023, with application number "202310906385.7" and invention name "Control method, device, cooking appliance and storage medium for cooking appliances", the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the technical field of household appliances, and in particular to a control method and device for a cooking appliance, a cooking appliance, and a storage medium. Background Art

[0003] In the related art, when a user uses a cooking utensil to obtain an image of the food being cooked, the camera of the cooking utensil will condense water droplets or water mist, resulting in water droplets or water mist on the food image, which in turn results in poor quality of the food image and affects the user experience.

[0004] Summary of the Invention

[0005] This application aims to solve at least one of the technical problems existing in the prior art.

[0006] To this end, a first aspect of the present application provides a method for controlling a cooking appliance.

[0007] A second aspect of the present application provides a control device for a cooking appliance.

[0008] A third aspect of the present application provides a control device for a cooking appliance.

[0009] A fourth aspect of the present application provides a cooking appliance.

[0010] A fifth aspect of the present application provides a readable storage medium.

[0011] A first aspect of the present application provides a method for controlling a cooking appliance, comprising: generating at least one food image of the food during the cooking process of the cooking appliance; detecting the at least one food image to determine at least one target image containing contaminants; and restoring the at least one target image to a contaminant-free state.

[0012] The cooking appliance control method provided in the present application can be used to obtain images of the cooking process of ingredients during the cooking process of the cooking appliance for users to view, thereby meeting the user's needs for the cooking process and improving the user experience.

[0013] Furthermore, the cooking appliance may be provided with an image acquisition device, such as a camera, etc. The image acquisition device is provided in the cooking cavity of the cooking appliance, so that images of the food can be acquired while the cooking appliance is cooking the food.

[0014] Specifically, while the cooking appliance is cooking, the image capture device of the cooking appliance can be controlled to capture the food within the cooking cavity, thereby generating at least one image of the food within the cooking cavity. It is understood that the user can, based on their needs, use the image capture device to capture the final image of the cooked food after cooking is complete. Alternatively, images of the food can be captured at each stage of the cooking process.

[0015] Furthermore, after at least one food image is generated, the at least one food image can be inspected to determine whether contaminants are present in the image content of the food image. Specifically, the contaminants can include water droplets or water mist, that is, whether water droplets or water mist are present in the image content. This determines whether the target image contains contaminants in the image content. It is understood that during the cooking process, the temperature within the cooking chamber rises, causing the water contained in the food to evaporate and then condense into water droplets or water mist on the camera of the image acquisition device. Furthermore, contaminants can also include oil droplets, greasy stains, or other impurities displayed in the image. If contaminants condense on the camera, the contaminants will appear in the captured food image, resulting in a poor quality food image and failure to meet user requirements for food image quality.

[0016] Furthermore, when it is detected that there are pollutants in the image content of at least one food image, that is, when it is determined that at least one food image includes a target image, the target image can be restored, that is, the part with pollutants in the image content of the food image is restored to remove the pollutants in the image content and retain the original effect of the food on the food image, thereby meeting the user's requirements for the quality of the food image.

[0017] Specifically, if there is only one food image, and contaminants are present in the food image, the contaminant-containing area can be restored based on other contaminant-free areas in the food image. If there are multiple food images, the contaminant-containing area can be restored based on areas corresponding to the contaminant-containing area in other images that do not contain contaminants.

[0018] The cooking appliance control method provided in this application captures images of ingredients during the cooking process to generate at least one ingredient image, thereby meeting the user's personalized needs during the cooking process and improving the user experience. Furthermore, if contaminants are present in the image content of at least one ingredient image, the contaminant-containing areas of the ingredient image can be restored to eliminate the impact of the contaminants on the quality of the ingredient image, preserving the original effect of the ingredients in the ingredient image, further improving the user's demand for the quality of the ingredient image and further enhancing the user experience of the cooking appliance.

[0019] In addition, the cooking appliance control method according to the above technical solution provided by this application may also have the following additional technical features:

[0020] In some technical solutions, optionally, the food image includes multiple sub-areas, and at least one food image is detected to determine at least one target image containing pollutants in the image content, including: traversing each food image to obtain image features of each sub-area; and determining at least one target image based on the image features.

[0021] In this technical solution, the food image can be divided into multiple sub-areas, so that in the process of detecting the food image, the multiple sub-areas of the food image are detected in turn to determine whether there are pollutants in the image content of each sub-area of ​​the food image. When there are pollutants in any sub-area of ​​the food image, the food image can be determined as a target image with pollutants in its image content.

[0022] Specifically, in the process of detecting food images, first, the food image can be traversed to obtain the image features of each sub-region of the food image, and then the image features of each sub-region can be analyzed to determine whether there are pollutants in the image content of each sub-region of the food image, that is, based on the image features of each sub-region of the food image, determine whether the food image is a target image with pollutants in the image content.

[0023] In some technical solutions, optionally, the image features include optical flow direction, brightness and color, and at least one target image is determined based on the image features, including: when the optical flow direction of any sub-area is toward the direction of gravity and the brightness is greater than a preset brightness, determining the food image as the target image; or when the optical flow direction of any sub-area is toward the direction of gravity and the color is white, determining the food image as the target image.

[0024] In this technical solution, the image features of the food image may include optical flow direction, brightness, and color. Thus, after traversing the food image, the presence of contaminants in each subregion of the food image can be determined based on the optical flow direction, brightness, or color of the subregion.

[0025] Specifically, for contaminants formed by water droplets or mist, if the optical flow direction of any subregion of the food image is oriented toward gravity and the brightness of that subregion is greater than a preset brightness, it can be determined that contaminants formed by water droplets or mist are present in that subregion. This means that the food image is a target image containing contaminants.

[0026] It's understandable that when contaminants are present in a food image, the light flow in the area containing the contaminants is oriented toward gravity, due to the contaminants' tendency to move downward under the influence of gravity. Furthermore, due to the reflective properties of contaminants, the brightness of the contaminant area will be higher than that of other areas. Therefore, if the light flow in any sub-area of ​​the food image is oriented toward gravity and the brightness is greater than a preset brightness, contaminants can be determined to be present in that sub-area.

[0027] Alternatively, for pollutants formed by water droplets or water mist, if the optical flow direction of any sub-region of the food image is toward the direction of gravity and the image color in the sub-region is white, it can be determined that pollutants formed by water droplets or water mist exist in the sub-region, that is, the food image is a target image with pollutants in the image content.

[0028] It is understood that when contaminants such as water droplets or mist are present in a food image, since these contaminants tend to move downward under the influence of gravity, the optical flow direction of the area containing these contaminants will be oriented toward the direction of gravity. Furthermore, contaminants such as water droplets or mist will appear white or near-white when illuminated by light. Therefore, if the optical flow direction of any sub-region of the food image is oriented toward gravity and the color is white or near-white, it can be determined that contaminants are present in that sub-region.

[0029] In some technical solutions, optionally, the image features include a refractive index, and at least one target image is determined based on the image features, including: when the refractive index of any sub-region is less than a preset refractive index, determining the food image as the target image.

[0030] In this technical solution, the image features may also include a refractive index. Accordingly, in the process of detecting each sub-region of the food image, if the refractive index in any sub-region of the food image is less than a preset refractive index, it indicates that there are contaminants in the sub-region, and the food image can be determined as a target image with contaminants in the image content.

[0031] It is understood that when contaminants such as water droplets or mist condense on the camera of an image acquisition device, the density of water is greater than that of air. Therefore, after the food image is generated, the refractive index of the subregion of the food image containing the contaminants will be significantly lower than the refractive index of other subregions. Accordingly, when inspecting the food image, if a subregion of the food image is determined to have a refractive index lower than a predetermined refractive index, the food image can be determined to be a target image containing contaminants such as water droplets or mist.

[0032] In some technical solutions, optionally, the image features include brightness, saturation and contrast, and at least one target image is determined based on the image features, including: when the brightness, saturation or contrast of any sub-area is greater than a preset value, determining the food image as the target image.

[0033] In this technical solution, the image features of the food image may also include brightness, saturation and contrast. Accordingly, in the process of detecting the food image, it is also possible to determine whether there are pollutants in the sub-area based on the brightness, saturation and contrast of each sub-area of ​​the food image.

[0034] Specifically, after traversing the food image, the brightness, saturation, and contrast of each subregion of the food image are determined. If the brightness within any subregion of the food image is greater than a preset brightness value, it indicates that a contaminant is present in that subregion. Alternatively, if the saturation within any subregion of the food image is greater than a preset saturation value, it indicates that a contaminant is present in that subregion. Alternatively, if the contrast within any subregion of the food image is greater than a preset contrast value, it indicates that a contaminant is present in that subregion.

[0035] In some technical solutions, optionally, at least one target image is determined based on image features, including: when the difference between the image features of any two adjacent sub-regions is greater than a first preset difference, determining the food image as the target image.

[0036] In this technical solution, after traversing the food image, the image features of each sub-region of the food image are determined, and further, whether there are pollutants in the sub-region can be determined based on the difference between the image features of adjacent sub-regions.

[0037] Specifically, if the difference between the image features of any two adjacent sub-regions is greater than a first preset difference, it can be determined that a contaminant exists in at least one of the two adjacent sub-regions. In other words, the continuity between the two adjacent sub-regions of the same food image is not satisfied.

[0038] It is understood that within the same food image, if no subregion contains contaminants, the image features of two adjacent subregions will exhibit a continuous transition. Therefore, if the difference between the image features of any two adjacent subregions is greater than a first predetermined difference, it indicates that the two adjacent subregions in the food image do not meet the continuity requirement, and therefore, it can be determined that contaminants are present in the food image content.

[0039] In some technical solutions, optionally, at least one target image is determined based on image features, including: when the difference between image features of the same sub-area of ​​two food images generated at adjacent times is greater than a second preset difference, the subsequently generated food image is determined to be the target image.

[0040] In this technical solution, when there are multiple food images, the area where the pollutants are present can be restored based on the area corresponding to the area where the pollutants are present in other images where the pollutants are not present.

[0041] After traversing each food image, image features of each corresponding sub-area in the two food images generated at adjacent times can be compared. If the difference between the image features of the same sub-area of ​​the two food images generated at adjacent times is greater than a second preset difference, it can be determined that there are contaminants in the image content of at least one of the two food images generated at adjacent times.

[0042] It is understood that, if no contaminants are present in the food images, the image features of the same subregion of food images generated at adjacent times will exhibit a continuous transition. Therefore, if the difference between the image features of the same subregion of two food images generated at adjacent times is greater than a second predetermined difference, it can be determined that a contaminant is present in the image content of at least one of the two food images generated at adjacent times.

[0043] In some technical solutions, optionally, at least one target image is restored, including: determining the change trend of food in the sub-area where pollutants exist in the target image based on at least one food image generated before the target image; and restoring the sub-area where pollutants exist in the target image based on the change trend of food in the sub-area where pollutants exist.

[0044] In this technical solution, after determining that a food image is a target image with pollutants in its image content, the sub-region of the target image with pollutants in its image content can be restored based on at least one food image generated before the target image to eliminate the pollutants in the target image and ensure the image quality of the target image.

[0045] Specifically, the changing trend of the food within the subregion containing contaminants can be determined based on at least one food image generated before the target image. That is, the changing trend of the food within the image content is predicted based on the at least one food image generated before. The subregion containing contaminants in the target image is then restored based on the changing area of ​​the food. This allows the food within the subregion containing contaminants in the target image to be restored to its true image, ensuring the quality of the food image.

[0046] In some technical solutions, optionally, at least one target image is restored, including: obtaining a first sub-region in the target image where pollutants exist; obtaining a second sub-region corresponding to the first sub-region in a food image before the target image; replacing the first sub-region with the second sub-region; and smoothing the replaced target image according to a preset algorithm to restore at least one target image.

[0047] In this technical solution, after determining that a food image is a target image with pollutants in its image content, the sub-region of the target image with pollutants in its image content can be restored based on at least one food image generated before the target image to eliminate the pollutants in the target image and ensure the image quality of the target image.

[0048] Specifically, the first subregion of the target image containing contaminants is first determined. A second subregion corresponding to the first subregion in the food image generated before the target image is obtained. The first subregion is replaced with the second subregion. Specifically, the first subregion containing contaminants in the target image is deleted, and the second subregion corresponding to the first subregion in the food image before the target image, which does not contain contaminants, is placed in the first subregion's place.

[0049] It is understood that since the food image displayed in the first and second sub-regions contains the same content, replacing the first sub-region with the second sub-region can eliminate the contaminants in the image content. Furthermore, the replaced target image is smoothed according to a preset algorithm to restore the target image.

[0050] In some technical solutions, optionally, at least one target image is restored, including: obtaining a third sub-region where contaminants exist in the target image; obtaining a fourth sub-region corresponding to the third sub-region in a food image before the target image; and adjusting the image features of the third sub-region based on the image features of the fourth sub-region to restore at least one target image.

[0051] In this technical solution, after determining that a food image is a target image with pollutants in its image content, the sub-region of the target image with pollutants in its image content can be restored based on at least one food image generated before the target image to eliminate the pollutants in the target image and ensure the image quality of the target image.

[0052] Specifically, the method first determines the third subregion of the target image where the contaminant is present. Then, a fourth subregion corresponding to the third subregion is obtained from a food image generated before the target image. Image features of the fourth subregion are then obtained, and the image features of the third subregion are adjusted based on the image features of the fourth subregion, thereby restoring the target image.

[0053] It can be understood that since the image content displayed by the third sub-region and the fourth sub-region are the same, and the generation time interval between the third sub-region and the fourth sub-region is short, the image features of the third sub-region can be adjusted according to the image features of the fourth sub-region, so as to eliminate the pollutants in the third sub-region and realize the restoration of the target image.

[0054] In some technical solutions, optionally, restoring at least one target image includes: inputting at least one target image into a preset neural network to restore the at least one target image.

[0055] In this technical solution, after determining the target image containing pollutants, the target image can be input into a preset neural network, and the preset neural network outputs a restored target image.

[0056] It is understood that the preset neural network is trained using a large amount of historical image data, including images with and without pollutants. Therefore, the preset neural network is used to fit a functional mapping relationship from blurred image information of the polluted mist area to the original clear image information. Based on the fitted functional mapping relationship, the original clear image of the polluted mist area is restored.

[0057] In some technical solutions, optionally, the pollutants include water droplets or water mist.

[0058] In this technical solution, the contaminants may include water droplets or water mist. That is, the target image is the image content containing water droplets or water mist. Furthermore, the contaminants may also include oil droplets, oil stains, or other impurities displayed in the image.

[0059] According to the second aspect of the present application, a control device for a cooking appliance is proposed, comprising: a generation unit, for generating at least one food image of food during the process of the cooking appliance cooking food; a detection unit, for detecting at least one food image to determine at least one target image containing contaminants in the image content; and a restoration unit, for restoring at least one target image to a state free of contaminants.

[0060] The control device for a cooking appliance provided in the present application captures an image of the ingredients through a generation unit during the cooking process to generate at least one ingredient image, thereby meeting the user's personalized needs during the cooking process and improving the user experience. Furthermore, the detection unit can detect at least one ingredient image containing contaminants. Furthermore, if contaminants are present in the image content of at least one ingredient image, the restoration unit restores the area containing the contaminants in the ingredient image to eliminate the impact of the contaminants on the quality of the ingredient image, retaining the original effect of the ingredients on the ingredient image, further improving the user's demand for the quality of the ingredient image, and further enhancing the user experience of the cooking appliance.

[0061] Furthermore, the food image includes multiple sub-regions, and the detection unit is specifically used to: traverse each food image to obtain image features of each sub-region; and determine at least one target image based on the image features.

[0062] Furthermore, the image features include optical flow direction, brightness and color, and the detection unit is also used to: determine that the food image is the target image when the optical flow direction of any sub-area is toward the direction of gravity and the brightness is greater than a preset brightness; or determine that the food image is the target image when the optical flow direction of any sub-area is toward the direction of gravity and the color is white.

[0063] Furthermore, the image feature includes a refractive index, and the detection unit is further configured to: determine that the food image is a target image when the refractive index of any sub-region is less than a preset refractive index.

[0064] Furthermore, the image features include brightness, saturation and contrast, and the detection unit is further used to: determine that the food image is the target image when the brightness, saturation or contrast of any sub-area is greater than a preset value.

[0065] Furthermore, the detection unit is further configured to: determine that the food image is the target image when the difference between the image features of any two adjacent sub-regions is greater than a first preset difference.

[0066] Furthermore, the detection unit is also used to: when the difference between the image features of the same sub-region of two food images generated at adjacent times is greater than a second preset difference, determine that the subsequently generated food image is the target image.

[0067] Furthermore, the restoration unit is specifically used to: determine the change trend of the food in the sub-area where pollutants exist in the target image based on at least one food image generated before the target image; and restore the sub-area where pollutants exist in the target image based on the change trend of the food in the sub-area where pollutants exist.

[0068] Furthermore, the restoration unit is also used to: obtain a first sub-region in the target image where pollutants exist; obtain a second sub-region corresponding to the first sub-region in the food image before the target image; replace the first sub-region with the second sub-region; and smooth the replaced target image according to a preset algorithm to restore at least one target image.

[0069] Furthermore, the restoration unit is also used to: obtain a third sub-region in the target image where contaminants exist; obtain a fourth sub-region corresponding to the third sub-region in the previous food image of the target image; and adjust the image features of the third sub-region based on the image features of the fourth sub-region to restore at least one target image.

[0070] Furthermore, the restoration unit is also used to: input at least one target image into a preset neural network to restore the at least one target image.

[0071] According to a third aspect of the present application, a control device for a cooking appliance is proposed, comprising: a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the control method for the cooking appliance provided in the first aspect are implemented.

[0072] The control device for a cooking appliance provided in the present application includes a memory and a processor, and also includes a program or instruction stored in the memory. When the program or instruction is executed by the processor, the steps of the control method for a cooking appliance of the first aspect mentioned above can be implemented. Therefore, the control device for the cooking appliance has all the beneficial effects of the control method for a cooking appliance mentioned above, which will not be repeated here.

[0073] According to a fourth aspect of the present application, a cooking appliance is provided, comprising a control device for the cooking appliance according to any one of the above technical solutions.

[0074] The cooking appliance provided in the present application includes the cooking appliance control device of any one of the above technical solutions. Therefore, the cooking appliance has all the beneficial effects of the above cooking appliance control device, which will not be repeated here.

[0075] According to a fifth aspect of the present application, a readable storage medium is proposed, on which a program or instruction is stored. When the program or instruction is executed by a processor, a control method for a cooking appliance as described in any one of the above technical solutions is implemented.

[0076] The readable storage medium provided in the present application stores a program or instruction thereon. When the program or instruction is executed by the processor, it can implement a control method for a cooking appliance as described in any one of the above-mentioned technical solutions. Therefore, the storage medium has all the beneficial effects of the control method for the above-mentioned cooking appliance, which will not be repeated here.

[0077] Additional aspects and advantages of the present application will become apparent in the following description or may be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0079] FIG1 is a schematic flow chart of a method for controlling a cooking appliance according to an embodiment of the present application;

[0080] FIG2 is a schematic flow chart of a method for controlling a cooking appliance according to another embodiment of the present application;

[0081] FIG3 is a schematic flow chart of a method for controlling a cooking appliance according to yet another embodiment of the present application;

[0082] FIG4 is a schematic flow chart showing a method for controlling a cooking appliance according to yet another embodiment of the present application;

[0083] FIG5 is a schematic flow chart showing a method for controlling a cooking appliance according to yet another embodiment of the present application;

[0084] FIG6 is a schematic flow chart showing a method for controlling a cooking appliance according to yet another embodiment of the present application;

[0085] FIG7 is a schematic flow chart showing a method for controlling a cooking appliance according to yet another embodiment of the present application;

[0086] FIG8 is a schematic flow chart showing a method for controlling a cooking appliance according to yet another embodiment of the present application;

[0087] FIG9 is a schematic flow chart showing a method for controlling a cooking appliance according to yet another embodiment of the present application;

[0088] FIG10 is a schematic flow chart showing a method for controlling a cooking appliance according to yet another embodiment of the present application;

[0089] FIG11 is a schematic flow chart showing a method for controlling a cooking appliance according to yet another embodiment of the present application;

[0090] FIG12 is a schematic flow chart of a method for controlling a cooking appliance according to yet another embodiment of the present application. DETAILED DESCRIPTION

[0091] In order to more clearly understand the above-mentioned objects, features and advantages of the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features therein can be combined with each other in the absence of conflict.

[0092] In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present application is not limited to the specific embodiments disclosed below.

[0093] The following describes a cooking appliance control method, device, cooking appliance, and storage medium according to some embodiments of the present application with reference to Figures 1 to 12.

[0094] As shown in FIG1 , according to one embodiment of the present application, a method for controlling a cooking appliance is proposed, including:

[0095] S102, generating at least one image of the food during the cooking process of the food by the cooking utensil;

[0096] S104, detecting at least one food image to determine at least one target image containing contaminants;

[0097] S106, restoring at least one target image to a pollutant-free state.

[0098] The cooking appliance control method provided in the present application can be used to obtain images of the cooking process of ingredients during the cooking process of the cooking appliance for users to view, thereby meeting the user's needs for the cooking process and improving the user experience.

[0099] Furthermore, the cooking appliance may be provided with an image acquisition device, such as a camera, etc. The image acquisition device is provided in the cooking cavity of the cooking appliance, so that images of the food can be acquired while the cooking appliance is cooking the food.

[0100] Specifically, while the cooking appliance is cooking, the image capture device of the cooking appliance can be controlled to capture the food within the cooking cavity, thereby generating at least one image of the food within the cooking cavity. It is understood that the user can, based on their needs, use the image capture device to capture the final image of the cooked food after cooking is complete. Alternatively, images of the food can be captured at each stage of the cooking process.

[0101] Furthermore, after at least one food image is generated, the at least one food image can be inspected to determine whether there are any contaminants in the image content of the food image. Specifically, the contaminants can include water droplets or water mist, that is, whether there are water droplets or water mist in the image content. This determines the target image whose image content contains contaminants. It is understood that during the cooking process, the temperature in the cooking chamber rises, causing the water contained in the food to evaporate and then condense into water droplets or water mist on the camera of the image acquisition device. Furthermore, contaminants can also include oil droplets, oil stains, or other impurities displayed in the image. If there are contaminants on the camera, the contaminants will appear in the acquired food image, resulting in a deterioration in the quality of the food image and failure to meet the user's requirements for food image quality.

[0102] Furthermore, when it is detected that there are pollutants in the image content of at least one food image, that is, when it is determined that at least one food image includes a target image, the target image can be restored, that is, the part with pollutants in the image content of the food image is restored to remove the pollutants in the image content and retain the original effect of the food on the food image, thereby meeting the user's requirements for the quality of the food image.

[0103] Specifically, if there is only one food image, and contaminants are present in the food image, the contaminant-containing area can be restored based on other contaminant-free areas in the food image. If there are multiple food images, the contaminant-containing area can be restored based on areas corresponding to the contaminant-containing area in other images that do not contain contaminants.

[0104] The cooking appliance control method provided in this application captures images of ingredients during the cooking process to generate at least one ingredient image, thereby meeting the user's personalized needs during the cooking process and improving the user experience. Furthermore, if contaminants are present in the image content of at least one ingredient image, the contaminant-containing areas of the ingredient image can be restored to eliminate the impact of the contaminants on the quality of the ingredient image, preserving the original effect of the ingredients in the ingredient image, further improving the user's demand for the quality of the ingredient image and further enhancing the user experience of the cooking appliance.

[0105] According to one embodiment of the present application, as shown in FIG2 , a method for controlling a cooking appliance is proposed, including:

[0106] S202, generating at least one image of the food during the cooking process of the food by the cooking utensil;

[0107] S204, traversing each food image to obtain image features of each sub-region;

[0108] S206, determining at least one target image based on the image features;

[0109] S208 , restoring at least one target image to a pollutant-free state.

[0110] In this embodiment, the food image can be divided into multiple sub-areas, so that in the process of detecting the food image, the multiple sub-areas of the food image are detected in turn to determine whether there are pollutants in the image content of each sub-area of ​​the food image. When there are pollutants in any sub-area of ​​the food image, the food image can be determined as a target image with pollutants in the image content.

[0111] Specifically, in the process of detecting food images, first, the food image can be traversed to obtain the image features of each sub-region of the food image, and then the image features of each sub-region can be analyzed to determine whether there are pollutants in the image content of each sub-region of the food image, that is, based on the image features of each sub-region of the food image, determine whether the food image is a target image with pollutants in the image content.

[0112] According to one embodiment of the present application, as shown in FIG3 , a method for controlling a cooking appliance is proposed, including:

[0113] S302, generating at least one image of the food during the cooking process of the food by the cooking utensil;

[0114] S304, traversing each food image to obtain image features of each sub-region;

[0115] S306, when the direction of optical flow in any sub-region is toward the direction of gravity and the brightness is greater than a preset brightness, determining the food image as the target image;

[0116] S308 , restoring at least one target image to a pollutant-free state.

[0117] In this embodiment, the image features of the food image may include optical flow direction, brightness, and color. Thus, after traversing the food image, it is possible to determine whether there is a contaminant in each sub-region of the food image based on the optical flow direction, brightness, or color of the sub-region.

[0118] Specifically, for contaminants formed by water droplets or mist, if the optical flow direction of any subregion of the food image is oriented toward gravity and the brightness of that subregion is greater than a preset brightness, it can be determined that contaminants formed by water droplets or mist are present in that subregion. This means that the food image is a target image containing contaminants.

[0119] It's understandable that when contaminants are present in a food image, the light flow in the area containing the contaminants is oriented toward gravity, due to the contaminants' tendency to move downward under the influence of gravity. Furthermore, due to the reflective properties of contaminants, the brightness of the contaminant area will be higher than that of other areas. Therefore, if the light flow in any sub-area of ​​the food image is oriented toward gravity and the brightness is greater than a preset brightness, contaminants can be determined to be present in that sub-area.

[0120] According to one embodiment of the present application, as shown in FIG4 , a method for controlling a cooking appliance is proposed, including:

[0121] S402, generating at least one image of the food during the cooking process of the food by the cooking utensil;

[0122] S404, traversing each food image to obtain image features of each sub-region;

[0123] S406, when the optical flow direction of any sub-region is toward the direction of gravity and the color is white, determining the food image as the target image;

[0124] S408: Restoring at least one target image to a pollutant-free state.

[0125] In this embodiment, the image features of the food image may include optical flow direction, brightness, and color. Thus, after traversing the food image, it is possible to determine whether there is a contaminant in each sub-region of the food image based on the optical flow direction, brightness, or color of the sub-region.

[0126] Specifically, for pollutants formed by water droplets or water mist, when the optical flow direction of any sub-region of the food image is toward the direction of gravity and the image color in the sub-region is white, it can be determined that pollutants formed by water droplets or water mist exist in the sub-region, that is, the food image is a target image with pollutants in the image content.

[0127] It is understood that when contaminants such as water droplets or mist are present in a food image, since these contaminants tend to move downward under the influence of gravity, the optical flow direction of the area containing these contaminants is oriented toward gravity. Furthermore, contaminants such as water droplets or mist appear white under illumination. Therefore, if the optical flow direction of any sub-region of the food image is oriented toward gravity and the color is white, it can be determined that contaminants such as water droplets or mist are present in that sub-region.

[0128] According to one embodiment of the present application, as shown in FIG5 , a method for controlling a cooking appliance is proposed, including:

[0129] S502, generating at least one image of the food during the cooking process of the food by the cooking utensil;

[0130] S504, traversing each food image to obtain image features of each sub-region;

[0131] S506, when the refractive index of any sub-region is less than a preset refractive index, determining the food image as a target image;

[0132] S508 , restoring at least one target image to a pollutant-free state.

[0133] In this embodiment, the image features may also include a refractive index. Accordingly, in the process of detecting each sub-region of the food image, if the refractive index in any sub-region of the food image is less than a preset refractive index, it indicates that there are contaminants in the sub-region, and the food image can be determined as a target image with contaminants in the image content.

[0134] It is understood that when contaminants such as water droplets or mist condense on the camera of an image acquisition device, the density of water is greater than that of air. Therefore, after the food image is generated, the refractive index of the subregion of the food image containing the contaminants will be significantly lower than the refractive index of other subregions. Accordingly, when inspecting the food image, if a subregion of the food image is determined to have a refractive index lower than a predetermined refractive index, the food image can be determined to be a target image containing contaminants such as water droplets or mist.

[0135] According to one embodiment of the present application, as shown in FIG6 , a method for controlling a cooking appliance is proposed, including:

[0136] S602, generating at least one image of the food during the cooking process of the food by the cooking utensil;

[0137] S604, traversing each food image to obtain image features of each sub-region;

[0138] S606, when the brightness, saturation or contrast of any sub-region is greater than a preset value, determining the food image as a target image;

[0139] S608: Restoring at least one target image to a pollutant-free state.

[0140] In this embodiment, the image features of the food image may also include brightness, saturation and contrast. Accordingly, in the process of detecting the food image, it is also possible to determine whether there are pollutants in each sub-area of ​​the food image based on the brightness, saturation and contrast of the sub-area.

[0141] Specifically, after traversing the food image, the brightness, saturation, and contrast of each subregion of the food image are determined. If the brightness of any subregion of the food image is greater than a preset brightness value, it indicates that a contaminant is present in that subregion. Alternatively, if the saturation of any subregion of the food image is greater than a preset saturation value, it indicates that a speed is present in that subregion. Alternatively, if the contrast of any subregion of the food image is greater than a preset contrast value, it indicates that a contaminant is present in that subregion.

[0142] According to one embodiment of the present application, as shown in FIG7 , a method for controlling a cooking appliance is proposed, including:

[0143] S702, generating at least one image of the food during the cooking process of the food by the cooking utensil;

[0144] S704, traversing each food image to obtain image features of each sub-region;

[0145] S706, when the difference between the image features of any two adjacent sub-regions is greater than a first preset difference, determining the food image as the target image;

[0146] S708 , restoring at least one target image to a pollutant-free state.

[0147] In this embodiment, after traversing the food image, the image features of each sub-region of the food image are determined, and further, whether there is a pollutant in the sub-region can be determined based on the difference between the image features of adjacent sub-regions.

[0148] Specifically, if the difference between the image features of any two adjacent sub-regions is greater than a first preset difference, it can be determined that a contaminant exists in at least one of the two adjacent sub-regions. In other words, the continuity between the two adjacent sub-regions of the same food image is not satisfied.

[0149] It is understood that within the same food image, if no subregion contains contaminants, the image features of two adjacent subregions will exhibit a continuous transition. Therefore, if the difference between the image features of any two adjacent subregions is greater than a first predetermined difference, it indicates that the two adjacent subregions in the food image do not meet the continuity requirement, and therefore, it can be determined that contaminants are present in the food image content.

[0150] According to one embodiment of the present application, as shown in FIG8 , a method for controlling a cooking appliance is proposed, including:

[0151] S802, generating at least one image of the food during the cooking process of the food by the cooking utensil;

[0152] S804, traversing each food image to obtain image features of each sub-region;

[0153] S806, when the difference between the image features of the same sub-region of two food images generated at adjacent times is greater than a second preset difference, determining that the subsequently generated food image is the target image;

[0154] S808 , restoring at least one target image to a pollutant-free state.

[0155] In this embodiment, when there are multiple food images, the area with pollutants can be restored based on the area corresponding to the area with pollutants in other images without pollutants.

[0156] After traversing each food image, image features of each corresponding sub-area in the two food images generated at adjacent times can be compared. If the difference between the image features of the same sub-area of ​​the two food images generated at adjacent times is greater than a second preset difference, it can be determined that there are contaminants in the image content of at least one of the two food images generated at adjacent times.

[0157] It is understood that, if no contaminants are present in the food images, the image features of the same subregion of food images generated at adjacent times will exhibit a continuous transition. Therefore, if the difference between the image features of the same subregion of two food images generated at adjacent times is greater than a second predetermined difference, it can be determined that a contaminant is present in the image content of at least one of the two food images generated at adjacent times.

[0158] According to one embodiment of the present application, as shown in FIG9 , a method for controlling a cooking appliance is proposed, including:

[0159] S902, generating at least one image of the food during the cooking process of the food by the cooking utensil;

[0160] S904, detecting at least one food image to determine at least one target image containing contaminants;

[0161] S906, determining a change trend of food in a sub-region containing pollutants in the target image based on at least one food image generated before the target image;

[0162] S908 , restoring the sub-region containing pollutants in the target image according to the change trend of the food ingredients in the sub-region containing pollutants.

[0163] In this embodiment, after determining that the food image is a target image with contaminants in its image content, the sub-region of the target image with contaminants in its image content can be restored based on at least one food image generated before the target image to eliminate the contaminants in the target image and ensure the image quality of the target image.

[0164] Specifically, the changing trend of the food within the subregion containing contaminants can be determined based on at least one food image generated before the target image. That is, the changing trend of the food within the image content is predicted based on the at least one food image generated before. The subregion containing contaminants in the target image is then restored based on the changing area of ​​the food. This allows the food within the subregion containing contaminants in the target image to be restored to its true image, ensuring the quality of the food image.

[0165] According to one embodiment of the present application, as shown in FIG10 , a method for controlling a cooking appliance is proposed, including:

[0166] S1002, generating at least one image of the food during the cooking process of the food by the cooking utensil;

[0167] S1004, detecting at least one food image to determine at least one target image containing contaminants;

[0168] S1006, obtaining a first sub-region containing pollutants in the target image;

[0169] S1008, obtaining a second sub-region corresponding to the first sub-region in the food image preceding the target image;

[0170] S1010, replacing the first sub-region with the second sub-region;

[0171] S1012: According to a preset algorithm, the replaced target image is smoothed to restore at least one target image to a pollutant-free state.

[0172] In this embodiment, after determining that the food image is a target image with contaminants in its image content, the sub-region of the target image with contaminants in its image content can be restored based on at least one food image generated before the target image to eliminate the contaminants in the target image and ensure the image quality of the target image.

[0173] Specifically, the first subregion of the target image containing contaminants is first determined. A second subregion corresponding to the first subregion in the food image generated before the target image is obtained. The first subregion is replaced with the second subregion. Specifically, the first subregion containing contaminants in the target image is deleted, and the second subregion corresponding to the first subregion in the food image before the target image, which does not contain contaminants, is placed in the first subregion's place.

[0174] It is understood that since the food image displayed in the first and second sub-regions contains the same content, replacing the first sub-region with the second sub-region can eliminate the contaminants in the image content. Furthermore, the replaced target image is smoothed according to a preset algorithm to restore the target image.

[0175] According to one embodiment of the present application, as shown in FIG11 , a method for controlling a cooking appliance is proposed, including:

[0176] S1102, generating at least one image of the food during the cooking process of the food by the cooking utensil;

[0177] S1104, detecting at least one food image to determine at least one target image containing contaminants;

[0178] S1106, obtaining a third sub-region where pollutants exist in the target image;

[0179] S1108, obtaining a fourth sub-region corresponding to the third sub-region in the food image preceding the target image;

[0180] S1110 , adjusting the image features of the third sub-region according to the image features of the fourth sub-region to restore at least one target image to a pollutant-free state.

[0181] In this embodiment, after determining that the food image is a target image with contaminants in its image content, the sub-region of the target image with contaminants in its image content can be restored based on at least one food image generated before the target image to eliminate the contaminants in the target image and ensure the image quality of the target image.

[0182] Specifically, the method first determines the third subregion of the target image where the contaminant is present. Then, a fourth subregion corresponding to the third subregion is obtained from a food image generated before the target image. Image features of the fourth subregion are then obtained, and the image features of the third subregion are adjusted based on the image features of the fourth subregion, thereby restoring the target image.

[0183] It can be understood that since the image content displayed by the third sub-region and the fourth sub-region are the same, and the generation time interval between the third sub-region and the fourth sub-region is short, the image features of the third sub-region can be adjusted according to the image features of the fourth sub-region, so as to eliminate the pollutants in the third sub-region and realize the restoration of the target image.

[0184] According to one embodiment of the present application, as shown in FIG12 , a method for controlling a cooking appliance is proposed, including:

[0185] S1202, generating at least one image of the food during the cooking process of the food by the cooking utensil;

[0186] S1204, detecting at least one food image to determine at least one target image containing contaminants;

[0187] S1206: Input the at least one target image into a preset neural network to restore the at least one target image to a pollutant-free state.

[0188] In this embodiment, after determining that a target image contains pollutants, the target image may be input into a preset neural network, and a restored target image may be outputted through the preset neural network.

[0189] It is understood that the preset neural network is trained using a large amount of historical image data, including images with and without pollutants. Therefore, the preset neural network is used to fit a functional mapping relationship from blurred image information of the polluted mist area to the original clear image information. Based on the fitted functional mapping relationship, the original clear image of the polluted mist area is restored.

[0190] According to the second aspect of the present application, a control device for a cooking appliance is proposed, comprising: a generation unit, for generating at least one food image of food during the process of the cooking appliance cooking food; a detection unit, for detecting at least one food image to determine at least one target image containing contaminants; and a restoration unit, for restoring at least one target image to a contaminant-free state.

[0191] The control device for a cooking appliance provided in the present application captures an image of the ingredients through a generation unit during the cooking process to generate at least one ingredient image, thereby meeting the user's personalized needs during the cooking process and improving the user experience. Furthermore, the detection unit can detect at least one ingredient image containing contaminants. Furthermore, if contaminants are present in the image content of at least one ingredient image, the restoration unit restores the area containing the contaminants in the ingredient image to eliminate the impact of the contaminants on the quality of the ingredient image, retaining the original effect of the ingredients on the ingredient image, further improving the user's demand for the quality of the ingredient image, and further enhancing the user experience of the cooking appliance.

[0192] Furthermore, the food image includes multiple sub-regions, and the detection unit is specifically used to: traverse each food image to obtain image features of each sub-region; and determine at least one target image based on the image features.

[0193] Furthermore, the image features include optical flow direction, brightness and color, and the detection unit is also used to: determine that the food image is the target image when the optical flow direction of any sub-area is toward the direction of gravity and the brightness is greater than a preset brightness; or determine that the food image is the target image when the optical flow direction of any sub-area is toward the direction of gravity and the color is white.

[0194] Furthermore, the image feature includes a refractive index, and the detection unit is further configured to: determine that the food image is a target image when the refractive index of any sub-region is less than a preset refractive index.

[0195] Furthermore, the image features include brightness, saturation and contrast, and the detection unit is further used to: determine that the food image is the target image when the brightness, saturation or contrast of any sub-area is greater than a preset value.

[0196] Furthermore, the detection unit is further configured to: determine that the food image is the target image when the difference between the image features of any two adjacent sub-regions is greater than a first preset difference.

[0197] Furthermore, the detection unit is also used to: when the difference between the image features of the same sub-region of two food images generated at adjacent times is greater than a second preset difference, determine that the subsequently generated food image is the target image.

[0198] Furthermore, the restoration unit is specifically used to: determine the change trend of the food in the sub-area where pollutants exist in the target image based on at least one food image generated before the target image; and restore the sub-area where pollutants exist in the target image based on the change trend of the food in the sub-area where pollutants exist.

[0199] Furthermore, the restoration unit is also used to: obtain a first sub-region in the target image where pollutants exist; obtain a second sub-region corresponding to the first sub-region in the food image before the target image; replace the first sub-region with the second sub-region; and smooth the replaced target image according to a preset algorithm to restore at least one target image.

[0200] Furthermore, the restoration unit is also used to: obtain a third sub-region in the target image where contaminants exist; obtain a fourth sub-region corresponding to the third sub-region in the previous food image of the target image; and adjust the image features of the third sub-region based on the image features of the fourth sub-region to restore at least one target image.

[0201] Furthermore, the restoration unit is also used to: input at least one target image into a preset neural network to restore the at least one target image.

[0202] According to a third aspect of the present application, a control device for a cooking appliance is proposed, comprising: a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the control method for the cooking appliance provided in the first aspect are implemented.

[0203] The control device for a cooking appliance provided in the present application includes a memory and a processor, and also includes a program or instruction stored in the memory. When the program or instruction is executed by the processor, the steps of the control method for a cooking appliance of the first aspect mentioned above can be implemented. Therefore, the control device for the cooking appliance has all the beneficial effects of the control method for a cooking appliance mentioned above, which will not be repeated here.

[0204] According to a fourth aspect of the present application, a cooking appliance is provided, comprising a control device for the cooking appliance according to any one of the above technical solutions.

[0205] The cooking appliance provided in the present application includes the cooking appliance control device of any one of the above technical solutions. Therefore, the cooking appliance has all the beneficial effects of the above cooking appliance control device, which will not be repeated here.

[0206] According to a fifth aspect of the present application, a readable storage medium is proposed, on which a program or instruction is stored. When the program or instruction is executed by a processor, a control method for a cooking appliance as described in any one of the above technical solutions is implemented.

[0207] The readable storage medium provided in the present application stores a program or instruction thereon. When the program or instruction is executed by the processor, it can implement a control method for a cooking appliance as described in any one of the above-mentioned technical solutions. Therefore, the storage medium has all the beneficial effects of the control method for the above-mentioned cooking appliance, which will not be repeated here.

[0208] The methods may be implemented in a variety of different ways depending on the specific features and / or example applications. For example, the methods may be implemented through a combination of hardware, firmware, and / or software. For example, in a hardware implementation, the processor may be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, electronic devices, other device units for performing the above functions, and / or combinations thereof.

[0209] A computer-readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. A computer-readable storage medium can be, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes: a portable computer floppy disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory card, a floppy disk, an encoding mechanical device (such as a punched card or a groove with a raised structure on which instructions are recorded), and any suitable combination of the foregoing. As used herein, a computer-readable storage medium should not be understood as a transmission signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium, or electrical signals transmitted through wires.

[0210] In the description of this application, the term "plurality" refers to two or more than two. Unless otherwise expressly defined, the orientation or positional relationship indicated by the terms "upper" and "lower" is based on the orientation or positional relationship shown in the accompanying drawings. It is only for the convenience of describing this application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on this application. The terms "connection", "installation", "fixed", etc. should be understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to the specific circumstances.

[0211] Throughout this specification, terms such as "one embodiment," "some embodiments," and "specific embodiments" mean that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0212] The above are merely preferred embodiments of the present application and are not intended to limit the present application. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A method for controlling a cooking appliance, wherein: include: During the process of the cooking utensil cooking the food, generating at least one food image of the food; Detecting at least one of the food images to determine at least one target image containing contaminants; Restoring at least one of the target images to a contaminant-free state.

2. The control method according to claim 1, wherein: The food image includes a plurality of sub-areas, and the detecting at least one of the food images to determine at least one target image containing contaminants includes: Traversing each of the food images to obtain image features of each of the sub-regions; At least one target image is determined according to the image features.

3. The control method according to claim 2, wherein: The image features include optical flow direction, brightness and color, and determining at least one target image according to the image features includes: In a case where the optical flow direction of any of the sub-areas is toward the gravity direction and the brightness is greater than a preset brightness, determining that the food image is the target image; or When the optical flow direction of any of the sub-regions is toward the gravity direction and the color is white, the food image is determined to be the target image.

4. The control method according to claim 2, wherein: The image feature includes a refractive index, and determining at least one target image according to the image feature includes: When the refractive index of any of the sub-regions is less than a preset refractive index, the food image is determined to be the target image.

5. The control method according to claim 2, wherein: The image features include brightness, saturation and contrast, and determining at least one target image according to the image features includes: When the brightness, the saturation or the contrast of any of the sub-areas is greater than a preset value, the food image is determined to be the target image.

6. The control method according to claim 2, wherein: The step of determining at least one target image according to the image features includes: When the difference between the image features of any two adjacent sub-areas is greater than a first preset difference, the food image is determined to be the target image.

7. The control method according to claim 2, wherein: The step of determining at least one target image according to the image features includes: When the difference between the image features of the same sub-region of two food images generated at adjacent times is greater than a second preset difference, it is determined that the subsequently generated food image is the target image.

8. The control method according to any one of claims 2 to 7, wherein: Restoring at least one of the target images to a pollutant-free state includes: Determining, based on at least one food image generated before the target image, a change trend of food in the sub-region where pollutants are present in the target image; The sub-region in the target image where the pollutants exist is restored according to the change trend of the food in the sub-region where the pollutants exist.

9. The control method according to any one of claims 2 to 7, wherein: Restoring at least one of the target images to a pollutant-free state includes: Acquire a first sub-region where pollutants exist in the target image; Acquire a second sub-region corresponding to the first sub-region in the food image preceding the target image; replacing the first sub-region with the second sub-region; According to a preset algorithm, the replaced target images are smoothed to restore at least one of the target images to a pollutant-free state.

10. The control method according to any one of claims 2 to 7, wherein: Restoring at least one of the target images to a pollutant-free state includes: Acquire a third sub-region where pollutants exist in the target image; Acquire a fourth sub-region corresponding to the third sub-region in the food image preceding the target image; The image features of the third sub-region are adjusted according to the image features of the fourth sub-region to restore at least one of the target images to a pollutant-free state.

11. The control method according to any one of claims 2 to 7, wherein: Restoring at least one of the target images to a pollutant-free state includes: At least one of the target images is input into a preset neural network to restore at least one of the target images to a pollutant-free state.

12. The control method according to any one of claims 1 to 7, wherein: The pollutants include water droplets or water mist.

13. A control device for a cooking appliance, wherein: include: a generating unit, which generates at least one food image of the food during the cooking process of the food by the cooking utensil; A detection unit, configured to detect at least one of the food images and determine at least one target image containing contaminants; The restoration unit is used to restore at least one of the target images to a state without contaminants.

14. A control device for a cooking appliance, wherein: include: A processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the control method of the cooking appliance according to any one of claims 1 to 12 are implemented.

15. A cooking appliance, wherein: include: A control device for a cooking appliance as claimed in claim 13 or 14.

16. A readable storage medium, wherein: The readable storage medium stores a program or an instruction, and when the program or the instruction is executed by a processor, the steps of the method for controlling a cooking appliance according to any one of claims 1 to 12 are implemented.

Images