A cooking image processing method and device, electronic equipment and storage medium
By acquiring and fusing cooking images, and combining them with a template cooking image set and a maturity classification model, the problem of misjudgment in ingredient maturity identification was solved, achieving higher recognition accuracy and precise control of the cooking process.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NINGBO FOTILE KITCHEN WARE CO LTD
- Filing Date
- 2026-04-16
- Publication Date
- 2026-07-21
AI Technical Summary
In existing technologies, methods for recognizing the ripeness of ingredients based on cooking images are prone to misjudgment, especially for meat ingredients that are similar in color and texture during cooking, which affects the precise control of the cooking process and the quality of the ingredients.
By acquiring images of ingredients in the cooking state, fusing the acquired images with target interference images, and using a template cooking image set for maturity recognition, including Poisson fusion and weighted fusion processing, the maturity of ingredients is classified in combination with a target maturity classification model.
It improves the accuracy and robustness of ingredient ripeness identification, ensures precise control of the cooking process, and enhances cooking quality.
Smart Images
Figure CN122435596A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of cooking image processing technology, and in particular to a cooking image processing method, apparatus, electronic device, and storage medium. Background Technology
[0002] In related technologies, the identification of food ripeness mostly relies on directly analyzing captured cooking images. Specifically, this involves comparing the captured cooking images with pre-defined standard cooking images and calculating the similarity to infer the ripeness of the food. However, this method has limitations in practical applications. Because different foods may have high similarity in appearance—for example, some meats may have similar colors, textures, and shapes during cooking—the above identification method is prone to errors, misjudging ripeness and misidentifying undercooked meat (A) as fully cooked meat (B). This error not only affects the precise control of the cooking process but may also lead to the final cooked food not meeting expectations. Therefore, improving the accuracy of food ripeness identification based on cooking images is a pressing technical challenge that needs to be addressed. Summary of the Invention
[0003] To address at least one of the aforementioned technical problems, this disclosure provides a cooking image processing method, apparatus, electronic device, and storage medium.
[0004] On one hand, the present invention provides a cooking image processing method, the method comprising: Images of ingredients in a cooking state are captured to obtain captured images; The acquired image and the first target interference image are fused to obtain the cooking image to be identified. The first target interference image is an interference image that corresponds to the food category in the acquired image. Based on a template cooking image set, maturity recognition processing is performed on the cooking image to be identified to obtain the maturity information corresponding to the cooking image to be identified. The template cooking image set includes multiple template cooking images, which are images obtained by fusing a standard cooking image with interference images corresponding to the ingredients in the standard cooking image. The standard cooking image is an image of different ingredients at different maturity levels. The pixel values of each pixel in the same interference image are the same, while the pixel values of the interference images corresponding to different ingredient categories are different. The color saturation values corresponding to different interference images are all greater than the preset color saturation threshold.
[0005] In an optional embodiment, the steps for obtaining a standard set of cooking images include the following: Determine the candidate sampling time period, which is the time period during the cooking process of the target ingredient when it changes from the first preset maturity to the second preset maturity. The target ingredient can be any ingredient among different ingredients. A preset number of time periods in the candidate sampling period are all determined as target sampling periods. The preset number of time periods correspond one-to-one with the maturity level. The time interval between the end time of the previous time period and the start time of the next time period in two adjacent time periods meets the preset duration. The duration of each of the preset number of time periods decreases as the cooking progresses. The cooking images of the target ingredients during the target sampling period are sampled to obtain a preset number of standard cooking images corresponding to the target ingredients; The standard cooking images corresponding to each ingredient are defined as the standard cooking image set.
[0006] In an optional embodiment, the step of obtaining the template cooking image set includes the following: Determine a first image region and a second image region in a target cooking image. The target cooking image is any standard cooking image. The first image region is the food region, and the second image region is any region other than the first image region. Poisson fusion processing is performed on the image regions associated with the first image region and the second target interference image to obtain the first fused image. The second target interference image is the interference image corresponding to the target cooking image. A weighted fusion process is performed on the second image region and the image regions in the second target interference image that are associated with the second image region to obtain the second fused image; Based on the first fused image and the second fused image, a template cooking image corresponding to the target cooking image is obtained; The template cooking images corresponding to each standard cooking image are determined as the template cooking image set.
[0007] In an optional embodiment, based on a template cooking image set, a maturity recognition process is performed on the cooking image to be identified to obtain maturity information corresponding to the cooking image to be identified, including: The cooking image to be identified is input into the target maturity classification model for maturity classification to obtain target maturity information. The target maturity classification model is a model trained on a preset classification model using a set of template cooking images. The target maturity information is used to indicate the classification result of the cooking image to be identified in a preset number of maturity levels. The target maturity information is defined as maturity information.
[0008] In an optional embodiment, each template cooking image in the template cooking image set carries a maturity label, which indicates the maturity level associated with the collection time period of each template cooking image. The training process of the target maturity classification model includes the following: Each template cooking image is input into a preset classification model for maturity classification, and sample maturity information is obtained. The sample maturity information indicates the classification result of each template cooking image in a preset number of maturity levels. Based on the sample maturity information and maturity labels, determine the loss information; Based on the loss information, the parameters of the preset classification model are adjusted until the preset parameter tuning stopping condition is reached. The preset classification model that reaches the preset parameter tuning stopping condition is used as the trained target maturity recognition model.
[0009] In an optional embodiment, based on a template cooking image set, a maturity recognition process is performed on the cooking image to be identified to obtain maturity information corresponding to the cooking image to be identified, including: Feature extraction is performed on each template cooking image to obtain the template image features corresponding to each template cooking image. Feature extraction is performed on the cooking image to be identified to obtain the cooking image features corresponding to the cooking image to be identified; The similarity between the features of the cooking image and the template image is compared to obtain the comparison results; If the comparison results indicate that the target image features exist in the template image features, the maturity level associated with the template cooking image corresponding to the target image features is determined as maturity information, and the similarity between the target image features and the cooking image features meets the preset similarity conditions.
[0010] In an optional embodiment, the method further includes: When the maturity information indicates that the current maturity meets the preset maturity, a cooking prompt message is generated to prompt the user to end cooking; If the maturity information indicates that the current maturity does not meet the preset maturity, the cooking equipment will continue to perform the cooking operation.
[0011] In a second aspect, the present invention also provides a cooking image processing apparatus, comprising: The first image acquisition module is used to acquire images of the food in the cooking state and obtain the acquired images; The image fusion module is used to fuse the acquired image with the first target interference image to obtain the cooking image to be identified. The first target interference image is an interference image corresponding to the food category in the acquired image. The maturity recognition module is used to perform maturity recognition processing on the cooking images to be identified based on a template cooking image set, and obtain the maturity information corresponding to the cooking images to be identified. The template cooking image set includes multiple template cooking images, which are images obtained by fusing a standard cooking image with interference images corresponding to the ingredients in the standard cooking image. The standard cooking image is an image of different ingredients at different maturity levels. The pixel values of each pixel in the same interference image are the same, while the pixel values of the interference images corresponding to different ingredient categories are different. The color saturation values corresponding to different interference images are all greater than the preset color saturation threshold.
[0012] Thirdly, the present invention also provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is used to execute instructions to implement the cooking image processing method described above.
[0013] Fourthly, the present invention also provides a storage medium that, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to perform the above-described cooking image processing method.
[0014] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.
[0015] Implementing this disclosure will have the following beneficial effects: Images of ingredients in a cooking state are acquired to obtain acquired images. The acquired images are then fused with a first target interference image to obtain a cooking image to be identified. The first target interference image is an interference image corresponding to the ingredient category in the acquired images. Based on a template cooking image set, the cooking image to be identified is processed for maturity recognition to obtain maturity information corresponding to the cooking image to be identified. The template cooking image set includes multiple template cooking images, which are images obtained by fusing a standard cooking image with interference images corresponding to the ingredients in the standard cooking image. The standard cooking images are images of different ingredients at different maturity levels. The pixel values of each pixel in the same interference image are the same, while the pixel values of the interference images corresponding to different ingredient categories are different. The color saturation values of different interference images are all greater than a preset color saturation threshold.
[0016] This disclosure obtains the cooking image to be identified and the template cooking image by fusing images of different ingredients and standard cooking images with the corresponding interference images of those ingredients. By fusing with interference images with single pixel values and high saturation, the difference between images of different ingredients can be increased, thereby improving the robustness of the matching process between the cooking image to be identified and the template cooking image. Furthermore, by performing maturity recognition processing on the cooking image to be identified based on the template cooking image set, the accuracy of maturity recognition can be improved.
[0017] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description
[0018] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The accompanying drawings are incorporated in and constitute a part of this specification, illustrating embodiments consistent with this disclosure, and are used together with the description to explain the principles of this disclosure, and do not constitute an improper limitation of this disclosure.
[0019] Figure 1 This is a schematic diagram of an implementation environment according to an exemplary embodiment; Figure 2 This is a flowchart illustrating a cooking image processing method according to an exemplary embodiment; Figure 3 This is a flowchart illustrating a method for obtaining a standard set of cooking images according to an exemplary embodiment; Figure 4 This is a schematic diagram illustrating a target sampling period according to an exemplary embodiment; Figure 5 This is a flowchart illustrating an exemplary embodiment for obtaining a template cooking image set; Figure 6 This is a flowchart illustrating cooking control based on ripeness information according to an exemplary embodiment; Figure 7 This is a schematic diagram of a cooking image processing apparatus according to an exemplary embodiment; Figure 8 This is a block diagram illustrating an electronic device for processing cooking images according to an exemplary embodiment. Detailed Implementation
[0020] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.
[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.
[0022] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. Like reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise. The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.
[0023] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.
[0024] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.
[0025] Please see Figure 1 , Figure 1This is a schematic diagram illustrating an application environment according to an exemplary embodiment, such as... Figure 1 As shown, the application environment may include server 01 and cooking equipment 02.
[0026] In an optional embodiment, the cooking device 02 can be used to process the cooking image using a cooking image processing method. For example, the cooking device 02 acquires images of ingredients in a cooking state to obtain acquired images. The cooking device 02 then fuses the acquired images with a first target interference image to obtain a cooking image to be identified. Subsequently, the cooking device 02 performs maturity recognition processing on the cooking image to be identified based on a template cooking image set to obtain maturity information corresponding to the cooking image to be identified.
[0027] It should be noted that the above-described application environment is merely one provided by this disclosure. In practical applications, the cooking device 02 can also be combined with the server 01 to process the cooking image. For example, the cooking device 02 can capture images of ingredients in a cooking state, obtain the captured images, and send them to the server 01. The server 01 then fuses the captured images with a first target interference image to obtain the cooking image to be identified. Based on a template cooking image set, the server performs maturity recognition processing on the cooking image to be identified to obtain the maturity information corresponding to the cooking image to be identified and sends it to the cooking device 02.
[0028] For example, server 01 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.
[0029] In the embodiments described in this specification, the server 01 and the cooking device 02 can be directly or indirectly connected via wired or wireless communication, and this disclosure does not impose any limitations.
[0030] Figure 2 This is a flowchart illustrating a cooking image processing method according to an exemplary embodiment, such as... Figure 2 As shown, the cooking image processing method includes the following: Step S201: Acquire images of the ingredients in the cooking state to obtain the acquired images.
[0031] In this embodiment of the disclosure, image acquisition of food during cooking can be achieved by capturing images of the inner pot of the cooking device through a camera configured on the cooking equipment. The acquisition time can be determined according to a preset acquisition time interval. For example, in order to enable the acquired image to better reflect the color and texture features of the food surface, the camera is a camera that supports red, green, and blue three channels, and the acquired image can be obtained after noise reduction processing.
[0032] Step S202: Fuse the acquired image with the first target interference image to obtain the cooking image to be identified. The first target interference image is an interference image corresponding to the food category in the acquired image.
[0033] In this embodiment of the present disclosure, fusing the acquired image and the first target interference image may involve first determining a first sub-image region and a second sub-image region in the acquired image, wherein the first sub-image region is the food region in the acquired image, and the second sub-image region is the other regions besides the first sub-image region; then, performing Poisson fusion processing on the first sub-image region and the image regions in the first target interference image associated with the first sub-image region to obtain a first sub-fused image; performing weighted fusion processing on the second sub-image region and the image regions in the first target interference image associated with the second sub-image region to obtain a second sub-fused image; finally, integrating the first sub-fused image and the second sub-fused image into a cooking image to be identified.
[0034] The first target interference image is an image with a single pixel value and a color saturation value greater than a preset color saturation threshold. The pixel value of the first target interference image corresponding to the acquired image is determined based on the food category in the acquired image. In this embodiment, a correspondence between different food categories and different image pixel values is pre-defined. After determining the food category in the acquired image, the interference image with the same pixel value as the food category is determined as the first target interference image. Optionally, the food category in the acquired image can be determined by user input.
[0035] Step S203: Based on the template cooking image set, perform maturity recognition processing on the cooking image to be identified to obtain the maturity information corresponding to the cooking image to be identified. The template cooking image set includes multiple template cooking images. The template cooking image is an image obtained by fusing a standard cooking image with the interference images corresponding to the ingredients in the standard cooking image. The standard cooking image is an image of different ingredients at different maturity levels. The pixel values of each pixel in the same interference image are the same. The pixel values of the interference images corresponding to different ingredient categories are different. The color saturation values corresponding to different interference images are all greater than the preset color saturation threshold.
[0036] Figure 3This is a flowchart illustrating an exemplary embodiment for obtaining a standard cooking image set. In an optional embodiment, the steps for obtaining the standard cooking image set include the following: Step S301: Determine the candidate sampling time period. The candidate sampling time period is the time period during which the target ingredient changes from the first preset maturity to the second preset maturity during the cooking process. The target ingredient is any ingredient among different ingredients.
[0037] In this embodiment of the disclosure, the degree of maturity of the ingredients indicated by the first preset maturity is greater than the degree of maturity of the ingredients indicated by the second preset maturity. The first preset maturity and the second preset maturity can be set according to the user's cooking needs.
[0038] When the user's cooking requirement is to cook ingredients from raw to cooked, and the cooking purpose is to cook the ingredients to a cooked state, the first preset maturity can be that the ingredients are raw, and the second preset maturity can be that the ingredients are cooked. Therefore, the candidate sampling time period in this case can be the time period of the complete ingredient cooking operation.
[0039] When the user's cooking requirement is to cook the ingredients from just cooked to a state with a certain appearance and color, and the cooking purpose is to color the outside of the ingredients by continuing to cook, the first preset degree of maturity can be that the ingredients are just cooked, and the second preset degree of maturity can be the degree of coloring specified by the user. Therefore, the candidate sampling time period in this case can be the time period corresponding to the cooking operation process from just cooked to the outside of the ingredients being charred, which is determined based on prior knowledge.
[0040] Step S302: Determine a preset number of time periods from the candidate sampling time periods as target sampling time periods. The preset number of time periods correspond one-to-one with the maturity level. The time interval between the end time of the previous time period and the start time of the next time period in two adjacent time periods meets the preset duration. The duration of each of the preset number of time periods decreases as the cooking progresses.
[0041] In this embodiment, maturity levels can be divided into first and second preset maturity levels based on the external changes in ingredients during the cooking process, from a first preset maturity level to a second preset maturity level. Multiple maturity levels are determined from these levels, and the preset number represents the number of maturity levels. For example, taking bread cooking as an example where the user's cooking requirement is to cook the bread from just baked to a caramel-colored state, four maturity levels can be determined: just baked, 30% caramel-colored, 70% caramel-colored, and fully caramel-colored. Therefore, it is necessary to determine the ingredient image sampling time periods that represent these four maturity levels from the candidate sampling time periods. Since the rate of change in the external features of the ingredients is faster towards the end of the cooking process, the sampling time period is shorter towards the end of the cooking process to prepare standard cooking images that represent the maturity levels. Therefore, the duration of each of the preset number of time periods decreases as the cooking progresses. To ensure that the standard cooking images collected in two adjacent time periods are not too similar, the time interval between the end time of the previous time period and the start time of the next time period must meet the preset duration.
[0042] Figure 4 This is a schematic diagram illustrating a target sampling period according to an exemplary embodiment. For example... Figure 4 As shown, on the cooking time axis 0-T, To preset a number of time periods, the first time period... The starting point τ represents the first preset maturity level, and the last time period The end time represents the second preset maturity level. to The duration of the sampling period decreases. The duration of the time period is , The duration of the time period is , The duration of the time period is , The duration of the time period is Specifically, the duration of the preceding time period is twice the duration of the following time period, and the time interval between the end of the preceding time period and the start of the following time period is... In this embodiment, to improve recognition accuracy, the aforementioned acquired images can also be obtained by acquiring images of the cooking cavity within the target sampling period.
[0043] Step S303: Sample the cooking images of the target ingredients during the target sampling period to obtain a preset number of standard cooking images corresponding to the target ingredients.
[0044] In this embodiment of the disclosure, sampling the cooking images of the target ingredients during the target sampling period can be performed by acquiring the cooking images of the target ingredients at any time within each time period, resulting in a preset number of standard cooking images corresponding to the target ingredients. To improve the accuracy of the standard cooking images, the cooking images of the target ingredients can also be acquired at the midpoint of each time period, resulting in a preset number of standard cooking images corresponding to the target ingredients. Each time period corresponds to one standard cooking image.
[0045] Step S304: Determine the standard cooking images corresponding to each ingredient as a standard cooking image set.
[0046] Based on the above, the sampling strategy adopted in this embodiment, which involves a preset number of time periods whose durations decrease with the cooking progress, can effectively maintain the consistency of food color and texture within the sampling interval, avoid the overlap and misalignment of sampling results, improve the independence of the color and texture of the sampling results, and thus improve the accuracy of the standard cooking image set.
[0047] Figure 5 This is a flowchart illustrating an exemplary embodiment for obtaining a template cooking image set. In an optional embodiment, the steps for obtaining the template cooking image set include the following: Step S501: Determine the first image region and the second image region in the target cooking image. The target cooking image is any standard cooking image. The first image region is the ingredient region, and the second image region is the region other than the first image region.
[0048] In this embodiment of the disclosure, determining the first image region and the second image region in the target cooking image may involve first confirming the position of the ingredients in the target cooking image, determining the rectangular range where the ingredients are located as the first image region, and determining the other regions in the target cooking image other than the first image region as the second image region.
[0049] Step S502: Perform Poisson fusion processing on the image regions associated with the first image region and the second target interference image to obtain the first fused image. The second target interference image is the interference image corresponding to the target cooking image.
[0050] In this embodiment of the disclosure, the Poisson fusion algorithm is an image fusion algorithm. The Poisson fusion processing of the first image region and the image region associated with the first image region in the second target interference image can be expressed as the following formula (1): (1) In equation (1), For the first fused image, This represents the Poisson fusion algorithm. This refers to the image region in the second target interference image that is associated with the first image region. This is the first image region.
[0051] Step S503: Perform weighted fusion processing on the second image region and the image regions in the second target interference image that are associated with the second image region to obtain the second fused image.
[0052] In this embodiment of the disclosure, the weighted fusion processing of the second image region and the image region associated with the second image region in the second target interference image can be expressed as the following formula (2): (2) In equation (2), For the second fused image, For weight values, , The image region in the second target interference image that is associated with the second image region. This is the second image region.
[0053] Step S504: Based on the first fused image and the second fused image, obtain the template cooking image corresponding to the target cooking image.
[0054] In this embodiment of the disclosure, obtaining the template cooking image corresponding to the target cooking image based on the first fused image and the second fused image can be achieved by merging the first fused image and the second fused image into a complete image of the same size as the target cooking image, and then determining the merged complete image as the template cooking image corresponding to the target cooking image.
[0055] Step S505: Determine the template cooking images corresponding to each standard cooking image as a template cooking image set.
[0056] Based on the above, in this embodiment of the present disclosure, by performing Poisson fusion processing on the image regions associated with the first image region in the first image region and the second target interference image, the texture features of the food part can be preserved; by performing weighted fusion processing on the image regions associated with the second image region in the second image region and the second target interference image, the independence of images of different food categories can be enhanced and the similarity of images of different food can be reduced.
[0057] In an optional embodiment, based on a template cooking image set, a maturity recognition process is performed on the cooking image to be identified to obtain maturity information corresponding to the cooking image to be identified, including: The cooking image to be identified is input into the target maturity classification model for maturity classification to obtain target maturity information. The target maturity classification model is a model trained on a preset classification model using a set of template cooking images. The target maturity information is used to indicate the classification result of the cooking image to be identified in a preset number of maturity levels.
[0058] In this embodiment of the disclosure, the target maturity classification model can classify the input cooking image to be identified, and then determine the maturity level to which the cooking image to be identified belongs. Specifically, the process of the target maturity classification model classifying the cooking image to be identified includes: classifying the cooking image to be identified by the target maturity classification model, outputting the probability values of the cooking image to be identified belonging to each preset number of maturity levels, and then determining the maturity level corresponding to the highest probability value that is greater than the preset probability value as the classification result, which is the target maturity information.
[0059] The target maturity information is defined as maturity information.
[0060] In this embodiment of the disclosure, the target maturity information output by the model is the maturity information of the cooking image to be identified.
[0061] As can be seen from the above embodiments, the present disclosure embodiments improve the accuracy of identifying the maturity information of the current ingredients by inputting the cooking image to be identified into the target maturity classification model for maturity classification and determining the target maturity information as maturity information, thereby improving the accuracy of cooking operations.
[0062] In an optional embodiment, each template cooking image in the template cooking image set carries a maturity label, which indicates the maturity level associated with the collection time period of each template cooking image. The training process of the target maturity classification model includes the following: Each template cooking image is input into a preset classification model for maturity classification, and sample maturity information is obtained. The sample maturity information indicates the classification result of each template cooking image in a preset number of maturity levels.
[0063] In this embodiment of the disclosure, the later the data collection time period, the higher the maturity level associated with that time period. Therefore, the number of classifications in the preset maturity classification model is determined based on a preset number of time periods, which is the preset number. The preset maturity classification model can classify each input template cooking image, thereby determining the maturity level to which the template cooking image belongs. Specifically, the process of the preset maturity classification model classifying the maturity of each template cooking image includes: classifying each template cooking image by the preset maturity classification model, outputting the probability value of each template cooking image belonging to each preset number of maturity levels, and then determining the maturity level corresponding to the highest value among the multiple probability values for each template cooking image as the maturity level corresponding to each template cooking image, thereby obtaining the sample maturity information.
[0064] Loss information is determined based on sample maturity information and maturity labels.
[0065] In this embodiment of the disclosure, determining loss information based on sample maturity information and maturity labels can identify the difference between sample maturity information and maturity labels as reference data required for calculating loss information. Then, the above data is substituted into a preset loss function to obtain loss information.
[0066] Based on the loss information, the parameters of the preset classification model are adjusted until the preset parameter tuning stopping condition is reached. The preset classification model that reaches the preset parameter tuning stopping condition is used as the trained target maturity recognition model.
[0067] In this embodiment of the disclosure, the preset parameter tuning stopping condition may be that the number of parameter tuning attempts reaches a preset number, or the loss information is less than a preset loss value.
[0068] With the preset parameter tuning stopping condition being that the number of parameter tuning attempts reaches a preset number, the parameters of the preset classification model are adjusted once after each model output, and the number of parameter tuning attempts is recorded once, until the number of parameter tuning attempts reaches the preset number, and a trained target maturity recognition model is obtained.
[0069] With the preset parameter tuning stopping condition being that the loss information is less than the preset loss value, after each loss is calculated, the loss information is compared to the preset loss value. When the loss information is not less than the preset loss value, the parameters of the preset classification model are adjusted until the loss information is less than the preset loss value, thus obtaining the trained target maturity recognition model.
[0070] As can be seen from the above, by grouping template cooking images of different ingredients within the same time period into one category, it is no longer necessary to train a dedicated recognition model for each type of ingredient, nor is it necessary to train a single model to distinguish different ingredients. This greatly reduces the number of classifications, lowers the complexity of model classification, and reduces the performance requirements of the model. By determining the loss information based on the sample maturity information and maturity labels, the loss value corresponding to the maturity classification function of the model can be determined. Based on the loss information, the parameters of the preset classification model can be adjusted, making the output of the preset classification model more accurate.
[0071] In an optional embodiment, based on a template cooking image set, ripeness recognition processing is performed on the cooking image to be identified to obtain ripeness information corresponding to the cooking image to be identified, and the process further includes: Feature extraction is performed on each template cooking image to obtain the template image features corresponding to each template cooking image.
[0072] In this embodiment of the present disclosure, feature extraction for each template cooking image may be performed by extracting features from the food region and non-food region of each template cooking image separately, to obtain the first sub-template image features corresponding to the food region and the second sub-template image features corresponding to the non-food region of each template cooking image. Then, the first sub-template image features and the second sub-template image features corresponding to each template cooking image are concatenated to obtain the template image features corresponding to each template cooking image.
[0073] Feature extraction is performed on the cooking image to be identified to obtain the cooking image features corresponding to the cooking image to be identified.
[0074] In this embodiment of the disclosure, feature extraction of the cooking image to be identified can be performed by extracting features from the food region and non-food region of the cooking image to be identified separately, to obtain a first sub-cooking image feature corresponding to the food region and a second sub-cooking image feature corresponding to the non-food region of the cooking image to be identified. Then, the first sub-cooking image feature and the second sub-cooking image feature corresponding to the cooking image to be identified are concatenated to obtain the cooking image feature corresponding to the cooking image to be identified.
[0075] The similarity between the features of the cooking image and the template image is compared to obtain the comparison results.
[0076] In this embodiment of the disclosure, the similarity comparison between cooking image features and template image features can be performed by sequentially comparing the cooking image features with the template image features corresponding to each template cooking image, and the similarity information between the cooking image features and each template image feature is used as the comparison result.
[0077] Similarity comparison between cooking image features and template image features can also be performed by dividing the images in the template cooking image set into a preset number of sub-template cooking image sets based on the order of the acquisition time periods of each template cooking image within a preset number of time periods. Template cooking images within the same sub-template cooking image set have the same acquisition time period order within the preset number of time periods, while template cooking images in different sub-template cooking image sets have different acquisition time periods order within the preset number of time periods. Subsequently, the cooking image features are compared with the template image features corresponding to the template cooking images in the first sub-template cooking image set to obtain a first comparison result. If the first comparison result indicates that there is no template image feature in the template image features corresponding to the template cooking images in the first sub-template cooking image set whose similarity to the cooking image features meets the preset similarity condition, then the cooking image features are compared with the template image features corresponding to the template cooking images in the second sub-template cooking image set to obtain a second comparison result. This process continues until a template image feature whose similarity to the cooking image features meets the preset similarity condition is found, or all sub-template image sets have completed the similarity comparison.
[0078] If the comparison results indicate that the target image features exist in the template image features, the maturity level associated with the template cooking image corresponding to the target image features is determined as maturity information, and the similarity between the target image features and the cooking image features meets the preset similarity conditions.
[0079] In this embodiment of the disclosure, if the comparison result indicates that there is a similarity between the template image features and the cooking image features that meets the preset similarity condition, the maturity level associated with the template cooking image corresponding to the template image features is determined as the maturity information corresponding to the cooking image to be identified, that is, the maturity information of the current ingredients.
[0080] As can be seen from the above embodiments, the present disclosure obtains a comparison result by comparing the similarity between cooking image features and template image features; when the comparison result indicates that there are target image features in the template image features, the maturity level associated with the template cooking image corresponding to the target image features is determined as maturity information, which can improve the accuracy of identifying the maturity information of the current ingredients, thereby improving the accuracy of cooking operations.
[0081] Figure 6 This is a flowchart illustrating cooking control based on ripeness information according to an exemplary embodiment. In an optional embodiment, the method further includes: Step S601: If the maturity information indicates that the current maturity meets the preset maturity, generate cooking prompt information to prompt the user to end cooking.
[0082] In this embodiment of the disclosure, the preset maturity level can indicate the appearance, color, and texture state that the user expects the ingredients to achieve after cooking. The preset maturity level can be obtained through user settings. Specifically, setting the preset maturity level can be achieved by displaying multiple interactive components representing different maturity levels on the display component of the cooking device, with the user determining the preset maturity level by triggering the interactive components. Setting the preset maturity level can also be achieved through multi-round voice interaction between the cooking device and the user to determine the user's desired cooking maturity level.
[0083] Cooking prompts can be displayed through the display components of the cooking equipment or played through the playback components of the cooking equipment.
[0084] Step S602: If the maturity information indicates that the current maturity does not meet the preset maturity, control the cooking equipment to continue the cooking operation.
[0085] In this embodiment of the disclosure, if the maturity information indicates that the current maturity does not meet the preset maturity, the cooking operation continues according to the set cooking control logic until the maturity information indicates that the current maturity meets the preset maturity.
[0086] As can be seen from the above embodiments, by comparing the maturity information with the preset maturity, the current cooking status of the ingredients can be understood in a timely manner; by generating cooking prompt information when the maturity information indicates that the current maturity meets the preset maturity, the user can be prompted to stop cooking in time to prevent the ingredients from being overcooked.
[0087] Figure 7 This is a block diagram illustrating a cooking image processing apparatus according to an exemplary embodiment. (Refer to...) Figure 7 The device includes a first image acquisition module 701, an image fusion module 702, and a maturity recognition module 703, wherein... The first image acquisition module 701 is used to acquire images of ingredients in the cooking state and obtain the acquired images; The image fusion module 702 is used to fuse the acquired image with the first target interference image to obtain the cooking image to be identified. The first target interference image is an interference image that corresponds to the food category in the acquired image. The maturity recognition module 703 is used to perform maturity recognition processing on the cooking image to be recognized based on the template cooking image set to obtain the maturity information corresponding to the cooking image to be recognized. The template cooking image set includes multiple template cooking images. The template cooking image is an image obtained by fusing a standard cooking image with interference images corresponding to the ingredients in the standard cooking image. The standard cooking image is an image of different ingredients at different maturity levels. The pixel values of each pixel in the same interference image are the same, while the pixel values of the interference images corresponding to different ingredient categories are different. The color saturation values corresponding to different interference images are all greater than the preset color saturation threshold.
[0088] In an optional embodiment, the apparatus further includes: The sampling time period candidate module is used to determine the candidate sampling time period. The candidate sampling time period is the time period during the cooking process of the target ingredient when it changes from the first preset maturity to the second preset maturity. The target ingredient can be any ingredient among different ingredients. The sampling period determination module is used to determine a preset number of time periods from the candidate sampling periods as the target sampling period. The preset number of time periods correspond one-to-one with the maturity level. The time interval between the end time of the previous time period and the start time of the next time period in two adjacent time periods meets the preset duration. The duration of each of the preset number of time periods decreases as the cooking progresses. The sampling module is used to sample the cooking images of the target ingredients during the target sampling period to obtain a preset number of standard cooking images corresponding to the target ingredients; The first image set determination module is used to determine the standard cooking images corresponding to each ingredient as a standard cooking image set.
[0089] In an optional embodiment, the apparatus further includes: The region division module is used to determine the first image region and the second image region in the target cooking image. The target cooking image is any standard cooking image, the first image region is the ingredient region, and the second image region is the other regions besides the first image region. The Poisson fusion module is used to perform Poisson fusion processing on the image regions associated with the first image region and the second target interference image in the first image region to obtain the first fused image. The second target interference image is the interference image corresponding to the target cooking image. The weighted fusion module is used to perform weighted fusion processing on the second image region and the image regions in the second target interference image that are associated with the second image region to obtain the second fused image; The stitching module is used to obtain a template cooking image corresponding to the target cooking image based on the first fused image and the second fused image; The second image set determination module is used to determine the template cooking images corresponding to each standard cooking image as a template cooking image set.
[0090] In an optional embodiment, the maturity identification module 703 includes: The first maturity recognition unit is used to input the cooking image to be recognized into the target maturity classification model for maturity classification to obtain target maturity information. The target maturity classification model is a model trained on a preset classification model using a set of template cooking images. The target maturity information is used to indicate the classification result of the cooking image to be recognized in a preset number of maturity levels.
[0091] The maturity determination unit is used to determine the target maturity information as maturity information.
[0092] In an optional embodiment, the apparatus further includes: The second maturity classification module is used to input each template cooking image into a preset classification model for maturity classification, and obtain sample maturity information. The sample maturity information indicates the classification result of each template cooking image in a preset number of maturity levels. The loss information determination module is used to determine loss information based on sample maturity information and maturity labels; The training module is used to adjust the parameters of a preset classification model based on loss information until a preset parameter tuning stopping condition is reached. The preset classification model that reaches the preset parameter tuning stopping condition is then used as the trained target maturity recognition model.
[0093] In an optional embodiment, the maturity identification module 703 includes: The first feature extraction unit is used to extract features from each template cooking image to obtain the template image features corresponding to each template cooking image. The second feature extraction unit is used to extract features from the cooking image to be identified, and obtain the cooking image features corresponding to the cooking image to be identified. The similarity comparison unit is used to compare the similarity between the features of the cooking image and the features of the template image to obtain the comparison results; The second maturity recognition unit is used to determine the maturity level associated with the template cooking image corresponding to the target image feature as maturity information when the comparison result indicates that the target image feature exists in the template image feature. The similarity between the target image feature and the cooking image feature meets the preset similarity condition.
[0094] In an optional embodiment, the apparatus further includes: The prompt module is used to generate cooking prompts when the maturity information indicates that the current maturity meets the preset maturity. The cooking prompts are used to prompt the user to end cooking. The cooking module is used to control the cooking equipment to continue cooking when the maturity information indicates that the current maturity does not meet the preset maturity.
[0095] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware such as processing circuitry or memory, or combinations thereof. Similarly, one or more processors or memories can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.
[0096] In an exemplary embodiment, an electronic device is also provided, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is used for the instructions to implement the cooking image processing method as described in the embodiments of this disclosure.
[0097] Figure 8 This is a block diagram illustrating an electronic device for a cooking image processing method according to an exemplary embodiment. The electronic device may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown, the electronic device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a cooking image processing method. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.
[0098] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present disclosure and does not constitute a limitation on the electronic device to which the present disclosure is applied. A specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0099] In an exemplary embodiment, a storage medium is also provided, which, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the cooking image processing method of the present disclosure embodiments.
[0100] In an exemplary embodiment, a computer program product is also provided, comprising a computer program stored in a readable storage medium, wherein at least one processor of a computer device reads from the readable storage medium and executes the computer program, causing the device to perform the cooking image processing method of the present disclosure embodiments.
[0101] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this disclosure can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.
[0102] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.
[0103] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.
Claims
1. A cooking image processing method, characterized by, The method includes: Images of ingredients in a cooking state are captured to obtain captured images; By fusing the acquired image with the first target interference image, a cooking image to be identified is obtained, wherein the first target interference image is an interference image corresponding to the food category in the acquired image; Based on a template cooking image set, the cooking image to be identified is processed for maturity recognition to obtain maturity information corresponding to the cooking image to be identified. The template cooking image set includes multiple template cooking images. The template cooking image is an image obtained by fusing a standard cooking image with interference images corresponding to the ingredients in the standard cooking image. The standard cooking image is an image of different ingredients at different maturity levels. The pixel values of each pixel in the same interference image are the same, while the pixel values of the interference images corresponding to different ingredient categories are different. The color saturation values corresponding to different interference images are all greater than a preset color saturation threshold.
2. The cooking image processing method of claim 1, wherein, The steps to obtain the standard cooking image set include the following: Determine candidate sampling time periods, which are the time periods during the cooking process of the target ingredient when it changes from a first preset maturity to a second preset maturity, and the target ingredient is any one of the different ingredients; A preset number of time periods in the candidate sampling period are all determined as target sampling periods. The preset number of time periods correspond one-to-one with the maturity level. The time interval between the end time of the previous time period and the start time of the next time period in two adjacent time periods meets the preset duration. The duration of each of the preset number of time periods decreases as the cooking progresses. The cooking images of the target ingredients during the target sampling period are sampled to obtain a preset number of standard cooking images corresponding to the target ingredients; The standard cooking images corresponding to each ingredient are determined as the standard cooking image set. 3.The cooking image processing method of claim 1, wherein, The steps to obtain the template cooking image set include: the following: Determine a first image region and a second image region in a target cooking image, wherein the target cooking image is any standard cooking image, the first image region is the food ingredient region, and the second image region is any region other than the first image region; A Poisson fusion process is performed on the image regions associated with the first image region and the second target interference image to obtain a first fused image. The second target interference image is the interference image corresponding to the target cooking image. A weighted fusion process is performed on the second image region and the image region in the second target interference image that is associated with the second image region to obtain a second fused image; Based on the first fused image and the second fused image, a template cooking image corresponding to the target cooking image is obtained; The template cooking images corresponding to each standard cooking image are determined as the template cooking image set.
4. The cooking image processing method according to claim 1, characterized in that, The process of performing maturity recognition processing on the cooking image to be identified based on the template cooking image set to obtain maturity information corresponding to the cooking image to be identified includes: The cooking image to be identified is input into a target maturity classification model for maturity classification to obtain target maturity information. The target maturity classification model is a model trained on a preset classification model using the template cooking image set. The target maturity information is used to indicate the classification result of the cooking image to be identified in a preset number of maturity levels. The target maturity information is determined as the maturity information.
5. The cooking image processing method according to claim 4, characterized in that, Each template cooking image in the template cooking image set carries a maturity label, which indicates the maturity level associated with the acquisition time period of each template cooking image. The training process of the target maturity classification model includes... the following: Each of the template cooking images is input into the preset classification model for maturity classification to obtain sample maturity information, which indicates the classification result of each of the template cooking images in the preset number of maturity levels; Based on the sample maturity information and the maturity label, the loss information is determined; Based on the loss information, the parameters of the preset classification model are adjusted until a preset parameter tuning stopping condition is reached. The preset classification model corresponding to the point where the preset parameter tuning stopping condition is reached is then used as the trained target maturity recognition model.
6. The cooking image processing method according to claim 1, characterized in that, The process of performing maturity recognition processing on the cooking image to be identified based on the template cooking image set to obtain maturity information corresponding to the cooking image to be identified includes: Feature extraction is performed on each of the template cooking images to obtain the template image features corresponding to each template cooking image. Feature extraction is performed on the cooking image to be identified to obtain the cooking image features corresponding to the cooking image to be identified; The similarity between the cooking image features and the template image features is compared to obtain the comparison results; If the comparison result indicates that the target image feature exists in the template image feature, the maturity level associated with the template cooking image corresponding to the target image feature is determined as the maturity information, and the similarity between the target image feature and the cooking image feature meets the preset similarity condition.
7. The cooking image processing method according to claim 1, characterized in that, The method further includes: When the maturity information indicates that the current maturity meets the preset maturity, a cooking prompt message is generated, which is used to prompt the user to end cooking; If the maturity information indicates that the current maturity does not meet the preset maturity, the cooking equipment is controlled to continue the cooking operation.
8. A cooking image processing apparatus, characterized in that, The device includes: The first image acquisition module is used to acquire images of the food in the cooking state and obtain the acquired images; An image fusion module is used to fuse the acquired image with a first target interference image to obtain a cooking image to be identified, wherein the first target interference image is an interference image corresponding to the food category in the acquired image; The maturity recognition module is used to perform maturity recognition processing on the cooking image to be recognized based on a template cooking image set to obtain the maturity information corresponding to the cooking image to be recognized. The template cooking image set includes multiple template cooking images, which are images obtained by fusing a standard cooking image with interference images corresponding to the ingredients in the standard cooking image. The standard cooking image is an image of different ingredients at different maturity levels. The pixel values of each pixel in the same interference image are the same, while the pixel values of the interference images corresponding to different ingredient categories are different. The color saturation values corresponding to different interference images are all greater than a preset color saturation threshold.
9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is used for the instructions to implement the cooking image processing method as described in any one of claims 1 to 7.
10. A storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the cooking image processing method as described in any one of claims 1 to 7.