A cooking control method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to a cooking control method and device, electronic equipment and storage medium. The method comprises: obtaining a target cooking image and target cooking result information under a target cooking mode; inputting the target cooking image and the target cooking result information into a target cooking control model to predict a cooking duration, and obtaining first target heating duration information; and controlling the cooking equipment based on the first target heating duration information. The present disclosure accurately adjusts the heating duration of each heating stage under the target cooking mode, accurately controls the cooking equipment, and improves the cooking efficiency and cooking effect.

Description

Technical Field

[0001] This invention relates to the field of cooking control technology, and in particular to a cooking control method, device, electronic device, and storage medium. Background Technology

[0002] In the cooking process, the training inputs for control models typically encompass the physical or chemical changes of ingredients at different stages, as well as parameters related to the cooking equipment. Using this data, the model's training objectives usually focus on two key aspects: first, accurately identifying the state of the ingredients, such as determining whether they have reached the ideal level of doneness, color, or texture; and second, rationally setting the total cooking time to ensure the cooking effect meets expectations. However, current models have certain limitations. Most can only optimize the total cooking time, lacking a model that can adapt to cooking modes involving multiple consecutive heating stages and precisely control the duration of each heating stage. This lack of a multi-stage duration control model restricts the development of automated and intelligent cooking, especially in complex cooking processes where precise time management of each stage is required to achieve optimal flavor and texture. Therefore, developing a control model that can cover the duration of each of multiple heating stages is of great significance for improving cooking efficiency and quality. Summary of the Invention

[0003] To address at least one of the aforementioned technical problems, this disclosure provides a cooking control method, apparatus, electronic device, and storage medium.

[0004] On one hand, the present invention provides a cooking control method, the method comprising: Acquire the target cooking image and target cooking result information under the target cooking mode. The target cooking mode represents the cooking mode of cooking food through multiple consecutive target heating stages. The target cooking image is the image of the food corresponding to the first heating stage under the target cooking mode. The target cooking result information describes the surface properties of the food after cooking under the target cooking mode as expected by the user. The target cooking image and target cooking result information are input into the target cooking control model to predict the cooking time, and the first target heating time information is obtained. The first target heating time information includes the first heating time corresponding to each of the multiple target heating stages. The target cooking control model is a model trained on a preset cooking control model using a sample dataset. The sample dataset includes sample control temperature information, sample cooking images and sample cooking result information in the sample cooking mode. The cooking equipment is controlled based on the heating time information of the first target.

[0005] In an optional embodiment, if the target cooking mode is interrupted, the method further includes: Get cooking instructions; When the cooking instruction instructs the cooking device to perform a reheating operation in the target cooking mode, multiple food images corresponding to the target time period are acquired. The target time period is the time period after the start of the reheating operation, which is associated with the start time of the reheating operation. The expected cooking result corresponding to the reheating operation is the expected result indicated by the target cooking result information. Multiple food images are integrated into a target image set according to the order in which they were acquired; The target image set and target cooking result information are input into the target cooking control model to predict the cooking time, and the second target heating time information is obtained. The second target heating time information includes the second heating time corresponding to each of the multiple target heating stages. The cooking equipment is controlled based on the second target heating time information.

[0006] In an optional embodiment, the training steps of the target cooking control model include the following: Obtain a sample dataset, which carries heating time labels. The heating time labels describe the heating time corresponding to each of the multiple heating stages required for the food to reach the state indicated by the sample cooking result information. Input the sample dataset into the preset cooking control model to predict the cooking time, and obtain the heating time information corresponding to each of the predicted heating stages. Based on heating duration information and heating duration labels, a preset cooking control model is trained to obtain a trained target cooking control model.

[0007] In an optional embodiment, the preset cooking control model includes an image feature extraction sub-model, a weight configuration sub-model, and a duration prediction sub-model. The sample dataset is input into the preset cooking control model to predict the cooking duration, obtaining the predicted heating duration information corresponding to each of the multiple heating stages, including: Based on the image feature extraction sub-model, the sample cooking images are segmented and feature extracted to obtain multiple sample cooking sub-images and their corresponding sample image features. Based on the sample cooking result information and the weight configuration sub-model, the weights of multiple sample cooking sub-images are configured to obtain the sample weights corresponding to each of the multiple sample cooking sub-images. By fusing sample image features and sample weights, the descriptive features of the sample image are obtained; The sample cooking result information, sample control temperature information, and sample image description features are input into the duration prediction sub-model to predict the duration and obtain the heating duration information.

[0008] In an optional embodiment, a preset cooking control model is trained based on heating time information and heating time labels to obtain a trained target cooking control model, including: Loss information is determined based on heating duration information and heating duration label; Based on the loss information, the parameters of the image feature extraction sub-model, weight configuration sub-model, and duration prediction sub-model are adjusted until the preset parameter tuning stopping condition is met, thus obtaining the trained image feature extraction sub-model, weight configuration sub-model, and duration prediction sub-model. Based on the trained image feature extraction sub-model, weight configuration sub-model, and duration prediction sub-model, the trained target cooking control model is determined.

[0009] In an optional embodiment, the process of obtaining sample cooking result information includes the following: Obtain images of ingredients after cooking using the sample cooking mode; Descriptive text generation is performed on the food images to obtain sample cooking result information.

[0010] Secondly, the present invention also provides a cooking control device, comprising: The target data acquisition module is used to acquire the target cooking image and target cooking result information under the target cooking mode. The target cooking mode represents a cooking mode that cooks food through multiple consecutive target heating stages. The target cooking image is the image of the food corresponding to the first heating stage under the target cooking mode. The target cooking result information describes the surface properties of the food after cooking under the target cooking mode as expected by the user. The first target duration prediction module is used to input the target cooking image and target cooking result information into the target cooking control model to predict the cooking duration and obtain the first target heating duration information. The first target heating duration information includes the first heating duration corresponding to each of the multiple target heating stages. The target cooking control model is a model obtained by training a preset cooking control model using a sample dataset. The sample dataset includes sample control temperature information, sample cooking images and sample cooking result information under the sample cooking mode. The first control module is used to control the cooking equipment based on the first target heating time information.

[0011] Thirdly, the present invention also provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is used to execute instructions to implement the cooking control method described above.

[0012] Fourthly, the present invention also provides a storage medium that, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to perform the above-described cooking control method.

[0013] Fifthly, the present invention also provides a computer program product comprising a computer program stored in a readable storage medium, wherein at least one processor of a computer device reads from the readable storage medium and executes the computer program, causing the device to perform the above-described cooking control method.

[0014] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0015] Implementing this disclosure will have the following beneficial effects: The process involves acquiring target cooking images and target cooking result information under a target cooking mode. The target cooking mode represents a cooking mode that cooks ingredients through multiple consecutive target heating stages. The target cooking image is the image of the ingredients corresponding to the first heating stage under the target cooking mode. The target cooking result information describes the surface properties of the ingredients after cooking under the target cooking mode, as desired by the user. The target cooking image and target cooking result information are then input into a target cooking control model to predict the cooking time, resulting in first target heating time information. This first target heating time information includes the first heating time corresponding to each of the multiple target heating stages. The target cooking control model is a model trained using a sample dataset on a preset cooking control model. The sample dataset includes sample control temperature information, sample cooking images, and sample cooking result information under sample cooking modes. Based on the first target heating time information, the cooking equipment is controlled.

[0016] This disclosure obtains first target heating time information by inputting the target cooking image and target cooking result information into the target cooking control model to predict the cooking time. This allows for obtaining the duration information corresponding to each heating stage in the target cooking mode that meets the user's expectations. By controlling the cooking equipment based on the first target heating time information, the heating time of each heating stage in the target cooking mode can be precisely adjusted, thereby achieving precise control of the cooking equipment and improving cooking efficiency and cooking effect.

[0017] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The accompanying drawings are incorporated in and constitute a part of this specification, illustrating embodiments consistent with this disclosure, and are used together with the description to explain the principles of this disclosure, and do not constitute an improper limitation of this disclosure.

[0019] Figure 1 This is a schematic diagram of an implementation environment according to an exemplary embodiment; Figure 2 This is a flowchart illustrating a cooking control method according to an exemplary embodiment; Figure 3 This is a schematic diagram illustrating a target cooking control model for predicting cooking time according to an exemplary embodiment; Figure 4 This is a schematic diagram illustrating the control temperature curve, food temperature curve, and food maturity curve corresponding to continuous cooking mode and reheat cooking mode under a target cooking mode according to an exemplary embodiment. Figure 5 This is a schematic diagram illustrating the food maturity curves corresponding to continuous cooking and reheating cooking methods under a target cooking mode, according to an exemplary embodiment. Figure 6 This is a flowchart illustrating a method for obtaining a target cooking control model by training a preset cooking control model, according to an exemplary embodiment. Figure 7 This is a schematic diagram illustrating a cooking control device according to an exemplary embodiment; Figure 8 This is a block diagram illustrating an electronic device for controlling cooking, according to an exemplary embodiment. Detailed Implementation

[0020] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0022] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. Like reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise. The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0023] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0024] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0025] Please see Figure 1 , Figure 1 This is a schematic diagram illustrating an application environment according to an exemplary embodiment, such as... Figure 1 As shown, the application environment may include server 01 and cooking equipment 02.

[0026] In an optional embodiment, the cooking device 02 can be used to process the cooking control method. For example, the cooking device 02 acquires a target cooking image and target cooking result information under a target cooking mode, and then inputs the target cooking image and target cooking result information into a target cooking control model to predict the cooking time, thereby obtaining first target heating time information; subsequently, the cooking device 02 controls the cooking device based on the first target heating time information.

[0027] It should be noted that the above-described application environment is merely one provided by this disclosure. In practical applications, the cooking device 02 can also be combined with the server 01 to process the cooking control method. For example, the cooking device 02 acquires the target cooking image and target cooking result information under the target cooking mode and sends them to the server 01. The server 01 then inputs the target cooking image and target cooking result information into the target cooking control model to predict the cooking time, obtains the first target heating time information, and sends it to the cooking device 02. Based on the first target heating time information, the cooking device 02 performs control operations on the cooking device.

[0028] For example, server 01 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0029] In the embodiments described in this specification, the server 01 and the cooking device 02 can be directly or indirectly connected via wired or wireless communication, and this disclosure does not impose any limitations.

[0030] Figure 2 This is a flowchart illustrating a cooking control method according to an exemplary embodiment, such as... Figure 2 As shown, the cooking control methods include the following: Step S201: Obtain the target cooking image and target cooking result information under the target cooking mode. The target cooking mode represents a cooking mode that cooks ingredients through multiple consecutive target heating stages. The target cooking image is the image of the ingredients corresponding to the first heating stage under the target cooking mode. The target cooking result information describes the surface properties of the ingredients after cooking under the target cooking mode as desired by the user.

[0031] In this embodiment of the present disclosure, the number of consecutive target heating stages in the target cooking mode is the same as the number of consecutive heating stages in the sample cooking mode, and the control temperature of each target heating stage in the consecutive target heating stages in the target cooking mode is the same as the control temperature of each heating stage in the consecutive heating stages in the sample cooking mode.

[0032] The process of acquiring the target cooking image includes the following: After starting the target cooking mode, in the first heating stage, i.e. the first heating stage, the camera configured in the cooking device is used to capture an image inside the cooking cavity to obtain the target cooking image, which includes an image of the ingredients in the cooking state.

[0033] The process of obtaining the target cooking result information includes the following: a preset interactive interface can be used to display cooking selection menus for different ingredients to the user. The cooking selection menus are equipped with selection controls for different cooking attributes. Users can trigger the selection controls to match different attribute cooking effects, obtain customized user expectations, and determine the target cooking result information.

[0034] The process of obtaining target cooking result information may also include the following: by configuring a generative large model in the cooking interaction interface, determining the user's cooking expectations through voice interaction with the user, and obtaining target cooking result information.

[0035] The target cooking result information describes the surface properties of different parts of the food after cooking using the sample cooking mode, which may include the food's color and texture properties.

[0036] Step S202: Input the target cooking image and target cooking result information into the target cooking control model to predict the cooking time and obtain the first target heating time information. The first target heating time information includes the first heating time corresponding to each of the multiple target heating stages. The target cooking control model is a model trained on a preset cooking control model using a sample dataset. The sample dataset includes sample control temperature information, sample cooking images and sample cooking result information under the sample cooking mode.

[0037] In this embodiment of the present disclosure, during the model training phase, the model learns the control temperature features of multiple target heating stages under the target cooking mode based on the input sample control temperature information. Therefore, during the use phase of the target cooking control model, it is not necessary to input the control temperatures of multiple heating stages. Instead, the target cooking image and target cooking result information can be directly input into the target cooking control model to predict the cooking time, thereby obtaining the first target heating time information that conforms to the target cooking result information.

[0038] Figure 3This is a schematic diagram illustrating a target cooking control model for predicting cooking time according to an exemplary embodiment. The target cooking image is a three-channel image, and the target cooking result information is denoted as... After inputting the target cooking image and target cooking result information into the target cooking control model, the image feature extraction sub-model in the target cooking control model performs segmentation and feature extraction on the target cooking image to obtain the target image features corresponding to each of the n target cooking sub-images. Then, the weight allocation sub-model in the target cooking control model allocates weights to the multiple target cooking sub-images to obtain the target weights corresponding to each of the n target cooking sub-images, denoted as […]. , , , , , … The target image features and target weights are fused to obtain the target image description features. The target cooking result information and the target image description features are then input into the duration prediction sub-model to predict the duration, thus obtaining the first target heating duration information. .

[0039] To further improve the accuracy of the model output, the control temperatures of multiple target heating stages under the target cooking mode can also be input into the target cooking control model. Therefore, the step of inputting the target cooking image and target cooking result information into the target cooking control model to predict the cooking time and obtain the first target heating time information can be adjusted to inputting the control temperatures of multiple target heating stages under the target cooking mode, the target cooking image, and the target cooking result information into the target cooking control model to predict the cooking time and obtain the first target heating time information.

[0040] Step S203: Control the cooking equipment based on the first target heating time information.

[0041] In this embodiment of the disclosure, the control operation of the cooking device based on the first target heating time information can be performed by setting the heating time of each of the multiple target heating stages according to the first target heating time information. After the heating time of a certain heating stage reaches its corresponding heating time, the stage ends and the next heating stage is started at the same time, until the target cooking mode is completed.

[0042] Based on the above, in this embodiment of the present disclosure, by inputting the target cooking image and target cooking result information into the target cooking control model to predict the cooking time, the first target heating time information is obtained. This allows for obtaining the duration information corresponding to each heating stage in the target cooking mode that meets the user's expectations. By controlling the cooking equipment based on the first target heating time information, the heating time of each heating stage in the target cooking mode can be precisely adjusted, thereby achieving precise control of the cooking equipment and improving cooking efficiency and cooking effect.

[0043] Figure 4 This is a schematic diagram illustrating the control temperature curve, food temperature curve, and food maturity curve corresponding to continuous cooking and reheating cooking methods under a target cooking mode, according to an exemplary embodiment. When cooking food using the target cooking mode, if multiple consecutive target heating stages are not interrupted, the correspondence between the control temperature curve, food temperature curve, and food maturity curve corresponding to the multiple consecutive target heating stages is as follows: Figure 4 The continuous cooking curve in black is shown in the image. After reheating is initiated, the corresponding relationships between the control temperature curves, food temperature curves, and food doneness curves for multiple target heating stages are as follows: Figure 4 The medium gray reheat cooking curve is shown.

[0044] Taking a multi-target heating stage that includes at least two heating stages, i.e., at least a first target heating stage and a second target heating stage, as an example, the control temperature corresponding to the first target heating stage is: The corresponding heating time is The control temperature corresponding to the second target heating stage is The corresponding heating time is In this mode, the food temperature reaches [a certain level] during the first target heating stage. Temperature, reached during the second target heating stage Temperature. Correspondingly, the doneness of ingredients changes from initial doneness... Gradually became At the end of the second target heating phase However, if the target cooking mode is interrupted, for example, if it is interrupted just as the first target heating stage begins, the food temperature will drop from... Gradually down to Since the doneness of ingredients depends not only on the external temperature control but also on the temperature of the ingredients themselves, if the temperature of the ingredients is within a certain range... Start reheating at the designated time, and continue as before. , Heating continues for the remaining time within the controlled duration. Because the food temperature is relatively low at this point, the food's maturity deviates from the original maturity curve, causing the maturity at the end of the second target heating stage to become... This can lead to errors in the cooking results and affect the precision of color control for the ingredients.

[0045] Therefore, in the event that the target cooking mode is interrupted, in order to maintain the cooked result of the reheated ingredients at the desired level of doneness... The above methods also include: Step S301: Obtain cooking instructions.

[0046] In this embodiment of the disclosure, the method of obtaining cooking instructions is not limited. For example, cooking instructions can be cooking instructions input by the user through touch operation, or cooking instructions given by voice.

[0047] Step S302: When the cooking instruction instructs the cooking device to perform a reheating operation in the target cooking mode, acquire multiple food images corresponding to the target time period. The target time period is the time period after the start of the reheating operation, which is associated with the start time of the reheating operation. The expected cooking result corresponding to the reheating operation is the expected result indicated by the target cooking result information.

[0048] In this embodiment of the disclosure, the reheating operation refers to the operation in the target cooking mode that uses the target cooking result information corresponding to the interrupted cooking operation as the cooking target. Therefore, the expected cooking result corresponding to the reheating operation is the expected result indicated by the target cooking result information, and the reheating operation does not require the user to input the target cooking result information again.

[0049] The target time period can be a 10-second interval after the start of the reheating operation. If the target cooking mode is interrupted during the first target heating stage, the target time period can be a 10-second interval within the remaining heating time of the first target heating stage. Acquiring multiple food images corresponding to the target time period can be achieved by capturing multiple food images within the cooking cavity of the cooking device during the target time period, with different capture times corresponding to different food images.

[0050] Step S303: Integrate multiple food images into a target image set according to the order of acquisition time.

[0051] In this embodiment of the disclosure, the food images acquired first are arranged in order of priority, and the food images acquired later are arranged in order of priority, and combined into a target image set.

[0052] Step S304: Input the target image set and target cooking result information into the target cooking control model to predict the cooking time and obtain the second target heating time information, which includes the second heating time corresponding to each of the multiple target heating stages.

[0053] In this embodiment, the target image set and target cooking result information are input into the target cooking control model to predict the cooking time. The second target heating time information can be obtained by the image feature extraction sub-model in the target cooking control model, which segments the images in the target image set to obtain multiple sub-images after each image segmentation. Features are extracted from each of the multiple sub-images after segmentation to obtain the target image features corresponding to each of the multiple sub-images after segmentation. Since the shooting position and angle are the same in each image, the target image features corresponding to the multiple sub-images at the same position in the cooking cavity are then combined according to the order of image acquisition time to obtain the combined features corresponding to that position. The combined features include the surface change features of the food at that position under reheating operation. The combined features corresponding to each position are determined as the target image set features. Then, the weight configuration sub-model in the target cooking control model configures the weights of the combined features corresponding to each position to obtain the target weights corresponding to each position. The target image set features and target weights are fused to obtain the target image set description features. The target cooking result information and the target image set description features are input into the duration prediction sub-model to predict the duration, thus obtaining the second target heating duration information. .

[0054] Step S305: Control the cooking equipment based on the second target heating time information.

[0055] In this embodiment of the disclosure, the control operation of the cooking device based on the second target heating time information can be performed by setting the heating time of each of the multiple target heating stages according to the second target heating time information. After the heating time of a certain heating stage reaches its corresponding heating time, the stage ends and the next heating stage is started at the same time, until the target cooking mode is completed.

[0056] Figure 5 This is a schematic diagram illustrating the food maturity curves corresponding to continuous cooking and reheating cooking methods under a target cooking mode, according to an exemplary embodiment. When cooking food using the target cooking mode, if multiple consecutive target heating stages are not interrupted, the correspondence between the food maturity curves corresponding to the multiple consecutive target heating stages is as follows: Figure 5 The continuous cooking curve in black is shown in the image. After initiating the reheating operation, the corresponding relationships between the food maturity curves for multiple target heating stages are as follows: Figure 5 The medium-gray reheat cooking curve is shown. If the cooking control method for situations where the target cooking mode is interrupted is not adopted, and the cooking operation is performed using the original first target heating time information, then the final result will be... The degree of coloring of the food after the control time ends, i.e., the degree of doneness of the food. The degree of ripeness indicated by the target cooking result information The difference is significant.

[0057] After implementing reheating control using a cooking control method for situations where the target cooking mode is interrupted, the model incorporates the variation characteristics of different internal temperatures of the ingredients extracted from the target image set. Therefore, during the inference of the target cooking control model, the heating duration under the target cooking mode can be corrected to... Then in the end After the controlled cooking time ends, the ingredients reach the required level of doneness. It is closer to the level of maturity indicated by the target cooking result information. The deviation in the precision control of food coloring has been corrected.

[0058] Based on the above, in this embodiment of the present disclosure, multiple food images corresponding to the target time period of the reheating operation are integrated into a target image set according to the order of acquisition time. The target image set and the target cooking result information are then input into the target cooking control model to predict the cooking time. This allows the model to learn the changes in the outer surface of the food under the reheating operation through multiple food images. The resulting second target heating time information is more consistent with the actual changes in the food under the reheating operation. By controlling the cooking equipment based on the second target heating time information, the cooking result of the reheating operation can meet the user's expected target cooking result.

[0059] Figure 6 This is a flowchart illustrating a method for obtaining a target cooking control model by training a preset cooking control model, according to an exemplary embodiment. Figure 6 As shown, the training steps for the target cooking control model include the following: Step S601: Obtain the sample dataset, which carries heating time labels. The heating time labels describe the heating time corresponding to each of the multiple heating stages required for the food to reach the state indicated by the sample cooking result information.

[0060] In this embodiment of the disclosure, the sample dataset includes sample control temperature information, sample cooking images, and sample cooking result information under a sample cooking mode. The sample cooking mode represents a cooking mode that cooks food through multiple consecutive heating stages. The control temperature corresponding to each heating stage in the sample cooking mode is different. The sample control temperature information represents the control temperature corresponding to each of the multiple heating stages. For example, in the case where the sample cooking mode involves cooking through two consecutive heating stages, namely a first heating stage and a second heating stage, the control temperature of the first heating stage is... The controlled temperature for the second heating stage is , Temperature and The temperature values ​​differ. In the sample cooking mode, which involves cooking through three consecutive heating stages—the first, second, and third heating stages—the controlled temperature for the first heating stage is... The controlled temperature for the second heating stage is The controlled temperature for the third heating stage is , temperature, Temperature and The temperature values ​​vary. Cooking control in the sample cooking mode can be achieved by setting the heating time for each heating stage. Different heating stages have different heating times, and different heating times for each stage result in different cooking effects.

[0061] In an optional embodiment, the sample cooking image is an image of the food in the first heating stage of multiple heating stages. The process of acquiring the sample cooking image includes the following: after starting the sample cooking mode, in the first heating stage, i.e. the first heating stage, the camera configured in the cooking device is used to collect an image inside the cooking cavity to obtain a sample cooking image, which includes an image of the food in the cooking state.

[0062] Alternatively, the cooking process of the ingredients under the sample cooking mode can be recorded as a video to obtain a sample cooking video. Any image frame in the video segment corresponding to the first heating stage can be identified as the sample cooking image. The sample cooking image is a three-channel image.

[0063] In an optional embodiment, the sample cooking result information is used to describe the surface properties of the food after cooking using the sample cooking mode. The process of obtaining the sample cooking result information includes the following: Step S6011: Obtain an image of the ingredients after cooking using the sample cooking mode.

[0064] In this embodiment of the disclosure, the image of the food after cooking in the sample cooking mode can be obtained by filming the process of cooking the food in the sample cooking mode, that is, by recording the process of cooking the food in the sample cooking mode to obtain a sample cooking video, and the last image frame in the video is determined as the image of the food after cooking in the sample cooking mode.

[0065] Step S6012: Perform descriptive text generation processing on the food ingredient images to obtain sample cooking result information.

[0066] In this embodiment of the disclosure, the descriptive text generation process for the food ingredient image can be performed by inputting the food ingredient image into a graph-to-text model for image-to-text conversion, thereby obtaining the sample cooking result information output by the graph-to-text model. Alternatively, the user's descriptive text of the food ingredient image can be obtained and identified as the sample cooking result information.

[0067] The sample cooking result information describes the surface properties of different parts of the food after cooking using the sample cooking mode, which may include the food's color and texture properties.

[0068] In an optional embodiment, the heating time label is the heating time corresponding to each of the consecutive heating stages in the cooking process of cooking the ingredients from the initial state to the intermediate state corresponding to the ingredients in the sample cooking image, and finally to the terminal state corresponding to the ingredients in the ingredient image, using the sample cooking mode.

[0069] Based on the above, in this embodiment of the present disclosure, by acquiring the food images after cooking in the sample cooking mode and processing the food images to generate descriptive text, sample cooking result information can be obtained, which can provide accurate cooking result description information, enabling the preset cooking control model to accurately learn the surface properties of the food in the cooking result, thereby achieving more accurate time prediction.

[0070] Step S602: Input the sample dataset into the preset cooking control model to predict the cooking time, and obtain the heating time information corresponding to each of the predicted heating stages.

[0071] In this embodiment of the disclosure, the preset cooking control model can be a preset cooking control model constructed using a single model, or it can be a preset cooking control model constructed by mixing multiple models.

[0072] In an optional embodiment, the preset cooking control model includes an image feature extraction sub-model, a weight configuration sub-model, and a duration prediction sub-model. The sample dataset is input into the preset cooking control model to predict the cooking duration, obtaining the predicted heating duration information corresponding to each of the multiple heating stages, including: Step S6021: Based on the image feature extraction sub-model, the sample cooking images are segmented and feature extracted to obtain multiple sample cooking sub-images and their corresponding sample image features.

[0073] In this embodiment of the disclosure, the segmentation and feature extraction of the sample cooking image can be performed firstly by dividing the sample cooking image into multiple sample cooking sub-images, each smaller than the original sample cooking image in both length and width. Then, feature extraction is performed on each sample cooking sub-image to obtain the sample image features corresponding to each of the multiple sample cooking sub-images. Accordingly, the image feature extraction sub-model includes an image segmentation layer and a feature extraction layer to respectively implement the above steps.

[0074] Step S6022: Based on the sample cooking result information and the weight configuration sub-model, perform weight configuration on multiple sample cooking sub-images to obtain the sample weights corresponding to each of the multiple sample cooking sub-images.

[0075] In this embodiment of the disclosure, the weight configuration sub-model adopts an embedding layer. Based on the sample cooking result information and the weight configuration sub-model, the weight configuration of multiple sample cooking sub-images can be achieved by using the embedding layer to nonlinearly map the sample cooking result information into attention weights for different sample cooking sub-images, thereby obtaining the sample weights corresponding to each of the multiple sample cooking sub-images.

[0076] Step S6023: Fuse sample image features and sample weights to obtain sample image descriptive features.

[0077] In this embodiment of the disclosure, fusing sample image features and sample weights to obtain sample image description features can be achieved by multiplying the sample image features corresponding to the same sample cooking sub-image with its corresponding sample weight to obtain the sample sub-image description features corresponding to that sample cooking sub-image. Multiple sample cooking sub-images... Step S6024: Input the sample cooking result information, sample control temperature information and sample image description features into the duration prediction sub-model to predict the duration and obtain the heating duration information.

[0078] In this embodiment of the disclosure, the duration prediction sub-model is constructed using a hierarchical visual transformer architecture (Swin-Transformer). The sample cooking result information, sample control temperature information, and sample image description features are input into the duration prediction sub-model, which then predicts the duration of each heating stage to obtain the heating duration information corresponding to each of the multiple heating stages.

[0079] Based on the above, in this embodiment of the present disclosure, by constructing an image feature extraction sub-model, a weight configuration sub-model, and a duration prediction sub-model, feature extraction, weight configuration, and duration prediction can be performed through the above three models respectively, and finally the duration prediction function can be realized to obtain heating duration information.

[0080] Step S603: Based on the heating time information and heating time label, train the preset cooking control model to obtain the trained target cooking control model.

[0081] In an optional embodiment, a preset cooking control model is trained based on heating time information and heating time labels to obtain a trained target cooking control model, including: Step S6031: Determine the loss information based on the heating duration information and the heating duration label.

[0082] In this embodiment of the disclosure, determining the loss information based on the heating time information and the heating time label can be based on the difference between the heating time information and the heating time label. To stabilize the heating duration information input to the model and ensure that the model output conforms to objective reality, loss information is determined based on the heating duration information and heating duration labels. The ratio between the heating duration of each consecutive heating stage indicated by the heating duration label and the heating duration information predicted by the model for each of the multiple heating stages can be used as reference data for calculating the loss information. Based on this reference data and a reasonable range of preset reference data, the loss information is determined. This method of calculating loss information ensures that the loss value is not directly dependent on the magnitude of the heating duration information predicted by the model for each of the multiple heating stages, making the loss information more accurate. Step S6032: Based on the loss information, adjust the parameters of the image feature extraction sub-model, weight configuration sub-model, and duration prediction sub-model until the preset parameter tuning stop condition is met, and obtain the trained image feature extraction sub-model, weight configuration sub-model, and duration prediction sub-model.

[0083] In this embodiment of the disclosure, the preset parameter tuning stopping condition may be that the number of parameter tuning attempts reaches a preset number, or the loss information is less than a preset loss value.

[0084] With the preset parameter tuning stopping condition being that the number of parameter tuning attempts reaches a preset number, the parameters of the image feature extraction sub-model, weight configuration sub-model, and duration prediction sub-model are adjusted once after each model output, and the number of parameter tuning attempts is recorded once, until the number of parameter tuning attempts reaches the preset number, and the trained image feature extraction sub-model, weight configuration sub-model, and duration prediction sub-model are obtained.

[0085] With the preset parameter tuning stopping condition being that the loss information is less than the preset loss value, after each loss calculation, the loss information is compared to the preset loss value. When the loss information is not less than the preset loss value, the parameters of the image feature extraction sub-model, weight configuration sub-model, and duration prediction sub-model are adjusted until the loss information is less than the preset loss value, thus obtaining the trained image feature extraction sub-model, weight configuration sub-model, and duration prediction sub-model.

[0086] Step S6033: Based on the trained image feature extraction sub-model, weight configuration sub-model, and duration prediction sub-model, determine the trained target cooking control model.

[0087] In this embodiment of the disclosure, determining the trained target cooking control model based on the trained image feature extraction sub-model, weight configuration sub-model, and duration prediction sub-model can be achieved by integrating the trained image feature extraction sub-model, weight configuration sub-model, and duration prediction sub-model into a complete model to obtain the trained target cooking control model.

[0088] Based on the above, in this embodiment of the present disclosure, by determining the loss information according to the heating duration information and the heating duration label, the loss value corresponding to the duration prediction function of the model can be determined. Based on the loss information, the parameters of the image feature extraction sub-model, the weight configuration sub-model, and the duration prediction sub-model are adjusted, which can make the outputs of the image feature extraction sub-model, the weight configuration sub-model, and the duration prediction sub-model more accurate.

[0089] The above training method utilizes the food image during the first heating stage of the sample cooking mode, the surface properties of the food after cooking in the sample cooking mode, the control temperature and heating time labels corresponding to each of the multiple heating stages in the sample cooking mode, to train a preset cooking control model and obtain a target cooking control model that can predict the duration of each heating stage in the sample cooking mode. This allows for the control of the duration of each heating stage in the sample cooking mode through the trained target cooking control model.

[0090] Figure 7 This is a block diagram illustrating a cooking control device according to an exemplary embodiment. (Refer to...) Figure 7 The device includes a target data acquisition module 701, a first target duration prediction module 702, and a first control module 703, wherein... The target data acquisition module 701 is used to acquire the target cooking image and target cooking result information under the target cooking mode. The target cooking mode represents a cooking mode that cooks food through multiple consecutive target heating stages. The target cooking image is the image of the food corresponding to the first heating stage under the target cooking mode. The target cooking result information describes the surface properties of the food after cooking under the target cooking mode as expected by the user. The first target duration prediction module 702 is used to input the target cooking image and target cooking result information into the target cooking control model to predict the cooking duration and obtain the first target heating duration information. The first target heating duration information includes the first heating duration corresponding to each of the multiple target heating stages. The target cooking control model is a model obtained by training a preset cooking control model using a sample dataset. The sample dataset includes sample control temperature information, sample cooking images and sample cooking result information in the sample cooking mode. The first control module 703 is used to control the cooking equipment based on the first target heating time information.

[0091] In an optional embodiment, the device further includes: [The following is an optional feature:] If the target cooking mode is interrupted, the device also includes: The cooking instruction acquisition module is used to acquire cooking instructions; The reheating image acquisition module is used to acquire multiple food images corresponding to a target time period when the cooking instruction instructs the cooking device to perform a reheating operation in a target cooking mode. The target time period is the time period after the start of the reheating operation, which is associated with the start time of the reheating operation. The expected cooking result corresponding to the reheating operation is the expected result indicated by the target cooking result information. The image set integration module is used to integrate multiple food images into a target image set according to the order in which they were acquired. The reheating time prediction module is used to input the target image set and target cooking result information into the target cooking control model to predict the cooking time and obtain the second target heating time information, which includes the second heating time corresponding to each of the multiple target heating stages. The second control module is used to control the cooking equipment based on the second target heating time information.

[0092] In an optional embodiment, the apparatus further includes: The sample data acquisition module is used to acquire sample datasets. The sample datasets carry heating time labels, which describe the heating time corresponding to each of the multiple heating stages required for the food to reach the state indicated by the sample cooking result information. The sample duration prediction module is used to input the sample dataset into the preset cooking control model to predict the cooking duration and obtain the heating duration information corresponding to each of the predicted heating stages. The training module is used to train a preset cooking control model based on heating time information and heating time labels to obtain a trained target cooking control model.

[0093] In an optional embodiment, the preset cooking control model includes an image feature extraction sub-model, a weight configuration sub-model, and a duration prediction sub-model. The sample duration prediction module includes: The image processing unit is used to perform segmentation and feature extraction on the sample cooking images based on the image feature extraction sub-model, so as to obtain multiple sample cooking sub-images and the sample image features corresponding to each of the multiple sample cooking sub-images. The weight configuration unit is used to configure the weights of multiple sample cooking sub-images based on the sample cooking result information and the weight configuration sub-model, so as to obtain the sample weights corresponding to each of the multiple sample cooking sub-images. The fusion unit is used to fuse sample image features and sample weights to obtain sample image descriptive features; The duration prediction unit is used to input the sample cooking result information, sample control temperature information, and sample image description features into the duration prediction sub-model to predict the duration and obtain heating duration information.

[0094] In an optional embodiment, the training module includes: The loss calculation unit is used to determine loss information based on heating duration information and heating duration label; The parameter tuning unit is used to adjust the parameters of the image feature extraction sub-model, weight configuration sub-model, and duration prediction sub-model based on the loss information until the preset parameter tuning stop condition is met, so as to obtain the trained image feature extraction sub-model, weight configuration sub-model, and duration prediction sub-model. The model determination unit is used to determine the trained target cooking control model based on the trained image feature extraction sub-model, weight configuration sub-model, and duration prediction sub-model.

[0095] In an optional embodiment, the sample data acquisition module includes: The cooking result acquisition unit is used to acquire images of ingredients after cooking using the sample cooking mode. The text generation unit is used to generate descriptive text from food images to obtain sample cooking result information.

[0096] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware such as processing circuitry or memory, or combinations thereof. Similarly, one or more processors or memories can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0097] In an exemplary embodiment, an electronic device is also provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is used for the instructions to implement the cooking control method as described in the embodiments of this disclosure.

[0098] Figure 8 This is a block diagram illustrating an electronic device for a cooking control method according to an exemplary embodiment. The electronic device may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown, the electronic device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a cooking control method. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.

[0099] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present disclosure and does not constitute a limitation on the electronic device to which the present disclosure is applied. A specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0100] In an exemplary embodiment, a storage medium is also provided, which, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to perform the cooking control method of the present disclosure embodiments.

[0101] In an exemplary embodiment, a computer program product is also provided, comprising a computer program stored in a readable storage medium, wherein at least one processor of a computer device reads from the readable storage medium and executes the computer program, causing the device to perform the cooking control method of the present disclosure embodiments.

[0102] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this disclosure can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0103] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0104] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A cooking control method, characterized in that, The method includes: Acquire the target cooking image and target cooking result information under the target cooking mode. The target cooking mode represents a cooking mode that cooks food through multiple consecutive target heating stages. The target cooking image is the image of the food corresponding to the first heating stage under the target cooking mode. The target cooking result information describes the surface properties of the food after cooking under the target cooking mode as desired by the user. The target cooking image and the target cooking result information are input into the target cooking control model to predict the cooking time, thereby obtaining the first target heating time information. The first target heating time information includes the first heating time corresponding to each of the multiple target heating stages. The target cooking control model is a model obtained by training a preset cooking control model using a sample dataset. The sample dataset includes sample control temperature information, sample cooking images, and sample cooking result information under the sample cooking mode. Based on the heating time information of the first target, the cooking equipment is controlled and operated.

2. The cooking control method according to claim 1, characterized in that, In the event that the target cooking mode is interrupted, the method further includes: Obtain the cooking instructions; When the cooking instruction instructs the cooking device to perform a reheating operation in the target cooking mode, multiple food images corresponding to the target time period are acquired. The target time period is the time period after the start of the reheating operation, which is associated with the start time of the reheating operation. The expected cooking result corresponding to the reheating operation is the expected result indicated by the target cooking result information. The multiple food images are integrated into a target image set according to the order in which they were acquired; The target image set and the target cooking result information are input into the target cooking control model to predict the cooking time, thereby obtaining the second target heating time information, which includes the second heating time corresponding to each of the multiple target heating stages. Based on the second target heating time information, the cooking equipment is controlled.

3. The cooking control method according to claim 1, characterized in that, The training steps for the target cooking control model include the following: Obtain the sample dataset, which carries heating time labels, and the heating time labels describe the heating time corresponding to each of the multiple heating stages required for the food to reach the state indicated by the sample cooking result information; The sample dataset is input into a preset cooking control model to predict the cooking time, and the predicted heating time information corresponding to each of the multiple heating stages is obtained. Based on the heating time information and the heating time label, the preset cooking control model is trained to obtain the trained target cooking control model.

4. The cooking control method according to claim 3, characterized in that, The preset cooking control model includes an image feature extraction sub-model, a weight configuration sub-model, and a duration prediction sub-model. The step of inputting the sample dataset into the preset cooking control model to predict the cooking duration, and obtaining the predicted heating duration information corresponding to each of the multiple heating stages, includes: Based on the image feature extraction sub-model, the sample cooking image is segmented and feature extracted to obtain multiple sample cooking sub-images and the sample image features corresponding to each of the multiple sample cooking sub-images. Based on the sample cooking result information and the weight configuration sub-model, weights are configured for the multiple sample cooking sub-images to obtain the sample weights corresponding to each of the multiple sample cooking sub-images; By fusing the sample image features and the sample weights, the sample image description features are obtained; The sample cooking result information, the sample control temperature information, and the sample image description features are input into the duration prediction sub-model to predict the duration and obtain the heating duration information.

5. The cooking control method according to claim 4, characterized in that, The step of training the preset cooking control model based on the heating time information and the heating time label to obtain the trained target cooking control model includes: Loss information is determined based on the heating duration information and the heating duration tag; Based on the loss information, the parameters of the image feature extraction sub-model, the weight configuration sub-model, and the duration prediction sub-model are adjusted until a preset parameter tuning stop condition is met, thereby obtaining the trained image feature extraction sub-model, the weight configuration sub-model, and the duration prediction sub-model. Based on the trained image feature extraction sub-model, the weight configuration sub-model, and the duration prediction sub-model, the trained target cooking control model is determined.

6. The cooking control method according to claim 1, characterized in that, The process of obtaining the sample cooking result information includes the following: Obtain images of the ingredients after cooking using the sample cooking mode; The food ingredient image is processed to generate descriptive text, thereby obtaining the sample cooking result information.

7. A cooking control device, characterized in that, The device includes: The target data acquisition module is used to acquire the target cooking image and target cooking result information under the target cooking mode. The target cooking mode represents a cooking mode that cooks food through multiple consecutive target heating stages. The target cooking image is the image of the food corresponding to the first heating stage under the target cooking mode. The target cooking result information describes the surface properties of the food after cooking under the target cooking mode as desired by the user. The first target duration prediction module is used to input the target cooking image and the target cooking result information into the target cooking control model to predict the cooking duration and obtain the first target heating duration information. The first target heating duration information includes the first heating duration corresponding to each of the multiple target heating stages. The target cooking control model is a model obtained by training a preset cooking control model using a sample dataset. The sample dataset includes sample control temperature information, sample cooking images, and sample cooking result information under the sample cooking mode. The first control module is used to control the cooking equipment based on the first target heating time information.

8. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is used for the instructions to implement the cooking control method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the cooking control method as described in any one of claims 1 to 6.

10. A computer program product comprising a computer program stored in a readable storage medium, wherein at least one processor of a computer device reads from and executes the computer program, causing the device to perform the cooking control method as described in any one of claims 1 to 6.

Images