Food maturity prediction model training method and food maturity early warning method

Abstract

The invention relates to a food maturity prediction model training method and a food maturity early warning method.The food maturity prediction model training method comprises the steps that a sample image set, sample time information and a sample reference set are obtained, and the sample image set comprises at least two sample images; the sample time information comprises time information corresponding to at least two sample images; the sample image set and the sample time information are input into a preset food maturity prediction model, a sample prediction set is obtained, and the sample prediction set comprises sample prediction images, sample prediction time information and sample prediction maturity information; and based on the sample prediction set and the sample reference set, training a preset food maturity prediction model to obtain a trained food maturity prediction model. According to the invention, the output result of the trained food maturity prediction model can be more accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of smart home appliances, and in particular to a food maturity prediction model training method and a food maturity early warning method. Background Art

[0002] With the continuous development of smart home appliance technology, more and more food cooking relies on the automatic control of kitchen appliances, such as steam ovens, microwave ovens, etc. In related technologies, food is generally cooked by controlling the heating time and power level. Due to the large variety of ingredients, different ingredients have different cooking firepower requirements and cooking times. If only the heating time and power level are set, it is easy for the ingredients to be immature or burnt. Therefore, it is necessary to understand the maturity of the food in a timely manner during the cooking process to improve the accuracy of food cooking. Summary of the invention

[0003] In order to solve at least one of the technical problems mentioned above, the present disclosure proposes a food maturity prediction model training method and a food maturity early warning method.

[0004] In one aspect, the present disclosure provides a method for training a food maturity prediction model, the method comprising:

[0005] Acquire a sample image set, sample time information, and a sample reference set, wherein the sample image set includes at least two sample images, the at least two sample images are images of food at different times during a food cooking process, the sample time information includes time information corresponding to each of the at least two sample images, and the sample reference set includes an image of the food at a first target time, time information corresponding to the first target time, and maturity information of the food at the first target time, wherein the time information corresponding to the first target time is greater than the time information corresponding to any sample image;

[0006] Inputting the sample image set and the sample time information into a preset food maturity prediction model to obtain a sample prediction set, the sample prediction set including the sample prediction image, the sample prediction time information and the sample prediction maturity information, the sample prediction image represents the image of the food at the predicted first target moment, the sample prediction time information represents the time information corresponding to the predicted first target moment, and the sample prediction maturity information represents the maturity information of the food at the predicted first target moment;

[0007] Based on the sample prediction set and the sample reference set, a preset food maturity prediction model is trained to obtain a trained food maturity prediction model.

[0008] In an optional embodiment, the preset food maturity prediction model includes a preset image prediction model and a preset image prediction model, and the sample image set and the sample time information are input into the preset food maturity prediction model to obtain a sample prediction set, including:

[0009] Based on the sample time information, determining a time weight corresponding to each of at least two sample images;

[0010] Determine the time weights corresponding to the at least two sample images as sample time weight information;

[0011] Inputting the sample image set and the sample time weight information into a preset image prediction model to obtain a sample prediction image;

[0012] Inputting the sample image set and the sample time information into a preset maturity prediction model to obtain sample prediction time information and sample prediction maturity information;

[0013] Based on the sample prediction image, the sample prediction time information and the sample prediction maturity information, a sample prediction set is determined.

[0014] In an optional embodiment, based on the sample prediction set and the sample reference set, a preset food maturity prediction model is trained to obtain a trained food maturity prediction model, including:

[0015] Determine loss information based on the sample prediction set and the sample reference set;

[0016] Based on the loss information, a preset image prediction model and a preset maturity prediction model are trained to obtain a trained image prediction model and a trained maturity prediction model;

[0017] Based on the trained image prediction model and the trained maturity prediction model, a trained food maturity prediction model is determined.

[0018] In a second aspect, the present invention further provides a food maturity early warning method, comprising:

[0019] Acquire a target image set and target time information, wherein the target image set includes at least two target images, the at least two target images are images of food at different times during a food cooking process, and the target time information includes time information corresponding to each of the at least two target images;

[0020] Input the target image set and the target time information into the trained food maturity prediction model obtained by training according to the above-mentioned food maturity prediction model training method, to obtain a target prediction set, the target prediction set including a target prediction image, a target prediction time information and a target prediction maturity information, the target prediction image represents the image of the food at the predicted second target moment, the target prediction time information represents the time information corresponding to the predicted second target moment, the target prediction maturity information represents the maturity information of the food at the predicted second target moment, and the time information corresponding to the second target moment is greater than the time information corresponding to any target image;

[0021] Based on the target prediction set, food maturity warning operations are performed.

[0022] In an optional embodiment, based on the target prediction set, a food maturity warning operation is performed, including:

[0023] When the target prediction set indicates that the food is ripe, a warning message is generated, and the warning message is used to indicate the ripeness information of the food.

[0024] In an optional embodiment, performing a food maturity warning operation based on the target prediction set further includes:

[0025] When the target prediction set indicates that the food is burnt, reduce the cooking fire power corresponding to the food.

[0026] In a third aspect, the present invention further provides a food maturity prediction model training device, comprising:

[0027] a first acquisition module, configured to acquire a sample image set, sample time information, and a sample reference set, wherein the sample image set includes at least two sample images, wherein the at least two sample images are images of food at different times during a food cooking process, the sample time information includes time information corresponding to each of the at least two sample images, and the sample reference set includes an image of the food at a first target time, time information corresponding to the first target time, and maturity information of the food at the first target time, wherein the time information corresponding to the first target time is greater than the time information corresponding to any sample image;

[0028] A sample prediction module is used to input a sample image set and sample time information into a preset food maturity prediction model to obtain a sample prediction set, wherein the sample prediction set includes a sample prediction image, sample prediction time information, and sample prediction maturity information, wherein the sample prediction image represents an image of the food at the predicted first target moment, the sample prediction time information represents time information corresponding to the predicted first target moment, and the sample prediction maturity information represents maturity information of the food at the predicted first target moment;

[0029] The training module is used to train a preset food maturity prediction model based on a sample prediction set and a sample reference set to obtain a trained food maturity prediction model.

[0030] In a fourth aspect, the present invention further provides a food maturity early warning device, comprising:

[0031] A second acquisition module is used to acquire a target image set and target time information, wherein the target image set includes at least two target images, and the at least two target images are images of food at different times during the food cooking process, and the target time information includes time information corresponding to each of the at least two target images;

[0032] A target prediction module, used for inputting a target image set and target time information into a trained food maturity prediction model obtained by training according to any food maturity prediction model training method according to claims 1-3, to obtain a target prediction set, the target prediction set including a target prediction image, target prediction time information and target prediction maturity information, the target prediction image represents an image of the food at a predicted second target moment, the target prediction time information represents time information corresponding to the predicted second target moment, the target prediction maturity information represents maturity information of the food at the predicted second target moment, and the time information corresponding to the second target moment is greater than the time information corresponding to any target image;

[0033] The early warning module is used to perform food maturity early warning operations based on the target prediction set.

[0034] In a fifth aspect, the present invention further provides an electronic device, comprising:

[0035] processor;

[0036] a memory for storing processor-executable instructions;

[0037] Among them, the processor is used to execute instructions to implement the above-mentioned food maturity prediction model training method or the above-mentioned food maturity warning method.

[0038] In a sixth aspect, the present invention further provides a storage medium, which, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute the above-mentioned food maturity prediction model training method or the above-mentioned food maturity warning method.

[0039] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

[0040] The implementation of this disclosure has the following beneficial effects:

[0041] A sample image set, sample time information and a sample reference set are obtained, wherein the sample image set includes at least two sample images, wherein the at least two sample images are images of food at different moments in a food cooking process, the sample time information includes time information corresponding to each of the at least two sample images, and the sample reference set includes an image of food at a first target moment, time information corresponding to the first target moment and maturity information of the food at the first target moment, wherein the time information corresponding to the first target moment is greater than the time information corresponding to any sample image; the sample image set and the sample time information are input into a preset food maturity prediction model to obtain a sample prediction set, wherein the sample prediction set includes a sample prediction image, a sample prediction time information and a sample prediction maturity information, wherein the sample prediction image represents the image of food at the predicted first target moment, the sample prediction time information represents the time information corresponding to the predicted first target moment, and the sample prediction maturity information represents the maturity information of food at the predicted first target moment; based on the sample prediction set and the sample reference set, a preset food maturity prediction model is trained to obtain a trained food maturity prediction model.

[0042] The present invention obtains a sample prediction set by inputting a sample image set and sample time information into a preset food maturity prediction model. The sample prediction set can predict the image of food at a future moment in the cooking process and the corresponding maturity information through the image of food at different moments in the cooking process and the corresponding time information. The preset food maturity prediction model is trained based on the sample prediction set and the sample reference set. The preset food maturity prediction model can be trained through the predicted results and the actual results to obtain a trained food maturity prediction model, so that the model output result is more accurate.

[0043] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the prior art description. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0045] Figure 1 is a schematic diagram of an implementation environment according to an exemplary embodiment;

[0046] Figure 2is a flow chart of a method for training a food maturity prediction model according to an exemplary embodiment;

[0047] Figure 3 is a schematic diagram of a food maturity prediction model according to an exemplary embodiment;

[0048] Figure 4 is a schematic diagram of a recognition process of an image recognition model according to an exemplary embodiment;

[0049] Figure 5 is a schematic diagram of a food maturity warning process according to an exemplary embodiment;

[0050] Figure 6 is a schematic diagram of a food maturity prediction model training device according to an exemplary embodiment;

[0051] Figure 7 is a schematic diagram of a food maturity early warning device according to an exemplary embodiment;

[0052] Figure 8 It is a block diagram of an electronic device for training a food maturity prediction model according to an exemplary embodiment. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0055] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise indicated. The word "exemplary" is used specifically herein to mean "serving as an example, embodiment, or illustrative". Any embodiment described herein as "exemplary" is not necessarily to be construed as being superior or better than other embodiments.

[0056] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0057] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0058] See also Figure 1 , Figure 1 is a schematic diagram of an application environment according to an exemplary embodiment. Figure 1 As shown, the application environment may include a server 01 and a terminal 02 .

[0059] In an optional embodiment, the server 01 can be used for calculation processing in the food maturity prediction model training method. Specifically, the server 01 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0060] In an optional embodiment, the terminal 02 can perform calculation processing in combination with the food maturity prediction model training method of the server 01. Specifically, the terminal 02 can include but is not limited to electronic devices such as smart phones, desktop computers, tablet computers, laptop computers, smart speakers, digital assistants, augmented reality (AR) / virtual reality (VR) devices, smart wearable devices, etc. Optionally, the operating system running on the electronic device can include but is not limited to Android system, IOS system, Linux system, Windows system, Unix system, etc.

[0061] For example, a sample image set, sample time information and a sample reference set are obtained through the terminal 02, and the sample image set, the sample time information and the sample reference set are transmitted to the server 01, the sample image set includes at least two sample images, and the at least two sample images are images of food at different times during the food cooking process. The sample time information includes the time information corresponding to each of the at least two sample images. The sample reference set includes the image of the food at a first target time, the time information corresponding to the first target time and the maturity information of the food at the first target time, and the time information corresponding to the first target time is greater than the time information corresponding to any sample image; the server 01 inputs the sample image set and the sample time information into a preset food maturity prediction model to obtain a sample prediction set, the sample prediction set includes the sample prediction image, the sample prediction time information and the sample prediction maturity information, the sample prediction image represents the image of the food at the predicted first target time, the sample prediction time information represents the time information corresponding to the predicted first target time, and the sample prediction maturity information represents the maturity information of the food at the predicted first target time; based on the sample prediction set and the sample reference set, the preset food maturity prediction model is trained to obtain the trained food maturity prediction model, and the training result is transmitted to the terminal 02.

[0062] In addition, it should be noted that Figure 1 What is shown is only one application environment provided by the present disclosure. In actual application, other application environments may also be included.

[0063] In the embodiments of this specification, the server 01 and the terminal 02 may be directly or indirectly connected via wired or wireless communication, which is not limited in this disclosure.

[0064] Figure 2 is a flow chart of a method for training a food maturity prediction model according to an exemplary embodiment. Figure 2 As shown, the above method includes the following:

[0065] Step S201: Obtain a sample image set, sample time information and a sample reference set, wherein the sample image set includes at least two sample images, and the at least two sample images are images of food at different times during the food cooking process. The sample time information includes time information corresponding to each of the at least two sample images. The sample reference set includes an image of the food at a first target time, time information corresponding to the first target time and maturity information of the food at the first target time, and the time information corresponding to the first target time is greater than the time information corresponding to any sample image.

[0066] In the embodiment of the present disclosure, the method of obtaining the sample image set, the sample time information and the sample reference set may be: collecting video data of a duration of T for the steaming and baking process of a food, and extracting the following information from each video frame in the video data: Z t =(t,x t ,m t ),t∈T. Among them, t is the number of seconds relative to time 0, that is, the video frame time information at time t, x t is the image of the video frame at time t, m t is the maturity of the food in the image of the video frame at time t. t The composition set is recorded as The collection Remove the maximum time t max The video frame corresponds to The resulting set is recorded as t max Greater than a set The time information corresponding to any moment in the above set With Collection The relationship between them can be expressed by the following formula (1):

[0067]

[0068] Similarly, the image set corresponding to any N interval video frames in the video is recorded as The collection Remove the maximum time t max The set of images corresponding to the video frames is recorded as The Collection is the sample image set. The images in are all sample images; the time information corresponding to any interval video frames N in the video is composed of a set and recorded as The collection Remove the maximum time t max The set obtained after the time information corresponding to the video frame is recorded as The Collection This is the sample time information.

[0069] In the embodiment of the present disclosure, the first target time may be t max The corresponding moment, is the image of food at the first target moment, t max is the time information corresponding to the first target moment, The maturity information of the food at the first target moment, This is the sample reference set.

[0070] Step S202: Input the sample image set and the sample time information into a preset food maturity prediction model to obtain a sample prediction set, which includes a sample prediction image, sample prediction time information and sample prediction maturity information. The sample prediction image represents the image of the food at the predicted first target moment, the sample prediction time information represents the time information corresponding to the predicted first target moment, and the sample prediction maturity information represents the maturity information of the food at the predicted first target moment.

[0071] In the embodiment of the present disclosure, the preset food maturity prediction model includes a preset image prediction model and a preset image prediction model. The sample image set and the sample time information are input into the preset food maturity prediction model to obtain a sample prediction set, including:

[0072] Step S2021: Based on the sample time information, determine the time weights corresponding to at least two sample images.

[0073] In the embodiment of the present disclosure, according to the mapping function according to the set The time sequence of each time information in is mapped into the corresponding time weight, and the corresponding time weight is determined as the time weight of the corresponding sample image according to the corresponding relationship between each time information and each sample image. The closer the time information in is to t max , the corresponding time weight is larger.

[0074] Step S2022: Determine the time weights corresponding to at least two sample images as sample time weight information.

[0075] In the embodiment of the present disclosure, the time weights corresponding to at least two sample images are combined into a set λ N-1 , set λ N-1 That is the sample time weight information.

[0076] Step S2023: input the sample image set and the sample time weight information into a preset image prediction model to obtain a sample prediction image.

[0077] In the embodiment of the present disclosure, the preset image prediction model may be a generator G in a generative adversarial network (GAN). Figure 3 As shown, the sample image set and sample time weight information λ N-1 Input generator G, and the generator G outputs the sample prediction image

[0078] Step S2024: input the sample image set and sample time information into a preset maturity prediction model to obtain sample prediction time information and sample prediction maturity information.

[0079] In the disclosed embodiment, the preset maturity prediction model is composed of two convolutional neural networks (CNN), a long short-term recurrent neural network (LSTM) and a fully connected neural network (FCNN). Figure 3 As shown, the sample image set and sample time information After inputting the preset maturity prediction model, the convolutional neural network CNN-X performs Perform image feature extraction to obtain a sample image set Corresponding spatial feature information, which represents the sample image set The image spatial features of each sample image in the convolutional neural network CNN-T are used to transform the sample time information Extract time features to obtain sample time information Corresponding time feature information, the spatial feature information and the time feature information are spliced ​​to obtain the spatiotemporal information, and the sample prediction time information t is calculated through the long short-term recursive neural network LSTM and the fully connected neural network FCNN ′ max and sample prediction maturity information

[0080]

[0081] Step S2025: Determine the sample prediction set based on the sample prediction image, sample prediction time information and sample prediction maturity information.

[0082] In the embodiment of the present disclosure, the sample prediction image Sample prediction time information t ′ max and sample prediction maturity information Composed of a collection Determine the sample prediction set

[0083] Based on the above, it can be seen that in the embodiment of the present disclosure, by inputting the sample image set and sample time weight information into the preset image prediction model to obtain the sample prediction image, the generated sample prediction image can be closer to the actual image; by inputting the sample image set and sample time information into the preset maturity prediction model, the sample prediction time information and sample prediction maturity information are obtained, which can predict the specific time and maturity of the future moment.

[0084] Step S203: Based on the sample prediction set and the sample reference set, a preset food maturity prediction model is trained to obtain a trained food maturity prediction model.

[0085] In the embodiment of the present disclosure, a preset food maturity prediction model is trained based on a sample prediction set and a sample reference set to obtain a trained food maturity prediction model, including:

[0086] Step S2031: Determine loss information based on the sample prediction set and the sample reference set.

[0087] In the embodiment of the present disclosure, according to the sample prediction set and the sample reference set, the loss information can be determined by the discriminator D in the GAN network. Figure 3 As shown, the discriminator D of the GAN network is t ′ max The reality of the moment Values ​​and predicted values For input, identify and Distinction, the discriminator D gives the sample reference set Higher score, sample prediction set Z t′ Lower scores are given to confirm loss information.

[0088] Step S2032: Based on the loss information, the preset image prediction model and the preset maturity prediction model are trained to obtain a trained image prediction model and a trained maturity prediction model.

[0089] In the disclosed embodiment, the preset image prediction model can be trained by alternately training the preset image prediction model (generator G) and the discriminator D according to the following formula (2) until a Nash equilibrium is reached to obtain a trained preset image prediction model.

[0090]

[0091] When training the discriminator D, the loss information is the loss function maxV(G,D), which makes the discrimination ability stronger and stronger. When training the generator G, the loss information is the loss function minV(G,D), the parameters of the discriminator D no longer change, and the preset image prediction model will be able to predict a more realistic sample prediction set.

[0092] In the disclosed embodiment, the preset maturity prediction model may be trained by feeding back the loss information to the preset maturity prediction model, and fine-tuning the parameters of the preset maturity prediction model until the loss information meets the conditions.

[0093] Step S2033: Determine a trained food maturity prediction model based on the trained image prediction model and the trained maturity prediction model.

[0094] In the disclosed embodiment, the trained image prediction model and the trained maturity prediction model are determined as the trained food maturity prediction model.

[0095] Figure 4 is a flow chart of a food maturity early warning method according to an exemplary embodiment. Figure 4 As shown, the above method includes the following:

[0096] Step S401: obtaining a target image set and target time information, wherein the target image set includes at least two target images, and the at least two target images are images of food at different times during the food cooking process, and the target time information includes time information corresponding to each of the at least two target images.

[0097] In the embodiment of the present disclosure, the method for obtaining the target image set and the target time information may be: during the steaming and baking process of food, video data corresponding to the steaming and baking process is collected, and frame extraction processing is performed on the video data, and the image set corresponding to any interval video frames of number Q in the video is recorded as The Collection is the target image set. The images in are all target images; the time information corresponding to each of the above target images is composed of a set and recorded as The Collection That is the target time information.

[0098] Step S402: Input the target image set and the target time information into the trained food maturity prediction model obtained by training according to the above-mentioned food maturity prediction model training method to obtain a target prediction set, the target prediction set including a target prediction image, a target prediction time information and a target prediction maturity information, the target prediction image represents the image of the food at the predicted second target moment, the target prediction time information represents the time information corresponding to the predicted second target moment, the target prediction maturity information represents the maturity information of the food at the predicted second target moment, and the time information corresponding to the second target moment is greater than the time information corresponding to any target image.

[0099] In the embodiment of the present disclosure, the target image set and target time information Input the trained food maturity prediction model obtained by the above-mentioned food maturity prediction model training method to obtain the target prediction set Z q (t q ,x q ,m q ), t q is the time information corresponding to the predicted second target moment, x q is the predicted image of food at the second target moment, m q The predicted maturity information of the food at a second target moment is a future moment relative to the current moment in the food cooking process.

[0100] Step S403: Perform food maturity warning operation based on the target prediction set.

[0101] Based on the above, it can be known that in the embodiment of the present disclosure, the target image set and target time information are input into the trained food maturity prediction model trained according to the above-mentioned food maturity prediction model training method to obtain a target prediction set, and based on the target prediction set, a food maturity warning operation is performed, which can predict the future cooking results and promptly remind the user to pay attention to the cooking process.

[0102] In the embodiment of the present disclosure, based on the target prediction set, a food maturity warning operation is performed, including:

[0103] When the target prediction set indicates that the food is ripe, a warning message is generated, and the warning message is used to indicate the ripeness information of the food.

[0104] In the disclosed embodiment, if the target prediction set indicates that the food is ripe, it means that the food will be ripe at the second target moment during the cooking process. When the target prediction set indicates that the food is ripe, the generated warning information may include the time information corresponding to the predicted second target moment, the image of the food at the predicted second target moment, and the maturity information of the food at the predicted second target moment. Specifically, an interactive screen is provided outside the cooking utensil used for cooking the food, which displays the time information corresponding to the predicted second target moment, the image of the food at the predicted second target moment, and the maturity information of the food at the predicted second target moment, or voice broadcasts the time information corresponding to the predicted second target moment and the maturity information of the food at the predicted second target moment.

[0105] Based on the above, it can be seen that in the embodiments of the present disclosure, by generating warning information when the target prediction set indicates that the food is ripe, the user can be prompted with information about the maturity of the food, and the user can be reminded to pay attention to the cooking process in a timely manner.

[0106] In the embodiment of the present disclosure, the food maturity warning operation is performed based on the target prediction set, and further includes:

[0107] When the target prediction set indicates that the food is burnt, reduce the cooking fire power corresponding to the food.

[0108] In the disclosed embodiment, if the target prediction set indicates that the food is burnt, it means that the food will be burnt at the second target moment during the cooking process. If the target prediction set indicates that the food is burnt, the current cooking firepower can be directly reduced to avoid or reduce the burnt condition at the second target moment.

[0109] Based on the above, it can be seen that in the embodiments of the present disclosure, by reducing the cooking fire corresponding to the food when the target prediction set indicates that the food is burnt, it is possible to avoid the food from being burnt and improve the cooking efficiency.

[0110] In a specific embodiment, the implementation process of the above-mentioned food maturity early warning method is as follows Figure 5 As shown, firstly, the target image set and target time information are obtained, and then input into the trained food maturity prediction model to obtain the prediction result, i.e., the target prediction set. When the target prediction set indicates that the food is not mature, the target image set and target time information are continuously obtained for prediction. When the target prediction set indicates that the food is mature, the warning information is output to prompt the user of the maturity information of the food. When the target prediction set indicates that the food is burnt, the firepower is automatically adjusted.

[0111] Figure 6is a block diagram of a food maturity prediction model training device according to an exemplary embodiment. Figure 6 The device includes a first acquisition module 601, a sample prediction module 602 and a training module 603, wherein:

[0112] The first acquisition module 601 is used to acquire a sample image set, sample time information and a sample reference set, wherein the sample image set includes at least two sample images, and the at least two sample images are images of food at different times during the food cooking process. The sample time information includes time information corresponding to each of the at least two sample images. The sample reference set includes an image of the food at a first target time, time information corresponding to the first target time and maturity information of the food at the first target time, and the time information corresponding to the first target time is greater than the time information corresponding to any sample image.

[0113] The sample prediction module 602 is used to input the sample image set and the sample time information into a preset food maturity prediction model to obtain a sample prediction set, wherein the sample prediction set includes a sample prediction image, sample prediction time information, and sample prediction maturity information, wherein the sample prediction image represents the image of the food at the predicted first target moment, the sample prediction time information represents the time information corresponding to the predicted first target moment, and the sample prediction maturity information represents the maturity information of the food at the predicted first target moment;

[0114] The training module 603 is used to train a preset food maturity prediction model based on a sample prediction set and a sample reference set to obtain a trained food maturity prediction model.

[0115] In an optional embodiment, the preset food maturity prediction model includes a preset image prediction model and a preset image prediction model, and the sample prediction module 602 includes:

[0116] A time weight determination module, used to determine the time weights corresponding to at least two sample images based on the sample time information;

[0117] A sample time weight information determination module, used to determine the time weights corresponding to at least two sample images as sample time weight information;

[0118] A first sample prediction module, used for inputting a sample image set and sample time weight information into a preset image prediction model to obtain a sample prediction image;

[0119] The second sample prediction module is used to input the sample image set and the sample time information into a preset maturity prediction model to obtain the sample prediction time information and the sample prediction maturity information;

[0120] The third sample prediction module is used to determine a sample prediction set based on the sample prediction image, the sample prediction time information and the sample prediction maturity information.

[0121] In an optional embodiment, the training module 603 includes:

[0122] A loss information determination module, used to determine loss information based on a sample prediction set and a sample reference set;

[0123] A first training submodule is used to train a preset image prediction model and a preset maturity prediction model based on the loss information to obtain a trained image prediction model and a trained maturity prediction model;

[0124] The second training submodule is used to determine a trained food maturity prediction model based on the trained image prediction model and the trained maturity prediction model.

[0125] Figure 7 is a block diagram of a food maturity warning device according to an exemplary embodiment. Figure 7 The device includes a second acquisition module 701, a target prediction module 702 and an early warning module 703, wherein:

[0126] The second acquisition module 701 is used to acquire a target image set and target time information, wherein the target image set includes at least two target images, and the at least two target images are images of food at different times during the food cooking process, and the target time information includes time information corresponding to each of the at least two target images;

[0127] The target prediction module 702 is used to input the target image set and the target time information into the trained food maturity prediction model obtained by the above-mentioned food maturity prediction model training method to obtain a target prediction set, wherein the target prediction set includes a target prediction image, target prediction time information and target prediction maturity information, wherein the target prediction image represents the image of the food at the predicted second target moment, the target prediction time information represents the time information corresponding to the predicted second target moment, and the target prediction maturity information represents the maturity information of the food at the predicted second target moment, and the time information corresponding to the second target moment is greater than the time information corresponding to any target image;

[0128] The warning module 703 is used to perform food maturity warning operations based on the target prediction set.

[0129] In an optional embodiment, the early warning module 703 includes:

[0130] The first warning submodule is used to generate warning information when the target prediction set indicates that the food is ripe, and the warning information is used to indicate the maturity information of the food.

[0131] In an optional embodiment, the early warning module 703 further includes:

[0132] The second warning submodule is used to reduce the cooking fire power corresponding to the food when the target prediction set indicates that the food is burnt.

[0133] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware such as processing circuits or memories, or a combination thereof. Similarly, one processor or multiple processors or memories can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0134] In an exemplary embodiment, an electronic device is also provided, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is used for the instructions to implement the food maturity prediction model training method as in the embodiment of the present disclosure.

[0135] Figure 8 is a block diagram of an electronic device for food maturity prediction model training or food maturity warning according to an exemplary embodiment. The electronic device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 8 As shown. The electronic device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a food maturity prediction model training method or a food maturity early warning method is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0136] Those skilled in the art will understand that Figure 8 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present disclosure, and does not constitute a limitation on the electronic device to which the scheme of the present disclosure is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0137] In an exemplary embodiment, a storage medium is also provided. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the food maturity prediction model training method or food maturity warning method in the embodiment of the present disclosure.

[0138] In an exemplary embodiment, a computer program product containing instructions is also provided, which, when executed on a computer, enables the computer to execute the food maturity prediction model training method or the food maturity early warning method in the embodiments of the present disclosure.

[0139] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present disclosure may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0140] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0141] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A food maturity prediction model training method, characterized in that: The method comprises: Acquire a sample image set, sample time information, and a sample reference set, wherein the sample image set includes at least two sample images, the at least two sample images are images of food at different moments in a food cooking process, the sample time information includes time information corresponding to each of the at least two sample images, and the sample reference set includes an image of the food at a first target moment, time information corresponding to the first target moment, and maturity information of the food at the first target moment, wherein the time information corresponding to the first target moment is greater than the time information corresponding to any of the sample images; Inputting the sample image set and the sample time information into a preset food maturity prediction model to obtain a sample prediction set, wherein the sample prediction set includes a sample prediction image, sample prediction time information, and sample prediction maturity information, wherein the sample prediction image represents an image of the food at the predicted first target moment, the sample prediction time information represents time information corresponding to the predicted first target moment, and the sample prediction maturity information represents maturity information of the food at the predicted first target moment; Based on the sample prediction set and the sample reference set, the preset food maturity prediction model is trained to obtain a trained food maturity prediction model.

2. The method according to claim 1, characterized in that The preset food maturity prediction model includes a preset image prediction model and a preset image prediction model, and the sample image set and the sample time information are input into the preset food maturity prediction model to obtain a sample prediction set, including: Based on the sample time information, determining a time weight corresponding to each of the at least two sample images; Determining the time weights corresponding to the at least two sample images as sample time weight information; Inputting the sample image set and the sample time weight information into the preset image prediction model to obtain the sample prediction image; Inputting the sample image set and the sample time information into the preset maturity prediction model to obtain sample prediction time information and sample prediction maturity information; The sample prediction set is determined based on the sample prediction image, the sample prediction time information and the sample prediction maturity information.

3. The method according to claim 2, characterized in that The step of training the preset food maturity prediction model based on the sample prediction set and the sample reference set to obtain a trained food maturity prediction model includes: Determining loss information according to the sample prediction set and the sample reference set; Based on the loss information, the preset image prediction model and the preset maturity prediction model are trained to obtain a trained image prediction model and a trained maturity prediction model; Based on the trained image prediction model and the trained maturity prediction model, the trained food maturity prediction model is determined.

4. A food maturity early warning method, characterized in that: The method comprises: Acquire a target image set and target time information, wherein the target image set includes at least two target images, the at least two target images are images of food at different times during a food cooking process, and the target time information includes time information corresponding to each of the at least two target images; Input the target image set and the target time information into a trained food maturity prediction model obtained by training according to any of the food maturity prediction model training methods according to claims 1-3, to obtain a target prediction set, the target prediction set including a target prediction image, target prediction time information and target prediction maturity information, the target prediction image represents an image of the food at the predicted second target moment, the target prediction time information represents the time information corresponding to the predicted second target moment, the target prediction maturity information represents the maturity information of the food at the predicted second target moment, and the time information corresponding to the second target moment is greater than the time information corresponding to any of the target images; Based on the target prediction set, a food maturity warning operation is performed.

5. The method according to claim 4, characterized in that The performing of food maturity warning operation based on the target prediction set includes: When the target prediction set indicates that the food is ripe, early warning information is generated, where the early warning information is used to prompt the maturity information of the food.

6. The method according to claim 4, characterized in that The performing of food maturity warning operation based on the target prediction set further includes: When the target prediction set indicates that the food is burnt, the cooking fire power corresponding to the food is reduced.

7. A food maturity prediction model training device, characterized in that: The device comprises: a first acquisition module, configured to acquire a sample image set, sample time information, and a sample reference set, wherein the sample image set includes at least two sample images, the at least two sample images are images of food at different moments in a food cooking process, the sample time information includes time information corresponding to each of the at least two sample images, and the sample reference set includes an image of the food at a first target moment, time information corresponding to the first target moment, and maturity information of the food at the first target moment, wherein the time information corresponding to the first target moment is greater than the time information corresponding to any of the sample images; A sample prediction module, used for inputting the sample image set and the sample time information into a preset food maturity prediction model to obtain a sample prediction set, wherein the sample prediction set includes a sample prediction image, sample prediction time information and sample prediction maturity information, wherein the sample prediction image represents an image of the food at the predicted first target moment, the sample prediction time information represents time information corresponding to the predicted first target moment, and the sample prediction maturity information represents maturity information of the food at the predicted first target moment; A training module is used to train the preset food maturity prediction model based on the sample prediction set and the sample reference set to obtain a trained food maturity prediction model.

8. A food maturity warning device, characterized in that: The device comprises: A second acquisition module is used to acquire a target image set and target time information, wherein the target image set includes at least two target images, and the at least two target images are images of food at different times during the food cooking process, and the target time information includes time information corresponding to each of the at least two target images; A target prediction module, used for inputting a target image set and target time information into a trained food maturity prediction model obtained by training according to any food maturity prediction model training method according to claims 1-3, to obtain a target prediction set, wherein the target prediction set includes a target prediction image, target prediction time information and target prediction maturity information, wherein the target prediction image represents an image of the food at a predicted second target moment, the target prediction time information represents time information corresponding to the predicted second target moment, the target prediction maturity information represents maturity information of the food at the predicted second target moment, and the time information corresponding to the second target moment is greater than the time information corresponding to any of the target images; The early warning module is used to perform food maturity early warning operations based on the target prediction set.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; Wherein, the processor is used for the instructions to implement the food maturity prediction model training method as described in any one of claims 1-3 or the food maturity early warning method as described in any one of claims 4-6.

10. A storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the food maturity prediction model training method as described in any one of claims 1-3 or the food maturity warning method as described in any one of claims 4-6.

Images