A food doneness determination method and a cooking apparatus

Abstract

The embodiment of the application discloses a food doneness determination method and a cooking device, which are applied to a cooking device, the cooking device is in communication connection with a cloud server; the method comprises the following steps: collecting a to-be-cooked food on a food bearing part to obtain a food initial image, and sending the food initial image to the cloud server, so that the cloud server generates a doneness cooking image set corresponding to the to-be-cooked food according to the food initial image; receiving the doneness cooking image set, for the doneness cooking image corresponding to each doneness in the doneness cooking image set, determining a multi-feature matching degree group corresponding to the doneness according to the doneness cooking image and a current food image; determining an actual doneness of the to-be-cooked food at a current time according to the multi-feature matching degree group corresponding to each doneness, and determining and cooking according to a remaining cooking time according to the actual doneness and a target doneness of the to-be-cooked food. The method reasonably and dynamically adjusts the cooking time according to the real-time cooking state of the food, and ensures the cooking effect of the to-be-cooked food.

Description

Technical Field

[0001] This invention relates to the field of home appliance control technology, and in particular to a method for determining the doneness of food and a cooking device. Background Technology

[0002] With the continuous advancement of automatic cooking technology, users can use cooking equipment equipped with automatic cooking technology to automatically cook the food to be cooked. Under normal circumstances, the cooking equipment will first identify the food to be cooked, select a preset cooking recipe from the database according to the user's cooking needs and the type of food to be cooked, and strictly execute the cooking operation according to the cooking recipe.

[0003] However, in actual cooking situations, the initial state of the food that users put in to cook is different. For example, the same food may be thick or thin, half-cooked or fully raw. Therefore, if you cook directly according to the preset cooking recipe and cooking method, the cooking effect will be poor due to the different properties of the food, which will lead to a decline in the user experience. Summary of the Invention

[0004] This invention provides a method and cooking device for determining the doneness of food, which enables reasonable and dynamic adjustment of cooking time based on the real-time cooking status of the food, ensuring the cooking effect of the food and thus improving the user's cooking experience.

[0005] In a first aspect, embodiments of the present invention provide a method for determining the doneness of food, applied to a cooking device, the cooking device being communicatively connected to a cloud server; the method includes: The system acquires an initial image of the food to be cooked on the food carrier and sends the initial image to the cloud server, so that the cloud server can generate a set of cooking images corresponding to the doneness of the food based on the initial image.

[0006] Receive a set of cooked food images at different levels of doneness. For each cooked food image at different levels of doneness in the set, determine a multi-feature matching degree group corresponding to the cooked food image at each level of doneness based on the cooked food image at each level of doneness and the current food image.

[0007] The actual doneness of the food to be cooked at the current moment is determined based on the multi-feature matching degree group corresponding to each degree of doneness. The food is then cooked according to the remaining cooking time based on the actual doneness and the target doneness of the food to be cooked.

[0008] The food doneness determination method provided in this invention first acquires an image of the food in its initial state before cooking begins, i.e., the initial food image. Then, a cloud server generates cooked images showing the food at different doneness levels. This achieves the acquisition of the initial state of the food to be cooked and enables the prediction of cooked images at different doneness levels based on the initial state. This solves the problem of not focusing on the initial state of the food during the cooking process, providing a foundation for accurately determining the doneness of the food during cooking. The method matches the current food image with multiple features using the cooked images at different doneness levels predicted from the initial state. This not only quantifies the matching degree between the cooked images at different doneness levels and the current food image from multiple feature perspectives, using the predicted cooked images from the initial state as a benchmark, but also provides an accurate basis for determining the actual doneness of the food at the current moment. The remaining cooking time can be continuously updated based on the actual and target doneness, enabling reasonable and dynamic adjustment of the cooking time according to the real-time cooking status of the food, ensuring the cooking effect of the food and thus improving the user's cooking experience.

[0009] Secondly, embodiments of the present invention also provide a food cooking degree determination device, which is applied to a cooking device and is communicatively connected to a cloud server; the device includes: The acquisition module is used to acquire the food to be cooked on the food carrier to obtain the initial image of the food, and send the initial image of the food to the cloud server so that the cloud server can generate a set of cooking images of the cooked food according to the initial image of the food.

[0010] The determination module receives a set of cooked food images of different levels of doneness. For each cooked food image in the set of cooked food images of different levels of doneness, it determines a group of multiple features matching degree corresponding to the cooked food image of each level of doneness based on the cooked food image of each level of doneness and the current food image.

[0011] The prediction module is used to determine the actual doneness of the food to be cooked at the current moment based on the multi-feature matching degree group corresponding to each degree of doneness, and to determine and cook the food according to the remaining cooking time based on the actual doneness and the target doneness of the food to be cooked.

[0012] Thirdly, embodiments of the present invention also provide a cooking apparatus, the cooking apparatus comprising: At least one processor; and A memory that is communicatively connected to at least one processor; wherein, The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to perform the food doneness determination method according to any embodiment of the present invention.

[0013] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the food cooking determination method of any embodiment of the present invention.

[0014] Fifthly, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the food cooking determination method of any embodiment of the present invention.

[0015] It should be noted that the aforementioned computer instructions may be stored, in whole or in part, on a computer-readable storage medium. This computer-readable storage medium may be packaged together with the processor of the food doneness determination device, or it may be packaged separately from the processor of the food doneness determination device; this application does not impose any limitations on this.

[0016] The descriptions of the second, third, fourth, and fifth aspects in this application can be referenced to the detailed description of the first aspect; and the beneficial effects of the descriptions of the second, third, fourth, and fifth aspects can be referenced to the analysis of the beneficial effects of the first aspect, which will not be repeated here.

[0017] In this application, the name of the aforementioned food doneness determination device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear under other names. As long as the function of each device or functional module is similar to that of this application, it falls within the scope of the claims of this application and its equivalents.

[0018] These or other aspects of this application will become more readily apparent in the following description. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating a method for determining the doneness of food provided in an embodiment of the present invention; Figure 2 A flowchart illustrating another method for determining food doneness provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a food ripeness determination device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a cooking device provided in an embodiment of the present invention. Detailed Implementation

[0021] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0022] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0023] The terms “initial” and “target” in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0024] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0025] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc. Moreover, embodiments and features in the embodiments of the present invention can be combined with each other without conflict.

[0026] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0027] In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0028] Figure 1 This is a flowchart illustrating a method for determining food doneness according to an embodiment of the present invention. This embodiment is applicable to situations where the remaining cooking time is adjusted based on the real-time actual doneness of the food during automatic cooking in a cooking device. This method can be executed by a food doneness determining device, which can be implemented in hardware and / or software and can be integrated into the cooking device. In this embodiment, the cooking device acts as the executing entity and can be an oven, stove, or other equipment capable of cooking food. Furthermore, in this embodiment, the cooking device is communicatively connected to a cloud server. (Continue to refer to...) Figure 1 This embodiment specifically includes: S101. Collect the food to be cooked on the food carrier to obtain the initial image of the food, and send the initial image of the food to the cloud server so that the cloud server can generate a set of cooking images of the doneness of the food to be cooked based on the initial image of the food.

[0029] In this context, the food carrier refers to a device prepared for the cooking equipment that can support and contain the food to be cooked. In this embodiment, if the cooking equipment is a stove, the food carrier can be a pot or similar appliance; if the cooking equipment is an oven or similar appliance, the food carrier is a carrier inside the oven. The food to be cooked is the food that the user needs to cook. The initial food image is the initial image of the food to be cooked captured when it is placed on the food carrier and cooking is not yet started, used to characterize the food state in its initial state. The cooked food image set includes cooked food images predicted for the food to be cooked at different levels of doneness. In this embodiment, the doneness of each cooked food image in the cooked food image set is higher than the doneness of the food to be cooked when the initial food image was captured.

[0030] Specifically, when the user places the food to be cooked on the food carrier of the cooking device, the built-in acquisition component of the cooking device can capture the food placed on the food carrier at a fixed angle to obtain an initial image of the food. The food doneness corresponding to the initial food image is actually the original doneness of the food before cooking begins. Then, the cooking device sends the initial food image to a cloud server, which can first perform image recognition on the initial food image, and then generate cooked doneness images at different doneness levels that have the same shape as the food to be cooked in the initial food image, based on the recognition results and the initial food image. The cooked doneness images corresponding to different doneness levels are then used as a set of cooked doneness images.

[0031] For example, the cloud server first identifies the food type corresponding to the food to be cooked based on the initial food image, and then generates cooking images of the food to be cooked at different levels of doneness based on the initial food image and the possible food forms of the food at different levels of doneness as determined in the experiment or test.

[0032] In this embodiment, an image of the food in its initial state is captured before cooking begins, i.e., the initial food image. The cloud server is then used to generate cooked images of the food at different levels of doneness. This achieves the acquisition of the initial state of the food to be cooked and enables the prediction of cooked images of the food at different levels of doneness based on the initial state of the food to be cooked. This solves the problem caused by the current food cooking process not focusing on the initial state of the food, and provides a foundation for accurately determining the doneness of the food to be cooked during the subsequent cooking process.

[0033] S102. Receive a set of cooked food images of different levels of doneness. For each cooked food image in the set of cooked food images of different levels of doneness, determine the multi-feature matching degree group corresponding to the level of doneness based on the cooked food image and the current food image.

[0034] The current food image refers to a food image with cooking food captured at the current moment. In this embodiment, the current moment can be real-time or a moment captured according to a preset acquisition cycle. The multi-feature matching degree group includes the matching degree corresponding to each feature under multiple features, used to characterize the matching degree between the cooked food image and the current food image under different features.

[0035] Specifically, after the cloud server obtains the cooked food images at different doneness levels, the cooking device receives these images from the cloud server. At this point, the cooking device begins cooking the food and, during the cooking process, collects images of the food at different "current moments" in real-time or according to a preset collection cycle. Then, it performs multi-feature matching between the current food image and each cooked food image at different doneness levels in the cooked food image set to obtain the degree of multi-feature matching between the current food image and each cooked food image corresponding to a different doneness level. Finally, the degree of multi-feature matching between the current food image and all cooked food images at different doneness levels is used as a multi-feature matching degree group.

[0036] For example, for each cooked food image corresponding to a cooked food level, the degree of matching between that cooked food image and the current food image in terms of features such as color, texture, and shape can be determined; where shape can include the area and perimeter of the food outline; texture can include the proportion of edge pixels. Furthermore, the feature matching degree between the cooked food image and the current food image under each feature can be used as a group of multi-feature matching degrees.

[0037] In this embodiment, the current food image can be matched with multiple features using cooking images of different degrees of doneness predicted from the initial state of the food. This not only enables the matching degree between cooking images of different degrees of doneness and the current food image from multiple feature perspectives, but also provides an accurate basis for accurately determining the actual doneness of the food to be cooked at the current moment.

[0038] S103. Determine the actual ripeness of the food to be cooked at the current moment based on the multi-feature matching degree group corresponding to each ripeness level, and cook according to the remaining cooking time based on the actual ripeness and the target ripeness of the food to be cooked.

[0039] The target doneness is the level of doneness that the user has preset or that is determined based on a recipe preset by the cooking equipment, and it is the level of doneness that the food to be cooked needs to reach in this cooking session. The remaining cooking time is the remaining cooking time for the food to be cooked.

[0040] Specifically, after determining the multi-feature matching degree group corresponding to each degree of ripeness, the ripeness corresponding to the cooking image that best matches the current food image in the multi-feature matching degree group can be selected as the actual ripeness of the food to be cooked at the current moment. Then, the remaining cooking time can be determined based on the difference between the actual ripeness and the target ripeness. Finally, the cooking device can perform the cooking operation according to the updated remaining cooking time.

[0041] In this embodiment, since the actual cooking degree is determined based on the cooking image of the food's initial state, which is the image with the highest matching degree with the current food image, it can solve the problem caused by not considering the initial state of the food when cooking. Furthermore, the remaining cooking time can be continuously updated according to the actual cooking degree and the target cooking degree, so as to realize the reasonable and dynamic adjustment of the cooking time according to the real-time cooking state of the food, ensuring the cooking effect of the food and thus improving the user's cooking experience.

[0042] The food doneness determination method provided in this invention first acquires an image of the food in its initial state before cooking begins, i.e., the initial food image. Then, a cloud server generates cooked images showing the food at different doneness levels. This achieves the acquisition of the initial state of the food to be cooked and enables the prediction of cooked images at different doneness levels based on the initial state. This solves the problem of not focusing on the initial state of the food during the cooking process, providing a foundation for accurately determining the doneness of the food during cooking. The method matches the current food image with multiple features using the cooked images at different doneness levels predicted from the initial state. This not only quantifies the matching degree between the cooked images at different doneness levels and the current food image from multiple feature perspectives, using the predicted cooked images from the initial state as a benchmark, but also provides an accurate basis for determining the actual doneness of the food at the current moment. The remaining cooking time can be continuously updated based on the actual and target doneness, enabling reasonable and dynamic adjustment of the cooking time according to the real-time cooking status of the food, ensuring the cooking effect of the food and thus improving the user's cooking experience.

[0043] Figure 2 This is a flowchart illustrating another method for determining food doneness according to an embodiment of the present invention. This embodiment is based on the above embodiment, specifying the determination of multi-feature matching degree groups, the determination of actual doneness, the determination of remaining cooking time, and other optional steps. In this embodiment, the method may specifically include: S201. Collect the food to be cooked on the food carrier to obtain the initial image of the food.

[0044] Specifically, after the user places the food to be cooked on the food carrier, the cooking device can control the acquisition component to scan the area of ​​the food to be cooked, and determine the target acquisition angle for the food to be cooked based on the scan results. The target acquisition angle is primarily determined to capture the overall appearance of the food to be cooked and its attribute information. Then, an initial image of the food can be obtained by acquiring images of the food to be cooked according to the target acquisition angle. Simultaneously, during subsequent cooking, the device will continue to acquire images of the food to be cooked according to the target acquisition angle as much as possible to obtain the current image of the food.

[0045] S202. Send the initial food image to the cloud server so that the cloud server calls the ripeness-generating artificial intelligence (AI) model to generate ripeness-based cooking images at different ripeness levels corresponding to the initial food image, and use the ripeness-based cooking images at different ripeness levels as a ripeness-based cooking image set.

[0046] Among them, the ripeness generative artificial intelligence (AI) model is a generative model trained using manually labeled sample food attribute information and initial images of sample food as training samples.

[0047] Specifically, an initial image of the food is sent to a cloud server. The cloud server can first perform simple image recognition on the initial image to determine the type and shape of the food to be cooked. Then, the recognition result is used as a simple annotation of the initial image and input into the cookedness generation AI model. The cookedness generation AI model can automatically generate standard cooking images that are the same as the food to be cooked in the initial image except for the degree of cookedness (which will cause slight differences in food color and shape). Other features (such as placement and food type) are the same. That is, cookedness images at different degrees of cookedness. Finally, all cookedness images are used as a cookedness image set.

[0048] For example, an initial image of food is sent to a cloud server, causing the cloud server to invoke a preheating-generative artificial intelligence (AI) model to generate cooked images at different preheating levels corresponding to the initial food image, including: (a) Send an initial image of the food to the cloud server so that the cloud server can identify the initial food attribute information of the food to be cooked based on the initial food image.

[0049] The initial food attribute information is the food attribute information directly identified by the cloud server based on the initial food image. Optionally, the food attribute information in this embodiment mainly includes: food type (such as meat, vegetables, mixed food, etc.), or even food name (such as beef, pork, green vegetables, cabbage, etc.), initial food state (such as raw, cold, semi-cooked, fully cooked, etc.), and the proportion of each food (limited to mixed food, the proportion of different types of food can be specifically determined).

[0050] Specifically, an initial image of the food can be sent directly to the cloud server, which can then identify the initial food attribute information of the food to be cooked based on the initial image.

[0051] (ii) Receive initial food attribute information and correct the initial food attribute information based on user feedback information to obtain target food attribute information.

[0052] Among them, the user feedback information is the actual food attribute information that the user provides feedback or correction to the initial food attribute information identified by the cloud server; optionally, in actual use, the user feedback information can be empty.

[0053] Specifically, the cooking equipment receives initial food attribute information and can display it on its control panel or send it directly to the user terminal. Based on user feedback (such as providing actual food attribute information or confirming the initial food attribute information is correct), the initial food attribute information is corrected to obtain the target food attribute information. If the user feedback is empty, the initial food attribute information is directly used as the target food attribute information.

[0054] (iii) Send the target food attribute information to the cloud server so that the cloud server can call the ripeness generative AI model and generate a set of ripeness cooking images at different ripeness levels corresponding to the initial food image based on the target food attribute information.

[0055] Specifically, after determining the target food attribute information, the target food attribute information can be directly sent to the cloud storage. The cloud storage can then retrieve the trained ripeness-generating AI model, input the initial food image and the target food attribute information into the ripeness-generating AI model, and the ripeness-generating AI model will generate a set of predicted ripeness cooking images of the food to be cooked at different ripeness levels.

[0056] Optionally, in this embodiment, the training method of the ripeness-generating AI model is as follows: initially, take pictures of a certain food used in the experiment at different ripeness levels during cooking from the same shooting angle to obtain a sample ripeness cooking image group corresponding to the food. Then, perform the same operation on multiple types of food to obtain a large number of sample ripeness cooking image groups corresponding to different types of food. Finally, manually annotate these sample ripeness cooking image groups (manual annotation can include ripeness, food attribute information, etc.) and input them into the AI ​​model to complete the initial learning of the model. Furthermore, different types of food are selected (starting with a single food, then progressing to single-type mixed foods, and finally multi-type mixed foods) as samples, which are then input into the pre-trained ripeness-generating AI model to generate a set of sample ripeness cooking images for each food at different ripeness levels. Next, these different types of food are cooked, and during the cooking process, actual food images at different ripeness levels are captured and labeled. These labels serve as correction samples for the sample ripeness cooking image set generated by the ripeness-generating AI model. By comparing the actual food images at different ripeness levels generated by the model with the corresponding ripeness images in the sample ripeness cooking image set generated by the model, the AI ​​model can be further trained. Finally, once the training accuracy of the ripeness-generating AI model reaches the preset accuracy, the training of the ripeness-generating AI model is considered complete. Other related operations for training the generative AI model are consistent with conventional model training methods and will not be elaborated upon here.

[0057] In this embodiment, the cloud server can directly call the trained ripeness-generating AI model to generate a set of cooked images at different ripeness levels, using the initial food image as a prototype. This provides an accurate prototype reference for determining the ripeness of the food to be cooked at different times in real time. Furthermore, in addition to inputting the initial food image into the model, the cloud server also inputs the target food attribute information confirmed by the user. This provides a guiding basis for the ripeness-generating AI model to generate more accurate cooked images at different ripeness levels, enabling the model to reasonably determine the image information of the food to be cooked at different ripeness levels based on accurate food attribute information.

[0058] S203, Receive a set of cooked images showing the doneness of the food.

[0059] Specifically, after a set of cooked images of ripeness is generated on a cloud server using a ripeness-generating AI model, the cooking equipment can receive the cooked images of ripeness.

[0060] In this embodiment, the cooking device receives a set of cooked food images, which can provide a cookedness comparison item for real-time comparison based on the current food images collected at different times. This provides an accurate comparison basis for determining the cookedness of the food at different times, making it easier for the cooking device to grasp the changes in the cookedness of the food during the cooking process.

[0061] S204. For each cooked food image in the cooked food image set corresponding to a cooked food level, extract the cooked food multi-feature information from the cooked food image and the current food image respectively to obtain the simulated cooked food feature group and the actual cooked food feature group.

[0062] The ripeness multi-feature information is used to characterize different features of the food in the image. In this embodiment, the features included in the ripeness multi-feature information can be information corresponding to features such as color, texture, and shape. The simulated ripeness feature group includes feature information corresponding to different features extracted from the cooked food image. The actual ripeness feature group includes feature information corresponding to different features extracted from the current food image.

[0063] Specifically, since the cooked food image set includes cooked food images at different levels of doneness, in order to determine the doneness of the food to be cooked at the current moment, multiple features of doneness can be extracted from the cooked food images corresponding to each level of doneness, and the feature information corresponding to each feature can be used as a simulated doneness feature group. At the same time, the same feature extraction method can be directly performed on the current food image, and the feature information corresponding to each feature can be used as the actual doneness feature group.

[0064] For example, in this embodiment, the features in the multi-feature information of ripeness include hue-saturation-value (HSV) color space features, texture features based on local binary pattern (LBP), food outline area features, perimeter features, edge pixel ratio features, and the ratio features of each food type. Among them, (1) for the HSV color space features, the image (cooking image of ripeness, current food image) can be preprocessed and food foreground segmented first, and then the red, green and blue image can be converted into HSV color space, the quantization intervals of each channel of H, S and V can be divided, the pixel frequency of each channel can be counted, a single channel histogram can be generated, and finally the image color histogram, i.e. HSV color histogram, can be obtained by splicing or combining. (2) For LBP-based texture features, the image can be preprocessed and the foreground segmented first, and then only the food pixels can be binary masked. After masking, the entire food pixel is converted into a grayscale image. Then, the LBP sampling neighborhood can be selected according to the specific cooking operation (for example, if the cooking operation involves constantly changing the placement of the food to be cooked, such as stir-frying, rotating the food, etc., the prototype neighborhood local binary mode can be selected; if the cooking operation keeps the placement of the food unchanged, the n×n square neighborhood or uniform square binary mode can be selected). After selecting the sampling neighborhood, the grayscale pixels of the neighborhood are sampled according to the sampling neighborhood. Then, the LBP binary code is calculated pixel by pixel, and the coding frequency is counted to construct the original histogram. Optionally, normalization is then performed to eliminate the influence of the food area size on the statistical results, and the LBP local binary mode histogram can be obtained. (3) For the food contour area feature, the food contour can be determined directly after the image is preprocessed and the food foreground segmented, and then the contour area value is calculated according to the food contour. (4) For the perimeter feature, it can be determined directly based on the food outline; if it is an irregular image, the perimeter feature of the entire irregular image can be determined by segmenting the food outline in the image to determine the outline of each segmented graphic. (5) For the edge pixel ratio feature, when obtaining the grayscale image according to (2), the edge operator is used to perform edge detection on the grayscale image to generate a binary edge image, and then the total number of pixels in the food mask and the number of edge pixels in the food area are counted. The ratio of the two quantities can be obtained to get the edge pixel ratio. (6) For the ratio feature of each food type, the number of pixels occupied by different types of food in the image can be directly used by the image recognition algorithm, and the ratio of each food type can be determined based on the number of pixels occupied by all types of food.

[0065] S205. For each pair of simulated maturity features and actual maturity features that correspond to the same feature in the simulated maturity feature group and the actual maturity feature group, calculate the feature matching degree between the simulated maturity features and the actual maturity features.

[0066] Specifically, after obtaining the simulated ripeness feature groups corresponding to cooking images of different ripeness levels and the actual ripeness feature groups corresponding to the current food image, the degree of feature matching between the simulated ripeness feature groups corresponding to each ripeness cooking image and the actual ripeness features can be calculated.

[0067] Optionally, the feature matching degree in this embodiment can be calculated using methods such as similarity calculation or feature difference statistics. For different features, the appropriate calculation method can be selected based on the feature information corresponding to the feature. For example, for the feature information of the histogram, the feature matching degree can be obtained by methods such as feature overlapping pixels or histogram overlap area calculation.

[0068] For example, in this embodiment, the features in the maturity feature group (i.e., the simulated maturity feature group and the actual maturity feature group) include the image color histogram and the local binary pattern histogram; after taking the feature matching degree of all simulated maturity features and actual maturity features corresponding to the same feature as a multi-feature matching degree group, it also includes: (i) For the graphic color histograms in the simulated maturity feature group and the actual maturity feature group, calculate the first feature matching degree between the simulated maturity graphic color histogram and the actual maturity graphic color histogram; and for the local binary pattern histograms in the simulated maturity feature group and the actual maturity feature group, calculate the second feature matching degree between the simulated maturity local binary pattern histogram and the actual maturity local binary pattern histogram.

[0069] In this embodiment, the graphic color histogram mainly refers to the HSV color histogram. The first feature matching degree is used to describe the feature matching degree between the simulated maturity graphic color histogram and the actual maturity graphic color histogram; the second feature matching degree is used to describe the feature matching degree between the simulated maturity local binary pattern histogram and the actual maturity local binary pattern histogram.

[0070] Specifically, after obtaining the simulated ripeness feature group and the actual ripeness feature group for each ripeness level, further processing can be performed on the two features (image color histogram and local binary pattern histogram) in the ripeness feature group, as they can more clearly show the color change information of food at different ripeness levels. That is, the first feature matching degree between the simulated ripeness image color histogram and the actual ripeness image color histogram is calculated, and the second feature matching degree between the simulated ripeness local binary pattern histogram and the actual ripeness local binary pattern histogram is also calculated.

[0071] For example, the calculation of the first feature matching degree can be performed by directly calculating the Bach distance or chi-square distance between the simulated maturity graphic color histogram and the actual maturity graphic color histogram. The calculation of the second feature matching degree can be performed by directly using methods such as Manhattan distance (L1 distance), Euclidean distance (L2 distance), or histogram intersection inverse distance.

[0072] (ii) Determine the first confidence coefficient of the first feature matching degree and the second confidence coefficient of the second feature matching degree based on the food attribute information corresponding to the food to be cooked.

[0073] In this embodiment, the food attribute information can be initial food attribute information, more preferably target food attribute information. The confidence coefficients (first confidence coefficient and second confidence coefficient) in this embodiment are used to characterize the reliability of the feature matching degree for the food to be cooked.

[0074] Specifically, since the image color histogram is mainly used to reflect the color change features of the food surface in the image, and the local binary mode histogram is mainly used to reflect the changes in the texture features of the food in the image, in the actual cooking process, different types of food have different main changes under heat, and even the same type or different types of food may be too similar in color or texture features. Therefore, in order to increase the confidence of different feature matching degrees, dynamic confidence coefficients can be set for the first feature matching degree and the second feature matching degree respectively based on the food attribute information, that is, to determine the first confidence coefficient of the first feature matching degree and the second confidence coefficient of the second feature matching degree.

[0075] Optionally, when actually determining the first confidence coefficient and the second confidence coefficient, the first confidence coefficient of the first feature matching degree and the second confidence coefficient of the second feature matching degree are determined based on the food attribute information corresponding to the food to be cooked, including: (1) Determine the offset ratio based on the matching degree of the first feature, the matching degree of the second feature, the food type information and the proportion of food types in the food attribute information.

[0076] The offset weight is used to correct the initial first confidence coefficient and the initial second confidence coefficient. In this embodiment, the offset weight may include a primary correction value and a secondary correction value. When the food to be cooked includes only one type of food, only the primary correction value is used. When the food to be cooked includes multiple types of food, the offset weight includes both the primary correction value and the secondary correction value.

[0077] Specifically, the specific type or even the name of the food to be cooked can be determined based on the food type information in the food attribute information. This allows for the determination of the degree of color change of the food when heated, with the degree of color change being the primary factor in determining the offset weight. Furthermore, if the food to be cooked includes multiple types of food, the offset weight can be further adjusted based on the proportion of each food type.

[0078] (2) The initial first confidence coefficient and the initial second confidence coefficient are corrected by using the offset weight.

[0079] The initial first confidence coefficient and the initial second confidence coefficient are the default first confidence coefficient and second confidence coefficient.

[0080] Specifically, the initial first confidence coefficient and the initial second confidence coefficient can be directly corrected using the offset weight.

[0081] For example, in this embodiment, the initial first confidence coefficient is optionally set to 1 and the initial second confidence coefficient is set to 1 by default. In one implementation, if the food to be cooked is a single food item, the first confidence coefficient of the first feature matching degree and the second confidence coefficient of the second feature matching degree can be determined based on the color change of the food item after heating. For example, for fish or tomatoes, the color change after heating is relatively small, so the first confidence coefficient can be smaller than the second confidence coefficient. Specifically, it can be determined based on the color change; for example, compared to fish, tomatoes show a smaller color change after heating, so the first confidence coefficient of fish can be greater than the first confidence coefficient of tomatoes. Another implementation method involves determining the primary correction value (e.g., primary correction precision of 0.1) for the default first and second confidence coefficients if the food to be cooked includes at least two different foods. The primary correction value is determined based on the food whose color change after heating. Then, secondary correction values ​​(e.g., secondary correction precision of 0.01) are dynamically determined based on the color change of the other foods after heating. Finally, the initial first and second confidence coefficients are adjusted by combining the primary and secondary correction values. For example, if the food to be cooked includes fish, celery stalks, and raw crayfish, the primary correction value is determined in the above manner. For fish that show the least color change after heating, the primary correction value is +0.7. For celery, which changes from light green to dark green and eventually to yellowish green after heating, the secondary correction value is -0.04. For crayfish, which changes from grayish-white to red after heating, the corresponding secondary correction value is -0.09. Therefore, the first confidence coefficient after correction is 1 - 0.7 - (-0.04) - (-0.09) = 0.43, and the second confidence coefficient after correction is 1 + 0.7 + (-0.04) + (-0.09) = 1.57.

[0082] (iii) The first feature matching degree, the first confidence coefficient, the second feature matching degree and the second confidence coefficient are weighted and summed to obtain the comprehensive matching degree of the two features.

[0083] Specifically, after obtaining the first feature matching degree, the first confidence coefficient, the second feature matching degree, and the second confidence coefficient, they can be weighted and summed, i.e., the first feature matching degree × the first confidence coefficient + the second feature matching degree × the second confidence coefficient. The resulting value is the comprehensive matching degree of the two features.

[0084] (iv) Replace the first feature matching degree and the second feature matching degree in the multi-feature matching degree group with the dual-feature comprehensive matching degree.

[0085] Specifically, the first feature matching degree and the second feature matching degree in the multi-feature matching degree group can be replaced with the comprehensive matching degree of the two features.

[0086] In this embodiment, since different types of food undergo different main changes under heat during actual cooking, and even foods of the same or different types may be too similar in color or texture features, in order to increase the credibility of different feature matching degrees and to better highlight the characteristics of the image color histogram and the local binary pattern histogram, dynamic confidence coefficients can be assigned to the first feature matching degree and the second feature matching degree based on the food attribute information of the food to be cooked. This can achieve the goal of highlighting the features of the food that change significantly after heating based on the food's own heat properties, while weakening the features of the food that change less after heating. This allows for the calculation of a more accurate dual-feature comprehensive matching degree between the cooked image and the current food image in terms of HSV color space features and LBP-based texture features, providing an accurate basis for determining the doneness of the food to be cooked.

[0087] S206. The degree of feature matching between all simulated maturity features and actual maturity features that correspond to the same feature is taken as a multi-feature matching degree group.

[0088] Specifically, after obtaining the feature matching degree between the simulated maturity feature and the actual maturity feature of each feature, the feature matching degree between the simulated maturity feature and the actual maturity feature of all features can be used as a multi-feature matching degree group.

[0089] In this embodiment, multiple features in the image that can reflect the changes of food after being heated are extracted, which can provide a basis for different dimensions of feature changes for accurately determining the doneness of the food. This ensures that the doneness of the food to be cooked will not cause the cooking equipment to cook incorrectly due to misjudgment of a single feature.

[0090] S207. For each maturity level corresponding to a multi-feature matching degree group, determine the total matching degree value corresponding to the maturity level based on the matching degree of all features in the multi-feature matching degree group.

[0091] Specifically, for each maturity level, there is a multi-feature matching degree group, which includes the matching degree values ​​of multiple features under that maturity level. Therefore, the total matching degree value corresponding to that maturity level can be directly calculated based on the matching degree of all features and their corresponding default weight coefficients.

[0092] For example, determining the total matching degree value corresponding to maturity based on the matching degree of all features in the multi-feature matching degree group includes: (i) Determine the weight coefficients of each feature in the multi-feature matching degree group based on the current cooking method, the food type, food form information and the proportion of food type in the food attribute information corresponding to the food to be cooked.

[0093] The food morphology information includes information such as the food's placement location, thickness, and size.

[0094] Specifically, since the multi-feature matching degree group includes features from different perspectives for evaluating the doneness of the food to be cooked, and the doneness of the food to be cooked depends not only on external factors such as cooking operation and heat level, but also on the food attribute information of the food to be cooked, such as the thicker and larger the food, the slower it takes to cook, and the more types of food there are in the food to be cooked, the different cooking times for different types of food, etc., it is necessary to determine the weight coefficients corresponding to each feature in the multi-feature matching degree group based on the current cooking method, the types of food, the food morphology information, and the proportion of food types in the food attribute information corresponding to the food to be cooked.

[0095] For example, default weights corresponding to different features can be pre-set for different types of food. For example, for vegetables (excluding HSV color space features and LBP-based texture features, since the two can be combined, their default weight coefficient is 1), their color texture feature (edge ​​pixel ratio feature) = 1.2, and food shape feature (perimeter feature, food outline area feature) = 0.7; for meat, their color texture feature = 1.3, and food shape feature = 1.5. Further, a first correction coefficient can be determined based on the current cooking method and the type of food. For example, if this type of food is easier to cook under this cooking method, the first correction coefficient is a positive number (the specific value depends on the specific situation and can be pre-set in terms of grading), otherwise it is a negative number. A second correction coefficient is determined based on the specific food name (such as beef, fish, etc. in meat category), food morphology information, and the proportion of the food type. Finally, the first and second correction coefficients are used to correct the weight coefficients of the default features to obtain the corrected coefficients for each feature.

[0096] (ii) The total matching degree value is calculated based on the weight coefficients corresponding to each feature and the numerical values ​​corresponding to each feature in the multi-feature matching degree group.

[0097] Specifically, the weight coefficients corresponding to each feature can be multiplied by the numerical values ​​corresponding to that feature, and then all the results can be summed. In other words, the total matching degree value can be obtained by using the weighted summation method.

[0098] In this embodiment, for each feature in the multi-feature matching degree group, the dynamic weight coefficient corresponding to each feature can be determined by quantifying the influence of the current cooking method on the doneness of the food to be cooked and the influence of the food attribute information of the food to be cooked on the doneness of the food during cooking. Then, the total matching degree value is calculated based on the weight coefficients corresponding to each feature and the values ​​corresponding to each feature in the multi-feature matching degree group. This enables the determination of the matching degree that can judge the actual doneness when further judging the doneness of the food to be cooked based on the heating condition of the food itself and the influence of the cooking method on the heating condition of the food.

[0099] S208. Take the largest value among all the total matching degree values ​​corresponding to the degree of cooking as the actual degree of cooking of the food to be cooked at the current moment.

[0100] Specifically, after calculating the total matching degree value corresponding to all degrees of doneness, the degree of doneness corresponding to the largest value can be directly used as the actual doneness of the food to be cooked at the current moment.

[0101] In this embodiment, based on the specific cooking method of the food to be cooked and the quantification and dynamic weighting of the changes in various characteristics of the food during the cooking process, the actual doneness of the food to be cooked can be accurately determined by using the characteristics that better reflect the actual morphological changes of the food to be cooked during the cooking process as the main judgment point and other characteristics as auxiliary judgment points. This realizes the determination of actual doneness based on the attribute changes of the food to be cooked during the cooking process.

[0102] S209. Determine the remaining cooking time based on the current cooking method, current heat information, and the difference between the actual doneness and the target doneness.

[0103] Among them, the current heat information refers to the current cooking heat level, which can be quantified data.

[0104] Specifically, the process begins by determining the difference between the actual doneness and the target doneness (if the doneness is only a doneness level, i.e., a non-quantified value, then the doneness can be quantified to obtain a quantified doneness value). This yields the doneness difference. Next, the current cooking method is determined. If the cooking method is static cooking (i.e., cooking without turning the food), the remaining cooking time can be directly estimated using the current heat information, doneness change information (determined using previous actual doneness and cooking time), and the doneness difference. If the cooking method is dynamic cooking (requiring turning the food), a time correction term needs to be deduced based on the different heat transfer methods of this cooking method. For example, in stir-frying, the food is turned more frequently, resulting in more even heating compared to pan-frying; therefore, the time correction term for stir-frying is smaller than that for pan-frying. After obtaining the time correction term, the initial remaining cooking time is calculated based on the current heat information, doneness change information, and the doneness difference, and then the initial remaining cooking time is corrected using the time correction term.

[0105] Optionally, in this embodiment, the obtained quantified ripeness value can be directly input into the model at the initial stage of training the ripeness-generating AI model by inputting experimental or test images corresponding to different ripeness values. Then, when the ripeness-generating AI model is used to output a set of ripeness cooking images, the output ripeness cooking images of different ripeness levels actually correspond to the ripeness cooking images at specific ripeness values. Therefore, the cooking device can determine the relationship between the ripeness value and the feature value under different features based on the "standard ripeness cooking images" in the ripeness cooking image set. Then, when determining the actual ripeness of the food to be cooked at the current moment based on the multi-feature matching degree group corresponding to each ripeness, the actual ripeness value can be directly determined by the relationship between the ripeness value and each feature value under different features.

[0106] In this embodiment, not only is the remaining cooking time predicted based on the current heat information and the difference in doneness, but the heating status of the food to be cooked is also analyzed according to different cooking methods to optimize the remaining cooking time. This realizes the determination of the heating status of the food to be cooked based on the specific cooking method, and then adjusts the change in doneness with cooking time, ensuring that the remaining cooking time can be predicted more accurately and improving the cooking effect of the food to be cooked.

[0107] S210. Cook according to the remaining cooking time.

[0108] Specifically, after determining the remaining cooking time, cooking can proceed according to that time. If the remaining cooking time is not zero, the process can return to step S204 to re-determine the doneness of the food, continuously adjusting the remaining cooking time based on the actual doneness until the remaining cooking time is close to zero (a threshold can be set, such as stopping when only 1 minute remains). Optionally, after returning to step S204, cooked images below the confirmed doneness level in the cooked image set can be removed to reduce the computational power of the cooking equipment and improve computational efficiency.

[0109] Figure 3 This is a schematic diagram of a food cooking degree determination device provided in an embodiment of the present invention, as shown below. Figure 3 As shown, this device is used in cooking equipment, which communicates with a cloud server; the device includes: The acquisition module 301 is used to acquire the food to be cooked on the food carrier to obtain the initial image of the food, and send the initial image of the food to the cloud server so that the cloud server can generate a set of cooking images of the cooked food according to the initial image of the food.

[0110] The determination module 302 is used to receive a set of cooked food images of different levels of doneness, and for each cooked food image corresponding to a level of doneness in the set of cooked food images, determine a group of multiple features matching degree corresponding to the level of doneness based on the cooked food image and the current food image.

[0111] The prediction module 303 is used to determine the actual ripeness of the food to be cooked at the current time based on the multi-feature matching degree group corresponding to each ripeness, and to determine and cook the food according to the remaining cooking time based on the actual ripeness and the target ripeness of the food to be cooked.

[0112] Optionally, the acquisition module 301 is specifically used for: Send an initial image of the food to the cloud server so that the cloud server can call a ripeness-generative artificial intelligence (AI) model to generate ripeness-based cooking images at different ripeness levels corresponding to the initial image of the food, and use the ripeness-based cooking images at different ripeness levels as a ripeness-based cooking image set.

[0113] Optionally, an initial food image is sent to a cloud server, so that the cloud server calls a ripeness-generating artificial intelligence (AI) model to generate cooked images at different ripeness levels corresponding to the initial food image. The acquisition module 301 is specifically used for: The process involves sending an initial food image to a cloud server, enabling the server to identify the initial food attribute information of the food to be cooked; receiving the initial food attribute information and correcting it based on user feedback to obtain the target food attribute information; sending the target food attribute information to the cloud server, enabling the server to invoke a ripeness-generating AI model and generate cooked images at different ripeness levels corresponding to the initial food image based on the target food attribute information; wherein, the ripeness-generating AI model is a generative model trained using manually labeled sample food attribute information and sample initial food images as training samples.

[0114] Optionally, the multi-feature matching degree group corresponding to the cooked food image is determined based on the cooked food image and the current food image. The determining module 302 is specifically used for: Multi-feature information of ripeness is extracted from the cooked image and the current food image to obtain simulated ripeness feature group and actual ripeness feature group. For each pair of simulated ripeness features and actual ripeness features that correspond to the same feature in the simulated ripeness feature group and actual ripeness feature group, the feature matching degree between simulated ripeness features and actual ripeness features is calculated. The feature matching degree of all simulated ripeness features and actual ripeness features that correspond to the same feature is taken as a multi-feature matching degree group.

[0115] Optionally, the features in the maturity feature group include image color histograms and local binary pattern histograms. After determining the feature matching degree between all simulated maturity features corresponding to the same feature and actual maturity features as a multi-feature matching degree group, the determining module 302 is further used for: For the graphic color histograms in the simulated ripeness feature group and the actual ripeness feature group, calculate the first feature matching degree between the simulated ripeness graphic color histogram and the actual ripeness graphic color histogram; and for the local binary pattern histograms in the simulated ripeness feature group and the actual ripeness feature group, calculate the second feature matching degree between the simulated ripeness local binary pattern histogram and the actual ripeness local binary pattern histogram; determine the first confidence coefficient of the first feature matching degree and the second confidence coefficient of the second feature matching degree based on the food attribute information corresponding to the food to be cooked; perform a weighted summation of the first feature matching degree, the first confidence coefficient, the second feature matching degree, and the second confidence coefficient to obtain the comprehensive matching degree of the two features; replace the first feature matching degree and the second feature matching degree in the multi-feature matching degree group with the comprehensive matching degree of the two features.

[0116] Optionally, the first confidence coefficient of the first feature matching degree and the second confidence coefficient of the second feature matching degree are determined based on the food attribute information corresponding to the food to be cooked. The determining module 302 is specifically used for: The offset weight is determined based on the degree of matching of the first feature, the degree of matching of the second feature, the food type information and the proportion of food types in the food attribute information; the offset weight is used to correct the initial first confidence coefficient and the initial second confidence coefficient.

[0117] Optionally, the actual doneness of the food to be cooked at the current moment is determined based on the multi-feature matching degree group corresponding to each degree of doneness. The prediction module 303 is specifically used for: For each level of ripeness, the total matching degree value corresponding to the ripeness is determined based on the matching degree of all features in the matching degree group. The ripeness corresponding to the largest value among all the total matching degree values ​​is taken as the actual ripeness of the food to be cooked at the current moment.

[0118] Optionally, the total matching degree value corresponding to the maturity is determined based on the matching degree of all features in the multi-feature matching degree group. The prediction module 303 is specifically used for: The weight coefficients of each feature in the multi-feature matching degree group are determined based on the current cooking method, the food type, the food form information, and the proportion of food types in the food attribute information corresponding to the food to be cooked; the total matching degree value is calculated based on the weight coefficients of each feature and the numerical values ​​of each feature in the multi-feature matching degree group.

[0119] Optionally, the remaining cooking time is determined based on the actual doneness and the target doneness of the food to be cooked. Specifically, the prediction module 303 is used for: The remaining cooking time is determined based on the current cooking method, current heat information, and the difference between the actual doneness and the target doneness.

[0120] The food ripeness determination device provided in the embodiments of the present invention can execute the food ripeness determination method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0121] It is worth noting that in the embodiments of the food cooking degree determination device described above, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0122] Figure 4This is a schematic diagram of a cooking apparatus provided for an embodiment of the present invention. The cooking apparatus is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The cooking apparatus can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0123] like Figure 4 As shown, the cooking device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer programs stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the cooking device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0124] Multiple components in the cooking appliance 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, optical disk, etc.; and a communication unit 19, such as a network card, modem, wireless transceiver, etc. The communication unit 19 allows the cooking appliance 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0125] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as methods for determining the doneness of food.

[0126] In some embodiments, the food doneness determination method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the cooking device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the food doneness determination method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the food doneness determination method by any other suitable means (e.g., by means of firmware).

[0127] This invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements, for example, the food cooking determination method provided in this invention.

[0128] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0129] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0130] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0131] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the food cooking determination method as provided in any embodiment of this invention.

[0132] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0133] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0134] Furthermore, the acquisition, storage, use, and processing of data in the technical solution of this invention all comply with the relevant provisions of national laws and regulations.

[0135] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for determining the doneness of food, characterized in that, The method is applied to a cooking device that is communicatively connected to a cloud server; the method includes: The system acquires an initial image of the food to be cooked on a food carrier and sends the initial image to the cloud server, so that the cloud server generates a set of cooking images corresponding to the doneness of the food based on the initial image. Receive the cooked food image set at different levels of doneness, and for each cooked food image at different levels of doneness in the cooked food image set, determine a multi-feature matching degree group corresponding to the cooked food image at each level of doneness based on the cooked food image at each level of doneness and the current food image; The actual doneness of the food to be cooked at the current moment is determined based on the multi-feature matching degree group corresponding to each degree of doneness. The food to be cooked is then cooked according to the remaining cooking time based on the actual doneness and the target doneness of the food to be cooked.

2. The method according to claim 1, characterized in that, Sending the initial food image to the cloud server, so that the cloud server generates a set of cooked food images corresponding to the doneness of the food based on the initial food image, includes: The initial image of the food is sent to the cloud server, so that the cloud server calls the ripeness-generating artificial intelligence (AI) model to generate ripeness-based cooking images at different ripeness levels corresponding to the initial image of the food, and the ripeness-based cooking images at different ripeness levels are used as the ripeness-based cooking image set.

3. The method according to claim 2, characterized in that, Sending the initial food image to the cloud server, so that the cloud server invokes a ripeness-generative artificial intelligence (AI) model to generate cooked images at different ripeness levels corresponding to the initial food image, includes: Send the initial image of the food to the cloud server so that the cloud server can identify the initial food attribute information of the food to be cooked based on the initial food image; Receive the initial food attribute information and correct the initial food attribute information according to user feedback information to obtain the target food attribute information; The target food attribute information is sent to the cloud server so that the cloud server can call the ripeness generative AI model and generate ripeness cooking images at different ripeness levels corresponding to the initial image of the food based on the target food attribute information. The ripeness generative AI model is a generative model trained using manually labeled sample food attribute information and initial images of sample food as training samples.

4. The method according to claim 1, characterized in that, The step of determining the multi-feature matching degree group corresponding to the cooked food based on the cooked food image and the current food image includes: The cooked food image and the current food image are extracted to obtain the simulated cooked food feature group and the actual cooked food feature group respectively. For each pair of simulated maturity features and actual maturity features that correspond to the same feature in the simulated maturity feature group and the actual maturity feature group, the feature matching degree between the simulated maturity features and the actual maturity features is calculated. The degree of feature matching between all simulated maturity features corresponding to the same feature and actual maturity features is defined as the multi-feature matching degree group.

5. The method according to claim 4, characterized in that, The features in the maturity feature group include image color histograms and local binary pattern histograms. After defining the feature matching degree between all simulated maturity features corresponding to the same feature and actual maturity features as the multi-feature matching degree group, it also includes: For the graphic color histograms in the simulated maturity feature group and the actual maturity feature group, calculate the first feature matching degree between the simulated maturity graphic color histogram and the actual maturity graphic color histogram; and for the local binary pattern histograms in the simulated maturity feature group and the actual maturity feature group, calculate the second feature matching degree between the simulated maturity local binary pattern histogram and the actual maturity local binary pattern histogram. The first confidence coefficient of the first feature matching degree and the second confidence coefficient of the second feature matching degree are determined based on the food attribute information corresponding to the food to be cooked; The first feature matching degree, the first confidence coefficient, the second feature matching degree, and the second confidence coefficient are weighted and summed to obtain the comprehensive matching degree of the two features; Replace the first feature matching degree and the second feature matching degree in the multi-feature matching degree group with the dual-feature comprehensive matching degree.

6. The method according to claim 5, characterized in that, The step of determining the first confidence coefficient of the first feature matching degree and the second confidence coefficient of the second feature matching degree based on the food attribute information corresponding to the food to be cooked includes: The offset weight is determined based on the first feature matching degree, the second feature matching degree, and the food type information and food type proportion information in the food attribute information; The initial first confidence coefficient and the initial second confidence coefficient are corrected using the offset weight.

7. The method according to claim 1, characterized in that, The step of determining the actual doneness of the food to be cooked at the current moment based on the multi-feature matching degree group corresponding to each degree of doneness includes: For each maturity level corresponding to a multi-feature matching degree group, the total matching degree value corresponding to the maturity level is determined based on the matching degree of all features in the multi-feature matching degree group. The maximum value among all the total matching values ​​corresponding to the degree of ripeness is taken as the actual ripeness of the food to be cooked at the current moment.

8. The method according to claim 7, characterized in that, The step of determining the total matching degree value corresponding to the maturity based on the matching degree of all features in the multi-feature matching degree group includes: The weight coefficients for each feature in the multi-feature matching degree group are determined based on the current cooking method, the food type, food form information and the proportion of food type in the food attribute information corresponding to the food to be cooked; The total matching degree value is calculated based on the weight coefficients corresponding to each feature and the numerical values ​​corresponding to each feature in the multi-feature matching degree group.

9. The method according to claim 1, characterized in that, The step of determining the remaining cooking time based on the actual doneness and the target doneness of the food to be cooked includes: The remaining cooking time is determined based on the current cooking method, current heat information, and the difference between the actual doneness and the target doneness.

10. A cooking device, characterized in that, include: One or more processors; Memory, used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the food cooking determination method as described in any one of claims 1 to 9.

Images