Dish identification method and device, kitchen equipment and storage medium

Abstract

The invention relates to a dish identification method and device, kitchen equipment and a storage medium. The method comprises the following steps: performing incremental learning on a current identification model by utilizing unknown dishes when the current identification model faces the unknown dishes, and enabling the current identification model to continuously absorb new information while keeping existing knowledge, so that the identification performance and generalization ability of the model are continuously improved; the incremental recognition model after incremental learning has the ability to recognize unknown dishes, so that the problem that the recognition accuracy of kitchen equipment is reduced when the kitchen equipment faces new types of dishes is solved.

Description

Technical Field

[0001] The present application relates to the field of dish recognition, and in particular to a dish recognition method, device, kitchen equipment and storage medium. Background Art

[0002] With the rapid development of smart home technology, the intelligence of kitchen equipment is constantly improving. Among them, the dish recognition function, as an important manifestation of the intelligence of kitchen equipment, can provide users with personalized eating suggestions. However, in practical applications, the dish recognition system of kitchen equipment faces many challenges.

[0003] As users’ cooking habits and needs become increasingly diverse, new types of dishes continue to emerge. Traditional machine learning models often need to be trained on fixed data sets and are difficult to adapt to the recognition needs of new categories of dishes. This results in a decrease in the recognition accuracy of kitchen equipment’s dish recognition models when faced with new categories of dishes, affecting user experience. Summary of the invention

[0004] The present application provides a dish recognition method, device, kitchen equipment and storage medium to solve the problem that existing kitchen equipment has low recognition accuracy for new categories of dishes.

[0005] In a first aspect, the present application provides a dish identification method, the method comprising:

[0006] When a dish collection image is acquired, the dish type of the dish collection image is determined according to a comparison result between a preset dish library and the dish collection image, wherein the preset dish library includes collection images of multiple different known dishes;

[0007] When the dish type of the dish captured image is an unknown dish, using the dish captured image to perform incremental learning on the current recognition model to obtain a first incremental recognition model;

[0008] determining a model performance of the first incremental recognition model based on a test result of the first incremental recognition model;

[0009] When the model performance of the first incremental recognition model is equal to or higher than the model performance of the current recognition model, the first incremental recognition model is used to recognize the target captured image to obtain a dish recognition result corresponding to the target captured image, wherein the target captured image includes the dish captured image and a captured image whose similarity to the dish captured image reaches a specified similarity.

[0010] Optionally, when the dish type in the dish captured image is an unknown dish, incremental learning is performed on the current recognition model using the dish captured image to obtain a first incremental recognition model, including:

[0011] When the dish type of the dish captured image is an unknown dish, outputting the dish captured image and the annotation prompt information corresponding to the dish captured image;

[0012] Upon receiving a labeling operation instruction for the dish captured image, constructing a first training sample corresponding to the dish captured image according to the labeling operation instruction, wherein the labeling operation instruction includes a plurality of labeling points and a labeling signature;

[0013] The first training sample corresponding to the captured dish image is used to perform incremental learning on the current recognition model to obtain a first incremental recognition model.

[0014] Optionally, incremental learning is performed on the current recognition model using the first training sample corresponding to the captured dish image to obtain a first incremental recognition model, including:

[0015] Acquire, from the preset dish library, similar images whose similarity to the dish captured image is greater than or equal to a preset similarity, and second training samples corresponding to the similar images;

[0016] Using the first training sample corresponding to the dish captured image and the second training sample corresponding to the similar image, incremental learning is performed on the current recognition model to obtain a first incremental recognition model.

[0017] Optionally, the step of performing incremental learning on the current recognition model using the first training sample corresponding to the captured dish image and the second training sample corresponding to the similar image to obtain a first incremental recognition model includes:

[0018] Cache the first training sample corresponding to the dish acquisition image into a cache queue;

[0019] When the number of first training samples in the cache queue reaches a preset number, all first training samples in the cache queue and second training samples related to each first training sample are used to perform incremental learning on the current recognition model to obtain a first incremental recognition model.

[0020] Optionally, using all the first training samples in the cache queue and second training samples related to each first training sample, incremental learning is performed on the current recognition model to obtain a first incremental recognition model, including:

[0021] Using a specified number of unlearned first training samples in the cache queue and each second training sample related to the first training sample, multiple rounds of incremental learning are performed on the current recognition model to obtain a first incremental recognition model, wherein each round of incremental learning uses a specified number of different first training data, and the specified number is less than the preset number.

[0022] Optionally, after determining the model performance of the first incremental recognition model based on the test result of the first incremental recognition model, the method further includes:

[0023] When the model performance of the first incremental recognition model is lower than the model performance of the current recognition model, a historical memory model is acquired, and the historical memory model is used as the current recognition model, wherein the model performance of the historical memory model is higher than the model performance of the first incremental recognition model;

[0024] Re-performing incremental learning on the current recognition model using the collected dish images to obtain a second incremental recognition model;

[0025] Determining a model performance of the second incremental recognition model based on a test result of the second incremental recognition model;

[0026] When the model performance of the second incremental recognition model is equal to or higher than the model performance of the current recognition model, the second incremental recognition model is saved as an updated historical memory model, and the target acquisition image is recognized using the second incremental recognition model to obtain a dish recognition result corresponding to the target acquisition image.

[0027] Optionally, after determining the model performance of the second incremental recognition model based on the test result of the second incremental recognition model, the method further includes:

[0028] When the model performance of the second incremental recognition model is lower than the model performance of the current recognition model, adjusting the model parameters of the current recognition model according to the test result of the second incremental recognition model to obtain an optimized recognition model;

[0029] Determining the model performance of the optimized recognition model based on the test results of the optimized recognition model;

[0030] When the model performance of the optimized recognition model is equal to or higher than the model performance of the current recognition model, the optimized recognition model is saved as an updated historical memory model, and the optimized recognition model is used to recognize the target acquisition image to obtain the dish recognition result corresponding to the target acquisition image.

[0031] In a second aspect, the present application provides a dish identification device, the device comprising:

[0032] A type determination module, configured to determine the dish type of the dish captured image according to a comparison result between a preset dish library and the dish captured image when the dish captured image is acquired, wherein the preset dish library includes captured images of multiple different known dishes;

[0033] An incremental learning module, configured to perform incremental learning on a current recognition model using the dish collection image to obtain a first incremental recognition model when the dish type of the dish collection image is an unknown dish;

[0034] a performance determination module, configured to determine a model performance of the first incremental recognition model based on a test result of the first incremental recognition model;

[0035] A dish recognition module is used to use the first incremental recognition model to recognize the target captured image when the model performance of the first incremental recognition model is equal to or higher than the model performance of the current recognition model, so as to obtain a dish recognition result corresponding to the target captured image, wherein the target captured image includes the dish captured image and a captured image whose similarity with the dish captured image reaches a specified similarity.

[0036] In a third aspect, the present application provides a kitchen device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned dish identification method when executing the computer program.

[0037] In a fourth aspect, the present application also provides a computer storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the above-mentioned dish identification method.

[0038] The above-mentioned technical scheme provided by the embodiment of the present application has the following advantages over the prior art: the method provided by the embodiment of the present application, when a dish capture image is acquired, determines the dish type of the dish capture image according to the comparison result between a preset dish library and the dish capture image, wherein the preset dish library includes capture images of multiple different known dishes; when the dish type of the dish capture image is an unknown dish, the dish capture image is used to perform incremental learning on the current recognition model to obtain a first incremental recognition model; the model performance of the first incremental recognition model is determined based on the test result of the first incremental recognition model; when the model performance of the first incremental recognition model is equal to or higher than the model performance of the current recognition model, the first incremental recognition model is used to recognize the target capture image to obtain a dish recognition result corresponding to the target capture image, wherein the target capture image includes the dish capture image and a capture image whose similarity with the dish capture image reaches a specified similarity.

[0039] Based on the above method, when faced with unknown dishes, the current recognition model is incrementally learned using the unknown dishes, so that the current recognition model can continuously absorb new information while maintaining existing knowledge, thereby continuously improving the recognition performance and generalization ability of the model. The incremental recognition model after incremental learning has the ability to recognize the unknown dishes it faces, thus solving the problem of decreased recognition accuracy of kitchen equipment when facing new categories of dishes. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0042] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0043] Figure 1 A schematic diagram of a method for identifying a dish provided in an embodiment of the present application;

[0044] Figure 2 A structural block diagram of a dish identification device provided in an embodiment of the present application;

[0045] Figure 3 A schematic diagram of the internal structure of a kitchen device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0047] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0048] In one embodiment, Figure 1 FIG. 1 is a flow chart of a dish identification method in an embodiment, referring to FIG. Figure 1 , a dish recognition method is provided, and the dish recognition method is applied to kitchen equipment, and the kitchen equipment can be any smart home appliance with a dish recognition function, such as a steamer, an oven, a steam oven, a refrigerator, an electric rice cooker, a cooking machine, etc. This embodiment is mainly illustrated by applying the method to a steam oven, that is, the kitchen equipment is taken as an example of a steam oven, and the dish recognition method specifically includes the following steps:

[0049] Step S210, when a dish capture image is acquired, the dish type of the dish capture image is determined according to a comparison result between a preset dish library and the dish capture image, wherein the preset dish library includes capture images of multiple different known dishes.

[0050] Specifically, a camera device is provided in the kitchen equipment for collecting images of dishes placed inside the kitchen equipment, that is, obtaining dish collection images. The preset dish library can be information stored locally in the kitchen equipment, or information obtained through the network. The preset dish library includes collection images of multiple different known dishes and dish information of each known dish. The collection images are used to display the appearance of the known dishes. The dish information is used to record the dish types, dish names, and training samples corresponding to the known dishes. The training samples are used to indicate the annotation boxes and dish labels of the known dishes in the collection images. The annotation boxes are used to frame the outlines of the known dishes in the collection images. The dish labels correspond to the dish names. The training samples are used to participate in the learning and training process of the dish recognition model. That is, the training samples corresponding to all dishes in the preset dish library have participated in the training process of the dish recognition model. Therefore, the dish recognition model built into the kitchen equipment can accurately recognize these dishes, so these dishes are regarded as known dishes corresponding to the dish recognition model.

[0051] When a dish collection image is detected, the dish collection image is first compared with the collection images of each dish in the preset dish library to determine whether there is a collection image in the preset dish library that is highly similar to the dish collection image. If so, it means that the dish collection image is a known dish corresponding to the dish recognition model, so the dish type of the dish collection image is determined to be a known dish; if not, it means that no collection image highly similar to the dish collection image is found in the preset dish library, and the dish collection image is determined to be an unknown dish that cannot be recognized by the dish recognition model, so the dish type of the dish collection image is determined to be an unknown dish. The dish collection image is an image that has been preprocessed, and the preprocessing specifically includes image denoising, scaling, normalization and other operations to improve image quality and recognition accuracy.

[0052] The steam oven is pre-installed with a basic dish recognition model when it leaves the factory. The model is trained based on a large number of labeled dish image data sets and has a certain generalization ability and recognition accuracy. After purchasing the steam oven, users can directly use the model for basic dish recognition functions without historical data. The steam oven uses a built-in camera to capture images of dishes in real time during the cooking process, and stores these image data together with the cooking information entered by the user (such as dish name, cooking time, etc.).

[0053] Step S220, when the dish type of the dish captured image is an unknown dish, incremental learning is performed on the current recognition model using the dish captured image to obtain a first incremental recognition model.

[0054] Specifically, when it is determined that the dish type of the dish captured image is an unknown dish, it means that the dish recognition model cannot recognize the dish captured image, and the dish recognition model needs to perform incremental learning on the unknown dish captured image to improve the recognition ability of the dish recognition model. Since the dish recognition model continuously uses the new dish captured images encountered for iterative learning, the version of the dish recognition model is dynamically increased over time. The current recognition model refers to the dish recognition model that has been iteratively learned to the current latest version. Since the current recognition model cannot recognize unknown dishes, the unknown dish captured images are used to perform incremental learning on the current recognition model, thereby obtaining an updated first incremental recognition model. This is equivalent to online learning of the current recognition model, and new dish captured images can be self-learned without suspending the normal operation of the kitchen equipment.

[0055] Using incremental learning technology, the steam oven can learn new dish data in real time and fine-tune the model without changing the original model structure to adapt to the recognition needs of new categories of dishes. Incremental learning allows the model to continuously absorb new information while maintaining existing knowledge, thereby continuously improving the model's recognition performance and generalization ability.

[0056] Step S230: determining the model performance of the first incremental recognition model based on the test result of the first incremental recognition model.

[0057] Specifically, after obtaining the first incremental recognition model, it is necessary to test it. Specifically, the test samples in the test set are substituted into the first incremental recognition model, and the recognition accuracy of the first incremental recognition model is determined according to the difference between the output result of the first incremental recognition model and the expected result, and the recognition accuracy is determined as the test result. The model performance of the first incremental recognition model is determined according to the model performance corresponding to the preset accuracy range to which the recognition accuracy belongs. For example, if the recognition accuracy is 91%, and the model performance corresponding to the preset accuracy range of 90% to 100% is the best performance, the model performance of the first incremental recognition model is the best at this time. That is, the recognition accuracy corresponds to the model performance. The higher the recognition accuracy, the better the model performance.

[0058] Step S240, when the model performance of the first incremental recognition model is equal to or higher than the model performance of the current recognition model, the target captured image is recognized using the first incremental recognition model to obtain a dish recognition result corresponding to the target captured image, wherein the target captured image includes the dish captured image and a captured image whose similarity to the dish captured image reaches a specified similarity.

[0059] Specifically, the model performance of the updated first incremental recognition model is compared with the model performance of the current recognition model. Only when the model performance of the first incremental recognition model at least reaches the model performance of the current recognition model, it is determined that the first incremental recognition model can be used as an updated version of the dish recognition model. At this time, the first incremental recognition model is used as the latest version of the current recognition model, and the target acquisition image is recognized and processed by the first incremental recognition model to obtain the dish recognition result corresponding to the target acquisition image. That is, the first incremental recognition model at this time has the function of recognizing the dishes and similar dishes in the dish acquisition image. The dish recognition result can be used to control the subsequent operating parameters of the kitchen equipment. For example, when the kitchen equipment is a steam oven, the dish recognition result is subsequently used to determine the corresponding cooking mode, that is, automatically adjust the cooking time, temperature, humidity and other parameters; when the kitchen equipment is a refrigerator, the dish recognition result is subsequently used to determine the corresponding preservation mode, that is, automatically adjust the compartment temperature, humidity, optimal storage time, etc.

[0060] Based on the above method, when faced with unknown dishes, the current recognition model is incrementally learned using the unknown dishes, so that the current recognition model can continuously absorb new information while maintaining existing knowledge, thereby continuously improving the recognition performance and generalization ability of the model. The incremental recognition model after incremental learning has the ability to recognize the unknown dishes it faces, thus solving the problem of decreased recognition accuracy of kitchen equipment when facing new categories of dishes.

[0061] In one embodiment, when the dish type in the dish captured image is an unknown dish, incremental learning is performed on the current recognition model using the dish captured image to obtain a first incremental recognition model, including:

[0062] When the dish type of the dish captured image is an unknown dish, outputting the dish captured image and the annotation prompt information corresponding to the dish captured image;

[0063] Upon receiving a labeling operation instruction for the dish captured image, constructing a first training sample corresponding to the dish captured image according to the labeling operation instruction, wherein the labeling operation instruction includes a plurality of labeling points and a labeling signature;

[0064] The first training sample corresponding to the captured dish image is used to perform incremental learning on the current recognition model to obtain a first incremental recognition model.

[0065] Specifically, when it is determined that the dish type of the dish collection image is an unknown dish, the dish collection image and the annotation prompt information are output. The annotation prompt information is used to remind the user to manually annotate the dish contour in the dish collection image that is unknown to the current recognition model, and prompt to annotate at least four contour points of the dish, which are the upper left, lower left, upper right and lower right. The more points the user manually annotates, the higher the recognition accuracy and efficiency of the dish in the dish collection image. That is, the user only needs to click on the four contour points of the new category of dishes (i.e., the four corner points of the upper left, upper right, lower right and lower left) through the touch screen of the steam oven or the supporting equipment, and the system can automatically generate the annotation box and construct the training sample in combination with the label name entered by the user. That is, the user only needs to click on the four contour points of the new category of dishes (i.e., the four corner points of the upper left, upper right, lower right and lower left) through the touch screen of the steam oven or the supporting equipment, and the system can automatically generate the annotation box and construct the training sample in combination with the label name entered by the user. This training method is not only simple and easy, but also can effectively improve the model's recognition ability for new categories of dishes.

[0066] The output dish collection image and annotation prompt information are displayed through a touch screen provided on the surface of the kitchen equipment. The touch screen also receives a user's click operation on the dish collection image. The click operation is used to generate an annotation operation instruction. The annotation operation instruction includes multiple annotation points and an annotation signature. The annotation point is the contour point marked by the user for the dish in the dish collection image. The annotation signature is the dish label input by the user. The first training sample corresponding to the dish collection image is constructed according to the multiple annotation points and the annotation signature in the annotation operation instruction. The first training sample corresponding to the dish collection image is used to perform incremental learning on the current recognition model, thereby obtaining a first incremental recognition model.

[0067] In one embodiment, incremental learning is performed on the current recognition model using the first training sample corresponding to the captured dish image to obtain a first incremental recognition model, including:

[0068] Acquire, from the preset dish library, similar images whose similarity to the dish captured image is greater than or equal to a preset similarity, and second training samples corresponding to the similar images;

[0069] Using the first training sample corresponding to the dish captured image and the second training sample corresponding to the similar image, incremental learning is performed on the current recognition model to obtain a first incremental recognition model.

[0070] Specifically, similar images whose similarity to the dish capture image reaches a preset similarity or above are searched in the preset dish library, that is, similar images related to the unknown dish capture image and the second training samples corresponding to the similar images are screened, and the second training samples of the known dishes that are most relevant to the dish capture image are used to perform incremental learning on the current recognition model together with the first training samples corresponding to the unknown dish capture image. In this way, the unknown dishes and related known dishes are combined for learning, which can improve the learning efficiency and recognition accuracy of unknown dishes.

[0071] In one embodiment, the method of performing incremental learning on the current recognition model using the first training sample corresponding to the dish captured image and the second training sample corresponding to the similar image to obtain the first incremental recognition model includes:

[0072] Cache the first training sample corresponding to the dish acquisition image into a cache queue;

[0073] When the number of first training samples in the cache queue reaches a preset number, all first training samples in the cache queue and second training samples related to each first training sample are used to perform incremental learning on the current recognition model to obtain a first incremental recognition model.

[0074] Specifically, after obtaining the first training sample corresponding to the dish captured image, the first training sample corresponding to the dish captured image is not immediately used to perform incremental learning on the current recognition model. Instead, the first training sample corresponding to the unknown dish captured image is cached in a cache queue, and the current recognition model is incrementally learned only when the number of first training samples corresponding to all unknown dishes in the cache queue reaches a preset number. This avoids the kitchen equipment performing incremental learning immediately every time it detects an unknown dish, and frequent learning leads to high energy consumption of the kitchen equipment.

[0075] Therefore, when an unknown dish is detected, its training data is cached first. When a certain amount of training data for new dishes is collected, the first training samples corresponding to all unknown dishes in the cache queue and the second training samples of known dishes related to each first training sample are uniformly used to perform incremental learning on the current recognition model, thereby reducing the frequency of incremental learning of the current recognition model, saving energy consumed by kitchen equipment for frequent incremental learning of dish recognition models, and ensuring the stability of kitchen equipment using dish recognition models to recognize dishes.

[0076] In one embodiment, all first training samples in the cache queue and second training samples related to each first training sample are used to perform incremental learning on the current recognition model to obtain a first incremental recognition model, including:

[0077] Using a specified number of unlearned first training samples in the cache queue and each second training sample related to the first training sample, multiple rounds of incremental learning are performed on the current recognition model to obtain a first incremental recognition model, wherein each round of incremental learning uses a specified number of different first training data, and the specified number is less than the preset number.

[0078] Specifically, when a certain number of training data for new dishes are collected, all the first training samples in the cache queue and the second training samples related to each first training data are not used at one time to perform incremental learning on the current recognition model. Instead, multiple first training samples in the cache queue are split into a specified number, and the current recognition model is incrementally learned once using the specified number of split first training samples and the second training samples related to the first training samples. In this way, multiple rounds of incremental learning are performed on the current recognition model, and the number of first training samples in each round of incremental learning is the specified number. The current recognition model is learned in batches to alleviate the learning pressure of the current recognition model at one time.

[0079] In one embodiment, after determining the model performance of the first incremental recognition model based on the test result of the first incremental recognition model, the method further includes:

[0080] When the model performance of the first incremental recognition model is lower than the model performance of the current recognition model, a historical memory model is acquired, and the historical memory model is used as the current recognition model, wherein the model performance of the historical memory model is higher than the model performance of the first incremental recognition model;

[0081] Re-performing incremental learning on the current recognition model using the collected dish images to obtain a second incremental recognition model;

[0082] Determining a model performance of the second incremental recognition model based on a test result of the second incremental recognition model;

[0083] When the model performance of the second incremental recognition model is equal to or higher than the model performance of the current recognition model, the second incremental recognition model is saved as an updated historical memory model, and the target acquisition image is recognized using the second incremental recognition model to obtain a dish recognition result corresponding to the target acquisition image.

[0084] Specifically, after incremental learning of the current recognition model, a first incremental recognition model is obtained. It is also necessary to determine whether the model performance of the first incremental recognition model has experienced performance regression. Therefore, it is determined whether the model performance of the first incremental recognition model is equal to or higher than the model performance of the current recognition model. If the model performance of the first incremental recognition model is lower than the model performance of the current recognition model, it means that the first incremental recognition model has experienced performance regression compared with the current recognition model. Performance regression will cause catastrophic forgetting, reduced recognition accuracy and other problems in the model. In order to ensure the stability of normal recognition of dishes by kitchen equipment, the first incremental recognition model with performance regression cannot be used. The memory point fallback mechanism is used to obtain the historical memory model. The historical memory model is the historical recognition model with the best performance. That is, each time a new model with better performance is obtained through iterative learning, it will be memorized to avoid catastrophic forgetting of the model due to performance regression, so as to forget all the data that has been learned.

[0085] The historical memory model is not only the best historical recognition model, but also has memory for previously learned data. Therefore, the historical memory model is used as the new current recognition model and incremental learning is performed to ensure that the performance of the dish recognition model is always maintained at a relatively good state. The second incremental recognition model is obtained after incremental learning based on the historical memory model. The model performance of the second incremental recognition model is also determined based on the test results of the second incremental recognition model. When its model performance at least reaches the model performance of the current recognition model, it is determined that the second incremental recognition model can be used normally and there is no performance regression. Therefore, the second incremental recognition model can be used to recognize and process the collected dish images to ensure the stable operation of the dish recognition model.

[0086] During the incremental learning process, the introduction of new data may cause the model performance to degrade. To solve this problem, a memory point fallback mechanism is introduced. When the model performance degrades after an update, the system will automatically fall back to the last memory point with better performance and re-perform incremental learning based on this memory point to ensure that the model performance always remains in a better state.

[0087] In one embodiment, after determining the model performance of the second incremental recognition model based on the test result of the second incremental recognition model, the method further includes:

[0088] When the model performance of the second incremental recognition model is lower than the model performance of the current recognition model, adjusting the model parameters of the current recognition model according to the test result of the second incremental recognition model to obtain an optimized recognition model;

[0089] Determining the model performance of the optimized recognition model based on the test results of the optimized recognition model;

[0090] When the model performance of the optimized recognition model is equal to or higher than the model performance of the current recognition model, the optimized recognition model is saved as an updated historical memory model, and the optimized recognition model is used to recognize the target acquisition image to obtain the dish recognition result corresponding to the target acquisition image.

[0091] Specifically, if the model performance of the second incremental recognition model is lower than the model performance of the current recognition model, it means that the second incremental recognition model has also experienced performance regression compared to the current recognition model. In this case, it is necessary to adjust the model parameters of the current recognition model according to the test results of the second incremental recognition model to obtain an optimized recognition model, and then determine its model performance according to the test results of the optimized recognition model. While ensuring that there is no performance regression in the model performance of the optimized recognition model, the optimized recognition model is saved as a new historical memory model and used to identify dishes.

[0092] The performance of the dish recognition model is also monitored and evaluated regularly, including indicators such as recognition accuracy and real-time performance. If the model performance is found to be degraded or unable to meet user needs, the model can be tuned, including adjusting model parameters, introducing new learning algorithms, or optimizing data preprocessing processes. At the same time, user feedback data can also be considered to guide the model tuning process to provide a smart cooking experience that better meets user needs.

[0093] Through the above-mentioned kitchen equipment control method, the kitchen equipment can continuously learn and optimize the dish recognition model during use, improve the recognition accuracy and generalization ability, and provide users with a more intelligent and convenient cooking experience.

[0094] Figure 1FIG. 1 is a flow chart of a method for identifying a dish in one embodiment. It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0095] In one embodiment, Figure 2 As shown, a dish identification device is provided, comprising:

[0096] A type determination module 310 is used to determine the dish type of the dish captured image according to the comparison result between a preset dish library and the dish captured image when the dish captured image is acquired, wherein the preset dish library includes captured images of multiple different known dishes;

[0097] An incremental learning module 320, configured to perform incremental learning on a current recognition model using the dish collection image to obtain a first incremental recognition model when the dish type of the dish collection image is an unknown dish;

[0098] A performance determination module 330, configured to determine the model performance of the first incremental recognition model based on the test result of the first incremental recognition model;

[0099] The dish recognition module 340 is used to use the first incremental recognition model to recognize the target captured image when the model performance of the first incremental recognition model is equal to or higher than the model performance of the current recognition model, so as to obtain the dish recognition result corresponding to the target captured image, wherein the target captured image includes the dish captured image and a captured image whose similarity with the dish captured image reaches a specified similarity.

[0100] In one embodiment, the incremental learning module 320 is further configured to:

[0101] When the dish type of the dish captured image is an unknown dish, outputting the dish captured image and the annotation prompt information corresponding to the dish captured image;

[0102] Upon receiving a labeling operation instruction for the dish captured image, constructing a first training sample corresponding to the dish captured image according to the labeling operation instruction, wherein the labeling operation instruction includes a plurality of labeling points and a labeling signature;

[0103] The first training sample corresponding to the captured dish image is used to perform incremental learning on the current recognition model to obtain a first incremental recognition model.

[0104] In one embodiment, the incremental learning module 320 is further used to:

[0105] Acquire, from the preset dish library, similar images whose similarity to the dish captured image is greater than or equal to a preset similarity, and second training samples corresponding to the similar images;

[0106] Using the first training sample corresponding to the dish captured image and the second training sample corresponding to the similar image, incremental learning is performed on the current recognition model to obtain a first incremental recognition model.

[0107] In one embodiment, the incremental learning module 320 is further configured to:

[0108] Cache the first training sample corresponding to the dish acquisition image into a cache queue;

[0109] When the number of first training samples in the cache queue reaches a preset number, all first training samples in the cache queue and second training samples related to each first training sample are used to perform incremental learning on the current recognition model to obtain a first incremental recognition model.

[0110] In one embodiment, the incremental learning module 320 is further configured to:

[0111] Using a specified number of unlearned first training samples in the cache queue and each second training sample related to the first training sample, multiple rounds of incremental learning are performed on the current recognition model to obtain a first incremental recognition model, wherein each round of incremental learning uses a specified number of different first training data, and the specified number is less than the preset number.

[0112] In one embodiment, the incremental learning module 320 is further configured to:

[0113] When the model performance of the first incremental recognition model is lower than the model performance of the current recognition model, a historical memory model is acquired, and the historical memory model is used as the current recognition model, wherein the model performance of the historical memory model is higher than the model performance of the first incremental recognition model;

[0114] Re-performing incremental learning on the current recognition model using the collected dish images to obtain a second incremental recognition model;

[0115] The performance determination module 330 is further configured to determine the model performance of the second incremental recognition model based on the test result of the second incremental recognition model;

[0116] The dish recognition module 340 is also used to save the second incremental recognition model as an updated historical memory model when the model performance of the second incremental recognition model is equal to or higher than the model performance of the current recognition model, and use the second incremental recognition model to recognize the target acquisition image to obtain the dish recognition result corresponding to the target acquisition image.

[0117] In one embodiment, the incremental learning module 320 is further configured to:

[0118] When the model performance of the second incremental recognition model is lower than the model performance of the current recognition model, adjusting the model parameters of the current recognition model according to the test result of the second incremental recognition model to obtain an optimized recognition model;

[0119] The performance determination module 330 is further used to determine the model performance of the optimized recognition model based on the test results of the optimized recognition model;

[0120] The dish recognition module 340 is also used to save the optimized recognition model as an updated historical memory model when the model performance of the optimized recognition model is equal to or higher than the model performance of the current recognition model, and use the optimized recognition model to recognize the target acquisition image to obtain the dish recognition result corresponding to the target acquisition image.

[0121] like Figure 3 As shown, an embodiment of the present application provides a kitchen device, including a processor 711, a communication interface 712, a memory 713 and a communication bus 714, wherein the processor 711, the communication interface 712, and the memory 713 communicate with each other through the communication bus 714;

[0122] Memory 713, used for storing computer programs;

[0123] The processor 711 is used to implement the dish identification method provided by any of the aforementioned method embodiments when executing the program stored in the memory 713.

[0124] Those skilled in the art will understand that Figure 3The structure shown in the figure is merely a block diagram of a partial structure related to the present application scheme, and does not constitute a limitation on the kitchen equipment to which the present application scheme is applied. The specific kitchen equipment may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0125] In one embodiment, the dish identification device provided in the present application can be implemented in the form of a computer program. The computer program can be Figure 3 The memory of the kitchen equipment can store various program modules constituting the dish recognition device, for example, Figure 2 The type determination module 310, the incremental learning module 320, the performance determination module 330 and the dish identification module 340 are shown. The computer program composed of various program modules enables the processor to execute the dish identification method of each embodiment of the present application described in this specification.

[0126] Figure 3 The kitchen equipment shown can be Figure 2 The type determination module 310 in the dish recognition device shown in the figure determines the dish type of the dish collection image according to the comparison result between the preset dish library and the dish collection image when the dish collection image is acquired, wherein the preset dish library includes collection images of multiple different known dishes. The kitchen device can use the incremental learning module 320 to perform incremental learning on the current recognition model using the dish collection image when the dish type of the dish collection image is an unknown dish to obtain a first incremental recognition model. The kitchen device can use the performance determination module 330 to determine the model performance of the first incremental recognition model based on the test result of the first incremental recognition model. The kitchen device can use the dish recognition module 340 to perform recognition of the target collection image using the first incremental recognition model when the model performance of the first incremental recognition model is equal to or higher than the model performance of the current recognition model to obtain a dish recognition result corresponding to the target collection image, wherein the target collection image includes the dish collection image and a collection image whose similarity with the dish collection image reaches a specified similarity.

[0127] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the dish identification method provided in any of the aforementioned method embodiments is implemented.

[0128] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0129] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, or of course by hardware. Based on this understanding, the above technical solution, in essence or in other words, the part that contributes to the relevant technology, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a kitchen device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0130] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative may be used.

[0131] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A dish identification method, characterized in that: The method comprises: When a dish collection image is acquired, the dish type of the dish collection image is determined according to a comparison result between a preset dish library and the dish collection image, wherein the preset dish library includes collection images of multiple different known dishes; When the dish type of the dish captured image is an unknown dish, using the dish captured image to perform incremental learning on the current recognition model to obtain a first incremental recognition model; determining a model performance of the first incremental recognition model based on a test result of the first incremental recognition model; When the model performance of the first incremental recognition model is equal to or higher than the model performance of the current recognition model, the first incremental recognition model is used to recognize the target captured image to obtain a dish recognition result corresponding to the target captured image, wherein the target captured image includes the dish captured image and a captured image whose similarity to the dish captured image reaches a specified similarity.

2. The method according to claim 1, characterized in that When the dish type in the dish collection image is an unknown dish, incremental learning is performed on the current recognition model using the dish collection image to obtain a first incremental recognition model, including: When the dish type of the dish captured image is an unknown dish, outputting the dish captured image and the annotation prompt information corresponding to the dish captured image; Upon receiving a labeling operation instruction for the dish captured image, constructing a first training sample corresponding to the dish captured image according to the labeling operation instruction, wherein the labeling operation instruction includes a plurality of labeling points and a labeling signature; The first training sample corresponding to the captured dish image is used to perform incremental learning on the current recognition model to obtain a first incremental recognition model.

3. The method according to claim 2, characterized in that Using the first training sample corresponding to the dish collection image to perform incremental learning on the current recognition model to obtain a first incremental recognition model, including: Acquire, from the preset dish library, similar images whose similarity to the dish captured image is greater than or equal to a preset similarity, and second training samples corresponding to the similar images; Using the first training sample corresponding to the dish captured image and the second training sample corresponding to the similar image, incremental learning is performed on the current recognition model to obtain a first incremental recognition model.

4. The method according to claim 3, characterized in that The method of using the first training sample corresponding to the dish captured image and the second training sample corresponding to the similar image to incrementally learn the current recognition model to obtain a first incremental recognition model includes: Cache the first training sample corresponding to the dish acquisition image into a cache queue; When the number of first training samples in the cache queue reaches a preset number, all first training samples in the cache queue and second training samples related to each first training sample are used to perform incremental learning on the current recognition model to obtain a first incremental recognition model.

5. The method according to claim 4, characterized in that Using all the first training samples in the cache queue and the second training samples related to each of the first training samples, incremental learning is performed on the current recognition model to obtain a first incremental recognition model, including: Using a specified number of unlearned first training samples in the cache queue and each second training sample related to the first training sample, multiple rounds of incremental learning are performed on the current recognition model to obtain a first incremental recognition model, wherein each round of incremental learning uses a specified number of different first training data, and the specified number is less than the preset number.

6. The method according to claim 1, characterized in that After determining the model performance of the first incremental recognition model based on the test result of the first incremental recognition model, the method further includes: When the model performance of the first incremental recognition model is lower than the model performance of the current recognition model, a historical memory model is acquired, and the historical memory model is used as the current recognition model, wherein the model performance of the historical memory model is higher than the model performance of the first incremental recognition model; Re-performing incremental learning on the current recognition model using the collected dish images to obtain a second incremental recognition model; Determining a model performance of the second incremental recognition model based on a test result of the second incremental recognition model; When the model performance of the second incremental recognition model is equal to or higher than the model performance of the current recognition model, the second incremental recognition model is saved as an updated historical memory model, and the target acquisition image is recognized using the second incremental recognition model to obtain a dish recognition result corresponding to the target acquisition image.

7. The method according to claim 6, characterized in that After determining the model performance of the second incremental recognition model based on the test result of the second incremental recognition model, the method further includes: When the model performance of the second incremental recognition model is lower than the model performance of the current recognition model, adjusting the model parameters of the current recognition model according to the test result of the second incremental recognition model to obtain an optimized recognition model; Determining the model performance of the optimized recognition model based on the test results of the optimized recognition model; When the model performance of the optimized recognition model is equal to or higher than the model performance of the current recognition model, the optimized recognition model is saved as an updated historical memory model, and the optimized recognition model is used to recognize the target acquisition image to obtain the dish recognition result corresponding to the target acquisition image.

8. A dish identification device, characterized in that: The device comprises: A type determination module, configured to determine the dish type of the dish captured image according to a comparison result between a preset dish library and the dish captured image when the dish captured image is acquired, wherein the preset dish library includes captured images of multiple different known dishes; An incremental learning module, configured to perform incremental learning on a current recognition model using the dish collection image to obtain a first incremental recognition model when the dish type of the dish collection image is an unknown dish; a performance determination module, configured to determine a model performance of the first incremental recognition model based on a test result of the first incremental recognition model; A dish recognition module is used to use the first incremental recognition model to recognize the target captured image when the model performance of the first incremental recognition model is equal to or higher than the model performance of the current recognition model, so as to obtain a dish recognition result corresponding to the target captured image, wherein the target captured image includes the dish captured image and a captured image whose similarity with the dish captured image reaches a specified similarity.

9. A kitchen device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images