Control method of a cooking navigation system

Abstract

The application discloses a control method of a cooking navigation system, which comprises the following steps: obtaining material image information of a cooking material in response to a cooking navigation starting instruction of a subject; performing semantic analysis on the material image information to obtain estimated recipe information; performing parameter estimation processing based on the estimated recipe information to obtain cooking parameter information; obtaining cooking image information of the cooking material, current working state information of a target cooking device corresponding to the cooking parameter information, and current cooking environment information of a cooking environment in the case that the subject cooks the cooking material based on the cooking parameter information; performing cooking state analysis based on the cooking image information, the current working state information, and the current cooking environment information to obtain cooking correction information, so that the subject cooks based on the cooking correction information. The cooking whole process is monitored, real-time interaction feedback with the subject is realized, the cooking process is optimized, and the problem of high cooking difficulty caused by the mode and single of cooking suggestions is solved.

Description

Technical Field

[0001] This application relates to the field of intelligent kitchenware technology, and in particular to a control method for a cooking navigation system. Background Technology

[0002] With the rapid development of artificial intelligence and Internet of Things technologies, new cooking methods established through intelligent technologies can achieve precise perception and intelligent control of the freshness and ripeness of ingredients as well as the cooking process, thereby improving the efficiency and quality of cooking and meeting users' personalized needs.

[0003] In existing technologies, cooking guidance devices are used to guide users' cooking behavior and recommend cooking methods based on preset programs or modes. However, because these devices often operate independently and lack the ability to interact with other kitchen equipment, they cannot automatically adjust recommended cooking parameters based on ingredient conditions or user preferences, and they lack oversight of the cooking process. Therefore, users often struggle to obtain real-time cooking suggestions when using these devices, especially when dealing with multiple ingredients and complex recipes, leading to significant cooking difficulties. Thus, there is an urgent need for a control method for a cooking navigation system that can reduce the difficulty of cooking and address these issues. Summary of the Invention

[0004] This application provides a control method for a cooking navigation system. Before cooking, the method identifies the dish to be cooked and, combined with professional cooking knowledge, assists the object in setting cooking parameters. During the cooking process, the method monitors the entire cooking process and interacts with the object in real time to provide feedback and optimize the cooking process. This solves the problem of high cooking difficulty caused by the standardized and simplistic cooking suggestions.

[0005] According to one aspect of this application, a control method for a cooking navigation system is proposed, the method comprising:

[0006] In response to the cooking navigation start command of the object, obtain the ingredient image information of the ingredients to be cooked;

[0007] Semantic analysis is performed on the food ingredient image information to obtain the predicted recipe information;

[0008] Based on the estimated recipe information, parameter estimation processing is performed to obtain cooking parameter information;

[0009] When the object cooks the food to be cooked based on the cooking parameter information, the cooking image information of the food to be cooked, the current working status information of the target cooking device corresponding to the cooking parameter information, and the current cooking environment information of the cooking environment are obtained.

[0010] Based on the cooking image information, the current working status information, and the current cooking environment information, cooking status analysis is performed to obtain cooking correction information, so that the object can make cooking corrections based on the cooking correction information.

[0011] In one possible implementation, the cooking assistance system includes an image acquisition module.

[0012] The process of acquiring the image information of the ingredients to be cooked includes:

[0013] The image acquisition module acquires images of the ingredients to be cooked, thereby obtaining image information of the ingredients.

[0014] In one possible implementation, the cooking navigation system includes a model invocation module that stores a pre-built target large-scale cooking model.

[0015] The step of performing semantic analysis on the food ingredient image information to obtain estimated recipe information includes:

[0016] The ingredient image information is input into the image recognition sub-model of the target cooking model, so that the image recognition sub-model performs semantic analysis on the ingredient image information to obtain the estimated recipe information.

[0017] In one possible implementation, the parameter estimation processing based on the estimated recipe information to obtain cooking parameter information includes:

[0018] Obtain the weight information of the ingredients to be cooked;

[0019] The estimated recipe information and the weight information of the ingredients to be cooked are input into the cooking parameter estimation sub-model of the target cooking model, so that the cooking parameter estimation sub-model performs parameter estimation processing based on the estimated recipe information and the weight information of the ingredients to be cooked to obtain the cooking parameter information.

[0020] In one possible implementation, the parameter estimation processing based on the estimated recipe information and the weight information of the ingredients to be cooked to obtain the cooking parameter information includes:

[0021] The preset cooking parameter library is queried based on the estimated recipe information and the weight information of the ingredients to be cooked to obtain the query results; the preset cooking parameter library includes multiple preset cooking parameters, which represent the cooking parameters of preset cooking equipment, and each preset cooking parameter is set based on the preset recipe and the weight of the preset ingredients;

[0022] If the query results meet the preset matching conditions, the preset cooking parameters corresponding to the query results that meet the preset matching conditions are determined as the cooking parameter information.

[0023] In one possible implementation, the cooking assistance system further includes a sensing module.

[0024] The acquisition of cooking image information of the food to be cooked, current working status information of the target cooking equipment corresponding to the cooking parameter information, and current cooking environment information of the cooking environment includes:

[0025] The image acquisition module acquires images of the ingredients to be cooked during cooking to obtain the cooking image information.

[0026] Based on the sensing module, the target cooking device is detected to obtain the current working status information; the current working status information represents the temperature information of the cooking device.

[0027] The cooking environment is detected by the sensing module to obtain the current cooking environment information; the current cooking environment information represents the gas detection information of the cooking environment.

[0028] In one possible implementation, the target cooking big model further includes a cooking state analysis sub-model.

[0029] The cooking state analysis based on the cooking image information, the current working state information, and the current cooking environment information to obtain cooking correction information includes:

[0030] The cooking image information, the current working status information, and the current cooking environment information are input into the cooking status analysis sub-model, so that the cooking status analysis sub-model performs cooking status analysis based on the cooking image information, the current working status information, and the current cooking environment information to obtain the cooking correction information.

[0031] In one possible implementation, after obtaining the cooking parameter information, the method further includes:

[0032] The prompt indicates that the object cooks based on the cooking parameter information and the ingredients to be cooked;

[0033] Accordingly,

[0034] After obtaining the cooking correction information, the method further includes:

[0035] The object is prompted to make cooking corrections based on the cooking correction information.

[0036] In one possible implementation, after the object completes cooking the ingredients to be cooked, the method further includes:

[0037] Based on the image acquisition module, images of the cooked ingredients are acquired to obtain images of the cooked dish.

[0038] Record the historical cooking parameters of the cooking equipment used in cooking;

[0039] Based on the cooked dish image and the historical cooking parameters, supplementary cooking parameters are generated;

[0040] The supplementary cooking parameters are entered into the preset cooking parameter library.

[0041] In one possible implementation, the system further includes a voice interaction module.

[0042] After obtaining the estimated recipe information, the method further includes:

[0043] The cooking voice signal of the object is obtained based on the voice interaction module;

[0044] The cooking voice signal is recognized to obtain the object's cooking instruction information;

[0045] The estimated recipe information is corrected based on the object cooking instruction information to obtain the target recipe information;

[0046] Accordingly,

[0047] The parameter estimation process based on the estimated recipe information to obtain cooking parameter information includes:

[0048] Based on the target recipe information, parameter estimation processing is performed to obtain the cooking parameter information.

[0049] On the other hand, a control device for a cooking navigation system is provided, the device comprising:

[0050] The instruction response module is used to respond to the cooking navigation start instruction of the object and obtain the ingredient image information of the ingredients to be cooked;

[0051] The semantic analysis module is used to perform semantic analysis on the food image information to obtain the estimated recipe information;

[0052] The parameter estimation module is used to perform parameter estimation processing based on the estimated recipe information to obtain cooking parameter information;

[0053] The cooking information acquisition module is used to acquire cooking image information of the food to be cooked, current working status information of the target cooking device corresponding to the cooking parameter information, and current cooking environment information of the cooking environment when the object cooks the food to be cooked based on the cooking parameter information.

[0054] The cooking correction information determination module is used to analyze the cooking state based on the cooking image information, the current working state information, and the current cooking environment information to obtain cooking correction information, so that the object can make cooking corrections based on the cooking correction information.

[0055] In one possible implementation, the cooking assistance system includes an image acquisition module.

[0056] The instruction response module is used for:

[0057] The image acquisition module acquires images of the ingredients to be cooked, thereby obtaining image information of the ingredients.

[0058] In one possible implementation, the cooking navigation system includes a model invocation module that stores a pre-built target large-scale cooking model.

[0059] The semantic analysis module is used for:

[0060] The ingredient image information is input into the image recognition sub-model of the target cooking model, so that the image recognition sub-model performs semantic analysis on the ingredient image information to obtain the estimated recipe information.

[0061] In one possible implementation, the parameter estimation module is used for:

[0062] Obtain the weight information of the ingredients to be cooked;

[0063] The estimated recipe information and the weight information of the ingredients to be cooked are input into the cooking parameter estimation sub-model of the target cooking model, so that the cooking parameter estimation sub-model performs parameter estimation processing based on the estimated recipe information and the weight information of the ingredients to be cooked to obtain the cooking parameter information.

[0064] In one possible implementation, the parameter estimation module is further configured to:

[0065] The preset cooking parameter library is queried based on the estimated recipe information and the weight information of the ingredients to be cooked to obtain the query results; the preset cooking parameter library includes multiple preset cooking parameters, which represent the cooking parameters of preset cooking equipment, and each preset cooking parameter is set based on the preset recipe and the weight of the preset ingredients;

[0066] If the query results meet the preset matching conditions, the preset cooking parameters corresponding to the query results that meet the preset matching conditions are determined as the cooking parameter information.

[0067] In one possible implementation, the cooking assistance system further includes a sensing module.

[0068] The cooking information acquisition module is used for:

[0069] The image acquisition module acquires images of the ingredients to be cooked during cooking to obtain the cooking image information.

[0070] Based on the sensing module, the target cooking device is detected to obtain the current working status information; the current working status information represents the temperature information of the cooking device.

[0071] The cooking environment is detected by the sensing module to obtain the current cooking environment information; the current cooking environment information represents the gas detection information of the cooking environment.

[0072] In one possible implementation, the target cooking big model further includes a cooking state analysis sub-model.

[0073] The cooking correction information determination module is used for:

[0074] The cooking image information, the current working status information, and the current cooking environment information are input into the cooking status analysis sub-model, so that the cooking status analysis sub-model performs cooking status analysis based on the cooking image information, the current working status information, and the current cooking environment information to obtain the cooking correction information.

[0075] In one possible implementation, the device further includes a prompting module.

[0076] After obtaining the cooking parameter information, the prompting module is used to:

[0077] The prompt indicates that the object cooks based on the cooking parameter information and the ingredients to be cooked;

[0078] Accordingly,

[0079] After receiving the cooking correction information, the prompting module is further configured to:

[0080] The object is prompted to make cooking corrections based on the cooking correction information.

[0081] In one possible implementation, the device further includes a recording module.

[0082] After the object completes the cooking of the ingredients to be cooked, the recording module is used to:

[0083] Based on the image acquisition module, images of the cooked ingredients are acquired to obtain images of the cooked dish.

[0084] Record the historical cooking parameters of the cooking equipment used in cooking;

[0085] Based on the cooked dish image and the historical cooking parameters, supplementary cooking parameters are generated;

[0086] The supplementary cooking parameters are entered into the preset cooking parameter library.

[0087] In one possible implementation, the system further includes a voice interaction module, and the device further includes a voice control module.

[0088] After obtaining the estimated recipe information, the voice control module is used to:

[0089] The cooking voice signal of the object is obtained based on the voice interaction module;

[0090] The cooking voice signal is recognized to obtain the object's cooking instruction information;

[0091] The estimated recipe information is corrected based on the object cooking instruction information to obtain the target recipe information;

[0092] Accordingly,

[0093] The parameter estimation module is also used for:

[0094] Based on the target recipe information, parameter estimation processing is performed to obtain the cooking parameter information.

[0095] On the other hand, an electronic device is provided, including a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement a control method for a cooking navigation system according to any of the above aspects.

[0096] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored therein, the at least one instruction or the at least one program being loaded and executed by a processor to implement the control method of the cooking navigation system as described above.

[0097] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the control method of the cooking navigation system according to any of the above aspects.

[0098] This application embodiment obtains the image information of the ingredients to be cooked in response to the cooking navigation start command of the object; performs semantic analysis on the ingredient image information to obtain estimated recipe information; performs parameter estimation processing based on the estimated recipe information to obtain cooking parameter information; when the object cooks the ingredients to be cooked based on the cooking parameter information, it obtains the cooking image information of the ingredients to be cooked, the current working status information of the target cooking equipment corresponding to the cooking parameter information, and the current cooking environment information of the cooking environment; performs cooking status analysis based on the cooking image information, the current working status information, and the current cooking environment information to obtain cooking correction information, so that the object can make cooking corrections based on the cooking correction information. By identifying the dish to be cooked before cooking, combining professional cooking knowledge, assisting the object in setting cooking parameters, monitoring the entire cooking process during cooking, and interacting with the object in real time to optimize the cooking process, this solves the problem of high cooking difficulty caused by the standardized and simplistic cooking suggestions. Attached Figure Description

[0099] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0100] Figure 1 This is a flowchart illustrating the control method of the cooking navigation system provided in an embodiment of this application;

[0101] Figure 2 This is a schematic diagram of the structure of the cooking navigation system provided in the embodiments of this application;

[0102] Figure 3 This is a flowchart illustrating the parameter estimation processing method provided in the embodiments of this application;

[0103] Figure 4 This is a schematic flowchart of the cooking state analysis method provided in the embodiments of this application;

[0104] Figure 5 This is a flowchart illustrating the method for recording cooking history data provided in an embodiment of this application;

[0105] Figure 6 This is a schematic diagram of the control device of the cooking navigation system provided in the embodiments of this application;

[0106] In the figure, the corresponding labels are: 1-image acquisition module, 2-model calling module, 3-weighing module, 4-voice interaction module, and 5-perception module. Detailed Implementation

[0107] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0108] Existing cooking navigation methods typically involve users manually selecting cooking parameters (mode, time) before cooking to customize the process or choosing a smart recipe to start one-click cooking. During cooking, the cooking status is displayed on the cooking device screen or an app, and a notification is given upon completion. This method requires users to manually select and set the cooking process, which can be difficult for inexperienced users. It also lacks real-time monitoring and automatic optimization of the entire cooking process from the user's perspective, as well as feedback recording and proactive interaction with the user.

[0109] This application provides a control method for a cooking navigation system, which is set up in a cooking environment. Before cooking, the system identifies the dish to be cooked, combines this with professional culinary knowledge to assist the user in setting cooking parameters, monitors the entire cooking process during cooking, and provides real-time interactive feedback to optimize the cooking process and reduce cooking difficulty. See also... Figure 1 The control method of the cooking navigation system includes steps S101 to S109.

[0110] In step S101, in response to the object's cooking navigation start command, the ingredient image information of the ingredients to be cooked is obtained.

[0111] In one possible implementation, the cooking assistance system includes an image acquisition module.

[0112] The process of acquiring the image information of the ingredients to be cooked includes:

[0113] The image acquisition module acquires images of the ingredients to be cooked, thereby obtaining image information of the ingredients.

[0114] In one possible implementation, the ingredients to be cooked are cooking ingredients prepared by the user.

[0115] In one possible implementation, such as Figure 2 As shown, the cooking assistance system includes an image acquisition module, which can be a camera, used to acquire images of the ingredients to be cooked.

[0116] In one possible implementation, the image acquisition module may include an augmented reality module, which is AR (Augmented Reality) glasses or a smart screen with AR functionality.

[0117] In one possible implementation, the image acquisition module acquires images of the cooking ingredients prepared by the user to obtain image information of the ingredients.

[0118] In one possible implementation, the image information of the ingredients to be cooked can also be obtained by the image acquisition module from the recipe image prepared by the user. Specifically, the recipe image prepared by the user can be a cookbook or a recipe image displayed on the screen.

[0119] In this implementation, the image acquisition module acquires images of the ingredients to be cooked, obtaining the image information of the ingredients. This facilitates the subsequent deduction of the recipe the user wants to cook based on the ingredients, thus improving the user experience.

[0120] In step S103, semantic analysis is performed on the food ingredient image information to obtain the estimated recipe information.

[0121] In one possible implementation, the cooking navigation system includes a model invocation module that stores a pre-built target large-scale cooking model.

[0122] The step of performing semantic analysis on the food ingredient image information to obtain estimated recipe information includes:

[0123] The ingredient image information is input into the image recognition sub-model of the target cooking model, so that the image recognition sub-model performs semantic analysis on the ingredient image information to obtain the estimated recipe information.

[0124] In one possible implementation, such as Figure 2 As shown, the cooking assistance system also includes a model invocation module, which stores a pre-built target cooking model.

[0125] In one possible implementation, the pre-built target cooking big model is fine-tuned based on a general multimodal generative big model and cooking-related data. Specifically, the general multimodal generative big model can be Qwen-VL, a large-scale visual language model with image and text recognition, description, question answering, and dialogue capabilities; or it can be LLaVA (Large Language and Vision Assistant), a large-scale language and vision assistant, which is an end-to-end trained large-scale multimodal model that connects the visual encoder and the large language model to achieve general visual and language understanding.

[0126] In one possible implementation, the general multimodal generative large model includes an image recognition sub-model, a cooking parameter estimation sub-model, and a speech recognition sub-model.

[0127] In one possible implementation, cooking-related data includes text sample data and image sample data.

[0128] In one possible implementation, the text sample data is obtained by collecting text data from food websites, cooking forums, cookbooks, and self-developed recipes, ingredients, and cooking knowledge on the Internet. This includes text information such as dish names, ingredient lists, cooking steps, cooking techniques, flavor descriptions, and cooking modes, as well as collected user comments and questions about the dishes.

[0129] In one possible implementation, the image sample data is obtained by collecting image information about recipes, including photos of ingredients, videos of the recipe cooking process, videos of ingredient processing, etc.

[0130] In one possible implementation, scene processing is performed on cooking-related data to obtain user dialogue sample data, cooking image sample data, and cooking parameter sample data.

[0131] In one possible implementation, the user dialogue sample data is obtained by constructing question-answer pairs for cooking scenarios, such as: Question: What ingredients are needed for steamed sea bass? Answer: Steamed sea bass requires…; and question-intent discriminative data, such as: Question: Let's start cooking steamed sea bass? Answer: Start cooking the recipe. The user dialogue sample data is used to train a general multimodal generative large model to improve the understanding of user dialogue and enhance response alignment.

[0132] In one possible implementation, the acquisition of the cooking image sample data involves processing it to obtain ingredient image-text description data, such as name, quantity, and freshness; and image-cooking state description during the recipe's cooking process, such as whether the food is raw / cooked, its color, and its shape. This cooking image sample data is used to train a general multimodal generative model to improve the ability to recognize and understand cooking objects and cooking states.

[0133] In one possible implementation, the acquisition of cooking parameter sample data includes recipe image-cooking curve data, recipe image-operator action data, etc. The cooking parameter sample data is used to train a general multimodal generative large model to improve the reliability of recommendations for equipment and environmental adjustments during the cooking process.

[0134] In one possible implementation, the image recognition sub-module of the general multimodal generative large model is fine-tuned using the user dialogue sample data and the SFT method to obtain the speech recognition sub-model of the target cooking large model.

[0135] In one possible implementation, the image recognition sub-module of the general multimodal generative large model is fine-tuned using the cooking image sample data and the LoRA method to obtain the image recognition sub-model of the target cooking large model.

[0136] In one possible implementation, the cooking parameter estimation sub-model of the general multimodal generative large model is fine-tuned using VPiT (VisionPrediction Instruction Tuning, a fine-tuning method combining vision and instruction adjustment) based on the cooking parameter sample data, thereby obtaining the cooking parameter estimation sub-model of the target cooking large model. Specifically, VPiT is particularly suitable for tasks that require understanding and predicting changes in actions or states in a visual scene.

[0137] In this implementation, based on the image semantic understanding capability of the target cooking model, the ingredients or text descriptions in the ingredient images are identified, and the recipe that the user wants to cook is inferred, thus improving the efficiency of recipe determination.

[0138] In step S105, parameter estimation processing is performed based on the estimated recipe information to obtain cooking parameter information.

[0139] In one possible implementation, the parameter estimation processing based on the estimated recipe information to obtain cooking parameter information includes:

[0140] Obtain the weight information of the ingredients to be cooked;

[0141] The estimated recipe information and the weight information of the ingredients to be cooked are input into the cooking parameter estimation sub-model of the target cooking model, so that the cooking parameter estimation sub-model performs parameter estimation processing based on the estimated recipe information and the weight information of the ingredients to be cooked to obtain the cooking parameter information.

[0142] In one possible implementation, the parameter estimation processing based on the estimated recipe information and the weight information of the ingredients to be cooked to obtain the cooking parameter information includes:

[0143] The preset cooking parameter library is queried based on the estimated recipe information and the weight information of the ingredients to be cooked to obtain the query results; the preset cooking parameter library includes multiple preset cooking parameters, which represent the cooking parameters of preset cooking equipment, and each preset cooking parameter is set based on the preset recipe and the weight of the preset ingredients;

[0144] If the query results meet the preset matching conditions, the preset cooking parameters corresponding to the query results that meet the preset matching conditions are determined as the cooking parameter information.

[0145] In one possible implementation, such as Figure 2 As shown, the system also includes a weighing module, which is used to obtain the weight information of the ingredients to be cooked.

[0146] In one possible implementation, the estimated recipe information and the weight information of the ingredients to be cooked are input into the cooking parameter estimation sub-model of the target cooking model, and the cooking parameter information is output. Specifically, the cooking parameter information includes the recipe name description, ingredient list, cooking equipment, cooking method, cooking parameters, cooking curve, etc., corresponding to the estimated recipe information. Specifically, the cooking parameters include cooking mode, duration, temperature, etc., and the cooking curve is a control curve in the specific cooking process, used to continuously output parameter information for the corresponding time.

[0147] In one possible implementation, the preset cooking parameter library includes introductions to professional cooking techniques, equipment user manuals, recipes, etc. The cooking parameter estimation sub-model queries the preset cooking parameter library using the Prompt command to obtain the cooking parameter information. Specifically, the preset cooking parameter library also includes a user memory vector library, which contains cooking parameter information recorded by the user after each cooking session. Specifically, in the large AI model, the Prompt command primarily serves to provide the AI ​​model with contextual information about the input and the parameter information input to the model.

[0148] In this implementation, a target cooking model that is more suitable for cooking scenarios is obtained by fine-tuning the general large model, which reduces the model training cost. Based on the target cooking model, the parameters of cooking recipes and ingredients are estimated to obtain cooking parameter information, which improves the accuracy and efficiency of cooking parameter recommendations and reduces the cooking difficulty for users.

[0149] In one possible implementation, the system further includes a voice interaction module.

[0150] After obtaining the estimated recipe information, the method further includes:

[0151] The cooking voice signal of the object is obtained based on the voice interaction module;

[0152] The cooking voice signal is recognized to obtain the object's cooking instruction information;

[0153] The estimated recipe information is corrected based on the object cooking instruction information to obtain the target recipe information;

[0154] Accordingly,

[0155] The parameter estimation process based on the estimated recipe information to obtain cooking parameter information includes:

[0156] Based on the target recipe information, parameter estimation processing is performed to obtain the cooking parameter information.

[0157] In one possible implementation, such as Figure 2 As shown, the system also includes a voice interaction module, through which user voice commands are received, such as: "I'm going to make a steamed sea bass."

[0158] In one possible implementation, the target cooking model further includes a speech recognition sub-model. The speech recognition sub-model analyzes the user's speech commands based on its semantic understanding capabilities to obtain speech command recognition results. If the speech command recognition results do not match the estimated recipe information, the speech command recognition results are determined to be the target recipe information. Based on the target recipe information, parameter estimation processing is performed to obtain the cooking parameter information.

[0159] In one possible implementation, such as Figure 3 As shown, the image information of the ingredients collected by the image acquisition module, the weight information of the ingredients to be cooked obtained by the weighing module, and the user's voice commands received by the voice interaction module are input into the target cooking model, so that the target cooking model can recommend cooking equipment and parameters based on the user memory vector library, generate the cooking parameter information, and output it to the user.

[0160] In this implementation, the recipe information is further confirmed based on the voice interaction module, which improves the accuracy of cooking suggestions. The voice interaction module also facilitates the capture of users' real-time needs and improves interactivity.

[0161] In step S107, when the object cooks the food to be cooked based on the cooking parameter information, the cooking image information of the food to be cooked, the current working status information of the target cooking device corresponding to the cooking parameter information, and the current cooking environment information of the cooking environment are obtained.

[0162] In one possible implementation, the cooking assistance system further includes a sensing module.

[0163] The acquisition of cooking image information of the food to be cooked, current working status information of the target cooking equipment corresponding to the cooking parameter information, and current cooking environment information of the cooking environment includes:

[0164] The image acquisition module acquires images of the ingredients to be cooked during cooking to obtain the cooking image information.

[0165] Based on the sensing module, the target cooking device is detected to obtain the current working status information; the current working status information represents the temperature information of the cooking device.

[0166] The cooking environment is detected by the sensing module to obtain the current cooking environment information; the current cooking environment information represents the gas detection information of the cooking environment.

[0167] In one possible implementation, the sensing module identifies the cooking status of the cooking equipment as the current operating status information. Specifically, the current operating status information may include information such as the oven's temperature and humidity, the temperature of the pot bottom on the stove, and the temperature of the stovetop glass.

[0168] In one possible implementation, the current cooking environment information is obtained through the sensing module. Specifically, the current cooking environment information may be information such as PM2.5 concentration (characterizing air quality) and carbon monoxide concentration.

[0169] In one possible implementation, the image acquisition module acquires cooking image information from the user's perspective. Specifically, the cooking image information may include images of cooking fumes, flames, and the food being cooked.

[0170] In this implementation, the cooking process is detected from all angles by the image acquisition module and the perception module to obtain real-time information about the cooking process. This allows for real-time cooking suggestions based on information about the ingredients, environment, and equipment, helping users to accurately control the cooking process and cook delicious dishes.

[0171] In step S109, cooking state analysis is performed based on the cooking image information, the current working state information, and the current cooking environment information to obtain cooking correction information, so that the object can make cooking corrections based on the cooking correction information.

[0172] In one possible implementation, the target cooking big model further includes a cooking state analysis sub-model.

[0173] The cooking state analysis based on the cooking image information, the current working state information, and the current cooking environment information to obtain cooking correction information includes:

[0174] The cooking image information, the current working status information, and the current cooking environment information are input into the cooking status analysis sub-model, so that the cooking status analysis sub-model performs cooking status analysis based on the cooking image information, the current working status information, and the current cooking environment information to obtain the cooking correction information.

[0175] In one possible implementation, after obtaining the cooking parameter information, the method further includes:

[0176] The prompt indicates that the object cooks based on the cooking parameter information and the ingredients to be cooked;

[0177] Accordingly,

[0178] After obtaining the cooking correction information, the method further includes:

[0179] The object is prompted to make cooking corrections based on the cooking correction information.

[0180] In one possible implementation, such as Figure 4 As shown, the cooking image information collected by the image acquisition module, the current cooking environment information detected by the perception module, and the user voice commands received by the voice interaction module are input into the target cooking model so that the target cooking model can perform cooking state analysis and suggestions, obtain cooking correction information, and output it to the user.

[0181] In one possible implementation, the cooking image information, current working status information, and current cooking environment information monitored in real time during the cooking process are input into the target cooking model. Then, through RAG (Retrieval-Enhanced Generation) technology, it connects to a cooking knowledge base (such as equipment user manuals, kitchen safety knowledge, and knowledge of recipe cooking process changes). The target cooking model uses Prompt commands to comprehensively analyze the operating status of the cooking equipment, the safety of the cooking environment, the status of the cooking recipe, and user requirements to draw conclusions and make suggestions. The model interacts with the user through a screen or voice interaction module, providing status descriptions, abnormal information, and action suggestions. After user confirmation, the corresponding cooking equipment is controlled.

[0182] In this implementation, prompts are given to the user during the cooking process, allowing the user to control the cooking equipment accordingly based on the suggestions. Real-time interaction and feedback with the user optimize the cooking process and achieve better cooking results.

[0183] In one possible implementation, after the object completes cooking the ingredients to be cooked, the method further includes:

[0184] The image acquisition module acquires images of the cooked ingredients to be cooked, thereby obtaining images of the cooked dish.

[0185] Record the historical cooking parameters of the cooking equipment used in cooking;

[0186] Based on the cooked dish image and the historical cooking parameters, supplementary cooking parameters are generated;

[0187] The supplementary cooking parameters are entered into the preset cooking parameter library.

[0188] In one possible implementation, user feedback is collected via a voice interaction module, and supplementary cooking parameters are generated based on the cooked dish image, the historical cooking parameters, and the feedback information.

[0189] In one possible implementation, such as Figure 5As shown, after the user confirms the completion of cooking through the voice interaction module or manual operation, the perception module acquires cooking process information (such as cooking time, cooking parameter curves, etc.); the image acquisition module acquires cooking image information from the user's perspective; the voice interaction module or screen interaction collects user cooking evaluations, such as scores or descriptions of taste and texture; after transmitting the above information to the target cooking model, the Prompt command allows the target cooking model to analyze and summarize the semantic features of the entire cooking process (such as the name of the dish, whether cooking time was extended, whether the recipe was burnt, the taste of the dish, the saltiness of the dish, etc.), and stores it in the user's cooking memory vector library through text vectorization, as a supplementary vector library for subsequent large model analysis of user cooking recipes and recommendations for cooking parameters, gradually aligning with user preferences and habits.

[0190] In this implementation, cooking result information is obtained through the perception module and image acquisition module. After analysis and summarization by the target cooking big model, user cooking evaluations are collected to form a user memory vector library, which serves as a supplementary memory library for subsequent multimodal cooking big models, making the recommendations of the cooking navigation system more closely aligned with user habits.

[0191] The above embodiments of this application have the following beneficial effects: This application obtains the image information of the ingredients to be cooked in response to the cooking navigation start command of the object; performs semantic analysis on the ingredient image information to obtain estimated recipe information; performs parameter estimation processing based on the estimated recipe information to obtain cooking parameter information; when the object cooks the ingredients to be cooked based on the cooking parameter information, it obtains the cooking image information of the ingredients to be cooked, the current working status information of the target cooking equipment corresponding to the cooking parameter information, and the current cooking environment information of the cooking environment; performs cooking status analysis based on the cooking image information, the current working status information, and the current cooking environment information to obtain cooking correction information, so that the object can make cooking corrections based on the cooking correction information. By identifying the dish to be cooked before cooking, combining professional cooking knowledge, assisting the object in setting cooking parameters, monitoring the entire cooking process during cooking, and interacting with the object in real time to optimize the cooking process, this application solves the problem of high cooking difficulty caused by the standardized and simplistic cooking suggestions.

[0192] like Figure 6 As shown in the embodiments, this application also discloses a control device 600 for a cooking navigation system, used to implement the control method of the cooking navigation system described in any of the above embodiments. The cooking navigation system is set in a cooking environment, and the device includes:

[0193] The instruction response module 601 is used to obtain the image information of the ingredients to be cooked in response to the cooking navigation start instruction of the object.

[0194] The semantic analysis module 602 is used to perform semantic analysis on the food image information to obtain the estimated recipe information;

[0195] The parameter estimation module 603 is used to perform parameter estimation processing based on the estimated recipe information to obtain cooking parameter information;

[0196] The cooking information acquisition module 604 is used to acquire cooking image information of the food to be cooked, current working status information of the target cooking device corresponding to the cooking parameter information, and current cooking environment information of the cooking environment when the object cooks the food to be cooked based on the cooking parameter information.

[0197] The cooking correction information determination module 605 is used to perform cooking state analysis based on the cooking image information, the current working state information and the current cooking environment information to obtain cooking correction information, so that the object can perform cooking correction based on the cooking correction information.

[0198] In one possible implementation, the cooking assistance system includes an image acquisition module.

[0199] The instruction response module 601 is used for:

[0200] The image acquisition module acquires images of the ingredients to be cooked, thereby obtaining image information of the ingredients.

[0201] In one possible implementation, the cooking navigation system includes a model invocation module that stores a pre-built target large-scale cooking model.

[0202] The semantic analysis module 602 is used for:

[0203] The ingredient image information is input into the image recognition sub-model of the target cooking model, so that the image recognition sub-model performs semantic analysis on the ingredient image information to obtain the estimated recipe information.

[0204] In one possible implementation, the parameter estimation module 603 is used to:

[0205] Obtain the weight information of the ingredients to be cooked;

[0206] The estimated recipe information and the weight information of the ingredients to be cooked are input into the cooking parameter estimation sub-model of the target cooking model, so that the cooking parameter estimation sub-model performs parameter estimation processing based on the estimated recipe information and the weight information of the ingredients to be cooked to obtain the cooking parameter information.

[0207] In one possible implementation, the parameter estimation module 603 is further configured to:

[0208] The preset cooking parameter library is queried based on the estimated recipe information and the weight information of the ingredients to be cooked to obtain the query results; the preset cooking parameter library includes multiple preset cooking parameters, which represent the cooking parameters of preset cooking equipment, and each preset cooking parameter is set based on the preset recipe and the weight of the preset ingredients;

[0209] If the query results meet the preset matching conditions, the preset cooking parameters corresponding to the query results that meet the preset matching conditions are determined as the cooking parameter information.

[0210] In one possible implementation, the cooking assistance system further includes a sensing module.

[0211] The cooking information acquisition module 604 is used for:

[0212] The image acquisition module acquires images of the ingredients to be cooked during cooking to obtain the cooking image information.

[0213] Based on the sensing module, the target cooking device is detected to obtain the current working status information; the current working status information represents the temperature information of the cooking device.

[0214] The cooking environment is detected by the sensing module to obtain the current cooking environment information; the current cooking environment information represents the gas detection information of the cooking environment.

[0215] In one possible implementation, the target cooking big model also includes a cooking state analysis sub-model.

[0216] The cooking correction information determination module 605 is used for:

[0217] The cooking image information, the current working status information, and the current cooking environment information are input into the cooking status analysis sub-model, so that the cooking status analysis sub-model performs cooking status analysis based on the cooking image information, the current working status information, and the current cooking environment information to obtain the cooking correction information.

[0218] In one possible implementation, the device further includes a prompting module 606.

[0219] After obtaining the cooking parameter information, the prompting module 606 is used to:

[0220] The prompt indicates that the object cooks based on the cooking parameter information and the ingredients to be cooked;

[0221] Accordingly,

[0222] After receiving the cooking correction information, the prompting module 606 is further configured to:

[0223] The object is prompted to make cooking corrections based on the cooking correction information.

[0224] In one possible implementation, the device further includes a recording module 607.

[0225] After the object completes the cooking of the ingredients to be cooked, the recording module 607 is used to:

[0226] The image acquisition module acquires images of the cooked ingredients to be cooked, thereby obtaining images of the cooked dish.

[0227] Record the historical cooking parameters of the cooking equipment used in cooking;

[0228] Based on the cooked dish image and the historical cooking parameters, supplementary cooking parameters are generated;

[0229] The supplementary cooking parameters are entered into the preset cooking parameter library.

[0230] In one possible implementation, the system further includes a voice interaction module, and the device further includes a voice control module 608.

[0231] After obtaining the estimated recipe information, the voice control module 608 is used for:

[0232] The cooking voice signal of the object is obtained based on the voice interaction module;

[0233] The cooking voice signal is recognized to obtain the object's cooking instruction information;

[0234] The estimated recipe information is corrected based on the object cooking instruction information to obtain the target recipe information;

[0235] Accordingly,

[0236] The parameter estimation module 603 is also used for:

[0237] Based on the target recipe information, parameter estimation processing is performed to obtain the cooking parameter information.

[0238] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0239] This application provides an electronic device including a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement any of the cooking navigation system control methods provided in the above method embodiments.

[0240] Memory can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. Memory can primarily include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for the functions, etc.; the data storage area can store data created based on the use of the device, etc. Furthermore, memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory can also include a memory controller to provide the processor with access to the memory.

[0241] Embodiments of this application also provide a computer-readable storage medium, which can be disposed in an electronic device to store at least one instruction or at least one program related to implementing a control method for a cooking navigation system. The at least one instruction or the at least one program is loaded and executed by the processor to implement any of the control methods for a cooking navigation system provided in the above-described method embodiments.

[0242] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0243] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0244] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0245] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0246] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A control method for a cooking navigation system, characterized in that, The cooking navigation system is set up in the cooking environment. The cooking navigation system includes a model invocation module, which stores a pre-constructed target cooking model. The target cooking model includes an image recognition sub-model, a cooking parameter estimation sub-model, and a cooking state analysis sub-model. The method includes: In response to the object's cooking navigation start command, obtain the ingredient image information of the ingredients to be cooked; Semantic analysis is performed on the ingredient image information to obtain estimated recipe information; the step of performing semantic analysis on the ingredient image information to obtain estimated recipe information includes: inputting the ingredient image information into the image recognition sub-model of the target cooking model, so that the image recognition sub-model performs semantic analysis on the ingredient image information to obtain the estimated recipe information; Based on the estimated recipe information, parameter estimation processing is performed to obtain cooking parameter information; the parameter estimation processing based on the estimated recipe information to obtain cooking parameter information includes: obtaining the weight information of the ingredients to be cooked; inputting the estimated recipe information and the weight information of the ingredients to be cooked into the cooking parameter estimation sub-model of the target cooking model, so that the cooking parameter estimation sub-model performs parameter estimation processing based on the estimated recipe information and the weight information of the ingredients to be cooked to obtain the cooking parameter information; When the object cooks the food to be cooked based on the cooking parameter information, the cooking image information of the food to be cooked, the current working status information of the target cooking device corresponding to the cooking parameter information, and the current cooking environment information of the cooking environment are obtained. Based on the cooking image information, the current working state information, and the current cooking environment information, a cooking state analysis is performed to obtain cooking correction information, so that the object can perform cooking corrections based on the cooking correction information; the step of performing cooking state analysis based on the cooking image information, the current working state information, and the current cooking environment information to obtain cooking correction information includes: inputting the cooking image information, the current working state information, and the current cooking environment information into the cooking state analysis sub-model, so that the cooking state analysis sub-model performs cooking state analysis based on the cooking image information, the current working state information, and the current cooking environment information to obtain the cooking correction information.

2. The control method of the cooking navigation system according to claim 1, characterized in that, The cooking navigation system includes an image acquisition module. The process of acquiring the image information of the ingredients to be cooked includes: The image acquisition module acquires images of the ingredients to be cooked, thereby obtaining image information of the ingredients.

3. The control method for the cooking navigation system according to claim 2, characterized in that, The step of performing parameter estimation based on the estimated recipe information and the weight information of the ingredients to be cooked to obtain the cooking parameter information includes: The preset cooking parameter library is queried based on the estimated recipe information and the weight information of the ingredients to be cooked to obtain the query results; the preset cooking parameter library includes multiple preset cooking parameters, which represent the cooking parameters of preset cooking equipment, and each preset cooking parameter is set based on the preset recipe and the weight of the preset ingredients; If the query results meet the preset matching conditions, the preset cooking parameters corresponding to the query results that meet the preset matching conditions are determined as the cooking parameter information.

4. The control method of the cooking navigation system according to claim 2, characterized in that, The cooking navigation system also includes a sensing module. The acquisition of cooking image information of the food to be cooked, current working status information of the target cooking equipment corresponding to the cooking parameter information, and current cooking environment information of the cooking environment includes: The image acquisition module acquires images of the ingredients to be cooked during cooking to obtain the cooking image information. Based on the sensing module, the target cooking device is detected to obtain the current working status information; the current working status information represents the temperature information of the cooking device. The cooking environment is detected by the sensing module to obtain the current cooking environment information; the current cooking environment information represents the gas detection information of the cooking environment.

5. The control method for the cooking navigation system according to claim 1, characterized in that, After obtaining the cooking parameter information, the method further includes: The prompt indicates that the object cooks based on the cooking parameter information and the ingredients to be cooked; Accordingly, After obtaining the cooking correction information, the method further includes: The object is prompted to make cooking corrections based on the cooking correction information.

6. The control method of the cooking navigation system according to claim 3, characterized in that, After the object completes cooking the ingredients to be cooked, the method further includes: The image acquisition module acquires images of the cooked ingredients to be cooked, thereby obtaining images of the cooked dish. Record the historical cooking parameters of the cooking equipment used in cooking; Based on the cooked dish image and the historical cooking parameters, supplementary cooking parameters are generated; The supplementary cooking parameters are entered into the preset cooking parameter library.

7. The control method of the cooking navigation system according to claim 1, characterized in that, The system also includes a voice interaction module. After obtaining the estimated recipe information, the method further includes: The cooking voice signal of the object is obtained based on the voice interaction module; The cooking voice signal is recognized to obtain the object's cooking instruction information; The estimated recipe information is corrected based on the object cooking instruction information to obtain the target recipe information; Accordingly, The parameter estimation process based on the estimated recipe information to obtain cooking parameter information includes: Based on the target recipe information, parameter estimation processing is performed to obtain the cooking parameter information.

Images