Cooking method based on cooking ai large model and text-to-video model, and intelligent cooking apparatus

Abstract

Provided in the present invention are a cooking method based on a cooking AI large model and a text-to-video model, and an intelligent cooking apparatus. The intelligent cooking apparatus is provided with an information collection module, a storage module, a processor, a display screen and a wireless communication module, wherein the information collection module is configured to receive cooking demand information inputted by a user; a cooking AI large model analyzes and trains the cooking demand information to generate a new cooking method; a text-to-video model generates a specific cooking video on the basis of the new cooking method; and when the user confirms the new cooking method by means of a user mobile terminal or the display screen to perform cooking, the display screen or the user mobile terminal outputs the specific cooking video. The present invention uses a cooking AI large model to convert brief cooking demand information of a user into prompt conditions that are longer, more detailed and more complete, and the prompt conditions are then sent to a text-to-video model, such that the text-to-video model generates, on the basis of the prompt conditions, a specific cooking video meeting user demands, thereby improving the cooking user experience.

Description

A cooking method based on a cooking AI large model and a text-to-video model and an intelligent cooking device thereof TECHNICAL FIELD

[0001] The present application relates to a cooking method and device thereof, in particular to a cooking method based on a cooking AI large model and a text-to-video model and an intelligent cooking device thereof. BACKGROUND

[0002] With the continuous progress of science and technology, intelligent cooking devices have become a topic of increasing concern. Intelligent cooking devices include various kitchen utensils, such as intelligent gas stoves, intelligent steam ovens, intelligent electric steamers, intelligent electric rice cookers, or intelligent frying pans, etc. These kitchen utensils can be controlled through a mobile phone APP or a human-computer interaction module of the intelligent cooking device, making the cooking process more convenient, fast and accurate.

[0003] The current situation of intelligent cooking devices is that users can be guided to perform cooking operations in the form of text or voice through the human-computer interaction module, but the interaction mode is single, fixed, and monotonous, which cannot meet the personalized cooking experience of users. SUMMARY

[0004] Based on the deficiencies of the prior art, the present application provides a cooking method based on a cooking AI large model and a text-to-video model and an intelligent cooking device thereof. The cooking AI large model is used to convert the user's short cooking requirement information into longer, more detailed and more complete prompt conditions, and then send them to the text-to-video model, so that the text-to-video model generates a specific cooking video with high quality, which can accurately reflect the user's requirements and meet the user's needs, thereby improving the user's cooking experience.

[0005] In order to solve the above-mentioned prior art problems, the present application provides a cooking method based on a cooking AI large model and a text-to-video model, applied to an intelligent cooking device, wherein the intelligent cooking device is provided with an information collection module, a storage module, a processor, a display screen and a wireless communication module, the information collection module, the storage module, the display screen, the wireless communication module and the processor are electrically connected, the information collection module is used to receive cooking demand information input by a user in the form of voice, video, text, picture or 3D model format, and push the cooking demand information to a cooking AI large model, the cooking AI large model analyzes and trains the cooking demand information to generate a new cooking method, the new cooking method is sent to a user mobile terminal and the intelligent cooking device through the wireless communication module after being completed by inference operation of the cooking AI large model, the cooking AI large model sends the new cooking method to a text-to-video model, the new cooking method includes food preparation instructions, intelligent cooking device stations, cooking operation steps and cooking parameters, the text-to-video model generates a specific cooking video according to the new cooking method, the text-to-video model sends the specific cooking video to a user mobile terminal or the display screen of the intelligent cooking device through the wireless communication module, when the user confirms the new cooking method for cooking through the user mobile terminal or the display screen, the display screen or the user mobile terminal outputs the specific cooking video to guide the user to cook using the new cooking method, and the specific cooking video includes food preparation instruction content, real-time voice and real-time cooking operation step instruction content meeting the user's demand.

[0006] As an improvement of the cooking method based on the cooking AI large model and the text-to-video model of the present application, the text-to-video model generates the specific cooking video, including the following steps:

[0007] Firstly, the cooking AI large model analyzes and trains the cooking demand information to generate a new cooking method, and converts the food preparation instructions, cooking operation steps and cooking parameters of the new cooking method into text content;

[0008] Secondly, the text-to-video model is called through an API, the text content of the food preparation instructions, cooking operation steps and cooking parameters is provided to the text-to-video model, and the specific cooking video is generated by the text-to-video model.

[0009] As an improvement of the cooking method based on the cooking AI large model and the text-to-video model of the present application, an AI virtual human system and a video synthesis software are further included, the information collection module is further used to receive virtual human demand information input by a user in the form of voice, video, text, picture or 3D model, and push the virtual human demand information to the AI virtual human system, the AI virtual human system outputs a specific virtual human after analysis and training, the specific virtual human and the specific cooking video generate a specific virtual human cooking video through the video synthesis software, and send the specific virtual human cooking video to a user mobile terminal or the display screen of the intelligent cooking device, when the user confirms the new cooking method for cooking through the user mobile terminal or the display screen, the display screen or the user mobile terminal outputs the specific virtual human cooking video to guide the user to cook using the new cooking method, and the specific virtual human cooking video includes food preparation guide content, a specific virtual human, real-time voice of the specific virtual human, and real-time cooking operation step guide content of the specific virtual human.

[0010] As an improvement of the cooking method based on the cooking AI large model and the text-to-video model of the present application, a camera is further included to record the cooking process to form an original cooking video, the original cooking video is sent to the cooking AI large model through the wireless communication module, the cooking AI large model analyzes and trains the original cooking video to generate a new cooking method and sends it to the text-to-video model, the text-to-video model generates a specific cooking video according to the new cooking method, the specific virtual human and the specific cooking video generate a specific virtual human cooking video through the video synthesis software, and send the specific virtual human cooking video to a user mobile terminal or the display screen of the intelligent cooking device, when the user confirms the new cooking method for cooking through the user mobile terminal or the display screen, the display screen or the user mobile terminal outputs the specific virtual human cooking video to guide the user to cook using the new cooking method, and the specific virtual human cooking video includes food preparation guide content, a specific virtual human, real-time voice of the specific virtual human, and real-time cooking operation step guide content of the specific virtual human.

[0011] As an improvement of the cooking method based on the cooking AI large model and the text-to-video model of the present application, the text-to-video model is sora, invideo AI, Outfit Anyone, Runway, Etna, Boximator, W.A.L.T, MagicVideo-V2, Stable Video Diffusion, FlashCut-AI digital person, KreadoAI, Star Group, Opera, Tongyi Dance King, Moxiaoxian, Wenxin Yige, Phenaki, Make-a-Video, Mobius, Yifang Miaochuang, Ju Ri AI, Yinying AI, D-Human digital human, new video large model, Doubao, magic has said, blue mark clone, Lingdong portrait, Meitu AI digital human DreamAvatar, Pika or Gen-2.

[0012] As an improvement of the cooking method based on the cooking AI large model and the text-to-video model of the present application, the cooking AI large model generates a new cooking method including two steps: building a cooking AI large model and calling a cooking AI large model, the building a cooking AI large model includes six steps:

[0013] The first step is to collect cooking data: collect all information about cooking methods of food materials presented by voice, video, text, pictures or 3D models;

[0014] The second step is to preprocess the cooking data: process all the collected information about cooking methods of food materials to ensure the integrity and availability of the information, including converting different formats of information into text, and editing the text information according to a certain format to facilitate the subsequent training of the AI large model;

[0015] The third step is to select AI large models applicable to cooking: select third-party AI large models at home and abroad, and measure them with accuracy, response speed and diversity indicators;

[0016] The fourth step is to train the cooking AI large model: after the cooking data set is sorted out in the second step, the cooking data set is fine-tuned and trained by the third-party AI large model, and the cooking AI large model with all related data of the cooking method is generated after the training is completed;

[0017] The fifth step is to verify and test the cooking AI large model: the cooking AI large model generated in the fourth step is detected and evaluated for specific tasks, if the evaluation effect does not pass, the steps of the first step, the second step, the third step and the fourth step are repeated to retrain until the effect evaluation passes, the cooking AI large model is generated and stored in the storage module of the cloud platform or the intelligent cooking device;

[0018] Sixth step, deploy and maintain the cooking AI large model: deploy the newly generated AI large model into the storage module of the cloud platform or the intelligent cooking device, and continuously maintain and update it, regularly update the data to ensure the timeliness and accuracy of the data; the cooking demand information input by the user in the form of voice, video, text, picture or 3D model is used to generate a new cooking method by calling the cooking AI large model, the calling of the cooking AI large model includes assembling a query statement, inference operation of the AI large model and return of the result of the AI large model, and the new cooking method is sent to the user's mobile terminal or the processor after the inference operation of the cooking AI large model is completed.

[0019] As an improvement of the cooking method based on the cooking AI large model and the text-to-video model of the present application, the method for outputting a specific virtual person by the AI virtual person system after analysis and training includes the steps of constructing a specific AI virtual person and calling the specific AI virtual person, the step of constructing a specific AI virtual person includes seven steps: character generation, voice generation, lip synchronization, action generation, synthesis display, test verification and deployment and maintenance, the virtual person demand information input by the user in the form of voice, video, text, picture or 3D model is used to generate a specific AI virtual person by the AI virtual person system, and the step of calling the specific AI virtual person includes generating the text instruction content of the specific AI virtual person, transmitting the text instruction content to the AI virtual person system, and generating a video by the AI virtual person system.

[0020] As an improvement of the cooking method based on the cooking AI large model and the text-to-video model of the present application, the step of calling the specific AI virtual person specifically includes the following steps:

[0021] First step, generate the text instruction content of the specific AI virtual person, which can be generated by the cooking AI large model or generated by the system program according to the business logic.

[0022] Second step, transmit the text instruction content to the AI virtual person system of the cloud platform by API calling, and generate a video by the AI virtual person system, which includes the following four processes:

[0023] First, voice generation: convert the text content into the voice of the specific AI virtual person by the voice generation module of the AI virtual person system, and can clone the voice uploaded by the user.

[0024] Second, lip generation: realize the lip synchronization of the specific AI virtual person by algorithm based on the voice content generated by the voice of the specific AI virtual person, which can realize the synchronization of Chinese and English.

[0025] Third, action generation: select the action template uploaded by the user, output the corresponding action posture of the specific AI virtual person according to the action template of the user, and combine the lip synchronization of the specific AI virtual person in the second step.

[0026] Fourth, video synthesis: the first step, the second step and the third step of the preceding step are output as a standard video format, and returned.

[0027] The application provides a kind of intelligent cooking device, it is characterized in that, it is equipped with information collection module, storage module, processor, display screen and wireless communication module, the information collection module, the storage module, the display screen, the wireless communication module are electrically connected with the processor, the information collection module is used to collect the information or virtual person demand information of all information about food cooking method that user inputs by voice, video, text, picture or 3D model format, the processor is connected between cloud platform and user mobile terminal by the wireless communication module, the processor is used to execute the new cooking method of the cooking AI large model generated in claim 1 or 2 or 3 and cook, the processor is also used to execute the specific cooking video or the specific virtual person cooking video in the display screen or user mobile terminal output.

[0028] As an improvement of the intelligent cooking device of the application, the intelligent cooking device is provided with a smart cooking device station for frying, exploding, rolling, frying, boiling, frying, pasting, burning, stewing, stewing, steaming, boiling, cooking, cooking, frying, frying, marinating, baking, stewing, freezing, pulling silk, honey juice, smoking, rolling, sliding or baking.

[0029] As an improvement of the intelligent cooking device of the application, the smart cooking device station for frying, exploding, rolling, frying, boiling, frying, pasting, burning, stewing, stewing, steaming, boiling, cooking, cooking, frying, frying, marinating, baking, stewing, freezing, pulling silk, honey juice, smoking, rolling, sliding or baking is provided with a corresponding operation detection feedback system, which is used to detect whether the cooking operation performed by the user meets the requirements of the cooking method of the cooking AI large model.

[0030] As an improvement of the intelligent cooking device of the application, the intelligent cooking device is provided with a man-machine interaction module, the man-machine interaction module is provided with a voice recognition device, the man-machine interaction module is used for information interaction and intelligent control between the intelligent cooking device and the user, the intelligent control includes confirming cooking parameters and cooking operation steps by voice or in the display screen by the user.

[0031] As an improvement of the intelligent cooking device of the application, the intelligent cooking device is provided with a projection device, the projection device is electrically connected with the processor, and the projection device is used to project the specific cooking video or the specific virtual person cooking video outside the intelligent cooking device.

[0032] The cooking method based on the cooking AI large model and the text-to-video model and the intelligent cooking device have the beneficial effects that: the user demand information collected by the information collection module of the intelligent cooking device is sent to the cooking AI large model, the cooking AI large model analyzes and trains the cooking demand information to generate a new cooking method, and then sends the new cooking method to the text-to-video model, the text-to-video model generates a specific cooking video according to the new cooking method, the cooking AI large model converts the short cooking demand information of the user into a longer, more detailed and more complete prompt condition, that is, a new cooking method, and then sends the new cooking method to the text-to-video model, so that the text-to-video model generates the specific cooking video with high quality, which can accurately reflect the requirements of the user and meet the demand of the user, thereby improving the cooking experience of the user, and when the user cooks, the display screen or the user mobile terminal outputs the specific cooking video, and the user can cook according to the material preparation guide content, real-time voice and real-time cooking operation step guide content of the specific cooking video, so as to more intuitively guide the user to cook and improve the cooking experience of the user. BRIEF DESCRIPTION OF DRAWINGS

[0033] Fig. 1 is a working flow chart of the preferred embodiment of the cooking method based on the cooking AI large model and the text-to-video model and the intelligent cooking device.

[0034] Fig. 2 is a working flow chart of the first other embodiment of the cooking method based on the cooking AI large model and the text-to-video model and the intelligent cooking device.

[0035] Fig. 3 is a working flow chart of the second other embodiment of the cooking method based on the cooking AI large model and the text-to-video model and the intelligent cooking device.

[0036] Fig. 4 is a working flow chart of the third other embodiment of the cooking method based on the cooking AI large model and the text-to-video model and the intelligent cooking device.

[0037] Fig. 5 is a working flow chart of the fourth other embodiment of the cooking method based on the cooking AI large model and the text-to-video model and the intelligent cooking device.

[0038] Fig. 6 is a working flow chart of the fifth other embodiment of the cooking method based on the cooking AI large model and the text-to-video model and the intelligent cooking device. DETAILED DESCRIPTION

[0039] In the following, the present application is further described in conjunction with Figs. 1-6, the specific embodiments and other embodiments, and it should be noted that the technical features described below can be combined in any manner to form new embodiments without conflict.

[0040] In a preferred embodiment, with reference to FIG. 1, the present application provides a cooking method based on a cooking AI large model 24 and a text-to-video model 23, applied to an intelligent cooking device, the intelligent cooking device is provided with an information collection module 31, a storage module 22, a processor 26, a display screen 33 and a wireless communication module 27, the information collection module 31, the storage module 22, the display screen 33, the wireless communication module 27 and the processor 26 are electrically connected, the information collection module 31 is used to receive the cooking demand information input by the user 36 in the form of voice, video, text, picture or 3D model format, and push the cooking demand information to the cooking AI large model 24, the cooking AI large model 24 analyzes and trains the cooking demand information to generate a new cooking method, the new cooking method is completed by the inference operation of the cooking AI large model 24 and is sent to the user mobile terminal 34 and the intelligent cooking device through the wireless communication module 27, the cooking AI large model 24 sends the new cooking method to the text-to-video model 23, the new cooking method includes food preparation instructions, intelligent cooking device stations, cooking operation steps and cooking parameters, the text-to-video model 23 generates a specific cooking video according to the new cooking method, the text-to-video model 23 sends the specific cooking video to the user mobile terminal 34 or the display screen 33 of the intelligent cooking device through the wireless communication module 27, when the user 36 confirms the new cooking method for cooking through the user mobile terminal 34 or the display screen 33, the display screen 33 or the user mobile terminal 34 outputs the specific cooking video to guide the user 36 to cook with the new cooking method, the specific cooking video includes food preparation guide content, real-time voice and real-time cooking operation step guide content that meet the user's 36 requirements; wherein the food preparation guide content displayed by the specific cooking video includes the food preparation instructions, the real-time cooking operation step guide content displayed by the specific cooking video includes the cooking operation steps and the intelligent cooking device stations, and the real-time voice displayed by the specific cooking video includes the explanation of the food preparation content, the cooking operation steps and the cooking parameters.For example, the user 36 can input the cooking demand information of the text or voice "I want to make red-braised pork ribs", the cooking AI large model 24 analyzes and trains the cooking demand information to generate a new cooking method and sends it to the user mobile terminal 34, when the user confirms on the user mobile terminal 34, the cooking AI large model 24 converts the new cooking method into a text content describing how to make red-braised pork ribs, which includes detailed descriptions of the preparation of pork ribs and seasonings, cooking operation steps and cooking parameters, the cooking operation steps include the use of intelligent cooking device stations, and then sends it to the text-to-video model 23, the text-to-video model 23 generates a specific cooking video according to "a text content describing how to make red-braised pork ribs", when the user clicks on the specific cooking video, the user will see the video sequentially and coherently showing the content of cooking red-braised pork ribs, the content shown by the video for the food preparation guide content includes preparing pork ribs, cutting pork ribs, and blanching treatment; the content shown by the video for the real-time cooking operation step guide content includes a series of cooking operations such as starting the cooking device, putting oil, putting the processed pork ribs into the pot, frying the pork ribs, putting seasonings, and plating, accompanied by the action of cooking, there are instant voiceovers such as the sound of tearing the packaging, the "drip-drip" sound when starting the cooking device, and the "sizzle" sound when frying, and there are also real-time voiceovers to explain each step of the cooking operation and set the cooking parameters, so that the user 36 can watch the cooking process of red-braised pork ribs as if he were there, enhance the user's experience, make it easy for the user to start cooking, and make the cooking process enjoyable and comfortable.

[0041] In a preferred embodiment, the text-to-video model 23 generates the specific cooking video includes the following steps:

[0042] First, the cooking AI large model 24 analyzes and trains the cooking demand information to generate a new cooking method and converts the food preparation instructions, cooking operation steps and cooking parameters of the new cooking method into text content;

[0043] Second, call the text-to-video model 23 through API, provide the text content of the food preparation instructions, cooking operation steps and cooking parameters to the text-to-video model 23, and generate the specific cooking video by the text-to-video model 23.

[0044] In the preferred embodiment, the AI virtual human system 25 and the video synthesis software are further included, the information collection module 31 is further used for receiving the virtual human demand information input by the user 36 in the form of voice, video, text, picture or 3D model, and pushing the virtual human demand information to the AI virtual human system 25, the AI virtual human system 25 outputs a specific virtual human after analysis and training, the specific virtual human and the specific cooking video generate a specific virtual human cooking video through the video synthesis software, and send it to the user mobile terminal 34 or the display screen 33 of the intelligent cooking device through the wireless communication module 27, when the user 36 confirms the new cooking method for cooking through the user mobile terminal 34 or the display screen 33, the display screen 33 or the user mobile terminal 34 outputs the specific virtual human cooking video to guide the user 36 to cook with the new cooking method, the specific virtual human cooking video includes the food preparation guide content, the specific virtual human, the specific virtual human real-time voice and the specific virtual human real-time cooking operation step guide content which meet the demand of the user 36; wherein the food preparation guide content displayed by the specific virtual human cooking video includes the food preparation instruction, the specific virtual human real-time cooking operation step guide content displayed by the specific virtual human cooking video includes the cooking operation step and the intelligent cooking device station, and the specific virtual human real-time voice displayed by the specific virtual human cooking video includes the explanation of the cooking operation step and the cooking parameter.The user 36 can customize the virtual human image in the cooking video. The user 36 can customize the virtual human image by inputting a picture or video of a friend or a relative, or by inputting the name of a star or a chef. For example, the user 36 inputs “Chef Zhou makes red-braised pork ribs”. The information collection module 31 pushes the information “Chef Zhou makes red-braised pork ribs” to the AI virtual human system 25. The AI virtual human system 25 extracts the relevant information of Chef Zhou from the user input information “Chef Zhou makes red-braised pork ribs” and analyzes to generate a specific AI virtual human. The information collection module 31 pushes the information “Chef Zhou makes red-braised pork ribs” to the cooking AI large model 24. The cooking AI large model 24 extracts the information “makes red-braised pork ribs” from the user input information “Chef Zhou makes red-braised pork ribs”. The cooking AI large model 24 analyzes and trains the information “makes red-braised pork ribs” to generate a new cooking method and sends it to the user mobile terminal 34. When the user 36 confirms on the user mobile terminal 34, the cooking AI large model 24 converts the new cooking method into a text content describing how to make red-braised pork ribs, which includes detailed descriptions of the preparation of pork ribs and seasonings, cooking operation steps, and cooking parameters, including the use of intelligent cooking device stations. Then it is sent to the text-to-video model 23. The text-to-video model 23 generates a specific cooking video according to the “text content describing how to make red-braised pork ribs”. The specific virtual human and the specific cooking video generate a specific virtual human cooking video through the video synthesis software, and send it to the user mobile terminal 34 or the display screen 33 of the intelligent cooking device through the wireless communication module 27. The display screen 33 or the user mobile terminal 34 outputs the specific virtual human cooking video, which sequentially and coherently displays the content of cooking red-braised pork ribs and Chef Zhou explaining each cooking operation step. The video synthesis software includes “Diyin”, “Fengyun Video Converter”, “Gihosoft Free Video Joiner”, “Adobe Premiere Pro”, or “FilmoraGo”. In this embodiment, the video synthesis software is “Adobe Premiere Pro”.

[0045] In the preferred embodiment, a camera is also included for recording the cooking process to form an original cooking video, which is sent to the cooking AI large model 24 through the wireless communication module 27. The cooking AI large model 24 analyzes and trains the original cooking video to generate a new cooking method and sends it to the text-to-video model 23. The text-to-video model 23 generates a specific cooking video according to the new cooking method. The specific virtual person and the specific cooking video generate a specific virtual person cooking video through the video synthesis software, and send the specific virtual person cooking video to the user mobile terminal 34 or the display screen 33 of the intelligent cooking device. When the user 36 confirms the new cooking method for cooking through the user mobile terminal 34 or the display screen 33, the display screen 33 or the user mobile terminal 34 outputs the specific virtual person cooking video to guide the user 36 to cook using the new cooking method. The specific virtual person cooking video includes food preparation guidance content, specific virtual person, specific virtual person real-time voice, and specific virtual person real-time cooking operation step guidance content that meet the user's 36 needs. For example, the user 36 records the cooking process of making "fried eggs" at home using existing kitchen utensils through the camera to form an original cooking video, and inputs the virtual person demand information "use of Happy Sheep cartoon characters". The cooking AI large model 24 analyzes and trains the original cooking video recorded by the user 36 to generate a new cooking method, and sends it to the text-to-video model 23 to generate the specific cooking video. The AI virtual person system 25 analyzes and trains the information "use of Happy Sheep cartoon characters" input by the user to generate a specific AI virtual person. The specific virtual person and the specific cooking video generate a specific virtual person cooking video through the video synthesis software, and send it to the user mobile terminal 34 or the display screen 33 of the intelligent cooking device through the wireless communication module 27. The specific virtual person cooking video output by the display screen 33 or the user mobile terminal 34 sequentially and coherently displays the content of making "fried eggs" and the Happy Sheep cartoon character explaining each cooking operation

[0046] In the preferred embodiment, the text-to-video model 23 is sora. In other embodiments, the text-to-video model 23 is invideo AI, Outfit Anyone, Runway, Etna, Boximator, W.A.L.T, MagicVideo-V2, Stable Video Diffusion, FlashCut-AI Digital Person, KreadoAI, Star Group, Opera, Tongyi Dance King, Moxiaoxian, Wenxin Yige, Phenaki, Make-a-Video, Mobius, Yifang Miaochuang, Ju Ri AI, Yinying AI, D-Human digital human, new video large model, bean bag, magic has said, blue mark clone, live portrait, Meitu AI digital human DreamAvatar, Pika or Gen-2.

[0047] In the preferred embodiment, the cooking AI large model 24 generates a new cooking method, which includes two steps: building a cooking AI large model 241 and calling a cooking AI large model 242, and the building a cooking AI large model 241 includes six steps:

[0048] The first step is to collect cooking data: collect all information about cooking methods of food materials presented by voice, video, text, pictures or 3D models;

[0049] The second step is to preprocess the cooking data: process all the collected information about cooking methods of food materials to ensure the integrity and availability of the information, including converting different formats of information into text and editing the text information according to a certain format to facilitate the subsequent training of the AI large model;

[0050] The third step is to select AI large models applicable to cooking: select third-party AI large models at home and abroad, and measure them with accuracy, response speed and diversity indicators;

[0051] The fourth step is to train the cooking AI large model 24: After the cooking data set is sorted out in the second step, the cooking data set is fine-tuned and trained by the third-party AI large model, and the cooking AI large model 24 with all related data of the cooking method is generated after the training is completed;

[0052] The fifth step is to verify and test the cooking AI large model 24: the cooking AI large model 24 generated in the fourth step is detected and evaluated for specific tasks, if the evaluation effect does not pass, the steps of the first step, the second step, the third step and the fourth step are repeated for retraining until the effect evaluation passes, the cooking AI large model 24 is generated and stored in the cloud platform 35 or the storage module 22 of the intelligent cooking device;

[0053] Step 6, deployment and maintenance of cooking AI large model 24: the newly generated AI large model is deployed to the cloud platform 35 or the storage module 22 of the intelligent cooking device, and is continuously maintained and updated, and the data is regularly updated to ensure the timeliness and accuracy of the data. The cooking demand information input by the user in the form of voice, video, text, picture or 3D model is used to generate a new cooking method by calling the cooking AI large model 24, which includes assembling a query statement, performing inference operation by the AI large model and returning the result by the AI large model, and the new cooking method is sent to the user mobile terminal 34 or the processor 26 after the inference operation of the cooking AI large model 24 is completed.

[0054] In the preferred embodiment, the method of outputting a specific virtual person by analyzing and training the AI virtual person system 25 includes the steps of constructing a specific AI virtual person 251 and calling a specific AI virtual person 252. The step of constructing a specific AI virtual person 251 includes seven steps: character generation, voice generation, lip synchronization, action generation, synthesis display, test verification and deployment and maintenance. The virtual person demand information input by the user in the form of voice, video, text, picture or 3D model is used to generate a specific AI virtual person by the AI virtual person system 25. The step of calling a specific AI virtual person 252 includes generating the text instruction content of a specific AI virtual person, transmitting the text instruction content to the AI virtual person system 25, and generating a video by the AI virtual person system 25.

[0055] In the preferred embodiment, the step of calling a specific AI virtual person 252 specifically includes the following steps:

[0056] Step 1, generating the text instruction content of a specific AI virtual person, which can be generated by the cooking AI large model 24 or generated by the system program according to the business logic.

[0057] Step 2, transmitting the text instruction content to the AI virtual person system 25 of the cloud platform 35 by API calling, and generating a video by the AI virtual person system 25, which includes the following four processes:

[0058] First, voice generation: converting the text content into the voice of a specific AI virtual person by the voice generation module of the AI virtual person system 25, and cloning the voice uploaded by the user.

[0059] Second, lip generation: synchronizing the lips of a specific AI virtual person by algorithm based on the voice of the specific AI virtual person, which can synchronize both Chinese and English.

[0060] Third, action generation: selecting the action template uploaded by the user, and outputting the corresponding action posture of a specific AI virtual person according to the action template of the user, and combining the lip synchronization of a specific AI virtual person in the second step.

[0061] Fourth, video synthesis: output the contents generated in the first, second and third steps above into a standard video format, and return.

[0062] Referring to FIG. 1, the present application provides an intelligent cooking device, which is provided with an information collection module 31, a storage module 22, a processor 26, a display screen 33 and a wireless communication module 27, the information collection module 31, the storage module 22, the display screen 33, the wireless communication module 27 and the processor 26 are electrically connected, the information collection module 31 is used to collect all information about cooking methods of food materials or virtual human demand information input by a user 36 in the form of voice, video, text, picture or 3D model format, the processor 26 is used to execute a new cooking method generated by the cooking AI large model 24 in claim 1 or 2 or 3 to cook, the processor 26 is also used to execute output of the specific cooking video or the specific virtual human cooking video on the display screen 33 or the user mobile terminal 34. The intelligent cooking device is provided with intelligent cooking device workstations for frying 11, frying 12, stewing 13, frying 14, steaming 15 and boiling 16, the intelligent cooking device workstation for frying 11 is a frying oven, an induction cooker, a microwave oven or an oven, the intelligent cooking device workstation for frying 12 is an air fryer, the intelligent cooking device workstation for stewing 13 is a slow cooker, an electric stewing pot or an electric stewing cup, the intelligent cooking device workstation for frying 14 is a frying machine, the intelligent cooking device workstation for steaming 15 is a steaming oven, an electric steaming pot, an electric rice cooker or an electric pressure cooker, and the intelligent cooking device workstation for boiling 16 is an electric boiling pot. One or more intelligent cooking devices use one or more cooking functions of frying, air frying, frying, stewing, steaming and boiling to cook food materials. The processor 26 can set the working parameters of the intelligent cooking device workstations for frying 11, frying 12, stewing 13, frying 14, steaming 15 and boiling 16, and control each intelligent cooking device workstation to execute related instructions such as standby, start, stop, cleaning and the like, when the cooking demand information of the user is analyzed by the cooking AI large model 24 and a new cooking method is generated, and then sent to the user mobile terminal 34 (or the display screen 33) by the wireless communication module 27, after the user confirms, the cooking AI large model 24 will send the new cooking method to the processor 26, the processor 26 will set the parameters of the related intelligent cooking device workstations for frying 11, frying 12, stewing 13, frying 14, steaming 15 and boiling 16 according to the new cooking method and control them to execute related instructions, at the same time, the display screen 33 or the user mobile terminal 34 outputs the specific cooking video or the specific virtual human cooking video.

[0063] Referring to FIG. 1, the cooking AI large model 24, the AI virtual human system 25 and the text-to-video model 23 of the cooking method based on the cooking AI large model and the text-to-video model and the intelligent cooking device thereof are arranged in the cloud platform 35.

[0064] Referring to FIG. 6, in other embodiment five, the cooking AI large model 24, the AI virtual human system 25 and the text-to-video model 23 of the cooking method based on the cooking AI large model and the text-to-video model and the intelligent cooking device thereof are built-in in the storage module 22; in addition, in other embodiments, the cooking AI large model 24, the AI virtual human system 25 and the text-to-video model 23 can be arranged in any combination of the storage module 22 or the cloud platform.

[0065] Referring to FIG. 1, in a preferred embodiment, the intelligent cooking device is provided with intelligent cooking device stations for frying 14, frying 12, frying 11, stewing 13, steaming 15 and boiling 16.

[0066] Referring to FIG. 2, in other embodiment one, the intelligent cooking device is provided with intelligent cooking device stations for frying 14, frying 12, frying 11, stewing 13 and steaming 15.

[0067] Referring to FIG. 3, in other embodiment two, the intelligent cooking device is provided with intelligent cooking device stations for frying 14, frying 12, frying 11 and stewing 13.

[0068] Referring to FIG. 4, in other embodiment three, the intelligent cooking device is provided with intelligent cooking device stations for frying 14, frying 12 and frying 11.

[0069] Referring to FIG. 5, in other embodiment four, the intelligent cooking device is provided with an intelligent cooking device station for frying 14.

[0070] Referring to FIG. 1, in the preferred embodiment, the frying 14, frying 12, frying 11, stewing 13, steaming 15, and boiling 16 intelligent cooking device stations are provided with corresponding operation detection feedback systems 21 for detecting whether the cooking operation performed by the user meets the requirements of the cooking method of the cooking AI large model 24. The operation detection feedback system 21 includes camera, infrared detection, radar detection, magnetic detection, weight detection, etc. detection devices for detecting whether the cooking operation performed by the user in the intelligent cooking device meets the requirements of the new cooking method of the cooking AI large model 24. For example, at the frying 14 intelligent cooking device station, the intelligent cooking device requires the user to pour a certain amount of cooking oil, but the user does not pour or uses the wrong ingredients. At this time, the operation detection feedback system 21 detects through the camera and feeds back the information to the processor 26 of the intelligent cooking device. The processor 26 controls the intelligent cooking device station to pause the next operation and sends information to the user for correction. When the user corrects, the operation detection feedback system 21 detects through the camera and feeds back the information to the processor 26. The processor 26 controls the intelligent cooking device station to perform the next operation. The above is only an example. In addition, infrared detection, radar detection, magnetic detection, and weight detection can be used singly or in combination for detection.

[0071] Referring to FIG. 1, in the preferred embodiment, a human-computer interaction module 32 is provided, which is provided with a voice recognition device, and is used for information interaction and intelligent control between the intelligent cooking device and the user 36, including confirmation of cooking parameters and cooking operation steps by the user 36 through voice or the display screen 33. The human-computer interaction module 32 further includes a touch screen (or display screen 33), an image input unit and a camera device, and is electrically connected with the processor 26, and is used for information interaction between the intelligent cooking device and the user. The user can input cooking demand information in the form of voice, video, text, picture or 3D model through the human-computer interaction module 32, which sends the cooking demand information to the processor 26. The processor 26 sends the cooking demand information to the cooking AI large model 24 through the wireless communication module 27. The cooking AI large model 24 analyzes and trains the cooking demand information to generate a new cooking method, which includes food preparation instructions, intelligent cooking device stations, cooking operation steps and cooking parameters. After the new cooking method is completed by inference operation of the cooking AI large model 24, it is sent to the human-computer interaction module 32 (or the user mobile terminal 34) and the intelligent cooking device through the wireless communication module 27. The cooking AI large model 24 sends the new cooking method to the text-to-video model 23, which generates a specific cooking video according to the new cooking method. The text-to-video model 23 sends the specific cooking video to the human-computer interaction module 32 (or the user mobile terminal 34) through the wireless communication module 27. When the user confirms the new cooking method through the human-computer interaction module 32 (or the user mobile terminal 34) for cooking, the human-computer interaction module 32 (or the user mobile terminal 34) outputs the specific cooking video. The processor 26 determines the stations of the intelligent cooking device according to the new cooking method, sets the relevant parameters of the stations of the intelligent cooking device, and then controls the stations of the intelligent cooking device to cook. During the cooking process, when some steps need to be operated by the user 36, the processor 26 sends information to the user through the human-computer interaction module 32 (or the user mobile terminal 34) in the form of image, voice or text.The user 36 can confirm the cooking parameters and cooking operation steps through the voice recognition module of the human-computer interaction module 32, for example, during the cooking process, the user needs to adjust the cooking power of the cooking parameters from 2000W to 800W, the user inputs "set the cooking power to 800W" by voice, the voice recognition module receives the instruction and feeds back to the processor 26, and the processor 26 sets the cooking power of the relevant intelligent cooking device station of the intelligent cooking device to 800W; for example, the user completes the cooking, the user inputs "stop cooking work" by voice, the voice recognition module receives the instruction and feeds back to the processor 26, and the processor 26 controls the relevant intelligent cooking device station of the intelligent cooking device to stop cooking work. In addition, the intelligent control includes that the user 36 confirms the cooking parameters and cooking operation steps on the touch screen (or the display screen 33), for example, during the cooking process, the user needs to adjust the cooking power of the cooking parameters from 2500W to 100W, the user inputs the cooking power of 1000W on the display screen 33 and feeds back the instruction to the processor 26, and the processor 26 controls the cooking power of the relevant intelligent cooking device station of the intelligent cooking device to be set to 1000W.

[0072] In other embodiments, the intelligent cooking device is one or a combination of a wok, an air fryer, an induction cooker, a microwave oven, an oven, a steam oven, an electric rice cooker, a beef steak machine, a meat roaster, a rice pot machine, an electric pressure cooker, a steam cooker, a multifunctional food processor, an electric boiling pot, a bread maker, a chef machine, a steam pot machine, an electric ceramic stove, a cooking pot, a frying and roasting machine, a slow cooker, an electric saucepan, an electric sauce cup, an integrated stove or a frying pan, and one or more intelligent cooking devices include one or more cooking functions of frying, exploding, rolling, frying, cooking, frying, sticking, burning, stewing, stewing, steaming, boiling, cooking, frying, frying, mixing, marinating, roasting, stewing, freezing, pulling silk, honey, smoking, rolling, sliding or baking to cook food.

[0073] In other embodiments, the intelligent cooking device includes various combinations of cooking functions such as frying, air frying, roasting, frying, stewing, stewing, steaming, boiling or baking, such as an intelligent cooking device that integrates air frying, roasting, frying, steaming and other functions, or an intelligent cooking device that integrates frying, air frying, stewing, stewing, boiling and other functions, and so on.

[0074] Referring to FIG. 1, in a preferred embodiment, the intelligent cooking device is provided with a projection device 28, which is electrically connected with the processor 26, and the projection device 28 is used to project the specific cooking video or the specific virtual person cooking video outside the intelligent cooking device. The projection device 28 can be set at any position according to the needs of the user to facilitate the user to watch during cooking.

[0075] The cooking method based on the cooking AI large model and the text-to-video model and the intelligent cooking device have the beneficial effects that: the user demand information collected by the information collection module of the intelligent cooking device is sent to the cooking AI large model, the cooking AI large model analyzes and trains the cooking demand information to generate a new cooking method, and then sends the new cooking method to the text-to-video model, the text-to-video model generates a specific cooking video according to the new cooking method, the cooking AI large model converts the short cooking demand information of the user into longer, more detailed and more complete prompt conditions, that is, a new cooking method, and then sends the new cooking method to the text-to-video model, so that the text-to-video model generates the specific cooking video with high quality, which can accurately reflect the requirements of the user and meet the demand of the user, so as to improve the cooking experience of the user, and when the user cooks, the display screen or the user mobile terminal outputs the specific cooking video, the user can cook according to the material preparation guide content of the specific cooking video, the intelligent cooking device work site guide content, the real-time voice and the real-time cooking operation step guide content, so as to more vividly guide the user to cook and improve the cooking experience of the user.

[0076] The above has made a detailed description of the present application, the above is only the preferred embodiment of the present application, which cannot limit the scope of the present application, that is, any equivalent change and modification made within the scope of the present application should still fall within the scope of the present application.

Claims

1. A cooking method based on a cooking AI large model and a Vincent video model, characterized in that: Applied to an intelligent cooking device, the intelligent cooking device is provided with an information collection module, a storage module, a processor, a display screen and a wireless communication module. The information collection module, the storage module, the display screen and the wireless communication module are electrically connected to the processor. The information collection module is used to receive cooking demand information input by the user in the form of voice, video, text, picture or 3D model, and push the cooking demand information to the cooking AI big model. The cooking AI big model analyzes and trains the cooking demand information to generate a new cooking method. The new cooking method is sent to the user's mobile terminal and the intelligent cooking device through the wireless communication module after the cooking AI big model completes the inference operation. The cooking AI big model will The new cooking method is sent to the Vincent video model, and the new cooking method includes ingredient preparation instructions, smart cooking device workstations, cooking operation steps and cooking parameters. The Vincent video model generates a specific cooking video based on the new cooking method. The Vincent video model sends the specific cooking video to the user mobile terminal or the display screen of the smart cooking device through the wireless communication module. When the user confirms the new cooking method through the user mobile terminal or the display screen to cook, the display screen or the user mobile terminal outputs the specific cooking video to guide the user to use the new cooking method to cook. The specific cooking video includes ingredient preparation guidance content, real-time voice and real-time cooking operation step guidance content that meet user needs.

2. A cooking method based on a cooking AI large model and a Vincent video model, characterized in that: The Wensheng video model generates the specific cooking video, comprising the following steps: In the first step, the cooking AI model analyzes and trains the cooking demand information to generate a new cooking method and converts the ingredient preparation instructions, cooking operation steps, and cooking parameters of the new cooking method into text content; In the second step, the Vincent video model is called through the API, and the text content of the ingredient preparation instructions, the cooking operation steps and the cooking parameters are provided to the Vincent video model, and the Vincent video model is used to generate the specific cooking video.

3. The cooking method based on the cooking AI large model and the Vincent video model according to claim 1 or 2, characterized in that: It also includes an AI virtual human system and video synthesis software. The information collection module is also used to receive virtual human demand information input by the user in voice, video, text, picture or 3D model format, and push the virtual human demand information to the AI ​​virtual human system. The AI ​​virtual human system outputs a specific virtual human after analysis and training. The specific virtual human and the specific cooking video generate a specific virtual human cooking video through the video synthesis software, and the specific virtual human cooking video is sent to the user's mobile terminal or the display screen of the intelligent cooking device. When the user confirms the new cooking method through the user's mobile terminal or the display screen to cook, the display screen or the user mobile terminal outputs the specific virtual human cooking video to guide the user to use the new cooking method to cook. The specific virtual human cooking video includes food preparation guidance content that meets user needs, real-time voice of the specific virtual human, and real-time cooking operation step guidance content of the specific virtual human.

4. The cooking method based on the cooking AI large model and the Vincent video model according to claim 3 is characterized in that: It also includes a camera for recording the cooking process to form an original cooking video, and the original cooking video is sent to the cooking AI big model through the wireless communication module. The cooking AI big model analyzes and trains the original cooking video to generate a new cooking method and sends it to the Vincent video model. The Vincent video model generates a specific cooking video according to the new cooking method. The specific virtual person and the specific cooking video generate a specific virtual person cooking video through the video synthesis software, and the specific virtual person cooking video is sent to the user mobile terminal or the display screen of the smart cooking device. When the user confirms the new cooking method through the user mobile terminal or the display screen to cook, the display screen or the user mobile terminal outputs the specific virtual person cooking video to guide the user to use the new cooking method to cook. The specific virtual person cooking video includes food preparation guidance content that meets user needs, a specific virtual person, real-time voice of a specific virtual person, and real-time cooking operation step guidance content of a specific virtual person.

5. The cooking method based on the cooking AI large model and the Vincent video model according to claim 1, wherein the Vincent video model is sora, invideoAI, OutfitAnyone, Runway, Etna, Boximator, WALT, MagicVideo-V2, Stable Video Diffusion, Flash-cut AI digital human, KreadoAI, Star Group, Opera, Tongyi Dance King, Mo Xiaoxian, Wenxin Yige, Phenaki, Make-a-Video, Mobius, Yijian Miaochuang, Giant Sun AI, Yiying AI, D-Human digital human, Xinyi Video Large Model, Doubao, Magic Words, Blue Label Clone, Liveportrait, Meitu AI digital human DreamAvatar, Pika or Gen-2.

6. The cooking method based on the cooking AI large model and the Vincent video model according to claim 1 is characterized by: The cooking AI big model generates a new cooking method, which includes two steps: building a cooking AI big model and calling the cooking AI big model. The building of the cooking AI big model includes six steps: Step 1: Collect cooking data: Collect all information about cooking methods of ingredients presented through voice, video, text, pictures or 3D models; Step 2: Preprocessing cooking data: All collected information about cooking methods is processed to ensure its integrity and usability. This includes converting information in different formats into text and editing the text according to a specific format to facilitate subsequent training of the AI ​​model. Step 3: Select AI models applicable to cooking: Select domestic and international third-party AI models, and measure them using accuracy, response speed, and diversity indicators; Step 4: Train the cooking AI model: After organizing the cooking dataset in step 2, fine-tune the cooking dataset using a third-party AI model. After training, a cooking AI model containing all relevant data about cooking methods is generated. Step 5: Verify and test the cooking AI big model: Perform a task-specific performance test on the cooking AI big model generated in step 4. If the performance fails, repeat steps 1, 2, 3, and 4 and retrain until the performance passes. Generate a cooking AI big model and store it on the cloud platform or in the storage module of the smart cooking device. Step 6. Deploy and maintain the cooking AI big model: deploy the newly generated AI big model to the storage module of the cloud platform or intelligent cooking device, and carry out continuous maintenance and updates, and update data regularly to ensure the timeliness and accuracy of the data; the cooking demand information input by the user in voice, video, text, picture or 3D model format is used to generate a new cooking method by calling the cooking AI big model. The calling of the cooking AI big model includes assembling query statements, the AI ​​big model performing reasoning operations and the AI ​​big model returning results. The new cooking method is sent to the user's mobile terminal or the processor after the reasoning operation of the cooking AI big model is completed.

7. The cooking method based on the cooking AI large model and the Vincent video model according to claim 3 is characterized by: The method of outputting a specific virtual human through analysis and training by the AI ​​virtual human system includes constructing a specific AI virtual human and calling a specific AI virtual human. The constructing of a specific AI virtual human includes seven steps: character generation, voice generation, lip synchronization, action generation, synthetic display, test verification and deployment maintenance. The user inputs virtual human demand information in the form of voice, video, text, picture or 3D model, and the AI ​​virtual human system generates a specific AI virtual human. The calling of a specific AI virtual human includes generating text instruction content for the specific AI virtual human, transmitting the text instruction content to the AI ​​virtual human system, and the AI ​​virtual human system generates the video.

8. The cooking method based on the cooking AI large model and the Vincent video model according to claim 7 is characterized in that: The calling of a specific AI virtual person specifically includes the following steps: The first step is to generate text instructions for a specific AI virtual person. This can be generated by a large cooking AI model or by a system program based on business logic. The second step is to use an API call to pass the text command content to the cloud platform's AI virtual human system, which then generates the video. This involves the following four steps: First, voice generation: The text content is converted into the voice of a specific AI virtual human through the voice generation module of the AI ​​virtual human system, and the voice uploaded by the user can be cloned. Second, lip generation: The speech of the specific AI virtual person is synchronized with the lips of the specific AI virtual person through an algorithm, so that both Chinese and English can be synchronized. Third, action generation: select the action template uploaded by the user, output the corresponding action posture of the specific AI virtual person according to the user's action template, and combine it with the lip synchronization of the specific AI virtual person in step 2. Fourth, video synthesis: output the content generated in the first, second and third steps to a standard video format and return it.

9. An intelligent cooking device, characterized in that: An information collection module, a storage module, a processor, a display screen and a wireless communication module are provided. The information collection module, the storage module, the display screen and the wireless communication module are electrically connected to the processor. The information collection module is used to collect all information about cooking methods of ingredients or virtual human demand information input by the user in voice, video, text, picture or 3D model format. The processor is connected to the cloud platform and the user mobile terminal through the wireless communication module. The processor is used to execute the new cooking method generated by the cooking AI large model in claim 1 or 2 or 3 for cooking. The processor is also used to output the specific cooking video or the specific virtual human cooking video on the display screen or the user mobile terminal.

10. The intelligent cooking device according to claim 9, characterized in that: There are intelligent cooking device stations for stir-frying, sautéing, deep-frying, cooking, frying, sticking, roasting, braising, stewing, steaming, blanching, boiling, stewing, sautéing, mixing, marinating, roasting, braising, freezing, candied, honey-glazed, smoking, rolling, sliding or baking.

11. The intelligent cooking device according to claim 10, characterized in that: The intelligent cooking device stations for stir-frying, frying, sautéing, deep-frying, cooking, frying, sticking, roasting, braising, stewing, steaming, blanching, boiling, stewing, sautéing, mixing, marinating, roasting, braising, freezing, candied, honey-glazed, smoking, rolling, sliding or baking are provided with corresponding operation detection feedback systems, and the operation detection feedback system is used to detect whether the cooking operations performed by the user meet the requirements of the cooking method of the cooking AI large model.

12. The intelligent cooking device according to claim 9, characterized in that: A human-computer interaction module is provided, which is equipped with a voice recognition device. The human-computer interaction module is used for information interaction and intelligent control between the intelligent cooking device and the user. The intelligent control includes the user confirming cooking parameters and cooking operation steps through voice or on the display screen.

13. The intelligent cooking device according to claim 9, characterized in that: A projection device is provided, which is electrically connected to the processor and is used to project the specific cooking video or the specific virtual person cooking video outside the intelligent cooking device.

Images