Cooking assisting method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a cooking assistance method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining and playing a corresponding cooking video in response to a cooking assistance operation of a target dish; in response to a process auxiliary operation triggered for the dish process of the target dish, intercepting a process auxiliary fragment corresponding to the process auxiliary operation in the cooking video; generating corresponding process auxiliary information based on the process auxiliary fragment; and after the process auxiliary fragment is played, prompting the current cooking user to execute the corresponding cooking process based on the process auxiliary information. According to the scheme, the efficiency and convenience of assisting the user in cooking can be improved, so that the auxiliary cooking effect and the user experience feeling are improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a cooking assistance method, device, electronic device and storage medium. Background Art

[0002] With the continuous development of science and technology, smart home has gradually penetrated into all aspects of family life. For example, the emergence of smart cooking equipment such as smart rice cookers, smart induction cookers and smart cooking machines has provided many conveniences for people's lives.

[0003] At present, when users use smart cooking devices to cook, if they are not familiar with the cooking process of the dishes, they often need to search for cooking videos through smart terminals (such as mobile phones or smart tablets) to watch and learn, and cook while watching the cooking videos. However, in actual applications, since the cooking videos are not synchronized with the actual cooking process, it is easy for users to need to adjust the playback of cooking videos multiple times to assist in cooking, resulting in low cooking efficiency and poor user experience. Summary of the invention

[0004] The present application provides a cooking assistance method, device, electronic device and storage medium, which can effectively improve the effect and convenience of assisting users in cooking, and improve the auxiliary cooking effect and user experience.

[0005] In a first aspect, the present application provides a cooking assistance method, comprising:

[0006] In response to the cooking assistance operation of the target dish, the corresponding cooking video is obtained and played;

[0007] In response to a process auxiliary operation triggered for a dish process of a target dish, intercepting a process auxiliary segment corresponding to the process auxiliary operation in the cooking video;

[0008] generating corresponding process auxiliary information based on the process auxiliary fragment;

[0009] After playing the process auxiliary clip, the current cooking user is prompted to perform the corresponding cooking process based on the process auxiliary information.

[0010] In a second aspect, the present application provides a cooking auxiliary device, comprising:

[0011] An acquisition module, used to respond to the cooking auxiliary operation of the target dish, acquire the corresponding cooking video and play it;

[0012] An interception module, configured to intercept, in response to a process auxiliary operation triggered by a dish process of a target dish, a process auxiliary segment corresponding to the process auxiliary operation in the cooking video;

[0013] A generating module, used for generating corresponding process auxiliary information based on the process auxiliary fragment;

[0014] The instruction module is used to prompt the current cooking user to perform the corresponding cooking process based on the process auxiliary information after playing the process auxiliary clip.

[0015] Optionally, in some embodiments of the present application, the interception module includes:

[0016] A first acquisition unit, used to acquire the voiceprint data of the current cooking user;

[0017] A second acquisition unit, configured to acquire a voice command corresponding to a current cooking user based on the voiceprint data;

[0018] A recognition unit, used to recognize the target dish and its cooking process corresponding to the voice instruction through a semantic recognition model;

[0019] A request unit, used for generating a corresponding process auxiliary request for the target dish and its process;

[0020] A triggering unit is used to trigger a process assistance operation corresponding to the process assistance request in response to the process assistance request.

[0021] Optionally, in some embodiments of the present application, the second acquiring unit is specifically configured to:

[0022] Acquire a voice command set at a current time, wherein the voice command set includes at least one user voice command;

[0023] Based on the voiceprint data of the current cooking user, the voice command corresponding to the current cooking user is matched in the voice command set.

[0024] Optionally, in some embodiments of the present application, the interception module further includes:

[0025] A first frame segmentation unit, configured to perform frame segmentation processing on the cooking video to obtain a plurality of cooking video segments;

[0026] A classification unit, configured to classify and label the plurality of cooking video clips according to a preset classification strategy, and obtain a classification label corresponding to each cooking video clip;

[0027] A first extraction unit, used to extract keywords corresponding to the process auxiliary operation;

[0028] The interception unit is used to intercept a plurality of cooking video frames corresponding to the keyword in the cooking video based on the keyword and the classification label as the process auxiliary segments corresponding to the process auxiliary operation.

[0029] Optionally, in some embodiments of the present application, the generating module includes:

[0030] A second frame division unit is used to perform frame division processing on the process auxiliary segment to obtain a plurality of cooking video segments;

[0031] A second extraction unit is used to extract cooking information from each cooking video frame;

[0032] The generating unit is used to generate corresponding process auxiliary information based on the cooking information in the plurality of frames of cooking video.

[0033] Optionally, in some embodiments of the present application, the generating unit is specifically used to:

[0034] Performing text recognition on the cooking information in each frame of the cooking video to obtain a process auxiliary text corresponding to each frame of the cooking video;

[0035] Extracting voice information from the plurality of frames of cooking video for voice recognition to obtain process auxiliary voice corresponding to the cooking video;

[0036] Based on the process assistance text and process assistance voice, corresponding process assistance information is generated.

[0037] Optionally, in some embodiments of the present application, the cooking auxiliary device further includes a display module, and the display module is specifically used for:

[0038] Displaying a prompt interface containing auxiliary information of the process;

[0039] In response to a user's selection operation on the prompt interface, current material preparation information is acquired, and based on the material preparation information and the process auxiliary information, prompt information for replenishing or replacing ingredients is output.

[0040] In a third aspect, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned cooking assistance method when executing the program.

[0041] In a fourth aspect, the present application further provides a storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above-mentioned cooking assistance method when executed by a processor.

[0042] As described above, the present application provides a cooking assistance method, device, electronic device and storage medium, which, in response to the cooking assistance operation of the target dish, obtains and plays the corresponding cooking video; in response to the process assistance operation triggered by the dish process of the target dish, intercepts the process assistance segment corresponding to the process assistance operation in the cooking video; generates corresponding process assistance information based on the process assistance segment; after playing the process assistance segment, prompts the current cooking user to perform the corresponding cooking process based on the process assistance information. In the cooking assistance scheme provided by the present application, in response to the cooking assistance operation triggered by the user's voice command, first obtain and play the cooking assistance video to assist the user in cooking; then, after obtaining the dish process of the target dish in the user's command, intercept the process assistance segment corresponding to the dish process in the cooking video, thereby generating the corresponding process assistance information, which can not only accurately assist the user in cooking by playing the process assistance segment, but also prompt the user to continue cooking through the process assistance information after the process assistance segment is played, thereby solving the problem in the prior art that the cooking video and the actual cooking progress of the user are inconsistent, which requires multiple adjustments and viewings, effectively improves the effect and convenience of assisting the user in cooking, improves the auxiliary cooking effect, and enables the user to have a better experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] in:

[0045] Figure 1 It is a scene diagram of the cooking assistance method provided in an embodiment of the present application.

[0046] Figure 2 is a flow chart of a cooking assistance method provided in an embodiment of the present application;

[0047] Figure 3 is a schematic diagram of a scenario for obtaining user instructions provided in an embodiment of the present application;

[0048] Figure 4 is a schematic diagram of a prompt interface in a cooking assistance method provided in an embodiment of the present application;

[0049] Figure 5 is another schematic diagram of a prompt interface in the cooking assistance method provided in an embodiment of the present application;

[0050] Figure 6is another schematic diagram of a prompt interface in the cooking assistance method provided in an embodiment of the present application;

[0051] Figure 7 is a schematic diagram of the structure of a cooking auxiliary device provided in an embodiment of the present application;

[0052] Figure 8 is another structural schematic diagram of the cooking auxiliary device provided in an embodiment of the present application;

[0053] Fig. 9 It is a structural schematic diagram of the intelligent cooking machine provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0055] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0057] The present application provides a cooking assistance method, device, electronic device and storage medium.

[0058] The cooking auxiliary device can be integrated into an electronic device, which can be a smart phone, a tablet computer, a laptop computer, a desktop computer or an intelligent cooking machine, but is not limited thereto. The electronic device can be directly or indirectly connected to the server through wired or wireless communication. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. This application does not limit this.

[0059] For example, see Figure 1The cooking assistance device is integrated in the intelligent cooking machine 10. The display screen of the intelligent cooking machine 10 can play cooking videos and process assistance clips, and display process assistance information. In addition, the user can interact with the intelligent cooking machine 10 by touching the display screen of the intelligent cooking machine 10. In the process of cooking assistance, the intelligent cooking machine 10 responds to the cooking assistance operation of the target dish, obtains the corresponding cooking video and plays it; then, the intelligent cooking machine 10 responds to the process assistance operation triggered by the dish process of the target dish, and intercepts the process assistance clip corresponding to the process assistance operation in the cooking video; then, the corresponding process assistance information is generated based on the process assistance clip; finally, after the intelligent cooking machine 10 plays the process assistance clip, it prompts the current cooking user to perform the corresponding cooking process based on the process assistance information.

[0060] The cooking assistance method provided by the present application first obtains and plays a cooking assistance video to assist the user in cooking; then, after obtaining the dish process of the target dish in the user's instruction, the process assistance segment corresponding to the dish process is intercepted in the cooking video, thereby generating corresponding process assistance information. Not only can the user be accurately assisted in cooking by playing the process assistance segment, but also the user can be prompted to continue cooking through the process assistance information after the process assistance segment is played. This solves the problem in the prior art that multiple adjustments and viewings are required due to the inconsistency between the cooking video and the user's actual cooking progress, effectively improves the effect and convenience of assisting users in cooking, improves the assisted cooking effect, and enables users to have a better experience.

[0061] It should be noted that the order of description of the following embodiments is not intended to limit the priority order of the embodiments.

[0062] A cooking assistance method comprises: in response to a cooking assistance operation of a target dish, obtaining a corresponding cooking video and playing it; in response to a process assistance operation triggered by a dish process of the target dish, intercepting a process assistance segment corresponding to the process assistance operation in the cooking video; generating corresponding process assistance information based on the process assistance segment; after playing the process assistance segment, prompting the current cooking user to perform the corresponding cooking process based on the process assistance information.

[0063] See also Figure 2 , Figure 2 This is a schematic diagram of the cooking assistance method provided in this application. The specific process of the cooking assistance method can be as follows:

[0064] 101. In response to a cooking auxiliary operation of a target dish, obtain and play a corresponding cooking video.

[0065] The cooking assistance operation may be triggered by the user, for example, a cooking assistance button is displayed on the display screen of the intelligent cooking machine, and in response to the user clicking the cooking assistance button, a cooking assistance instruction for the target dish is triggered, and then the cooking assistance operation is triggered according to the cooking assistance instruction. In addition, the user may also pre-set the cooking assistance time in the intelligent cooking machine, that is, when the current time reaches the cooking assistance time, the cooking assistance operation is triggered, and the cooking assistance time may be set by the user or by the system. Optionally, in some embodiments, a cooking assistance instruction issued by the user through the intelligent terminal may be received, and the cooking assistance operation may be triggered by monitoring whether the cooking assistance instruction initiated by the user is received, that is, in response to the cooking assistance operation, this step may not only be triggered by pre-setting the trigger time of the intelligent cooking machine, but may also be triggered by receiving the cooking assistance instruction triggered by the user clicking the cooking assistance button at the current moment, or may be triggered by receiving the cooking assistance instruction sent by the user at the current moment. After responding to the cooking assistance operation of the target dish, the smart cooking machine obtains the corresponding cooking video according to the target dish and plays it, wherein the cooking video can be obtained from the cloud server through crawler technology and downloaded to the smart cooking machine, or it can be obtained from the local database of the smart cooking machine. The cooking video includes reference videos of cooking processes at different stages corresponding to different dishes, such as the cooking reference process of the target dish at a certain cooking stage. After obtaining the cooking video corresponding to the target dish, it is played through the display screen of the smart cooking machine. For example, after the user initiates a cooking assistance instruction for potatoes in "Stir-fried Potatoes with Chicken", the smart cooking machine obtains the cooking assistance video for potatoes in "Stir-fried Potatoes with Chicken" from the cloud server or local database and plays it to assist the user in cooking the potatoes.

[0066] In addition, after responding to the cooking assistance operation, the current cooking menu will also be obtained. The cooking menu can be obtained from the cloud server through the network or from the local database. The cooking menu includes cooking dish information, cooking dish sorting, and preparation information and cooking processes corresponding to different cooking dishes. By obtaining the current cooking menu, the next dish to be cooked after the target dish is determined, and the cooking video of the next dish to be cooked is obtained, so that the corresponding cooking video can be played when the next dish to be cooked is cooked later, without the user having to manually initiate the cooking assistance command again to obtain the corresponding cooking video when the next dish to be cooked needs to be cooked, thereby improving the convenience and efficiency of cooking assistance. For example, when the user is cooking potatoes in "Stir-fried Potatoes with Chicken" at the current time, the smart cooking machine obtains the cooking video corresponding to the next dish to be cooked, the chicken, so that the pre-acquired cooking video can be played immediately when the chicken is cooked later.

[0067] 102. In response to a process auxiliary operation triggered by a dish process of a target dish, a process auxiliary segment corresponding to the process auxiliary operation is captured in the cooking video.

[0068] Among them, the process auxiliary operation is triggered by the user for the cooking process of the target dish. When the smart cooking machine obtains the voice instruction for the cooking process of the target dish issued by the current cooking user, it responds to the process auxiliary operation triggered by the cooking process of the target dish, thereby intercepting the process auxiliary segment corresponding to the process auxiliary operation in the cooking video of the target dish that has been obtained in advance. Figure 3 As shown, when the cooking user issues a voice command of "how to fry potatoes", the intelligent cooking machine receives the voice command and triggers the corresponding "fry potatoes" process auxiliary operation in response to the voice command, thereby obtaining the process auxiliary segment about frying potatoes in the cooking video, so as to subsequently play the process auxiliary segment or extract the corresponding process auxiliary message to prompt the user, thereby assisting the user in cooking.

[0069] 103. Generate corresponding process auxiliary information based on the process auxiliary fragment.

[0070] Specifically, according to the process auxiliary segment corresponding to the dish process of the target dish, the corresponding process auxiliary text, voice and other data are extracted from the process auxiliary segment, so as to comprehensively generate process auxiliary information. Among them, the process auxiliary information can be process auxiliary text, process auxiliary voice, or process auxiliary text plus process auxiliary voice, which is not specifically limited here. For example, after obtaining the voice instruction of "how to fry potatoes" issued by the user, the intelligent cooking machine obtains the process auxiliary segment about frying potatoes in the cooking video, and then extracts the corresponding text and voice data from the process auxiliary segment of frying potatoes, so as to generate process auxiliary information about frying potatoes.

[0071] 104. After playing the process auxiliary clip, prompt the current cooking user to perform the corresponding cooking process based on the process auxiliary information.

[0072] Specifically, after the intelligent cooking machine responds to the process assistance operation triggered for the dish process of the target dish, and obtains the process assistance information generated based on the process assistance clip, it prompts the current cooking user how to perform the corresponding cooking process by displaying or playing the process assistance information, so as to achieve the effect of accurate cooking assistance and improve the synchronization and convenience of cooking assistance. For example, after the process assistance clip corresponding to the dish process of the target dish is played by the intelligent cooking machine, in order to avoid the situation where the user needs to adjust again due to missing the process assistance clip, the process assistance information corresponding to the process assistance clip can be played or displayed to prompt the user for the next operation. For example, after the intelligent cooking machine has played the process assistance clip of fried potatoes, the process assistance information of fried potatoes is displayed on the display screen of the intelligent cooking machine, and / or the process assistance information of fried potatoes is played by the intelligent cooking machine, so that the current cooking user can be prompted how to perform the cooking process of fried potatoes, without the user having to adjust the progress of the cooking video again to watch how to cook the dish, which greatly improves the convenience of cooking assistance.

[0073] Optionally, in some embodiments, the step of “responding to a process auxiliary operation triggered by a dish process for a target dish” may specifically include:

[0074] (11) Obtaining the voiceprint data of the current cooking user;

[0075] (12) Based on the voiceprint data, obtain the voice command corresponding to the current cooking user;

[0076] (13) identifying the target dish and its cooking process corresponding to the voice command through a semantic recognition model;

[0077] (14) generating corresponding process auxiliary requests for the target dish and its process;

[0078] (15) In response to a process assistance request, triggering a process assistance operation corresponding to the process assistance request.

[0079] Specifically, after responding to the cooking assistance operation of the target dish, the smart cooking machine obtains the cooking video corresponding to the cooking assistance operation and plays it, and will pre-identify the voiceprint of the current cooking user. First, the voiceprint data of the current cooking user is obtained for subsequent execution of the voice command of the current cooking user based on the voiceprint data; based on the pre-identified voiceprint data, the voice command issued by the current cooking user is collected, and then, the currently obtained voice command is recognized through the semantic recognition model to identify the dish corresponding to the voice command and its dish process. For example, after the user issues a voice command of "how to fry potatoes", the semantic recognition model is used to identify that the target dish corresponding to the voice command of "how to fry potatoes" is "potatoes", and its dish process is "frying"; then, for the target dish and its dish process, a corresponding process assistance request is automatically generated, for example, for potatoes and their "frying" process, a corresponding process assistance request of "frying potatoes" is generated; finally, in response to the process assistance request triggered by the dish process of the target dish, the process assistance operation corresponding to the process assistance request is triggered.

[0080] In addition, in this embodiment, in order to improve the accuracy and flexibility of user voice command recognition, the acquired user voice commands can be recognized through a semantic recognition model. In order to improve the recognition accuracy of the semantic recognition model, it is necessary to expand the data set of the voice recognition model by expanding the existing local nouns with synonyms and antonyms. For example, "potato" can be expanded to "potato", "potato", "potato", etc., so as to facilitate the subsequent accurate and flexible recognition of nouns in user voice commands. Similarly, in addition to expanding the nouns of dishes, the names of seasonings can also be expanded, or the nouns of processes can be expanded, which are not specifically limited here.

[0081] Optionally, in some embodiments, the step of “obtaining the voice command corresponding to the current cooking user based on the voiceprint data” may specifically include:

[0082] (121) obtaining a voice command set at a current time, the voice command set including at least one user voice command;

[0083] (122) Based on the voiceprint data of the current cooking user, match the voice command corresponding to the current cooking user in the voice command set.

[0084] Specifically, in the present embodiment, the voiceprint data of the current cooking user is first acquired and recognized in advance. In the actual cooking assistance process, since the voices of other users besides the cooking user may appear, in order to avoid misidentifying the voice commands of other users and thus executing the wrong actions, the present embodiment only executes the control actions corresponding to the voice commands of the current cooking user. During the user's cooking process, the intelligent cooking machine acquires a voice command set at the current time, wherein the voice command set includes at least one user voice command. Then, based on the pre-recognized voiceprint data of the current cooking user, the voice command corresponding to the voiceprint data of the current cooking user is matched in the voice command set. Thus, the voice commands issued by the current cooking user can be accurately identified in a noisy cooking environment and the corresponding control actions can be executed, thus avoiding being affected by the surrounding environment and other users.

[0085] Optionally, in some embodiments, the step of “cutting off the process auxiliary segments corresponding to the process auxiliary operations in the cooking video” may specifically include:

[0086] (21) performing frame processing on the cooking video to obtain multiple cooking video clips;

[0087] (22) classifying and labeling the multiple cooking video clips according to a preset classification strategy to obtain a classification label corresponding to each cooking video clip;

[0088] (23) Extract keywords corresponding to auxiliary operations of the process;

[0089] (24) Based on the keywords and classification labels, several frames of cooking video clips corresponding to the keywords are captured in the cooking video as process auxiliary clips corresponding to the process auxiliary operations.

[0090] Specifically, after the intelligent cooking machine responds to the process auxiliary operation initiated by the user for the dish process of the target dish, the intelligent cooking machine captures the process auxiliary segment corresponding to the process auxiliary operation from the cooking video that is pre-acquired and played. First, the cooking video acquired in response to the cooking auxiliary operation is framed to obtain multiple cooking video segments after framing. Then, according to a pre-set classification strategy, each frame of the cooking video segment after framing is classified and labeled. For example, the cooking video segments can be classified according to different dishes and their different processing steps. After the classification is completed, each frame of the cooking video segment is labeled to mark the category of each frame of the cooking video segment. Then, the corresponding keywords in the process auxiliary operation are extracted, such as the target dish and the processing step. After determining the keywords, the required corresponding classification labels are found according to the keywords, so as to capture the required several frames of cooking video segments in the cooking video according to the classification labels. Finally, several frames of cooking video segments are integrated to generate the process auxiliary segment corresponding to this process auxiliary operation.

[0091] This embodiment divides the cooking video into frames, classifies and labels it, which not only facilitates the subsequent matching of corresponding cooking video clips according to keywords extracted from the process auxiliary operations, thereby improving the matching efficiency of cooking video clips, but also can further generate process auxiliary clips corresponding to the process auxiliary operations, thereby improving the accuracy of cooking assistance.

[0092] Optionally, in some embodiments, the step of “generating corresponding process auxiliary information based on the process auxiliary fragment” may specifically include:

[0093] (31) performing frame processing on the process auxiliary segment to obtain a plurality of cooking video segments;

[0094] (32) extracting cooking information from each frame of the cooking video clip;

[0095] (33) Generate corresponding process auxiliary information based on cooking information in several frames of cooking video.

[0096] Specifically, after the process auxiliary segment corresponding to the process auxiliary operation is generated, the process auxiliary segment is firstly framed again. After the frame division is completed, a multi-frame cooking video segment is obtained. Then, cooking information is extracted from each frame of the cooking video. The cooking information includes information such as the cooking process of the target dish. Finally, based on the cooking information extracted from each frame of the cooking video, the corresponding process auxiliary information is summarized and generated, thereby achieving the effect of prompting the user to assist in cooking. This embodiment can effectively assist the user in cooking by obtaining the cooking information corresponding to each frame from the process auxiliary segment, thereby generating the cooking information corresponding to the current cooking auxiliary operation.

[0097] Optionally, in some embodiments, the step of “generating corresponding process auxiliary information based on cooking information in a plurality of frames of cooking videos” may specifically include:

[0098] (41) performing text recognition on the cooking information in each frame of the cooking video to obtain a process auxiliary text corresponding to each frame of the cooking video;

[0099] (42) extracting voice information from a plurality of frames of a cooking video for voice recognition to obtain a process-assisted voice corresponding to the cooking video;

[0100] (43) Based on the process auxiliary text and process auxiliary voice, generate corresponding process auxiliary information.

[0101] Among them, after extracting the cooking information of multiple frames of cooking videos, the present embodiment also provides further identification and extraction of the cooking information. First, text recognition is performed on the cooking information in each frame of the cooking video after framing. For example, by extracting the subtitles in each frame of the cooking video or the keywords, key tags and other information in the video screen, the process auxiliary text corresponding to each frame of the cooking video is obtained. For example, the required number of potatoes and processing precautions text appear in the current frame of the cooking video. By performing text recognition on the current frame of the cooking video, the corresponding process auxiliary text is extracted; then, the voice information in each frame of the cooking video is extracted and voice recognition is performed, so as to identify the process auxiliary voice corresponding to each frame of the cooking video; finally, the process auxiliary text and process auxiliary voice corresponding to all frames of the cooking video are integrated. During the integration process, the process auxiliary text can be deduplicated, or the process auxiliary voice can be voice-to-text processed, and finally the process auxiliary information corresponding to the current cooking auxiliary operation is integrated, so as to prompt the user to perform subsequent cooking processes, thereby improving the effect and convenience of cooking assistance.

[0102] Further, after prompting the current cooking user to perform the corresponding cooking process based on the process auxiliary information, a prompt interface may be displayed, and then the user may obtain the process auxiliary information through the prompt interface to perform the next cooking action, that is, optionally, in some embodiments, after the step of "prompting the current cooking user to perform the corresponding cooking process based on the process auxiliary information", it may also specifically include:

[0103] (51) Displaying a prompt interface containing process auxiliary information;

[0104] (52) In response to the user's selection operation on the prompt interface, the current material preparation information is obtained, and based on the material preparation information and process auxiliary information, prompt information for replenishing or replacing ingredients is output.

[0105] For example, see Figure 4 The present application provides a prompt interface, which displays auxiliary process information, including the required quantity of the target dish and the processing process, etc. The auxiliary process information can be prompted by voice broadcast and text, or by video playback and text, such as Figure 5 As shown, the process auxiliary text of "frying potatoes" is displayed through the prompt interface. Figure 6As shown, a process auxiliary video of "frying potatoes" is displayed through the prompt interface; the process auxiliary voice of "frying potatoes" can also be played through the speaker of the intelligent cooking machine, so that the user can adjust the cooking process in real time according to the process auxiliary information. In addition, this embodiment also provides a preparation reminder function for cooking dishes. In response to the user's selection operation on the prompt interface, the intelligent cooking machine obtains the preparation information of the current target dish, and based on the preparation information and process auxiliary information, outputs reminder information for replenishing or replacing ingredients, or plays reminder voice for replenishing or replacing ingredients.

[0106] Furthermore, in some embodiments, the embodiments of the present application also provide the functions of dish replacement recommendation and menu recommendation. After obtaining the preparation information of the current target dish, if it is monitored that the current target dish is insufficiently prepared, the existing preparation information can be obtained to recommend corresponding substitutes and output prompt information for replacing ingredients. In addition, based on the existing preparation information, several menus with a dish matching degree greater than a preset threshold can be obtained from a cloud server or from a local database through the network, thereby outputting prompt information for recommending alternative menus, providing users with options, and improving the flexibility of assisted cooking.

[0107] The above is the cooking assistance process of this application.

[0108] As described above, the present application provides a cooking assistance method, in response to the cooking assistance operation of the target dish, the corresponding cooking video is obtained and played; in response to the process assistance operation triggered for the dish process of the target dish, the process assistance segment corresponding to the process assistance operation is intercepted in the cooking video; the corresponding process assistance information is generated based on the process assistance segment; after playing the process assistance segment, the current cooking user is prompted to perform the corresponding cooking process based on the process assistance information. In the cooking assistance scheme provided by the present application, in response to the cooking assistance operation triggered according to the user's voice command, the cooking assistance video is first obtained and played to assist the user in cooking; then, after obtaining the dish process of the target dish in the user's command, the process assistance segment corresponding to the dish process is intercepted in the cooking video, thereby generating the corresponding process assistance information, which can not only accurately assist the user in cooking by playing the process assistance segment, but also prompt the user to continue cooking through the process assistance information after the process assistance segment is played, thereby solving the problem in the prior art that the cooking video and the actual cooking progress of the user are inconsistent, which requires multiple adjustments and viewings, effectively improving the effect and convenience of assisting the user in cooking, improving the auxiliary cooking effect, and allowing the user to have a better experience.

[0109] In order to facilitate better implementation of the cooking assistance method of the embodiment of the present application, the embodiment of the present invention also provides a device based on the above cooking assistance method (hereinafter referred to as the assistance device). The meanings of the terms are the same as those in the above cooking assistance method, and the specific implementation details can refer to the description in the method embodiment.

[0110] See also Figure 7 , Figure 7 This is a schematic diagram of the structure of a cooking auxiliary device provided in an embodiment of the present application, wherein the adjustment device may include an acquisition module 201, an interception module 202, a generation module 203 and an indication module 204.

[0111] The acquisition module 201 is used to respond to the cooking auxiliary operation of the target dish, acquire the corresponding cooking video and play it.

[0112] For the acquisition module 201, it mainly responds to the user's click operation on the cooking assistance button, triggers the cooking assistance instructions for the target dish, and then triggers the cooking assistance operation according to the cooking assistance instructions, and then obtains the corresponding cooking video according to the target dish and plays it.

[0113] The interception module 202 is used to respond to the process auxiliary operation triggered by the dish process of the target dish, and intercept the process auxiliary segment corresponding to the process auxiliary operation in the cooking video.

[0114] For the interception module 202, after obtaining the voice instructions for the cooking process of the target dish issued by the current cooking user, it responds to the process auxiliary operation triggered by the cooking process of the target dish, thereby intercepting the process auxiliary segment corresponding to the process auxiliary operation in the cooking video of the target dish that has been obtained in advance.

[0115] The generating module 203 is used to generate corresponding process auxiliary information based on the process auxiliary fragment.

[0116] For the generation module 203, according to the process auxiliary fragment corresponding to the dish process of the target dish, the corresponding process auxiliary text, voice and other data are extracted from the process auxiliary fragment, so as to comprehensively generate process auxiliary information.

[0117] The instruction module 204 is used to prompt the current cooking user to perform the corresponding cooking process based on the process auxiliary information after playing the process auxiliary clip.

[0118] For the instruction module 204, after obtaining the process auxiliary information generated based on the process auxiliary fragment, the process auxiliary information is displayed or played to prompt the current cooking user how to perform the corresponding cooking process, thereby achieving the effect of precise cooking assistance and improving the synchronization and convenience of cooking assistance.

[0119] Optionally, in some embodiments, the interception module 202 may specifically include:

[0120] A first acquisition unit, used to acquire the voiceprint data of the current cooking user;

[0121] A second acquisition unit, configured to acquire a voice command corresponding to the current cooking user based on the voiceprint data;

[0122] A recognition unit, used to recognize the target dish and its cooking process corresponding to the voice command through a semantic recognition model;

[0123] A request unit, used for generating corresponding process auxiliary requests for target dishes and their process;

[0124] The trigger unit is used to trigger the process assistance operation corresponding to the process assistance request in response to the process assistance request.

[0125] Optionally, in some embodiments, the second acquiring unit is specifically configured to:

[0126] Obtaining a voice command set at a current time, where the voice command set includes at least one user voice command;

[0127] Based on the voiceprint data of the current cooking user, the voice command corresponding to the current cooking user is matched in the voice command set.

[0128] Optionally, in some embodiments, the interception module 202 may further include:

[0129] A first frame segmentation unit is used to perform frame segmentation processing on the cooking video to obtain multiple cooking video segments;

[0130] A classification unit, used to classify and label the multiple cooking video clips according to a preset classification strategy to obtain a classification label corresponding to each cooking video clip;

[0131] A first extraction unit is used to extract keywords corresponding to the auxiliary operations of the process;

[0132] The interception unit is used to intercept a number of cooking video clips corresponding to the keywords in the cooking video based on the keywords and classification tags as process auxiliary clips corresponding to the process auxiliary operations.

[0133] Optionally, in some embodiments, the generating module 203 may specifically include:

[0134] A second frame division unit is used to perform frame division processing on the process auxiliary segment to obtain a plurality of cooking video segments;

[0135] A second extraction unit is used to extract cooking information from each cooking video frame;

[0136] The generating unit is used to generate corresponding process auxiliary information based on cooking information in a plurality of frames of cooking videos.

[0137] Optionally, in some embodiments, the generating unit is specifically used for:

[0138] Perform text recognition on the cooking information in each frame of the cooking video to obtain the process auxiliary text corresponding to each frame of the cooking video;

[0139] Extract voice information from several frames of cooking videos for voice recognition to obtain process-assisted voice corresponding to the cooking videos;

[0140] Based on the process auxiliary text and process auxiliary voice, corresponding process auxiliary information is generated.

[0141] Optionally, in some embodiments, see Figure 8 The auxiliary device of the present application may further include a display module 205, which is specifically used to: display a prompt interface containing process auxiliary information; obtain current material preparation information in response to a user's selection operation on the prompt interface, and output prompt information to remind users to replenish or replace ingredients based on the material preparation information and process auxiliary information.

[0142] It can be seen that the embodiment of the present application provides a cooking assistance device, in which the acquisition module 201 responds to the cooking assistance operation of the target dish, obtains the corresponding cooking video and plays it, the interception module 202 responds to the process assistance operation triggered by the dish process of the target dish, intercepts the process assistance segment corresponding to the process assistance operation in the cooking video, and generates the corresponding process assistance information based on the process assistance segment through the generation module 203. Finally, after the process assistance segment is played through the indication module 204, the current cooking user is prompted to perform the corresponding cooking process based on the process assistance information. In the cooking assistance solution provided in the present application, after responding to the cooking assistance operation triggered by the user's voice command, a cooking assistance video is first obtained and played to assist the user in cooking; then, after obtaining the dish process of the target dish in the user's command, the process assistance segment corresponding to the dish process is captured in the cooking video, thereby generating corresponding process assistance information. Not only can the user be accurately assisted in cooking by playing the process assistance segment, but the user can also be prompted to continue cooking through the process assistance information after the process assistance segment is played. This solves the problem in the prior art that multiple adjustments and viewings are required due to the inconsistency between the cooking video and the user's actual cooking progress, effectively improving the effect and convenience of assisting users in cooking, improving the assisted cooking effect, and allowing users to have a better experience.

[0143] It should be noted that in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM, Read Only Memory), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0144] Correspondingly, an embodiment of the present application also discloses a storage medium storing a computer program that can be loaded by a processor and execute the above-mentioned cooking assistance method.

[0145] The present application also discloses a smart cooking device, such as Fig. 9 As shown, it includes a processor 100, at least one communication bus 200, a user interface 300, at least one external communication interface 400 and a memory 500. The processor 100 is used to provide computing and control capabilities, the communication bus 200 is configured to achieve connection and communication between these components, the user interface 300 may include a display screen, the external communication interface 400 may include a standard wired interface and a wireless interface, and the memory 500 stores a cooking assistance method. The processor 100 is also used to execute the steps of the cooking assistance method stored in the memory 500.

[0146] Optionally, in some embodiments, the processor implements the following steps when executing the computer program:

[0147] In response to the cooking assistance operation of the target dish, the corresponding cooking video is obtained and played; in response to the process assistance operation triggered by the dish process of the target dish, the process assistance segment corresponding to the process assistance operation is intercepted in the cooking video; the corresponding process assistance information is generated based on the process assistance segment; after playing the process assistance segment, the current cooking user is prompted to perform the corresponding cooking process based on the process assistance information.

[0148] In the embodiments of the present application, not only can the user be accurately assisted in cooking by playing the process assistance clips, but the user can also be prompted to continue cooking through the process assistance information after the process assistance clips are played, thereby solving the problem in the prior art that multiple adjustments and viewings are required due to the inconsistency between the cooking video and the user's actual cooking progress, effectively improving the effect and convenience of assisting the user in cooking, improving the assisted cooking effect, and allowing users to have a better experience.

[0149] It should be understood that "one embodiment" or "some embodiments" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments.

[0150] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0151] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0152] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0153] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0154] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks or optical disks.

[0155] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or in other words, the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks or optical disks.

[0156] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A cooking auxiliary method, characterized in that: include: In response to the cooking assistance operation of the target dish, the corresponding cooking video is obtained and played; In response to a process auxiliary operation triggered for a dish process of a target dish, intercepting a process auxiliary segment corresponding to the process auxiliary operation in the cooking video; generating corresponding process auxiliary information based on the process auxiliary fragment; After playing the process auxiliary clip, the current cooking user is prompted to perform the corresponding cooking process based on the process auxiliary information.

2. The cooking assistance method according to claim 1, characterized in that: The process auxiliary operation triggered in response to the dish process of the target dish includes: Get the voiceprint data of the current cooking user; Based on the voiceprint data, obtaining a voice command corresponding to the current cooking user; Identify the target dish and its cooking process corresponding to the voice command through a semantic recognition model; Generate a corresponding process auxiliary request for the target dish and its process; In response to the process assistance request, a process assistance operation corresponding to the process assistance request is triggered.

3. The cooking auxiliary method according to claim 2, characterized in that: The acquiring, based on the voiceprint data, a voice command corresponding to the current cooking user includes: Acquire a voice command set at a current time, wherein the voice command set includes at least one user voice command; Based on the voiceprint data of the current cooking user, the voice command corresponding to the current cooking user is matched in the voice command set.

4. The cooking auxiliary method according to claim 1, characterized in that: The step of intercepting the process auxiliary segment corresponding to the process auxiliary operation in the cooking video includes: Performing frame processing on the cooking video to obtain multiple cooking video clips; According to a preset classification strategy, the multiple cooking video clips are classified and labeled to obtain a classification label corresponding to each cooking video clip; Extracting keywords corresponding to the auxiliary operations of the process; Based on the keywords and classification labels, several frames of cooking video clips corresponding to the keywords are intercepted in the cooking video as process auxiliary clips corresponding to the process auxiliary operations.

5. The cooking auxiliary method according to claim 1, characterized in that: The generating corresponding process auxiliary information based on the process auxiliary fragment includes: Performing frame processing on the process auxiliary segment to obtain a plurality of cooking video segments; Extract cooking information from each frame of cooking video clip respectively; Based on the cooking information in the several frames of cooking video, corresponding process auxiliary information is generated.

6. The cooking auxiliary method according to claim 5, characterized in that: The generating corresponding process auxiliary information based on the cooking information in the plurality of frames of cooking videos includes: Performing text recognition on the cooking information in each frame of the cooking video to obtain a process auxiliary text corresponding to each frame of the cooking video; Extracting voice information from the plurality of frames of cooking video for voice recognition to obtain process auxiliary voice corresponding to the cooking video; Based on the process assistance text and process assistance voice, corresponding process assistance information is generated.

7. The cooking auxiliary method according to any one of claims 1 to 6, characterized in that: After prompting the current cooking user to perform the corresponding cooking process based on the process auxiliary information, the method further includes: Displaying a prompt interface containing auxiliary information of the process; In response to a user's selection operation on the prompt interface, current material preparation information is acquired, and based on the material preparation information and the process auxiliary information, prompt information for replenishing or replacing ingredients is output.

8. A cooking auxiliary device, characterized in that: include: An acquisition module, used to respond to the cooking auxiliary operation of the target dish, acquire the corresponding cooking video and play it; An interception module, configured to intercept, in response to a process auxiliary operation triggered by a dish process of a target dish, a process auxiliary segment corresponding to the process auxiliary operation in the cooking video; A generating module, used for generating corresponding process auxiliary information based on the process auxiliary fragment; The instruction module is used to prompt the current cooking user to perform the corresponding cooking process based on the process auxiliary information after playing the process auxiliary clip.

9. An electronic device, characterized in that: include: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the cooking assistance method according to any one of claims 1 to 7 are implemented.

10. A storage medium, characterized in that: A computer program is stored thereon, wherein when the computer program is executed by a processor, the steps of the cooking assistance method according to any one of claims 1 to 7 are implemented.

Images