A recipe generation method, apparatus, electronic device, and storage medium

Abstract

The present application provides a recipe generation method, apparatus, electronic device and storage medium, relating to the technical field of recipe generation. The key points of its technical solution are as follows: only obtain cooking videos with complete cooking time series and containing cooking operation step picture information and / or cooking operation step sound information; play the cooking video on the user terminal and automatically generate an editing interface, the editing interface at least includes an input area and a label area, and recognition labels are placed in the label area; obtain the recognition labels from the signature area input by the user in the input area and the corresponding text information; add the recognition labels together with the corresponding text information to the cooking time series to obtain cooking process information; generate a recipe that can be recognized and executed by a cooking robot according to the cooking process information. The recipe generation method, apparatus, electronic device and storage medium provided by the present application have the advantages of being convenient for users to make recipes and having high applicability.

Description

Technical Field

[0001] The present application relates to the technical field of recipe generation, and in particular to a recipe generation method, device, electronic device and storage medium. Background Art

[0002] With the development of technology, cooking robots have gradually become popular. Besides commercial applications, more and more individual users are also using them to cook. However, existing cooking robots generally only follow pre-set recipes. These recipes are usually limited in number, and the variety and quantity of these robots are gradually failing to meet user needs. Therefore, some existing technologies have proposed recipe generation methods and recipe sharing methods for cooking robots.

[0003] Recipe sharing typically involves users sharing their own recipes, allowing other users to download the shared recipes online and send them to a cooking robot, which then cooks the dishes according to the downloaded recipes. However, this approach often results in users having different tastes, making it difficult for users to create dishes that suit their tastes after downloading other users' recipes.

[0004] Therefore, for users, the preferred method is to create their own recipes. The advantage of creating your own recipes is that you can make them completely according to your own tastes, and ultimately the dishes cooked by the cooking robot will be as close to your tastes as possible, which can bring a better experience when using the cooking robot. However, for users to create their own recipes and create recipes that can be recognized and executed by the cooking robot is a very difficult task for the vast majority of users. In order to create a recipe that can be recognized and executed by the cooking robot, it is usually necessary to enter a large number of cooking parameters, which are very abstract for ordinary users. In addition, the cooking skills of users who use cooking robots are generally low, making it difficult for them to create their own recipes.

[0005] In this regard, with the development of intelligent technology, some existing technologies have proposed some automatic recipe generation methods for cooking robots.

[0006] For example, the Chinese invention patent with application number 201911086129.8 proposes a recipe making method, device and cooking machine. By obtaining the cooking video stream and cooking temperature data of the cooking process, and automatically extracting the ingredient addition timing and the ingredient addition category corresponding to each ingredient addition point from the cooking video stream, and automatically recording this information into a recipe template in a standard format to automatically generate the corresponding cooking recipe, there is no need to manually record the cooking process data, reducing the complexity of the recipe making process and improving the recipe making efficiency.

[0007] For example, in the Chinese invention patent with the application number 202310877558.7, a method, device, system and medium for automatically producing video recipes are proposed. By intercepting and integrating the obtained cooking videos based on operation prompt nodes according to the working time of the cooking device, and at the same time, adding control instructions for the cooking device. This enables users to have the cooking device read the control instructions while watching the video recipe. So that users do not need to manually adjust the cooking device, reducing the cooking labor and pressure of users and enhancing the user experience.

[0008] For example, in the Chinese invention patent with the application number 202311317039.1, a method, device and storage medium for automatically generating cloud recipes are proposed. The device status information on the cooking devices participating in the cooking work in the kitchen is obtained through pre-set Internet of Things; then the cooking data stream is analyzed to determine the user's cooking behavior, and a simplified cooking process is generated according to the user's cooking behavior and the device status information. The parts that require manual participation are analyzed through video, and the parts that require the participation of the devices are determined through the device status information, facilitating the step-by-step analysis of the human-machine integrated cooking work; the simplified cooking process is sent to the user and the correction information feedback by the user is obtained, and finally, a recipe is generated according to the correction information and the simplified cooking process and shared to the cloud platform. Thus, the semi-automatic recording function of the recipe is realized. It can effectively reduce the compilation work of the user for the recipe on the premise of ensuring that the recipe content fully meets the user's needs, and the introduced device status information can accurately meet the cooking requirements, facilitating the perfect reproduction of the step process of the cloud recipe on other devices.

[0009] In the above prior art, it is necessary to use a cooking robot to cook in advance, and generate a recipe by obtaining the video during the cooking process of the cooking robot and the device parameters of the cooking robot during the corresponding process. That is, the user needs to first cook using the cooking robot according to his own taste style, record the cooking video and obtain the device parameters of the cooking robot during this process, and only in this way can a recipe be generated. However, in this process, there are some relatively enclosed cooking robots, such as a pocket-sized full-automatic cooking machine disclosed in the applicant's prior application with the application number 202222334287.4 and a full-automatic cooking robot disclosed in the application number 202123025058.6. Its cooking pan is in a relatively enclosed environment, which is not suitable for users to cook by themselves, and it is also difficult to record the cooking process of the cooking robot. In this case, the above prior art cannot be used to generate a recipe. Therefore, the prior art has the disadvantage of a small scope of application.

[0010] In view of the above problems, the present application proposes a new technical solution. Summary of the Invention

[0011] The purpose of this application is to provide a recipe generation method, device, electronic device, and storage medium, which have the advantages of facilitating users to make recipes and high applicability.

[0012] In the first aspect, this application provides a recipe generation method, and the technical solution is as follows:

[0013] It includes:

[0014] Only obtain cooking videos with a complete cooking time series and containing cooking operation step screen information and / or cooking operation step sound information;

[0015] Play the cooking video on the user terminal and automatically generate an editing interface at the time node corresponding to the cooking time series according to the cooking operation step screen information and / or the cooking operation step sound information. The editing interface at least includes an input area and a label area. Identification labels are placed in the label area, and the identification labels at least include ingredient labels, seasoning labels, heat labels, and portion labels;

[0016] Obtain the identification labels from the label area and the corresponding text information input by the user in the input area;

[0017] Add the identification label together with the corresponding text information to the cooking time series to obtain cooking process information;

[0018] Generate a recipe that can be recognized and executed by a stir-fry robot according to the cooking process information.

[0019] By playing a cooking video with a complete cooking time series and containing cooking operation step screen information and / or cooking operation step sound information on the user terminal, and then automatically generating an editing interface at the corresponding time node during the playing process for the user to perform editing operations, enabling the user to edit according to the cooking operation step screen information and / or the cooking operation step sound information during the editing process. Therefore, it can facilitate the user to edit and generate recipes. Moreover, in this process, the recording of the cooking video does not require using a stir-fry robot to stir-fry, but can shoot a video of stir-frying using an ordinary wok. Since the cooking video has a complete cooking time series and contains cooking operation step screen information and / or cooking operation step sound information, the user does not need to use the parameter information during the stir-frying process of the stir-fry robot during the process of editing and generating the recipe. Therefore, there is no need to shoot a specific video for stir-frying with a stir-fry robot, but a video of stir-frying using ordinary cookware can be shot, which has the beneficial effects of facilitating users to make recipes and high applicability.

[0020] Further, in the present application, the step of playing the cooking video on the user terminal and automatically generating an editing interface at the time nodes corresponding to the cooking time sequence according to the cooking operation step screen information and / or the cooking operation step sound information includes:

[0021] Obtain a playback request message;

[0022] Play the cooking video on the user terminal according to the playback request message;

[0023] Calculate a first standard value of the cooking operation step screen information and a second standard value of the cooking operation step sound information at the time nodes corresponding to the cooking time sequence;

[0024] When the first standard value is greater than a first preset value, reduce the cooking operation step screen information and generate the editing interface, so that the cooking operation step screen information and the editing interface are on the same display screen, and loop-play the cooking operation step screen information within a preset range before and after the time nodes corresponding to the cooking time sequence that has been speed-reduced;

[0025] When the second standard value is greater than a second preset value, generate the editing interface, and loop-play the cooking operation step sound information within a preset range before and after the time nodes corresponding to the cooking time sequence that has been speed-reduced and noise-reduced on the editing interface.

[0026] Further, in the present application, the step of calculating the first standard value of the cooking operation step screen information and the second standard value of the cooking operation step sound information at the time nodes corresponding to the cooking time sequence includes:

[0027] Extract a first image frame from the cooking operation step screen information or a first keyword from the cooking operation step sound information to generate the time nodes corresponding to the cooking time sequence;

[0028] Obtain second image frames and second keywords corresponding to a preset range before and after the time nodes corresponding to the cooking time sequence;

[0029] Calculate the first standard value according to the second image frames;

[0030] Calculate the second standard value according to the second keywords;

[0031] The first standard value refers to the degree of image change in the cooking operation step screen information;

[0032] The second standard value refers to the degree of speech recognition in the cooking operation step sound information.

[0033] Further, in the present application, it further includes:

[0034] Adjust the first preset value or adjust the second preset value.

[0035] Further, in the present application, the step of adjusting the first preset value or adjusting the second preset value includes:

[0036] Calculate a first difference between the first standard value exceeding the first preset value;

[0037] Calculate a second difference between the second standard value exceeding the second preset value;

[0038] Compare the first difference and the second difference to obtain a comparison result, and adjust the first preset value or the second preset value according to the comparison result;

[0039] The step of adjusting the first preset value or the second preset value according to the comparison result includes:

[0040] When the first difference is greater than the second difference, adjust the second preset value;

[0041] When the second difference is greater than the first difference, adjust the first preset value;

[0042] The step of adjusting the second preset value includes:

[0043] When the first standard value is greater than the first preset value, decrease the second preset value;

[0044] The step of adjusting the first preset value includes:

[0045] When the second standard value is greater than the second preset value, decrease the first preset value.

[0046] Further, in the present application, the step of when the first difference is greater than the second difference and adjusting the second preset value includes:

[0047] When the first difference is greater than the second difference, determine whether the second standard value is greater than a fourth preset value, and the fourth preset value is less than the second preset value;

[0048] When the second standard value is greater than the fourth preset value, adjust the second preset value, and the adjusted second preset value is not less than the fourth preset value;

[0049] The step of when the second difference is greater than the first difference and adjusting the first preset value includes:

[0050] When the second difference is greater than the first difference, determining whether the first standard value is greater than a third preset value, and the third preset value is less than the first preset value;

[0051] When the first standard value is greater than the third preset value, the first preset value is adjusted, and the adjusted first preset value is not less than the third preset value.

[0052] Furthermore, in the present application, the step of obtaining a cooking video having a complete cooking time sequence and including cooking operation step image information and / or cooking operation step sound information includes:

[0053] Sending request information to the second software terminal through the first software terminal;

[0054] The second software terminal retrieves the camera permission of the user terminal according to the request information to shoot a video, thereby obtaining the cooking video having the complete cooking time sequence and including the cooking operation step image information and / or the cooking operation step sound information;

[0055] sending the cooking video to a server;

[0056] The first software terminal or the second software terminal accesses the server to obtain the cooking video having the complete cooking time sequence and including the cooking operation step screen information and / or the cooking operation step sound information.

[0057] In a second aspect, the present application further provides a recipe generation device for executing the above-mentioned recipe generation method, comprising:

[0058] The first module is used to obtain only cooking videos that have a complete cooking time sequence and contain cooking operation step image information and / or cooking operation step sound information;

[0059] The second module is configured to play the cooking video on a user terminal and automatically generate an editing interface at a time node corresponding to the cooking time sequence based on the cooking step visual information and / or the cooking step audio information, wherein the editing interface includes at least an input area and a label area, wherein the label area is provided with identification labels, and the identification labels include at least an ingredient label, a seasoning label, a heat label, and a portion label;

[0060] A third module is configured to obtain the identification tag and corresponding text information input by the user in the input area from the tag area;

[0061] A fourth module is configured to add the identification tag and the corresponding text information to the cooking time sequence to obtain cooking process information;

[0062] A fifth module is configured to generate a recipe that can be recognized and executed by a cooking robot according to the cooking process information.

[0063] In a third aspect, the present application further provides an electronic device, including a processor and a memory. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the steps in the above method are run.

[0064] In a fourth aspect, the present application further provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method are run.

[0065] As can be seen from the above, a recipe generation method, device, electronic device and storage medium provided by the present application play a cooking video with a complete cooking time sequence and containing cooking operation step picture information and / or cooking operation step sound information on a user terminal, and then, during the playing process, an editing interface is automatically generated at corresponding time nodes for the user to perform editing operations, enabling the user to edit according to the cooking operation step picture information and / or cooking operation step sound information during the editing process. Therefore, it is convenient for the user to edit and generate a recipe. Moreover, during this process, the recording of the cooking video does not require using a cooking robot to cook, but a video of cooking using an ordinary wok can be shot. Since the cooking video has a complete cooking time sequence and contains cooking operation step picture information and / or cooking operation step sound information, the user does not need the parameter information during the cooking process of the cooking robot when editing and generating the recipe. Therefore, there is no need to shoot a specific video for cooking with a cooking robot, but a video of cooking using ordinary cookware can be shot, which has the beneficial effects of facilitating the user to make a recipe and having high applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a flowchart of a recipe generation method provided by the present application.

[0067] Figure 2 It is a schematic structural diagram of a recipe generation device provided by the present application.

[0068] Figure 3 It is a schematic structural diagram of an electronic device provided by the present application.

[0069] Figure 4 It is a schematic structural diagram of a cooking robot targeted by the method of the present application.

[0070] In the figure: 210, the first module; 220, the second module; 230, the third module; 240, the fourth module; 250, the fifth module; 310, the processor; 320, the memory; 400, the cookware; 500, the housing. DETAILED DESCRIPTION

[0071] The technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. The components of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application.

[0072] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0073] Usually, when making a recipe, the user is generally required to have a very good understanding of the entire dish production process, and then record the entire dish production process to form a recipe. In this process, the user can manually record each operation step during the cooking process. This method is complicated and tedious to operate. The user needs to record the steps while cooking, that is, the recipe cannot be edited anytime and anywhere.

[0074] In this regard, users can record the cooking process by shooting a video during a specific cooking session, so that they can edit the recipe at a suitable time anytime and anywhere. However, this method requires users to constantly press the pause button while watching the recorded video, and then record manually. The operation is still inconvenient and has many shortcomings, which will greatly affect users' enthusiasm for making their own recipes.

[0075] With the development of automation technology, more and more stir-fry robot products have emerged. Stir-fry robots can replace manual operation by humans to achieve automatic stir-frying. During the working process of stir-fry robots, they need to execute corresponding operations according to recipes. However, there are significant differences between traditional recipes and the recipes for stir-fry robots. Stir-fry robots can only recognize and execute corresponding operations according to accurate specific parameters. For example, a specific amount of a grams of seasoning is added, heated at a temperature of b for a time of c, etc. These parameters are abstract and not intuitive for users. Therefore, this makes it very difficult for users to create recipes for stir-fry robots, especially for users with low cooking skills. Creating recipes for stir-fry robots is extremely difficult. This leads to the situation that users with low cooking skills often have to make do with using the originally preset recipes or recipes shared by other users, which often do not fully suit their own tastes, resulting in a low user experience. Moreover, users who use stir-fry robots generally have low cooking skills, which leads to a very small number of actually shared recipes, greatly limiting the user experience.

[0076] In response to this, in some existing technologies, methods for automatically generating recipes for stir-fry robots have been proposed, such as the solutions described in Chinese patent applications with application numbers 201911086129.8, 202310877558.7, 202311317039.1, etc. In these existing solutions, it is necessary to shoot videos of the stir-fry robot cooking. And during this process, the device parameters or control instructions of the stir-fry robot are obtained synchronously, and combined with the video images obtained by shooting to automatically generate recipes.

[0077] The above-mentioned existing technologies do solve the problem of users having difficulty creating recipes well. They can automatically generate recipes by combining video images with device parameters. Users only need to cook on the stir-fry robot and record the cooking process, without the need for additional recipe editing operations for users.

[0078] However, there are limitations in the above-mentioned existing technologies. First, most users who use stir-fry robots do not have good cooking skills themselves. Even if users operate the stir-fry robot to cook and record the entire process, the recipes finally made may not meet the users' tastes, which is a problem caused by the users' own cooking skills. Second, the stir-fry robot needs to be in an open environment, which is convenient for users to manually operate the stir-fry robot and also for recording the cooking process video. If the cookware of the stir-fry robot is in a relatively closed environment, it is difficult for users to operate it and also difficult to record videos, which will greatly affect the difficulty of finally creating recipes.

[0079] In this regard, if the above-mentioned existing technologies are adopted, in order to finally obtain a satisfactory recipe, the user himself / herself needs to have good cooking skills, or the user needs to find someone with relatively high cooking skills to use the cooking robot for cooking. At the same time, the cooking robot needs to be in an open state for easy operation and video recording. This has very great limitations. Through user surveys and market research, it can be known that the users who use cooking robots are generally migrant workers away from home and have a low cooking level. The flavors they like are often the cooking flavors of their parents at home. However, due to the low cooking skills of the users themselves, they cannot cook the flavors they like. Even if they operate the cooking robot to cook and make a recipe through video recording, they still cannot finally obtain a recipe that satisfies the users. This means that in some specific scenarios, if users want to make a satisfactory recipe, it often requires their parents to come from other places to use the cooking robot for cooking, then record the cooking process and finally generate a recipe. Obviously, this has huge limitations and is not convenient to operate.

[0080] Moreover, among the existing cooking robots, although there are cooking robot products in an open state, the automation degree of such products is relatively low, and there will be large fluctuations in their safety and working stability. For the solutions described in the Chinese patent applications with application numbers 202222334287.4 and 202123025058.6 previously applied by the applicant, the cookware is in a relatively closed environment, so it has better safety and working stability. However, because the cookware of this cooking robot product is in a relatively closed environment, it is difficult for users to observe the change process of the dishes in the cookware, resulting in very difficult manual operation. At the same time, it is also difficult to record the cooking process through a camera. If a camera is set inside the cooking robot, since oil fumes and steam will be generated during the cooking process, which will block the camera, there is also a problem of difficult video recording.

[0081] Therefore, the existing methods for automatically generating recipes for cooking robots have great limitations and are difficult to truly be popularized in the commercial market.

[0082] In this regard, the present application proposes a new solution for making recipes for cooking robots. Since the fully automated recipe generation method in the above-mentioned prior art has very large limitations, the solution provided by the present application abandons the solution of fully automatic recipe generation, and instead adopts the idea of assisting users in making recipes. By allowing users to participate in the recipe editing and production process, the many limitations brought about by fully automatic recipe generation are reduced. At the same time, assistance is provided to users in the process of editing and making recipes, reducing the difficulty of users in the recipe making process, so that even if users need to participate in the editing and production of recipes themselves, there will not be too many difficulties. At the same time, the limitations of the above-mentioned prior art are eliminated, so that users can finally easily make recipes that suit their taste and let the cooking robot make dishes that suit their taste according to the recipes.

[0083] For details, please refer to Figure 1 , this application proposes a recipe generation method, the technical solution specifically includes:

[0084] S110, only obtaining cooking videos that have a complete cooking time sequence and include cooking operation step image information and / or cooking operation step audio information;

[0085] S120: Playing a cooking video on a user terminal and automatically generating an editing interface at a time point corresponding to a cooking time sequence based on the cooking step visual information and / or the cooking step audio information, the editing interface comprising at least an input area and a label area, wherein the label area comprises identification labels, and the identification labels comprise at least an ingredient label, a seasoning label, a heat label, and a portion label;

[0086] S130, obtaining the identification tag from the tag area and the corresponding text information input by the user in the input area;

[0087] S140, adding the identification tag and the corresponding text information to the cooking time sequence to obtain cooking process information;

[0088] S150: Generate a recipe that can be recognized and executed by the cooking robot based on the cooking process information.

[0089] Preferably, the cooking robot proposed in this application is as follows Figure 4 As shown, the pot 400 of the cooking robot is surrounded by the outer shell 500, and the outer shell 500 forms a shield on the top of the pot 400, so that the pot 400 is in a relatively sealed state. The specific structure can refer to the scheme recorded in the applicant's prior Chinese invention patent applications with application numbers 202222334287.4 and 202123025058.6. Since the cooking robot is not the focus of this application, its specific structure will not be described in detail.

[0090] Among them, a complete cooking time series refers to a complete and continuous process from the start to the end of stir-frying. That is, a cooking video with a complete cooking time series refers to an unedited video that contains the complete process from the start to the end of stir-frying.

[0091] Among them, the cooking operation step screen information refers to the image screen of manual stir-frying. Specifically, it can include the image screens of operation steps such as the operator putting in ingredients, putting in seasonings, and adjusting the heat.

[0092] Among them, the cooking operation step sound information refers to the sound information of manual stir-frying. Specifically, it can include the sound information of operation steps such as the operator putting in ingredients, putting in seasonings, and adjusting the heat. Preferably, when the operator performs the relevant operation steps, the operator can say the relevant operation steps, and the sound emitted by the operator saying the operation steps can be used as the cooking operation step sound information.

[0093] Among them, the cooking video can be shot by a user terminal. Specifically, the user terminal can be a device with video shooting capabilities such as a smart phone or a camera.

[0094] Among them, the cooking video does not need to be recorded when stir-frying on a cooking robot, but can be recorded when stir-frying on an ordinary cooking utensil. Its core purpose is to obtain the cooking operation step screen information and / or the cooking operation step sound information and a complete cooking time series.

[0095] Among them, after the cooking video is shot, the cooking video can be sent and downloaded. In some specific application scenarios, the user can let their parents record a cooking video when stir-frying on an ordinary cooking utensil and then send the cooking video to the user.

[0096] Among them, the cooking video is played on the user terminal. Specifically, a software corresponding to the cooking robot can be provided. After the user downloads or shoots the cooking video, the cooking video is imported into the software for playing.

[0097] Based on the above software corresponding to the cooking robot when playing the cooking video, through image recognition technology and / or sound recognition technology, an editing interface is automatically generated at the time node corresponding to the cooking time series according to the cooking operation step screen information and / or the cooking operation step sound information, allowing the user to edit on the editing interface.

[0098] Among them, the editing interface includes an input area and a label area. Recognition labels are placed in the label area. The recognition labels at least include ingredient labels, seasoning labels, heat labels, and portion labels. During the editing process, the user can slide and drag the recognition labels from the label area into the input area, and then input corresponding text information in the recognition labels. For example, drag the ingredient label into the input area and then input the specific ingredient name such as eggs, then drag the seasoning label into the input area and then input the specific seasoning such as soy sauce, then drag the portion label into the input area and then input the specific portion such as 20g, and then drag the heat label into the input area and then input the specific heat information such as low heat. Among them, specific parameters recognized by the cooking robot corresponding to common colloquial operation steps can be established in advance. For example, the heating temperature corresponding to very low heat is 20 to 40 degrees Celsius, the heating temperature corresponding to low heat is 40 to 60 degrees Celsius, the temperature corresponding to medium heat is 60 to 80 degrees Celsius, the heating temperature corresponding to high heat is above 80 degrees Celsius, or one tablespoon corresponds to 15 milliliters or different grams according to different specific seasonings, etc.

[0099] That is, the user edits in the generated editing interface according to the cooking operation step screen information and / or cooking operation step sound information at the corresponding time nodes. During this process, since the cooking video has cooking operation step screen information and / or cooking operation step sound information, when the user edits in the editing interface, the user can edit according to the cooking operation step screen information and / or cooking operation step sound information received during the playback of the cooking video, rather than directly editing abstract cooking parameters by imagination alone. Therefore, the editing difficulty of the recipe is greatly reduced. And during this process, the editing interface is generated at the time nodes corresponding to the cooking time sequence. Therefore, the user does not need to consider the time corresponding to each cooking step during the editing process. Compared with the original recipe making method, the user does not need to record every operation step during the cooking process, nor does the user need to face various abstract cooking parameters, greatly reducing the editing difficulty of the recipe. Compared with the existing method for automatically generating recipes for cooking robots, the cooking video captured in the solution of this application does not need to be recorded when cooking on the cooking robot, and can be recorded when cooking with ordinary cookware, removing the restriction on the cooking robot.

[0100] After the user finishes editing in each editing interface, cooking process information can be obtained, and then the cooking process information can be sent to the server. The server compiles the cooking process information to obtain a recipe that can be recognized and executed by the cooking robot, and the obtained recipe can be loaded onto the corresponding cooking robot by means of data download.

[0101] The core inventive concept of this application lies in playing a cooking video with a complete cooking time sequence and containing cooking operation step screen information and / or cooking operation step sound information on a user terminal. During this process, an editing interface is automatically generated at corresponding time nodes based on image recognition or sound recognition technology, enabling the user to perform recipe editing operations on the editing interface. Moreover, during the user's editing process, the editing operations can be completed according to the cooking operation step screen information and / or cooking operation step sound information received during the playback of the cooking video. In this process, the user does not directly face abstract cooking parameters but completes the editing work under the guidance of the cooking operation step screen information and / or cooking operation step sound information. After the user has completed the editing work on all editing interfaces, cooking process information is obtained, and then a recipe that can be recognized and executed by the stir-fry robot is generated based on the cooking process information.

[0102] Compared with the existing solutions for automatically generating recipes for stir-fry robots, this application adopts a different technical route. This application adopts the technical concept of assisting the user in editing and producing recipes. Therefore, an editing interface is automatically generated at the time nodes corresponding to the cooking time sequence, and the editing interface includes an input area and a label area, which facilitates the user to perform recipe editing operations.

[0103] In summary, this application plays a cooking video with a complete cooking time sequence and containing cooking operation step screen information and / or cooking operation step sound information on a user terminal, and then during the playback process, an editing interface is automatically generated at corresponding time nodes for the user to perform editing operations, enabling the user to edit according to the cooking operation step screen information and / or cooking operation step sound information during the editing process. Therefore, it is convenient for the user to edit and generate a recipe. Moreover, during this process, the recording of the cooking video does not require using a stir-fry robot to cook but can be a video of cooking using an ordinary wok. Since the cooking video has a complete cooking time sequence and contains cooking operation step screen information and / or cooking operation step sound information, the user does not need to use the parameter information during the cooking process of the stir-fry robot when editing and generating the recipe. Therefore, there is no need to shoot a specific video of cooking with a stir-fry robot, but a video of cooking with ordinary cookware can be shot, which has the beneficial effects of facilitating the user to make recipes and having high applicability.

[0104] Specifically, in some embodiments, the method provided in the present application can be implemented based on a software terminal installed on a user's user terminal. The software terminal can be installed on the user terminals of different users, and the software terminals installed by different users can establish a connection relationship through a network. For example, a user installs a first software terminal on their own user terminal, and the user's parents or relatives and friends install a second software terminal on their user terminals. The first software terminal and the second software terminal can establish a connection relationship. Through the first software terminal, a shooting request message for a cooking video can be sent to the second software terminal. After other users discover the request message on the second software terminal, they can call the camera permission of the user terminal through the second software terminal to shoot a video, obtaining a cooking video. The obtained cooking video can be directly uploaded to the server. At this time, the server automatically generates an editing interface at the time node corresponding to the cooking time sequence according to the cooking operation step screen information and / or cooking operation step sound information in the cooking video. After uploading the cooking video to the server, the user can access the cooking video on the server through the first software terminal. After sending a play request, the cooking video can be played on the user terminal, and the generated editing interface is displayed when playing the cooking video, enabling the user to perform editing operations through the editing interface.

[0105] In this process, the user of the second software terminal can also access the server through the second software terminal and perform editing operations on the editing interface of the cooking video.

[0106] In some specific embodiments, when the user edits on the first software terminal, the corresponding editing operations are saved on the cloud server. The user of the first software terminal can send a help request message to the user of the second software terminal to enable the user of the second software terminal to help complete the editing work.

[0107] In addition, in some specific embodiments, the user can select a specified time node in the cooking time sequence by themselves when playing the cooking video, and then generate an editing interface at the specified time node.

[0108] In addition, in some specific embodiments, the user can manually cancel the editing interface.

[0109] Through the above settings, the user can correct the editing interface generated by mistake and the situation where the editing interface should be generated at the appropriate position but is not generated.

[0110] Among them, in some of the above embodiments, when the user edits a recipe on the editing interface, it can also be done by voice input.

[0111] In addition, after completing all the edits, the edited video can be saved and stored on the server. The server analyzes the edited video and generates a recipe that can be recognized and executed by the cooking robot based on the editing operations in the cooking time series.

[0112] Meanwhile, since the edited video is saved separately, when the user accesses the server through the first software terminal, the user can operate on the edited video. For example, the user can separately perform operations to add flavors, such as spicier, saltier, sweeter, etc. After adding the flavor, a new recipe can be regenerated separately, and in the newly generated recipe, the use of seasonings will be adjusted according to the selected flavor. In this process, the user can perform the operation of adding flavors by voice input.

[0113] In the solution of this application, the user can use the software terminal to shoot a cooking video. After shooting a video with a complete cooking time series, the video will be uploaded to the server. The server analyzes the cooking video based on image recognition technology and / or sound recognition technology to automatically find the time nodes corresponding to the cooking operations in the cooking time series. When the cooking video is played on the user terminal, an editing interface will be generated at the corresponding time nodes. After completion of the editing, the edited video can also be saved on the server, and the automatic adjustment operation of adding flavors can be completed separately later. Among them, the saved video can be named with the corresponding dish name. When the user accesses the server, the user can directly add flavors by voice to generate a new recipe. For example, on the server, there is a video of shredded pork with green peppers edited by the user. When the user accesses the server and enters the corresponding interface, the user can input "shredded pork with green peppers, salty flavor" by voice or text. At this time, the server will adjust the flavor of the originally saved edited video and automatically generate a new recipe that can be recognized and executed by the cooking robot. Specifically, the server can separately adjust the dosage and / or type of seasonings according to the user's input.

[0114] Further, in some embodiments, the step of playing the cooking video on the user terminal and automatically generating an editing interface at the time nodes corresponding to the cooking time series according to the cooking operation step screen information and / or cooking operation step sound information includes:

[0115] Obtain the play request information;

[0116] Play the cooking video on the user terminal according to the play request information;

[0117] Calculate the first standard value of the cooking operation step screen information and the second standard value of the cooking operation step sound information at the time nodes corresponding to the cooking time series;

[0118] When the first standard value is greater than a first preset value, the cooking operation step screen information is reduced in size and an editing interface is automatically generated, so that the cooking operation step screen information and the editing interface are displayed on the same screen, and the cooking operation step screen information within a set range before and after the time node corresponding to the cooking time sequence that has been slowed down is played in a loop on the editing interface;

[0119] When the second standard value is greater than the second preset value, an editing interface is automatically generated, and the sound information of the cooking operation steps that has been slowed down and noise-reduced within a set range before and after the time node corresponding to the cooking time sequence is played in a loop on the editing interface.

[0120] Since this application adopts a technical concept different from the existing technology and is intended to assist users in editing and making recipes, the cooking video is played on the user terminal, which can specifically refer to a smart phone or other smart terminal. During the playing of the cooking video, an editing interface is automatically generated at the corresponding time node for the user to edit.

[0121] On this basis, the present application further proposes to calculate the first standard value of the cooking operation step screen information and the second standard value of the cooking operation step sound information at the corresponding time node, wherein the first standard value refers to the degree of image change in the cooking operation step screen information, and the second standard value refers to the degree of voice recognition in the cooking operation step sound information.

[0122] An editing interface is automatically generated based on the comparison results of the first standard value and the first preset value, and the comparison results of the second standard value and the second preset value, so as to better assist the user in editing and making recipes in the editing interface.

[0123] Specifically, when the first standard value is greater than the first preset value, the cooking operation step screen information is reduced and an editing interface is automatically generated, so that the cooking operation step screen information and the editing interface are on the same display screen, and the cooking operation step screen information within a set range before and after the time node corresponding to the cooking time sequence that has been slowed down is played in a loop on the editing interface.

[0124] When the first standard value is greater than the first preset value, it indicates that the image changes significantly. This means that the user can receive clear information from the cooking operation step screen information to edit the recipe. Therefore, in this case, the cooking operation step screen information can be reduced and displayed on the same display screen as the editing interface. That is, at this time, the screen of the user terminal is divided into two display areas. One area displays the cooking video, and the other area displays the editing interface for the user to edit. In this way, the user can watch the cooking video while editing, making it more convenient for the user to edit. Moreover, during this process, when the cooking video and the editing interface are displayed on the same screen, the cooking operation step screen information within a set range before and after the corresponding time node is looped and played, enabling the user to clearly understand the operation steps required at the corresponding time node through the cooking operation step screen information, which is beneficial for the user to edit and produce the recipe. In addition, the cooking operation step screen information that is looped and played can be processed to slow down the speed, making it easier for the user to obtain the corresponding operation information from the cooking operation step screen information, thus being more conducive to the user to edit and produce the recipe.

[0125] In the solution of this application, both the cooking operation step screen information and the cooking operation step sound information are used to enable the user to obtain sufficient information to judge the operation steps performed at the corresponding time node. Among them, the user can rely solely on the cooking operation step screen information or solely on the cooking operation step sound information, or can also rely on both the cooking operation step screen information and the cooking operation step sound information simultaneously to assist the user in editing and producing the recipe.

[0126] When the first standard value is greater than the first preset value, since the first standard value is calculated at the corresponding time node, it indicates that the cooking operation step screen information has changed significantly. Generally speaking, the user can obtain specific operation steps from the significantly changed image, thereby completing the editing and production of the recipe. Therefore, when the first standard value is greater than the first preset value, it usually means that the user can complete the editing and production of the recipe solely by relying on the cooking operation step screen information. So, the cooking operation step screen information is reduced and an editing interface is automatically generated, making the cooking operation step screen information and the editing interface displayed on the same display screen, and the cooking operation step screen information within a set range before and after the time node corresponding to the cooking time sequence that has been processed to slow down is looped and played on the editing interface, enabling the user to complete the editing and production of the recipe.

[0127] Among them, the above-mentioned editing and production of the recipe can specifically refer to the editing and production of a certain step in the recipe.

[0128] In addition, when the second standard value is greater than the second preset value, an editing interface is automatically generated, and the voice information of the cooking operation steps within a set range before and after the time nodes corresponding to the cooking time series, which has been processed by speed reduction and noise reduction, is looped and played on the editing interface.

[0129] When the second standard value is greater than the second preset value, it indicates a high degree of speech recognition in the voice information of the cooking operation steps. This means that the user can receive clear information from the voice information of the cooking operation steps to edit the recipe. Therefore, in this case, the voice information of the cooking operation steps within a set range before and after the time nodes corresponding to the cooking time series, which has been processed by speed reduction and noise reduction, is looped and played on the editing interface so that the user can more easily complete the editing and production of the recipe.

[0130] In addition, in some preferred embodiments, if the first standard value is greater than the first preset value and the second standard value is greater than the second preset value, the above methods can be executed simultaneously, that is, after reducing the display size of the cooking operation steps screen information so that the editing interface and the cooking video are on the same display screen, the cooking operation steps screen information that has been processed by speed reduction and the corresponding voice information of the cooking operation steps that has been processed by speed reduction and noise reduction are looped and played.

[0131] In addition, in some preferred embodiments, when the first standard value is not greater than the first preset value and the second standard value is not greater than the second preset value, a suggestion message for reshooting the cooking video is sent to the user.

[0132] In some of the above embodiments, the present application proposes that the cooking video can be shot through the first software terminal and the second software terminal. After sending the suggestion message for reshooting the cooking video to the user, a reshooting option is generated on the first software terminal or the second software terminal. When the user selects the reshooting option to shoot, the reminder suggestion mode is triggered, and voice or text suggestion messages are generated in the reminder suggestion mode to remind the user to shoot.

[0133] Preferably, in the reminder suggestion mode, the smoke in the shooting screen is detected in real time. When the smoke volume reaches the set value, the user is reminded to remove the smoke.

[0134] Preferably, in the reminder suggestion mode, the background noise during the shooting process is detected in real time. When the background noise reaches the set value, the user is reminded to raise the voice.

[0135] Preferably, in the reminder suggestion mode, the cooking operation steps screen information and the voice information of the cooking operation steps during the shooting process are detected and compared in real time;

[0136] When it is detected that there is cooking operation steps screen information in the shooting screen but no voice information of the cooking operation steps, the user is reminded to emit the voice information of the cooking operation steps.

[0137] When the sound information of the cooking operation steps is detected but the picture information of the cooking operation steps does not exist, the user is reminded to take the correct picture.

[0138] In addition, when the user sends the sound information of the cooking operation steps or takes the correct picture after the reminder, there is a certain lag in the resulting sound information or picture information of the cooking operation steps. Therefore, a lag label can be generated at the corresponding position.

[0139] When automatically generating an editing interface at the time node corresponding to the cooking time series according to the picture information and / or sound information of the cooking operation steps, it is judged whether there is a lag label. When there is a lag label, an editing interface is automatically generated at the time node corresponding to the cooking time series according to the lag label and the picture information and / or sound information of the cooking operation steps.

[0140] That is, in some specific embodiments, when automatically generating an editing interface, if there is a lag label, when the two adjacent editing interfaces are separately generated according to the picture information and sound information of the cooking operation steps, the subsequent editing interface can be cancelled.

[0141] In addition, in the above solution, since there is a certain lag in the obtained sound information or picture information of the cooking operation steps, when there is a lag label, the set range in the picture information of the cooking operation steps within the set range before and after the time node corresponding to the cooking time series that is loop-played in the editing interface after being speeded down can be adjusted to extend the set range.

[0142] When there is a lag label, the set range in the sound information of the cooking operation steps within the set range before and after the time node corresponding to the cooking time series that is loop-played in the editing interface after being speeded down and noise-reduced can be adjusted to extend the set range.

[0143] Furthermore, in some of these embodiments, the steps of calculating the first standard value of the picture information of the cooking operation steps and the second standard value of the sound information of the cooking operation steps at the time node corresponding to the cooking time series include:

[0144] Extracting the first image frame in the picture information of the cooking operation steps or the first keyword in the sound information of the cooking operation steps to generate the time node corresponding to the cooking time series;

[0145] Obtaining the corresponding second image frame and second keyword within the set range before and after the time node corresponding to the cooking time series;

[0146] Calculating the first standard value according to the second image frame;

[0147] Calculating the second standard value according to the second keyword.

[0148] Generate time nodes corresponding to the cooking time series by extracting the first image frame from the cooking operation step screen information or the first keyword from the cooking operation step sound information. The corresponding time nodes indicate cooking operation steps judged based on image recognition technology or speech recognition technology. For example, in the previous image frame, the operator's hand did not hold a seasoning bottle, and in the next image frame, i.e., the first image frame, the operator's hand holds a seasoning bottle. Then, a time node corresponding to the cooking time series can be generated at this first image frame. In addition, judgment can also be made through speech recognition technology. For example, when recording a video, the operator is required to say each operation step, and based on the extraction of keywords, such as extracting keywords like "put in", "add", "join", etc., so as to generate time nodes corresponding to the cooking time series at the corresponding positions.

[0149] After generating the time nodes corresponding to the cooking time series, obtain the corresponding second image frames and second keywords within a set range before and after the time nodes corresponding to the cooking time series. Calculate the first standard value through the second image frames and calculate the second standard value through the second keywords. Among them, the second image frames can include the first image frame, and the second keywords can include the first keyword. Calculating the first standard value and the second standard value corresponding to the second image frames and the second keywords is to judge whether the user can obtain clear operation steps according to the second image frames or the second keywords for recipe editing and production.

[0150] It should be noted that in the solution of the present application, the time nodes corresponding to the cooking time series are the time points when cooking operations occur judged based on image recognition technology or speech recognition technology. On this basis, since the technical concept of the present application is to provide assistance to users to help them edit and make recipes, the present application proposes to play a cooking video with cooking operation step screen information and / or cooking operation step sound information on the user terminal, and automatically generate an editing interface at the time nodes corresponding to the cooking time series for the user to edit, enabling the user to edit after receiving the cooking operation step screen information and / or cooking operation step sound information, reducing the editing difficulty. In this process, since the user needs to obtain information through the cooking operation step screen information and / or cooking operation step sound information, if the cooking operation step screen information and / or cooking operation step sound information cannot enable the user to accurately and clearly obtain information, it will cause trouble for the user to edit and make the recipe. At this time, the user can be advised to reshoot the cooking video, that is, when the first standard value is less than or equal to the first preset value and the second standard value is less than or equal to the second preset value, send a suggestion message to the user to reshoot the cooking video, avoiding the user from editing and making the recipe when unable to clearly and accurately obtain the operation steps, which not only consumes the user's time and energy, but also the finally made recipe may not satisfy the user.

[0151] Therefore, in the solution of the present application, the first standard value is calculated based on the second image frame, and the second standard value is calculated based on the second keyword, aiming to ensure that the user can clearly and accurately obtain the cooking operation steps from the cooking operation step screen information and / or cooking operation step sound information.

[0152] Furthermore, in some embodiments, it further includes:

[0153] Adjust the first preset value or adjust the second preset value.

[0154] In the above-mentioned some embodiments, the present application proposes to measure whether the user can clearly and accurately obtain the cooking operation steps from the cooking operation step screen information and / or cooking operation step sound information by calculating the first standard value and the second standard value. When the first standard value is greater than the first preset value, it means that the user can clearly and accurately obtain the cooking operation steps from the cooking operation step screen information. When the second standard value is greater than the second preset value, it means that the user can clearly and accurately obtain the cooking operation steps from the cooking operation step sound information. As a preferred method, the user should obtain the cooking operation steps from both the cooking operation step screen information and the cooking operation step sound information.

[0155] When the first standard value is greater than the first preset value and the second standard value is not greater than the second preset value, the user will only be able to obtain the cooking operation steps through the cooking operation step screen information. However, in this case, the user can already clearly and accurately obtain the cooking operation steps based on the cooking operation step screen information. At this time, even if the voice recognition level of the cooking operation step sound information is low, combined with the cooking operation step screen information, the user can obtain the cooking operation steps more clearly and accurately. Therefore, in this case, the second preset value can be lowered so that the user can obtain the cooking operation steps based on both the cooking operation step screen information and the cooking operation step sound information.

[0156] Similarly, when the second standard value is greater than the second preset value and the first standard value is not greater than the first preset value, the user will only be allowed to obtain the cooking operation steps through the cooking operation step sound information. However, in this case, the user can already clearly and accurately obtain the cooking operation steps based on the cooking operation step sound information. At this time, even if the degree of change in the cooking operation step screen information is low, combined with the cooking operation step sound information, the user can obtain the cooking operation steps more clearly and accurately. Therefore, in this case, the first preset value can be lowered so that the user can obtain the cooking operation steps based on both the cooking operation step screen information and the cooking operation step sound information.

[0157] Specifically, the step of adjusting the first preset value or adjusting the second preset value includes:

[0158] Calculating a first difference between the first standard value and the first preset value;

[0159] Calculating a second difference between the second standard value and the second preset value;

[0160] Comparing the first difference and the second difference to obtain a comparison result, and adjusting the first preset value or the second preset value according to the comparison result;

[0161] The step of adjusting the first preset value or the second preset value according to the comparison result includes:

[0162] When the first difference is greater than the second difference, adjusting the second preset value;

[0163] When the second difference is greater than the first difference, adjusting the first preset value;

[0164] The step of adjusting the second preset value includes:

[0165] When the first standard value is greater than the first preset value, lowering the second preset value;

[0166] The step of adjusting the first preset value includes:

[0167] When the second standard value is greater than the second preset value, reduce the first preset value.

[0168] In some of the above embodiments, in order to enable users to more easily edit recipes in the editing interface, it is proposed to adjust the first preset value or the second preset value. As to whether to adjust the first preset value or the second preset value, this embodiment proposes to calculate the first difference between the first standard value exceeding the first preset value and the second difference between the second standard value exceeding the second preset value. Among them, the first difference and the second difference can be negative numbers. When the first difference is negative, it indicates that the first standard value is less than the first preset value. When the second difference is negative, it indicates that the second standard value is less than the second preset value. When the first difference is greater than the second difference, it indicates that the degree of image change in the cooking operation step screen information is higher. Users can mainly obtain the cooking operation steps through the cooking operation step screen information. Therefore, the second preset value can be adjusted. When the second difference is greater than the first difference, it indicates that the voice recognition degree in the cooking operation step voice information is higher. Users can mainly obtain the cooking operation steps through the cooking operation step voice information. Therefore, the first preset value can be adjusted.

[0169] Further, in some of these embodiments, when the first difference is greater than the second difference, the steps of adjusting the second preset value include:

[0170] When the first difference is greater than the second difference, determine whether the second standard value is greater than the fourth preset value, and the fourth preset value is less than the second preset value;

[0171] When the second standard value is greater than the fourth preset value, adjust the second preset value, and the adjusted second preset value is not less than the fourth preset value;

[0172] When the second difference is greater than the first difference, the steps of adjusting the first preset value include:

[0173] When the second difference is greater than the first difference, determine whether the first standard value is greater than the third preset value, and the third preset value is less than the first preset value;

[0174] When the first standard value is greater than the third preset value, adjust the first preset value, and the adjusted first preset value is not less than the third preset value.

[0175] In some of the above embodiments, the present application proposes to enable the user to obtain cooking operation steps based on the cooking operation step screen information and the cooking operation step sound information as much as possible. However, in this process, if the first difference is greater than the second difference, it means that the user can mainly obtain cooking operation steps through the cooking operation step screen information. At this time, the second preset value can be appropriately reduced to enable the user to receive the cooking operation step sound information as much as possible. However, if the second standard value is too small and less than or equal to the fourth preset value, then it is considered that it is difficult for the user to obtain useful information from the cooking operation step sound information. At this time, the cooking operation step sound information will not help the user, but may instead cause hindrance. For example, the sound generated during actual stir-frying will cover the cooking operation step sound information spoken by the operator. In this case, the sound information can be regarded as a kind of noise, which not only cannot help the user, but also has a bad impact on the user and distracts the user's attention. Therefore, only when the second standard value is greater than the fourth preset value will the second preset value be adjusted, and the adjusted second preset value is not less than the fourth preset value.

[0176] Similarly, only when the first standard value is greater than the third preset value will the first preset value be adjusted, and the adjusted first preset value is not less than the third preset value.

[0177] That is, the present application sets the third preset value and the fourth preset value as the lower limits of the first standard value and the second standard value respectively, to avoid displaying the cooking operation step screen information when the first standard value is equal to or lower than the third preset value, and to avoid displaying the cooking operation step sound information when the second standard value is equal to or lower than the fourth preset value. When the first standard value is equal to or lower than the third preset value, the cooking operation step screen information will interfere with the user. In this case, not displaying the cooking operation step screen information can enable the user to focus on the cooking operation step sound information. When the second standard value is equal to or lower than the fourth preset value, the cooking operation step sound information will interfere with the user. In this case, not displaying the cooking operation step sound information can enable the user to focus on the cooking operation step screen information. Through the above method, it can help the user better complete the editing and production work of the recipe.

[0178] Further, in some of the embodiments, after the step of adding the recognition label and the corresponding text information to the cooking time series to obtain the cooking process information, the following steps are further included:

[0179] Play a cooking video with cooking process information, and display the recognition label along with the cooking time series during the playback.

[0180] By playing a cooking video with cooking process information and displaying identification labels, users can double-check whether there are any omissions or errors in the entire cooking process, thereby avoiding the final generation of an incorrect recipe.

[0181] Once the user confirms that everything is correct, the cooking process information can be uploaded to the server, which will automatically compile and generate a recipe that can be recognized and executed by the cooking robot.

[0182] Secondly, refer to Figure 2 The present application also provides a recipe generation device for executing the above-mentioned recipe generation method, comprising:

[0183] The first module 210 is used to obtain a cooking video with a complete cooking time sequence and including cooking operation step image information and / or cooking operation step sound information;

[0184] The second module 220 is configured to play a cooking video on a user terminal and automatically generate an editing interface at a time point corresponding to a cooking time sequence based on the cooking step visual information and / or cooking step audio information. The editing interface includes at least an input area and a label area. The label area includes identification labels. The identification labels include at least an ingredient label, a seasoning label, a heat label, and a portion label.

[0185] The third module 230 is used to obtain the identification tag and corresponding text information from the tag area input by the user in the input area;

[0186] The fourth module 240 is configured to add the identification tag and the corresponding text information to the cooking time sequence to obtain cooking process information;

[0187] The fifth module 250 is used to generate a recipe that can be recognized and executed by the cooking robot based on the cooking process information.

[0188] By playing a cooking video with a complete cooking time sequence and containing cooking operation step picture information and / or cooking operation step sound information on a user terminal, and then automatically generating an editing interface at corresponding time nodes during the playing process for the user to perform editing operations, enabling the user to edit according to the cooking operation step picture information and / or cooking operation step sound information during the editing process. Therefore, it is convenient for the user to edit and generate a recipe. Moreover, in this process, the recording of the cooking video does not require using a cooking robot to cook, but can shoot a video of cooking with an ordinary wok. Since the cooking video has a complete cooking time sequence and contains cooking operation step picture information and / or cooking operation step sound information, the user does not need to use the parameter information during the cooking process of the cooking robot when editing and generating a recipe. Therefore, there is no need to shoot a specific video for cooking with a cooking robot, but can shoot a video of cooking with ordinary cookware, which has the beneficial effects of facilitating the user to make a recipe and having high applicability.

[0189] In addition, in some preferred embodiments, a recipe generation device proposed in the present application can execute any one of the steps in the above method.

[0190] In a third aspect, referring to Figure 3 , the present application further provides an electronic device, including a processor 310 and a memory 320. The memory 320 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 310, the steps in the above method are run.

[0191] Through the above technical solution, the processor 310 and the memory 320 are interconnected and communicate with each other through a communication bus and / or other forms of connection mechanisms (not marked). The memory 320 stores computer-readable instructions executable by the processor 310. When the electronic device runs, the processor 310 executes the computer-readable instructions to execute the method in any optional implementation manner of the above embodiments to achieve the following functions: obtaining a cooking video with a complete cooking time sequence and containing cooking operation step picture information and / or cooking operation step sound information; playing the cooking video on a user terminal and automatically generating an editing interface at the time nodes corresponding to the cooking time sequence according to the cooking operation step picture information and / or cooking operation step sound information. The editing interface at least includes an input area and a label area. The label area is provided with identification labels, and the identification labels at least include ingredient labels, seasoning labels, heat labels, and portion labels; obtaining the identification labels from the label area and the corresponding text information input by the user in the input area; adding the identification labels together with the corresponding text information to the cooking time sequence to obtain cooking process information; generating a recipe that can be recognized and executed by a cooking robot according to the cooking process information.

[0192] Fourthly, the present application further provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method are run.

[0193] Through the above technical solution, when the computer program is executed by a processor, the method in any optional implementation manner of the above embodiment is executed to achieve the following functions: obtaining a cooking video with a complete cooking time series and including cooking operation step screen information and / or cooking operation step sound information; playing the cooking video on a user terminal and automatically generating an editing interface at a time node corresponding to the cooking time series according to the cooking operation step screen information and / or the cooking operation step sound information, where the editing interface at least includes an input area and a label area, and identification labels are placed in the label area, and the identification labels at least include ingredient labels, seasoning labels, heat labels, and portion labels; obtaining the identification labels from the label area and the corresponding text information input by the user in the input area; adding the identification labels together with the corresponding text information to the cooking time series to obtain cooking process information; generating a recipe that can be recognized and executed by a cooking robot according to the cooking process information.

[0194] Among them, the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM for short), electrically erasable programmable read-only memory (EEPROM for short), erasable programmable read-only memory (EPROM for short), programmable read-only memory (PROM for short), read-only memory (ROM for short), magnetic memory, flash memory, a magnetic disk, or an optical disc.

[0195] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some communication interfaces. The indirect coupling or communication connection of the devices or units can be in an electrical, mechanical, or other form.

[0196] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or may be distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0197] Furthermore, the functional modules in various embodiments of the present application may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part.

[0198] The above are only the embodiments of the present application and are not used to limit the protection scope of the present application. For those skilled in the art, the present application may have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

Claims

1. A recipe generation method, characterized in that, Including: Only obtaining cooking videos that have a complete cooking time series and contain cooking operation step screen information and / or cooking operation step sound information; Playing the cooking video on a user terminal and automatically generating an editing interface at the time node corresponding to the cooking time series according to the cooking operation step screen information and / or the cooking operation step sound information. The editing interface at least includes an input area and a label area. Identification labels are placed in the label area, and the identification labels at least include ingredient labels, seasoning labels, heat labels, and portion labels; Obtaining the identification labels from the label area and the corresponding text information input by the user in the input area; Adding the identification labels together with the corresponding text information to the cooking time series to obtain cooking process information; Generating a recipe that can be recognized and executed by a stir-fry robot according to the cooking process information; The step of playing the cooking video on the user terminal and automatically generating an editing interface at the time node corresponding to the cooking time series according to the cooking operation step screen information and / or the cooking operation step sound information includes: Obtaining a play request message; Playing the cooking video on the user terminal according to the play request message; Calculating a first standard value of the cooking operation step screen information and a second standard value of the cooking operation step sound information at the time node corresponding to the cooking time series; When the first standard value is greater than a first preset value, reducing the cooking operation step screen information and generating the editing interface, so that the cooking operation step screen information and the editing interface are on the same display screen, and looping and playing the cooking operation step screen information within a set range before and after the time node corresponding to the cooking time series that has been speed-reduced; When the second standard value is greater than a second preset value, generating the editing interface, and looping and playing the cooking operation step sound information within a set range before and after the time node corresponding to the cooking time series that has been speed-reduced and noise-reduced; 2. The recipe generation method according to claim 1, wherein The step of calculating the first standard value of the cooking operation step screen information and the second standard value of the cooking operation step sound information at the time node corresponding to the cooking time series includes: Extracting a first image frame from the cooking operation step screen information or a first keyword from the cooking operation step sound information to generate the time node corresponding to the cooking time series; Obtaining corresponding second image frames and second keywords within a set range before and after the time node corresponding to the cooking time series; Calculating the first standard value according to the second image frames; Calculating the second standard value according to the second keywords; The first standard value refers to the degree of image change in the cooking operation step screen information; The second standard value refers to the degree of speech recognition in the cooking operation step sound information; 3. The method for generating a recipe according to claim 2, wherein, Also including: Adjusting the first preset value or adjusting the second preset value; 4. The recipe generation method according to claim 3, characterized in that The step of adjusting the first preset value or adjusting the second preset value includes: Calculating a first difference value by which the first standard value exceeds the first preset value; Calculating a second difference between the second standard value and the second preset value; Comparing the first difference and the second difference to obtain a comparison result, and adjusting the first preset value or the second preset value according to the comparison result; The step of adjusting the first preset value or the second preset value according to the comparison result includes: When the first difference is greater than the second difference, adjusting the second preset value; When the second difference is greater than the first difference, adjusting the first preset value; The step of adjusting the second preset value includes: When the first standard value is greater than the first preset value, reducing the second preset value; The step of adjusting the first preset value includes: When the second standard value is greater than the second preset value, the first preset value is reduced.

5. The recipe generation method according to claim 4, wherein When the first difference is greater than the second difference, the step of adjusting the second preset value includes: When the first difference is greater than the second difference, determining whether the second standard value is greater than a fourth preset value, and the fourth preset value is less than the second preset value; When the second standard value is greater than the fourth preset value, adjusting the second preset value, and the adjusted second preset value is not less than the fourth preset value; When the second difference is greater than the first difference, the step of adjusting the first preset value includes: When the second difference is greater than the first difference, determining whether the first standard value is greater than a third preset value, and the third preset value is less than the first preset value; When the first standard value is greater than the third preset value, the first preset value is adjusted, and the adjusted first preset value is not less than the third preset value.

6. The recipe generation method according to claim 1, wherein The step of obtaining a cooking video having a complete cooking time sequence and including cooking operation step image information and / or cooking operation step sound information includes: Sending request information to the second software terminal through the first software terminal; The second software terminal retrieves the camera permission of the user terminal according to the request information to shoot a video, thereby obtaining the cooking video having the complete cooking time sequence and including the cooking operation step image information and / or the cooking operation step sound information; sending the cooking video to a server; The first software terminal or the second software terminal accesses the server to obtain the cooking video having the complete cooking time sequence and including the cooking operation step screen information and / or the cooking operation step sound information.

7. A recipe generation device, characterized in that, The method for executing the recipe generation method according to any one of claims 1 to 6 comprises: The first module is used to obtain only cooking videos that have a complete cooking time sequence and contain cooking operation step image information and / or cooking operation step sound information; A second module, configured to play the cooking video on a user terminal and automatically generate an editing interface at time nodes corresponding to the cooking time series according to the cooking operation step screen information and / or the cooking operation step sound information, where the editing interface at least includes an input area and a label area, and identification labels are placed in the label area, and the identification labels at least include ingredient labels, seasoning labels, heat labels, and portion labels; A third module, configured to obtain the identification labels from the label area and the corresponding text information input by the user in the input area; A fourth module, configured to add the identification labels together with the corresponding text information to the cooking time series to obtain cooking process information; A fifth module, configured to generate a recipe that can be recognized and executed by a cooking robot according to the cooking process information.

8. An electronic device, characterized in that, It includes a processor and a memory, and the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method according to any one of claims 1-6 are run.

9. A storage medium, on which a computer program is stored, characterized in that, When the computer program is executed by the processor, the steps in the method according to any one of claims 1-6 are run.

Images