Structured data generation and job method, kitchen robot, and storage medium

Abstract

The embodiment of the present application provides a kind of structured data generation and job method, kitchen robot and storage medium.In the embodiment of the present application, in the process of generating structured data, at least for the job step of a kind of needing to rely on data object, leave the margin of user intervention, by setting corresponding prompt information, user is guided to intervene, finally generates a kind of semi-automatic structured data, rather than fully automated structured data.In this way, in the process that any kitchen robot executes job task according to the semi-automatic structured data, when executing the job step provided with prompt information, can output prompt information to prompt user to execute the auxiliary operation adapted to the job step, to cooperate with the kitchen robot to complete job task.Since allowing user to adjust and intervene the process that kitchen robot executes job task, the flexibility of kitchen robot executing job task is stronger, it is advantageous to satisfy the job demand of user.

Description

[0001] This case involves application number 202110678876.1, application date June 18, 2021, and patent title "Structured..." A divisional application of the patent application for “Data generation and operation method, kitchen robot and storage medium”. Technical Field

[0002] This application relates to the field of artificial intelligence technology, and in particular to a method for generating and operating structured data, a kitchen robot, and a storage medium. Background Technology

[0003] With the rapid development of artificial intelligence, more and more intelligent machines are being applied to people's lives. For example, intelligent cooking machines allow users to complete the automated cooking process with very few steps, bringing great convenience to cooking delicious food.

[0004] In the existing technology, there are some electronic recipes that allow smart cooking machines to automatically cook various dishes. However, the process of cooking food according to existing electronic recipes lacks flexibility, which may result in unsatisfactory food quality that fails to meet the user's cooking needs. Summary of the Invention

[0005] This application provides a structured data generation and operation method, a kitchen robot, and a storage medium to improve the flexibility of performing operation tasks.

[0006] This application provides a structured data generation method, including: recording multiple work steps performed by the kitchen robot and the execution order between the multiple work steps during the execution of a work task; generating prompt information corresponding to a first type of work step among the multiple work steps, wherein the first type of work step includes at least work steps that require data objects; generating reference structured data based on the multiple work steps, the execution order between the multiple work steps, and the prompt information corresponding to the first type of work step; wherein the prompt information is used to prompt the user to perform an auxiliary operation adapted to the first type of work step when any kitchen robot performs a work task based on the reference structured data and reaches the first type of work step, so as to cooperate with any kitchen robot to complete the work task.

[0007] This application embodiment also provides a structured data generation method, including: during the execution of a kitchen robot's work tasks, recording multiple work steps performed by the kitchen robot and the execution order between the multiple work steps; for a third type of work step among the multiple work steps, responding to the user's configuration operation, recording the attribute information of the data objects that the user configures for the third type of work step, wherein the third type of work step is a work step among the multiple work steps that needs to depend on data objects but does not provide data object identification information; and generating structured data based on the multiple work steps, the execution order between the multiple work steps, and the attribute information of the data objects that the third type of work step depends on.

[0008] This application embodiment also provides a method for executing a task, including: acquiring reference structured data, the reference structured data including multiple task steps in the kitchen robot's task execution, the execution order of the multiple task steps, and prompt information corresponding to a first type of task step among the multiple task steps; executing the multiple task steps sequentially according to the execution order of the multiple task steps; and outputting prompt information when executing the first type of task step to prompt the user to perform auxiliary operations adapted to the first type of task step in order to cooperate with the kitchen robot to complete the task.

[0009] This application also provides a structured data generation method applicable to terminal devices. The method includes: displaying a work control interface; controlling a kitchen robot to perform work tasks based on controls on the work control interface; receiving and displaying the work steps currently being performed by the kitchen robot during the execution of the work tasks; when the currently performed work step is a specified work step, generating prompt information corresponding to the first type of work step in response to a user's marking operation of marking it as a first type of work step; generating reference structured data based on the work steps sequentially performed by the kitchen robot and the prompt information corresponding to the first type of work steps; wherein, the prompt information is used to prompt the user to perform auxiliary operations adapted to the first type of work step when any kitchen robot performs a work task based on the reference structured data and reaches the first type of work step, so as to assist any kitchen robot in completing the work task.

[0010] This application also provides a structured data generation method applicable to terminal devices. The method includes: displaying a job control interface; controlling a kitchen robot to perform job tasks based on controls on the job control interface; receiving and displaying the job steps currently being performed by the kitchen robot during the job task performance; when the currently performed job step is a specified job step, displaying an object configuration interface in response to a user's marking operation as a third type of job step, wherein the third type of job step refers to a job step that depends on a data object; recording the attribute information of the data objects that the user configures for the third type of job step in response to the user's configuration operation on the object configuration interface; and generating structured data based on the job steps sequentially performed by the kitchen robot and the attribute information of the data objects that the third type of job step depends on.

[0011] This application embodiment also provides a kitchen robot, including: a memory and a processor; the memory for storing a computer program; the processor, coupled to the memory, for executing the computer program, for: recording multiple work steps performed by the kitchen robot and the execution order between the multiple work steps during the execution of a work task; generating prompt information corresponding to a first type of work step among the multiple work steps, wherein the first type of work step includes at least work steps that require a data object; generating reference structured data based on the multiple work steps, the execution order between the multiple work steps, and the prompt information corresponding to the first type of work step; wherein the prompt information is used to prompt the user to perform an auxiliary operation adapted to the first type of work step when any kitchen robot performs a work task based on the reference structured data and reaches the first type of work step, so as to cooperate with any kitchen robot to complete the work task.

[0012] This application embodiment also provides a kitchen robot, including: a memory and a processor; the memory for storing a computer program; the processor, coupled to the memory, for executing the computer program, for: recording multiple work steps performed by the kitchen robot and the execution order between the multiple work steps during the execution of a work task; for a third type of work step among the multiple work steps, responding to a user's configuration operation, recording the attribute information of the data objects that the user configures for the third type of work step, wherein the third type of work step is a work step among the multiple work steps that needs to depend on data objects but does not provide data object identification information; and generating structured data based on the multiple work steps, the execution order between the multiple work steps, and the attribute information of the data objects that the third type of work step depends on.

[0013] This application also provides a kitchen robot, including: a memory and a processor; the memory for storing a computer program; the processor, coupled to the memory, for executing the computer program to: acquire reference structured data, the reference structured data including multiple work steps in the kitchen robot's work tasks, the execution order between the multiple work steps, and prompt information corresponding to a first type of work step among the multiple work steps; execute the multiple work steps sequentially according to the execution order between the multiple work steps; and when the first type of work step is executed, output prompt information to prompt the user to perform auxiliary operations adapted to the first type of work step to cooperate with the kitchen robot to complete the work tasks.

[0014] This application embodiment also provides a terminal device, including: a memory and a processor; the memory for storing computer programs; the processor, coupled to the memory, for executing the computer programs to: display a job control interface, and control a kitchen robot to perform job tasks based on controls on the job control interface; during the execution of job tasks by the kitchen robot, receive and display the job steps currently being performed by the kitchen robot; when the currently performed job step is a specified job step, in response to a user marking it as a first type of job step, generate prompt information corresponding to the first type of job step; generate reference structured data based on the job steps sequentially performed by the kitchen robot and the prompt information corresponding to the first type of job steps; wherein, the prompt information is used to prompt the user to perform auxiliary operations adapted to the first type of job step when any kitchen robot performs a job task based on the reference structured data and reaches the first type of job step, so as to cooperate with any kitchen robot to complete the job task.

[0015] This application embodiment also provides a terminal device, including: a memory and a processor; the memory for storing computer programs; the processor, coupled to the memory, for executing the computer programs for: displaying a job control interface, controlling a kitchen robot to perform job tasks based on controls on the job control interface; receiving and displaying the job steps currently being performed by the kitchen robot during the execution of job tasks; when the currently performed job step is a specified job step, displaying an object configuration interface in response to a user's marking operation of marking it as a third type of job step, wherein the third type of job step refers to a job step that requires a data object; recording the attribute information of the data objects that the user configures for the third type of job step in response to the user's configuration operation on the object configuration interface; and generating structured data based on the job steps sequentially performed by the kitchen robot and the attribute information of the data objects that the third type of job step depends on.

[0016] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps in the structured data generation method and job task execution method provided in this application.

[0017] In this embodiment, during the generation of structured data, at least for one type of operational step that relies on data objects, room for user intervention is provided. By setting corresponding prompts, users are guided to intervene, ultimately generating semi-automated structured data, rather than fully automated structured data. Thus, when any kitchen robot performs a task based on this semi-automated structured data, upon reaching a step with provided prompts, the robot can output prompts to guide the user to perform auxiliary operations appropriate to that step, thereby assisting the kitchen robot in completing the task. Because users are allowed to adjust and intervene in the process of the kitchen robot performing tasks, the flexibility of the kitchen robot in performing tasks is greater, which is beneficial for meeting user needs. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0019] Figure 1a A flowchart illustrating a structured data generation method provided for an exemplary embodiment of this application;

[0020] Figure 1b A flowchart illustrating the steps of a structured data generation method before a kitchen robot performs a task, provided as an exemplary embodiment of this application;

[0021] Figure 1c A flowchart illustrating another structured data generation method provided for an exemplary embodiment of this application;

[0022] Figure 1d A flowchart illustrating another structured data generation method provided for an exemplary embodiment of this application;

[0023] Figure 1e A flowchart illustrating yet another structured data generation method provided for an exemplary embodiment of this application;

[0024] Figure 2a A flowchart illustrating a job task execution method provided as an exemplary embodiment of this application;

[0025] Figure 2b A schematic diagram of the structure of a voice module provided for an exemplary embodiment of this application;

[0026] Figure 2c A schematic diagram of the structure of a voice system provided for an exemplary embodiment of this application;

[0027] Figure 3 A schematic diagram of the structure of a cooking device provided for an exemplary embodiment of this application;

[0028] Figure 4a This is a schematic diagram of a creative mode interface.

[0029] Figure 4b This is a schematic diagram of a pre-editing interface;

[0030] Figure 4c This is a schematic diagram of another pre-editing interface;

[0031] Figure 4d This is a schematic diagram of a material box weighing interface;

[0032] Figure 4e This is a schematic diagram of another type of weighing interface for a material box;

[0033] Figure 4f This is a schematic diagram of a cooking control interface;

[0034] Figure 4g This is a schematic diagram of another cooking control interface;

[0035] Figure 4h This is a schematic diagram of a creation preview interface;

[0036] Figure 5 A schematic diagram of the structure of a kitchen robot provided as an exemplary embodiment of this application;

[0037] Figure 6 This is a schematic diagram of the structure of a terminal device provided for an exemplary embodiment of this application. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0039] In existing technologies, when intelligent cooking machines cook food according to existing electronic recipes, users cannot intervene. The machines can only follow the steps in the recipes, lacking flexibility and potentially resulting in suboptimal food quality that fails to meet user needs. To address this issue, in this embodiment, during the generation of structured data, at least for one type of operational step that relies on data objects, user intervention is permitted. By setting appropriate prompts, users are guided to intervene, ultimately generating semi-automated structured data, rather than fully automated structured data. Thus, when any kitchen robot performs a task based on this semi-automated structured data, upon reaching a step with provided prompts, the robot can output prompts to guide the user to perform appropriate auxiliary operations, thereby assisting the robot in completing the task. Because users can adjust and intervene in the process of the kitchen robot performing tasks, the robot's flexibility is enhanced, better meeting user needs.

[0040] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0041] Figure 1 is a flowchart illustrating a structured data generation method provided in an exemplary embodiment of this application. As shown in Figure 1, the method includes:

[0042] 101a. During the kitchen robot's task execution, record the multiple work steps performed by the kitchen robot and the execution order between the multiple work steps;

[0043] 102a. For the first type of operation steps in multiple operation steps, generate prompt information corresponding to the first type of operation steps. The first type of operation steps shall include at least the operation steps that require data objects.

[0044] 103a. Generate reference structured data based on multiple work steps, the execution order between the multiple work steps, and the prompt information corresponding to the first type of work steps; wherein, the prompt information is used to prompt the user to perform auxiliary operations adapted to the first type of work steps when any kitchen robot performs a work task based on the reference structured data and reaches the first type of work step, so as to cooperate with any kitchen robot to complete the work task.

[0045] In this embodiment, the kitchen robot can be any electronic device capable of autonomously performing tasks in a kitchen environment based on structured data, such as, but not limited to, a smart stir-fry machine, a smart rice cooker, a smart electric griddle, or a smart dishwasher. Depending on the type of kitchen robot, the tasks it performs will also differ. For example, if the kitchen robot is a smart stir-fry machine, it can perform stir-frying tasks; if the kitchen robot is a smart rice cooker, it can perform rice cooking tasks, etc.

[0046] In this embodiment, the kitchen robot can establish a binding relationship with a terminal device. The terminal device can be any terminal device that can provide interface operation functions, such as a mobile phone, tablet computer, smartwatch, laptop computer, or desktop computer.

[0047] In this embodiment, the kitchen robot can perform tasks using, but is not limited to, the following implementation methods. Examples are given below.

[0048] Implementation method A1: The kitchen robot can receive a work instruction that directs it to perform a task based on basic structured data. In this case, the kitchen robot can acquire the basic structured data, which contains multiple work steps required for the kitchen robot to perform the task and the execution order of these steps. Then, it executes the multiple work steps sequentially according to the execution order of these steps included in the basic structured data to complete the task.

[0049] Optionally, the operation instructions can be issued by the user through the display screen of the kitchen robot; or by the user through voice; or by the user through an APP on a terminal device bound to the kitchen robot; or by the server issuing the instructions to the kitchen robot.

[0050] Optionally, the basic structured data can be pre-loaded structured data for the kitchen robot, in which case it can be obtained locally. Alternatively, the basic structured data can be downloaded from the network or server, and this basic structured data can be created and published on the network or server by other users.

[0051] Implementation method A2:Users can continuously send control commands to the kitchen robot, sequentially controlling it to perform multiple work steps to complete the task. Optionally, users can send control commands to the kitchen robot via the display screen on the terminal device or the display screen on the kitchen robot itself, or they can send control commands via voice. Each control command instructs the kitchen robot to perform one or more work steps. The kitchen robot can continuously receive control commands from the user and execute multiple work steps sequentially according to the received commands to complete the task.

[0052] In implementation A2, the order in which the user sends control signals to the kitchen robot can reflect the execution order among multiple work steps performed by the kitchen robot.

[0053] Regardless of which method the kitchen robot uses to perform its tasks, it can record the steps it takes and the order in which these steps are executed to generate structured data. In this embodiment, the kitchen robot primarily generates semi-automatic structured data. Fully automated structured data guides the kitchen robot to complete tasks automatically without user intervention, while semi-automatic structured data requires user participation; the robot can only complete tasks with user cooperation. For ease of distinction and description, in this embodiment, this semi-automatic structured data is referred to as reference structured data; correspondingly, the fully automated structured data is referred to as follow-up structured data.

[0054] In this embodiment, the process of the kitchen robot generating reference structured data involves recording the work steps it performs and the execution order of these steps while simultaneously identifying work steps suitable for user intervention. For these work steps, prompts are added to guide user intervention. In this embodiment, the work steps suitable for user intervention among the work steps performed by the kitchen robot are referred to as first-type work steps, and prompts corresponding to the first-type work steps are generated.

[0055] In this embodiment, the kitchen robot's task execution involves data objects. These data objects refer to external data objects, such as those that need to be provided, added, increased, or decreased during task execution. Since these data objects are external to the kitchen robot, they can be manually operated by the user, rather than necessarily automatically by the robot. Therefore, the first type of task steps includes at least the following: task steps that rely on data objects. For example, if the kitchen robot is a smart cooking machine and the task is cooking dishes, the data objects could be ingredients, seasonings, or other auxiliary tools needed for cooking.

[0056] In this embodiment, each first-type operation step corresponds to a prompt message. After identifying all first-type operation steps and generating corresponding prompt messages for each, reference structured data can be generated based on the recorded multiple operation steps performed by the kitchen robot, the execution order of the multiple operation steps, and the prompt messages corresponding to the first-type operation steps. This reference structured data includes at least the multiple operation steps required for the kitchen robot to perform its task, the execution order of the multiple operation steps, and the prompt messages corresponding to the first-type operation steps among the multiple operation steps. The reference structured data generated in this embodiment can be used to guide any kitchen robot to perform its task. "Any kitchen robot" here includes both the kitchen robot that generated the reference structured data and other kitchen robots. The kitchen robot that generated the reference structured data can publish the reference structured data to the internet or a server, and other kitchen robots can download the reference structured data from the internet or a server. Specifically, the prompt messages corresponding to the first-type operation steps are used to prompt the user to perform auxiliary operations adapted to the first-type operation steps when any kitchen robot performs its task according to the reference structured data and reaches the first-type operation step, in order to assist any kitchen robot in completing its task.

[0057] In this embodiment, during the generation of structured data, at least for one type of operational step that relies on data objects, room for user intervention is provided. By setting corresponding prompts, users are guided to intervene, ultimately generating semi-automated structured data, rather than fully automated structured data. Thus, when any kitchen robot performs a task based on this semi-automated structured data, upon reaching a step with provided prompts, the robot can output prompts to guide the user to perform auxiliary operations appropriate to that step, thereby assisting the kitchen robot in completing the task. Because users are allowed to adjust and intervene in the process of the kitchen robot performing tasks, the flexibility of the kitchen robot in performing tasks is greater, which is beneficial for meeting user needs.

[0058] In an optional embodiment, when the first type of operation steps includes operation steps that depend on data objects, the implementation method for generating prompt information corresponding to the first type of operation steps includes: generating prompt information to prompt the user to operate on the data objects on which the first type of operation steps depend, based on the dependency behavior of the first type of operation steps on data objects. In this case, the prompt information is used to prompt the user to operate on the data objects on which the first type of operation steps depend, and correspondingly, the auxiliary operation performed by the user is to operate on the data objects on which the first type of operation steps depend. Taking a kitchen robot as a cooking machine as an example, the reference structured data can be implemented as a reference electronic recipe, and the prompt information corresponding to the data objects included in the reference electronic recipe can be "Please add xxx ingredient", "Add xxx seasoning", "Retrieve xxx seasoning from the pot", etc.

[0059] Optionally, when the first type of work steps includes work steps that depend on data objects, before generating the prompt information corresponding to the first type of work steps, it can be determined which work steps are first type work steps that depend on data objects. For work steps that clearly depend on data objects, keywords corresponding to these types of work steps can be pre-configured. Based on this, during the kitchen robot's work task execution, the currently executed work step can be matched with the pre-configured keywords; if a match is found with the pre-configured keywords, the currently executed work step is determined to be a first type of work step. For work steps that clearly do not depend on data objects, because they cannot match the pre-configured keywords, they will not be identified as first type of work steps.

[0060] However, in practical applications, some operation steps may or may not depend on data objects. These types of operation steps cannot be easily identified by matching preset keywords. In this embodiment, these operation steps that may or may not depend on data objects are referred to as specified operation steps. Specified operation steps are pre-configured, and marking options are set for them, allowing users to mark whether a specified operation step depends on data objects as needed during the actual execution of the kitchen robot's tasks. If the user marks a specified operation step as needing to depend on data objects, then that specified operation step is a first-category operation step that depends on data objects; otherwise, the specified operation step is not a first-category operation step that depends on data objects.

[0061] Based on the above, during the kitchen robot's operation, in addition to recording the multiple operation steps performed by the kitchen robot, the current operation step can be displayed to the user on the kitchen robot's display screen or the display screen of the terminal device bound to the kitchen robot, according to the execution order of these multiple operation steps. It should be noted that the display screen can show not only the operation steps currently being performed by the kitchen robot, but also the operation steps already performed or those about to be performed. When the kitchen robot performs a specific operation step, a marking option can be displayed for that specific operation step, allowing the user to mark whether the specified operation step depends on a data object. Specifically, through the marking option for a specified operation step, the user can mark that the specified operation step depends on a data object, or mark that the specified operation step does not depend on a data object. When the user marks that the specified operation step depends on a data object through this marking option, the marking information can also reflect the dependency behavior of the operation step on the data object. For example, taking a cooking machine as an example, the designated operation step is the "opening the lid" step. The marked options can be, but are not limited to: opening the lid to add / remove ingredients, opening the lid to add / remove seasonings, opening the lid to flip the ingredients, or opening the lid to cook. Among these, actions such as "adding," "removing," or "flipping" are dependent behaviors on data objects. If a designated operation step is marked as dependent on a data object, the user can also configure the identification information of the data objects that the designated operation step depends on. Then, the kitchen robot can also obtain the identification information of the data objects that the user configured for the designated operation step.

[0062] In this embodiment, the implementation method of obtaining the identification information of the data objects that the user has configured for a specified job step is not limited to. Examples are given below.

[0063] Implementation Method B1: Before the kitchen robot performs its tasks, the user pre-edits the data objects required for the task to obtain a list of data objects, including identification information for at least one data object. Based on this, when the kitchen robot reaches a specific task step and the user marks the data objects dependent on that step using the aforementioned marking option, the list of data objects required for the task can be further displayed. The user can select the identification information of the data objects dependent on the specified task from the list. The kitchen robot can respond to the user's selection of the data object list and obtain the identification information of the selected data object as the identification information of the data object dependent on the specified task.

[0064] Optionally, before the kitchen robot performs its tasks, the process of pre-editing the data objects required for the tasks includes: displaying a pre-editing interface, which includes candidate data objects and / or editing controls, for the user to edit the list of data objects required for the kitchen robot to perform its tasks; and generating a list of data objects required for the kitchen robot to perform its tasks in response to the user's pre-editing operation. The user can select the data objects required for the kitchen robot to perform its tasks from the candidate data objects and add them to the data object list; or, if the list of data objects required for the kitchen robot to perform its tasks does not exist in the candidate data objects, the user can automatically add or edit the information of the data objects required by the kitchen robot using the editing controls. Alternatively, the editing controls can be directly displayed on the pre-editing interface, allowing the user to directly edit the information of the data objects required for the kitchen robot to perform its tasks and add them to the data object list.

[0065] Implementation method B2: When the kitchen robot performs a specified operation step and the user marks the data object that the specified operation step depends on using the marking option mentioned above, a data object editing interface is further displayed to the user. The data object editing interface includes: editing controls, through which the user can edit the identification information of the data object that the specified operation step depends on; the kitchen robot can respond to the user's editing operation and obtain the identification information of the data object that the specified operation step depends on.

[0066] Implementation Method B3: When the kitchen robot reaches a designated task step, and the user marks the data objects dependent on that task step using the marking options mentioned above, a list of data objects required for the kitchen robot to perform the task and an add control are further displayed. If the data object list contains the identification information of the data objects dependent on the designated task step, the user can directly select the identification information of the data objects from the list. The kitchen robot can respond to the user's selection operation and obtain the identification information of the selected data objects as the identification information of the data objects dependent on the designated task step. If the data object list does not contain the identification information of the data objects dependent on the designated task step, the user can click the add control to enter the data object editing interface. This interface has editing controls, allowing the user to edit the identification information of the data objects dependent on the designated task step. The kitchen robot can respond to the user's editing operation and obtain the identification information of the data objects dependent on the designated task step.

[0067] It should be noted that the first type of operation steps in this application embodiment includes not only operation steps that depend on data objects, but also other operation steps that users can participate in, such as operation steps that depend on user-configurable operation parameters. User-configurable operation parameters refer to operation parameters that allow users to adjust or configure in real time during the kitchen robot's operation without affecting the execution of the current task. Optionally, these user-configurable operation parameters can be operation parameters that are easy for users to configure, such as, but not limited to, operation parameters for which configuration controls (physical buttons or touch buttons on a touch panel) are provided on the kitchen robot and / or remote control. For example, users can configure or adjust user-configurable operation parameters through touch buttons on the kitchen robot's touch panel; or, users can configure or adjust user-configurable operation parameters through physical buttons on the kitchen robot's remote control. In the case where the kitchen robot is a smart cooking machine, the operation parameters that the smart cooking machine allows users to configure in real time during its operation can be, but are not limited to, heat level, stirring speed, etc., and the cooking machine's touch panel is equipped with heat adjustment controls and stirring speed adjustment controls. For operation steps that rely on user-configurable operation parameters, the adjustment or configuration of operation parameters can be participated in by the user and does not necessarily have to be automatically executed by the kitchen robot. Therefore, it can also be regarded as the first type of operation step in the embodiments of this application.

[0068] When the first type of operation steps includes operation steps that depend on user-configurable operation parameters, the method for generating prompt information corresponding to the first type of operation steps includes generating prompt information to prompt the user to configure user-configurable operation parameters. That is, this prompt information is used to prompt the user to set or adjust the operation parameters in the first type of operation steps. Accordingly, under the prompt of this prompt information, the auxiliary operation performed by the user is to set or adjust the user-configurable operation parameters. Taking a kitchen robot as a cooking machine as an example, the reference structured data can be implemented as a reference electronic recipe. The prompt information corresponding to the user-configurable operation parameters included in the reference electronic recipe can be "Please increase the heat", "Adjust the stirring speed", "Decrease the heat", etc.

[0069] In some optional embodiments of this application, the multiple operation steps performed by the kitchen robot may include not only the first type of operation steps but also a second type of operation steps. The second type of operation steps are other operation steps besides the first type. It should be noted that for the kitchen robot, the second type of operation steps are optional; that is, all the multiple operation steps performed by the kitchen robot can be of the first type. When the kitchen robot's multiple operation steps include a second type of operation step, this second type of operation step can be an operation step using non-user-configurable operation parameters. These non-user-configurable operation parameters are preset in the operation step and cannot be adjusted or configured by the user through interaction. Therefore, during the generation of reference structured data, it is not necessary to generate corresponding prompt information for the second type of operation steps. However, for the second type of operation steps among the multiple operation steps, the non-user-configurable operation parameters used by the kitchen robot when performing the second type of operation steps can be recorded. In other words, the reference structured data includes the second type of operation steps and their corresponding non-user-configurable operation parameters.

[0070] It should be noted that the user-configurable and non-user-configurable operation parameters mentioned above refer to the operation parameters that the kitchen robot needs to use when performing corresponding operation steps. These operation parameters can be reflected in the working status or status parameters of the kitchen robot.

[0071] In some alternative embodiments of this application, the kitchen robot supports two creation modes: a first creation mode and a second creation mode. The first creation mode generates reference structured data, and the second creation mode generates follow-up structured data. Follow-up structured data is structured data that guides the kitchen robot to automatically perform tasks. In practical applications, users can select one of the creation modes according to their needs for creating structured data, instructing the kitchen robot to perform the corresponding structured data creation process based on the user-selected mode. Based on this, as... Figure 1b As shown, before the kitchen robot performs its tasks, i.e. before step 101a, the structured data generation method further includes the following steps:

[0072] 101b. Display mode settings interface, which includes at least a first creation mode and a second creation mode;

[0073] 102b. Respond to the user's selection action and confirm that the user has selected the first creation mode.

[0074] In this embodiment, if the user selects the first creation mode, it means that the kitchen robot needs to generate reference structured data. Therefore, the kitchen robot performs the operations of steps 101a-103a to generate reference structured data.

[0075] In an optional embodiment, the user can also select a second creation mode, in which the kitchen robot can generate and process structured data while performing tasks. Figure 1c This application provides a flowchart illustrating a method for generating structured data, as shown in the embodiments below. Figure 1c As shown, the method includes:

[0076] 101c. During the kitchen robot's task execution, record the multiple work steps performed by the kitchen robot and the execution order between the multiple work steps;

[0077] 102c. For the third type of job steps among multiple job steps, respond to the user's configuration operation and record the attribute information of the data objects that the user configures for the third type of job steps. The third type of job steps are job steps that need to depend on data objects but do not provide data object identification information.

[0078] 103c. Generate structured data based on multiple job steps, the execution order between multiple job steps, and the attribute information of the data objects on which the third type of job steps depend.

[0079] For a detailed description of the implementation of step 101c, please refer to the description of step 101a in the foregoing embodiments, which will not be repeated here. In this embodiment, the multiple operation steps executed by the kitchen robot include: a third type of operation step, which is an operation step that depends on a data object but does not provide data object identification information, that is, it is known that the operation step depends on the operation step, but it is not known which data object(s) the operation step specifically needs; for the third type of operation step, the user can configure the attribute information of the data object it depends on, and the kitchen robot can respond to the user's configuration operation and record the attribute information of the data object it depends on configured for the third type of operation step, such as the identifier, type or quantity of the data object; then, according to the multiple operation steps, the execution order between the multiple operation steps and the attribute information of the data object depended on by the third type of operation step, structured data is generated.

[0080] Optionally, before configuring the attribute information of the data objects that the third type of work steps depend on, it is possible to first determine which of the multiple work steps are third type work steps. For work steps that clearly depend on data objects, keywords corresponding to this type of work step can be pre-configured. Based on this, during the kitchen robot's work task execution, the currently executed work step can be matched with the pre-configured keywords. If a match is found with the pre-configured keywords, it is further determined whether the currently executed work step contains the attribute information of the data object. If not, the currently executed work step is determined to be a third type work step; otherwise, it is determined that the currently executed work step is not a third type work step. For work steps that clearly do not depend on data objects, because they cannot match the pre-configured keywords, they will not be identified as third type work steps.

[0081] However, in practical applications, some operation steps may or may not depend on data objects. These types of operation steps cannot be easily identified by matching preset keywords. In this embodiment, these operation steps that may or may not depend on data objects are referred to as specified operation steps. Specified operation steps are pre-configured, and marking options are set for them, allowing users to mark whether a specified operation step depends on a data object as needed during the actual execution of the kitchen robot's tasks. Because it is unknown whether a specified operation step depends on a data object, the attribute information of the data object it depends on is also unknown. Therefore, if the user marks a specified operation step as needing to depend on a data object, that specified operation step is a third-category operation step; otherwise, it is not a third-category operation step.

[0082] Based on the above, during the kitchen robot's operation, in addition to recording the multiple operation steps performed by the kitchen robot, the current operation step can be displayed to the user through the kitchen robot's display screen or the display screen of a terminal device bound to the kitchen robot, according to the execution order of the multiple operation steps. Furthermore, when the kitchen robot performs a specific operation step, a marking option for that specific operation step can be displayed, allowing the user to mark whether the specified operation step is a dependent data object. If the specified operation step is marked as a dependent data object, but the attribute information of the data object on which the specified operation step depends cannot be obtained in advance, then the specified operation step is classified as a third type of operation step, and the user configures the attribute information of the data object on which the specified operation step depends. Here, a specified operation step is a type of operation step that may require a dependent data object. The method by which the user configures the attribute information of the data object on which the specified operation step depends is similar to the implementation methods B1-B3 above, and will not be repeated here.

[0083] Optionally, the kitchen robot uses certain operational parameters when performing each task step. Therefore, in addition to recording the multiple task steps performed by the kitchen robot, the operational parameters used by the kitchen robot in each task step can also be recorded. These operational parameters will be stored in the structured data. That is to say, the structured data includes not only the multiple task steps, the execution order between the multiple task steps, and the attribute information of the data objects on which the third type of task steps depend, but also the operational parameters used when performing each task step.

[0084] In one optional embodiment, after the kitchen robot generates a follow-up data package or a reference data package, it can also perform modification operations on the generated follow-up data package or reference data package. The modification operations include at least one of the following: adding work steps, deleting work steps, modifying work parameters, adding videos or images corresponding to the work steps, and modifying the name of structured data. Based on this, if the kitchen robot generates a follow-up data package, the follow-up structured data is displayed on a first preview interface, and the user can perform modification operations on the follow-up structured data; the kitchen robot can perform modification operations on the follow-up structured data in response to the modification operation initiated by the user on the first preview interface; if the kitchen robot generates a follow-up data package, the reference structured data is displayed on a second preview interface, and the user can perform modification operations on the reference structured data; the reference structured data is modified in response to the modification operation initiated by the user on the second preview interface. The correction operation includes at least one of the following operations: adding or deleting a stirring step; adding or deleting a feeding step; increasing or decreasing heating time; increasing or decreasing heating power; increasing or decreasing stirring speed; adding or deleting images, videos, or pictures that guide the operation steps; adding or deleting images, videos, or pictures that display the operation progress in real time; adding or deleting a lid-opening step in any step of following or referencing the structured data; adding or deleting a lid-closing step in any step of following or referencing the structured data.

[0085] Alternatively, for safety reasons, a step to adjust the stirring spatula needs to be added after the closing step; to ensure the stability of the structured data, a step to open the lid needs to be added at the end of the structured data; or a step to close the lid needs to be added at the end of the structured data.

[0086] In terms of product implementation, the kitchen robot can simultaneously support two creation modes, allowing users to choose between them. Based on the user's selection, it can generate corresponding follow-up structured data or reference structured data. Alternatively, the kitchen robot can also support only the first creation mode or only the second creation mode. If the kitchen robot only supports the first creation mode, it can then... Figure 1a The process shown generates reference structured data; if the kitchen robot only supports the second creation mode, then it can be done according to... Figure 1c The process shown generates and processes structured data.

[0087] In addition, the above Figures 1a-1c The execution subject of the method embodiment shown can be a kitchen robot, meaning all operations are performed on the kitchen robot; or it can be implemented in conjunction with a terminal device. When implemented in conjunction with a terminal device, the process of generating structured data mainly consists of three parts: ① The user performs pre-editing operations through the terminal device to determine the data objects required for the task, and the terminal device sends the pre-edited data objects to the kitchen robot; ② The user controls the kitchen robot to generate reference or follow-up structured data while performing the task, and controls the kitchen robot to provide the generated structured data to the terminal device; ③ The user performs correction operations on the reference or follow-up structured data through the terminal device to obtain the final structured data. Detailed descriptions of each part can be found above and will not be repeated here.

[0088] In this embodiment, the process of generating structured data can be completed not only by the kitchen robot alone, or by the kitchen robot and the terminal device working together, but also by the terminal device alone. Figure 1d Another structured data generation method provided as an exemplary embodiment of this application is applicable to terminal devices bound to kitchen robots, such as... Figure 1d As shown, the method includes:

[0089] 101d. Display the operation control interface and control the kitchen robot to perform operation tasks based on the controls on the operation control interface;

[0090] 102d. During the kitchen robot's task execution, receive and display the current task steps performed by the kitchen robot;

[0091] 103d. When the currently executed task is a specified task, in response to the user's marking operation of marking it as a first-class task, generate a prompt message corresponding to the first-class task.

[0092] 104d. Generate reference structured data based on the work steps executed sequentially by the kitchen robot and the prompt information corresponding to the first type of work steps; wherein, the prompt information is used to prompt the user to perform auxiliary operations adapted to the first type of work steps when any kitchen robot performs a work task based on the reference structured data and reaches the first type of work steps, so as to cooperate with any kitchen robot to complete the work task.

[0093] In this embodiment, the controls on the operation control interface can control the kitchen robot to perform actions. If the kitchen robot is a cooking device, the operation control interface includes, but is not limited to, controls for heat output, stirring, opening / closing the lid, and pausing.

[0094] In this embodiment, the tagging operation refers to the process by which a user configures the identification information of the data objects that a specified job step depends on. For details regarding the process of configuring the identification information of the data objects that a user depends on a specified job step, or other aspects of this embodiment, please refer to the foregoing embodiments, which will not be repeated here.

[0095] Figure 1e Another structured data generation method provided for exemplary embodiments of this application, applicable to terminal devices bound to kitchen robots, such as... Figure 1e As shown, the method includes:

[0096] 101e. Display the operation control interface and control the kitchen robot to perform operation tasks based on the controls on the operation control interface;

[0097] 102e. During the kitchen robot's task execution, receive and display the current task steps performed by the kitchen robot;

[0098] 103e. When the currently executed job step is a specified job step, in response to the user's marking operation of marking it as a third type of job step, the object configuration interface is displayed. The third type of job step refers to a job step that depends on a data object.

[0099] 104e. Respond to the user's configuration operation on the object configuration interface and record the attribute information of the data objects that the user configures for the third type of operation steps and which they depend on.

[0100] 105e. Generate structured data based on the sequential operation steps performed by the kitchen robot and the attribute information of the data objects on which the third type of operation steps depend.

[0101] In this embodiment, the object configuration interface allows users to configure the attribute information of data objects. Through this interface, users can manually or verbally input the attribute information of data objects. Alternatively, the object configuration interface can also display a list of data objects selected for the kitchen robot during the pre-editing operation, allowing users to select the data objects that the third type of operation step depends on, and further display the attribute information configuration interface for that data object, including configuration items such as data object type, quantity, and name, for users to configure the attribute information of the data object. Other aspects of this embodiment can be found in the foregoing embodiments and will not be repeated here.

[0102] In this embodiment, regardless of which method is used to generate the structured data, the generated structured data can be used to guide any kitchen robot in performing its tasks. The following description illustrates the process of a kitchen robot performing tasks using a user's reference to the structured data.

[0103] Figure 2a A flowchart illustrating a job task execution method provided as an exemplary embodiment of this application is shown below. Figure 2a As shown, the method includes:

[0104] 201a. Obtain reference structured data, which includes multiple operation steps in the kitchen robot's task, the execution order between the multiple operation steps, and the prompt information corresponding to the first type of operation step among the multiple operation steps;

[0105] 202a. Execute multiple job steps sequentially according to their execution order;

[0106] 203a. When the first type of operation step is executed, output a prompt message to prompt the user to perform auxiliary operations that are compatible with the first type of operation step in order to cooperate with the kitchen robot to complete the operation task.

[0107] In this embodiment, the method of generating the reference structured data is not limited. For example, the reference structured data may be created by the user based on the creation mode of the kitchen robot, in which case the reference structured data can be directly obtained from the local storage of the kitchen robot; another example is that other users publish their created reference structured data to the Internet or a server, and the kitchen robot obtains the reference structured data published by other users to the Internet or a server; yet another example is that a terminal device bound to the kitchen robot obtains the reference structured data published by other users to the Internet or a server and provides it to the kitchen robot.

[0108] In this embodiment, the reference structured data includes prompts. With the prompts, users can adjust and intervene in the process of the kitchen robot performing tasks, thus realizing semi-automatic execution of tasks by the kitchen robot. The execution of tasks has a certain degree of flexibility, which is conducive to meeting user needs.

[0109] In one optional embodiment, the first type of job steps includes job steps that depend on data objects, and the prompt information is used to prompt the user to operate on the data objects on which the first type of job steps depend; or the first type of job steps includes job steps that depend on user-configurable job parameters, and the prompt information is used to prompt the user to configure or adjust the user-configurable job parameters. For details regarding job steps that depend on data objects and job steps that depend on user-configurable job parameters, please refer to the foregoing embodiments, and they will not be repeated here.

[0110] Optionally, during the execution of work steps by the kitchen robot, if the first type of work steps includes work steps that depend on user-configurable work parameters, the kitchen robot will issue a prompt to the user, who can then configure or adjust the user-configurable work parameters based on the prompt. Specifically, the user can send a parameter adjustment command to the kitchen robot, which includes new user-configurable work parameters required for the kitchen robot to perform its work. In response to the user's parameter adjustment command, the kitchen robot can be controlled to execute subsequent work steps based on the new user-configurable work parameters.

[0111] In one optional embodiment, after performing the auxiliary operation, the user can send a confirmation message to the kitchen robot. For example, the confirmation message can be sent through the kitchen robot's display screen or the display screen of a terminal device bound to the kitchen robot. Alternatively, the confirmation message can be sent through the kitchen robot's remote control. Another example is that the kitchen robot includes a voice module, allowing the user to send confirmation messages via voice. The kitchen robot's voice module recognizes the user's voice and receives the confirmation message. Upon receiving the confirmation message, the kitchen robot, in response to the confirmation message sent by the user after performing the auxiliary operation, can determine whether the following operation steps exist in the first type of operation step. If they exist, it means that the reference structured data has not yet been completed, and the next operation step is executed; if they do not exist, it means that the reference structured data has been completed, and the operation task is terminated.

[0112] In addition, after the kitchen robot issues a prompt, the user does not need to send a confirmation message. The kitchen robot can be set to countdown. After the countdown ends, the user will have completed the auxiliary operation by default. The kitchen robot can then proceed to the next operation step or end the operation.

[0113] Scenario-based implementation examples:

[0114] In this embodiment, taking a kitchen robot as the cooking device, structured data as the reference electronic recipe, and following the structured data as following the electronic recipe as an example, the generation and usage of structured data are explained. The structure of the cooking device is not limited. Figure 3 An exemplary structure of a cooking device is given. For example... Figure 3 As shown, the cooking device 300 includes: a pot body 301, a pot lid 302 and an MCU 303, a seasoning container 304 (including at least one seasoning container, each containing at least one seasoning), and a stirring spatula located inside the pot body 301. Figure 3(Not shown in the image), heating base 305, base 306 for supporting heating base 305, bracket 307, display screen 308, and measuring device 309. The MCU 303, acting as the brain of the cooking device 300, controls the cooking device 300 to perform cooking actions. For example, the MCU 303 can control the opening and closing of the pot lid 302; the MCU 303 can control the rotation of the seasoning container 304 to dispense seasonings; the MCU 303 can also control the rotation speed of the stirring spatula or the heating power of the heating base 305, etc.

[0115] Furthermore, the cooking device also includes a communication module 310, which can implement short-range communication technologies such as Wi-Fi, Bluetooth, infrared, or radio frequency identification (RFID). The cooking device can establish a wireless connection with the terminal device based on the communication module.

[0116] In this embodiment, the terminal device is equipped with an APP corresponding to the cooking equipment. The user enters the creation mode through the APP. The creation mode includes: creating reference electronic recipes and creating follow-up electronic recipes. Figure 4a This is the interface for the creation mode. At the top of the screen is a help video explaining the differences between the two modes, making it easier for users to understand. At the bottom of the screen is the selection menu for creation mode; users can choose to create a reference e-recipe or create a follow-up e-recipe. Users can click... Figure 4a The first control in the app is for creating and making electronic recipes, or you can click... Figure 4a The second control in the system is used to create reference electronic recipes.

[0117] When a user selects the first control to create and execute an electronic recipe, the user can perform pre-editing operations on the terminal device, recording the types of ingredients and seasonings required for the cooking equipment to perform its tasks, such as... Figure 4b and 4c As shown. Figure 4b As shown, the terminal device displays a pre-editing interface to the user, which shows various common ingredients, including but not limited to: pork belly, ribs, fish, chicken, mushrooms, etc. The user can select the type of ingredient needed to perform the task. If the required ingredient is not among the common ingredients, the user can click the ingredient addition control on the pre-editing interface. The terminal device responds to the user's click by displaying an information input interface for the user to enter the identifier information of the ingredient to be added. For example... Figure 4cAs shown, the pre-editing interface also allows users to select the types of seasonings to be added to each seasoning box. Each seasoning box can contain one or more seasonings; there are no restrictions. For each seasoning box, the pre-editing interface displays common seasonings, which may include, but are not limited to: peanut oil, fine salt, light soy sauce, dark soy sauce, aged vinegar, rice vinegar, etc. Users can select the seasonings needed for the task. If the required seasoning is not listed among the common seasonings, the user can click the corresponding seasoning addition control on the pre-editing interface. The terminal device responds to the user's click by displaying an information input interface for the user to input the identifier information of the seasoning to be added. For example... Figure 4c As shown, the seasonings selected by the user for seasoning box 1 include: peanut oil, fine salt, light soy sauce, and aged vinegar; the seasoning selected by the user for seasoning box 2 is fine salt. In addition, in Figure 4b In addition, users can also add names to data packets.

[0118] Next, after the user performs pre-editing operations through the terminal device's app, the terminal device sends the pre-edited data to the cooking device. The cooking device can then assist the user in setting the seasoning containers based on this data, that is, assist the user in adding the seasonings to the corresponding containers. For example... Figure 4d The image shows the weighing interface of the seasoning boxes. Seasoning box 1 requires the addition of peanut oil, fine salt, light soy sauce, and aged vinegar. Figure 4d The system only displays the addition of peanut oil and fine salt; light soy sauce and aged vinegar are not shown. Users can weigh the seasonings to be added to seasoning container 1 using the measuring device of the cooking equipment, such as an electronic scale. The currently weighed seasoning is highlighted. After the measuring device obtains the amount of seasoning used, the cooking equipment can record the amount. Users add the weighed seasonings to seasoning container 1. Subsequently, using the same method, other seasonings in seasoning container 1 or other seasoning containers can be weighed, recorded, and added to the container sequentially, ending the seasoning process and starting cooking. For seasonings not in the user's pre-edited data, they can be added to the container during the container settings via the weighing interface, and then weighed, recorded, and added. Figure 4d As shown, the ingredient weighing interface has an add control that users can trigger to add seasonings to the container. The cooking equipment can respond to user clicks by displaying a common seasoning selection interface for the user to choose from. Alternatively, if the common seasonings do not meet the user's needs, the user can trigger the seasoning add control on the common seasoning selection interface and enter the identifier information of the seasoning to be added.

[0119] In this embodiment, in addition to supporting the addition of seasonings to the seasoning box via the weighing interface, the user can also delete the identification information of seasonings to be added to the seasoning box, such as... Figure 4eAs shown, users can select the identifier of the seasoning to be deleted and slide it to the left. The cooking device responds to the user's sliding operation and displays a seasoning deletion control. Users can click on the seasoning deletion control to delete the seasoning's identifier.

[0120] In this embodiment, the user can view a cooking control interface on the display screen of the cooking device (pot end). The user performs tasks based on this interface while simultaneously generating structured data. For example, Figure 4f This is a schematic diagram of a cooking control interface. On the left side of the interface, the specified operation steps of the cooking equipment are displayed. For example, the specified operation step is the currently executed operation step, or it may show operation steps that the cooking equipment has already completed and the currently executed operation step. On the right side of the interface, interactive controls for the cooking equipment are displayed. These controls include: heat control, stirring control, ingredient addition control, lid opening control, lid closing control, pause control, and end control. The heat control is used to adjust the heat level during the operation, with options for: off, low, medium, and high. Figure 4f The diagram illustrates the target heat level as high. The stirring control adjusts the stirring spatula's rotation speed, which can be set to off, low, medium, or high. The ingredient dispensing control instructs the seasoning dispenser to rotate, dispensing the appropriate seasoning. The open and close lid controls are combined buttons. Clicking the open lid control opens the lid, and the combined control displays "Lid Closed." Clicking the close lid control closes the lid, and the combined control displays "Lid Open." The pause control pauses the cooking process. In this state, the stirring spatula stops rotating, the heat is turned off, ingredient dispensing stops, and the lid remains in its current state. Users can continue to click the pause control while the cooking process is paused, and the cooking equipment will resume its operation. After the cooking process is complete, users can click the end control to terminate the operation.

[0121] In this embodiment, for a specific operation step, such as the lid-opening operation step, the cooking control interface can display marking options for the lid-opening operation step. For example... Figure 4f As shown, the options for the "open lid" action are: open lid to add ingredients, open lid to cook, or open lid for other operations. Furthermore, when the user selects the "open lid to add ingredients" option, the cooking device can display an ingredient selection interface, such as... Figure 4gAs shown, the ingredient selection interface displays the identifiers of pre-edited ingredients. Users can select the identifiers of ingredients to be added during the current lid-opening action. If additional ingredients besides the pre-edited ones are added during this action, the user can input their identifiers. If the user selects a marking option for other lid-opening operations, the cooking device provides editing controls for inputting specific actions to be performed. Specifically, during task execution, a lid-opening marking function is provided. If the user marks the lid-opening action, the structured data after generating the electronic recipe will directly display the lid-opening action and any accompanying actions, such as adding ingredients and seasonings. This addresses the issue where the cooking device cannot detect the user adding ingredients when opening the lid, resulting in the generated electronic recipe lacking corresponding actions for the lid-opening action. The marking option significantly reduces the user's memory burden.

[0122] In this embodiment, after the user controls the cooking equipment to perform a task, the kitchen robot can automatically generate a follow-up electronic recipe. The cooking equipment can display the follow-up electronic recipe on the terminal device's screen, and the user can also perform correction operations on the follow-up electronic recipe. Figure 4h The preview interface shown displays a follow-up electronic recipe for "Changzhou Spicy Beef Noodles." The preview includes a "Go to Edit" control, which users can click. The terminal device responds to the user's click, displaying the editing interface. Users can modify the cooking parameters in the follow-up electronic recipe, such as adjusting the heat, stirring speed, and cooking time for certain steps. In addition, users can add or delete cooking steps. When adding steps, fault tolerance controls are implemented to prevent data packet anomalies caused by step adjustments. For example, an "open lid" step can be added at the end of the follow-up electronic recipe; a "close lid" step can be added at the end. Furthermore, for safety reasons, a stirring step is added after the "close lid" step to prevent the stirring motion of the spatula from harming the user's health when the lid is opened. In addition, users can add videos or images corresponding to the cooking steps, modify the names of structured data, or adjust the types or weights of ingredients or seasonings.

[0123] In this embodiment, the process of creating a follow-up data package is the same as or similar to creating a reference data package. The difference lies in that when creating a reference electronic recipe, the user does not need to accurately weigh the seasonings or put them into the seasoning box. The user can choose the type and amount of seasonings according to their preferences while opening the lid. Correspondingly, during the execution of the task, the user needs to mark the task steps for adding data objects (e.g., seasonings and ingredients) and generate prompt information for that task step. The follow-up electronic recipe includes multiple task steps, the execution order between multiple task steps, and attribute information of some data objects in some task steps (e.g., the type, identification, or amount of ingredients or seasonings, etc.), while the reference electronic recipe includes multiple task steps, the execution order between multiple task steps, and prompt information corresponding to some task steps. Other content is the same as or similar to the reference electronic recipe and the follow-up electronic recipe; for details, please refer to the content of the follow-up electronic recipe.

[0124] In this embodiment, users can perform tasks by following an electronic recipe or by referring to a reference electronic recipe. The difference lies in the following: when performing tasks by following an electronic recipe, the cooking equipment automatically performs the task, and the user does not need to perform any auxiliary operations. In this mode, the user cannot adjust or intervene in the task steps. However, when performing tasks by referring to a reference electronic recipe, the user can adjust and intervene in the task execution process based on prompts, allowing the user to perform tasks according to their own preferences. This makes the task execution more flexible and better meets the user's needs. At the same time, the prompts can greatly reduce the user's memory burden and improve the user experience.

[0125] The following describes the process of users performing cooking tasks using reference electronic recipes. The cooking device supports both follow-up and reference modes. In follow-up mode, users do not need to intervene or adjust the cooking process; the kitchen robot can perform the task according to the electronic recipe. Reference mode is mainly for users with some culinary skills. Users who want to intervene or adjust the cooking process can choose reference mode to perform the cooking task based on the reference electronic recipe.

[0126] Users can pre-edit the types of ingredients and seasonings on the terminal device. For example, the ingredients are tomatoes, and the seasonings are cooking oil, salt, and chicken bouillon. After confirming the ingredients and seasonings, the terminal device provides the pre-edited content to the cooking device. The cooking device executes the cooking process, turning on high heat and automatically opening the lid. It then prompts the user to add cooking oil, which the user adds as instructed. The amount of oil is controlled by the user. After a 5-second countdown, the cooking device assumes the user has added oil. Next, the cooking device prompts the user to add chopped tomatoes. After adding the tomatoes, the user confirms their addition via controls on the display screen. Upon receiving confirmation, the cooking device closes the lid, runs the stirring spatula at medium speed, and after a short time, closes the stirring spatula, opens the lid, and prompts the user to add salt and chicken bouillon. After adding these, the user sends a confirmation message to the cooking device, which then turns off the heat after 5 seconds, ending the cooking task.

[0127] In this embodiment, the kitchen robot has a voice function, allowing users to interact with it via voice. Users can issue control commands to the robot via voice, and the robot can also output various information to the user via voice. The voice function of the kitchen robot in this embodiment includes: voice recognition, sound pickup, and voice broadcasting. The sound pickup function is mainly used to pick up the voice signal emitted by the user, or other audio signals from the environment in which the kitchen robot is located. The voice recognition function is mainly used to convert the picked-up voice signal or other audio signal into text, and then identify control commands for controlling the kitchen robot from the text information. The voice broadcasting function is used to convert some information that the kitchen robot needs to output into voice signals and broadcast them to the user via voice.

[0128] For example, users can issue voice commands such as "Please start work" or "Execute work task" to instruct the kitchen robot to begin performing its tasks. Furthermore, users can issue voice commands such as "Please increase power" or "Please open the lid" to control the kitchen robot in real time during task execution. Alternatively, users can issue voice commands such as "Current working status," "Current power used," or "Current work progress" to query the kitchen robot's operating / working status. The kitchen robot can then return its operating / working status information to the user based on the voice commands. Moreover, the kitchen robot can also pick up other audio signals in the working environment, such as alarm sounds from the fume sensor, the sound of broken porcelain, oil popping, noise from food burning in a pot, or noise from the range hood. It can identify these other audio signals, and if these sounds are specific to certain events, it can output reminders or alarm messages to the user.

[0129] In this embodiment, the kitchen robot may include a voice module that provides voice functionality to the kitchen robot. The voice module may be a hardware module, such as a voice chip, but is not limited thereto. Figure 2b An implementation structure for the voice module of a kitchen robot is presented. For example... Figure 2b As shown, the voice module 200b of the kitchen robot includes: a sound receiving module 201b, a voice recognition module 202b, a main control module 203b, and a broadcasting module 204b.

[0130] The audio receiving module 201b is used to pick up the user's voice signal (hereinafter referred to as user voice) and / or other sound signals in the environment where the kitchen robot is located. The audio receiving module 201b can be, but is not limited to, a microphone, microphone array, Bluetooth microphone, voice recorder, voice recorder, recorder, microphone, etc. The audio output module 204b is used to output sound signals to the outside world, such as a speaker, megaphone, loudspeaker, etc.

[0131] The speech recognition module 202b analyzes and processes the sound signals (e.g., user voice or noise in the kitchen robot's environment) collected by the sound receiving module 201b. Specifically, the speech recognition module 202b inputs the user voice picked up by the sound receiving module 201b into a deep neural network model for recognition. It matches the recognized voice commands against a pre-stored correspondence between voice commands and function codes. If a matching function code is found, it sends the corresponding function code to the main control module 203b, which then controls the kitchen robot to perform corresponding actions based on the function code. The function code refers to machine language that the main control module 203b can recognize. Based on the function code, the main control module 203b can control other modules of the kitchen robot (e.g., the pot lid drive module or the stirring spatula drive module) to perform corresponding actions.

[0132] One voice command may have multiple different voice expressions. For example, if the voice command is "open the lid," the voice recognition module 202b may recognize the voice signal as, but is not limited to, "open the lid," "please open the lid," or "open the pot lid." In this embodiment, the voice recognition module 202b uses a deep neural network model, which can accurately recognize voice signals under different expressions. The deep neural network model can be an offline model, meaning it is pre-trained offline for voice recognition and stored in the kitchen robot itself. This method is not network-dependent, has low implementation costs, and provides users with a more convenient and intelligent operating experience.

[0133] In this embodiment, the main control module 203b can control the kitchen robot to perform corresponding actions according to the function codes sent by the voice recognition module 202b. Taking the kitchen robot as a cooking device as an example, the functional modules of the cooking device include, but are not limited to: a pot lid control module, a heating module, a seasoning box control module, a stirring spatula control module, and a voice module, etc. The functions that the main control module 203b can control the functional modules of the cooking device to achieve are shown in Table 1 below.

[0134] In this embodiment, the kitchen robot stores not only the correspondence between voice commands and function codes, but also the correspondence between voice commands and query codes. The query code is a machine language that the main control module 203b can recognize, used to query the operating status / working status of various functional modules of the kitchen robot. Users can query the operating status / working status of various functional modules of the kitchen robot via voice. Specifically, the voice recognition module 202b recognizes the user's voice with a query purpose and matches the recognized voice command against the pre-stored correspondence between voice commands and query codes. If a matching query code is successfully found, it is sent to the main control module 203b. The main control module 203b can then query the operating status of the corresponding functional module or sensor based on the query code. After the main control module 203b queries the operating status of the functional module or sensor, it broadcasts the queried content through the broadcasting module 204b. For example, if the user's voice is "What is the current firepower level?", the sound receiving module 201b picks up the voice, and the voice recognition module 202b recognizes the voice to obtain...

[0135] Table 1

[0136]

[0137] The voice command is: "Query firepower". The voice recognition module 202b sends the query code corresponding to this voice command to the main control module 203b. The main control module 203b queries the operating status of the heating module. If the current firepower is "medium", the main control module 203b provides the query information "current firepower is medium" to the broadcast module 204b, and the broadcast module 204b broadcasts the query information "current firepower is medium".

[0138] In this embodiment, in addition to recognizing user speech, the voice recognition module 202b can also identify and process noise in the environment where the kitchen robot is located, as picked up by the audio receiving module 201b. Upon identifying specific noise, the broadcasting module 204b will issue reminder or alarm information. Specifically, the noise identification process includes: pre-collecting specific noise, inputting the collected noise into a neural network model for training to form a set of specific noise; comparing the noise in the kitchen robot's environment, picked up in real-time by the audio receiving module 201b, with the specific noise; if the similarity between the picked-up noise and the specific noise in the set exceeds a set similarity threshold, the picked-up noise is considered the specific noise.

[0139] Furthermore, upon detecting specific noise, the voice recognition module 202b sends the function code corresponding to that specific noise to the main control module 203b, which then performs the corresponding operation based on the function code. The function code corresponding to the specific noise can instruct the main control module 203b to perform a specific action. For example, if the function code corresponding to the specific noise instructs the main control module 203b to perform a corresponding action, when the voice recognition module 202b recognizes "dry-frying ingredients," it sends the corresponding function code—that is, the instruction to reduce or turn off the heat—to the main control module 203b, which then performs the action of reducing or turning off the heat.

[0140] Alternatively, upon detecting a specific noise, the voice recognition module 202b sends a notification to the main control module 203b. Based on this notification, the main control module 203b then uses the broadcast module to issue a reminder or alarm to the user. For example, when the voice recognition module 202b detects an oil popping sound, it sends a notification to the main control module 203b. The main control module 203b then retrieves a reminder message: "Please turn down the heat or turn off the heater," and provides this message to the broadcast module 204b. The broadcast module 204b then broadcasts the reminder message, and the user, upon hearing it, turns down the heat or turns off the heater.

[0141] Alternatively, the kitchen robot may be equipped with a smoke detector. When the smoke detector detects that the amount of smoke in the robot's environment exceeds a set smoke threshold, it sends a reminder to the main control module 203b, prompting it to check if the range hood is on. Upon receiving this reminder, the main control module 203b uses the voice recognition module 202b to identify the noise in the robot's environment, which is picked up in real-time by the audio receiver 201b. The voice recognition module 202b then determines whether the picked-up noise is generated by the range hood. If the similarity threshold between the picked-up noise and the noise generated by the range hood exceeds a set similarity threshold, the noise is considered to be generated by the range hood, indicating that the range hood is on. If the similarity threshold does not exceed the set similarity threshold, the noise is considered not to be generated by the range hood, indicating that the range hood is not on. When the voice recognition module 202b detects that the range hood is not turned on, it sends a notification message to the main control module 203b to notify the main control module 203b that the range hood is not turned on. The main control module 203b then sends a reminder message to the user through the broadcast module 204b to remind the user to turn on the range hood.

[0142] In this embodiment, the installation location of the microphone module 201b is not limited. In one optional embodiment, the microphone module 201b adopts a detachable design. It can be embedded in the bottom of the kitchen robot's display screen, facing the user. The position of the microphone module 201b is fixed and isolated from the microphone module 201b with soft adhesive, which reduces vibration, enhances the sound reception effect, and effectively reduces oil stains. Simultaneously, the microphone module 201b uses a wired transmission mode with other modules, increasing data transmission speed. In another optional embodiment, the microphone module 201b adopts a detachable design and can be worn around the user's collar. Wireless communication technology is used to achieve data transmission between the microphone module 201b and other modules, ensuring sound reception quality. In yet another optional embodiment, the microphone module 201b can have two types: one installed on the kitchen robot to collect noise from the environment in which the robot operates, and the other worn around the user's collar to ensure the reception of audio commands. The radio module 201b features a detachable design, making it easy to clean and preventing oil stains from forming on its surface due to prolonged uncleanliness of the kitchen environment, which could affect the radio reception.

[0143] In an optional embodiment, the kitchen robot can also perform voice conversion on the structured data or firmware files required to perform its tasks to obtain corresponding voice signals and output them. The firmware files include prompts such as user guides, fault alerts, or operation tips. For ease of distinction and description, in this embodiment, [the following will be used:] Figure 1a In the illustrated embodiment, the prompt information referencing the structured data is called the first prompt information. Figure 2c The prompt information in the firmware file in the illustrated embodiment is called the second prompt information.

[0144] For structured data, such as Figure 2c In the system shown, terminal device 23b or server 22b can send structured data to kitchen robot 21b. The kitchen robot (main control module 203b) receives the structured data via wireless communication module 210b (such as a Bluetooth module) and sends it to voice module 200b. Within the voice module, the structured data is parsed to obtain its name, execution flow information, data object attribute information, and execution status information of the kitchen robot 21b during task execution. Based on the parsed content, speech text is generated and sent to the TTS module in speech recognition module 202b. The TTS module converts the received speech text into a speech signal. Optionally, it can... Figure 1a In the embodiment shown, the first prompt information in the generated reference structured data is used to generate speech text, which is then converted into a speech signal by a TTS module.

[0145] For firmware files, such as Figure 2c In the system shown, terminal device 23b provides the necessary firmware files for kitchen robot 21b via Over-The-Air (OTA) technology. In this embodiment, the firmware file is referred to as the firmware OTA file. This file includes secondary prompts such as user guides, fault alerts, or operation tips. The firmware OTA file remains unchanged regardless of the specific task being performed. Modifying the prompts in the firmware OTA file requires updating the device firmware via OTA technology. Furthermore, the main control module 203b can parse the firmware OTA file, generate speech-to-text based on the parsed content, and send the generated speech-to-text to the TTS module in the speech recognition module 202b.

[0146] like Figure 2c As shown, the voice module 200b of the kitchen robot includes: a microphone module 201b, a voice recognition module 202b, a main control module 203b, and a broadcasting module 204b (the microphone module 201b is not included in the main control module 203b). Figure 2cAs shown in the diagram, it also includes: a power amplifier module 205b and a storage module 206b. The main control module 203b can be part of the voice module 200b, or it can be the main controller of the kitchen robot; the storage module 206b is used to store structured data or firmware OTA files; and the power amplifier module 205b is used to amplify the voice signal.

[0147] In this embodiment, after the main control module 203b generates the voice text, it provides the voice text to the TTS module in the voice recognition module 202b through a Universal Asynchronous Receiver / Transmitter (UART) interface or a Serial Peripheral Interface (SPI) interface. The TTS module synthesizes the voice text into a voice signal and outputs it to the power amplifier module 205b. The power amplifier module 205b amplifies the voice signal and outputs it to the playback module 204b. The playback module 204b then plays the voice signal.

[0148] In one optional embodiment, the user can send control commands to the kitchen robot via an app on a terminal device. There are two main types of control commands. The first type controls the kitchen robot, such as controlling its current state (wake-up or standby) or its voice broadcast function (volume adjustment). In the case of volume adjustment, the main control module 203b receives and parses the user's control command and adjusts the playback volume of the broadcast module 204b accordingly. The second type queries the kitchen robot's status, such as instructing the user to obtain the current power level. In this case, the main control module 203b obtains the current power level and provides it as text-to-speech (TTS) data to the TTS module in the voice recognition module 202b. The TTS module converts the text-to-speech data into a voice signal, which is then amplified by the power amplifier module 205b and broadcast by the broadcast module 204b.

[0149] In this embodiment, the process for the main control module 203b to control voice broadcast is not limited. In an alternative embodiment, the voice broadcast duration varies with the length of broadcast content, and the main control module 203b uses a First Input First Output (FIFO) queue to buffer voices to be broadcast. In another alternative embodiment, voice broadcast includes sequential broadcast and insert broadcast. For example, sequential broadcast is adopted for cooking steps, and insert broadcast is adopted for fault alerts (such as over-temperature or communication abnormality). Sequential broadcast can use FIFO API for broadcasting, and insert broadcast can use Last In First Out (LIFO) API for preferential fault broadcasting. In another alternative embodiment, after the kitchen robot is powered on, it enters a novice guidance voice broadcast, and the main control module 203b calls the FIFO API; if a user chooses to actively skip the novice guidance voice broadcast, the Over Write API is called to skip the novice guidance voice broadcast.

[0150] In an alternative embodiment, the pronunciation of polyphonic characters can be adjusted according to semantics. For example, homophones are used to replace characters prone to wrong pronunciation. For example, if the broadcast system may pronounce "dried (gān, first tone) chili" as "dry (gàn, fourth tone) chili", the main control module 203b can replace the character with the homophone "gān" when generating the voice text, so as to avoid broadcast errors of the broadcast module.

[0151] In this embodiment, it is not necessary to make a voice package, and reminder information such as data objects, execution steps, temperature warnings, timing and timekeeping in the process of performing different operation tasks can be personalized and presented in the form of voice broadcast. In addition, adding, deleting or modifying voices also does not require long-time production of single voice files for OTA; addition, deletion or modification of process voices and early warning voices can be achieved through OTA of device firmware; voice broadcast of data pairs and execution steps in the cooking process is performed through automatic identification of structured data, which does not require independent maintenance, and modifying the content of structured data is equivalent to modifying the broadcast content.

[0152] It should be noted that the execution subject of each step of the method provided in the above embodiments may be the same device, or different devices may serve as the execution subjects of the method. For example, the execution subject of steps 101a to 103a may be device A; for another example, the execution subject of steps 101a and 102a may be device A, and the execution subject of step 103a may be device B; and so on.

[0153] Furthermore, in some processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 101a, 102a, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0154] Figure 5 This is a schematic diagram of the structure of a kitchen robot provided as an exemplary embodiment of this application. Figure 5 As shown, the kitchen robot includes a memory 54 and a processor 55.

[0155] Memory 54 is used to store computer programs and can be configured to store various other data to support operation on the kitchen robot. Examples of this data include instructions for any application or method used to operate on the kitchen robot.

[0156] The processor 55, coupled to the memory 54, is used to execute the computer program in the memory 54 for: recording multiple work steps performed by the kitchen robot and the execution order between the multiple work steps during the kitchen robot's work task; generating prompt information corresponding to the first type of work steps among the multiple work steps, wherein the first type of work steps includes at least work steps that require data objects; generating reference structured data based on the multiple work steps, the execution order between the multiple work steps, and the prompt information corresponding to the first type of work steps; wherein the prompt information is used to prompt the user to perform auxiliary operations adapted to the first type of work steps when any kitchen robot performs a work task based on the reference structured data and reaches the first type of work steps, so as to cooperate with any kitchen robot to complete the work task.

[0157] In an optional embodiment, when the processor 55 generates prompt information corresponding to the first type of job steps among multiple job steps, it is specifically used to: if the first type of job step is a job step that depends on a data object, then based on the dependency behavior of the first type of job step on the data object, generate prompt information to prompt the user to operate on the data object that the first type of job step depends on, and the auxiliary operation performed by the user is to operate on the data object that the first type of job step depends on.

[0158] In an optional embodiment, before generating the prompt information corresponding to the first type of operation step, the processor 55 is further configured to: display a marking option for the specified operation step when the kitchen robot performs the specified operation step, so that the user can mark whether the specified operation step depends on a data object; if the specified operation step is marked as depending on a data object, the specified operation step is regarded as the first type of operation step, and the identification information of the data object that the user has configured for the specified operation step depends on is obtained.

[0159] In an optional embodiment, when the processor 55 obtains the identification information of the data objects configured by the user for a specified work step, it is specifically used to: display a list of data objects required for the kitchen robot to perform the work task, the data object list including the identification information of at least one data object; and in response to the user's selection operation on the data object list, obtain the identification information of the data object selected by the user as the identification information of the data object on which the specified work step depends.

[0160] In an optional embodiment, before the kitchen robot performs a task, the processor 55 is further configured to: display a pre-editing interface, the pre-editing interface including candidate data objects and / or editing controls, for the user to edit a list of data objects required for the kitchen robot to perform a task; and generate a list of data objects required for the kitchen robot to perform a task in response to the user's pre-editing operation.

[0161] In an optional embodiment, the first type of operation steps further includes operation steps that depend on user-configurable operation parameters. When the processor 55 generates prompt information corresponding to the first type of operation steps among multiple operation steps, it is specifically used to: if the first type of operation steps are operation steps that depend on user-configurable operation parameters, generate prompt information to prompt the user to configure user-configurable operation parameters, and the auxiliary operation performed by the user is to set or adjust user-configurable operation parameters.

[0162] In an optional embodiment, the processor 55 is further configured to: record, for a second type of operation step among multiple operation steps, non-user-configurable operation parameters used by the kitchen robot when performing the second type of operation step; the reference structured data also includes non-user-configurable operation parameters corresponding to the second type of operation step; the second type of operation step is an operation step other than the first type of operation step.

[0163] In an optional embodiment, before the kitchen robot performs its task, the processor 55 is further configured to: display a mode setting interface, the mode setting interface including at least a first creation mode and a second creation mode; and respond to the user's selection operation to determine that the user has selected the first creation mode; wherein the first creation mode is used to instruct the generation of reference structured data, and the second creation mode is used to instruct the generation of follow-up structured data.

[0164] In an optional embodiment, if the user selects the second creation mode, the processor 55 is further configured to: record multiple work steps performed by the kitchen robot and the execution order between the multiple work steps during the kitchen robot's work task; for a third type of work step among the multiple work steps, in response to the user's configuration operation, record the attribute information of the data object that the user configures for the third type of work step, wherein the third type of work step is a work step among the multiple work steps that needs to depend on a data object but does not provide data object identification information; and generate structured data based on the multiple work steps, the execution order between the multiple work steps, and the attribute information of the data object that the third type of work step depends on.

[0165] In an optional embodiment, the processor 55 is further configured to: record, for each operation step, the operation parameters used by the kitchen robot when performing each operation step, and the structured data also includes the operation parameters used when performing each operation step.

[0166] In an optional embodiment, the processor 55 is further configured to perform at least one of the following operations: displaying follow-up structured data on a first preview interface, and performing a correction operation on the follow-up structured data in response to a correction operation initiated by the user on the first preview interface; displaying reference structured data on a second preview interface, and performing a correction operation on the reference structured data in response to a correction operation initiated by the user on the second preview interface; wherein the correction operation includes at least one of adding a job step, deleting a job step, modifying job parameters, adding a video or image corresponding to the job step, and modifying the name of the structured data.

[0167] In an optional embodiment, the processor 55 adds a job step specifically to perform at least one of the following operations: adding a stirring step after the closing step; adding an opening step at the end of the follow-up or reference structured data; and adding a closing step at the end of the follow-up or reference structured data.

[0168] Furthermore, such as Figure 5 As shown, the kitchen robot also includes other components such as a communication component 56, a display screen 57, a power supply component 58, and an audio component 59. Figure 5 The diagram only shows some components and does not mean that the kitchen robot only includes... Figure 5 The components shown. It should be noted that... Figure 5 The components within the dashed box are optional, not mandatory, and their specific requirements depend on the product type of the kitchen robot.

[0169] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the functions provided in embodiments of this application. Figures 1a-1cThe steps in the method shown.

[0170] This application also provides a kitchen robot, the implementation structure of which is similar to... Figure 5 The kitchen robot shown has the same or similar implementation structure, which can be referred to. Figure 5 The structural implementation of the kitchen robot shown is illustrated. The kitchen robot provided in this embodiment is similar to... Figure 5 The main difference between the kitchen robots in the illustrated embodiments lies in the different functions implemented by the computer programs stored in the processor's memory. For the kitchen robot provided in this embodiment, the computer programs stored in its processor's memory can be used to: record multiple work steps performed by the kitchen robot and the execution order between these steps during the robot's task execution; for a third type of work step among these steps, in response to the user's configuration operation, record the attribute information of the data objects that the user configures for the third type of work step, where the third type of work step is a work step that depends on a data object but for which no data object identification information is provided; and generate structured data based on the multiple work steps, the execution order between them, and the attribute information of the data objects that the third type of work step depends on.

[0171] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the functions provided in embodiments of this application. Figure 1c The steps in the method shown.

[0172] This application also provides a kitchen robot, the implementation structure of which is similar to... Figure 5 The kitchen robot shown has the same or similar implementation structure, which can be referred to. Figure 5 The structural implementation of the kitchen robot shown is illustrated. The kitchen robot provided in this embodiment is similar to... Figure 5 The main difference between the kitchen robots in the illustrated embodiments lies in the different functions implemented by the computer programs stored in the memory executed by the processor. For the kitchen robot provided in this embodiment, the computer programs stored in the memory executed by its processor can be used to: acquire reference structured data, which includes multiple work steps in the kitchen robot's task, the execution order of these multiple work steps, and prompt information corresponding to the first type of work steps; execute the multiple work steps sequentially according to the execution order; and output prompt information when the first type of work step is executed, prompting the user to perform auxiliary operations adapted to the first type of work step to assist the kitchen robot in completing the task.

[0173] In one optional embodiment, the first type of operation steps includes operation steps that require data objects, and the prompt information is used to prompt the user to operate on the data objects on which the first type of operation steps depend; or the first type of operation steps includes operation steps that require user-configurable operation parameters, and the prompt information is used to prompt the user to configure or adjust the user-configurable operation parameters.

[0174] In an optional embodiment, during the kitchen robot's execution of work steps, the processor is further configured to: receive a parameter adjustment instruction from a user, the parameter adjustment instruction including new user-configurable work parameters required for the kitchen robot to perform the work; and control the kitchen robot to execute subsequent work steps according to the new user-configurable work parameters.

[0175] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the functions provided in embodiments of this application. Figure 2a The steps in the method shown.

[0176] Figure 6 This is a schematic diagram of the structure of a terminal device provided for an exemplary embodiment of this application. For example... Figure 6 As shown, the terminal device includes a memory 64 and a processor 65. It also includes a display screen 67.

[0177] Memory 64 is used to store computer programs and can be configured to store various other data to support operation on the terminal device. Examples of this data include instructions for any application or method used to operate on the terminal device.

[0178] The processor 65, coupled to the memory 64, is used to execute the computer program in the memory 64 for: displaying a work control interface on the display screen 67, and controlling the kitchen robot to perform work tasks based on the controls on the work control interface; receiving and displaying the work steps currently being performed by the kitchen robot during the execution of work tasks; generating prompt information corresponding to the first type of work steps in response to a user marking the currently performed work step as a specified work step when the currently performed work step is a specified work step; generating reference structured data based on the work steps sequentially performed by the kitchen robot and the prompt information corresponding to the first type of work steps; wherein the prompt information is used to prompt the user to perform auxiliary operations adapted to the first type of work steps when any kitchen robot performs a work task based on the reference structured data and reaches the first type of work step, so as to assist any kitchen robot in completing the work task.

[0179] Furthermore, such as Figure 6 As shown, the terminal device also includes other components such as a communication component 66, a power supply component 68, and an audio component 69. Figure 6 The diagram only shows some components and does not mean that the terminal device only includes... Figure 6 The components shown. It should be noted that... Figure 6 The components within the dashed box are optional, not mandatory, and their selection depends on the product form of the terminal device.

[0180] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the functions provided in embodiments of this application. Figure 1d The steps in the method shown.

[0181] This application embodiment also provides a terminal device, the implementation structure of which is similar to... Figure 6 The implementation structure of the terminal devices shown is the same or similar, and can be referred to. Figure 6 The terminal device shown in this embodiment is structurally implemented. The terminal device provided in this embodiment is similar to... Figure 6 The main difference between the terminal devices in the illustrated embodiments lies in the different functions implemented by the computer programs stored in the memory executed by the processor. For the terminal device provided in this embodiment, the computer programs stored in the memory executed by its processor can be used to: display a job control interface; control the kitchen robot to perform job tasks based on the controls on the job control interface; receive and display the job steps currently being performed by the kitchen robot during the job task execution process; when the currently performed job step is a specified job step, in response to the user's marking operation as a third type of job step, display an object configuration interface, where the third type of job step refers to a job step that depends on a data object; in response to the user's configuration operation on the object configuration interface, record the attribute information of the data objects that the user configures for the third type of job step; and generate structured data based on the job steps sequentially performed by the kitchen robot and the attribute information of the data objects that the third type of job step depends on.

[0182] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the functions provided in embodiments of this application. Figure 1e The steps in the method shown.

[0183] The communication components in the above embodiments are configured to facilitate wired or wireless communication between the device housing the communication component and other devices. The device housing the communication component can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0184] The memory in the above embodiments can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0185] The display screen in the above embodiments includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe action, but also the duration and pressure associated with the touch or swipe operation.

[0186] The power supply component in the above embodiments provides power to various components of the device in which the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply component is located.

[0187] The audio component in the above embodiments can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0188] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0189] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0190] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0191] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0192] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0193] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0194] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0195] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0196] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for generating structured data, characterized in that, include: During the kitchen robot's operation, multiple operation steps performed by the kitchen robot and the execution order between the multiple operation steps are recorded, and the multiple operation steps include at least the lid opening step; When the kitchen robot performs the lid-opening step, a marking option for the lid-opening step is displayed so that the user can mark whether the lid-opening step depends on a data object; If the opening step is marked as a dependent data object, obtain the identification information of the data object that the user has configured for the opening step. Based on the marking options used by the user when marking the opening step, generate prompt information to prompt the user to operate on the data object that the opening step depends on. The marking options can reflect the dependency behavior of the opening step on the data object. Based on the execution order of the multiple operation steps and the identification and prompt information of the data objects on which the lid-opening step depends, reference structured data is generated.

2. The method according to claim 1, characterized in that, Obtain the identification information of the data objects that the user has configured for the lid-opening step, including: Display a list of data objects required for the kitchen robot to perform its tasks, the list of data objects including identification information for at least one data object; In response to the user's selection operation on the list of data objects, the identification information of the selected data object is obtained as the identification information of the data object on which the lid opening step depends.

3. The method according to claim 2, characterized in that, Also includes: Display added controls; In response to a click operation on the added control, a data object editing interface is displayed, the data object editing interface including: an editing control; In response to an editing operation on the editing control, the identification information of the data object on which the lid-opening step depends is obtained.

4. The method according to claim 1, characterized in that, Obtain the identification information of the data objects that the user has configured for the lid-opening step, including: A data object editing interface is displayed, which includes: editing controls; In response to an editing operation on the editing control, the identification information of the data object on which the lid-opening step depends is obtained.

5. A method for executing a job task, characterized in that, include: Obtain the reference structured data as described in any one of claims 1-4, wherein the reference structured data includes multiple operation steps in the kitchen robot's operation task, the execution order between the multiple operation steps, and the identification information and prompt information corresponding to the lid opening step among the multiple operation steps; The multiple job steps are executed sequentially according to their execution order. as well as When the lid-opening step is executed, the prompt and identification information are output to prompt the user to perform auxiliary operations adapted to the lid-opening step, so as to cooperate with the kitchen robot to complete the task.

6. A method for generating structured data, applicable to terminal devices, characterized in that, The method includes: Display the operation control interface, and control the kitchen robot to perform operation tasks based on the controls on the operation control interface; During the kitchen robot's task execution, the system receives and displays the current task step being performed by the kitchen robot. When the currently executed operation step is the lid opening step, in response to the user's marking operation that marks it as a dependent data object, a prompt message corresponding to the lid opening step is generated, and the identification information of the data object dependent on the lid opening step is obtained; Based on the work steps sequentially executed by the kitchen robot and the prompts and identification information corresponding to the data relied upon by the lid-opening step, reference structured data is generated; The process of generating prompt information corresponding to the opening step includes: generating prompt information to prompt the user to perform operations on the data objects on which the opening step depends, based on the marking options used by the user when marking the opening step, wherein the marking options can reflect the dependency behavior of the opening step on the data objects.

7. A kitchen robot, characterized in that, include: The device body includes: a memory and a processor; the memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement the steps of the method according to any one of claims 1-5.

8. A terminal device, characterized in that, include: Memory and processor; The memory is used to store computer programs; The processor, coupled to the memory, is used to execute the computer program for implementing the steps of the method of claim 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it causes the processor to perform the steps of the method according to any one of claims 1-6.

Images