Intelligent kitchen control method and device and electronic equipment

Abstract

The invention provides a control method and device of an intelligent kitchen and electronic equipment. The method comprises the steps that a target cooking menu is obtained, the cooking process of each dish in the target cooking menu is obtained, the target cooking menu comprises a plurality of dishes, the cooking process at least comprises the dish making process and the setting requirement of the cooking electric appliance corresponding to the dishes, and the setting requirement at least comprises the time requirement; the cooking processes of the dishes with the same making process and / or the same setting requirements of the cooking electric appliances are combined, a total cooking process of the target cooking menu is generated, and the total cooking process comprises a plurality of cooking steps; the current user behavior is recognized, a plurality of cooking electric appliances in the intelligent kitchen are controlled to operate according to the current user behavior and the total cooking process, and the current user behavior represents the cooking action currently performed by the user. Through the application, the problem that in the prior art, cooperative work among multiple pieces of intelligent kitchen equipment cannot be realized is solved.

Description

Technical Field

[0001] The present application relates to the field of smart home technology, and more specifically, to a control method, device, computer-readable storage medium, and electronic device for a smart kitchen. Background Art

[0002] With the accelerated pace of life, cooking is both a pleasure and a challenge for modern people. The traditional cooking process requires manual processing of ingredients, adjustment of cooking equipment parameters, and constant attention to cooking progress, which undoubtedly increases the complexity and time cost of cooking. In recent years, with the rapid development of the Internet of Things, artificial intelligence, and machine vision technology, smart home devices have gradually entered people's daily lives, bringing new possibilities to the cooking process. However, existing smart home systems can often only achieve intelligent control of a single device, and cannot achieve collaborative work between multiple devices, let alone intelligent process planning based on user cooking needs. Summary of the invention

[0003] The main purpose of the present application is to provide a control method, device, computer-readable storage medium and electronic device for a smart kitchen, so as to at least solve the problem in the prior art that it is impossible to achieve collaborative work among multiple smart kitchen devices.

[0004] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a control method for a smart kitchen is provided, comprising: obtaining a target cooking menu, and obtaining a cooking process for each dish in the target cooking menu, wherein the target cooking menu includes multiple dishes, and the cooking process at least includes a preparation process of the dishes and setting requirements of the cooking appliances corresponding to the dishes, and the setting requirements at least include time requirements; merging the cooking processes of the dishes with the same preparation process and / or the same setting requirements of the cooking appliances to generate a total cooking process for the target cooking menu, wherein the total cooking process includes multiple cooking steps; identifying current user behavior, and controlling the operation of multiple cooking appliances in the smart kitchen according to the current user behavior and the total cooking process, wherein the current user behavior represents the cooking action currently being performed by the user.

[0005] Optionally, identifying the current user behavior includes: collecting images of users in the smart kitchen at predetermined time intervals within a target time period to obtain multiple user images, wherein the predetermined time interval is less than the target time period; analyzing the user images through a neural network model to determine whether the current user behavior is a food processing behavior, wherein the neural network model is obtained by training multiple sets of data, and each set of the multiple sets of data includes historical user images and food processing behavior labels.

[0006] Optionally, the method also includes: determining the first position coordinates of the user based on multiple user images, and obtaining the second position coordinates and usage status of each cooking appliance, wherein the usage status at least includes an in-use state and an unused state; calculating the absolute value of the difference between the second position coordinates and the first position coordinates, and determining that the current user behavior is an appliance usage behavior when the absolute value of the difference is less than a preset threshold and the usage status is the in-use state.

[0007] Optionally, controlling the operation of multiple cooking appliances in the smart kitchen according to the current user behavior and the overall cooking process includes: obtaining user instructions and the device status of each cooking appliance, determining a current cooking step according to the current user behavior, the user instructions and the device status, wherein the user instructions are instructions input by the user for operating the cooking appliance, and the device status represents the cooking status of the cooking appliance; obtaining current cooking content of the current cooking step, and controlling multiple current cooking appliances to operate according to a current target status according to the current cooking content, wherein the current cooking content includes at least the current cooking appliance and its current target status.

[0008] Optionally, after controlling multiple current cooking appliances to operate according to the target state according to the current cooking content, the method further includes: obtaining the next cooking content of the next cooking step, outputting the next cooking content to the user interface and / or voice broadcasting, wherein the next cooking step is the next step of the current cooking step, and the next cooking content includes at least the target cooking appliance and its next target state; controlling the target cooking appliance to the next target state.

[0009] Optionally, the method further includes: in the current cooking step, obtaining a current operating state of the cooking appliance, wherein the current operating state indicates that the cooking appliance is in a normal state or an abnormal state; when the current operating state indicates that the cooking appliance is in an abnormal state, outputting an alarm signal, wherein the alarm signal is a signal indicating that the cooking appliance is in the abnormal state.

[0010] Optionally, generating the total cooking process of the target cooking menu further includes: merging the cooking processes of the dishes having different preparation processes and using different cooking appliances to generate the total cooking process of the target cooking menu.

[0011] According to another aspect of the present application, a control device for a smart kitchen is provided, comprising: a first acquisition unit, used to acquire a target cooking menu, and acquire the cooking process of each dish in the target cooking menu, wherein the target cooking menu includes multiple dishes, and the cooking process at least includes the preparation process of the dishes and the setting requirements of the cooking appliances corresponding to the dishes, and the setting requirements at least include time requirements; a generation unit, used to merge the cooking processes of the dishes with the same preparation process and / or the same setting requirements of the cooking appliances, to generate a total cooking process for the target cooking menu, wherein the total cooking process includes multiple cooking steps; a first control unit, used to identify current user behavior, and control the operation of multiple cooking appliances in the smart kitchen according to the current user behavior and the total cooking process, wherein the current user behavior represents the cooking action currently being performed by the user.

[0012] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the control methods for the smart kitchen.

[0013] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a control method for executing any one of the smart kitchens.

[0014] Apply the technical solution of the present application, obtain the target cooking menu, and obtain the cooking process of each dish in the target cooking menu, wherein the target cooking menu includes multiple dishes, the cooking process at least includes the production process of the dishes and the setting requirements of the cooking appliances corresponding to the dishes, and the setting requirements at least include the time requirements; merge the cooking processes of dishes with the same production process and / or the same setting requirements of the cooking appliances to generate the total cooking process of the target cooking menu, wherein the total cooking process includes multiple cooking steps; identify the current user behavior, and control the operation of multiple cooking appliances in the smart kitchen according to the current user behavior and the total cooking process, and the current user behavior represents the cooking action currently being performed by the user. Compared with the prior art, the smart home system can often only realize the intelligent control of a single device, cannot realize the collaborative work between multiple devices, and cannot perform intelligent process planning according to the user's cooking needs. In this application, by merging some cooking processes, that is, the same cooking process can be merged and carried out simultaneously, so as to control multiple cooking appliances to run simultaneously, thereby realizing the collaborative work of multiple kitchen devices. Therefore, it can solve the problem that the collaborative work between multiple smart kitchen devices cannot be realized in the prior art, and achieve the effect of collaborative control of the operation of multiple devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings constituting part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0016] Figure 1 A schematic diagram showing a flow chart of a control method for a smart kitchen provided in an embodiment of the present application is shown;

[0017] Figure 2 A schematic diagram showing a specific flow chart of a control method for a smart kitchen provided in an embodiment of the present application is shown;

[0018] Figure 3 A structural block diagram of a control device for a smart kitchen provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0019] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0020] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] As introduced in the background technology, the smart home system in the prior art can often only realize the intelligent control of a single device, and cannot realize the collaborative work among multiple devices, let alone perform intelligent process planning according to the user's cooking needs. In order to solve the problem that the collaborative work among multiple smart kitchen devices cannot be realized, the embodiments of the present application provide a control method, device, computer-readable storage medium and electronic device for a smart kitchen.

[0023] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0024] In this embodiment, a control method for a smart kitchen running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0025] Figure 1 FIG. 1 is a flow chart of a control method for a smart kitchen according to an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0026] Step S201, obtaining a target cooking menu and obtaining a cooking process of each dish in the target cooking menu, wherein the target cooking menu includes a plurality of dishes, the cooking process at least includes a preparation process of the dishes and setting requirements of a cooking appliance corresponding to the dishes, and the setting requirements at least include a time requirement;

[0027] Specifically, the user enters the names of multiple dishes they want to cook on the main control interface of the smart home system to obtain a target cooking menu. The smart home system exports the cooking process of the corresponding dishes on the target cooking menu through pre-stored recipes, as well as the setting requirements of the cooking appliances needed (such as time requirements), and allows the user to customize and add new cooking processes for dishes.

[0028] Step S202, merging the cooking processes of the dishes with the same preparation process and / or the same setting requirements of the cooking appliances to generate a total cooking process of the target cooking menu, wherein the total cooking process includes a plurality of cooking steps;

[0029] Specifically, in order to further help users optimize cooking efficiency, the cooking processes of different dishes can be combined into an overall process. For example, each dish requires washing and cutting vegetables, which means they are of the same type of production process. Therefore, the washing and cutting processes of multiple dishes can be combined and performed at the same time. Different dishes sometimes require the same cooking appliance, such as using the same oven temperature and the same time, which can also be combined. Alternatively, if the production process is the same and the setting requirements of the cooking appliance are also the same, they can be combined.

[0030] Step S203, identifying the current user behavior, and controlling the operation of the plurality of cooking appliances in the smart kitchen according to the current user behavior and the overall cooking process, wherein the current user behavior represents the cooking action currently being performed by the user.

[0031] Specifically, cameras and other devices based on smart kitchens can identify the current user behavior. Further, through user instructions and device status as well as the current user behavior identified by the camera, it can accurately determine which cooking step the user is in the overall cooking process, and pre-run the cooking appliances that need to be prepared in the next step, such as preheating the oven, and control the cooking appliances running in parallel in the current step to ensure the smooth progress of the user's entire cooking process. Cooking actions can include cutting vegetables, frying vegetables, washing vegetables, and using the oven.

[0032] Through this embodiment, a target cooking menu is obtained, and a cooking process of each dish in the target cooking menu is obtained, wherein the target cooking menu includes multiple dishes, and the cooking process at least includes the production process of the dishes and the setting requirements of the cooking appliances corresponding to the dishes, and the setting requirements at least include time requirements; the cooking processes of dishes with the same production process and / or the same setting requirements of the cooking appliances are merged to generate a total cooking process of the target cooking menu, wherein the total cooking process includes multiple cooking steps; the current user behavior is identified, and the operation of multiple cooking appliances in the smart kitchen is controlled according to the current user behavior and the total cooking process, and the current user behavior represents the cooking action currently being performed by the user. Compared with the prior art, the smart home system can often only realize the intelligent control of a single device, cannot realize the collaborative work between multiple devices, and cannot perform intelligent process planning according to the user's cooking needs. In this application, by merging some cooking processes, that is, the same cooking process can be merged and performed simultaneously, so as to control multiple cooking appliances to run simultaneously, thereby realizing the collaborative work of multiple kitchen devices. Therefore, it can solve the problem that the collaborative work between multiple smart kitchen devices cannot be realized in the prior art, and achieve the effect of collaborative control of the operation of multiple devices.

[0033] In the specific implementation process, the above step S203 identifies the current user behavior, which can be achieved through the following steps: Step S2031: Collect images of users in the smart kitchen at predetermined time intervals within the target time period to obtain multiple user images, wherein the predetermined time interval is less than the target time period; Step S2032: Analyze the user images through a neural network model to determine whether the current user behavior is a food processing behavior, wherein the neural network model is obtained by training multiple sets of data, and each set of data in the multiple sets of data includes historical user images and food processing behavior labels. The method identifies the current user behavior through the above steps, so that the operation of the cooking appliance can be controlled according to the current user behavior.

[0034] Specifically, the system uses the kitchen camera image data and the status of each kitchen appliance to determine the current user behavior, such as cutting vegetables, cooking, using the oven, using the refrigerator, etc. After the user starts the cooking process, a recurrent neural network is used to determine the user's actions, such as cutting vegetables, cooking, washing vegetables, etc., using the kitchen images captured by the camera over a continuous period of time.

[0035] In some optional embodiments, the method further includes the following steps: Step S204: determining the first position coordinates of the user based on the plurality of user images, and obtaining the second position coordinates and the use status of each cooking appliance, wherein the use status at least includes the use status and the unuse status; Step S205: calculating the absolute value of the difference between the second position coordinates and the first position coordinates, and determining that the current user behavior is the use of an appliance behavior when the absolute value of the difference is less than a preset threshold and the use status is the use status. The method determines whether the current user behavior is the use of an appliance behavior through the above steps, so that the current user behavior can be further judged.

[0036] In the specific implementation process, when the camera is turned on, the system first uses the deep learning network to identify the kitchen equipment in the field of view and divides the possible positions of the user when using the equipment, such as oven, gas stove, sink, rice cooker, etc. If the user is not in the action of cutting vegetables, cooking, washing vegetables, etc. that does not rely on kitchen equipment, the user is judged to be using the corresponding kitchen equipment based on the user's position and the status reported by the corresponding kitchen equipment. For example, if the user is next to the oven and detects that the oven door is open or the oven is being operated, the user behavior is identified as "using the oven".

[0037] In some optional implementations, the above step S203 controls the operation of multiple cooking appliances in the smart kitchen according to the current user behavior and the overall cooking process, which can be achieved by the following steps: Step S2031: Obtain user instructions and the device status of each cooking appliance, and determine the current cooking step according to the current user behavior, the user instructions and the device status, wherein the user instructions are instructions input by the user for operating the cooking appliance, and the device status represents the cooking status of the cooking appliance; Step S2032: Obtain the current cooking content of the current cooking step, and control multiple current cooking appliances to operate according to the current target state according to the current cooking content, wherein the current cooking content at least includes the current cooking appliance and its current target state. The method first determines the current cooking step through the above steps, and then controls the operation of the cooking appliance according to the current cooking step.

[0038] Specifically, through the user instructions and device status reported by the user and the user behavior recognized by the camera in the previous step, it is possible to accurately determine which step of the cooking stage the user is in, determine which cooking step in the overall cooking process the user is in, obtain the current cooking step, and control the automatically running equipment in parallel in the current cooking step to ensure the smooth progress of the user's entire cooking process.

[0039] In some optional embodiments, after controlling multiple current cooking appliances to operate in a target state according to the current cooking content, the method further comprises the following steps: Step S206: obtaining the next cooking content of the next cooking step, outputting the next cooking content to the user interface and / or performing voice broadcasting, wherein the next cooking step is the next step of the current cooking step, and the next cooking content at least includes a target cooking appliance and its next target state; Step S207: controlling the target cooking appliance to the next target state. The method controls the control step of the next cooking step to start in advance through the above steps, so that preparation work can be performed in advance.

[0040] Specifically, according to the overall cooking process, the next cooking content of the next cooking step is obtained, and the cooking appliances that need to be prepared in the next step are pre-operated, such as preheating the oven, etc. In each cooking step, the system provides prompts and help to the user through the APP interface and voice, mainly including the specific content of the next step, the time required, precautions, etc.

[0041] In the specific implementation process, the method further includes the following steps: Step S208: in the current cooking step, obtaining the current operating state of the cooking appliance, wherein the current operating state indicates that the cooking appliance is in a normal state or an abnormal state; Step S209: when the current operating state indicates that the cooking appliance is in an abnormal state, outputting an alarm signal, wherein the alarm signal is a signal indicating that the cooking appliance is in the abnormal state. Through the above steps, the method outputs an alarm signal to warn when the cooking appliance is abnormal, so as to prevent further damage to the cooking appliance.

[0042] Specifically, the system also monitors the operating status of various kitchen appliances for users, such as oven temperature, gas stove power, oil fume concentration and other information, and promptly alerts users in case of abnormalities to ensure the safety and smooth progress of the cooking process.

[0043] In some optional implementations, the above step S202 generates the total cooking process of the target cooking menu, which can also be achieved by the following steps: merging the cooking processes of the dishes with different production processes and different cooking appliances to generate the total cooking process of the target cooking menu. The method generates the total cooking process even when the production processes are different through the above steps, which can improve cooking efficiency.

[0044] Specifically, the merging of cooking processes is divided into the following situations: common steps in the cooking process of different dishes, such as cutting vegetables, blanching, etc.; different dishes are cooked using the same equipment under the same cooking parameters, and can be cooked together when there is enough space for the equipment; different dishes are cooked using different equipment and can be cooked at the same time. The first two situations correspond to the merging of the cooking processes of the dishes with the same production process and / or the same setting requirements of the cooking appliances mentioned above, and the third situation corresponds to the merging of the cooking processes of the dishes with different production processes and different cooking appliances. In other words, in order to improve cooking efficiency, the cooking processes in the above situations are merged.

[0045] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the control method of the smart kitchen of the present application will be described in detail below in combination with specific embodiments.

[0046] This embodiment relates to a specific control method of a smart kitchen. Figure 2 As shown, the following steps are included:

[0047] Step S1: Select a dish on the oven touch screen or APP interface;

[0048] Step S2: Check whether there is a corresponding dish in the dish database. If yes, automatically plan the cooking process (total cooking process). If no, manually input the dish-related data.

[0049] Step S3: automatically planning the cooking process;

[0050] Step S4: display and adjustment;

[0051] Step S5: User behavior identification (current user behavior);

[0052] Step S6: real-time cooking prompts and multi-device collaborative control;

[0053] Step S7: cooking execution and feedback.

[0054] The present application embodiment also provides a control device for a smart kitchen. It should be noted that the control device for the smart kitchen of the present application embodiment can be used to execute the control method for a smart kitchen provided by the present application embodiment. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions thereof will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and contemplated.

[0055] The following is an introduction to the control device of the smart kitchen provided in the embodiment of the present application.

[0056] Figure 3 Schematic diagram of a control device for a smart kitchen according to an embodiment of the present application. Figure 3 As shown, the device comprises:

[0057] A first acquisition unit 10 is used to acquire a target cooking menu and acquire a cooking process of each dish in the target cooking menu, wherein the target cooking menu includes a plurality of dishes, the cooking process at least includes a preparation process of the dishes and setting requirements of the cooking appliance corresponding to the dishes, and the setting requirements at least include a time requirement;

[0058] Specifically, the user enters the names of multiple dishes they want to cook on the main control interface of the smart home system to obtain a target cooking menu. The smart home system exports the cooking process of the corresponding dishes on the target cooking menu through pre-stored recipes, as well as the setting requirements of the cooking appliances needed (such as time requirements), and allows the user to customize and add new cooking processes for dishes.

[0059] A generating unit 20, configured to merge the cooking processes of the dishes having the same preparation process and / or the same setting requirements of the cooking appliances to generate a total cooking process of the target cooking menu, wherein the total cooking process includes a plurality of cooking steps;

[0060] Specifically, in order to further help users optimize cooking efficiency, the cooking processes of different dishes can be combined into an overall process. For example, each dish requires washing and cutting vegetables, which means they are of the same type of production process. Therefore, the washing and cutting processes of multiple dishes can be combined and performed at the same time. Different dishes sometimes require the same cooking appliance, such as using the same oven temperature and the same time, which can also be combined. Alternatively, if the production process is the same and the setting requirements of the cooking appliance are also the same, they can be combined.

[0061] The first control unit 30 is used to identify the current user behavior and control the operation of the multiple cooking appliances in the smart kitchen according to the current user behavior and the overall cooking process, wherein the current user behavior represents the cooking action currently being performed by the user.

[0062] Specifically, cameras and other devices based on smart kitchens can identify the current user behavior. Further, through user instructions and device status as well as the current user behavior identified by the camera, it can accurately determine which cooking step the user is in the overall cooking process, and pre-run the cooking appliances that need to be prepared in the next step, such as preheating the oven, and control the cooking appliances running in parallel in the current step to ensure the smooth progress of the user's entire cooking process. Cooking actions can include cutting vegetables, frying vegetables, washing vegetables, and using the oven.

[0063] Through this embodiment, a target cooking menu is obtained, and a cooking process of each dish in the target cooking menu is obtained, wherein the target cooking menu includes multiple dishes, and the cooking process at least includes the production process of the dishes and the setting requirements of the cooking appliances corresponding to the dishes, and the setting requirements at least include time requirements; the cooking processes of dishes with the same production process and / or the same setting requirements of the cooking appliances are merged to generate a total cooking process of the target cooking menu, wherein the total cooking process includes multiple cooking steps; the current user behavior is identified, and the operation of multiple cooking appliances in the smart kitchen is controlled according to the current user behavior and the total cooking process, and the current user behavior represents the cooking action currently being performed by the user. Compared with the prior art, the smart home system can often only realize the intelligent control of a single device, cannot realize the collaborative work between multiple devices, and cannot perform intelligent process planning according to the user's cooking needs. In this application, by merging some cooking processes, that is, the same cooking process can be merged and performed simultaneously, so as to control multiple cooking appliances to run simultaneously, thereby realizing the collaborative work of multiple kitchen devices. Therefore, it can solve the problem that the collaborative work between multiple smart kitchen devices cannot be realized in the prior art, and achieve the effect of collaborative control of the operation of multiple devices.

[0064] In the specific implementation process, the first control unit includes a collection module and a first determination module. The collection module is used to collect images of users in the smart kitchen at predetermined time intervals within a target time period to obtain multiple user images, wherein the predetermined time interval is less than the target time period; the first determination module is used to analyze the user images through a neural network model to determine whether the current user behavior is a food processing behavior, wherein the neural network model is obtained by training multiple sets of data, and each set of data in the multiple sets of data includes historical user images and food processing behavior labels. The method identifies the current user behavior through the above steps, so that the operation of the cooking appliance can be controlled according to the current user behavior.

[0065] Specifically, the system uses the kitchen camera image data and the status of each kitchen appliance to determine the current user behavior, such as cutting vegetables, cooking, using the oven, using the refrigerator, etc. After the user starts the cooking process, a recurrent neural network is used to determine the user's actions, such as cutting vegetables, cooking, washing vegetables, etc., using the kitchen images captured by the camera over a continuous period of time.

[0066] In some optional embodiments, the method further includes a second acquisition unit and a determination unit, the second acquisition unit is used to determine the first position coordinates of the user based on the multiple user images, and obtain the second position coordinates and the use status of each cooking appliance, wherein the use status at least includes the use status and the unuse status; the determination unit is used to calculate the absolute value of the difference between the second position coordinate and the first position coordinate, and when the absolute value of the difference is less than a preset threshold and the use status is the use status, determine that the current user behavior is the use of the appliance behavior. The method determines whether the current user behavior is the use of the appliance behavior through the above steps, so that the current user behavior can be further judged.

[0067] In the specific implementation process, when the camera is turned on, the system first uses the deep learning network to identify the kitchen equipment in the field of view and divides the possible positions of the user when using the equipment, such as oven, gas stove, sink, rice cooker, etc. If the user is not in the action of cutting vegetables, cooking, washing vegetables, etc. that does not rely on kitchen equipment, the user is judged to be using the corresponding kitchen equipment based on the user's position and the status reported by the corresponding kitchen equipment. For example, if the user is next to the oven and detects that the oven door is open or the oven is being operated, the user behavior is identified as "using the oven".

[0068] In some optional embodiments, the first control unit further includes a second determination module and a control module, the second determination module is used to obtain user instructions and the device status of each cooking appliance, and determine the current cooking step according to the current user behavior, the user instructions and the device status, wherein the user instructions are instructions input by the user to operate the cooking appliance, and the device status represents the cooking status of the cooking appliance; the control module is used to obtain the current cooking content of the current cooking step, and control multiple current cooking appliances to operate according to the current target state according to the current cooking content, wherein the current cooking content at least includes the current cooking appliance and its current target state. The method first determines the current cooking step through the above steps, and then controls the operation of the cooking appliance according to the current cooking step.

[0069] Specifically, through the user instructions and device status reported by the user and the user behavior recognized by the camera in the previous step, it is possible to accurately determine which step of the cooking stage the user is in, determine which cooking step in the overall cooking process the user is in, obtain the current cooking step, and control the automatically running equipment in parallel in the current cooking step to ensure the smooth progress of the user's entire cooking process.

[0070] In some optional embodiments, after controlling multiple current cooking appliances to operate in a target state according to the current cooking content, the method further includes a first output unit and a second control unit, the first output unit is used to obtain the next cooking content of the next cooking step, output the next cooking content to the user interface and / or perform voice broadcast, wherein the next cooking step is the next step of the current cooking step, and the next cooking content at least includes a target cooking appliance and its next target state; the second control unit is used to control the target cooking appliance to the next target state. The method controls the control step of the next cooking step to start in advance through the above steps, so that preparation work can be done in advance.

[0071] Specifically, according to the overall cooking process, the next cooking content of the next cooking step is obtained, and the cooking appliances that need to be prepared in the next step are pre-operated, such as preheating the oven, etc. In each cooking step, the system provides prompts and help to the user through the APP interface and voice, mainly including the specific content of the next step, the time required, precautions, etc.

[0072] In the specific implementation process, the method further includes a third acquisition unit and a second output unit: the third acquisition unit is used to acquire the current operating state of the cooking appliance in the current cooking step, wherein the current operating state indicates that the cooking appliance is in a normal state or an abnormal state; the second output unit is used to output an alarm signal when the current operating state indicates that the cooking appliance is in an abnormal state, wherein the alarm signal is a signal indicating that the cooking appliance is in the abnormal state. Through the above steps, the method outputs an alarm signal to warn when the cooking appliance is abnormal, so as to prevent further damage to the cooking appliance.

[0073] Specifically, the system also monitors the operating status of various kitchen appliances for users, such as oven temperature, gas stove power, oil fume concentration and other information, and promptly alerts users in case of abnormalities to ensure the safety and smooth progress of the cooking process.

[0074] In some optional embodiments, the generation unit further includes a generation module for merging the cooking processes of the dishes with different production processes and different cooking appliances to generate a total cooking process for the target cooking menu. The method generates a total cooking process even when the production processes are different through the above steps, which can improve cooking efficiency.

[0075] Specifically, the merging of cooking processes is divided into the following situations: common steps in the cooking process of different dishes, such as cutting vegetables, blanching, etc.; different dishes are cooked using the same equipment under the same cooking parameters, and can be cooked together when there is enough space for the equipment; different dishes are cooked using different equipment and can be cooked at the same time. The first two situations correspond to the merging of the cooking processes of the dishes with the same production process and / or the same setting requirements of the cooking appliances mentioned above, and the third situation corresponds to the merging of the cooking processes of the dishes with different production processes and different cooking appliances. In other words, in order to improve cooking efficiency, the cooking processes in the above situations are merged.

[0076] The control device of the smart kitchen includes a processor and a memory. The first acquisition unit, the generation unit and the first control unit are stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions. The modules are all located in the same processor; or, the modules are located in different processors in any combination.

[0077] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the problem that the coordination between multiple smart kitchen devices cannot be achieved in the prior art can be solved by adjusting the kernel parameters.

[0078] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0079] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the control method of the smart kitchen.

[0080] Specifically, the control method of the smart kitchen includes:

[0081] Step S201, obtaining a target cooking menu and obtaining a cooking process of each dish in the target cooking menu, wherein the target cooking menu includes a plurality of dishes, the cooking process at least includes a preparation process of the dishes and setting requirements of a cooking appliance corresponding to the dishes, and the setting requirements at least include a time requirement;

[0082] Specifically, the user enters the names of multiple dishes they want to cook on the main control interface of the smart home system to obtain a target cooking menu. The smart home system exports the cooking process of the corresponding dishes on the target cooking menu through pre-stored recipes, as well as the setting requirements of the cooking appliances needed (such as time requirements), and allows the user to customize and add new cooking processes for dishes.

[0083] Step S202, merging the cooking processes of the dishes with the same preparation process and / or the same setting requirements of the cooking appliances to generate a total cooking process of the target cooking menu, wherein the total cooking process includes a plurality of cooking steps;

[0084] Specifically, in order to further help users optimize cooking efficiency, the cooking processes of different dishes can be combined into an overall process. For example, each dish requires washing and cutting vegetables, which means they are of the same type of production process. Therefore, the washing and cutting processes of multiple dishes can be combined and performed at the same time. Different dishes sometimes require the same cooking appliance, such as using the same oven temperature and the same time, which can also be combined. Alternatively, if the production process is the same and the setting requirements of the cooking appliance are also the same, they can be combined.

[0085] Step S203, identifying the current user behavior, and controlling the operation of the plurality of cooking appliances in the smart kitchen according to the current user behavior and the overall cooking process, wherein the current user behavior represents the cooking action currently being performed by the user.

[0086] Specifically, cameras and other devices based on smart kitchens can identify the current user behavior. Further, through user instructions and device status as well as the current user behavior identified by the camera, it can accurately determine which cooking step the user is in the overall cooking process, and pre-run the cooking appliances that need to be prepared in the next step, such as preheating the oven, and control the cooking appliances running in parallel in the current step to ensure the smooth progress of the user's entire cooking process. Cooking actions can include cutting vegetables, frying vegetables, washing vegetables, and using the oven.

[0087] An embodiment of the present invention provides an electronic device, including a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, at least the following steps are implemented:

[0088] Step S201, obtaining a target cooking menu and obtaining a cooking process of each dish in the target cooking menu, wherein the target cooking menu includes a plurality of dishes, the cooking process at least includes a preparation process of the dishes and setting requirements of a cooking appliance corresponding to the dishes, and the setting requirements at least include a time requirement;

[0089] Specifically, the user enters the names of multiple dishes they want to cook on the main control interface of the smart home system to obtain a target cooking menu. The smart home system exports the cooking process of the corresponding dishes on the target cooking menu through pre-stored recipes, as well as the setting requirements of the cooking appliances needed (such as time requirements), and allows the user to customize and add new cooking processes for dishes.

[0090] Step S202, merging the cooking processes of the dishes with the same preparation process and / or the same setting requirements of the cooking appliances to generate a total cooking process of the target cooking menu, wherein the total cooking process includes a plurality of cooking steps;

[0091] Specifically, in order to further help users optimize cooking efficiency, the cooking processes of different dishes can be combined into an overall process. For example, each dish requires washing and cutting vegetables, which means they are of the same type of production process. Therefore, the washing and cutting processes of multiple dishes can be combined and performed at the same time. Different dishes sometimes require the same cooking appliance, such as using the same oven temperature and the same time, which can also be combined. Alternatively, if the production process is the same and the setting requirements of the cooking appliance are also the same, they can be combined.

[0092] Step S203, identifying the current user behavior, and controlling the operation of the plurality of cooking appliances in the smart kitchen according to the current user behavior and the overall cooking process, wherein the current user behavior represents the cooking action currently being performed by the user.

[0093] Specifically, cameras and other devices based on smart kitchens can identify the current user behavior. Further, through user instructions and device status as well as the current user behavior identified by the camera, it can accurately determine which cooking step the user is in the overall cooking process, and pre-run the cooking appliances that need to be prepared in the next step, such as preheating the oven, and control the cooking appliances running in parallel in the current step to ensure the smooth progress of the user's entire cooking process. Cooking actions can include cutting vegetables, frying vegetables, washing vegetables, and using the oven.

[0094] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0095] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the method described in each embodiment of the present application:

[0096] Step S201, obtaining a target cooking menu and obtaining a cooking process of each dish in the target cooking menu, wherein the target cooking menu includes a plurality of dishes, the cooking process at least includes a preparation process of the dishes and setting requirements of a cooking appliance corresponding to the dishes, and the setting requirements at least include a time requirement;

[0097] Specifically, the user enters the names of multiple dishes they want to cook on the main control interface of the smart home system to obtain a target cooking menu. The smart home system exports the cooking process of the corresponding dishes on the target cooking menu through pre-stored recipes, as well as the setting requirements of the cooking appliances needed (such as time requirements), and allows the user to customize and add new cooking processes for dishes.

[0098] Step S202, merging the cooking processes of the dishes with the same preparation process and / or the same setting requirements of the cooking appliances to generate a total cooking process of the target cooking menu, wherein the total cooking process includes a plurality of cooking steps;

[0099] Specifically, in order to further help users optimize cooking efficiency, the cooking processes of different dishes can be combined into an overall process. For example, each dish requires washing and cutting vegetables, which means they are of the same type of production process. Therefore, the washing and cutting processes of multiple dishes can be combined and performed at the same time. Different dishes sometimes require the same cooking appliance, such as using the same oven temperature and the same time, which can also be combined. Alternatively, if the production process is the same and the setting requirements of the cooking appliance are also the same, they can be combined.

[0100] Step S203, identifying the current user behavior, and controlling the operation of the plurality of cooking appliances in the smart kitchen according to the current user behavior and the overall cooking process, wherein the current user behavior represents the cooking action currently being performed by the user.

[0101] Specifically, cameras and other devices based on smart kitchens can identify the current user behavior. Further, through user instructions and device status as well as the current user behavior identified by the camera, it can accurately determine which cooking step the user is in the overall cooking process, and pre-run the cooking appliances that need to be prepared in the next step, such as preheating the oven, and control the cooking appliances running in parallel in the current step to ensure the smooth progress of the user's entire cooking process. Cooking actions can include cutting vegetables, frying vegetables, washing vegetables, and using the oven.

[0102] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0103] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0104] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0105] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0107] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0108] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0109] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0110] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0111] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0112] 1) In the control method of the smart kitchen of the present application, a target cooking menu is obtained, and a cooking process of each dish in the target cooking menu is obtained, wherein the target cooking menu includes multiple dishes, and the cooking process includes at least the production process of the dishes and the setting requirements of the cooking appliances corresponding to the dishes, and the setting requirements include at least the time requirements; the cooking processes of dishes with the same production process and / or the same setting requirements of the cooking appliances are merged to generate a total cooking process of the target cooking menu, wherein the total cooking process includes multiple cooking steps; the current user behavior is identified, and the operation of multiple cooking appliances in the smart kitchen is controlled according to the current user behavior and the total cooking process, and the current user behavior represents the cooking action currently being performed by the user. Compared with the prior art, the smart home system can often only realize the intelligent control of a single device, and cannot realize the collaborative work between multiple devices, and cannot perform intelligent process planning according to the user's cooking needs. In the present application, by merging some cooking processes, that is, the same cooking process can be merged and carried out simultaneously, so as to control multiple cooking appliances to run simultaneously, thereby realizing the collaborative work of multiple kitchen devices. Therefore, it can solve the problem that the collaborative work between multiple smart kitchen devices cannot be realized in the prior art, and achieve the effect of collaborative control of the operation of multiple devices.

[0113] 2) In the control device of the smart kitchen of the present application, a target cooking menu is obtained, and a cooking process of each dish in the target cooking menu is obtained, wherein the target cooking menu includes multiple dishes, and the cooking process includes at least the production process of the dishes and the setting requirements of the cooking appliances corresponding to the dishes, and the setting requirements include at least the time requirements; the cooking processes of the dishes with the same production process and / or the same setting requirements of the cooking appliances are merged to generate a total cooking process of the target cooking menu, wherein the total cooking process includes multiple cooking steps; the current user behavior is identified, and the operation of multiple cooking appliances in the smart kitchen is controlled according to the current user behavior and the total cooking process, and the current user behavior represents the cooking action currently being performed by the user. Compared with the prior art, the smart home system can often only realize the intelligent control of a single device, and cannot realize the collaborative work between multiple devices, and cannot perform intelligent process planning according to the user's cooking needs. In the present application, by merging some cooking processes, that is, the same cooking process can be merged and performed simultaneously, so as to control multiple cooking appliances to run simultaneously, thereby realizing the collaborative work of multiple kitchen devices. Therefore, it can solve the problem that the collaborative work between multiple smart kitchen devices in the prior art cannot be realized, and achieve the effect of collaborative control of the operation of multiple devices.

[0114] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A control method for a smart kitchen, characterized in that: include: Obtaining a target cooking menu and obtaining a cooking process for each dish in the target cooking menu, wherein the target cooking menu includes a plurality of the dishes, the cooking process at least includes a preparation process for the dishes and setting requirements for the cooking appliances corresponding to the dishes, and the setting requirements at least include time requirements; Merging the cooking processes of the dishes with the same preparation process and / or the same setting requirements of the cooking appliances to generate a total cooking process of the target cooking menu, wherein the total cooking process includes a plurality of cooking steps; Identify the current user behavior, and control the operation of the plurality of cooking appliances in the smart kitchen according to the current user behavior and the overall cooking process, wherein the current user behavior represents the cooking action currently being performed by the user.

2. The control method of the smart kitchen according to claim 1, characterized in that: Identify current user behavior, including: Collecting images of users in the smart kitchen at predetermined time intervals within a target time period to obtain a plurality of user images, wherein the predetermined time interval is shorter than the target time period; The user image is analyzed by a neural network model to determine whether the current user behavior is a food processing behavior, wherein the neural network model is obtained by training multiple sets of data, and each set of the multiple sets of data includes historical user images and food processing behavior labels.

3. The control method of the smart kitchen according to claim 2, characterized in that: The method further comprises: Determine the first position coordinates of the user according to the plurality of user images, and obtain the second position coordinates and the use status of each cooking appliance, wherein the use status at least includes an in-use state and an unused state; The absolute value of the difference between the second position coordinate and the first position coordinate is calculated, and when the absolute value of the difference is less than a preset threshold and the usage state is the in-use state, it is determined that the current user behavior is an appliance usage behavior.

4. The control method of the smart kitchen according to claim 1, characterized in that: Controlling the operation of the plurality of cooking appliances in the smart kitchen according to the current user behavior and the overall cooking process includes: Acquire a user instruction and a device state of each of the cooking appliances, and determine a current cooking step according to the current user behavior, the user instruction and the device state, wherein the user instruction is an instruction input by a user for operating the cooking appliance, and the device state indicates a cooking state of the cooking appliance; The current cooking content of the current cooking step is obtained, and a plurality of current cooking appliances are controlled to operate according to a current target state according to the current cooking content, wherein the current cooking content at least includes the current cooking appliances and their current target states.

5. The control method of the smart kitchen according to claim 4, characterized in that: After controlling the multiple current cooking appliances to operate according to the target state according to the current cooking content, the method further includes: Acquire the next cooking content of the next cooking step, and output the next cooking content to the user interface and / or perform voice broadcast, wherein the next cooking step is the next step of the current cooking step, and the next cooking content at least includes a target cooking appliance and its next target state; The target cooking appliance is controlled to be in the next target state.

6. The control method of the smart kitchen according to claim 1, characterized in that: The method further comprises: In the current cooking step, a current operating state of the cooking appliance is obtained, wherein the current operating state indicates whether the cooking appliance is in a normal state or an abnormal state; In the case where the current operating state indicates that the cooking appliance is in an abnormal state, an alarm signal is output, wherein the alarm signal is a signal indicating that the cooking appliance is in the abnormal state.

7. The control method of the smart kitchen according to claim 1, characterized in that: The overall cooking process for generating the target cooking menu also includes: The cooking processes of the dishes with different preparation processes and using different cooking appliances are combined to generate a total cooking process of the target cooking menu.

8. A control device for a smart kitchen, characterized in that: include: A first acquisition unit is used to acquire a target cooking menu and acquire a cooking process of each dish in the target cooking menu, wherein the target cooking menu includes a plurality of dishes, the cooking process at least includes a preparation process of the dishes and setting requirements of the cooking appliance corresponding to the dishes, and the setting requirements at least include a time requirement; a generating unit, configured to merge the cooking processes of the dishes having the same preparation process and / or the same setting requirements of the cooking appliances, to generate a total cooking process of the target cooking menu, wherein the total cooking process includes a plurality of cooking steps; The first control unit is used to identify the current user behavior and control the operation of multiple cooking appliances in the smart kitchen according to the current user behavior and the overall cooking process, wherein the current user behavior represents the cooking action currently being performed by the user.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the control method of the smart kitchen according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing the control method of the smart kitchen described in any one of claims 1 to 7.

Images