Cooking method, device, wearable device, storage medium and program product

Abstract

The application discloses a cooking method and device, a wearable device, a storage medium and a program product, and relates to the technical field of intelligent kitchen appliances. The method can be executed by the wearable device, and the method comprises the following steps: in response to a completion instruction that a target user wears the wearable device, determining a target utensil type of a target cooking device used by the target user; determining a target operation action of the target cooking device by the target user; determining a target audio-visual effect based on the target utensil type and the target operation action, and sending the target audio-visual effect to a target response device, so that the target response device displays the target audio-visual effect. According to the scheme, the operation experience of the user can be improved, a relaxed and happy cooking atmosphere can be created, and the cooking experience of the user can be improved.

Description

Technical Field

[0001] This invention relates to the field of smart kitchen appliances, and in particular to a cooking method, apparatus, wearable device, storage medium, and program product. Background Technology

[0002] With the rapid development of smart kitchen appliances and wearable devices, the intelligentization of the kitchen cooking process has become a hot topic in research and product development. Currently, some products on the market have attempted to assist cooking through cameras or sensors, such as food recognition and temperature control, but the integration support for user action recognition and feedback interaction is still relatively limited.

[0003] How to enhance the user experience and create a relaxed and enjoyable cooking atmosphere during the cooking process is a key research topic in the industry. Summary of the Invention

[0004] This invention provides a cooking method, apparatus, wearable device, storage medium, and program product to enhance the user's operating experience, create a relaxed and pleasant cooking atmosphere, and improve the user's cooking experience during the cooking process.

[0005] According to one aspect of the present invention, a cooking method is provided, performed by a wearable device, the method comprising:

[0006] In response to the completion command from the wearable device worn by the target user, determine the type of target utensil of the target cooking equipment used by the target user;

[0007] Determine the target user's target operational actions on the target cooking equipment;

[0008] The target audiovisual effect is determined based on the type of target device and the target operation action, and then sent to the target response device so that the target response device can display the target audiovisual effect.

[0009] Optionally, determining the target appliance type of the target cooking equipment used by the target user includes: identifying the RFID tag of the target cooking equipment through the RFID module of the wearable device, and determining the target appliance type of the target cooking equipment based on the identification result of the RFID tag.

[0010] Optionally, determining the target utensil type of the target cooking equipment used by the target user includes: collecting appearance image data of the target cooking equipment through the image recognition module of the wearable device, and performing image recognition on the appearance image data; determining the target utensil type of the target cooking equipment based on the image recognition results.

[0011] Optionally, determining the target user's target operation action on the target cooking device includes: collecting first motion data of the wearable device in real time through the inertial measurement unit of the wearable device; performing attitude estimation on the first motion data to obtain first motion state data of the wearable device; determining a first operation action that matches the first motion state data based on a pre-determined correspondence between the motion state data and the operation action, and identifying the first operation action as the target operation action.

[0012] Optionally, determining the target user's target operation action on the target cooking equipment includes: acquiring first video data of the target user operating the target cooking equipment in real time through the image recognition module of the wearable device; filtering out either the first frame image data or the second frame image data when the similarity between the first frame image data and the second frame image data of the first video data is greater than a set threshold to obtain target video data; and inputting the target video data into a pre-trained action recognition model to obtain the target operation action that matches the target video data.

[0013] Optionally, the action recognition model is trained through the following steps: acquiring a cooking video dataset, determining the operation action corresponding to each cooking video data in the cooking video dataset; inputting each cooking video data and the operation action corresponding to each cooking video data into the target machine learning model for iterative training, and obtaining the action recognition model when the iteration stops.

[0014] Optionally, the target audiovisual effect is determined based on the target appliance type and the target operation action, including: determining the target user's cooking purpose based on the target appliance type and the target operation action; selecting the target sound from preset sound files and the target light from preset light files based on the target user's cooking purpose; and generating the target audiovisual effect based on the target sound and the target light.

[0015] Optionally, the cooking purpose of the target user can be determined based on the target utensil type and the target operation action, including: comparing the target utensil type and the target operation action with a pre-built cooking purpose database, and determining the cooking purpose of the target user based on the comparison results; wherein, the cooking purpose database stores the correspondence between different utensil types and operation actions and cooking purposes.

[0016] According to another aspect of the present invention, a cooking apparatus is provided, which is deployed in a wearable device, the apparatus comprising:

[0017] The first determining module is used to determine the type of target utensil of the target cooking equipment used by the target user in response to the completion instruction of the target user wearing the wearable device;

[0018] The second determining module is used to determine the target user's target operation action on the target cooking equipment;

[0019] The third determining module is used to determine the target audiovisual effect based on the target device type and the target operation action, and send the target audiovisual effect to the target response device so that the target response device can display the target audiovisual effect.

[0020] According to another aspect of the present invention, a wearable device is provided, the wearable device comprising:

[0021] At least one processor; and

[0022] A memory that is communicatively connected to at least one processor; wherein,

[0023] The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to perform the cooking method of any embodiment of the present invention.

[0024] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute and implement a cooking method according to any embodiment of the present invention.

[0025] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements a cooking method according to any embodiment of the present invention.

[0026] The technical solution of this invention, in response to a completion command from a wearable device worn by a target user, determines the type of utensil used by the target user in the target cooking equipment; determines the target user's target operation on the target cooking equipment; it can quickly determine the type of utensil used by the target user and the operation performed by the target user on the target cooking equipment, providing a basis for subsequently determining the target audiovisual effect; based on the target utensil type and the target operation, the target audiovisual effect is determined and sent to the target response device so that the target response device can display the target audiovisual effect. This allows for the rapid determination of the target audiovisual effect based on the target operation and the target utensil type, enhancing the user's operating experience during cooking, creating a relaxed and pleasant cooking atmosphere, and improving the user's cooking experience.

[0027] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart of a cooking method provided according to Embodiment 1 of the present invention;

[0030] Figure 2 This is a flowchart of a cooking method provided according to Embodiment 2 of the present invention;

[0031] Figure 3 This is a flowchart of a cooking method provided according to Embodiment 3 of the present invention;

[0032] Figure 4 This is a control flowchart of a cooking wristband provided according to Embodiment 3 of the present invention;

[0033] Figure 5 This is a schematic diagram of the structure of a cooking device according to Embodiment 4 of the present invention;

[0034] Figure 6 This is a schematic diagram of the structure of a wearable device that implements the cooking method of the present invention.

[0035] Icons: 510-First determining module; 520-Second determining module; 530-Third determining module; 10-Wearable device; 11-Processor; 12-ROM; 13-RAM; 14-Bus; 15-I / O interface; 16-Input unit; 17-Output unit; 18-Storage unit; 19-Communication unit. Detailed Implementation

[0036] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0037] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0038] Example 1

[0039] Figure 1 This is a flowchart of a cooking method according to Embodiment 1 of the present invention. This embodiment is applicable to situations where the user's operating experience is enhanced and a relaxed and pleasant cooking atmosphere is created during the cooking process. The method can be executed by a cooking device, which can be implemented in hardware and / or software. The cooking device can be configured in wearable devices such as smartwatches, smart bracelets, smart rings, or smart glasses. Figure 1 As shown, the method includes:

[0040] Step 110: In response to the completion instruction of the target user wearing the wearable device, determine the target utensil type of the target cooking device used by the target user.

[0041] The target user can be an adult or a minor who is preparing ingredients or cooking in the kitchen; this embodiment does not limit the target user.

[0042] In this embodiment, the wearable device can be a smartwatch, smart bracelet, smart ring, or smart glasses, etc., and is not limited thereto.

[0043] In one optional implementation of this embodiment, the wearable device itself can determine whether the user has finished wearing the wearable device. For example, the wearable device can detect the ambient temperature, and when it detects that the contact temperature is close to the human body temperature, it can determine that the target user has finished wearing the wearable device; alternatively, the user can actively issue a command to complete wearing the wearable device, which is not limited in this embodiment.

[0044] Optionally, in this embodiment, after determining that the target user has finished wearing the wearable device, the type of cooking equipment used by the target user can be further determined, which is referred to as the target equipment type in this embodiment. For example, the type of cooking equipment can be: cutting tools, such as a kitchen knife, peeler, or scissors; stirring and carrying tools, such as a spatula, ladle, or slotted spoon; measuring tools, such as a measuring cup or measuring spoon; auxiliary tools, such as a cutting board, egg beater, or rolling pin; or other smart kitchen appliances, such as a steamer, oven, microwave oven, or integrated stove, which are not limited in this embodiment.

[0045] In one optional implementation of this embodiment, the appearance image or model information of the target cooking device can be collected by a wearable device, and the target utensil type of the target cooking device can be determined based on the appearance image or model information of the target cooking device.

[0046] Step 120: Determine the target user's target operation actions on the target cooking equipment.

[0047] Optionally, in this embodiment, after the wearable device determines the type of the target cooking device, it can further determine the target user's operation on the target cooking device, which is referred to as the target operation action in this embodiment. For example, the target operation action can be cutting vegetables, chopping bones, stir-frying vegetables, or rolling noodles, etc., and is not limited to it in this embodiment.

[0048] In one optional implementation of this embodiment, the target operation action can be determined by acquiring the motion data of the target cooking device; alternatively, the motion data of the wrist can be determined by acquiring the motion data of the part wearing the wearable device (e.g., the wrist), and further, the target operation action can be determined based on the motion data of the wrist. This embodiment does not limit this.

[0049] Step 130: Determine the target audiovisual effect based on the target device type and target operation action, and send the target audiovisual effect to the target response device so that the target response device can display the target audiovisual effect.

[0050] The audiovisual effects may include visual effects, such as changes in lighting, or auditory effects, such as different music or sound effects. This embodiment does not limit the specific effects.

[0051] The target response device can be a range hood, integrated stove, Bluetooth speaker, or water heater equipped with a light-changing module and a speaker, and this embodiment does not limit it.

[0052] Optionally, in this embodiment, after determining the type of target appliance used by the target user and the target operation action of the target user on the target cooking equipment, the target audiovisual effects (target lighting effects and target sound effects) can be further determined based on the type of target appliance and the target operation action. Furthermore, the target audiovisual effects can be transmitted to the target response device via Bluetooth or wireless communication, so that the target response device can display the target audiovisual effects. For example, the target lighting effects and target sound effects can be displayed simultaneously through a Bluetooth speaker, or the target sound effects can be played through a Bluetooth speaker, and the target lighting effects can be displayed through the integrated stove.

[0053] For example, when a user quickly cuts vegetables with a kitchen knife, it can trigger background music with a strong rhythm and rhythmic lighting; when stir-frying with a spatula, it can play sound effects and light up the range hood light strip, showing a marquee effect; when tossing the pan with a wok, it can play electronically synthesized music that sounds like the food touching the pan, along with lighting effects that match the rhythm of tossing the pan.

[0054] The advantage of this setup is that it not only enhances the user's cooking enjoyment but can also be used for auxiliary functions such as prompting the operating rhythm and optimizing attention allocation.

[0055] The solution in this embodiment determines the type of cooking appliance used by the target user in response to a completion command from a wearable device; it also determines the target user's operation on the cooking appliance. This allows for the rapid determination of the type of cooking appliance and the user's operation, providing a basis for subsequently determining the target audiovisual effects. Based on the type of appliance and the operation, the target audiovisual effects are determined and sent to the target response device for display. This rapid determination of the target audiovisual effects enhances the user experience during cooking, creating a relaxed and enjoyable atmosphere and improving the overall cooking experience.

[0056] Example 2

[0057] Figure 2 This is a flowchart of a cooking method according to Embodiment 2 of the present invention. This embodiment is a further refinement of the above technical solution, and the technical solution in this embodiment can be combined with various optional solutions in one or more of the above embodiments. Figure 2 As shown, the method includes:

[0058] Step 210: Identify the RFID tag of the target cooking equipment through the RFID module of the wearable device, and determine the target utensil type of the target cooking equipment based on the identification result of the RFID tag.

[0059] Optionally, in this embodiment, the wearable device can be a smart bracelet, smartwatch, or smart ring, etc., worn on the user's wrist; at the same time, the smart bracelet, smartwatch, or smart ring, etc., are internally deployed with a radio frequency identification (RFID) module; it should be noted that in this embodiment, each cooking device has an RFID tag affixed to its surface, and each cooking device's RFID tag is unique.

[0060] In an optional implementation of this embodiment, after determining that the target user is using the target cooking device, for example, after detecting that the target user is holding the target cooking device in their hand, the RFID tag of the target cooking device can be obtained through the RFID module of the smart bracelet, and the obtained RFID tag can be identified. Based on the identification result of the RFID tag, the type of appliance of the target cooking device can be determined.

[0061] For example, when a target user picks up a kitchen knife, the RFID module of the smart bracelet senses the RFID tag on the knife, reads its contents, and parses the read RFID data to determine that the target user is currently using a kitchen knife as a cooking utensil.

[0062] Step 220: Collect first motion data of the wearable device in real time through the inertial measurement unit of the wearable device; perform attitude estimation on the first motion data to obtain first motion state data of the wearable device; based on the pre-determined correspondence between motion state data and operation action, determine the first operation action that matches the first motion state data, and determine the first operation action as the target operation action.

[0063] Optionally, in this embodiment, an inertial measurement unit is also deployed inside the smart bracelet, which can be used to detect the motion data of the smart bracelet; it is understood that when the target user wears the smart bracelet and makes hand movements, the motion data detected by the inertial measurement unit is the motion data of the target user's hand.

[0064] Optionally, in this embodiment, after determining that the target user is using the target cooking device, the motion data (e.g., acceleration or angular velocity) of the smart bracelet can be collected in real time through the inertial measurement unit of the smart bracelet. Furthermore, outlier processing, missing value processing, or normalization processing can be performed on each motion data to obtain the first motion data. Further, attitude estimation can be performed on the first motion data. For example, the angular velocity in the first motion data can be calculated to determine the orientation of the target cooking device in three-dimensional space (e.g., roll angle, pitch angle, or yaw angle). The determined orientation of the target cooking device in three-dimensional space can be the motion state data of the target cooking device, that is, the motion state data of the smart bracelet, which is referred to as the first motion state data in this embodiment.

[0065] Furthermore, based on the pre-determined correspondence between motion state data and operation actions, a first operation action matching the first motion state data can be determined, and the first operation action can be identified as the target operation action.

[0066] It is understood that in this embodiment, a database of the correspondence between different motion state data and operation actions is pre-established. The motion state data can correspond one-to-one with the operation action, or multiple motion state data can correspond to the same operation action. For example, the first motion state data and the second motion state data correspond to the operation action of cutting vegetables; the third motion state data corresponds to the operation action of rolling noodles.

[0067] In an optional implementation of this embodiment, after determining the first motion state data of the smart bracelet, the first motion state data can be further compared with each motion state data in the pre-determined correspondence between motion state data and operation actions. If the comparison is consistent with the reference motion state data, the first operation action corresponding to the reference motion state data can be determined as the target operation action.

[0068] Step 230: Determine the target user's cooking purpose based on the target appliance type and target operation action; based on the target user's cooking purpose, select the target sound from the preset sound files and the target light from the preset light files; generate the target audiovisual effect based on the target sound and target light.

[0069] Optionally, after determining the target appliance type and target operation based on the above steps, the target user's cooking purpose can be further determined based on the target operation and target appliance type (e.g., preparing ingredients, stir-frying, stewing, or steaming / baking, etc., which are not limited in this embodiment); further, the target sound can be selected from preset sound documents and the target light can be selected from preset light documents based on the cooking purpose; further, the target audiovisual effect can be generated based on the target sound and target light, that is, the target sound and target light are combined to obtain the target audiovisual effect.

[0070] Optionally, in this embodiment, determining the target user's cooking purpose based on the target appliance type and the target operation action may include: comparing the target appliance type and the target operation action with a pre-built cooking purpose database, and determining the target user's cooking purpose based on the comparison results.

[0071] The cooking purpose database stores different utensil types and the correspondence between operation actions and cooking purposes; for example, a piece of data stored in the cooking purpose database can be: utensil type A, operation action B - cooking purpose C.

[0072] In an optional implementation of this embodiment, after determining the target appliance type and the target operation action, the target appliance type and the target operation action can be compared with each appliance type and operation action in the cooking purpose database; when the target appliance type matches appliance type A and the target operation action matches operation action B, then the target user's cooking purpose is cooking purpose C.

[0073] The advantage of this setup is that it allows for quick and accurate identification of the target user's intended use, which helps in determining the target audiovisual effects.

[0074] In another optional implementation of this embodiment, after determining the target utensil type and the target operation action, the target utensil type and the target operation action can be further input into a pre-trained cooking purpose determination model to output the cooking purpose corresponding to the target utensil type and the target operation action.

[0075] Among them, the cooking purpose determination model can be a machine learning model obtained by training a deep learning model with various utensil types, operation actions, and cooking purposes.

[0076] In one example of this embodiment, after the user wears this smart cooking bracelet, he holds a kitchen knife with an embedded RFID tag in his right hand. The bracelet automatically reads the RFID information and identifies the current utensil as a "kitchen knife". Subsequently, the inertial measurement unit collects the user's continuous up-and-down cutting motion and determines it as "quickly cutting vegetables" through the built-in motion recognition algorithm. Furthermore, the system matches the corresponding audiovisual feedback template: plays rhythmic sound effects and controls the stove light to light up in a flashing manner.

[0077] In another example of this embodiment, the user swings the spatula in his hand to stir-fry. The wristband identifies the current utensil type as "spatula" by recognizing the spatula's RFID tag, and the inertial measurement unit determines the cooking action as "medium-speed stir-frying". Furthermore, the system matches a feedback template with soft sound effects and slowly changing stove lights.

[0078] In another example of this embodiment, the user performs a tossing operation with the handheld wok. The wristband identifies the current utensil type as "wok" by recognizing the RFID tag on the wok, and the inertial measurement unit identifies the cooking action as "tossing". Furthermore, the system matches electronically synthesized music that plays when the food comes into contact with the wok with the lighting effects that match the tossing rhythm.

[0079] Step 240: Send the target audiovisual effects to the target response device so that the target response device can display the target audiovisual effects.

[0080] In this embodiment, when the wearable device is a smart device worn on the user's hand, the target cooking appliance type can be identified through the RFID module of the wearable device; motion state data can be determined through the inertial measurement unit of the wearable device, and the target operation action can be determined based on the motion state data. This allows for the rapid and accurate determination of the target appliance type and the target operation action; and the target user's cooking purpose can be accurately determined based on the target appliance type and the target operation action. The target audiovisual effects can also be accurately determined based on the cooking purpose, creating a relaxed and pleasant cooking atmosphere and enhancing the user's cooking experience.

[0081] Example 3

[0082] Figure 3 This is a flowchart of a cooking method according to Embodiment 3 of the present invention. This embodiment is a further refinement of the above technical solution, and the technical solution in this embodiment can be combined with various optional solutions in one or more of the above embodiments. Figure 3 As shown, the method includes:

[0083] Step 310: Collect appearance image data of the target cooking equipment through the image recognition module of the wearable device, and perform image recognition on the appearance image data; determine the target utensil type of the target cooking equipment based on the image recognition results.

[0084] Optionally, in this embodiment, the wearable device can be a smart device worn on the user's head, such as smart glasses or a smart helmet; and it is equipped with an image recognition module.

[0085] In an optional implementation of this embodiment, after determining that the target user is using the target cooking device, for example, after detecting that the target user is holding the target cooking device, the appearance image data of the target cooking device can be collected by the image recognition module of the smart glasses, and the appearance image can be recognized. Then, the target utensil type of the target cooking device can be determined based on the image recognition result.

[0086] It is understandable that the image recognition module of smart glasses can integrate an image recognition model. After acquiring the appearance image data of the target cooking equipment, the appearance image can be processed based on the image recognition model to determine the type of the target utensil.

[0087] Step 320: Collect first video data of the target user operating the target cooking equipment in real time through the image recognition module of the wearable device; if the similarity between the first frame image data and the second frame image data of the first video data is greater than a set threshold, filter out the first frame image data or the second frame image data to obtain the target video data; input the target video data into the pre-trained action recognition model to obtain the target operation action that matches the target video data.

[0088] The first frame image data and the second frame image data can be any two adjacent frames of image data in the first video data.

[0089] Optionally, in this embodiment, after determining that the target user is using the target cooking equipment, video data of the target user operating the target cooking equipment can be collected in real time through the image recognition module of the smart glasses. In this embodiment, this is referred to as the first video data. Further, each video frame of the first video data can be detected. When the similarity between the first frame image data and the second frame image data of the first video data is greater than a set threshold (e.g., 0.99 or 0.98), the first frame image data or the second frame image data can be filtered out to obtain the target video data. This allows for the filtering out of identical image frames, reducing algorithm processing time without affecting the recognition accuracy of the operation.

[0090] Furthermore, the target video data can be input into a pre-trained action recognition model, and the action recognition model can process the target video data to obtain the target operation action that matches the target video data.

[0091] Optionally, in this embodiment, the action recognition model can be trained by the following steps: acquiring a cooking video dataset, determining the operation action corresponding to each cooking video data in the cooking video dataset; inputting each cooking video data and the operation action corresponding to each cooking video data into the target machine learning model for iterative training, and obtaining the action recognition model when the iteration stops.

[0092] The cooking video dataset can include video data of different users preparing ingredients (e.g., washing or chopping vegetables) or cooking processes (e.g., stir-frying, pan-frying recipes, or stewing soup) at different times.

[0093] The target machine learning model can be a deep learning model, an image recognition model, or a multimodal large language model, etc., and is not limited to it in this embodiment.

[0094] In this embodiment, after obtaining the cooking video dataset, the operation action corresponding to each video data in the cooking video dataset can be further determined. For example, image recognition can be performed on each image frame data in each video data, and then the image recognition results can be stitched together to obtain the operation action corresponding to each video data.

[0095] Furthermore, each cooking video data, along with the corresponding operation actions, can be input into the target machine learning model for iterative training. Under the condition of stopping the iteration (e.g., the output accuracy is greater than a set accuracy threshold, or the number of iterations reaches a set number), the action recognition model can be output.

[0096] Step 330: Determine the target user's cooking purpose based on the target appliance type and target operation action; based on the target user's cooking purpose, select the target sound from the preset sound files and the target light from the preset light files; generate the target audiovisual effect based on the target sound and target light.

[0097] Step 340: Send the target audiovisual effects to the target response device so that the target response device can display the target audiovisual effects.

[0098] In this embodiment, when the wearable device is a smart device worn on the user's head, the image recognition module of the wearable device can collect an image of the appearance of the target cooking equipment, and the type of the target utensil can be determined based on the recognition result of the appearance image. At the same time, the image recognition module can collect cooking video data, thereby determining the target user's target operation action. The target utensil type and target operation action can be determined based on different wearable devices, which helps to determine the target audiovisual effects and create a relaxed and pleasant cooking atmosphere.

[0099] To better understand the cooking method involved in this embodiment, a specific example is used below for illustration. Figure 4 This is a control flowchart of a cooking wristband according to Embodiment 3 of the present invention, including:

[0100] Step 410: The user puts on the cooking bracelet;

[0101] Step 420: The user picks up the cooking utensils;

[0102] Step 430: The cooking wristband identifies the type of cooking appliance by reading the RFID tag of the cooking appliance;

[0103] Step 440: Match the corresponding audiovisual database according to the identified device type;

[0104] Step 450: Cook using cooking utensils;

[0105] Step 460: The terminal smart device responds with corresponding sound effects or lights based on the type of cooking appliance movement.

[0106] In this embodiment, when a user uses a cooking utensil (e.g., a kitchen knife), the RFID module built into the cooking bracelet reads the RFID tag information of the cooking utensil, thereby completing the identification of the utensil type; the identification result will be used to match the corresponding motion analysis algorithm and audiovisual effect template.

[0107] Furthermore, the IMU (Inertial Measurement Unit) sensor inside the cooking bracelet continuously collects hand motion data when the user uses cooking utensils; through preset motion feature models (such as chopping vegetables, stir-frying, tossing the pan, etc.), the cooking bracelet can identify specific cooking actions.

[0108] Furthermore, by combining the identified appliance type and action pattern, the system can call the corresponding audiovisual feedback template and send the signal to the audiovisual response device (range hood, stove or other kitchen equipment) via wireless connection.

[0109] The solution of this invention incorporates an IMU sensor and an RFID reading module into the cooking wristband, and sets an RFID tag on the handle of the cooking utensil to identify the type of utensil used during cooking and collect motion characteristics during cooking. This triggers corresponding audiovisual feedback effects, which not only enhances the user's cooking enjoyment but also prompts the operating rhythm and optimizes attention allocation.

[0110] Example 4

[0111] Figure 5This is a schematic diagram of a cooking device according to Embodiment 4 of the present invention, which can be deployed in a wearable device. Figure 5 As shown, the device includes: a first determining module 510, a second determining module 520, and a third determining module 530.

[0112] The first determining module 510 is used to determine the type of target utensil of the target cooking equipment used by the target user in response to the completion instruction of the target user wearing the wearable device;

[0113] The second determining module 520 is used to determine the target user's target operation action on the target cooking equipment;

[0114] The third determining module 530 is used to determine the target audiovisual effect based on the target device type and the target operation action, and send the target audiovisual effect to the target response device so that the target response device can display the target audiovisual effect.

[0115] In this embodiment, the solution involves a first determining module responding to a completion command from a wearable device worn by a target user to determine the type of utensil used by the target user in the target cooking device; a second determining module determining the target user's target operation on the target cooking device; and a third determining module determining the target audiovisual effect and sending the target audiovisual effect to a target response device so that the target response device can display the target audiovisual effect. This can enhance the user's operating experience during the cooking process, create a relaxed and pleasant cooking atmosphere, and improve the user's cooking experience.

[0116] In an optional implementation of this embodiment, the first determining module 510 is specifically used for:

[0117] The target cooking equipment's RFID tag is identified by the RFID module of a wearable device, and the target cooking equipment's utensil type is determined based on the RFID tag identification result.

[0118] In an optional implementation of this embodiment, the first determining module 510 is further specifically used for:

[0119] The image recognition module of a wearable device collects the appearance image data of the target cooking equipment and performs image recognition on the appearance image data;

[0120] The target cooking equipment type is determined based on the image recognition results.

[0121] In an optional implementation of this embodiment, the second determining module 520 is specifically used for:

[0122] The inertial measurement unit of the wearable device collects the first motion data of the wearable device in real time.

[0123] The first motion data is used to estimate the attitude, thus obtaining the first motion state data of the wearable device;

[0124] Based on the pre-determined correspondence between motion state data and operation actions, a first operation action matching the first motion state data is determined, and the first operation action is identified as the target operation action.

[0125] In an optional implementation of this embodiment, the second determining module 520 is further specifically used for:

[0126] The first video data of the target user operating the target cooking equipment is collected in real time through the image recognition module of the wearable device;

[0127] If the similarity between the first frame image data and the second frame image data of the first video data is greater than a set threshold, the first frame image data or the second frame image data is filtered out to obtain the target video data.

[0128] The target video data is input into a pre-trained action recognition model to obtain the target operation action that matches the target video data.

[0129] In an optional implementation of this embodiment, the action recognition model is trained through the following steps:

[0130] Obtain a cooking video dataset and determine the corresponding action for each cooking video data in the dataset.

[0131] Each cooking video data, along with the corresponding operation actions, is input into the target machine learning model for iterative training. When the iteration stops, the action recognition model is obtained.

[0132] In an optional implementation of this embodiment, the third determining module 530 is specifically used for:

[0133] Determine the target user's cooking purpose based on the type of target appliance and the target user's operating actions;

[0134] The target sound is selected from a set of preset sound files based on the target user's cooking purpose, and the target light is selected from a set of light files.

[0135] Generate target audiovisual effects based on target sound and target light.

[0136] In an optional implementation of this embodiment, the third determining module 530 includes a cooking purpose determining unit, used for:

[0137] The target appliance type and target operation action are compared with a pre-built database of cooking uses, and the cooking purpose of the target user is determined based on the comparison results.

[0138] The cooking purpose database stores different types of utensils and the correspondence between operating actions and cooking purposes.

[0139] The cooking apparatus provided in the embodiments of the present invention can perform the cooking method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for performing the method.

[0140] The collection, storage, use, processing, transmission, provision, and disclosure of user cooking information involved in the technical solutions of this invention comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0141] Example 5

[0142] Figure 6 A schematic diagram of the structure of a wearable device 10, such as a helmet, glasses, watch, etc., which can be used to implement embodiments of the present invention is shown. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0143] like Figure 6 As shown, the wearable device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer programs stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the wearable device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0144] Multiple components in the wearable device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, optical disk, etc.; and a communication unit 19, such as a network card, modem, wireless transceiver, etc. The communication unit 19 allows the wearable device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0145] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 executes the various methods and processes described above, such as a cooking method, which can be executed by a wearable device and may include: in response to a completion instruction from a target user wearing the wearable device, determining the target utensil type of the target cooking device used by the target user; determining the target operation action of the target user on the target cooking device; determining the target audiovisual effect based on the target utensil type and the target operation action; and sending the target audiovisual effect to a target response device so that the target response device displays the target audiovisual effect.

[0146] In some embodiments, the cooking method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the wearable device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the cooking method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the cooking method by any other suitable means (e.g., by means of firmware).

[0147] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0148] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0149] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0150] To provide interaction with a user, the systems and techniques described herein can be implemented on a wearable device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the wearable device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or haptic feedback); and input from the user can be received in any form (including sound input, voice input, or haptic input).

[0151] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0152] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0153] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0154] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

[0155] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements a database detection method as provided in any embodiment of this application.

[0156] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0157] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, they do not mean that the applicant has used or necessarily used the solution.

[0158] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A cooking method performed by a wearable device, characterized in that, include: In response to a completion command from a target user wearing the wearable device, the target utensil type of the target cooking device used by the target user is determined; Determine the target user's target operation on the target cooking equipment; Based on the target device type and the target operation action, the target audiovisual effect is determined, and the target audiovisual effect is sent to the target response device so that the target response device can display the target audiovisual effect; Determining the target audiovisual effect based on the target appliance type and the target operation action includes: determining the target user's cooking purpose based on the target appliance type and the target operation action; selecting the target sound from preset sound files and the target light from preset light files based on the target user's cooking purpose; and generating the target audiovisual effect based on the target sound and the target light.

2. The cooking method according to claim 1, characterized in that, Determining the type of target appliance used by the target user in the target cooking equipment includes: The wearable device identifies the RFID tag of the target cooking device using its RFID module, and determines the target utensil type based on the identification result of the RFID tag.

3. The cooking method according to claim 1, characterized in that, Determining the type of target appliance used by the target user in the target cooking equipment includes: The wearable device's image recognition module acquires the appearance image data of the target cooking device and performs image recognition on the appearance image data; The target utensil type of the target cooking equipment is determined based on the image recognition results.

4. The cooking method according to claim 2, characterized in that, Determining the target user's target operation on the target cooking device includes: The first motion data of the wearable device is collected in real time by the inertial measurement unit of the wearable device; The first motion data is used to estimate the attitude to obtain the first motion state data of the wearable device; Based on the pre-determined correspondence between motion state data and operation actions, a first operation action matching the first motion state data is determined, and the first operation action is identified as the target operation action.

5. The cooking method according to claim 3, characterized in that, Determining the target user's target operation on the target cooking device includes: The image recognition module of the wearable device collects first video data in real time of the target user operating the target cooking equipment; If the similarity between the first frame image data and the second frame image data of the first video data is determined to be greater than a set threshold, the first frame image data or the second frame image data is filtered out to obtain the target video data. The target video data is input into a pre-trained action recognition model to obtain the target operation action that matches the target video data.

6. The cooking method according to claim 5, characterized in that, The action recognition model is trained through the following steps: Acquire a cooking video dataset and determine the operation action corresponding to each cooking video data in the cooking video dataset; Each of the cooking video data and the corresponding operation actions are input into the target machine learning model for iterative training. When the iteration stops, the action recognition model is obtained.

7. The cooking method according to claim 6, characterized in that, Determining the target user's cooking purpose based on the target appliance type and the target operation includes: The target appliance type and the target operation action are compared with a pre-built database of cooking uses, and the cooking use of the target user is determined based on the comparison results. The cooking purpose database stores different types of utensils and the correspondence between operating actions and cooking purposes.

8. A cooking device deployed in a wearable device, characterized in that, include: The first determining module is used to determine the type of target utensil of the target cooking device used by the target user in response to the completion instruction of the target user wearing the wearable device; The second determining module is used to determine the target user's target operation action on the target cooking device; The third determining module is used to determine the target audiovisual effect based on the target device type and the target operation action, and send the target audiovisual effect to the target response device so that the target response device can display the target audiovisual effect; The third determining module is specifically used to determine the cooking purpose of the target user based on the target appliance type and the target operation action; and to select the target sound from preset sound files and the target light from preset light files based on the target user's cooking purpose. The target audiovisual effect is generated based on the target sound and the target light.

9. A wearable device, characterized in that, The wearable device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the cooking method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the cooking method of any one of claims 1-7.

11. A computer program product comprising a computer program that, when executed by a processor, implements the cooking method according to any one of claims 1-7.

Images