Cooking control method and device, cooking equipment and storage medium
By installing a movable camera module in the steam oven, the system can automatically identify ingredients on multi-layer trays and match them with recipes, solving the problem that existing steam ovens cannot recognize multi-layer foods. This enables intelligent cooking and personalized sharing functions, enhancing the user experience.
Patent Information
- Application Number
- CN202511057208.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-30
- Publication Date
- 2025-10-28
AI Technical Summary
Existing steam ovens cannot flexibly obtain information about the food on multiple trays, resulting in poor cooking results, making it difficult to meet users' personalized sharing needs, and affecting the user experience.
A camera module is installed inside the transparent layer in the middle of the steam oven door. It can move vertically and horizontally to capture images of the ingredients on each tray rack. The module can also automatically match the target recipe by recognizing the ingredients through the cloud. It supports automatic cooking after user confirmation and can dynamically acquire high-definition images or videos during the cooking process.
It simplifies the intelligent cooking process, improves food recognition and cooking accuracy, meets users' personalized recording needs, and enhances ease of operation and user experience.
Smart Images
Figure CN120848236A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent control technology, specifically to a cooking control method, device, cooking equipment, and storage medium. Background Technology
[0002] With the rapid development of technology, people have increasingly higher demands for quality of life. When using cooking equipment, such as steam ovens, users hope for automatic cooking after food recognition, and the ability to record videos or photos during the cooking process for sharing on various short video platforms or social media. However, when users place different foods on multiple trays in a steam oven simultaneously, the fixed installation of cameras in existing steam ovens prevents them from recognizing the food on each tray, thus hindering accurate matching of corresponding set menus. Furthermore, it's impossible to record videos or photos of the food on the next tray during cooking, and even more so, it's difficult to personalize the sharing of food placed on any particular tray, severely impacting the user experience. Summary of the Invention
[0003] In view of this, the present invention provides a cooking control method, device, cooking equipment and storage medium to solve the problem that existing cooking equipment cannot flexibly access food placed on any layer of tray rack inside its cavity, resulting in poor food cooking effect, difficulty in meeting users' personalized food sharing needs, and serious impact on user experience.
[0004] In a first aspect, the present invention provides a cooking control method applied to a cooking device, the cooking device including a cabinet door and a housing, the cabinet door being multiple transparent layers stacked sequentially, and the housing having an inner cavity with multiple layers of tray racks, each tray rack being used to place food ingredients; a camera module and a moving area are provided within the middle transparent layer of the cabinet door; wherein the camera module can move vertically and horizontally along the moving area, for capturing corresponding images of the food ingredients on each tray rack, the method comprising:
[0005] In response to a user placing food on at least one tray rack, the camera module is controlled to move vertically along the moving area to acquire images of the corresponding food placed on each tray rack.
[0006] The target recipe is determined based on the images of each ingredient and pushed to the user so that the user can confirm whether to use the target recipe for cooking.
[0007] In response to the user's confirmation of cooking using the target recipe, the cooking equipment is controlled to cook based on the target recipe;
[0008] If a shooting command is detected during the cooking process, the shooting command is parsed to obtain control parameters, and the camera module is controlled to acquire corresponding shooting data according to the control parameters; the shooting command is generated by the user.
[0009] The cooking control method provided by this invention allows the camera module to move vertically along a moving area to identify the corresponding ingredients after the user places them. It then automatically matches the target recipe based on the identified ingredients, effectively avoiding the tedious process of manually searching for recipes. Furthermore, after the user confirms the use of the target recipe, the cooking device automatically calls the target recipe to perform the corresponding cooking, simplifying the cooking process and achieving intelligent cooking. Additionally, after the user issues a shooting command, the camera can be dynamically controlled to move to a designated position to obtain high-definition images or videos, greatly satisfying the user's personalized recording needs. This method offers significant advantages such as convenient operation, precise control, high cooking accuracy, and excellent results.
[0010] In one optional implementation, the moving area includes a vertical moving area with multiple shooting points; wherein the number of shooting points is determined by the number of layers of the tray rack, and the position of the shooting points is determined according to the corresponding position of each layer of the tray rack; controlling the camera module to move vertically along the moving area to acquire images of the food items placed on each layer of the tray rack, including:
[0011] The camera module is controlled to move from a preset initial position to the current shooting point, and to take pictures of the food placed on the tray rack corresponding to the current shooting point to obtain the corresponding food image;
[0012] The camera module is controlled to move from the current shooting point to the next shooting point to repeatedly shoot until it reaches the last shooting point and obtains the corresponding food image. Then, the camera module is controlled to return to the preset initial position.
[0013] This invention standardizes the vertical movement path of the camera module, specifically by moving the camera module in an orderly manner to avoid repetitive operations, thereby shortening image acquisition time and improving shooting efficiency. Furthermore, by setting a one-to-one correspondence between the shooting point and the corresponding tray layer, the invention effectively eliminates shooting position deviations, thereby improving image recognition accuracy and achieving high efficiency, accuracy, and streamlined control in food image acquisition.
[0014] In one optional implementation, determining the target recipe based on the images of each ingredient includes:
[0015] When the cooking equipment and camera module are connected to the wireless network, the camera module is controlled to send images of each ingredient to the cloud via the wireless network. The cloud then recognizes the images of each ingredient, generates a target recipe, and transmits the target recipe to the cooking equipment via the camera module.
[0016] This invention improves the accuracy and efficiency of food identification by uploading food images to the cloud for processing, thus fully utilizing the computing power and data advantages of the cloud and greatly ensuring the accuracy of the target recipe recommendation.
[0017] In one optional implementation, the control parameters include the subject being photographed and its corresponding shooting parameters; controlling the camera module to acquire corresponding shooting data according to the control parameters includes:
[0018] Control the camera module to move to the location of the subject being filmed;
[0019] The camera module is configured based on the shooting parameters, and the configured camera module is used to shoot the subject to obtain the corresponding shooting data.
[0020] This invention designs a dynamic shooting process in which the camera module moves directly to the target food location according to control parameters, which can achieve targeted positioning of the shooting object, thereby avoiding invalid shooting area coverage. At the same time, it can automatically adapt the shooting parameters for different foods, greatly satisfying the different shooting needs of users and thus improving the user experience.
[0021] In one optional embodiment, the cooking device further includes a display screen for displaying a human-machine interface; the process of determining the shooting command includes:
[0022] In response to the target food and corresponding shooting method determined by the user through the human-computer interaction interface, a shooting command is generated accordingly.
[0023] Alternatively, when cooking equipment is connected to a mobile terminal, a shooting command can be generated in response to the target ingredients and corresponding shooting methods determined by the user through the mobile terminal.
[0024] This invention designs two methods to generate shooting commands: through the human-computer interaction interface on the display screen of the cooking equipment and through interconnection with a mobile terminal. This can meet the user's operating habits in different scenarios, thereby improving the convenience and flexibility of operation.
[0025] In one optional implementation, after controlling the camera module to acquire corresponding shooting data according to control parameters, the cooking control method further includes:
[0026] The captured data is shared with users through a human-computer interaction interface;
[0027] Alternatively, when cooking equipment is connected to a mobile terminal, the captured data can be transmitted to the mobile terminal so that the user can access the captured data through the mobile terminal.
[0028] This invention designs a multi-terminal data sharing method that connects a human-computer interaction interface on a display screen with a mobile terminal, which can enhance user participation and convenience, thereby meeting users' personalized recording needs.
[0029] Secondly, the present invention provides a cooking control device applied to a cooking apparatus, the cooking apparatus including a cabinet door and a housing, the cabinet door comprising multiple transparent layers stacked sequentially, and the housing having an inner cavity with multiple layers of tray racks, each layer of tray racks for placing food ingredients; a camera module and a moving area are provided within the middle transparent layer of the cabinet door; wherein the camera module can move vertically and horizontally along the moving area, for capturing corresponding images of the food ingredients on each layer of tray racks, the device comprising:
[0030] The acquisition module is used to respond to the user's operation of placing food on at least one layer of tray racks, and control the camera module to move vertically along the moving area to acquire images of the corresponding food placed on each layer of tray racks respectively;
[0031] The determination module is used to determine the target recipe based on the images of each ingredient and push the target recipe to the user so that the user can confirm whether to use the target recipe for cooking.
[0032] The cooking module is used to respond to the user's confirmation operation of cooking with the target recipe, and to control the cooking equipment to cook based on the target recipe;
[0033] The shooting module is used to parse the shooting command to obtain control parameters if a shooting command is detected during the cooking process, and then control the camera module to acquire corresponding shooting data according to the control parameters; the shooting command is generated by the user.
[0034] The cooking control device provided by this invention identifies placed ingredients by controlling a camera module to move vertically along a moving area, and automatically matches a target recipe based on the identified ingredients, effectively avoiding the tedious steps of manually searching for recipes. Furthermore, once the user confirms the use of the target recipe for cooking, the device can automatically call up the target recipe for corresponding cooking, simplifying the cooking process, achieving intelligent cooking, and streamlining the operation process, thereby improving cooking control efficiency. Simultaneously, after the user issues a shooting command, the device can dynamically control the camera to move to a designated position to obtain high-definition images or videos, effectively meeting the user's personalized recording needs. It has significant advantages such as convenient operation, precise control, high cooking accuracy, and good results.
[0035] Thirdly, the present invention provides a cooking device, which includes a display screen, a cabinet door, and a housing. The display screen is used to display a human-computer interaction interface. The cabinet door consists of multiple transparent layers stacked sequentially. The inner cavity of the housing is provided with multiple tray racks, each of which is used to place food ingredients. A camera module and a moving area are provided in the middle transparent layer of the cabinet door. The camera module can move vertically and horizontally along the moving area to capture corresponding images of the food ingredients on each tray rack. The cooking device also includes a controller, which includes a memory and a processor. The memory and the processor are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the cooking control method of the first aspect or any corresponding embodiment described above.
[0036] In one alternative implementation, the cooking device is a steam oven.
[0037] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the cooking control method of the first aspect or any corresponding embodiment thereof. Attached Figure Description
[0038] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0039] Figure 1 This is a structural block diagram of the cooking device according to an embodiment of the present invention;
[0040] Figure 2 This is a schematic diagram of the controller structure of the cooking device according to an embodiment of the present invention;
[0041] Figure 3(A) is a schematic diagram of the steam oven structure;
[0042] Figure 3(B) is a schematic diagram of another steam oven structure;
[0043] Figure 4 This is a schematic diagram of the cabinet door structure of a steam oven;
[0044] Figure 5 This is a diagram illustrating the camera's field of view;
[0045] Figure 6 This is a schematic diagram of the cabinet door structure of another steam oven;
[0046] Figure 7(A) is a schematic diagram of the location identification setup;
[0047] Figure 7(B) is another schematic diagram of the location identification setup;
[0048] Figure 8 This is a schematic flowchart of the cooking control method according to an embodiment of the present invention;
[0049] Figure 9 This is a schematic flowchart of another cooking control method according to an embodiment of the present invention;
[0050] Figure 10 This is a flowchart illustrating the cooking control method;
[0051] Figure 11 This is a structural block diagram of the cooking control device according to an embodiment of the present invention. Detailed Implementation
[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0053] This embodiment provides a cooking device. Figure 1 This is a structural block diagram of the cooking device according to an embodiment of the present invention, such as... Figure 1 As shown, the cooking device includes a display screen 110, a cabinet door 120, and a housing 130. The display screen 110 is used to display a human-computer interaction interface. The cabinet door 120 consists of multiple transparent layers stacked sequentially. The inner cavity of the housing 130 is provided with multiple tray racks 131, each of which is used to place food ingredients. The middle transparent layer of the cabinet door 120 is provided with a camera module 121 and a moving area 122. The camera module 121 can move vertically and horizontally along the moving area 122 to capture corresponding images of the food ingredients on each tray rack 131.
[0054] In this embodiment, the specific type of cooking equipment is not limited and can be adapted according to actual needs. For example, the cooking equipment may be a steam oven, oven, steam oven, etc. This is only an example.
[0055] It should be noted that in this embodiment, the human-machine interface of the display screen 110 presents operation options in a graphical and menu-driven manner. The specific installation location of the display screen and the operation options of the human-machine interface can be determined according to the actual design. For example, the display screen is typically embedded in the front of the cooking equipment (such as the outside of the cabinet door or the front of the cabinet body), using a touch-screen LCD display with a size designed according to the equipment specifications (such as 10-15 inches), facilitating intuitive user operation. The visual human-machine interface supports user input commands (such as selecting recipes, setting cooking parameters, controlling the movement of the camera module, etc.); real-time feedback on equipment status (such as current temperature, remaining time, and ingredient image previews), and displays the steps of the target recipe generated in the cloud; it also serves as an output terminal for captured data, allowing users to view the ingredient images captured by the camera module on the screen or share them to mobile terminals.
[0056] In this embodiment, the cabinet door 120 is composed of multiple layers of transparent materials (such as tempered glass or acrylic) stacked sequentially. This avoids the risk of damage and bumps to the camera module 121, which is located on the outside of the cabinet door 120, thus providing both protection and visibility. Its specific structure includes: an outer layer (i.e., wear-resistant and scratch-resistant glass, protecting the internal structure and providing a smooth operating surface), a middle transparent layer (i.e., the core functional layer, housing the camera module 121 and the moving area 122, enabling unobstructed shooting), and an inner layer (i.e., heat-insulating glass, isolating the high-temperature environment inside the cabinet and ensuring the normal operation of the camera module 121). The camera module 121 is embedded inside the middle transparent layer and connected to the moving mechanism via a bracket, allowing it to move along the trajectory of the moving area 122. The moving area 122 serves as the motion track for the camera module 121. Its structural design employs a hidden guide rail or sliding rail system, embedded inside the middle transparent layer, so as not to occupy the cabinet space. The vertical guide rail is usually arranged along the central axis of the cabinet door, while the horizontal guide rail is designed according to the width of the tray frame (e.g., extending 30cm to the left and right). It can achieve precise positioning by using a built-in micro motor (e.g., a stepper motor) and setting corresponding control parameters (e.g., moving speed, positioning accuracy).
[0057] It should be noted that the specific model of the camera module 121 is adaptively adjusted according to the actual project requirements to achieve movement at any angle in the vertical, horizontal, and lateral directions. For example, the camera module 121 supports high-definition pixels (such as 1080P or 4K), is equipped with a wide-angle lens to cover the multi-layer tray rack 131, has a built-in fill light module (such as an LED ring light) to ensure the clarity of shooting when the light inside the cabinet 130 is insufficient, and has automatic focus and white balance functions to adapt to the color and distance changes of different ingredients. This is only an example for illustration purposes.
[0058] In this embodiment, the cabinet 130 serves as the core space for food cooking and storage. Its internal cavity structure is constructed using heat-insulating materials (such as polyurethane foam), with an internal temperature uniformity of ≤±5℃ to ensure consistent cooking results. Furthermore, the internal cavity of the cabinet 130 can be layered vertically (e.g., 3-7 layers) to accommodate multiple tray racks 131. The spacing between layers is adjustable (e.g., 5-10cm) to accommodate food containers of different heights (e.g., baking pans, grills, steamers, etc.). It is important to note that the specific number of tray racks 131 in this embodiment directly determines the number of vertical shooting points in the camera module 121 (e.g., 5 tray racks correspond to 5 vertical shooting positions); the horizontal position of each tray rack 131 corresponds to the horizontal movement range of the camera module 121, ensuring that food can be captured at any tray rack position.
[0059] In this embodiment, the cooking device also includes a controller; please refer to [link / reference]. Figure 2 , Figure 2 This is a schematic diagram of the controller structure of the cooking device described above in an optional embodiment of the present invention, as shown below. Figure 2 As shown, the controller includes one or more processors 210, memory 220, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise as required. The processors can process instructions executed within the controller, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple controllers can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 2 Take a processor 210 as an example.
[0060] Processor 210 may be a central processing unit, a network processor, or a combination thereof. Processor 210 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.
[0061] The memory 220 stores instructions executable by at least one processor 210 to cause the at least one processor 210 to perform the method shown in the above embodiments.
[0062] The memory 220 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the controller. Furthermore, the memory 220 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 220 may optionally include memory remotely located relative to the processor 210, and these remote memories may be connected to the controller via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0063] The memory 220 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 220 may also include a combination of the above types of memory.
[0064] The controller also includes an input device 230 and an output device 240. The processor 210, memory 220, input device 230, and output device 240 can be connected via a bus or other means. Figure 2 Taking the example of a connection between China and Israel via a bus.
[0065] Input device 230 can receive input digital or character information, and generate signal inputs related to user settings and function control of the thermal power unit's operation control unit, such as a touch screen, keypad, mouse, trackpad, touchpad, indicator, one or more mouse buttons, trackball, joystick, etc. Output device 240 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touch screen.
[0066] In this embodiment, the cooking device is a steam oven. Specifically, integrating the cooking control method described below into the cooking device enables it to achieve highly stable and reliable cooking results. This not only simplifies the cumbersome process of manual control and greatly improves control efficiency, but also meets the user's personalized recording needs, thereby enhancing the user experience.
[0067] In one specific embodiment, the cooking device is a steam oven. Figure 3(A) is a schematic diagram of the steam oven structure. As shown in Figure 3(A), the steam oven includes a cabinet door 310 and a housing 320. Figure 3(B) is a schematic diagram of another steam oven structure, showing the interior of the steam oven cavity after opening the cabinet door 310. Specifically, it is provided with 1, 2, and 3 tray racks for placing multiple baking trays or racks to hold food, i.e., the housing 320 in Figure 3(B) contains a total of three tray racks 321 with 1, 2, and 3 layers.
[0068] In this embodiment, Figure 4 This is a schematic diagram of the cabinet door structure of a steam oven. Figure 4 It can be seen that the cabinet door includes a camera moving module (i.e., camera module 121, which is in the set initial position when not working) and a movable area of the camera moving module (i.e., movable area 122). The camera moving module can move to any position within this area. The vertical direction is the main direction of movement of the camera moving module. A, B, and C correspond to the corresponding food positions in the 1, 2, and 3 multi-layer tray racks inside the cavity that the camera moving module can identify.
[0069] It should be noted that the camera movement module in this embodiment can rotate at a 60° angle along the horizontal direction, which facilitates the camera to adjust to a more comprehensive angle to select the imaging angle and take pictures. The specific shooting range of the camera can be found in [reference needed]. Figure 5 A diagram illustrating the camera's shooting range.
[0070] In this embodiment, A, B, and C are the three camera recognition positions, and their specific positions are located between the current layer and the previous layer (note that multi-layer trays are named sequentially from top to bottom according to the layer number, such as the topmost being the first layer, and each tray is placed on the tray rack of each layer for placing food), that is, 2 / 3 of the distance from the current layer and 1 / 3 of the distance from the previous layer. See also... Figure 6 Figures 7(A) and 7(B) illustrate this. For example, position A is located between the first and second baking trays, at a distance of 1 / 3 from the first tray and 2 / 3 from the second tray. This design allows the camera to capture all the food on each tray more comprehensively, improving recognition accuracy. It should be explained that when position A is set at 1 / 3 from the first tray, the area the camera can capture is larger than when it is set in the middle or at 2 / 3 from the first tray, resulting in a smaller blind spot. Assuming the distance between recognition positions A, B, and C and the previous tray is d (i.e., the distance between the recognition position and the previous baking tray), and the distance between the current baking tray and the previous baking tray is D (i.e., the distance between the tray rack layers), then the relationship between d and D satisfies: 0 < d / D ≤ 1 / 3.
[0071] According to embodiments of the present invention, based on the cooking apparatus mentioned in the above embodiments, a corresponding embodiment of a cooking control method is also provided, applied to the cooking apparatus. It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in a different order than that shown here.
[0072] This embodiment provides a cooking control method. Figure 8 This is a schematic flowchart of a cooking control method according to an embodiment of the present invention, such as... Figure 8 As shown, the process includes the following steps:
[0073] In step S801, in response to the user placing food on at least one layer of tray rack, the camera module is controlled to move vertically along the moving area to acquire images of the corresponding food placed on each layer of tray rack.
[0074] It should be noted that the specific detection method for the food placement operation in this embodiment is not limited. For example, high-precision pressure sensors (such as strain gauge load cells) can be embedded in the support structure of each tray rack (such as the bottom of the support or the four corners of the tray rack) to obtain weight data, and the determination can be based on the changes in weight data; or a camera module (such as a 4K high-definition camera) on the cabinet door can be used with a wide-angle lens (such as a field of view ≥120°) to cover all tray rack areas, and the determination can be made through real-time image detection. This is only an example.
[0075] Step S802: Determine the target recipe based on the images of each ingredient and push the target recipe to the user so that the user can confirm whether to use the target recipe for cooking.
[0076] In this embodiment, the target recipe can be determined by using an ingredient recognition model (such as YOLO or EfficientDet object detection model) to identify ingredients and generate a corresponding recipe based on the specific ingredients.
[0077] It should be noted that the specific method of pushing the target recipe to the user in this embodiment so that the user can confirm whether to use the target recipe for cooking is not limited here. For example, the target recipe can be sent to a mobile terminal that communicates with the cooking device, such as a mobile APP, so that the user can make a corresponding confirmation operation; or it can be displayed on the display screen and the user can make a corresponding confirmation through the human-computer interaction interface. This is only an example.
[0078] In step S803, in response to the user's confirmation operation of cooking using the target recipe, the cooking equipment is controlled to cook based on the target recipe.
[0079] In this embodiment, the specific content of the target recipe is adaptively adjusted according to actual needs; for example, the target recipe includes a specific dish name (such as beef stew with potatoes) and corresponding cooking parameters (such as heating temperature, cooking time, humidity control, etc.), which is only used as an example.
[0080] Step S804: If a shooting command is detected during the cooking process, the shooting command is parsed to obtain control parameters, and the camera module is controlled to acquire corresponding shooting data according to the control parameters; wherein, the shooting command is generated by user operation.
[0081] It should be noted that the specific content and detection method of the shooting command in this embodiment are not limited. For example, the shooting command may include shooting mode, resolution, focal length, angle, time interval, etc. The shooting command can be generated by the user through touch operation on the human-computer interaction interface of the cooking device (i.e., a "shoot" button or shortcut icon is set on the human-computer interaction interface of the display screen, and the user clicks it to directly trigger the shooting command; or it can be generated through gesture swiping operation, such as having "up / down" swiping buttons on the screen to select the target layer for shooting, i.e., the corresponding tray rack layer, and the user clicks to confirm after swiping to generate the command; or it can be generated through voice interaction, such as issuing a voice command ("shoot the current ingredients", "shoot the second tray", etc.) to trigger shooting, and the voice recognition module parses the command to generate the corresponding parameters); this is only an example.
[0082] The cooking control method of this invention, after the user places ingredients, controls the camera module to move vertically along the moving area to identify the corresponding ingredients, and automatically matches the target recipe based on the identified ingredients, effectively avoiding the tedious steps of manually searching for recipes. Simultaneously, after the user confirms the use of the target recipe for cooking, the cooking device automatically calls the target recipe to perform the corresponding cooking, simplifying the cooking process and achieving intelligent cooking. Furthermore, after the user issues a shooting command, the camera can be dynamically controlled to move to a designated position to obtain high-definition images or videos, greatly satisfying the user's personalized recording needs. It has significant advantages such as convenient operation, precise control, high cooking accuracy, and good results.
[0083] This embodiment provides a cooking control method. Figure 9 This is a schematic flowchart of another cooking control method according to an embodiment of the present invention, such as... Figure 9 As shown, the process includes the following steps:
[0084] In step S901, in response to the user placing food on at least one layer of tray rack, the camera module is controlled to move vertically along the moving area to acquire images of the corresponding food placed on each layer of tray rack.
[0085] In this embodiment, the moving area includes a vertical moving area, on which multiple shooting points are provided; wherein, the number of shooting points is determined by the number of layers of the tray frame, and the position of the shooting points is determined according to the corresponding position of each layer of the tray frame.
[0086] It should be noted that the shooting point in this embodiment is the camera recognition location, and its details can be found in the previous text, so they will not be repeated here.
[0087] Specifically, step S901 includes:
[0088] Step S9011, in response to the user placing food on at least one tray rack.
[0089] In this embodiment, the specific details of the food placement operation can be found in the preceding text and will not be repeated here.
[0090] Step S9012: Control the camera module to move from the preset initial position to the current shooting point, and take pictures of the food placed on the tray rack corresponding to the current shooting point to obtain the corresponding food image.
[0091] In this embodiment, the specific content of the preset initial position is determined adaptively according to actual needs. For example, the preset initial position is located at the top of the first baking tray, that is, the top position of the inner cavity of the steam oven. The advantage of this setting is that when the steam oven is idle, the camera module is located at the top layer, which can avoid obstructing the visible area of the cabinet door to the greatest extent, thus making it easier for users to check whether there is any food left behind or forgotten to be taken out inside the steam oven. In addition, the camera module being at the top position also makes it easier for the steam oven to start shooting and recognizing the food placed on each tray more quickly when it is turned on again.
[0092] Step S9013: Control the camera module to move from the current shooting point to the next shooting point to repeat shooting until the camera module reaches the last shooting point and obtains the corresponding food image, then control the camera module to return to the preset initial position.
[0093] In this embodiment of the invention, the standardized design of the vertical movement path of the camera module avoids repetitive operations through the orderly movement of the camera module, thereby shortening the image acquisition time and improving shooting efficiency. Furthermore, the one-to-one correspondence between the shooting point and the corresponding tray layer is set, which can effectively eliminate shooting position deviation and improve image recognition accuracy, thereby achieving high efficiency, accuracy and process control of food image acquisition.
[0094] Step S902: Determine the target recipe based on the images of each ingredient, and push the target recipe to the user so that the user can confirm whether to use the target recipe for cooking.
[0095] Specifically, step S902 includes:
[0096] Step S9021: When the cooking equipment and the camera module are connected to the wireless network, control the camera module to send the images of each ingredient to the cloud via the wireless network, so that the cloud can recognize the images of each ingredient, generate the target recipe, and transmit the target recipe to the cooking equipment via the camera module.
[0097] In this embodiment, the food images are uploaded to the cloud for processing, so as to make full use of the cloud's computing power and data advantages to improve the accuracy and efficiency of food recognition, and greatly ensure the accuracy of the target recipe recommendation.
[0098] Step S9022: Push the target recipe to the user so that the user can confirm whether to use the target recipe for cooking.
[0099] In this embodiment, the relevant content of step S9022 is as described above and will not be repeated here.
[0100] In this embodiment of the invention, food images are uploaded to the cloud for processing, so as to make full use of the computing power and data advantages of the cloud to improve the accuracy and efficiency of food recognition, and greatly ensure the accuracy of the target recipe recommendation.
[0101] Step S903: In response to the user's confirmation operation of cooking using the target recipe, the cooking equipment is controlled to perform cooking based on the target recipe. For details, please refer to [link to details]. Figure 8 Step S803 of the illustrated embodiment will not be repeated here.
[0102] Step S904: If a shooting command is detected during the cooking process, the shooting command is parsed to obtain control parameters, and the camera module is controlled to acquire corresponding shooting data according to the control parameters; wherein, the shooting command is generated by user operation.
[0103] Specifically, step S904 includes:
[0104] In step S9041, if a shooting command is detected during the cooking process, the shooting command is parsed to obtain control parameters.
[0105] In this embodiment, the control parameters include the subject being photographed and its corresponding shooting parameters, the details of which are described above and will not be repeated here.
[0106] It should be noted that, since the cooking equipment also includes a display screen for displaying the human-computer interaction interface, the process of determining the shooting command in step S9041 of this embodiment includes:
[0107] Step a1: In response to the target food and its corresponding shooting method determined by the user through the human-computer interaction interface, a shooting instruction is generated accordingly.
[0108] In this embodiment, the generation of the shooting command is described above and will not be repeated here.
[0109] Step a2, or, when the cooking equipment is connected to the mobile terminal, in response to the target ingredients and their corresponding shooting methods determined by the user through the mobile terminal, a shooting instruction is generated accordingly.
[0110] In this embodiment, the specific type of mobile terminal is determined according to actual needs, such as smartphones, smartwatches, and tablets.
[0111] In this embodiment of the invention, shooting instructions are generated through two methods: the human-computer interaction interface on the display screen of the cooking device and the interconnection with a mobile terminal. This can meet the user's operating habits in different scenarios, thereby improving the convenience and flexibility of operation.
[0112] Step S9042: Control the camera module to move to the location of the subject being photographed.
[0113] Step S9043: Configure the camera module based on the shooting parameters, and use the configured camera module to shoot the subject to obtain the corresponding shooting data.
[0114] In this embodiment, the shooting parameters are the imaging parameters of the camera. The specific parameters are adaptively adjusted according to the actual project requirements. For example, the imaging parameters include resolution, shooting angle (including top angle, such as 45° top shot, eye-level angle, such as perpendicular to the surface of the food, and bottom angle, etc.), horizontal / vertical displacement, that is, the distance the camera module moves in the horizontal (left and right) or vertical (up and down) direction, such as horizontal leftward movement of 10cm, rising to the 3rd tray, and lens tilt, that is, the angle between the lens and the horizontal plane (such as 0° horizontal, 15° tilt), to avoid reflection or obstruction (such as adjusting the angle to reduce reflection when shooting glossy meat).
[0115] In this embodiment of the invention, a dynamic shooting process is designed in which the camera module moves directly to the target food position according to the control parameters. This can achieve targeted positioning of the shooting object, thereby avoiding invalid shooting area coverage. At the same time, it can automatically adapt the shooting parameters for different foods, greatly satisfying the different shooting needs of users and thus improving the user experience.
[0116] In practical applications, users share specific cooking methods for ingredients, such as how to make braised pork, through images or videos, allowing others to learn frame by frame or share the user's current cooking experience. Therefore, after controlling the camera module to acquire the corresponding shooting data according to the control parameters, the cooking control method in this embodiment further includes:
[0117] Step b1: Share the captured data with the user through the human-computer interaction interface.
[0118] In this embodiment, the captured data may be at least one image of the cooking ingredients or a cooking video, which is only used as an example.
[0119] Step b2, or, when the cooking device is connected to the mobile terminal, the captured data is transmitted to the mobile terminal so that the user can obtain the captured data through the mobile terminal.
[0120] This invention presents a multi-terminal data sharing method that connects a human-computer interaction interface on a display screen with a mobile terminal, which enhances user engagement and convenience, thereby meeting users' personalized recording needs.
[0121] In one specific embodiment, a control scheme for a fully automatic steam oven is proposed to reduce manual operation steps and improve convenience. (See also...) Figure 10 The specific control methods include the following steps:
[0122] Step 1: The user puts in the ingredients.
[0123] In this embodiment, step 1 can be determined by the user opening the cabinet door, placing one or more layers of food into the steam oven cavity, and then closing the door. Specifically, after the steam oven is turned on, it detects that the cabinet door has been open for more than 5 seconds (other values can be set) and then closes the door to determine that the user has placed food in, thus initiating the step of starting the camera for recognition.
[0124] Step 2: The camera begins to move, identifies each layer of food, and recommends menu items.
[0125] In this embodiment, the camera movement module automatically starts moving along the vertical main direction, identifying and recording food on each layer. After identification, it automatically recommends the most suitable cooking meal. Specifically, the camera in this embodiment has network transmission capabilities, identifying the currently recorded food through the cloud network and using network big data analysis to identify the specific ingredients of the food; and combining the food identified on each layer with the cloud big data network analysis to recommend a cooking meal menu that matches the desired recipe.
[0126] Step 3: Start cooking.
[0127] In this embodiment, step 3 mainly involves user confirmation to start cooking. Specifically, confirmation can be sent to the user both on the oven's display interface and on their mobile phone, breaking the limitations of device usage and allowing users to start cooking anytime, anywhere. It should be noted that after the user confirms the start of cooking, the camera movement module returns to its initial position.
[0128] Step 4: Whether or not to record the cooking process.
[0129] In this embodiment, step 4 is mainly used to detect whether the user generates a shooting instruction, that is, whether the user needs to record a cooking video or take photos.
[0130] Step 5: The user controls the camera movement via their mobile phone.
[0131] In this embodiment, when a user needs to record cooking videos or take photos during the cooking process, they can connect their mobile phone to the steam oven (e.g., via WiFi) and control the camera movement module through the APP software. This allows the camera to move freely up, down, left, and right within a set movable area (i.e., vertically, horizontally, and rotating the camera at a 60° angle, allowing users to record or take more precise cooking videos and photos from different angles and positions at any time). This also allows users to choose their preferred food layers and different angles for recording and shooting, greatly improving the user experience.
[0132] It's worth noting that controlling the camera's movement via a mobile phone offers greater convenience, allowing users to operate the device anytime, anywhere, without being limited to the kitchen. Furthermore, recorded videos or photos can be shared and sent directly from the phone, providing a more user-friendly experience. Alternatively, the camera can be controlled via the oven's own LCD screen for recording or filming.
[0133] In summary, the cooking control method of this invention can achieve automated ingredient recognition and cooking control, while also supporting dynamic shooting, thereby meeting personalized recording needs and greatly improving the user experience.
[0134] This embodiment also provides a cooking control device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, a "module" can be a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.
[0135] This invention provides a cooking control device applied to cooking equipment. The cooking equipment includes a cabinet door and a housing. The cabinet door consists of multiple transparent layers stacked sequentially. The inner cavity of the housing has multiple layers of tray racks, each used to hold food ingredients. A camera module and a moving area are located within the middle transparent layer of the cabinet door. The camera module can move vertically and horizontally along the moving area to capture images of the food ingredients on each tray rack, such as... Figure 11 As shown, the device includes:
[0136] The acquisition module 1101 is used to control the camera module to move vertically along the moving area in response to the user's operation of placing food on at least one layer of tray rack, so as to acquire the images of the corresponding food placed on each layer of tray rack.
[0137] The determination module 1102 is used to determine the target recipe based on the images of each ingredient and push the target recipe to the user so that the user can confirm whether to use the target recipe for cooking.
[0138] The cooking module 1103 is used to control the cooking equipment to cook based on the target recipe in response to the user's confirmation operation of cooking with the target recipe.
[0139] The shooting module 1104 is used to parse the shooting command to obtain control parameters if a shooting command is detected during the cooking process, and control the camera module to acquire corresponding shooting data according to the control parameters; wherein, the shooting command is generated by the user operation.
[0140] In some optional implementations, the acquisition module 1101 includes: a first acquisition submodule, a second acquisition submodule, and a third acquisition submodule; wherein, the first acquisition submodule is used to respond to a user placing food on at least one layer of tray rack; the second acquisition submodule is used to control the camera module to move from a preset initial position to the current shooting point, and to take pictures of the food placed on the tray rack corresponding to the current shooting point to obtain the corresponding food image; the third acquisition submodule is used to control the camera module to move from the current shooting point to the next shooting point to repeat the shooting until the camera module reaches the last shooting point and obtains the corresponding food image, and then controls the camera module to return to the preset initial position.
[0141] In some optional implementations, the determining module 1102 includes: a first determining submodule and a second determining submodule; wherein, the first determining submodule is used to control the camera module to send images of each ingredient to the cloud via the wireless network when the cooking device and the camera module are connected to the wireless network, so that the cloud can identify each ingredient image and generate a target recipe, and transmit the target recipe to the cooking device via the camera module; the second determining submodule is used to push the target recipe to the user so that the user can confirm whether to use the target recipe for cooking.
[0142] In some optional implementations, the shooting module 1104 includes: a first shooting submodule, a second shooting submodule, and a third shooting submodule; wherein, the first shooting submodule is used to parse the shooting command to obtain control parameters if a shooting command is detected during the cooking process; the second shooting submodule is used to control the camera module to move to the position of the shooting object; and the third shooting submodule is used to configure the camera module based on the shooting parameters and use the configured camera module to shoot the shooting object to obtain corresponding shooting data.
[0143] In some optional implementations, the first shooting submodule includes: a first shooting unit and a second shooting unit; wherein, the first shooting unit is used to generate a shooting instruction in response to the target food and its corresponding shooting method determined by the user through the human-computer interaction interface; the second shooting unit is used, or, when the cooking equipment is interconnected with the mobile terminal, to generate a shooting instruction in response to the target food and its corresponding shooting method determined by the user through the mobile terminal.
[0144] In some alternative implementations, the device further includes: a sharing module for sharing the captured data with the user through a human-computer interaction interface; or, when the cooking device is interconnected with a mobile terminal, transmitting the captured data to the mobile terminal so that the user can obtain the captured data through the mobile terminal.
[0145] Further functional descriptions of the above modules are the same as those in the corresponding embodiments described above, and will not be repeated here.
[0146] In this embodiment, the cooking control device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.
[0147] The cooking control device of this invention identifies placed ingredients by controlling the camera module to move vertically along a moving area, and automatically matches a target recipe based on the identified ingredients, effectively avoiding the tedious steps of manually searching for recipes. Furthermore, once the user confirms the use of the target recipe for cooking, the device automatically calls the target recipe to perform the corresponding cooking, simplifying the cooking process, achieving intelligent cooking, and streamlining the operation process, thereby improving cooking control efficiency. Simultaneously, after the user issues a shooting command, the device dynamically controls the camera to move to a designated position to obtain high-definition images or videos, effectively meeting the user's personalized recording needs. It has significant advantages such as convenient operation, precise control, high cooking accuracy, and good results.
[0148] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor main control chips, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the methods shown in the above embodiments are implemented.
[0149] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.
Claims
1. A cooking control method applied to a cooking device, the cooking device comprising a cabinet door and a housing, the cabinet door being multiple transparent layers stacked sequentially, and the housing having an inner cavity equipped with multiple layers of tray racks, wherein each layer of tray racks is used for placing food ingredients, characterized in that... The cabinet door has a transparent middle layer housing a camera module and a moving area; the camera module can move vertically and horizontally along the moving area to capture images of the food items on each tray shelf. The method includes: In response to a user placing food on at least one tray rack, the camera module is controlled to move vertically along the moving area to acquire images of the corresponding food placed on each tray rack. A target recipe is determined based on the images of each ingredient, and the target recipe is pushed to the user so that the user can confirm whether to use the target recipe for cooking. In response to the user's confirmation operation of cooking using the target recipe, the cooking equipment is controlled to cook based on the target recipe; If a shooting command is detected during the cooking process, the shooting command is parsed to obtain control parameters, and the camera module is controlled to acquire corresponding shooting data according to the control parameters; wherein, the shooting command is generated by the user operation.
2. The cooking control method according to claim 1, characterized in that, The moving area includes a vertical moving area, on which multiple shooting points are provided; wherein, the number of shooting points is determined by the number of layers of the tray rack, and the position of the shooting points is determined according to the corresponding position of each layer of the tray rack; the control camera module moves vertically along the moving area to acquire images of the food items placed on each layer of the tray rack, including: The camera module is controlled to move from a preset initial position to the current shooting point, and the food placed on the tray rack corresponding to the current shooting point is photographed to obtain the corresponding food image; The camera module is controlled to move from the current shooting point to the next shooting point to repeatedly shoot until the camera module reaches the last shooting point and obtains the corresponding food image. Then, the camera module is controlled to return to the preset initial position.
3. The cooking control method according to claim 1, characterized in that, The step of determining the target recipe based on the images of each ingredient includes: When the cooking equipment and the camera module are connected to the wireless network, the camera module is controlled to send the images of each ingredient to the cloud through the wireless network, so that the cloud can identify each ingredient image, generate a target recipe, and transmit the target recipe to the cooking equipment through the camera module.
4. The cooking control method according to any one of claims 1 to 3, characterized in that, The control parameters include the object being photographed and its corresponding shooting parameters; controlling the camera module to acquire corresponding shooting data according to the control parameters includes: Control the camera module to move to the location of the subject being filmed; The camera module is configured based on the shooting parameters, and the configured camera module is used to shoot the subject to obtain the corresponding shooting data.
5. The cooking control method according to claim 1, characterized in that, The cooking device also includes a display screen for displaying a human-computer interaction interface; the process of determining the shooting command includes: In response to the target food and corresponding shooting method determined by the user through the human-computer interaction interface, a shooting command is generated accordingly. Alternatively, when the cooking equipment is interconnected with the mobile terminal, a shooting command is generated in response to the target ingredient and its corresponding shooting method determined by the user through the mobile terminal.
6. The cooking control method according to claim 5, characterized in that, After controlling the camera module to acquire corresponding shooting data according to the control parameters, the method further includes: The captured data is shared with the user through a human-computer interaction interface; Alternatively, when the cooking equipment is interconnected with the mobile terminal, the captured data can be transmitted to the mobile terminal so that the user can obtain the captured data through the mobile terminal.
7. A cooking control device applied to a cooking appliance, the cooking appliance comprising a cabinet door and a housing, the cabinet door being multiple transparent layers stacked sequentially, and the housing having an inner cavity provided with multiple layers of tray racks, wherein each layer of tray racks is used for placing food ingredients, characterized in that... The cabinet door has a transparent middle layer housing a camera module and a moving area; the camera module can move vertically and horizontally along the moving area to capture images of the food items on each tray shelf. The device includes: The acquisition module is used to respond to the user's operation of placing food on at least one layer of tray racks, and control the camera module to move vertically along the moving area to acquire images of the corresponding food placed on each layer of tray racks respectively; The determination module is used to determine a target recipe based on the images of each ingredient, and push the target recipe to the user so that the user can confirm whether to use the target recipe for cooking. The cooking module is used to control the cooking equipment to cook based on the target recipe in response to the user's confirmation operation of cooking with the target recipe; The shooting module is used to parse the shooting command to obtain control parameters if a shooting command is detected during the cooking process, and control the camera module to acquire corresponding shooting data according to the control parameters; wherein the shooting command is generated by the user operation.
8. A cooking device, comprising a display screen, a cabinet door, and a housing, wherein the display screen is used to display a human-computer interaction interface, the cabinet door consists of multiple transparent layers stacked sequentially, and the inner cavity of the housing is provided with multiple tray racks, wherein each tray rack is used to place food ingredients, characterized in that... The cabinet door has a transparent middle layer with a camera module and a moving area. The camera module can move vertically and horizontally along the moving area to capture images of the food on each tray shelf. The cooking device also includes a controller, which includes a memory and a processor. The memory and the processor are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to perform the cooking control method according to any one of claims 1 to 6.
9. The cooking apparatus according to claim 8, characterized in that, The cooking equipment is a steam oven.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the cooking control method according to any one of claims 1 to 6.