Baking control method and device, baking equipment and storage medium

By using display screens, visual cameras and robots in baking equipment, food posture detection and correction are realized, and the baking process is automated, which solves the problems of complex operation of existing baking equipment and insufficient requirements in the internal area of ​​the oven, improving user experience and Easy to operate.

CN118927251AActive Publication Date: 2024-11-12GREE ELECTRIC APPLIANCE INC OF ZHUHAI

Patent Information

Application Number
CN202411175874.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-11-12
Estimated Expiration
2044-08-26

AI Technical Summary

Technical Problem

The existing baking equipment is complex to operate. Users need to manually set parameters and the temperature is high after removing the tray, which affects the user experience and lacks the need to meet the internal area of ​​the oven.

Method used

Design a baking control method, using display screen, visual camera and robotic hand to realize food posture detection and correction, automatically grab the ingredients from the ingredient placement area and place them on the baking tray, and determine based on the image that the baking is completed after baking is completed. , simplify the baking process.

Benefits of technology

It realizes automated, efficient and beautiful control of ingredients baking, meets users' efficient and convenient customization needs, and improves user experience and simplicity of operation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118927251A_ABST
    Figure CN118927251A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of baking control, and discloses a baking control method and device, baking equipment and a storage medium, the baking control method and device are applied to the baking equipment, a display screen is arranged outside the baking equipment to display a human-computer interaction interface, and at least one visual camera, a manipulator, a food material placing area, a baking tray placing area and a dinner plate placing area are arranged inside the baking equipment. At least one food material is placed in the food material placing area, and the method comprises the following steps: controlling the manipulator to grab the food material from the food material placing area and then carrying out posture detection; if the posture deviation is detected, the mechanical arm is controlled to conduct correction, and the corrected food materials are placed in a baking tray containing area; after all the food materials on the food material placing area are placed in the baking tray placing area, baking is conducted, a current food material image of the baking tray placing area is obtained, and after it is judged that baking is completed, the mechanical arm is controlled to place the food materials in the baking tray placing area in the dinner plate placing area in response to the tray placing mode determined by the user through the man-machine interaction interface. The operation process can be simplified, and then the baking control efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of baking control, and in particular to a baking control method, device, baking equipment and storage medium. Background Art

[0002] At present, baking equipment, such as ovens, has gradually entered people's daily lives and become a common tool for people to make and cook. However, during the baking process, users are still required to manually set relevant parameters, which greatly increases the difficulty and complexity of the operation. In addition, the existing ovens have a single structure. For example, there is no placement table for placing the tray after taking out the tray. After the food is baked, it is inconvenient for the user to take out the tray with the food, because the temperature of the tray taken out from the oven is high and there is nowhere to put it, which seriously affects the user's experience.

[0003] Based on the above problems, the related technology improves the interior of the oven and increases the tray placement area, but ignores the different setting requirements of the internal areas of the oven, such as the food placement area. In addition, the control of each area set inside the oven still requires manual assistance to achieve the corresponding baking operation, which has the defects of high control cost and complex operation, and it is difficult to meet the use requirements of this type of baking equipment. Summary of the invention

[0004] In view of this, the present invention provides a baking control method, device, baking equipment and storage medium to solve the problem that the manual control operation of the existing baking equipment is cumbersome and difficult to meet the user's needs for efficient and convenient use.

[0005] In a first aspect, the present invention provides a baking control method, which is applied to a baking device, wherein a display screen is arranged outside the baking device, and the display screen is used to display a human-computer interaction interface, and at least one visual camera, a manipulator, a food placement area, a baking tray placement area, and a dinner plate placement area are arranged inside the baking device, wherein at least one food is placed in the food placement area, and the method comprises:

[0006] Control the robot to grab the food from the food placement area in sequence;

[0007] Perform posture detection on the food ingredients one by one;

[0008] If the posture deviation of the food is detected, the robot arm is controlled to make corrections and the corrected food is placed on the baking tray placement area;

[0009] After all the ingredients on the ingredient placement area are placed on the baking tray placement area, baking is performed and a current ingredient image of the baking tray placement area is obtained;

[0010] Determine whether baking is complete based on the current food image;

[0011] If baking is completed, in response to the plating method determined by the user through the human-computer interaction interface, the robot is controlled to place the ingredients in the baking tray placement area into the dinner plate placement area.

[0012] The baking control method provided by the present invention controls the robot to detect and correct the posture of grabbing food from the food placement area, so as to adjust the placement of food on the baking tray placement area; determines whether baking is completed through real-time detection of the current food image during the baking process, and completes the corresponding food plating with the help of the robot based on the user's own plating needs after baking is completed. It can simplify the food baking process and realize automatic, efficient and beautiful control of the food baking process, which greatly meets the user's demand for efficient and convenient customized use of baking equipment.

[0013] In an optional implementation, performing posture detection on food includes:

[0014] The robot arm that grabs the food moves to the location of the first visual camera and obtains a first image of the food;

[0015] Determine whether there is an angle deviation between the first image and the preset image;

[0016] If there is an angle deviation, it is determined that the food has a posture deviation;

[0017] If there is no angle deviation, it is determined that the food has no posture deviation.

[0018] The present invention is designed to detect the posture of the robot arm grabbing food. It determines whether there is a posture deviation by judging the angle deviation between the first image corresponding to the food and the set standard image. This can ensure that the corresponding position of the food is in the standard orientation, which helps to enhance the subsequent baking efficiency of the food and greatly improves the user experience.

[0019] In an optional embodiment, controlling the manipulator to perform correction includes:

[0020] determining a correction angle based on an angular deviation between the first image and a preset image;

[0021] The robot is controlled to make corrections based on the correction angle.

[0022] The present invention controls the manipulator to correct the position of the food based on the angular deviation between the first image corresponding to the food and the set standard image, thereby ensuring that the corresponding position of the food is in the standard orientation, thereby facilitating the accurate placement of the food in the baking tray placement area.

[0023] In an optional implementation, judging whether baking is completed based on the current food image includes:

[0024] Upload the current food image to the cloud server so that the cloud server can identify the current color of the current food image, determine the current food state based on the current color and the preset food state, determine whether baking is completed based on the current food state, and receive the judgment result fed back by the cloud server, wherein the preset food state includes the correspondence between different colors and corresponding baking states;

[0025] Alternatively, the current food image is uploaded to the human-computer interaction interface, so that the user can confirm whether the baking is completed through the human-computer interaction interface.

[0026] The present invention uploads the current food image to the cloud server for automatic food baking progress judgment, and uploads it to the human-computer interaction interface for manual semi-automatic food baking progress judgment. Based on the design of the above two food baking progress judgment processes, the accurate judgment of the food baking process is improved to a certain extent, which helps to ensure the accuracy of the judgment of the completion of food baking.

[0027] In an optional implementation, before determining whether baking is completed based on the current food image, the baking control method further includes:

[0028] Upload the current food image to the cloud server or human-computer interaction interface to determine whether a flipping operation is required;

[0029] When turning over is required, the robot is controlled to turn over the food.

[0030] The present invention takes into account the routine operations of the actual food baking process, and uses a cloud server or a human-computer interaction interface to determine the food baking process based on the current food image. This not only improves the food baking operation process, but also increases user satisfaction with the use of the baking equipment.

[0031] In an optional embodiment, in response to the plating method determined by the user through the human-computer interaction interface, controlling the robot to place the ingredients in the baking tray placement area to the dinner plate placement area includes:

[0032] Get the current plate data of the baking tray placement area, which includes the plate size, shape and position;

[0033] Determine a target plating method based on the plate data and the plating method;

[0034] Control the robot to place the ingredients in the baking tray placement area into the dinner plate placement area according to the target plating method.

[0035] The present invention takes into account the personalized plating needs of users, and through customized food plating methods and combined with the plate data in the current baking equipment, adaptively realizes customized plating after the food is baked, which can greatly enhance the user experience.

[0036] In an optional embodiment, after placing the calibrated food into the baking tray placement area, the baking control method further includes:

[0037] Acquire a second image of the food;

[0038] Detecting whether there is a deviation in the placement position of the food based on the second image;

[0039] If there is a deviation in the placement of the food, the deviation angle is determined, and the robot is controlled to make corrections based on the deviation angle;

[0040] If there is no deviation in the placement position of the ingredients, the step of baking and obtaining a current ingredient image of the baking tray placement area is performed after all ingredients on the ingredient placement area are placed on the baking tray placement area.

[0041] The present invention is also designed to detect and correct whether there is any deviation in the corresponding placement positions of all ingredients on the food placement area, ensuring that all ingredients on the food placement area are in the set standard placement orientation, which can also speed up the subsequent food baking process to a certain extent.

[0042] In an optional embodiment, the baking control method further includes:

[0043] If no posture deviation of the food is detected, the robot arm is controlled to place the food on the baking tray placement area.

[0044] In an optional embodiment, before controlling the manipulator to sequentially grab food from the food placement area, the baking control method further includes:

[0045] Check whether all ingredients on the food placement area are in place;

[0046] If yes, executing the step of controlling the manipulator to grab the food from the food placement area in sequence;

[0047] If not, the process returns to the step of checking whether all the ingredients on the ingredient placement area are in place.

[0048] Before controlling the manipulator to grab food, the present invention designs a detection system to detect whether food is loaded in place, thereby ensuring the normal operation of subsequent processes of grabbing, correcting, baking and plating food, thereby achieving automatic and efficient control of food baking.

[0049] In a second aspect, the present invention provides a baking control device, which is applied to a baking device. A display screen is arranged outside the baking device, and the display screen is used to display a human-computer interaction interface. At least one visual camera, a manipulator, a food placement area, a baking tray placement area, and a dinner plate placement area are arranged inside the baking device, wherein at least one food is placed in the food placement area. The device comprises:

[0050] The grabbing module is used to control the robot to grab the food from the food placement area in sequence;

[0051] A detection module, used to detect the posture of the food in turn;

[0052] A correction module is used to control the robot to make corrections if a posture deviation of the food is detected, and to place the corrected food on the baking tray placement area;

[0053] A baking module, used for baking and obtaining a current image of the ingredients in the baking tray placement area after all the ingredients in the ingredient placement area are placed in the baking tray placement area;

[0054] A judgment module, used to judge whether baking is completed based on the current food image;

[0055] The plating module is used to control the robot to place the ingredients in the baking tray placement area into the dinner plate placement area in response to the plating method determined by the user through the human-computer interaction interface if baking is completed.

[0056] The baking control device provided by the present invention combines baking equipment, a manipulator and a visual system. It controls the manipulator to detect and correct the posture of grabbing food from the food placement area, determines whether baking is completed based on real-time detection of the current food image during the baking process, and uses the manipulator to complete the corresponding food plating based on the user's own plating needs after baking is completed. This greatly simplifies the baking process of food, realizes automatic, efficient and beautiful baking control of food, and meets the user's efficient, convenient and personalized use needs.

[0057] In a third aspect, the present invention provides a baking device, wherein a display screen is arranged on the outside of the baking device, and the display screen is used to display a human-computer interaction interface. The baking device is internally provided with at least one visual camera, a manipulator, a food placement area, a baking tray placement area, and a dinner plate placement area, wherein at least one food is placed in the food placement area. The baking device also includes a controller, and the controller includes: a memory and a processor, and the memory and the processor are communicatively connected to each other, and computer instructions are stored in the memory. The processor executes the computer instructions to execute the baking control method of the first aspect or any corresponding embodiment thereof.

[0058] In an optional embodiment, the baking device is an oven.

[0059] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the baking control method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0061] Figure 1 is a flow chart of a baking control method according to an embodiment of the present invention;

[0062] Figure 2 is a flow chart of another baking control method according to an embodiment of the present invention;

[0063] Figure 3 It is a structural schematic diagram of baking equipment;

[0064] Figure 4 It is a schematic diagram of the structure of the robot;

[0065] Figure 5 It is a schematic diagram of the process of food grabbing and correction;

[0066] Figure 6 is a structural block diagram of a baking control device according to an embodiment of the present invention;

[0067] Figure 7 Schematic diagram of the structure of a baking controller according to an embodiment of the present invention. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0069] According to an embodiment of the present invention, a baking control method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0070] In this embodiment, a baking control method is provided, which is applied to a baking device. A display screen is arranged outside the baking device, and the display screen is used to display a human-computer interaction interface. At least one visual camera, a manipulator, a food placement area, a baking tray placement area, and a dinner plate placement area are arranged inside the baking device, wherein at least one food is placed in the food placement area. Figure 1 is a flow chart of a baking control method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0071] Step S101, controlling the robot arm to grab food from the food placement area in sequence.

[0072] It should be noted that the specific type and control method of the manipulator in this embodiment are not limited here, and are determined based on the adaptability of the actual project. For example, the manipulator is a three-degree-of-freedom manipulator that can perform three independent rotations or translations in three-dimensional space. This type of manipulator is usually composed of a device connected by three rotating joints, each of which can make the manipulator move in one axis, such as rotating the manipulator around its axis, so as to achieve the grasping, movement and operation of objects, which is only an exemplary description.

[0073] Step S102, performing posture detection on the food ingredients in sequence.

[0074] In this embodiment, the specific content of posture detection is not limited here, and is adaptively adjusted based on actual needs. For example, the use of a pre-trained deep learning algorithm model to implement posture detection of food is only an exemplary description.

[0075] Step S103: If it is detected that the food has posture deviation, the robot arm is controlled to make corrections and the corrected food is placed on the baking tray placement area.

[0076] In this embodiment, if there is a posture deviation of the food, it means that the current position of the food grasped by the robot is not correct. Placing it directly in the baking tray placement area may increase the subsequent baking time. Therefore, it is necessary to correct its posture with the set standard orientation to ensure the completion efficiency of the subsequent food baking process.

[0077] Step S104, after all the ingredients on the ingredient placement area are placed on the baking tray placement area, baking is performed and a current ingredient image of the baking tray placement area is obtained.

[0078] It should be noted that the method for acquiring the current food image in this embodiment is not limited here, and can be adaptively adjusted based on actual needs. For example, the real-time acquisition of the current food image is achieved by using a visual camera installed inside the oven to monitor the baking tray placement area, which is only an exemplary description.

[0079] Step S105, judging whether baking is completed based on the current food image.

[0080] In this embodiment, the specific basis for judging whether the baking is completed by the current food image is not limited here, and is adaptively set based on actual needs. For example, the judgment is made by detecting the color change of the food image, such as the bright red meat gradually changes color under the charcoal fire, and the surface has an attractive oily sheen, which is an obvious sign that the food is cooked. It should be noted that different ingredients will have different color changes. For example, meat will change from red to golden yellow, and vegetables will change from raw colors to brighter colors. Alternatively, it can be judged by changes in the shape, texture, and gloss of the food. For example, when the cake is baked, in addition to judging by the golden or light brown color on its surface, it can also be judged by its corresponding edge texture, degree of expansion, gloss and other key features for detailed judgment and analysis, which is only an exemplary description.

[0081] Step S106, if baking is completed, in response to the plating method determined by the user through the human-computer interaction interface, the robot is controlled to place the ingredients in the baking tray placement area into the dinner plate placement area.

[0082] The baking control method of the embodiment of the present invention controls the robot to detect and correct the posture of grabbing food from the food placement area, so as to adjust the placement of food on the baking tray placement area; determines whether baking is completed through real-time detection of the current food image during the baking process, and after baking is completed, completes the corresponding food plating with the help of the robot based on the user's own plating requirements. This can simplify the food baking process and realize automatic, efficient and beautiful control of the food baking process, which greatly meets the user's demand for efficient and convenient customized use of baking equipment.

[0083] In this embodiment, a baking control method is provided, which is applied to a baking device. A display screen is arranged outside the baking device, and the display screen is used to display a human-computer interaction interface. At least one visual camera, a manipulator, a food placement area, a baking tray placement area, and a dinner plate placement area are arranged inside the baking device, wherein at least one food is placed in the food placement area. Figure 2 is a flow chart of another baking control method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0084] It should be noted that before the upgrade control process in this embodiment is executed, it is necessary to detect whether the food in the food placement area has been loaded. Specifically, before controlling the manipulator to grab food from the food placement area in sequence, the baking control method of this embodiment also includes:

[0085] Step a1, checking whether all the ingredients on the food placement area are in place.

[0086] In actual applications, if the user is loading food onto the food placement area while the robot is grabbing food, the robot may rub against the user, causing injury to the user. Therefore, this embodiment performs food grabbing by the robot after loading the food, which can ensure the safety of the user and avoid potential dangers.

[0087] It should be noted that the detection method of whether the ingredients are loaded in place in this embodiment is not limited here, and can be adaptively adjusted based on actual needs. For example, it can be determined by visually detecting whether there is a user loading action within a certain period of time; or by detecting the opening and closing state of a baking device, such as an oven, and confirming that all ingredients have been loaded in place when the door lock body of the oven is in the closed state. This is only an exemplary description.

[0088] Step a2: If yes, execute step S201 of controlling the robot arm to grab food from the food placement area in sequence.

[0089] Step a3: If not, return to step a1 to check whether all the ingredients passing through the ingredient placement area are loaded in place.

[0090] Before controlling the robot arm to grab food, the embodiment of the present invention designs a detection system to detect whether the food is loaded in place, thereby ensuring the normal operation of subsequent processes of grabbing, correcting, baking and plating the food, thereby achieving automatic and efficient control of food baking.

[0091] Step S201, control the robot to grab food from the food placement area in sequence. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0092] Step S202, performing posture detection on the food ingredients in sequence.

[0093] Specifically, the posture detection of food in the above step S202 includes:

[0094] Step S2021, moving the robot arm that grabs the food to the location of the first visual camera, and acquiring a first image of the food.

[0095] In this embodiment, the corresponding installation positions and specific models of the manipulator and the first visual camera can be adaptively adjusted based on actual project requirements.

[0096] Step S2022, determining whether there is an angle deviation between the first image and the preset image.

[0097] In this embodiment, the specific content of the preset image is not limited here, and is adaptively adjusted based on the type of food and actual project requirements. For example, for the first image of the food ingredient being potatoes, the preset image is set to include potatoes and the potatoes are placed in the positive direction, which is only used as an exemplary description.

[0098] Step S2023: If there is an angle deviation, it is determined that the food has a posture deviation.

[0099] Step S2024: If there is no angle deviation, it is determined that there is no posture deviation of the food.

[0100] In the embodiment of the present invention, a posture detection is designed for the robot arm to grab food. The angle deviation between the first image corresponding to the food and the set standard image is judged to determine whether there is a posture deviation. This can ensure that the corresponding position of the food is in the standard orientation, which helps to enhance the subsequent baking efficiency of the food and greatly improves the user experience.

[0101] Step S203: If it is detected that the food has posture deviation, the robot arm is controlled to make corrections and the corrected food is placed on the baking tray placement area.

[0102] Specifically, in the above step S203, controlling the manipulator to perform calibration includes:

[0103] Step b1, determining a correction angle based on an angle deviation between the first image and a preset image.

[0104] In this embodiment, assuming that the food is potato, its corresponding preset image is the potato in a set standard orientation, and the angle between the corresponding first image and the preset image is obtained through the deep learning algorithm model to determine the corresponding correction angle, which is only for illustrative purposes.

[0105] Step b2, controlling the robot to perform correction based on the correction angle.

[0106] In the embodiment of the present invention, by controlling the robot arm to correct the position of the food based on the angular deviation between the first image corresponding to the food and the set standard image, it can be ensured that the corresponding position of the food is in the standard orientation, which helps to accurately place the food in the baking tray placement area.

[0107] In this embodiment, if no posture deviation of the food is detected, the robot is controlled to place the food on the baking tray placement area. Specifically, placing the food without posture deviation directly on the baking tray placement area can speed up the operation of the baking process and further improve the efficiency of baking control.

[0108] It should be noted that, in order to improve the baking efficiency in this embodiment, after placing the food on the baking tray placement area, it is necessary to perform a deviation detection on the food placement position again. Specifically, after placing the corrected food on the baking tray placement area in the above step S203, the baking control method of this embodiment further includes:

[0109] Step c1, obtaining a second image of the food.

[0110] In this embodiment, the same visual camera installed during the posture detection process of grabbing food can be used to obtain the food image corresponding to the baking tray placement area; a corresponding visual camera can also be set in the baking tray placement area to collect real-time images of food. It should be noted that the number of visual cameras used in this embodiment is adaptively adjusted based on the actual camera model and project requirements. For example, if the camera is wide-angle and the current project cost is low, you can choose to install a visual camera to complete different image acquisitions. This is only an exemplary description.

[0111] Step c2: detecting whether there is any deviation in the placement position of the food based on the second image.

[0112] Step c3: if there is a deviation in the placement position of the food, determine the deviation angle and control the robot to make corrections based on the deviation angle.

[0113] It should be noted that the deviation detection and correction of the food placement position in this embodiment can refer to the relevant content in the previous text and will not be repeated here.

[0114] Step c4, if there is no deviation in the placement position of the ingredients, then after all the ingredients on the ingredient placement area are placed on the baking tray placement area, baking is performed and a current ingredient image of the baking tray placement area is obtained.

[0115] The embodiment of the present invention is designed to detect and correct whether there is a deviation in the corresponding placement positions of all ingredients on the ingredient placement area, which can ensure that all ingredients on the ingredient placement area are in a set standard placement orientation, greatly speeding up the subsequent ingredient baking process.

[0116] Step S204: After all ingredients on the ingredient placement area are placed on the baking tray placement area, baking is performed and the current ingredient image of the baking tray placement area is obtained. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0117] Step S205, judging whether baking is completed based on the current food image.

[0118] In this embodiment, a fully automatic form of baking progress determination based on a visual algorithm and a semi-automatic baking completion determination form based on manual experience are designed. Specifically, the above step S205 includes:

[0119] Step d1, upload the current ingredient image to the cloud server so that the cloud server can identify the current color of the current ingredient image, determine the current ingredient state based on the current color and the preset ingredient state, determine whether baking is completed based on the current ingredient state, and receive the judgment result fed back by the cloud server, wherein the preset ingredient state includes the correspondence between different colors and corresponding baking states.

[0120] It should be noted that the specific content of the preset food status in this embodiment is not limited here and is adaptively set based on actual needs.

[0121] Step d2, uploading the current food image to the human-computer interaction interface, so that the user can confirm whether the baking is completed through the human-computer interaction interface.

[0122] In the embodiment of the present invention, the current food image is uploaded to the cloud server for automatic food baking progress judgment, and uploaded to the human-computer interaction interface for manual semi-automatic food baking progress judgment. Based on the design of the above two food baking progress judgment processes, the accurate judgment of the food baking process is improved to a certain extent, which helps to ensure the accuracy of the judgment of whether the food baking is completed.

[0123] It should be noted that in this embodiment, the food in the baking process can also be stir-fried. Specifically, before judging whether the baking is completed based on the current food image, the baking control method of this embodiment also includes:

[0124] The current food image is uploaded to the cloud server or the human-computer interaction interface to determine whether a flipping operation is required; when a flipping operation is required, the robot arm is controlled to flip the food.

[0125] In the embodiment of the present invention, the routine operation of the actual food baking process is taken into consideration, and the design of judging the food baking process based on the current food image, namely, a cloud server or a human-computer interaction interface, is used. This not only improves the food baking operation process, but also increases the user satisfaction with the use of the baking equipment.

[0126] Step S206, if baking is completed, in response to the plating method determined by the user through the human-computer interaction interface, the robot is controlled to place the ingredients in the baking tray placement area into the dinner plate placement area.

[0127] Specifically, in the above step S206, in response to the dish arrangement method determined by the user through the human-computer interaction interface, controlling the robot to place the ingredients in the baking tray placement area into the dinner plate placement area includes:

[0128] Step S2061, obtaining the current plate data of the baking tray placement area, where the plate data includes the size, shape and position of the plate.

[0129] In this embodiment, the specific method for acquiring the plate data is not limited here, and is known based on conventional data acquisition methods in the art.

[0130] Step S2062, determining a target plating method based on the plate data and the plating method.

[0131] In this embodiment, the target plating method is the best way to place the ingredients. Specifically, in this embodiment, the size, shape and position of the actual plate are taken into consideration, and combined with the set plating method, the robot can place the currently baked ingredients in the best plating.

[0132] Step S2063, controlling the robot arm to place the ingredients in the baking tray placement area into the dinner plate placement area according to the target plating method.

[0133] The embodiment of the present invention takes into account the personalized plating needs of users, and adaptively realizes customized plating after the ingredients are baked by customizing the plating method of ingredients and combining the plate data in the current baking device, which can greatly enhance the user experience.

[0134] In a specific embodiment, a control scheme for a fully automatic oven is proposed to reduce manual operation steps and improve life convenience. Figure 3 On the basis of the original oven, the food placement area, plate placement area, robot, and visual camera are added, and the human-computer interaction interface is combined to realize the full automatic control of the oven. That is, through the human-computer interaction interface, the robot can be customized to control the robot to simulate and replace human hands to achieve the precise positioning and placement of food. Specifically, the food placement area and plate placement area are expanded on one side of the oven. The inner side of the expanded side can be opened. The robot is installed on the X-axis slide and a Z-axis telescopic cylinder is installed above it for clamping. The specific installation position is as follows: Figure 3 shown.

[0135] In this embodiment, a robot with three axial degrees of freedom, XYZ, is used. Figure 4 It is a schematic diagram of the structure of the manipulator. It should be noted that the Y-axis where the Y-axis motor is located in the figure is the angle correction axis. The manipulator is based on a high-precision motion control system based on vision and servo control, which can accurately and quickly pick up food. Through the visual camera 1, it can check whether the food is square when it is grabbed, and when there is an angle deviation, it can control the visual manipulator to rotate the corresponding angle through correction to correct the current posture of the food. For example, when the visual camera 1 detects that the chicken wing is a little off the center, the rotation and gripping angle of the manipulator can be adjusted to grip the chicken wing in the center and then put it on the baking tray.

[0136] In a specific embodiment, when the food is placed on the baking tray, the visual camera 2 will also provide correction feedback on the food. If a deviation is detected when the food is placed, the manipulator will be moved again to correct the angle. Specifically, the manipulator places the food on the baking tray, and the visual camera 2 detects and scans the baking tray to determine whether the position of the food is deviated. The visual system sends the detected posture position data to the visual controller. After data processing, the controller sends data to the manipulator control motor, and the drive motor controls the manipulator to correct the angle. Through the visual system and the manipulator control system, the food can be placed on the baking tray in sequence.

[0137] In a specific embodiment, the robot first performs posture inspection based on the visual camera 1, that is, it calibrates whether the food grasped in the food placement area is grasped squarely. Figure 5 As shown; then place the corrected ingredients on the baking tray, and perform a calibration process to see if there is any deviation in the position of the ingredients on the baking tray. Through the above two methods of correcting the position of ingredients, it can be ensured that all the ingredients on the baking tray are in a square position.

[0138] In a specific embodiment, during the baking stage, the color change detection of the food is realized by the visual camera 2 to control the manipulator to grab and turn the food; at the same time, the visual camera 2 will provide real-time feedback on the current baking state of the food to determine whether the baking is completed. Specifically, the visual camera 2 can transmit the food state picture to the cloud or the user interaction end (i.e., the human-computer interaction interface) in real time, and compare the color change of different foods with the color change of the menu library to determine whether the food executes the turning instruction or the baking end instruction; or the user can manually control the turning of the food or end the baking by transmitting and displaying the real-time food picture through the human-computer interaction interface.

[0139] In this embodiment, the user can independently select the plating method through the human-computer interaction interface. The visual camera scans the plate and accurately positions the plate based on the recording of the size and shape of the plate, thereby controlling the robot arm to accurately clamp the baked food onto the plate, and the corresponding plating method is achieved in combination with the set exquisite plating method.

[0140] In summary, the baking control method of the embodiment of the present invention combines a precision manipulator with a visual system and is equipped with two visual cameras to adjust the position and posture of the food, and detect and feedback the target position, thereby realizing all-round, automated rapid grasping, correction, and positioning of the food, and realizing data transmission conversion and intelligent control through a human-computer interaction interface and an intelligent controller, thereby truly freeing the hands and completing automated and personalized baking and placement of the food.

[0141] In this embodiment, a baking control device is also provided, which is used to implement the above embodiments and preferred implementations, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0142] The present invention provides a baking control device, which is applied to a baking device. A display screen is arranged outside the baking device, and the display screen is used to display a human-computer interaction interface. At least one visual camera, a manipulator, a food placement area, a baking tray placement area, and a dinner plate placement area are arranged inside the baking device, wherein at least one food is placed in the food placement area, such as Figure 6 As shown, the device comprises:

[0143] The grabbing module 601 is used to control the robot to grab food from the food placement area in sequence.

[0144] The detection module 602 is used to detect the posture of the food in sequence.

[0145] The correction module 603 is used to control the robot to perform correction if a posture deviation of the food is detected, and place the corrected food on the baking tray placement area.

[0146] The baking module 604 is used to bake and obtain the current ingredient image of the baking tray placement area after all ingredients on the ingredient placement area are placed on the baking tray placement area.

[0147] The judgment module 605 is used to judge whether baking is completed based on the current food image.

[0148] The plating module 606 is used to control the robot to place the ingredients in the baking tray placement area to the dinner plate placement area in response to the plating method determined by the user through the human-computer interaction interface if baking is completed.

[0149] In some optional embodiments, the detection module 602 includes: a first detection submodule, a second detection submodule, a third detection submodule and a fourth detection submodule; wherein the first detection submodule is used to move the robot arm after grabbing the food to the position of the first visual camera and obtain a first image of the food; the second detection submodule is used to determine whether there is an angle deviation between the first image and the preset image; the third detection submodule is used to determine whether there is a posture deviation of the food if there is an angle deviation; the fourth detection submodule is used to determine whether there is no posture deviation of the food if there is no angle deviation.

[0150] In some optional embodiments, the correction module 603 includes: a first correction submodule and a second correction submodule; wherein the first correction submodule is used to determine the correction angle based on the angle deviation between the first image and the preset image; and the second correction submodule is used to control the manipulator to perform correction based on the correction angle.

[0151] In some optional embodiments, the correction module 603 also includes: a first correction submodule, a second correction submodule, a third correction submodule and a fourth correction submodule; wherein the first correction submodule is used to obtain a second image of the food; the second correction submodule is used to detect whether there is a deviation in the placement position of the food based on the second image; the third correction submodule is used to determine the deviation angle if there is a deviation in the placement position of the food, and control the manipulator to perform correction based on the deviation angle; the fourth correction submodule is used to execute the steps of baking and obtaining the current image of the food in the baking tray placement area after all the food in the food placement area is placed in the baking tray placement area if there is no deviation in the placement position of the food.

[0152] In some optional implementations, the correction module 603 further includes: a control submodule, configured to control the robot arm to place the food on the baking tray placement area if no posture deviation of the food is detected.

[0153] In some optional embodiments, the judgment module 605 includes: a first judgment submodule and a second judgment submodule; wherein, the first judgment submodule is used to upload the current ingredient image to the cloud server so that the cloud server can identify the current color of the current ingredient image, determine the current ingredient status based on the current color and the preset ingredient status, determine whether baking is completed based on the current ingredient status, and receive the judgment result fed back by the cloud server, wherein the preset ingredient status includes the correspondence between different colors and corresponding baking status; the second judgment submodule is used to upload the current ingredient image to the human-computer interaction interface, so that the user can confirm whether baking is completed through the human-computer interaction interface.

[0154] In some optional embodiments, the judgment module 605 also includes: a third judgment submodule, used to upload the current food image to the cloud server or the human-computer interaction interface to determine whether a flipping operation is required; when a flipping operation is required, controlling the robot arm to flip the food.

[0155] In some optional embodiments, the plating module 606 includes: a first plating plate module, a second plating plate module and a third plating plate module; wherein the first plating plate module is used to obtain the current plate data of the baking tray placement area, the plate data including the plate size, shape and position; the second plating plate module is used to determine the target plating method based on the plate data and the plating method; the third plating plate module is used to control the robot to place the ingredients in the baking tray placement area to the plate placement area according to the target plating method.

[0156] In some optional embodiments, the device also includes: a judgment module, used to detect whether all ingredients on the food placement area are in place; if so, execute the step of controlling the robot arm to grab ingredients from the food placement area in sequence; if not, return to the step of detecting whether all ingredients on the food placement area are in place.

[0157] The further functional description of each of the above modules is the same as that of the above corresponding embodiments and will not be repeated here.

[0158] The baking control device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0159] The baking control device of the embodiment of the present invention can solve the problem of complicated operation existing in the existing manual baking control. It controls the robot arm to detect and correct the posture of the grabbed food, determines the current baking process based on the real-time status of the food during the baking process, and realizes customized plating output after the baking of the food is completed. It greatly simplifies the baking process of the food, realizes automatic, efficient and beautiful baking control of the food, and meets the user's efficient, convenient and personalized use needs.

[0160] The embodiment of the present invention further provides a baking device, wherein a display screen is arranged outside the baking device, and the display screen is used to display a human-computer interaction interface. The baking device is internally provided with at least one visual camera, a manipulator, a food placement area, a baking tray placement area, and a dinner plate placement area, wherein at least one food is placed in the food placement area. The baking device also includes a controller, see Figure 7 , Figure 7 is a schematic diagram of the structure of the controller provided by an optional embodiment of the present invention, such as Figure 7As shown, the controller includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the controller, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple controllers can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.

[0161] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0162] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0163] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created according to the use of the controller, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the controller via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0164] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0165] The controller also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 7The example of connecting through bus is taken in the following.

[0166] The input device 30 can receive input digital or character information, and generate signal input related to the user settings and function control of the thermal power unit operation control unit, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device may be a touch screen.

[0167] In this embodiment, the baking device is an oven. Specifically, integrating the baking control method of this embodiment on the baking device can make the baking device have a very stable and reliable baking effect, simplify the cumbersome process of manual control, greatly improve the efficiency of baking control, and enhance the user experience.

[0168] A computer-readable storage medium is also provided in an embodiment of the present invention. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor main control chip or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0169] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A baking control method, applied to a baking device, wherein a display screen is provided outside the baking device, and the display screen is used to display a human-computer interaction interface, characterized in that: The baking device is provided with at least one visual camera, a manipulator, a food placement area, a baking tray placement area, and a dinner plate placement area, wherein at least one food is placed in the food placement area, and the method comprises: Control the robot to grab the food from the food placement area in sequence; Performing posture detection on the food ingredients in sequence; If it is detected that the food has a posture deviation, the robot is controlled to make corrections and the corrected food is placed on the baking tray placement area; After all the ingredients on the ingredient placement area are placed on the baking tray placement area, baking is performed and a current ingredient image of the baking tray placement area is obtained; Determining whether baking is completed based on the current food image; If baking is completed, in response to the plating method determined by the user through the human-computer interaction interface, the robot is controlled to place the ingredients in the baking tray placement area into the dinner plate placement area.

2. The baking control method according to claim 1, characterized in that: The performing posture detection on the food includes: The robot arm that grabs the food moves to the location of the first visual camera and obtains a first image of the food; Determining whether there is an angle deviation between the first image and a preset image; If there is an angle deviation, it is determined that the food has a posture deviation; If there is no angle deviation, it is determined that the food has no posture deviation.

3. The baking control method according to claim 2, characterized in that: The controlling the manipulator to perform correction comprises: determining a correction angle based on an angular deviation between the first image and a preset image; The robot is controlled to perform correction based on the correction angle.

4. The baking control method according to claim 1, characterized in that: The determining whether baking is completed based on the current food image includes: Uploading the current food image to a cloud server so that the cloud server can identify the current color of the current food image, correspondingly determining the current food state based on the current color and a preset food state, determining whether baking is completed based on the current food state, and receiving a judgment result fed back by the cloud server, wherein the preset food state includes a correspondence between different colors and corresponding baking states; Alternatively, the current food image is uploaded to a human-computer interaction interface, so that the user can perform a confirmation operation on whether baking is completed through the human-computer interaction interface.

5. The baking control method according to claim 4, characterized in that: Before determining whether baking is completed based on the current food image, the method further includes: Uploading the current food image to a cloud server or a human-computer interaction interface to determine whether a turning operation is required; When a turning operation is required, the robot is controlled to turn the food over.

6. The baking control method according to claim 1, characterized in that: The method of controlling the robot to place the food in the baking tray placement area to the dinner plate placement area in response to the dish placement method determined by the user through the human-computer interaction interface comprises: Acquire current plate data of the baking tray placement area, wherein the plate data includes plate size, shape and position; Determining a target plating method based on the plate data and the plating method; The robot is controlled to place the ingredients in the baking tray placement area into the dinner plate placement area according to the target plating method.

7. The baking control method according to claim 1, characterized in that: After placing the corrected food on the baking tray placement area, the method further includes: Acquire a second image of the food; Detecting whether there is a deviation in the placement position of the food based on the second image; If there is a deviation in the placement position of the food, a deviation angle is determined, and the manipulator is controlled to make corrections based on the deviation angle; If there is no deviation in the placement position of the ingredients, the step of baking and obtaining a current ingredient image of the baking tray placement area is performed after all ingredients on the ingredient placement area are placed on the baking tray placement area.

8. The baking control method according to claim 1, characterized in that: The method further comprises: If no posture deviation of the food is detected, the robot arm is controlled to place the food on the baking tray placement area.

9. The baking control method according to any one of claims 1 to 8, characterized in that: Before the controlling manipulator sequentially grabs food from the food placement area, the method further includes: Check whether all ingredients on the food placement area are in place; If yes, executing the step of controlling the manipulator to grab the food from the food placement area in sequence; If not, the process returns to the step of checking whether all the ingredients on the ingredient placement area are in place.

10. A baking control device, applied to a baking device, wherein a display screen is provided outside the baking device, and the display screen is used to display a human-computer interaction interface, characterized in that: The baking device is provided with at least one visual camera, a manipulator, a food placement area, a baking tray placement area and a dinner plate placement area, wherein at least one food is placed in the food placement area, and the device comprises: The grabbing module is used to control the robot to grab the food from the food placement area in sequence; A detection module, used to detect the postures of the food in turn; A correction module, for controlling the robot to perform correction if a posture deviation of the food is detected, and placing the corrected food on the baking tray placement area; A baking module, used for baking and obtaining a current image of the ingredients in the baking tray placement area after all the ingredients in the ingredient placement area are placed in the baking tray placement area; A judging module, used for judging whether baking is completed based on the current food image; The plating module is used to control the robot to place the ingredients in the baking tray placement area into the dinner plate placement area in response to the plating method determined by the user through the human-computer interaction interface if baking is completed.

11. A baking device, characterized in that: A display screen is arranged on the outside of the baking device, and the display screen is used to display a human-computer interaction interface. At least one visual camera, a manipulator, a food placement area, a baking tray placement area and a dinner plate placement area are arranged inside the baking device, wherein at least one food is placed in the food placement area. The baking device also includes a controller, and the controller includes: a memory and a processor, and the memory and the processor are communicatively connected to each other, and computer instructions are stored in the memory, and the processor executes the baking control method described in any one of claims 1 to 9 by executing the computer instructions.

12. The baking device according to claim 11, characterized in that: The baking equipment is an oven.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the baking control method according to any one of claims 1 to 9.

Citation Information

Patent Citations

  • Food material arrangement system, method, control device, storage medium and equipment

    CN109739139A

  • Target object moving system

    CN111152222A

  • Method and device for identifying food maturity and kitchen appliance

    CN113758879A

  • Full-automatic intelligent skewering equipment

    CN114391763A

  • Automated kitchen system for assisting human worker prepare food

    US20220386807A1

Cited By

  • Intelligent cutting method and device for large white squid slices

    CN119278993A