A baking control method, apparatus, baking equipment, and storage medium

Abstract

This invention relates to the field of baking control technology, and discloses a baking control method, apparatus, baking equipment, and storage medium. Applied to a baking equipment, it has an external display screen for displaying a human-machine interface, and internally includes at least one vision camera, a robotic arm, a food placement area, a baking tray placement area, and a dinner plate placement area. At least one food item is placed in the food placement area. The method includes: controlling the robotic arm to pick up the food item from the food placement area and performing posture detection; if a posture deviation is detected, controlling the robotic arm to correct it and placing the corrected food item in the baking tray placement area; after all the food items in the food placement area are placed in the baking tray placement area, baking is performed, and the current food image in the baking tray placement area is acquired to determine if baking is complete. In response to the plating method determined by the user through the human-machine interface, the robotic arm is controlled to place the food items in the baking tray placement area into the dinner plate placement area. This invention simplifies the operation process, thereby improving baking control efficiency.

Description

Technical Field

[0001] This invention relates to the field of baking control technology, specifically to a baking control method, apparatus, baking equipment, and storage medium. Background Technology

[0002] Currently, baking equipment, such as ovens, has gradually entered people's daily lives, becoming a common tool for food preparation and cooking. However, during the baking process, users still need to manually set relevant parameters, which greatly increases the difficulty and complexity of operation. In addition, existing ovens have a simple structure and do not have a place to put the tray after it has been removed. After baking, it is inconvenient for users to remove the tray containing food because the tray is still hot from inside the oven and has nowhere to be placed, which seriously affects the user experience.

[0003] Based on the aforementioned problems, related technologies have improved the interior of ovens by adding tray placement areas, but they have neglected the different layout requirements of various areas within the oven, such as food placement areas. Furthermore, controlling each area within the oven still requires manual assistance to achieve the corresponding baking operations, resulting in high control costs and complex operation, making it difficult to meet the usage needs of this type of baking equipment. Summary of the Invention

[0004] In view of this, the present invention provides a baking control method, device, baking equipment and storage medium to solve the problem that the manual control operation of existing baking equipment is cumbersome and cannot meet the user's needs for efficient and convenient use.

[0005] In a first aspect, the present invention provides a baking control method applied to a baking device. The baking device has an external display screen for displaying a human-machine interface. The baking device has at least one vision camera, a robotic arm, a food placement area, a baking tray placement area, and a plate placement area internally. The food placement area contains at least one food item. The method includes:

[0006] The robotic arm is controlled to sequentially grab ingredients from the ingredient placement area;

[0007] The food ingredients are sequentially subjected to posture detection;

[0008] If a deviation in the posture of the food is detected, the robotic arm is controlled to correct it and the corrected food is placed in the baking tray area.

[0009] After all the ingredients in the ingredient placement area are placed into the baking tray placement area, the baking process is performed and the current ingredient image in the baking tray placement area is obtained.

[0010] Determine whether baking is complete based on the current image of the ingredients;

[0011] If baking is complete, the robotic arm will move the food from the baking tray area to the serving tray area in response to the plating method determined by the user through the human-computer interaction interface.

[0012] The baking control method provided by this invention controls the robotic arm to detect and correct the posture of the food being picked up from the food placement area, thereby adjusting the placement position of the food on the baking tray. It determines whether baking is complete by real-time detection of the current food image during the baking process, and after baking, uses the robotic arm to arrange the corresponding food on the tray according to the user's desired presentation. This simplifies the food baking process and enables automatic, efficient, and aesthetically pleasing control of the baking process, greatly satisfying users' needs for efficient and convenient customized use of baking equipment.

[0013] In one alternative implementation, the food ingredient's posture is detected, including:

[0014] The robotic arm, after grasping the food, is moved to the location of the first vision camera and acquires the first image of the food.

[0015] Determine whether there is an angular deviation between the first image and the preset image;

[0016] If there is an angular deviation, it is determined that the food has a posture deviation;

[0017] If there is no angular deviation, then it is determined that the food does not have any posture deviation.

[0018] This invention designs a posture detection method for a robotic arm to grasp food. By judging the angular deviation between the first image corresponding to the food and a set standard image, it can determine whether there is a posture deviation. This can ensure that the corresponding position of the food is in the standard orientation, thereby helping to enhance the subsequent baking efficiency of the food and greatly improving the user experience.

[0019] In one alternative implementation, controlling the robotic arm to perform correction includes:

[0020] The correction angle is determined based on the angular deviation between the first image and the preset image;

[0021] The control robot arm performs corrections based on the adjustment angle.

[0022] This invention corrects the position of ingredients by controlling a robotic arm based on the angular deviation between the first image corresponding to the ingredients and a set standard image, ensuring that the corresponding position of the ingredients is in a standard orientation, which helps to accurately place the ingredients in the baking tray area.

[0023] In one optional implementation, determining whether baking is complete based on the current image of the food ingredients includes:

[0024] The current food image is uploaded to the cloud server so that the cloud server can identify the current color of the current food image, determine the current food status based on the current color and the preset food status, determine whether baking is complete based on the current food status, and receive the judgment result from the cloud server. The preset food status includes the correspondence between different colors and corresponding baking statuses.

[0025] Alternatively, the current food image can be uploaded to the human-computer interaction interface, allowing users to confirm whether baking is complete through the interface.

[0026] This invention achieves automatic determination of the food baking process by uploading the current food image to a cloud server, and semi-automatic determination of the food baking process by uploading it to a human-computer interaction interface. Based on the design of the above two food baking process determination processes, the accuracy of food baking process determination is improved to a certain extent, which helps to ensure the accuracy of determining the completion of food baking.

[0027] In one optional implementation, before determining whether baking is complete based on the current food image, the baking control method further includes:

[0028] Upload the current food image to a cloud server or human-computer interaction interface to determine whether a flipping operation is needed;

[0029] When it is necessary to flip the food, control the robotic arm to flip the food.

[0030] This invention takes into account the conventional operation of actual food baking process, and uses two designs to determine the baking process of food based on the current food image, namely cloud server or human-computer interaction interface. This not only improves the food baking operation process, but also increases user satisfaction with the baking equipment.

[0031] In one alternative implementation, in response to the plating method determined by the user through a human-computer interaction interface, the robotic arm is controlled to place the food from the baking tray area to the serving tray area, including:

[0032] Retrieve the current plate data in the baking tray placement area. The plate data includes the plate size, shape, and position.

[0033] Determine the target plating method based on plate data and plating patterns;

[0034] The robotic arm is controlled to place the ingredients from the baking tray area into the serving plate area according to the target plating method.

[0035] This invention takes into account users' personalized plating needs. By customizing the food plating method and combining the current plate data in the baking equipment, it adaptively realizes the custom plating after the food is baked, which can greatly enhance the user experience.

[0036] In one alternative implementation, after the calibrated ingredients are placed in the baking tray placement area, the baking control method further includes:

[0037] Obtain a second image of the ingredients;

[0038] The second image is used to detect whether there is a deviation in the placement of the ingredients;

[0039] If there is a deviation in the placement of the ingredients, the deviation angle is determined, and the robotic arm is controlled to make corrections based on the deviation angle.

[0040] If there is no deviation in the placement of the ingredients, then the steps are performed: after all the ingredients in the ingredient placement area are placed in the baking tray placement area, baking is carried out and the current ingredient image in the baking tray placement area is obtained.

[0041] The invention also includes a detection and correction mechanism to check for any deviations in the placement of all ingredients in the food placement area. This ensures that all ingredients are placed in the set standard positions, which can also speed up the subsequent baking process to some extent.

[0042] In one optional implementation, the baking control method further includes:

[0043] If no deviation in the food's posture is detected, the robotic arm is controlled to place the food into the baking tray area.

[0044] In one alternative implementation, before controlling the robotic arm to sequentially grab ingredients from the ingredient placement area, the baking control method further includes:

[0045] Check that all ingredients in the ingredient placement area have been properly loaded;

[0046] If so, execute the steps of controlling the robotic arm to sequentially grab ingredients from the ingredient placement area;

[0047] If not, return to the step of checking whether all ingredients in the ingredient placement area have been properly placed.

[0048] Before the robotic arm grasps the food ingredients, this invention designs a detection system to check whether the food ingredients are properly loaded. This ensures the normal operation of the subsequent processes of grasping, correcting, baking, and plating the food ingredients, thus achieving automatic and efficient control of food baking.

[0049] Secondly, the present invention provides a baking control device applied to a baking equipment. The baking equipment has an external display screen for displaying a human-machine interface. The baking equipment has at least one vision camera, a robotic arm, a food placement area, a baking tray placement area, and a plate placement area internally. The food placement area contains at least one food item. The device includes:

[0050] The gripping module is used to control the robotic arm to sequentially grip food items from the food placement area;

[0051] The detection module is used to sequentially detect the posture of the ingredients;

[0052] The calibration module is used to control the robotic arm to correct any posture deviations detected in the food and then place the corrected food into the baking tray area.

[0053] The baking module is used to bake all the ingredients in the food placement area after they have been placed in the baking tray placement area and to obtain the current food image in the baking tray placement area.

[0054] The judgment module is used to determine whether baking is complete based on the current image of the food ingredients;

[0055] The plating module is used to control the robotic arm to place the food from the baking tray area to the serving tray area in response to the plating method determined by the user through the human-computer interaction interface once baking is complete.

[0056] The baking control device provided by this invention combines baking equipment, a robotic arm, and a vision system. By controlling the robotic arm to detect and correct the posture of the food being picked up from the food placement area, it determines whether baking is complete based on real-time detection of the current food image during the baking process. After baking, the robotic arm completes the corresponding food plating according to the user's own plating requirements, which greatly simplifies the food baking process and realizes automatic, efficient, and aesthetically pleasing baking control, meeting the user's needs for efficient, convenient, and personalized use.

[0057] Thirdly, the present invention provides a baking device, wherein the baking device is provided with a display screen on the outside for displaying a human-machine interface, and the baking device is provided with at least one vision camera, a robotic arm, an ingredient placement area, a baking tray placement area and a plate placement area inside the baking device, wherein at least one ingredient is placed in the ingredient placement area, and the baking device further includes a controller, which includes a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the baking control method of the first aspect or any corresponding embodiment described above.

[0058] In one alternative implementation, the baking device is an oven.

[0059] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the baking control method of the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0060] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0061] Figure 1 This is a schematic flowchart of the baking control method according to an embodiment of the present invention;

[0062] Figure 2 This is a schematic flowchart of another baking control method according to an embodiment of the present invention;

[0063] Figure 3 This is a schematic diagram of the baking equipment;

[0064] Figure 4 This is a schematic diagram of the robotic arm.

[0065] Figure 5 This is a flowchart illustrating the process of food ingredient grabbing and alignment.

[0066] Figure 6 This is a structural block diagram of the baking control device according to an embodiment of the present invention;

[0067] Figure 7 This is a schematic diagram of the baking controller according to an embodiment of the present invention. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0069] According to an embodiment of the present invention, a baking control method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0070] This embodiment provides a baking control method applied to a baking device. The baking device has an external display screen for displaying a human-machine interface. The baking device has at least one vision camera, a robotic arm, a food placement area, a baking tray placement area, and a plate placement area inside. The food placement area contains at least one food item. Figure 1 This is a schematic flowchart of a baking control method according to an embodiment of the present invention, as shown below. Figure 1 As shown, the process includes the following steps:

[0071] Step S101: Control the robotic arm to grab ingredients sequentially from the ingredient placement area.

[0072] It should be noted that the specific type and control method of the robotic arm in this embodiment are not limited, but determined based on the adaptability of the actual project. For example, the robotic arm is a three-degree-of-freedom robotic arm that can perform three independent rotational or translational movements in three-dimensional space. This type of robotic arm is usually composed of a device connected by three rotary joints. Each joint can make the robotic arm move in one axis, such as making the robotic arm rotate around its axis, thereby realizing the grasping, moving and manipulating of objects. This is only an example for illustration.

[0073] Step S102: Perform posture detection on the ingredients sequentially.

[0074] In this embodiment, the specific content of pose detection is not limited and can be adaptively adjusted based on actual needs. For example, using a pre-trained deep learning algorithm model to implement pose detection for food ingredients is merely an example.

[0075] In step S103, if a posture deviation of the food is detected, the robotic arm is controlled to correct it and the corrected food is placed in the baking tray placement area.

[0076] In this embodiment, if the food has a posture deviation, it means that the food being grasped by the robotic arm is not in the correct position. Placing it directly in the baking tray area may increase the subsequent baking time. Therefore, it is necessary to correct its posture relative to the set standard orientation to ensure the efficiency of the subsequent food baking process.

[0077] Step S104: After all the ingredients in the ingredient placement area are placed into the baking tray placement area, bake them and obtain the current ingredient image in the baking tray placement area.

[0078] It should be noted that the method of acquiring the current food image in this embodiment is not limited and can be adapted based on actual needs. For example, a vision camera installed inside the oven to monitor the baking tray placement area can be used to achieve real-time acquisition of the current food image; this is only an example.

[0079] Step S105: Determine whether baking is complete based on the current image of the ingredients.

[0080] In this embodiment, the specific criteria for determining whether baking is complete based on the current food image are not limited and are adaptively set according to actual needs. For example, it can be determined by detecting color changes in the food image. For instance, a bright red piece of meat gradually changes color under the heat of charcoal, and its surface develops an enticing sheen; this is a clear sign that the food is cooked. It should be noted that different foods will exhibit different color changes; for example, meat will change from red to golden yellow, while vegetables will change from a raw color to a more vibrant one. Alternatively, it can be determined by changes in the food's shape, texture, and gloss. For example, when a cake is baked, in addition to judging by the golden yellow or light brown color of its surface, a detailed analysis can be performed based on its corresponding edge texture, degree of expansion, and gloss. This is only an example for illustration.

[0081] In step S106, if baking is complete, the robotic arm is controlled to place the food from the baking tray area into the serving tray area in response to the plating method determined by the user through the human-computer interaction interface.

[0082] The baking control method of this invention adjusts the placement of ingredients on the baking tray by controlling the robotic arm to detect and correct the posture of the robotic arm as it picks up ingredients from the food placement area. It determines whether baking is complete by real-time detection of the current ingredient image during the baking process, and then uses the robotic arm to arrange the ingredients according to the user's desired presentation. This simplifies the baking process, enables automatic, efficient, and aesthetically pleasing control of the baking process, and greatly satisfies users' needs for efficient and convenient customized use of baking equipment.

[0083] This embodiment provides a baking control method applied to a baking device. The baking device has an external display screen for displaying a human-machine interface. The baking device has at least one vision camera, a robotic arm, a food placement area, a baking tray placement area, and a plate placement area inside. The food placement area contains at least one food item. Figure 2 This is a schematic flowchart of another baking control method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0084] It should be noted that before the upgrade control process in this embodiment begins, it needs to detect whether the food loading in the food placement area has been completed. Specifically, before controlling the robotic arm to sequentially grab food from the food placement area, the baking control method in this embodiment further includes:

[0085] Step a1: Check whether all ingredients in the ingredient placement area have been properly placed.

[0086] In practical applications, if a user is feeding food into the food storage area while a robotic arm is simultaneously grasping the food, friction may occur between the robotic arm and the user, potentially causing injury. Therefore, in this embodiment, the robotic arm grasps the food only after the feeding process is complete, ensuring user safety and preventing potential dangers.

[0087] It should be noted that the detection method for whether the ingredients have been properly placed in this embodiment is not limited and can be adapted based on actual needs. For example, it can be determined by visually detecting whether a user has placed the ingredients within a certain period of time; or by detecting the opening and closing status of baking equipment, such as an oven door. When the oven door lock is in the closed state, it can be confirmed that all ingredients have been placed in place. This is only an example for illustration.

[0088] If step a2 is correct, execute step S201, which involves controlling the robotic arm to sequentially grab ingredients from the ingredient placement area.

[0089] If not, proceed to step a3. If not, return to step a1 to check if all ingredients passing through the ingredient placement area have been properly placed.

[0090] In this embodiment of the invention, before the robotic arm grasps the food ingredients, a detection system is designed to check whether the food ingredients are properly loaded. This ensures the normal operation of the subsequent processes of grasping, correcting, baking, and plating the food ingredients, thus achieving automatic and efficient control of food ingredient baking.

[0091] Step S201: Control the robotic arm to sequentially grab ingredients from the ingredient placement area. For details, please refer to [link to details]. Figure 1 Step S101 of the illustrated embodiment will not be described again here.

[0092] Step S202: Perform posture detection on the ingredients sequentially.

[0093] Specifically, the posture detection of the food ingredients in step S202 above includes:

[0094] Step S2021: Move the robotic arm that has grasped the food to the location of the first vision camera and acquire the first image of the food.

[0095] In this embodiment, the installation positions and specific models of the robotic arm and the first vision camera can be adaptively adjusted based on the actual project requirements.

[0096] Step S2022: Determine whether there is an angular deviation between the first image and the preset image.

[0097] In this embodiment, the specific content of the preset image is not limited and can be adjusted adaptively based on the type of food and the actual project requirements. For example, for the first image where the food is a potato, the preset image is set to include a potato and place it in a positive orientation position, which is only used as an example.

[0098] Step S2023: If there is an angle deviation, it is determined that the food has a posture deviation.

[0099] Step S2024: If there is no angular deviation, then it is determined that the food has no posture deviation.

[0100] In this embodiment of the invention, a posture detection method for the robotic arm to grasp food is designed. The angular deviation between the first image corresponding to the food and the set standard image is used to determine whether there is a posture deviation. This ensures that the corresponding position of the food is in the standard orientation, which helps to enhance the subsequent baking efficiency of the food and greatly improves the user experience.

[0101] In step S203, if a posture deviation of the food is detected, the robotic arm is controlled to correct it and the corrected food is placed in the baking tray placement area.

[0102] Specifically, the step S203 above, which involves controlling the robotic arm to perform calibration, includes:

[0103] Step b1: Determine the correction angle based on the angular deviation between the first image and the preset image.

[0104] In this embodiment, assuming the ingredient is a potato, the corresponding preset image is the potato in a set standard orientation. The angle between the corresponding first image and the preset image is obtained through a deep learning algorithm model, and then the corresponding correction angle is determined. This is only an example for illustration.

[0105] Step b2: Control the robotic arm to perform correction based on the correction angle.

[0106] In this embodiment of the invention, by controlling the robotic arm to correct the position of the food based on the angular deviation between the first image corresponding to the food and the set standard image, it is possible to ensure that the corresponding position of the food is in the standard orientation, which helps to accurately place the food in the baking tray placement area.

[0107] In this embodiment, if no posture deviation of the food is detected, the robotic arm is controlled to place the food into the baking tray placement area. Specifically, placing food without posture deviation directly into the baking tray placement area can speed up the baking process and further improve the efficiency of baking control.

[0108] It should be noted that, in order to improve baking efficiency in this embodiment, after placing the ingredients in the baking tray placement area, it is necessary to detect the deviation of the ingredient placement position again. Specifically, after placing the corrected ingredients in the baking tray placement area in step S203 above, the baking control method of this embodiment further includes:

[0109] Step c1: Obtain the second image of the ingredients.

[0110] In this embodiment, the same vision camera installed during the posture detection process for food handling can be used to acquire images of the food corresponding to the baking tray placement area; alternatively, a separate vision camera can be set up in the baking tray placement area for real-time image acquisition of the food. It should be noted that the number of vision cameras used in this embodiment is adaptively adjusted based on the actual camera model and project requirements. For example, if the camera is wide-angle and the current project cost is low, one vision camera can be installed to complete different image acquisition tasks; this is only an example.

[0111] Step c2: Detect whether there is a deviation in the placement of the ingredients based on the second image.

[0112] Step c3: If there is a deviation in the placement of the ingredients, determine the deviation angle and control the robotic arm to make corrections based on the deviation angle.

[0113] It should be noted that the detection and correction of the deviation in the placement of ingredients in this embodiment can refer to the relevant content above, and will not be repeated here.

[0114] Step c4: If there is no deviation in the placement of the ingredients, then perform the step of placing all the ingredients in the ingredient placement area into the baking tray placement area, baking, and obtaining the current ingredient image in the baking tray placement area.

[0115] In this embodiment of the invention, the design detects and corrects for any deviations in the placement positions of all ingredients in the food placement area, ensuring that all ingredients are in the set standard placement positions, which greatly speeds up the subsequent food baking process.

[0116] Step S204: After all ingredients in the ingredient placement area have been placed into the baking tray placement area, baking is performed and the current ingredient image in the baking tray placement area is acquired. For details, please refer to [link to details]. Figure 1 Step S104 of the illustrated embodiment will not be described again here.

[0117] Step S205: Determine whether baking is complete based on the current image of the ingredients.

[0118] In this embodiment, a fully automatic method for determining the baking process based on a visual algorithm and a semi-automatic method for determining the completion of baking based on human experience are designed. Specifically, step S205 includes:

[0119] Step d1: Upload the current food image to the cloud server so that the cloud server can identify the current color of the current food image, determine the current food status based on the current color and the preset food status, determine whether baking is complete based on the current food status, and receive the judgment result from the cloud server. The preset food status includes the correspondence between different colors and corresponding baking statuses.

[0120] It should be noted that the specific content of the preset food ingredient state in this embodiment is not limited, but is set adaptively based on actual needs.

[0121] Step d2: Upload the current food image to the human-computer interaction interface so that the user can confirm whether baking is complete through the human-computer interaction interface.

[0122] In this embodiment of the invention, the baking process of the food is automatically determined by uploading the current food image to the cloud server, and semi-automated by uploading it to the human-computer interaction interface for manual determination. Based on the design of the above two food baking process determination processes, the accuracy of food baking process determination is improved to a certain extent, which helps to ensure the accuracy of the determination of food baking completion.

[0123] It should be noted that, in this embodiment, the ingredients can also be stirred during the baking process. Specifically, before determining whether baking is complete based on the current image of the ingredients, the baking control method in this embodiment further includes:

[0124] The current food image is uploaded to a cloud server or human-computer interaction interface to determine whether a flipping operation is needed; when a flipping operation is needed, the robotic arm is controlled to flip the food.

[0125] In this embodiment of the invention, taking into account the conventional operation of the actual food baking process, two designs based on the current food image are used to determine the food baking process, namely cloud server or human-computer interaction interface. This not only improves the food baking operation process, but also increases user satisfaction with the baking equipment.

[0126] In step S206, if baking is complete, the robotic arm is controlled to place the food from the baking tray area into the serving tray area in response to the plating method determined by the user through the human-computer interaction interface.

[0127] Specifically, in step S206 above, in response to the plating method determined by the user through the human-computer interaction interface, the robotic arm is controlled to place the ingredients from the baking tray area to the dinner plate area, including:

[0128] Step S2061: Obtain the current plate data of the baking tray placement area. The plate data includes the plate size, shape and position.

[0129] In this embodiment, the specific method of obtaining the plate data is not limited, but is based on conventional data acquisition methods known in the art.

[0130] Step S2062: Determine the target plating method based on the plate data and plating method.

[0131] In this embodiment, the target plating method is the optimal food placement method. Specifically, this embodiment takes into account the actual size, shape, and position of the plate, and combines this with the set plating method to enable the robotic arm to achieve the optimal plating placement of the currently baked food.

[0132] Step S2063: Control the robotic arm to place the ingredients from the baking tray placement area into the dinner plate placement area according to the target plating method.

[0133] In this embodiment of the invention, taking into account the user's personalized plating needs, a customized food plating method is used, combined with the current plate data in the baking equipment, to adaptively realize customized plating after the food is baked, which can greatly enhance the user experience.

[0134] In one specific embodiment, a control scheme for a fully automatic oven is proposed to reduce manual operation steps and improve convenience. See also... Figure 3 Based on the original oven, a food placement area, a tray placement area, a robotic arm, and a vision camera have been added. Combined with a human-machine interface, this enables fully automatic control of the oven. Specifically, the robotic arm can be customized through the human-machine interface to simulate and replace human hands, achieving precise placement of food. Specifically, the food placement area and tray placement area have been expanded on one side of the oven's interior. The expanded inner cavity can be opened. The robotic arm is mounted on an X-axis slide, with a Z-axis telescopic cylinder mounted above it for gripping. The specific installation location is as follows... Figure 3 As shown.

[0135] In this embodiment, a robotic arm with three degrees of freedom in the XYZ axes is used. Figure 4 This is a schematic diagram of the robotic arm. It should be noted that the Y-axis, where the Y-axis motor is located, is the angle correction axis. Based on a high-precision motion control system combining vision and servo control, the robotic arm can accurately and quickly pick up food. Vision camera 1 can check whether the food is square when grasped, and if there is an angular deviation, it can correct the rotation of the robotic arm by a corresponding angle to straighten the food's current posture. For example, when vision camera 1 detects that a chicken wing is slightly off-center, the robotic arm's rotation angle can be adjusted to grasp the wing in the center before placing it on the baking tray.

[0136] In one specific embodiment, after the ingredients are placed on the baking tray, the vision camera 2 provides feedback on the ingredients. If a deviation is detected during placement, the robotic arm will be moved back to correct the angle. Specifically, the robotic arm places the ingredients on the baking tray, and the vision camera 2 scans the tray to determine if the ingredients are misaligned. The vision system sends the detected posture and position data to the vision controller. After data processing, the controller sends data to the robotic arm's motor, which in turn drives the robotic arm to correct the angle. Through the vision system and the robotic arm control system, the ingredients can be sequentially placed on the baking tray.

[0137] In one specific embodiment, the robotic arm first performs posture verification based on the vision camera 1, that is, it corrects whether the food grasped in the food placement area is square. (See reference...) Figure 5 As shown; then place the corrected ingredients on the baking tray and check whether there is any deviation in the position of the ingredients on the baking tray. Through the above two methods of ingredient position correction, it can be ensured that all the ingredients on the baking tray are in a square position.

[0138] In one specific embodiment, during the baking stage, the visual camera 2 detects color changes in the food to control the robotic arm to grasp and flip the food. Simultaneously, the visual camera 2 provides real-time feedback on the food's baking status to determine whether baking is complete. Specifically, the visual camera 2 can transmit food status images in real-time to the cloud or a user interface (i.e., a human-computer interaction interface). Based on the color changes of different foods, it compares these changes with color changes in the menu library to determine whether a flipping instruction or a baking end instruction has been executed. Alternatively, the user can manually control the food to flip or end baking by transmitting and displaying real-time food images through the human-computer interaction interface.

[0139] In this embodiment, the user can independently select the plating method through the human-computer interaction interface. The vision camera scans the plate and, based on the recorded size and shape of the plate, achieves precise positioning of the plate, thereby controlling the robotic arm to accurately pick up the baked food and place it onto the plate. Combined with the set exquisite plating method, the corresponding plating is achieved.

[0140] In summary, the baking control method of this invention combines a precision robotic arm with a vision system, and is equipped with two vision cameras to adjust the position and posture of the ingredients and detect and provide feedback on the target position. This achieves all-round, automated, rapid grasping, correction, and positioning of ingredients. Data transmission conversion and intelligent control are realized through a human-computer interaction interface and an intelligent controller, thereby truly freeing up hands and completing automated and personalized baking and placement of ingredients.

[0141] This embodiment also provides a baking control device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, a "module" can be a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0142] This invention provides a baking control device applied to baking equipment. The baking equipment has an external display screen for displaying a human-machine interface. The baking equipment internally includes at least one vision camera, a robotic arm, a food placement area, a baking tray placement area, and a plate placement area. The food placement area contains at least one food item, such as... Figure 6 As shown, the device includes:

[0143] The gripping module 601 is used to control the robotic arm to grip food items sequentially from the food placement area.

[0144] The detection module 602 is used to sequentially detect the posture of the ingredients.

[0145] The calibration module 603 is used to control the robotic arm to perform calibration if a posture deviation of the food is detected, and then place the calibrated food into the baking tray placement area.

[0146] The baking module 604 is used to bake and acquire the current image of the ingredients in the baking tray after all the ingredients in the ingredient placement area have been placed in the baking tray placement area.

[0147] The judgment module 605 is used to determine whether baking is complete based on the current image of the food ingredients.

[0148] The plating module 606 is used to control the robotic arm to place the food from the baking tray area to the serving tray area in response to the plating method determined by the user through the human-machine interface when baking is complete.

[0149] In some optional implementations, the detection module 602 includes: a first detection submodule, a second detection submodule, a third detection submodule, and a fourth detection submodule; wherein, the first detection submodule is used to move the robotic arm after grasping the food to the location of the first vision camera and acquire a first image of the food; the second detection submodule is used to determine whether there is an angular deviation between the first image and a preset image; the third detection submodule is used to determine that the food has a posture deviation if there is an angular deviation; and the fourth detection submodule is used to determine that the food has no posture deviation if there is no angular deviation.

[0150] In some optional implementations, the correction module 603 includes: a first correction submodule and a second correction submodule; wherein, the first correction submodule is used to determine a correction angle based on the angular deviation between the first image and the preset image; and the second correction submodule is used to control the robot to perform correction based on the correction angle.

[0151] In some optional embodiments, the correction module 603 further includes: a first correction submodule, a second correction submodule, a third correction submodule, and a fourth correction submodule; wherein, the first correction submodule is used to acquire a second image of the food ingredients; the second correction submodule is used to detect whether there is a deviation in the placement position of the food ingredients based on the second image; the third correction submodule is used to determine the deviation angle if there is a deviation in the placement position of the food ingredients, and control the robot arm to correct based on the deviation angle; the fourth correction submodule is used to perform the step of placing all the food ingredients in the food ingredient placement area into the baking tray placement area, baking, and acquiring the current food ingredient image in the baking tray placement area if there is no deviation in the placement position of the food ingredients.

[0152] In some alternative implementations, the correction module 603 further includes a control submodule for controlling the robotic arm to place the food into the baking tray placement area if no posture deviation of the food is detected.

[0153] In some optional implementations, the judgment module 605 includes: a first judgment submodule and a second judgment submodule; wherein, the first judgment submodule is used to upload the current food image to the cloud server so that the cloud server can identify the current color of the current food image, determine the current food state based on the current color and the preset food state, determine whether baking is complete based on the current food state, and receive the judgment result fed back by the cloud server, wherein the preset food state includes the correspondence between different colors and corresponding baking states; the second judgment submodule is used to upload the current food image to the human-computer interaction interface so that the user can confirm whether baking is complete through the human-computer interaction interface.

[0154] In some optional implementations, the judgment module 605 further includes: a third judgment submodule, used to upload the current food image to a cloud server or human-computer interaction interface to determine whether a flipping operation is required; when a flipping operation is required, controlling the robotic arm to flip the food.

[0155] In some optional implementations, the plating module 606 includes: a first plating module, a second plating module, and a third plating module; wherein, the first plating module is used to acquire the current plate data of the baking pan placement area, the plate data including the plate size, shape, and position; the second plating module is used to determine the target plating method based on the plate data and the plating method; and the third plating module is used to control the robotic arm to place the ingredients in the baking pan placement area into the plate placement area according to the target plating method.

[0156] In some optional embodiments, the device further includes: a determination module for detecting whether all ingredients in the ingredient placement area are in place; if so, executing the step of controlling the robotic arm to sequentially grab ingredients from the ingredient placement area; if not, returning to the step of detecting whether all ingredients in the ingredient placement area are in place.

[0157] Further functional descriptions of the above modules are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0158] In this embodiment, the baking control device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0159] The baking control device of this invention can solve the problem of cumbersome operation in existing manual baking control. By controlling the robotic arm to perform posture detection and correction on the food being grasped, the current baking progress is determined based on the real-time status of the food during the baking process. After the food is baked, a customized plating output is achieved, which greatly simplifies the baking process and realizes automatic, efficient and beautiful baking control of food, meeting the user's needs for efficient, convenient and personalized use.

[0160] This invention also provides a baking device. The baking device has an external display screen for displaying a human-machine interface. The baking device internally includes at least one vision camera, a robotic arm, a food placement area, a baking tray placement area, and a plate placement area. The food placement area contains at least one food item. The baking device also includes a controller. (See attached image.) Figure 7 , Figure 7 This is a schematic diagram of the structure of the controller provided in an optional embodiment of the present invention, as shown below. Figure 7As shown, the controller includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise as required. The processors can process instructions executed within the controller, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple controllers can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 7 Take a processor 10 as an example.

[0161] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0162] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0163] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the controller. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the controller via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0164] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0165] The controller also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means. Figure 7Taking the example of a connection between China and Israel via a bus.

[0166] Input device 30 can receive input digital or character information, and generate signal inputs related to user settings and function control of the thermal power unit's operation control unit, such as a touch screen, keypad, mouse, trackpad, touchpad, indicator, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touch screen.

[0167] In this embodiment, the baking device is an oven. Specifically, integrating the baking control method described above into the baking device enables the device to achieve a very stable and reliable baking effect, simplifies the cumbersome manual control process, greatly improves the efficiency of baking control, and enhances the user experience.

[0168] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor main control chips, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the methods shown in the above embodiments are implemented.

[0169] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A baking control method, applied to a baking device, wherein a display screen is externally disposed on the baking device, the display screen being used to display a human-machine interface, characterized in that... The baking equipment is internally equipped with at least one vision camera, a robotic arm, an ingredient placement area, a baking tray placement area, and a serving tray placement area, wherein at least one ingredient is placed in the ingredient placement area, and the method includes: The robotic arm is controlled to sequentially grab ingredients from the ingredient placement area; The food ingredients are sequentially subjected to posture detection; If a posture deviation is detected in the food, the robotic arm is controlled to correct it and the corrected food is placed in the baking tray area. After all the ingredients in the ingredient placement area are placed into the baking tray placement area, baking is performed and the current ingredient image in the baking tray placement area is obtained; Determine whether baking is complete based on the current food image; If baking is complete, the robotic arm will be controlled to place the food from the baking tray area into the serving tray area in response to the plating method determined by the user through the human-computer interaction interface. The process of detecting the posture of the food ingredient includes: moving the robotic arm that has grasped the food ingredient to the location of the first vision camera and acquiring a first image of the food ingredient; determining whether there is an angular deviation between the first image and a preset image; if there is an angular deviation, determining that the food ingredient has a posture deviation; if there is no angular deviation, determining that the food ingredient has no posture deviation. After placing the corrected ingredients into the baking tray placement area, the method further includes: acquiring a second image of the ingredients; detecting whether there is a deviation in the placement position of the ingredients based on the second image; if there is a deviation in the placement position of the ingredients, determining the deviation angle and controlling the robotic arm to correct based on the deviation angle; if there is no deviation in the placement position of the ingredients, then performing the steps of baking after all the ingredients in the ingredient placement area have been placed into the baking tray placement area and acquiring the current ingredient image of the baking tray placement area.

2. The baking control method according to claim 1, characterized in that, The control of the robotic arm to perform calibration includes: The correction angle is determined based on the angular deviation between the first image and the preset image; The robotic arm is controlled to make corrections based on the corrected angle.

3. The baking control method according to claim 1, characterized in that, The step of determining whether baking is complete based on the current food image includes: The current food image is uploaded to the cloud server so that the cloud server can identify the current color of the current food image, determine the current food state based on the current color and the preset food state, determine whether baking is complete based on the current food state, and receive the judgment result fed back by the cloud server. The preset food state includes the correspondence between different colors and corresponding baking states. Alternatively, the image of the current ingredients can be uploaded to a human-computer interaction interface, allowing the user to confirm whether baking is complete through the interface.

4. The baking control method according to claim 3, characterized in that, Before determining whether baking is complete based on the current food image, the method further includes: The current food image is uploaded to a cloud server or human-computer interaction interface to determine whether a flipping operation is needed. When it is necessary to flip the food, the robotic arm is controlled to flip the food.

5. The baking control method according to claim 1, characterized in that, The step of controlling the robotic arm to place the ingredients from the baking tray area to the serving tray area in response to the plating method determined by the user through the human-computer interaction interface includes: Obtain the current plate data of the baking tray placement area, the plate data including plate size, shape and position; The target plating method is determined based on the plate data and the plating method. The robotic arm is controlled to place the ingredients from the baking tray placement area into the serving plate placement area according to the target plating method.

6. The baking control method according to claim 1, characterized in that, The method further includes: If no posture deviation is detected in the food, the robotic arm is controlled to place the food into the baking tray placement area.

7. The baking control method according to any one of claims 1 to 6, characterized in that, Before the robotic arm sequentially picks up ingredients from the ingredient placement area, the method further includes: Check that all ingredients in the ingredient placement area have been properly loaded; If so, execute the steps of controlling the robotic arm to sequentially grab ingredients from the ingredient placement area; If not, return to the step of checking whether all ingredients in the ingredient placement area have been properly placed.

8. A baking control device, applied to baking equipment, wherein a display screen is externally disposed on the baking equipment, the display screen being used to display a human-machine interface, characterized in that, The baking equipment is internally equipped with at least one vision camera, a robotic arm, an ingredient placement area, a baking tray placement area, and a serving tray placement area, wherein at least one ingredient is placed in the ingredient placement area. The device includes: The gripping module is used to control the robotic arm to sequentially grip food items from the food placement area; The detection module is used to sequentially detect the posture of the food ingredients; The correction module is used to control the robotic arm to correct the posture of the food if a posture deviation is detected, and then place the corrected food into the baking tray placement area. The baking module is used to bake all the ingredients in the ingredient placement area after they have been placed in the baking tray placement area and to acquire the current ingredient image in the baking tray placement area. The judgment module is used to determine whether baking is complete based on the current food image; The plating module is used to control the robotic arm to place the food from the baking tray area to the serving tray area in response to the plating method determined by the user through the human-computer interaction interface when baking is complete. The process of detecting the posture of the food ingredient includes: moving the robotic arm that has grasped the food ingredient to the location of the first vision camera and acquiring a first image of the food ingredient; determining whether there is an angular deviation between the first image and a preset image; if there is an angular deviation, determining that the food ingredient has a posture deviation; if there is no angular deviation, determining that the food ingredient has no posture deviation. After placing the corrected ingredients into the baking tray placement area, the method further includes: acquiring a second image of the ingredients; detecting whether there is a deviation in the placement position of the ingredients based on the second image; if there is a deviation in the placement position of the ingredients, determining the deviation angle and controlling the robotic arm to correct based on the deviation angle; if there is no deviation in the placement position of the ingredients, then performing the steps of baking after all the ingredients in the ingredient placement area have been placed into the baking tray placement area and acquiring the current ingredient image of the baking tray placement area.

9. A baking apparatus, characterized in that, The baking equipment is externally equipped with a display screen for displaying a human-machine interface. The baking equipment is internally equipped with at least one vision camera, a robotic arm, an ingredient placement area, a baking tray placement area, and a plate placement area. The ingredient placement area contains at least one ingredient. The baking equipment also includes a controller, which includes a memory and a processor. The memory and the processor are communicatively connected. The memory stores computer instructions, and the processor executes the computer instructions to perform the baking control method according to any one of claims 1 to 7.

10. The baking apparatus according to claim 9, characterized in that, The baking equipment is an oven.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the baking control method according to any one of claims 1 to 7.

Images