Oven control method and system and electronic equipment

Abstract

The invention relates to an oven control method and system and electronic equipment, a camera, a depth sensor and a cutting tool are arranged in an oven, food materials are placed at the bottom of the oven, and the method comprises the steps that a two-dimensional image, shot by the camera, of the food materials is obtained; based on a preset image identification model, identifying a food material type corresponding to the two-dimensional image; generating a three-dimensional model of the food material based on a depth map measured by the depth sensor and the two-dimensional image; determining cutting information according to a food material type corresponding to the two-dimensional image and a three-dimensional model of the food material; planning a cutting path of the cutting tool according to the cutting information and the three-dimensional model of the food materials; and based on the cutting path, the cutting tool is controlled to cut the food materials. The path of the cutting tool can be precisely planned, manual intervention is not needed, the automatic cutting process is achieved, the intelligent level of the oven is improved, and the cooking efficiency is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of intelligent kitchen appliances, and in particular to a control method, system and electronic device for an oven. Background Art

[0002] In the cooking process, cutting ingredients with a flower knife is a common technique, especially when preparing to bake or steam ingredients. Cutting with a flower knife can effectively increase the surface area of ​​the ingredients, promote uniform heating and flavor absorption, and thus improve the taste and quality of the dishes.

[0003] Traditional steaming and baking equipment requires manual cutting, which relies on manual experience and skills. Improper operation will lead to uneven cutting of ingredients, affecting the heating and flavor of the ingredients. In addition, manual operation is cumbersome and takes a lot of time and effort. Therefore, there is an urgent need for an intelligent steaming and baking equipment that can automatically cut ingredients to improve cutting accuracy and optimize cooking effects. Summary of the invention

[0004] In order to solve at least one of the above-mentioned technical problems, the present disclosure provides a control method, system and electronic device for an oven.

[0005] According to a first aspect of an embodiment of the present disclosure, a method for controlling an oven is provided, wherein a camera, a depth sensor, and a cutting tool are disposed inside the oven, and food is placed at the bottom of the oven, the method comprising:

[0006] Acquire a two-dimensional image of the food taken by the camera;

[0007] Based on a preset image recognition model, identifying the type of food corresponding to the two-dimensional image;

[0008] generating a three-dimensional model of the food based on the depth map measured by the depth sensor and the two-dimensional image;

[0009] Determining cutting information according to the type of food corresponding to the two-dimensional image and the three-dimensional model of the food, the cutting information including cutting type, cutting width, cutting depth, cutting speed and cutting angle;

[0010] Planning a cutting path of the cutting tool according to the cutting information and the three-dimensional model of the food;

[0011] Based on the cutting path, the cutting tool is controlled to cut the food.

[0012] In one embodiment, determining the cutting information according to the type of food corresponding to the two-dimensional image and the three-dimensional model of the food includes:

[0013] Establishing a cutting information database; the cutting information database includes first cutting data, the first cutting data includes the name of each type of food, multiple cutting types corresponding to each type of food and cutting parameters thereof, the cutting parameters are the cutting width, the cutting speed and the cutting angle corresponding to the target cutting type of the target food type;

[0014] Determine the first cutting data corresponding to the type of food corresponding to the two-dimensional image in the cutting information database;

[0015] Based on the three-dimensional model of the food, extracting thickness information of the food; the thickness information is data information of the three-dimensional model in the vertical direction;

[0016] Based on the thickness information of the food, second cutting data is determined; the second cutting data is the cutting depth.

[0017] In one embodiment, the preset image recognition model is a food image recognition model, and the food image recognition model is trained by the following method:

[0018] Acquire a sample food image, wherein the sample food image includes annotation data for indicating the food type of the sample food;

[0019] Inputting the sample food image into the food image recognition model to obtain a classification prediction result;

[0020] Calculating a loss value based on the difference between the labeled data and the classification prediction result;

[0021] Based on the loss value, the parameters of the food image recognition model are adjusted.

[0022] In one embodiment, planning the cutting path of the cutting tool according to the cutting information and the three-dimensional model of the food includes:

[0023] Based on the three-dimensional model of the food, the cutting type and the cutting width, a plurality of cutting lines are determined; each of the cutting lines is tangent to the upper surface of the three-dimensional model of the food;

[0024] Determining a plurality of cutting areas inside the food material according to the cutting lines, the cutting angles and the cutting depths;

[0025] Based on the multiple cutting areas, the cutting path of the cutting tool is planned.

[0026] In one embodiment, controlling the cutting tool to cut the food based on the cutting path includes:

[0027] determining the coordinates of each cutting line in the plurality of cutting lines;

[0028] According to the coordinates of an initial cutting line among the plurality of cutting lines, moving the cutting tool to the initial cutting line;

[0029] Based on the cutting path and the coordinates of each cutting line, the cutting tool is controlled to cut the food at the cutting speed.

[0030] In one embodiment, the cutting type includes a dicing type and a shredding type;

[0031] The determining of a plurality of cutting lines based on the three-dimensional model of the food material, the cutting type and the cutting width comprises:

[0032] When the cutting type is the block cutting type or the wire cutting type, multiple first cutting lines and multiple second cutting lines are determined; each of the first cutting lines and each of the second cutting lines are tangent to the upper surface of the three-dimensional model, the distance between adjacent first cutting lines is consistent with the cutting width, the distance between adjacent second cutting lines is consistent with the cutting width, and the first cutting line is perpendicular to the second cutting line.

[0033] In one embodiment, the cutting type includes a slicing type;

[0034] The determining of a plurality of cutting lines based on the three-dimensional model of the food material, the cutting type and the cutting width further comprises:

[0035] When the cutting type is the slicing type, a plurality of third cutting lines are determined; each of the third cutting lines is tangent to the upper surface of the three-dimensional model, and a distance between adjacent third cutting lines is consistent with the cutting width.

[0036] In one embodiment, generating the three-dimensional model of the food based on the depth map measured by the depth sensor and the two-dimensional image includes:

[0037] Fusing the depth map and the two-dimensional image to obtain a four-channel image including color information and depth information;

[0038] Based on the depth information, each pixel point in the four-channel image is mapped into a three-dimensional space to generate point cloud data; each point in the point cloud data includes a three-dimensional coordinate and the color information;

[0039] Based on the point cloud data, a three-dimensional model of the food is constructed.

[0040] According to a second aspect of an embodiment of the present disclosure, a control system of an oven is provided, wherein a camera, a depth sensor, and a cutting tool are arranged inside the oven, and food is placed at the bottom of the oven, and the system comprises:

[0041] An acquisition module, used for acquiring a two-dimensional image of the food taken by the camera;

[0042] A recognition module, used to recognize the type of food corresponding to the two-dimensional image based on a preset image recognition model;

[0043] A generating module, configured to generate a three-dimensional model of the food based on the depth map measured by the depth sensor and the two-dimensional image;

[0044] a determination module, configured to determine cutting information according to the type of food corresponding to the two-dimensional image and the three-dimensional model of the food, wherein the cutting information includes a cutting type, a cutting width, a cutting depth, a cutting speed, and a cutting angle;

[0045] A planning module, used for planning a cutting path of the cutting tool according to the cutting information and the three-dimensional model of the food;

[0046] A control module is used to control the cutting tool to cut the food based on the cutting path.

[0047] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor implements the oven control method as described in the first aspect above by executing the instructions stored in the memory.

[0048] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

[0049] The implementation of this disclosure has the following beneficial effects:

[0050] By determining the matching cutting information based on the characteristics of different ingredients, the oven can process a variety of ingredients, increasing the oven's applicability and flexibility; through the combination of cameras and depth sensors, it can automatically identify ingredients and generate their three-dimensional models; based on the three-dimensional model and cutting information of the ingredients, the path of the cutting tool can be accurately planned to ensure the accuracy and consistency of the cutting, improve the quality of food processing, and achieve the intelligence level of the oven without human intervention; the automated cutting process reduces manual preparation time, improves cooking efficiency, provides users with a convenient user experience, and reduces the labor intensity and risks of manual cutting.

[0051] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions and advantages in the embodiments of this specification or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 A schematic flow chart of a method for controlling an oven according to an embodiment of the present disclosure is shown.

[0054] Figure 2 A schematic diagram of a process for determining cutting information according to an embodiment of the present disclosure is shown.

[0055] Figure 3 A flowchart of a method for training a food image recognition model according to an embodiment of the present disclosure is shown.

[0056] Figure 4 A schematic diagram showing a flow chart of planning a cutting path of a cutting tool according to an embodiment of the present disclosure.

[0057] Figure 5 A schematic diagram showing a process of generating a three-dimensional model of food according to an embodiment of the present disclosure.

[0058] Figure 6 A module schematic diagram of a control system of an oven according to an embodiment of the present disclosure is shown.

[0059] Figure 7 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0062] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0063] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0064] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0065] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0066] In order to solve the limitations of traditional ovens, the present invention provides an intelligent oven, wherein a camera, a depth sensor and a cutting tool are arranged inside the oven, and food is placed at the bottom of the oven. Figure 1 A flow chart of a method for controlling an oven according to an embodiment of the present disclosure is shown as follows: Figure 1 , the above method comprises:

[0067] S101, obtaining a two-dimensional image of food taken by a camera.

[0068] The present disclosure does not limit the position of the camera. In one embodiment, the camera is fixedly mounted on the side wall of the oven. A rotatable platform is provided inside the oven, and the food is placed on the rotatable platform. The camera takes images at multiple angles by rotating the platform to ensure that all parts of the food are fully covered and a complete two-dimensional image is obtained.

[0069] S102: Identify the type of food corresponding to the two-dimensional image based on a preset image recognition model.

[0070] The preset image recognition model is the food image recognition model. Figure 3 A flow chart showing a method for training a food image recognition model according to an embodiment of the present disclosure is shown. Figure 3 , the food image recognition model is trained by the following method:

[0071] S301: Acquire a sample food image, where the sample food image includes annotation data for indicating the type of the sample food.

[0072] The sample food images cover a variety of food types, each with different shapes and sizes to ensure the generalization ability of the model. For example, various specific food ingredients in different categories such as vegetables, fruits, meat, fish, etc. The sample food images have a high enough resolution to ensure clear details for easy model recognition, and each food has enough sample food images to ensure the training effect of the model.

[0073] Preprocess the sample food images, including resizing, standardization, and data enhancement. Use image processing libraries such as OpenCV and PI L to adjust the images to a uniform size; perform data enhancement on the sample food images by rotating, scaling, cropping, flipping, etc. to increase the diversity of training samples and improve the robustness of the model; add images with different lighting conditions, backgrounds, and angles to simulate image acquisition in a real oven environment.

[0074] S302: Input the sample food image into the food image recognition model to obtain a classification prediction result.

[0075] The present disclosure does not limit the food image recognition model. In one embodiment, the food image recognition model is AlexNet in the Convolutional Neural Network (CNN) model, and AlexNet includes multiple convolutional layers, pooling layers, and fully connected layers. The convolutional layer is used to extract local features of the image, the pooling layer is used to reduce the dimension and extract the main features, and the fully connected layer is used to map the features to the classification results.

[0076] S303: Calculate the loss value based on the difference between the labeled data and the classification prediction result.

[0077] The input sample food image is forward propagated through the AlexNet model to obtain the classification prediction result of the output layer; the cross entropy loss function is used to calculate the difference between the classification prediction result and the labeled data. The cross entropy loss function can quantify the difference between the probability distribution of the model output and the actual distribution. Through the feedback mechanism of the loss function, the model parameters are continuously adjusted so that the model can be continuously optimized during the training process, improving its performance and robustness in practical applications.

[0078] S304: Adjust the parameters of the food image recognition model based on the loss value.

[0079] Based on the loss value, the gradient of each layer parameter is calculated through the back propagation algorithm; an optimizer such as the Adam optimizer is used to update the model parameters according to the calculated gradient, so that the loss value is gradually reduced. The loss value is calculated by the difference between the labeled data and the classification prediction results, and the model parameters are continuously adjusted so that the model can gradually improve the recognition accuracy, and finally achieve high-precision recognition of different ingredients.

[0080] By training with a large number of sample food images, the model can learn and adapt to the characteristics of various ingredients, improve the model's recognition capabilities in diverse and complex environments, and ensure accurate classification of ingredients of different types and shapes. The food image recognition model with high accuracy and strong adaptability can significantly improve the user experience when using the smart oven and reduce operational inconveniences caused by recognition errors.

[0081] S103: Generate a three-dimensional model of the food based on the depth map and the two-dimensional image measured by the depth sensor.

[0082] The present disclosure does not limit the position of the depth sensor. In one embodiment, the depth sensor is fixed to the upper side wall inside the oven, opposite to or adjacent to the camera, and at the same height as the camera to ensure that the depth information can be obtained from the same viewing angle. The rotatable platform is started to rotate at predetermined angle intervals, for example, 10 degrees each time. At each angle interval, the camera captures a two-dimensional image and the depth sensor obtains a depth map. By controlling the rotation angle of the platform, images and depth information can be captured at different angle intervals to ensure that every part of the food can be accurately recorded.

[0083] Figure 5 A schematic diagram showing a process of generating a three-dimensional model of food according to an embodiment of the present disclosure is shown. Figure 5 , the method for generating a three-dimensional model of food includes:

[0084] S501 , fusing the depth map and the two-dimensional image to obtain a four-channel image including color information and depth information.

[0085] Specifically, a two-dimensional image captured by a camera and a depth map measured by a depth sensor are collected; a calibration plate is used for calibration to ensure that the viewing angles and coordinate systems of the camera and the depth sensor are aligned. In one embodiment, a checkerboard pattern can be used for calibration; the intrinsic and extrinsic parameters of the camera and the depth sensor are calculated. The intrinsic parameters are used to describe the internal optical characteristics of the device, such as focal length and principal point coordinates. The extrinsic parameters are used to describe the relative position and orientation between the devices, such as rotation matrices and translation vectors. In one embodiment, the computer vision algorithm OpenCV can be used to detect the corner points of the calibration plate and calculate the intrinsic and extrinsic parameters; the coordinate system of the depth map is converted to the coordinate system of the camera through the intrinsic and extrinsic parameters. In the embodiment, a four-channel image matrix is ​​initialized to store the fused RGB and depth information, wherein the four-channel information of each pixel includes three RGB color channels and one depth channel; the two-dimensional image and the depth map are traversed pixel by pixel, and the RGB value of the two-dimensional image and the depth value of the corresponding pixel are combined into the four-channel image.

[0086] By fusing the depth map and the 2D image, the resulting four-channel image provides richer data, helping to more accurately capture the shape and structure of the food.

[0087] S502 . Based on the depth information, map each pixel point in the four-channel image into a three-dimensional space to generate point cloud data.

[0088] Specifically, color information and depth information are extracted from the four-channel image; the two-dimensional coordinates of each pixel are converted into three-dimensional coordinates using internal parameters; each pixel is mapped into three-dimensional space through depth information; and the three-dimensional coordinates and color information of each pixel are combined into point cloud data. Each point in the point cloud data includes three-dimensional coordinates and color information, which can construct a high-resolution three-dimensional model.

[0089] S503: construct a three-dimensional model of the food based on the point cloud data.

[0090] Specifically, before constructing a three-dimensional model, the point cloud data needs to be preprocessed. The present disclosure does not limit the preprocessing method, which may include steps such as noise removal and downsampling; surface reconstruction is performed after preprocessing, that is, the preprocessed point cloud data is converted into a three-dimensional mesh model. The present disclosure does not limit the surface reconstruction method, and a convex hull algorithm or a triangular mesh reconstruction method may be used; the generated three-dimensional model is post-processed. The present disclosure does not limit the post-processing method, and mesh simplification and hole filling may be used. Simplifying the mesh can reduce the number of triangles and improve rendering efficiency. Filling holes in the three-dimensional model can improve the integrity of the model, thereby improving the quality of the three-dimensional model. Generating an accurate three-dimensional model helps the oven accurately identify ingredients and provide personalized cooking operations, providing users with a more convenient and intelligent user experience.

[0091] S104: Determine cutting information according to the type of food corresponding to the two-dimensional image and the three-dimensional model of the food.

[0092] The present disclosure does not limit the cutting information. In one embodiment, the cutting information includes cutting type, cutting width, cutting depth, cutting speed and cutting angle.

[0093] In one embodiment, the cutting information may be determined by an automated method. Figure 2 A schematic diagram of a process for determining cutting information according to an embodiment of the present disclosure is shown. Figure 2 , the method to determine the cutting information is as follows:

[0094] S201, establishing a cutting information database.

[0095] The cutting information database includes first cutting data, which includes the name of each type of food, multiple cutting types corresponding to each type of food, and cutting parameters thereof, wherein the cutting parameters are the cutting width, cutting speed, and cutting angle corresponding to the target cutting type of the target food type. The present disclosure does not limit the cutting type, which may be a dicing type, a shredding type, or a slicing type, and each cutting type corresponds to a cutting width, a cutting speed, and a cutting angle applicable to the type. The cutting width refers to the distance between two adjacent cuts made by the cutting tool on the food, the cutting speed refers to the speed at which the cutting tool moves when cutting the food, and the cutting angle refers to the inclination angle of the cutting tool relative to the surface of the food.

[0096] The present disclosure does not limit the method for obtaining the first cutting data, which may be obtained by actually testing the food ingredients or by consulting a professional chef.

[0097] The present disclosure does not limit the specific design of the cutting information database. In one embodiment, the cutting information database includes an ingredient type table, a cutting type table, and a cutting parameter table. Specifically, the ingredient type table includes a primary key IngredientID and an ingredient name IngredientName, the cutting type table includes a primary key CuttingTypeID and a cutting type CuttingTypeName, and the cutting parameter table includes a primary key ParameterID, a foreign key IngredientID, a foreign key CuttingTypeID, a cutting width CuttingWidth, a cutting speed CuttingSpeed, and a cutting angle CuttingAngle. The foreign key IngredientID is used to associate the ingredient type, and the foreign key CuttingTypeID is used to associate the cutting type. By establishing a cutting information database, it is possible to select suitable cutting types and cutting parameters according to different types of ingredients to ensure that each ingredient can achieve the best cutting effect.

[0098] S202: Determine first cutting data corresponding to the type of food corresponding to the two-dimensional image in the cutting information database.

[0099] Specifically, a connection is established between the food image recognition model and the cutting information database; the name of the food type is extracted according to the output of the food image recognition model; and the corresponding first cutting data is queried in the cutting information database according to the name of the extracted food type. In one embodiment, one food may correspond to multiple suitable cutting types, and the user may select a default mode to determine the cutting type. In the default mode, the oven will randomly select a cutting type.

[0100] Through the application of cutting information database, suitable cutting data can be quickly found and matched, which improves the efficiency of cutting path planning, reduces processing time, and improves overall work efficiency.

[0101] S203: extracting thickness information of the food based on the three-dimensional model of the food.

[0102] S204: Determine second cutting data based on the thickness information of the food.

[0103] The thickness information is the data information of the three-dimensional model in the vertical direction, and the second cutting data is the cutting depth. Specifically, the height data of the three-dimensional model is analyzed to determine the thickness range of the food in the vertical direction; based on the thickness range, the second cutting data, i.e., the cutting depth, is determined. The present disclosure does not specifically limit the cutting depth. In one embodiment, the cutting depth can be greater than the minimum value in the thickness range and less than the maximum value in the thickness range. By extracting the thickness information of the food based on the three-dimensional model, a personalized cutting plan can be formulated according to the actual size of the food, which can adapt to food of different shapes and thicknesses, avoid the problem of excessive or insufficient cutting, ensure the cutting accuracy, and improve the quality of food processing.

[0104] In one embodiment, the user can select a manual mode to determine the cutting type. In the manual mode, the oven prompts the user to select the cutting type through an external display screen, and after selecting the cutting type, prompts the user to manually input the cutting information.

[0105] S105: planning a cutting path of the cutting tool according to the cutting information and the three-dimensional model of the food.

[0106] Figure 4 FIG. 1 is a schematic diagram showing a process of planning a cutting path of a cutting tool according to an embodiment of the present disclosure. Figure 4 , the method of planning the cutting path of the cutting tool includes:

[0107] S401, determining a plurality of cutting lines based on the three-dimensional model of the food material, the cutting type and the cutting width.

[0108] Each cutting line is tangent to the upper surface of the three-dimensional model of the food. In one embodiment, when the cutting type is a dicing type or a shredding type, a plurality of first cutting lines and a plurality of second cutting lines are determined, each first cutting line and each second cutting line are tangent to the upper surface of the three-dimensional model, the distance between adjacent first cutting lines is consistent with the cutting width, the distance between adjacent second cutting lines is consistent with the cutting width, and the first cutting line is perpendicular to the second cutting line. By respectively determining the first cutting line and the second cutting line of the dicing type or the shredding type, and ensuring that the first cutting line and the second cutting line are perpendicular, the shape of the food after cutting can be accurately controlled.

[0109] In one embodiment, when the cutting type is a slicing type, a plurality of third cutting lines are determined, each third cutting line is tangent to the upper surface of the three-dimensional model, and the distance between adjacent third cutting lines is consistent with the cutting width.

[0110] Determine the spacing between adjacent cutting lines based on the cutting width, so that the width of each portion of food is consistent, ensuring uniform cutting; clarify multiple cutting lines to reduce invalid operations during the cutting process, improve cutting efficiency, and quickly plan and execute cutting operations, shortening processing time.

[0111] Based on the 3D model of the food and the cutting type, the cutting line tangent to the surface of the food is determined to ensure the accuracy of the cutting path, make the cutting process more precise and reduce errors; according to different cutting types and cutting widths, multiple cutting lines can be flexibly determined to adapt to various food and cutting needs, providing a variety of cutting solutions.

[0112] S402: Determine multiple cutting areas inside the food according to the cutting lines, cutting angles and cutting depths.

[0113] Specifically, each cutting line is extended downward at a set cutting angle. The present disclosure does not limit the cutting angle, which can be a right angle or an oblique angle, which is specifically set according to the cutting requirements; in combination with the cutting depth determined above, the cutting line is extended inside the food to the set cutting depth to ensure that the cutting does not exceed the thickness range of the food; based on the extended cutting line, multiple cutting areas, i.e., cutting surfaces, are determined. By combining the cutting line, cutting angle, and cutting depth, multiple cutting areas inside the food are accurately determined, making the path planning of the cutting tool more reasonable and efficient, reducing unnecessary tool movement, and improving cutting efficiency.

[0114] S403: Planning a cutting path of a cutting tool based on multiple cutting areas.

[0115] Specifically, the initial cutting position of the cutting tool is determined, which is usually the cutting area where the edge cutting line is located; all cutting areas are sorted, and the cutting order is determined according to a preset rule.

[0116] The present disclosure does not limit the preset rules. In one embodiment, when the cutting type is a dicing type or a shredding type, the preset rule is to cut in sequence according to the arrangement order of the cutting areas where the first cutting line is located, and then to cut in sequence according to the arrangement order of the cutting areas where the second cutting line is located. In one embodiment, when the cutting type is the slicing type, the preset rule is to cut in sequence according to the arrangement order of the cutting areas where the third cutting line is located.

[0117] S106: Based on the cutting path, control the cutting tool to cut the food.

[0118] Specifically, the coordinates of each cutting line among the multiple cutting lines are determined; according to the coordinates of the initial cutting line among the multiple cutting lines, the cutting tool is moved to the initial cutting line; based on the cutting path and the coordinates of each cutting line, the cutting tool is controlled to cut the food at the above-set cutting speed. According to the coordinates of the cutting line, the cutting tool is controlled to accurately position and move to each cutting line, thereby achieving accurate cutting of the food.

[0119] By determining the coordinates of each cutting line and moving the cutting tool according to these coordinates, precise cutting of ingredients can be achieved to ensure the accuracy and consistency of cutting; by controlling the cutting tool to cut ingredients at a set cutting speed, the efficiency of cutting can be improved, saving time and labor costs; based on the pre-set cutting path and speed, the stability and reliability of cutting can be improved, reducing the possibility of errors.

[0120] Figure 6 A schematic diagram of a control system of an oven according to an embodiment of the present disclosure is shown. A camera, a depth sensor and a cutting tool are arranged inside the oven, and food is placed at the bottom of the oven. Figure 6 , the system includes:

[0121] An acquisition module, used for acquiring a two-dimensional image of the food taken by the camera;

[0122] A recognition module, used to recognize the type of food corresponding to the two-dimensional image based on a preset image recognition model;

[0123] A generating module, configured to generate a three-dimensional model of the food based on the depth map measured by the depth sensor and the two-dimensional image;

[0124] a determination module, configured to determine cutting information according to the type of food corresponding to the two-dimensional image and the three-dimensional model of the food, wherein the cutting information includes a cutting type, a cutting width, a cutting depth, a cutting speed, and a cutting angle;

[0125] A planning module, used for planning a cutting path of the cutting tool according to the cutting information and the three-dimensional model of the food;

[0126] A control module is used to control the cutting tool to cut the food based on the cutting path.

[0127] In some embodiments, the functions or modules included in the system provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0128] The embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to perform the method.

[0129] The electronic device may be provided as a terminal, a server, or a device in other forms.

[0130] Figure 7 1 shows a block diagram of an electronic device according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server. Figure 7, the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0131] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like.

[0132] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0133] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0134] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0135] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the above-mentioned module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the standard function in the square box can also occur in a sequence different from the standard in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a special hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0136] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for controlling an oven, characterized in that: The oven is provided with a camera, a depth sensor and a cutting tool, and food is placed at the bottom of the oven. The method comprises: Acquire a two-dimensional image of the food taken by the camera; Based on a preset image recognition model, identifying the type of food corresponding to the two-dimensional image; generating a three-dimensional model of the food based on the depth map measured by the depth sensor and the two-dimensional image; Determining cutting information according to the type of food corresponding to the two-dimensional image and the three-dimensional model of the food, the cutting information including cutting type, cutting width, cutting depth, cutting speed and cutting angle; Planning a cutting path of the cutting tool according to the cutting information and the three-dimensional model of the food; Based on the cutting path, the cutting tool is controlled to cut the food.

2. The method for controlling an oven according to claim 1, characterized in that: The determining of cutting information according to the type of food corresponding to the two-dimensional image and the three-dimensional model of the food includes: Establishing a cutting information database; the cutting information database includes first cutting data, the first cutting data includes the name of each type of food, multiple cutting types corresponding to each type of food and cutting parameters thereof, the cutting parameters are the cutting width, the cutting speed and the cutting angle corresponding to the target cutting type of the target food type; Determine the first cutting data corresponding to the type of food corresponding to the two-dimensional image in the cutting information database; Based on the three-dimensional model of the food, extracting thickness information of the food; the thickness information is data information of the three-dimensional model in the vertical direction; Based on the thickness information of the food, second cutting data is determined; the second cutting data is the cutting depth.

3. The method for controlling an oven according to claim 1, characterized in that: The preset image recognition model is a food image recognition model, and the food image recognition model is trained by the following method: Acquire a sample food image, wherein the sample food image includes annotation data for indicating the food type of the sample food; Inputting the sample food image into the food image recognition model to obtain a classification prediction result; Calculating a loss value based on the difference between the labeled data and the classification prediction result; Based on the loss value, the parameters of the food image recognition model are adjusted.

4. The method for controlling an oven according to claim 1 or 2, characterized in that: The planning of the cutting path of the cutting tool according to the cutting information and the three-dimensional model of the food comprises: Based on the three-dimensional model of the food, the cutting type and the cutting width, a plurality of cutting lines are determined; each of the cutting lines is tangent to the upper surface of the three-dimensional model of the food; Determining a plurality of cutting areas inside the food material according to the cutting lines, the cutting angles and the cutting depths; Based on the multiple cutting areas, the cutting path of the cutting tool is planned.

5. The method for controlling an oven according to claim 4, characterized in that: The controlling the cutting tool to cut the food based on the cutting path comprises: determining the coordinates of each cutting line in the plurality of cutting lines; According to the coordinates of an initial cutting line among the plurality of cutting lines, moving the cutting tool to the initial cutting line; Based on the cutting path and the coordinates of each cutting line, the cutting tool is controlled to cut the food at the cutting speed.

6. The method for controlling an oven according to claim 4, characterized in that: The cutting types include dicing and shredding; The determining of a plurality of cutting lines based on the three-dimensional model of the food material, the cutting type and the cutting width comprises: When the cutting type is the block cutting type or the wire cutting type, multiple first cutting lines and multiple second cutting lines are determined; each of the first cutting lines and each of the second cutting lines are tangent to the upper surface of the three-dimensional model, the distance between adjacent first cutting lines is consistent with the cutting width, the distance between adjacent second cutting lines is consistent with the cutting width, and the first cutting line is perpendicular to the second cutting line.

7. The method for controlling an oven according to claim 4, characterized in that: The cutting type includes a slicing type; The determining of a plurality of cutting lines based on the three-dimensional model of the food material, the cutting type and the cutting width further comprises: When the cutting type is the slicing type, a plurality of third cutting lines are determined; each of the third cutting lines is tangent to the upper surface of the three-dimensional model, and a distance between adjacent third cutting lines is consistent with the cutting width.

8. The method for controlling an oven according to claim 1, characterized in that: The step of generating the three-dimensional model of the food based on the depth map measured by the depth sensor and the two-dimensional image comprises: Fusing the depth map and the two-dimensional image to obtain a four-channel image including color information and depth information; Based on the depth information, each pixel point in the four-channel image is mapped into a three-dimensional space to generate point cloud data; each point in the point cloud data includes a three-dimensional coordinate and the color information; Based on the point cloud data, a three-dimensional model of the food is constructed.

9. A control system for an oven, characterized in that: The oven is provided with a camera, a depth sensor and a cutting tool, and food is placed at the bottom of the oven. The system includes: An acquisition module, used for acquiring a two-dimensional image of the food taken by the camera; A recognition module, used to recognize the type of food corresponding to the two-dimensional image based on a preset image recognition model; A generating module, configured to generate a three-dimensional model of the food based on the depth map measured by the depth sensor and the two-dimensional image; a determination module, configured to determine cutting information according to the type of food corresponding to the two-dimensional image and the three-dimensional model of the food, wherein the cutting information includes a cutting type, a cutting width, a cutting depth, a cutting speed, and a cutting angle; A planning module, used for planning a cutting path of the cutting tool according to the cutting information and the three-dimensional model of the food; A control module is used to control the cutting tool to cut the food based on the cutting path.

10. An electronic device, characterized in that: It includes at least one processor and a memory that is communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the at least one processor implements the control method of the oven as described in any one of claims 1 to 8 by executing the instructions stored in the memory.

Images