Cooking equipment control methods, devices, cooking control equipment, and cooking equipment

Abstract

This application relates to a cooking equipment control method, apparatus, cooking control device, and cooking equipment. The method includes: acquiring image data of the interior of the cooking equipment and the position of the ingredients within the cooking equipment when ingredients are placed inside; the image data includes image information of the ingredients; determining the area and quantity of the ingredients based on the image data and their position within the cooking equipment; determining cooking parameters for the cooking equipment based on the position, quantity, and area of ​​the ingredients; and controlling the cooking equipment to operate according to the cooking parameters. This method can improve the cooking effect of the cooking equipment.

Description

Technical Field

[0001] This application relates to the field of cooking control technology, and in particular to a cooking equipment control method, apparatus, cooking control equipment, cooking equipment, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] With the continuous development of cooking technology, the cooking modes of cooking equipment are also gradually increasing. Users can flexibly adjust the cooking mode of cooking equipment according to parameters such as the type and quantity of ingredients. Therefore, how to select the most suitable cooking parameters for different ingredients has become one of the problems to be solved.

[0003] Traditional cooking equipment typically has 2-3 cooking time and temperature gradients. Users can choose the most suitable cooking mode from several available cooking modes based on parameters such as the type and weight of the ingredients.

[0004] However, the cooking modes of the aforementioned traditional cooking equipment are limited and have a small scope. The existing cooking modes may not be able to meet the cooking needs of the ingredients, resulting in poor cooking results. Summary of the Invention

[0005] Therefore, it is necessary to provide a cooking equipment control method, apparatus, cooking control device, cooking equipment, computer equipment, computer-readable storage medium, and computer program product that can improve cooking results, in order to address the above-mentioned technical problems.

[0006] In a first aspect, this application provides a method for controlling a cooking device, comprising:

[0007] When ingredients are placed in a cooking device, image data of the interior of the cooking device and the position of the ingredients within the cooking device are acquired; the image data includes image information of the ingredients.

[0008] Based on the image data and the position of the ingredients in the cooking equipment, determine the area and number of ingredients;

[0009] The cooking parameters of the cooking equipment are determined based on the position of the ingredients in the cooking equipment, the number of ingredients, and the area of ​​the ingredients.

[0010] Control the cooking equipment to operate at the cooking parameters.

[0011] Secondly, this application also provides a cooking equipment control device, comprising:

[0012] The data acquisition module is used to acquire image data inside the cooking device and the position of the food in the cooking device when the food is placed in the cooking device; the image data includes image information of the food.

[0013] The ingredient analysis module is used to determine the area and number of ingredients based on the image data and the position of the ingredients in the cooking equipment.

[0014] A cooking parameter determination module is used to determine the cooking parameters of the cooking equipment based on the position of the ingredients in the cooking equipment, the number of ingredients, and the area of ​​the ingredients.

[0015] A cooking module is used to control the cooking equipment to operate with the cooking parameters.

[0016] Thirdly, this application also provides a cooking control device, which is used to control a cooking device according to the steps in the cooking device control method embodiments described above.

[0017] Fourthly, this application also provides a cooking device, including a cooking control device, a pressure sensing device and an image acquisition device connected to the cooking control device, wherein the pressure sensing device is used to acquire the position of the food in the cooking device and send it to the cooking control device, and the image acquisition device is used to acquire image data inside the cooking device and send it to the cooking control device.

[0018] Fifthly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described embodiments of the cooking equipment control method.

[0019] Sixthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described embodiments of the cooking equipment control method.

[0020] In a seventh aspect, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps in the above-described embodiments of the cooking equipment control method.

[0021] The aforementioned cooking equipment control method, apparatus, cooking control device, cooking equipment, computer-readable storage medium, and computer program product differ from traditional solutions that have limited cooking modes and struggle to meet the diverse cooking needs of various ingredients. This solution acquires image data containing image information of the ingredients within the cooking equipment, along with the ingredients' positions within the equipment, to determine the area and quantity of the ingredients. Based on the ingredients' positions, quantities, and areas, the cooking parameters of the cooking equipment are determined, and finally, the cooking equipment is controlled to operate according to these parameters. In other words, this solution is no longer limited to existing cooking modes but rather sets cooking parameters based on the actual state of the ingredients within the equipment, broadening the adjustable range of cooking parameters. This allows for a more tailored approach to the cooking needs of the ingredients, improving the adaptability and accuracy of the cooking parameters, and ultimately enhancing the cooking results. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is an application environment diagram of a cooking equipment control method in one embodiment;

[0024] Figure 2 This is a flowchart illustrating a cooking equipment control method in one embodiment;

[0025] Figure 3 This is a schematic diagram of the cooking equipment in one embodiment;

[0026] Figure 4 This is a flowchart illustrating the cooking equipment control method in another embodiment;

[0027] Figure 5 This is a schematic diagram of image data in one embodiment;

[0028] Figure 6 This is a schematic diagram of the food ingredient location partitions in one embodiment;

[0029] Figure 7 This is a flowchart illustrating the cooking equipment control method in yet another embodiment;

[0030] Figure 8 This is a flowchart illustrating the steps for determining cooking parameters in one embodiment;

[0031] Figure 9This is a flowchart illustrating the cooking parameter determination step in another embodiment;

[0032] Figure 10 This is a flowchart illustrating the steps for acquiring image data in one embodiment;

[0033] Figure 11 This is a flowchart illustrating the steps for acquiring image data in another embodiment;

[0034] Figure 12 This is a structural block diagram of a cooking equipment control device in one embodiment;

[0035] Figure 13 This is a structural block diagram of the cooking device in one embodiment;

[0036] Figure 14 This is a structural block diagram of the cooking device in another embodiment;

[0037] Figure 15 This is an internal structural diagram of a computer device in one embodiment.

[0038] Figure labeling: 1. Steam oven; 2. Control panel; 3. Inner cavity; 3-1-1. Ingredients; 3-3-1. Shelf; 3-2-5. Pressure sensor; 4. Camera. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0040] The cooking equipment control method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, the data acquisition device 102 communicates with the cooking control device 104 via a network. The data storage system can store the data that the cooking control device 104 needs to process. The data storage system can be integrated into the cooking control device 104 or placed in the cloud or on another network server.

[0041] Specifically, when ingredients are placed into the cooking equipment, a data acquisition device 102 deployed inside the cooking equipment can collect image data inside the cooking equipment and the position of the ingredients in the cooking equipment. The image data contains image information of the ingredients and is sent to the cooking control device 104. The cooking control device 104 determines the area and number of ingredients based on the image data and the position of the ingredients in the cooking equipment. Then, based on the position, number, and area of ​​the ingredients in the cooking equipment, it determines the cooking parameters of the cooking equipment and finally controls the cooking equipment to work according to the cooking parameters.

[0042] The data acquisition device 102 can be, but is not limited to, various image acquisition devices, pressure sensing devices, etc., and the cooking control device 104 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0043] In one exemplary embodiment, such as Figure 2 As shown, a method for controlling a cooking device is provided, which is applied to... Figure 1 Taking the cooking control equipment in the middle as an example, the following steps are included:

[0044] S100: When ingredients are placed in the cooking equipment, image data inside the cooking equipment and the position of the ingredients in the cooking equipment are acquired.

[0045] The acquired image data includes image information of the ingredients. This image data can be obtained by an image acquisition device installed inside the cooking equipment, such as a high-definition camera, capturing images of the inside of the equipment. The position of the ingredients within the cooking equipment can be acquired by a pressure sensor. When the ingredients are placed in the cooking equipment, the pressure sensor can sense the pressure generated by the ingredients, thereby locating their position within the equipment.

[0046] Taking a steam oven as an example of cooking equipment, such as Figure 3 As shown, the steam oven 1 is equipped with an operation screen 2, an inner cavity 3, and a camera 4. The camera 4 can be positioned in the center of the oven door or other locations, or adaptably placed in other parts of the oven. When the user places food into the inner cavity 3, the camera 4 is triggered to capture image data of the oven's interior. Alternatively, the user can perform touch operations on the operation screen 2 to set initial cooking parameters, triggering the camera 4 to capture image data of the oven's interior. Furthermore, the steam oven typically has multiple storage layers, numbered from bottom to top as 3-3-1. Each layer has a pressure sensor, such as pressure sensor 3-2-5, indicating a pressure sensor located on the fifth layer from the bottom. Due to the weight of the food, the protruding part of pressure sensor 3-2-5 descends, sending feedback to the cooking control device, thus quickly locating the layer where the food is located. Within the same storage layer, food can also be placed in different positions, for example... Figure 3In the fifth shelf, the first ingredient from right to left is designated as ingredient 3-1-1. Thus, through the camera 4 and pressure sensors 3-2-5 within the steam oven 1, image data of the interior of the steam oven 1 and the position of the ingredients within it can be collected. It should be noted that if multiple shelves inside the steam oven 1 are filled with ingredients, since camera 4 can only capture image data from the highest shelf, subsequent data analysis will be based on the placement of ingredients on the highest shelf. However, multiple cameras 4 can be installed in the steam oven 1 to collect image data from each shelf, allowing the placement of ingredients on different shelves to be considered when setting cooking parameters.

[0047] S200 determines the area and number of ingredients based on image data and the position of the ingredients in the cooking equipment.

[0048] The number of ingredients significantly impacts cooking parameters. More ingredients mean more energy is needed to complete the cooking process, potentially requiring higher temperatures or longer cooking times. For example, 3 300-gram sweet potatoes require 40 minutes to bake, while 8 require 50 minutes. Similarly, the surface area of ​​the ingredients also affects cooking parameters. For instance, 280-gram (smaller surface area) sweet potatoes require 35 minutes to bake, while 500-gram (larger surface area) sweet potatoes require 50 minutes.

[0049] Specifically, the acquired image data may contain distortion and noise, requiring preprocessing to reduce these issues and improve accuracy. Then, image processing algorithms, such as edge detection, can be used to extract the food outlines from the image data, allowing for the estimation of food area. It's important to note that for cooking appliances with multiple internal shelves (such as steam ovens), the distance between the camera and each shelf varies. Therefore, the position of the food within the appliance must be considered when estimating its area to ensure accuracy. The number of food items can be determined directly from the image data using specific algorithms. For example, a target recognition neural network model can be used to extract feature information from the image data, including but not limited to shape, outline, color, texture, and relative position. This feature information can then be used to determine which parts of the image data belong to the same food item and which belong to different food items, thus counting the number of food items.

[0050] S300 determines the cooking parameters of the cooking equipment based on the position, number, and area of ​​the ingredients within the equipment.

[0051] Cooking parameters include, but are not limited to, cooking temperature and cooking time. Taking a steam oven as an example, the position of the ingredients within the cooking equipment includes which shelf the ingredients are on and their distribution within the same shelf. Accordingly, the cooking temperature and cooking time will differ depending on the shelf on which the ingredients are located. Even within the same shelf, different quantities and areas of ingredients will require different cooking temperatures and times.

[0052] For example, steam ovens typically have an upper heating element and a lower heating element. If the user selects a mode that enables the upper heating element and disables the lower heating element, then for the same food, the closer the food is to the upper heating element, the lower the corresponding cooking temperature should be. The larger the food area and the more food there is, the higher the corresponding cooking temperature and the longer the cooking time should be.

[0053] S400 controls the cooking equipment to operate according to cooking parameters.

[0054] Specifically, once the cooking parameters are determined based on factors such as the position of the ingredients in the cooking equipment, the number of ingredients, and the area of ​​the ingredients, the cooking control equipment (or controller, control system, etc.) in the cooking equipment will convert these parameters into specific control instructions. The cooking control equipment will send corresponding control instructions to each working module of the cooking equipment to control each module to work according to the cooking parameters. For example, it controls the heating module to heat the ingredients according to the calculated cooking temperature, and the heating time is the calculated cooking time.

[0055] The aforementioned cooking equipment control method differs from traditional solutions that limit cooking modes and struggle to meet the diverse cooking needs of various ingredients. This solution acquires image data containing image information of the ingredients within the cooking equipment, along with the ingredients' positions within the equipment, to determine the area and quantity of the ingredients. Based on these factors, the cooking parameters of the equipment are determined, and the equipment is then controlled to operate according to these parameters. In other words, this solution is no longer limited to existing cooking modes but rather sets cooking parameters based on the actual state of the ingredients within the equipment. This broadens the adjustable range of cooking parameters, allowing for a more tailored approach to the cooking needs of different ingredients, improving the adaptability and accuracy of the parameters, and ultimately enhancing the cooking results.

[0056] In one embodiment, such as Figure 4 As shown, S200 includes:

[0057] S210, Based on the image data, determine the number of ingredients and the area of ​​each ingredient in the image data.

[0058] S220 determines the area ratio coefficient based on the position of the ingredients in the cooking equipment.

[0059] S230, determine the area of ​​the food ingredient based on its area and area ratio coefficient in the image data.

[0060] Taking a steam oven as an example, when ingredients are placed on the same shelf, the size of the same ingredient will vary depending on its distance from the camera. Ingredients placed closer to the camera will appear larger than normal, while those further away will appear smaller. When ingredients are placed on different shelves, the same ingredient may appear smaller on lower shelves and larger on higher shelves. Therefore, it's necessary to determine the area ratio based on the ingredient's position within the cooking equipment to calculate its true area. The number of ingredients determined by processing image data is usually quite accurate and generally doesn't have significant deviations.

[0061] Specifically, features such as shape, outline, color, texture, and relative position of ingredients can be extracted from image data. These features can then be used to determine which parts of the image data belong to the same individual ingredient and which belong to different individual ingredients, thus counting the number of ingredients. Furthermore, when calculating the area of ​​ingredients, edge detection algorithms can be used to identify the ingredient outlines, thereby determining the area of ​​the ingredient in the image data. Then, the area of ​​the ingredient in the image data needs to be converted into the actual area of ​​the ingredient. This process involves an area scaling factor, which is related to the position of the ingredient in the cooking equipment. For example, decision trees, random forests, and other methods can be used to perform feature analysis and ingredient classification in the image data. Then, linear regression, multinomial regression, and other methods can be used to establish the relationship between the area of ​​the ingredient and the feature information of the ingredient in the image data, thereby determining the area of ​​the ingredient. Alternatively, the area of ​​the ingredient can be calculated based on deep learning model algorithms, such as using a CNN model for regression tasks. The input image data is used, and the output is the predicted area of ​​the ingredient. A hybrid model can also be used, combining traditional features and deep learning features to construct a hybrid model to predict the area of ​​the ingredient; no limitation is made here. It's important to note that when using deep learning algorithms to calculate food area, model training is necessary first. Training data can be obtained by collecting multiple images of the same food from different angles, under varying lighting conditions, and against different backgrounds. This reduces the impact of occlusion and shadows and helps separate the foreground and background, thus improving robustness. Then, each image is labeled, including marking the food's boundaries, generating a pixel-level mask, and labeling the actual food area (which can be achieved by first calculating the food area in the image data and then converting it to the actual food area using an area scaling factor). This process yields the training data.

[0062] For example, the area ratio factor may include a vertical area ratio factor and a horizontal area ratio factor, so as to Figure 3 Taking the steam oven 1 as an example, since the steam oven 1 has five storage layers, and the distance between each storage layer and the camera 4 is not the same, a vertical area ratio coefficient can be set for each storage layer to characterize the influence of the vertical distribution of food on the food area. Furthermore, for the same storage layer, the image data collected by the camera 4 is as follows... Figure 5 As shown, ingredients 1-6 are distributed in different areas of the shelf. The shelf can be pre-divided into areas 1-9, as shown in the area distribution diagram. Figure 6 As shown, the distance from each area to camera 4 is different. Therefore, a horizontal area ratio coefficient can be set for each area to characterize the influence of the horizontal distribution of ingredients on the area of ​​the ingredients. It should be noted that the aforementioned vertical and horizontal area ratio coefficients can be pre-calibrated for the cooking equipment (a steam oven in this embodiment), for example, based on historical image data and the actual area of ​​each ingredient in the historical image data. After determining the area ratio coefficients and the area of ​​the ingredients in the image data, the area of ​​the ingredients can be further determined, for example, by multiplying the area of ​​the ingredients in the image data by the vertical and horizontal area ratio coefficients to obtain the area of ​​the ingredients.

[0063] In this embodiment, considering that the distance between the food and the camera will result in different ratios between the food area in the image data and the actual food area, different area ratio coefficients are set for different layers and different positions in the same layer. This allows for the calculation of more accurate food areas, thereby improving the accuracy of cooking parameters and enhancing the cooking effect.

[0064] In one embodiment, such as Figure 7 As shown, S220 includes:

[0065] S221, determine the vertical area ratio coefficient based on the vertical position of the ingredients in the cooking equipment.

[0066] S222, determine the horizontal area ratio coefficient based on the horizontal position of the ingredients in the cooking equipment.

[0067] The position of the ingredients in the cooking equipment includes the horizontal position of the ingredients in the horizontal direction and the vertical position of the ingredients in the vertical direction, and the area ratio coefficient includes the vertical area ratio coefficient and the horizontal area ratio coefficient.

[0068] Specifically, also with Figure 3Taking the steam oven 1 as an example, the following description continues from the previous embodiment. The steam oven 1 has 5 storage layers, each of which is divided into 9 areas, which can be represented by (horizontal distribution, vertical distribution), numbered 1-9. For example, these 9 areas are (top, left), (top, middle), (top, right), (middle, left), (middle, middle), (middle, right), (bottom, left), (bottom, middle), and (bottom, right). Correspondingly, the area of ​​the food in the image data can be divided into the areas of each region in the image data, denoted as... For example, the area ratio coefficients in steam oven 1 can be as shown in Table 1:

[0069] Table 1 Area Proportion Coefficient Table

[0070]

[0071] As shown in Table 1, the vertical area ratio coefficient P represents the magnification or reduction ratio of the same food item placed on different shelves in the steam oven 1, and the horizontal area ratio coefficient k represents the magnification or reduction ratio of the same food item placed on the same shelf in the steam oven 1 within the nine zones. The areas of the food items in the nine zones are as follows: (It can be predicted by the CNN model in the above embodiments).

[0072] Furthermore, after determining the area ratio coefficient and the area of ​​the ingredients in the nine areas, the area of ​​all ingredients in steam oven 1 can be further determined. For example, with Figure 5 Taking the distribution of ingredients in the text as an example, we can see that Figure 5 There are 6 ingredients, all located on the 3rd shelf and numbered 1-6. The area occupied by each ingredient is different, as shown in Table 2.

[0073] Table 2. Distribution of Ingredients

[0074]

[0075] It should be noted that the area (3,1) where ingredient 1 is located means that ingredient 1 is in area 1 of the third shelf. The areas (3,2) and (3,5) where ingredient 2 is located mean that ingredient 2 is in areas 2 and 5 of the third shelf. The same applies to other ingredients.

[0076] Then the area of ​​ingredients It can be determined based on equation (1):

[0077] (1)

[0078] If n is the number of ingredients, then the average area of ​​the ingredients is... It can be determined based on equation (2):

[0079] (2)

[0080] In this embodiment, the vertical and horizontal distribution of ingredients are taken into account, and different vertical area ratio coefficients P and horizontal area ratio coefficients k are set. This allows for a more accurate determination of the ingredient area, leading to more precise cooking parameters that better meet the cooking requirements of the ingredients.

[0081] In one embodiment, such as Figure 8 As shown, S300 includes:

[0082] S310 determines the cooking temperature coefficient of the ingredients based on their position in the cooking equipment.

[0083] S320 determines the cooking time coefficient for ingredients based on the number and area of ​​the ingredients.

[0084] S330 adjusts the preset initial cooking temperature of the cooking equipment according to the cooking temperature coefficient to determine the cooking temperature of the cooking equipment.

[0085] S340, adjusts the preset initial cooking time of the cooking equipment according to the cooking time coefficient, and determines the cooking time of the cooking equipment.

[0086] Following the above embodiment, the steam oven 1 has two working modes: steaming and baking. The user can first select the working mode of the steam oven 1 through the operation screen 2 on the steam oven 1. When the user selects the baking mode, the initial cooking temperature corresponding to the baking mode has been pre-written into the cooking control device (or controller, control system). At this point, the initial cooking temperature needs to be adjusted according to the position of the ingredients in the cooking equipment. The adjustment can be made by multiplying by a cooking temperature coefficient m. The relationship between the cooking temperature coefficient and the position of the food in the cooking equipment is shown in Table 3.

[0087] Table 3. Relationship between cooking temperature coefficient and the position of ingredients in cooking equipment.

[0088]

[0089] It should be noted that the impact of different areas of the same layer on the cooking temperature is relatively small. Therefore, only the influence of the food being placed on different shelves and the heating method in the baking mode on the cooking temperature coefficient m is considered. Specifically, when the steam oven 1 uses both upper and lower heating elements, the food inside the steam oven 1 is heated relatively evenly, so the food can be heated at the initial cooking temperature. When the steam oven uses only the upper heating element, the food closer to the upper heating element is heated more significantly. In order to meet the heating requirements of the food, the cooking temperature coefficient m corresponding to the shelf closer to the upper heating element is smaller.

[0090] When the steam oven 1 is in steam mode, the preset initial cooking temperature is... Typically set at 100 degrees Celsius, the temperature difference between ingredients placed on different shelves is minimal, so there's no need to adjust the initial cooking temperature. Alternatively, set the cooking temperature coefficient m to 1. The adjusted cooking temperature... As shown in equation (3):

[0091] (3)

[0092] Similarly, users can select the operating mode of the steam oven 1 through the operation screen 2. The initial cooking time corresponding to the operating mode of the steam oven 1 has been pre-programmed into the cooking control device (or controller, control system). Since the area and quantity of ingredients both affect the processing time, a cooking time coefficient can be determined based on the number and area of ​​ingredients, and then the initial cooking time can be adjusted according to the cooking time coefficient L. The adjustment method can also be to adjust the cooking time coefficient L relative to the initial cooking time. Then the adjusted cooking time As shown in equation (4):

[0093] (4)

[0094] In this embodiment, the cooking temperature coefficient and cooking time coefficient are determined based on the location, quantity, and area of ​​the ingredients, and the temperature and time of the cooking equipment are adjusted accordingly. This allows for the creation of targeted cooking parameters for the ingredients, enabling more precise and personalized control over the cooking process and improving the cooking results.

[0095] In one embodiment, such as Figure 9 As shown, S320 includes:

[0096] S321, determine the area coefficient corresponding to the area of ​​the ingredient based on the area range of the ingredient.

[0097] S322, determine the number coefficient corresponding to the number of ingredients based on the range of ingredient numbers in which the ingredient number falls.

[0098] S323, determine the cooking time coefficient of the ingredients based on the number of ingredients, the area of ​​the ingredients, the area coefficient, and the number coefficient.

[0099] Following the above embodiments, the cooking time of ingredients is related to both the area and the number of ingredients. Therefore, the cooking time coefficient of ingredients can be determined based on the respective ranges of the area and the number of ingredients.

[0100] Specifically, for each type of ingredient, a pre-defined average area distribution table can be created, containing different area ranges and quantity ranges for each ingredient. This table is based on the average area of ​​each ingredient. The area coefficient Q is determined by the area range of the food ingredient, and the number coefficient N is determined by the number of food ingredients n. For example, the distribution table of area coefficients and number coefficients is shown in Table 4:

[0101] Table 4 Distribution of Area Coefficient and Number Coefficient

[0102]

[0103] After determining the area coefficient Q and the number coefficient N, different weight values ​​can be set for the area coefficient Q and the number coefficient N. The cooking time coefficient L of the ingredients can be obtained by weighted summation. For example, the weight value of the area coefficient Q is 0.6 and the weight value of the number coefficient N is 0.4. This means that the area of ​​the ingredients has a higher cooking attribute than the number of ingredients. The cooking time coefficient L can be determined based on equation (5):

[0104] (5)

[0105] In this embodiment, the impact of two key factors, the area and number of ingredients, on cooking time is comprehensively considered. A cooking time coefficient that accurately reflects actual cooking needs is calculated by weighted summation, which helps to formulate more accurate cooking times and improve cooking results.

[0106] In one embodiment, after S340, the method further includes: receiving a cooking time adjustment signal carrying a target cooking time, and adjusting the cooking time of the cooking time to the target cooking time.

[0107] Following the above embodiment, after adjusting the initial cooking time, the control panel 2 of the steam oven 1 will display the adjusted cooking time. For example, it can be displayed as a time progress bar. In addition, users can also use touch operation on the operation screen 2 to customize and adjust the cooking time, such as sliding the time progress bar to the target cooking time, thereby adjusting the cooking time. Adjust to the target cooking time.

[0108] In this embodiment, the user can customize and adjust the cooking time based on the cooking time calculated by the cooking control device, which improves the flexibility of the cooking device and can also meet the user's cooking needs.

[0109] In one embodiment, such as Figure 10 As shown, image data of the interior of the cooking equipment is acquired, including:

[0110] S110: Acquire initial image data of the interior of the cooking equipment.

[0111] S120, the initial image data is distorted to obtain the distorted initial image data.

[0112] S130: Preprocess the corrected initial image data to obtain image data of the interior of the cooking equipment.

[0113] Due to the characteristics of the optical lens of the image acquisition device and factors such as its installation position and angle, the initial image data acquired often exhibits a certain degree of distortion.

[0114] by Figure 3 Taking the steam oven 1 as an example, the focal length of camera 4 is usually relatively short and it is a fixed-focus static lens. This means that the optimal focal length is achieved when the focus ring is rotated to obtain the clearest view. However, when a short-focal-length lens (i.e., a wide-angle lens) captures images inside the steam oven 1, transmission distortion occurs, resulting in a fisheye lens effect. Therefore, when camera 4 acquires initial image data, the relative proportions of near and far features change, causing the image to bend or distort. Consequently, the initial image data exhibits barrel distortion, requiring distortion correction to obtain corrected initial image data.

[0115] The distortion correction process can begin by identifying key feature points in the initial image data (e.g., the titanium frame and rear metal plate in oven 1). These key feature points are prominent objects and lines in the initial image data, which can be used to determine the perspective relationships. Further, based on the identified key feature points, a mathematical model can be constructed to describe the perspective relationships of the initial image data. This mathematical model can be an affine transformation model, a transmission transformation model, or other types of transformation models. Then, the established transformation model is applied to the initial image data, and barrel distortion is corrected by adjusting various parts of the initial image data to match the expected perspective relationships.

[0116] Furthermore, noise may still exist in the initial image data after distortion correction. To further improve image quality, the corrected initial image data can be further preprocessed, such as removing noise, enhancing image contrast, and enhancing details. The image data obtained after preprocessing is clearer and of higher quality.

[0117] In this embodiment, by performing a series of processes on the initial image data, such as distortion correction, noise reduction, and detail enhancement, the image quality can be effectively improved, making the subsequent image data-based analysis process more efficient and accurate, and improving the accuracy of cooking parameters.

[0118] In one embodiment, such as Figure 11 As shown, S120 includes:

[0119] S121, Identify key feature points in the initial image data, and determine the perspective relationship model of the initial image data based on the key feature points.

[0120] S122, Based on the perspective relationship model, the initial image data is distorted to obtain the corrected initial image data.

[0121] Following the above embodiments, in image recognition, the repair of distorted images involves perspective transformation or homography transformation, the purpose of which is to map one quadrilateral region in the image data to another quadrilateral region, thereby correcting the perspective distortion of the image.

[0122] Specifically, feature point detection algorithms (such as SIFT, SURF, ORB, etc.) can be used to detect key feature points in the initial image, and descriptor matching can be performed to find the actual points corresponding to these key feature points from different viewpoints. Then, four key feature points and their corresponding actual points can be selected to define a quadrilateral region in the initial image data. Assuming the coordinates of the four key feature points in the initial image data are... , , , The corrected coordinates of these four key feature points are as follows: , , , The coordinates of the key feature points before and after correction satisfy the following relationship:

[0123] (6)

[0124] (7)

[0125] (8)

[0126] For each pair of key feature points, the following two equations can be written:

[0127] (9)

[0128] (10)

[0129] Since four pairs of key feature points are known, eight equations such as equation (9) or equation (10) can be obtained. These eight equations are combined into an 8*9 matrix A and an 8*1 zero vector B. Then, the linear equation system AH=B is solved, where H is a 9-dimensional vector as shown in equation (6). Finally, the homography matrix H can be obtained by solving the linear equation system using the least squares method, which serves as the perspective relation model.

[0130] Then, based on the perspective relationship model, distortion correction can be performed on the initial image data. Specifically, other points in the initial image data are subjected to transmission transformation according to the same transmission relationship as the key feature points, thereby obtaining the corrected initial image data.

[0131] In this embodiment, a homography matrix is ​​constructed to describe the transmission transformation relationship of image data before and after distortion correction, thereby correcting the distortion of the initial image data and providing an accurate data basis for subsequent formulation of cooking parameters, thus improving the accuracy of the cooking parameters.

[0132] To provide a clearer explanation of the cooking equipment control method proposed in this application, detailed embodiments are described below:

[0133] by Figure 3 Taking the steam oven 1 as an example, if a user wants to cook sweet potatoes in the steam oven 1, the user places three sweet potatoes on a baking tray and puts the tray into the steam oven 1. Data feedback from the pressure sensor in the steam oven 1 indicates that the user has placed the baking tray on the second shelf. By referring to a table, the vertical area ratio coefficient P = 1.05. The moment the user closes the steam oven door, the camera 4 on the door activates white balance mode and stabilizes for one second. After one second, it takes a picture of the inside of the steam oven 1 and uploads it to the cooking control device (or controller, control system). The cooking control device analyzes the data to determine the area of ​​each of the nine pre-divided areas on the shelf. Combined with the horizontal area ratio coefficient k corresponding to each area, the area of ​​the sweet potato is calculated. On the other hand, the cooking control equipment identifies that there are 3 sweet potatoes on the shelf, meaning the total number of ingredients is 3, and then calculates the average area of ​​the sweet potatoes. Furthermore, by looking up the table, we found that the area coefficient Q = 1.1 and the number coefficient N = 1.0. After weighting the area coefficient Q and the number coefficient N, we obtain L = 1.06. Based on the user's selected cooking mode (steaming or baking), the initial cooking time is pre-programmed into the cooking control device. =40 minutes, so the final cooking time displayed on control panel 2 is 40 minutes. =42.5 minutes. A time progress bar is displayed on the control panel 2. Users can adjust the cooking time by dragging the slider. The baking tray is placed on the second layer, and the cooking mode does not involve activating the lower heating element. The initial cooking temperature is pre-programmed into the cooking control device. =200 degrees Celsius. Looking up the table, the cooking temperature coefficient m = 102%. Therefore, the adjusted cooking temperature... The temperature is 204 degrees Celsius, displayed on the control screen 2. Users can start cooking by clicking "Start Cooking" on the control screen 2.

[0134] In another detailed embodiment, the user turns on the steam oven 1, places two plates on the steaming tray, each plate containing five steamed buns, and sends the steaming tray into the steam oven 1. Data feedback from the pressure sensor indicates that the user has placed the steaming tray on the third layer. By referring to a table, the vertical area ratio coefficient P = 1. The moment the user closes the steam oven door, the camera 4 located on the door activates white balance mode and stabilizes for one second. One second later, it takes an image of the interior of the steam oven 1 and uploads it to the cooking control device (or controller, control system). The cooking control device analyzes the data to determine the area of ​​each of the nine pre-divided areas on the shelf. Combined with the horizontal area ratio coefficient k corresponding to each area, the area of ​​the steamed buns is calculated. On the other hand, the cooking control equipment identifies that there are 10 steamed buns on the shelf, meaning the total number of ingredients is 10, and then calculates the average area of ​​the steamed buns. After data processing by the cooking control equipment, the area coefficient Q = 1.1 and the number coefficient N = 1.1 for the steamed buns. Weighting the area coefficient Q and the number coefficient N, we get L = 1.1. Based on the user's selected cooking mode, the initial cooking time is pre-programmed into the cooking control equipment. =20 minutes, so the final cooking time displayed on control panel 2 is 20 minutes. =22 minutes, and the operation screen 2 also has a time progress bar. If the user wants to adjust the cooking time, they can drag the slider to adjust the cooking time to the target cooking time. The initial cooking temperature is pre-programmed into the cooking control device. =100 degrees Celsius. The cooking method for steamed buns is steaming, so the initial cooking temperature should be kept constant, i.e., the cooking temperature... The temperature is set to 100 degrees Celsius and displayed on the control screen 2. Users can start cooking by clicking "Start Cooking" on the control screen 2.

[0135] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0136] Based on the same inventive concept, this application also provides a cooking equipment control device for implementing the cooking equipment control method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the cooking equipment control device provided below can be found in the limitations of the cooking equipment control method described above, and will not be repeated here.

[0137] In one exemplary embodiment, such as Figure 12 As shown, a cooking equipment control device 500 is provided, including: a data acquisition module 510, an ingredient analysis module 520, a cooking parameter determination module 530, and a cooking module 540, wherein:

[0138] The data acquisition module 510 is used to acquire image data inside the cooking equipment and the position of the ingredients in the cooking equipment when the ingredients are placed in the cooking equipment; the image data includes image information of the ingredients.

[0139] The ingredient analysis module 520 is used to determine the area and number of ingredients based on image data and the position of ingredients in the cooking equipment.

[0140] The cooking parameter determination module 530 is used to determine the cooking parameters of the cooking equipment based on the position, number, and area of ​​the ingredients in the cooking equipment.

[0141] The cooking module 540 is used to control the cooking equipment to operate according to cooking parameters.

[0142] In one embodiment, the ingredient analysis module 520 is further configured to determine the number of ingredients and the area of ​​the ingredients in the image data based on the image data, determine the area ratio coefficient based on the position of the ingredients in the cooking device, and determine the area of ​​the ingredients based on the area of ​​the ingredients in the image data and the area ratio coefficient.

[0143] In one embodiment, the position includes the horizontal position of the ingredient in the horizontal direction and the vertical position in the vertical direction, and the area ratio coefficient includes the vertical area ratio coefficient and the horizontal area ratio coefficient. The ingredient analysis module 520 is also used to determine the vertical area ratio coefficient based on the vertical position of the ingredient in the cooking device and to determine the horizontal area ratio coefficient based on the horizontal position of the ingredient in the cooking device.

[0144] In one embodiment, the cooking parameters include cooking temperature and cooking time. The cooking parameter determination module 530 is further configured to determine the cooking temperature coefficient of the ingredients based on their position in the cooking device, determine the cooking time coefficient of the ingredients based on the number and area of ​​the ingredients, adjust the preset initial cooking temperature of the cooking device based on the cooking temperature coefficient, determine the cooking temperature of the cooking device, adjust the preset initial cooking time of the cooking device based on the cooking time coefficient, and determine the cooking time of the cooking device.

[0145] In one embodiment, the cooking parameter determination module 530 is further configured to determine the area coefficient corresponding to the area of ​​the ingredient based on the area range of the ingredient, determine the number coefficient corresponding to the number of ingredients based on the number of ingredients based on the number of ingredients, and determine the cooking time coefficient of the ingredients based on the number of ingredients, the area of ​​the ingredients, the area coefficient, and the number coefficient.

[0146] In one embodiment, the cooking equipment control device 500 is further configured to receive a cooking time adjustment signal carrying a target cooking time, and adjust the cooking time of the cooking time to the target cooking time.

[0147] In one embodiment, the data acquisition module 510 is used to acquire initial image data inside the cooking device, perform distortion correction on the initial image data to obtain corrected initial image data, and preprocess the corrected initial image data to obtain image data inside the cooking device.

[0148] In one embodiment, the data acquisition module 510 is used to identify key feature points in the initial image data, determine the perspective relationship model of the initial image data based on the key feature points, and perform distortion correction on the initial image data according to the perspective relationship model to obtain the corrected initial image data.

[0149] Each module in the aforementioned cooking equipment control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0150] In one embodiment, this application provides a cooking control device for controlling a cooking device according to the cooking device control method in any of the above-mentioned cooking device control method embodiments, which will not be described in detail here.

[0151] In one embodiment, this application provides a cooking apparatus 600, such as... Figure 13 As shown, the cooking apparatus 600 includes a cooking control device 610, a pressure sensor 620 connected to the cooking control device 610, and an image acquisition device 630. The pressure sensor 620 is used to collect the position of the food ingredients in the cooking apparatus 600 and send it to the cooking control device 610. The image acquisition device 630 is used to collect image data inside the cooking apparatus 600 and send it to the cooking control device 610. Taking the cooking apparatus 600 as an example... Figure 3 Taking the steam oven 1 as an example, the pressure sensing device 620 is equivalent to the pressure sensor 3-2-5 in the steam oven 1, and the image acquisition device 630 is equivalent to the camera 4 in the steam oven 1.

[0152] In one embodiment, such as Figure 14 As shown, the cooking appliance 600 also includes a touch device 640 connected to the cooking control device 610, used to generate and send a cooking time adjustment signal carrying a target cooking time to the cooking control device 610 in response to a touch operation. Taking the cooking appliance 600 as an example... Figure 3 Taking the steam oven 1 as an example, the touch device 640 is equivalent to the operation screen 2. While responding to the user's touch operation, the touch device 640 can also serve as a display device to present relevant cooking parameters to the user.

[0153] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 15 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data such as image data from within the cooking equipment. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When the computer program is executed by the processor, it implements a cooking equipment control method.

[0154] Those skilled in the art will understand that Figure 14 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0155] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in any of the above embodiments of the cooking device control method.

[0156] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in any of the above embodiments of the cooking equipment control method.

[0157] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the cooking equipment control method.

[0158] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0159] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0160] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0161] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for controlling a cooking device, characterized in that, The method includes: When ingredients are placed in a cooking device, image data of the interior of the cooking device and the position of the ingredients within the cooking device are acquired; the image data includes image information of the ingredients; the position includes the horizontal position of the ingredients in the horizontal direction and the vertical position in the vertical direction. Based on the image data, the number of ingredients is determined; based on the vertical position of the ingredients in the cooking equipment, a vertical area ratio coefficient is determined, which characterizes the influence of the vertical distribution of the ingredients on the area of ​​the ingredients; based on the horizontal position of the ingredients in the cooking equipment, a horizontal area ratio coefficient is determined, which characterizes the influence of the horizontal distribution of the ingredients on the area of ​​the ingredients; based on the area of ​​the ingredients in the image data, the vertical area ratio coefficient, and the horizontal area ratio coefficient, the area of ​​the ingredients is determined. The cooking parameters of the cooking equipment are determined based on the position of the ingredients in the cooking equipment, the number of ingredients, and the area of ​​the ingredients. Control the cooking equipment to operate at the cooking parameters.

2. The method according to claim 1, characterized in that, The cooking parameters include cooking temperature and cooking time. Determining the cooking parameters of the cooking equipment based on the position of the ingredients in the cooking equipment, the number of ingredients, and the area of ​​the ingredients includes: The cooking temperature coefficient of the ingredient is determined based on its position in the cooking equipment. The cooking time coefficient of the ingredients is determined based on the number of ingredients and the area of ​​the ingredients. Based on the cooking temperature coefficient, adjust the preset initial cooking temperature of the cooking equipment to determine the cooking temperature of the cooking equipment; Based on the cooking time coefficient, the preset initial cooking time of the cooking equipment is adjusted to determine the cooking time of the cooking equipment.

3. The method according to claim 2, characterized in that, The step of determining the cooking time coefficient of the ingredients based on the number and area of ​​the ingredients includes: Determine the area coefficient corresponding to the area of ​​the food ingredient based on the food ingredient area range in which the food ingredient area is located; Based on the range of ingredient quantities in which the ingredient quantity falls, determine the corresponding quantity coefficient for the ingredient quantity; The cooking time coefficient of the ingredients is determined based on the number of ingredients, the area of ​​the ingredients, the area coefficient, and the number coefficient.

4. The method according to claim 2, characterized in that, After adjusting the preset initial cooking time of the cooking equipment according to the cooking time coefficient and determining the cooking time of the cooking equipment, the method further includes: Receive a cooking time adjustment signal carrying the target cooking time; Adjust the cooking time to the target cooking time.

5. The method according to any one of claims 1 to 4, characterized in that, The acquisition of image data inside the cooking device includes: Acquire initial image data of the interior of the cooking device; The initial image data is subjected to distortion correction to obtain the corrected initial image data; The corrected initial image data is preprocessed to obtain image data of the interior of the cooking device.

6. The method according to claim 5, characterized in that, The process of performing distortion correction on the initial image data to obtain corrected initial image data includes: Identify key feature points in the initial image data; Based on the key feature points, determine the perspective relationship model of the initial image data; Based on the perspective relationship model, the initial image data is distorted to obtain the corrected initial image data.

7. A cooking equipment control device, characterized in that, The device includes: The data acquisition module is used to acquire image data inside the cooking device and the position of the food in the cooking device when the food is placed in the cooking device; the image data includes image information of the food; the position includes the horizontal position of the food in the horizontal direction and the vertical position in the vertical direction. The ingredient analysis module is used to determine the number of ingredients based on the image data; determine a vertical area ratio coefficient based on the vertical position of the ingredients in the cooking equipment, the vertical area ratio coefficient being used to characterize the influence of the vertical position distribution of the ingredients on the ingredient area; determine a horizontal area ratio coefficient based on the horizontal position of the ingredients in the cooking equipment, the horizontal area ratio coefficient being used to characterize the influence of the horizontal position distribution of the ingredients on the ingredient area; and determine the ingredient area based on the area of ​​the ingredients in the image data, the vertical area ratio coefficient, and the horizontal area ratio coefficient. A cooking parameter determination module is used to determine the cooking parameters of the cooking equipment based on the position of the ingredients in the cooking equipment, the number of ingredients, and the area of ​​the ingredients. A cooking module is used to control the cooking equipment to operate with the cooking parameters.

8. The apparatus according to claim 7, characterized in that, The cooking parameters include cooking temperature and cooking time. The cooking parameter determining module is further configured to determine the cooking temperature coefficient of the ingredients based on their position in the cooking equipment; determine the cooking time coefficient of the ingredients based on their number and area; adjust the preset initial cooking temperature of the cooking equipment based on the cooking temperature coefficient to determine the cooking temperature of the cooking equipment; and adjust the preset initial cooking time of the cooking equipment based on the cooking time coefficient to determine the cooking time of the cooking equipment.

9. The apparatus according to claim 8, characterized in that, The cooking parameter determination module is further configured to determine the area coefficient corresponding to the area of ​​the ingredient based on the area range of the ingredient; determine the number coefficient corresponding to the number of ingredients based on the number of ingredients based on the number of ingredients; and determine the cooking time coefficient of the ingredient based on the number of ingredients, the area of ​​the ingredient, the area coefficient, and the number coefficient.

10. A cooking control device, characterized in that, The cooking control device is used to control the cooking device according to any one of claims 1 to 6.

11. A cooking appliance, characterized in that, The method includes a cooking control device, a pressure sensing device and an image acquisition device connected to the cooking control device. The pressure sensing device is used to acquire the position of the food in the cooking device and send it to the cooking control device. The image acquisition device is used to acquire image data inside the cooking device and send it to the cooking control device. The cooking control device is used to control the cooking device according to any one of claims 1 to 6.

12. The cooking apparatus according to claim 11, characterized in that, The cooking equipment also includes a touch device connected to the cooking control device, for generating and sending a cooking time adjustment signal carrying a target cooking time to the cooking control device in response to a touch operation.

Images