Cooking equipment control method and device, cooking control equipment and cooking equipment

Abstract

The invention relates to a cooking equipment control method and device, cooking control equipment and cooking equipment. The method comprises the following steps: under the condition that a food material is put into the cooking equipment, acquiring image data in the cooking equipment and the position of the food material in the cooking equipment; the image data comprises image information of the food materials, determining the food material area and the food material number according to the image data and the positions of the food materials in the cooking equipment, determining cooking parameters of the cooking equipment according to the positions of the food materials in the cooking equipment, the food material number and the food material area, and controlling the cooking equipment to work according to the cooking parameters. By adopting the method, the cooking effect of the cooking equipment can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of cooking control, and in particular to a cooking equipment control method, device, cooking control equipment, cooking equipment, computer equipment, computer-readable storage medium and computer program product. Background Art

[0002] With the continuous development of cooking technology, the cooking modes of cooking equipment are gradually increasing. Users can flexibly adjust the cooking mode of cooking equipment according to parameters such as the type and quantity of ingredients. Therefore, how to select the most suitable cooking parameters for different ingredients has become one of the problems to be solved.

[0003] Traditional cooking equipment usually has 2-3 cooking time gradients and cooking temperature gradients. Users can choose the cooking mode that best suits the ingredients from several existing cooking modes based on the type, weight and other parameters of the ingredients.

[0004] However, the above-mentioned traditional cooking devices have limited cooking modes and cover a small amount. The existing cooking modes may be difficult to meet the cooking needs of the ingredients, resulting in poor cooking results. Summary of the invention

[0005] Based on this, it is necessary to provide a cooking equipment control method, device, cooking control equipment, cooking equipment, computer equipment, computer-readable storage medium and computer program product that can improve cooking effects in response to the above-mentioned technical problems.

[0006] In a first aspect, the present application provides a cooking device control method, comprising:

[0007] When food is placed in a cooking device, image data of the interior of the cooking device and the position of the food in the cooking device are acquired; the image data includes image information of the food;

[0008] Determine the area and number of ingredients according to the image data and the position of the ingredients in the cooking device;

[0009] Determining cooking parameters of the cooking device according to the position of the food in the cooking device, the number of the food, and the area of ​​the food;

[0010] The cooking device is controlled to operate according to the cooking parameters.

[0011] In a second aspect, the present application also provides a cooking equipment control device, comprising:

[0012] A data acquisition module, used to acquire image data inside the cooking device and the position of the food in the cooking device when food is placed in the cooking device; the image data includes image information of the food;

[0013] An ingredient analysis module, used to determine the area and number of ingredients according to the image data and the position of the ingredients in the cooking device;

[0014] a cooking parameter determination module, configured to determine the cooking parameters of the cooking device according to the position of the food in the cooking device, the number of the food and the area of ​​the food;

[0015] The cooking module is used to control the cooking device to work according to the cooking parameters.

[0016] In a third aspect, the present application further provides a cooking control device, which is used to control a cooking device according to the steps in any one of the above cooking device control method embodiments.

[0017] In a fourth aspect, the present application also provides a cooking device, comprising a cooking control device, a pressure sensing device and an image acquisition device connected to the cooking control device, wherein the pressure sensing device is used to collect the position of food in the cooking device and send it to the cooking control device, and the image acquisition device is used to collect image data inside the cooking device and send it to the cooking control device.

[0018] In a fifth aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above cooking device control method embodiment when executing the computer program.

[0019] In a sixth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the above-mentioned cooking device control method embodiment are implemented.

[0020] In a seventh aspect, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps in the above-mentioned cooking device control method embodiment.

[0021] The above cooking equipment control method, device, cooking control device, cooking equipment, computer-readable storage medium and computer program product are different from the traditional schemes in which the cooking modes are limited and it is difficult to meet the cooking needs of diversified ingredients. This scheme obtains the image data containing the image information of the ingredients inside the cooking equipment and the position of the ingredients in the cooking equipment, thereby determining the area and number of ingredients, and determines the cooking parameters of the cooking equipment according to the position of the ingredients in the cooking equipment, the number of ingredients and the area of ​​the ingredients, and finally controls the cooking equipment to work with the cooking parameters. In other words, this scheme is no longer limited to the existing modes of the cooking equipment, but formulates cooking parameters according to the actual state of the ingredients in the current cooking equipment, broadens the adjustable range of the cooking parameters, so that it can better meet the cooking needs of the ingredients, improve the adaptability and accuracy of the cooking parameters, and thus improve the cooking effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A diagram showing an application environment of a cooking device control method in an embodiment;

[0024] Figure 2 A schematic diagram of a cooking device control method according to an embodiment;

[0025] Figure 3 is a schematic diagram of a cooking device in one embodiment;

[0026] Figure 4 is a flow chart of a cooking device control method in another embodiment;

[0027] Figure 5 A schematic diagram of image data in one embodiment;

[0028] Figure 6 This is a schematic diagram of food location zoning in one embodiment;

[0029] Figure 7 is a flow chart of a cooking device control method in yet another embodiment;

[0030] Figure 8 A schematic flow chart of a cooking parameter determination step in one embodiment;

[0031] Fig. 9A schematic flow chart of a cooking parameter determination step in another embodiment;

[0032] Fig.10 A schematic flow chart of steps for acquiring image data in one embodiment;

[0033] Fig.11 A schematic flow chart of steps for acquiring image data in another embodiment;

[0034] Fig.12 is a structural block diagram of a cooking equipment control device in one embodiment;

[0035] Fig.13 is a structural block diagram of a cooking device in one embodiment;

[0036] Fig.14 is a structural block diagram of a cooking device in another embodiment;

[0037] Fig.15 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment.

[0038] Explanation of the reference numerals: 1. Steam oven; 2. Operation panel; 3. Inner cavity; 3-1-1. Food; 3-3-1. Storage layer; 3-2-5. Pressure sensor; 4. Camera. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0040] The cooking device control method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the data acquisition device 102 communicates with the cooking control device 104 through a network. The data storage system can store data that the cooking control device 104 needs to process. The data storage system can be integrated on the cooking control device 104, or placed on the cloud or other network servers.

[0041] Specifically, when the ingredients are put into the cooking equipment, the data acquisition device 102 deployed in the cooking equipment may collect image data inside the cooking equipment and the position of the ingredients in the cooking equipment. The image data contains image information of the ingredients and is sent to the cooking control device 104. The cooking control device 104 determines the area and number of ingredients based on the image data and the position of the ingredients in the cooking equipment, and then determines the cooking parameters of the cooking equipment based on the position of the ingredients in the cooking equipment, the number of ingredients and the area of ​​the ingredients. Finally, the cooking equipment is controlled to work according to the cooking parameters.

[0042] Among them, the data acquisition device 102 can be but is not limited to various image acquisition devices, pressure sensing devices, etc., and the cooking control device 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0043] In an exemplary embodiment, Figure 2 As shown, a cooking device control method is provided, and the method is applied to Figure 1 The cooking control device in the example is used to illustrate, including the following steps:

[0044] S100, when food is put into a cooking device, image data of the interior of the cooking device and the position of the food in the cooking device are obtained.

[0045] The collected image data includes image information of the food, and the image data can be obtained by an image acquisition device disposed in the cooking device, such as a high-definition camera, photographing the interior of the cooking device. The position of the food in the cooking device can be acquired by a pressure sensing device. When the food is placed in the cooking device, the pressure sensing device can sense the pressure generated by the food, and then locate the position of the food in the cooking device.

[0046] Take the cooking equipment as a steam oven as an example. Figure 3 As shown, the steam oven 1 is provided with an operation screen 2, an inner cavity 3 and a camera 4. The camera 4 can be arranged at the center position or other position of the door of the steam oven 1, or can be adaptively arranged at other positions in the steam oven 1. When the user puts the food into the inner cavity 3, the camera 4 is triggered to collect the image data of the interior of the steam oven. Alternatively, the user performs a touch operation on the operation screen 2 to set some initial cooking parameters and trigger the camera 4 to collect the image data of the interior of the steam oven. In addition, there are usually many storage layers in the steam oven. Counting from the bottom to the top, the bottom storage layer is denoted as 3-3-1. A pressure sensing device is arranged in each storage layer. For example, pressure sensor 3-2-5 represents a pressure sensor arranged in the fifth storage layer from the bottom to the top. Due to the gravity of the food, the raised part of the pressure sensor 3-2-5 drops and is fed back to the cooking control device, so that the level of the food can be quickly located. In the same storage layer, the food can also have different placement positions. For example Figure 3In the fifth storage layer, the first ingredient counted from right to left is recorded as ingredient 3-1-1. In this way, the image data inside the steam oven 1 and the position of the ingredients in the steam oven 1 can be collected through the camera 4 and the pressure sensor 3-2-5 in the steam oven 1. It should be noted that if ingredients are placed on multiple storage layers inside the steam oven 1, since the camera 4 can only collect image data of the highest storage layer, the subsequent data analysis will be performed according to the placement of ingredients on the highest storage layer. However, multiple cameras 4 can also be set in the steam oven 1 to collect image data of each storage layer, so that the placement of ingredients on different storage layers can be considered when formulating cooking parameters later.

[0047] S200, determining the area and number of ingredients according to the image data and the positions of the ingredients in the cooking device.

[0048] The number of ingredients has a significant impact on cooking parameters. More ingredients means more energy is needed to complete cooking, and the cooking temperature or cooking time may need to be increased. For example, for the same 300-gram sweet potatoes, 3 need to be baked for 40 minutes, while 8 need to be baked for 50 minutes. Similarly, the area of ​​the ingredients will also affect the cooking parameters. For example, for the same number of sweet potatoes, 280 grams (small area) sweet potatoes need to be baked for 35 minutes, while 500 grams (large area) sweet potatoes need to be baked for 50 minutes.

[0049] Specifically, the collected image data may have some distortion, noise and other problems, and it is necessary to pre-process it to reduce the distortion, noise and other problems, so that the processed image data is more accurate. Then, some image processing algorithms, such as edge detection algorithms, can be used to extract the outline of the food in the image data, and then estimate the area of ​​the food according to the outline of the food. It should be noted that for cooking equipment (such as a steam oven) with multiple storage layers inside, since the distance between the camera and different storage layers is different, when estimating the area of ​​the food, it is also necessary to consider the position of the food in the cooking equipment, so as to obtain an accurate area of ​​the food. The number of food can be directly obtained by analyzing the image data using a specific algorithm, for example, through a target recognition neural network model, first extracting the characteristic information of the food according to the image data, including but not limited to shape, outline, color, texture, relative position, etc., and then judging which parts of the image data belong to the same food individual and which belong to different food individuals according to the above characteristic information, and then counting the number of food.

[0050] S300, determining cooking parameters of the cooking device according to the position of the ingredients in the cooking device, the number of ingredients and the area of ​​the ingredients.

[0051] Among them, the cooking parameters include but are not limited to parameters such as cooking temperature and cooking time. Taking the cooking device as a steam oven as an example, the position of the ingredients in the cooking device includes which storage layer the ingredients are on and the distribution position of the ingredients on the same storage layer. Accordingly, when the ingredients are on different storage layers, the corresponding cooking temperatures and cooking times are also different. Even if the ingredients are on the same storage layer, the cooking temperatures and cooking times corresponding to different numbers of ingredients and areas of ingredients are also different.

[0052] For example, a steam oven usually has an upper heating tube and a lower heating tube. If the user selects a mode that enables the upper heating tube and disables the lower heating tube, then for the same food, the closer the food layer is to the upper heating tube, the lower the corresponding cooking temperature should be. The larger the food area and the more food there are, the higher the corresponding cooking temperature should be and the longer the cooking time should be.

[0053] S400, controlling the cooking device to work according to the cooking parameters.

[0054] Specifically, after the cooking parameters are determined based on factors such as the position of the ingredients in the cooking equipment, the number of ingredients, and the area of ​​the ingredients, the cooking control device (or controller, control system, etc.) in the cooking equipment will convert these parameters into specific control instructions. The cooking control device will send corresponding control instructions to each working module of the cooking equipment to control each module to work according to the cooking parameters. For example, the heating module is controlled to heat the ingredients according to the calculated cooking temperature, and the heating time is the calculated cooking time.

[0055] The above cooking equipment control method is different from the traditional scheme in which the cooking modes are limited and it is difficult to meet the cooking needs of diversified ingredients. This scheme obtains the image data containing the image information of the ingredients inside the cooking device and the position of the ingredients in the cooking device, and then determines the area and number of ingredients, and determines the cooking parameters of the cooking device according to the position of the ingredients in the cooking device, the number of ingredients and the area of ​​the ingredients, and finally controls the cooking device to work with the cooking parameters. In other words, this scheme is no longer limited to the existing modes of the cooking device, but formulates cooking parameters according to the actual state of the ingredients in the current cooking device, broadens the adjustable range of the cooking parameters, so that it can better meet the cooking needs of the ingredients, improve the adaptability and accuracy of the cooking parameters, and thus improve the cooking effect.

[0056] In one embodiment, Figure 4 As shown, S200 includes:

[0057] S210, determining the number of ingredients and the area of ​​the ingredients in the image data according to the image data.

[0058] S220: Determine an area ratio coefficient according to the position of the food in the cooking device.

[0059] S230: Determine the area of ​​the food based on the area of ​​the food in the image data and the area ratio coefficient.

[0060] For example, when the cooking equipment is a steam oven, when the ingredients are placed on the same storage layer, the ingredients placed closer to the camera will appear larger than normal, while the ingredients placed farther away from the camera will appear smaller than normal. When the ingredients are placed on different storage layers, the same ingredients may appear smaller than actual when placed on the lower layer, and larger than actual when placed on the upper layer. Therefore, it is necessary to determine the area ratio coefficient according to the position of the ingredients in the cooking equipment, and then calculate the actual area of ​​the ingredients. The number of ingredients determined by processing image data is usually more accurate and generally does not have too much deviation.

[0061] Specifically, the shape, outline, color, texture, relative position and other characteristic information of the food can be extracted from the image data, and then the above characteristic information can be used to determine which parts of the image data belong to the same food individual and which belong to different food individuals, and then the number of food is counted. In addition, when calculating the area of ​​food, some edge detection algorithms can be used to identify the outline of the food, and then the area of ​​the food in the image data can be determined according to the outline of the food. Then, it is necessary to convert the area of ​​the food in the image data into the actual area of ​​the food. This process involves the area ratio coefficient, and the area ratio coefficient is related to the position of the food in the cooking device. Exemplarily, the food in the image data can be analyzed and classified by decision trees, random forests and other methods, and then the relationship between the food area and the characteristic information of the food in the image data can be established by linear regression, polynomial regression and other methods, and then the food area can be determined. In addition, the food area can also be calculated based on a deep model algorithm, such as using a CNN model of a regression task, inputting image data, and outputting the food area predicted by the model, or using a combination model to combine traditional features and deep learning features to construct a hybrid model to predict the food area, which is not limited here. It should be noted that when using the deep model algorithm to calculate the area of ​​ingredients, model training is required first. The training data can be obtained in the following ways: multiple image data can be collected for the same ingredient at multiple angles, different lighting conditions, and different backgrounds to reduce the impact of occlusion and shadows and help separate the foreground and background, thereby improving robustness. Then, data annotation is performed for each image data, including annotating the boundaries of the ingredients, generating pixel-level masks, and annotating the actual area of ​​the ingredients (the area of ​​the ingredients in the image data can be calculated first, and then converted into the actual area of ​​the ingredients in combination with the area ratio coefficient) to obtain training data.

[0062] For example, the area ratio coefficient may include a vertical area ratio coefficient and a horizontal area ratio coefficient. Figure 3 Taking the steam oven 1 in the figure as an example, since there are five storage layers in the steam oven 1, and the distance between each storage layer and the camera 4 is different, a vertical area ratio coefficient can be set for each storage layer to represent the influence of the vertical position distribution of the food on the food area. In addition, for the same storage layer, the image data collected by the camera 4 is as follows Figure 5 As shown, it can be seen that ingredients 1-6 are distributed in different areas of the storage layer. The storage layer can be pre-divided into areas 1-9. The area distribution diagram is shown in FIG. Figure 6 As shown, the distance between each area and the camera 4 is also different, so a horizontal area ratio coefficient can be set for each area to characterize the influence of the position distribution of the food in the horizontal direction on the food area. It should be noted that the vertical area ratio coefficient and the horizontal area ratio coefficient can be pre-calibrated for the cooking equipment (the steam oven in this embodiment), for example, based on the historical image data and the actual food area of ​​each food in the historical image data. After determining the area ratio coefficient and the area of ​​the food in the image data, the food area can be further determined, for example, by multiplying the area of ​​the food in the image data with the vertical area ratio coefficient and the horizontal area ratio coefficient to obtain the food area.

[0063] In this embodiment, it is taken into consideration that the different distances between the food and the camera will result in a different proportional relationship between the food area in the image data and the actual food area. Therefore, different area proportional coefficients are set for areas on different layers and at different positions on the same layer, so that a more accurate food area can be calculated, thereby improving the accuracy of cooking parameters and enhancing cooking effects.

[0064] In one embodiment, Figure 7 As shown, S220 includes:

[0065] S221, determining a vertical area ratio coefficient according to the vertical position of the food in the cooking device.

[0066] S222: Determine a horizontal area ratio coefficient according to the horizontal position of the food in the cooking device.

[0067] The position of the food in the cooking device includes the horizontal position of the food in the horizontal direction and the vertical position of the food in the vertical direction, and the area ratio coefficient includes the vertical area ratio coefficient and the horizontal area ratio coefficient.

[0068] Specifically, Figure 3The steam oven 1 in the embodiment is used as an example for explanation. Following the above embodiment, there are 5 storage layers in the steam oven 1, and each storage layer is divided into 9 areas, which can be represented by (horizontal distribution, vertical distribution), labeled 1-9, for example, these 9 areas are (upper, left), (upper, middle), (upper, right), (middle, left), (middle, middle), (middle, right), (lower, left), (lower, middle), (lower, right). Correspondingly, the area of ​​the food in the image data can be divided into the area of ​​each area in the image data, recorded as For example, the area ratio coefficient of the steam oven 1 may be as shown in Table 1:

[0069] Table 1 Area ratio coefficient table

[0070]

[0071] According to Table 1, the vertical area ratio coefficient P is the enlargement or reduction ratio when the same food is placed on different storage layers in the steam oven 1, and the horizontal area ratio coefficient k is the enlargement or reduction ratio of the same food in the 9 partitions when the same food is placed on the same layer of the steam oven 1. The food areas in the nine zones are (It can be predicted by the CNN model in the above embodiment).

[0072] Furthermore, after determining the area ratio coefficient and the food areas in the nine areas, the food areas of all the food in the steam oven 1 can be further determined. For example, Figure 5 As an example, we can see that Figure 5 There are 6 ingredients in the game, all of which are placed in the third storage layer and are numbered 1-6. Different ingredients occupy different areas. The specific distribution of ingredients is shown in Table 2:

[0073] Table 2 Food distribution table

[0074]

[0075] It should be noted that the area (3, 1) where ingredient 1 is located means that ingredient 1 is in area 1 of the third storage layer, and the areas (3, 2) and (3, 5) where ingredient 2 is located means that ingredient 2 is in areas 2 and 5 of the third storage layer, and the same applies to other ingredients.

[0076] Food area It can be determined based on formula (1):

[0077] (1)

[0078] n is the number of ingredients, then the average area of ​​the ingredients It can be determined based on formula (2):

[0079] (2)

[0080] In this embodiment, the position distribution of ingredients in the vertical direction and the horizontal direction are taken into consideration, and different vertical area ratio coefficients P and horizontal area ratio coefficients k are set, so that the area of ​​ingredients can be determined more accurately, and more accurate cooking parameters can be formulated to better meet the cooking needs of the ingredients.

[0081] In one embodiment, Figure 8 As shown, S300 includes:

[0082] S310, determining a cooking temperature coefficient of the food according to the position of the food in the cooking device.

[0083] S320: Determine the cooking time coefficient of the ingredients according to the number of ingredients and the area of ​​the ingredients.

[0084] S330, adjusting a preset initial cooking temperature of the cooking device according to the cooking temperature coefficient, and determining a cooking temperature of the cooking device.

[0085] S340: According to the cooking time coefficient, the preset initial cooking time of the cooking device is adjusted to determine the cooking time of the cooking device.

[0086] In accordance with the above embodiment, the steam oven 1 has two working modes: steaming and baking. The user can first select the working mode of the steam oven 1 through the operation screen 2 on the steam oven 1. When the user selects the baking mode, the cooking control device (or controller, control system) has pre-written the initial cooking temperature corresponding to the baking mode. At this time, you need to adjust the initial cooking temperature according to the position of the ingredients in the cooking equipment. , the adjustment method can be multiplied by a cooking temperature coefficient m. The relationship between the cooking temperature coefficient and the position of the food in the cooking device can be shown in Table 3:

[0087] Table 3 Correlation between cooking temperature coefficient and the position of ingredients in cooking equipment

[0088]

[0089] It should be noted that the influence of ingredients in different areas of the same layer on the cooking temperature is small, so only the influence of ingredients in different storage layers and the heating method in the baking mode on the cooking temperature coefficient m is considered. Specifically, when the heating method of the steam oven 1 is to enable the upper and lower heating tubes for heating, the ingredients inside the steam oven 1 are heated more evenly, so the ingredients can be heated at the initial cooking temperature. When the heating method of the steam oven is to enable only the upper heating tube for heating, the closer the ingredients are to the upper heating tube, the more obvious the heating is. In order to meet the heating needs of the ingredients, the cooking temperature coefficient m corresponding to the storage layer closer to the upper heating tube is smaller.

[0090] When the working mode of steam oven 1 is steam mode, the preset initial cooking temperature Usually 100 degrees Celsius. At this time, the ingredients placed on different layers are not heated to the same extent, so there is no need to adjust the initial cooking temperature. , or set the cooking temperature coefficient m to 1. Adjusted cooking temperature As shown in formula (3):

[0091] (3)

[0092] Similarly, the user can select the working mode of the steam oven 1 through the operation screen 2, and the cooking control device (or controller, control system) has pre-written the initial cooking time corresponding to the working mode of the steam oven 1. Since the area and number of ingredients will affect the processing time of the ingredients, the cooking time coefficient can be determined according to the number of ingredients and the area of ​​ingredients, and then the initial cooking time can be adjusted according to the cooking time coefficient L The adjustment method can also be to adjust the cooking time coefficient L and the initial cooking time , then the adjusted cooking time As shown in formula (4):

[0093] (4)

[0094] In this embodiment, the cooking temperature coefficient and the cooking time coefficient are determined based on the position, number of ingredients and area of ​​the ingredients, and the temperature and time of the cooking equipment are adjusted accordingly. This allows targeted cooking parameters to be formulated for the ingredients, achieving more precise and personalized control of the cooking process and improving the cooking effect.

[0095] In one embodiment, Fig. 9 As shown, S320 includes:

[0096] S321, determining an area coefficient corresponding to the food area according to the food area interval in which the food area is located.

[0097] S322, determining a number coefficient corresponding to the number of ingredients according to the ingredient number interval in which the number of ingredients is located.

[0098] S323, determining the cooking time coefficient of the ingredients according to the number of ingredients, the area of ​​the ingredients, the area coefficient and the number coefficient.

[0099] Continuing from the above embodiment, the cooking time of the ingredients is related to both the area of ​​the ingredients and the number of ingredients, so the cooking time coefficient of the ingredients can be determined according to the intervals in which the area of ​​the ingredients and the number of ingredients are respectively located.

[0100] Specifically, for each type of food, an average area distribution table can be preset, with different food area intervals and food number intervals, according to the average food area The area coefficient Q is determined according to the food area interval, and the number coefficient N is determined according to the number of food n. For example, the area coefficient and number coefficient distribution table are shown in Table 4:

[0101] Table 4 Area coefficient and number coefficient distribution table

[0102]

[0103] After determining the area coefficient Q and the number coefficient N, different weight values ​​can be set for the area coefficient Q and the number coefficient N, and the weighted sum is used to obtain the cooking time coefficient L of the food. For example, the weight value of the area coefficient Q is 0.6, and the weight value of the number coefficient N is 0.4, which means that the area of ​​the food has a higher cooking attribute than the number of the food. The cooking time coefficient L can be determined based on formula (5):

[0104] (5)

[0105] In this embodiment, the effects of two key factors, the area of ​​ingredients and the number of ingredients, on the cooking time are comprehensively considered, and a cooking time coefficient that can accurately reflect actual cooking needs is calculated by weighted summation, which is conducive to formulating a more accurate cooking time and improving cooking effects.

[0106] In one embodiment, after S340, the method further includes: receiving a cooking time adjustment signal carrying a target cooking time, and adjusting the cooking time of the cooking time to the target cooking time.

[0107] Following the above embodiment, after adjusting the initial cooking time, the adjusted cooking time will be displayed on the operation screen 2 of the steam oven 1. , for example, in the form of a time progress bar. In addition, the user can also perform touch operations on the operation screen 2 to customize the cooking time, such as sliding the time progress bar to the target cooking time, thereby adjusting the cooking time. Adjust to target cooking time.

[0108] In this embodiment, the user can customize the cooking time based on the cooking time calculated by the cooking control device, which improves the flexibility of using the cooking device and can also meet the cooking needs of the user.

[0109] In one embodiment, Fig.10 As shown, obtaining image data inside the cooking device includes:

[0110] S110, acquiring initial image data inside the cooking device.

[0111] S120, performing distortion correction on the initial image data to obtain corrected initial image data.

[0112] S130, pre-processing the corrected initial image data to obtain image data of the interior of the cooking device.

[0113] Among them, due to the optical lens characteristics of the image acquisition device and its installation position, angle and other factors, the collected initial image data often has a certain degree of distortion.

[0114] by Figure 3 Taking the steam oven 1 in the figure as an example, the focal length of the camera 4 is usually short and fixed-focus static, that is, when the focus ring is rotated to obtain the clearest field of view, it is the best. However, when the short focal length lens (i.e., wide-angle lens) collects images inside the steam oven 1, transmission deformation will occur, forming a fisheye lens. Therefore, when the camera 4 collects the initial image data, the relative proportion change of the near and far features will cause the image to bend or deform, so the initial image data has barrel distortion, and the initial image data needs to be distorted to obtain the corrected initial image data.

[0115] The process of distortion correction may be to first identify key feature points in the initial image data (e.g., the titanium frame and the rear metal plate in the steam oven 1). These key feature points are obvious objects and lines in the initial image data, which can be used to determine the perspective relationship of the initial image data. Further, based on the identified key feature points, a mathematical model may be constructed to describe the perspective relationship of the initial image data. The mathematical model may be an affine transformation model, a transmission transformation model, or other types of transformation models. Then, the established transformation model is applied to the initial image data, and the barrel distortion is corrected by adjusting various parts of the initial image data to match the expected perspective relationship.

[0116] Furthermore, there may still be noise in the initial image data after distortion correction. In order to further improve the image quality, the corrected initial image data can be further preprocessed, such as removing noise, enhancing image contrast, enhancing details, etc. The image data obtained after preprocessing is clearer and of higher quality.

[0117] In this embodiment, by performing a series of processing such as distortion correction, denoising, and detail enhancement on the initial image data, the image quality can be effectively improved, so that the subsequent analysis process based on the image data can be more efficient and accurate, thereby improving the accuracy of the cooking parameters.

[0118] In one embodiment, Fig.11 As shown, S120 includes:

[0119] S121, identifying key feature points in the initial image data, and determining a perspective relationship model of the initial image data based on the key feature points.

[0120] S122, performing distortion correction on the initial image data according to the perspective relationship model to obtain corrected initial image data.

[0121] Following the above embodiment, in image recognition, repairing a distorted image involves perspective transformation or homography transformation, the purpose of which is to map a quadrilateral region in the image data to another quadrilateral region, thereby correcting the perspective distortion of the image.

[0122] Specifically, we can use feature point detection algorithms (such as SIFT, SURF, ORB, etc.) to detect key feature points in the initial image, and match them through descriptors to find the actual points corresponding to these key feature points under different viewing angles. Then, we can select four key feature points and their corresponding actual points to define the quadrilateral area in the initial image data. Assume that the coordinates of the four key feature points in the initial image data are , , , The corrected coordinates of these four key feature points are , , , , the coordinates of the key feature points before and after correction satisfy the following relationship:

[0123] (6)

[0124] (7)

[0125] (8)

[0126] For each key feature point pair, the following two equations can be written:

[0127] (9)

[0128] (10)

[0129] Since four pairs of key feature points are known, eight equations such as equation (9) or equation (10) can be obtained. These eight equations are combined into an 8*9 matrix A and an 8*1 zero vector B, and then the linear equation system AH=B is solved, where H is a 9-dimensional vector as shown in equation (6). Finally, the linear equation system can be solved by the least squares method to obtain the homography matrix H as the perspective relationship model.

[0130] Then, the distortion correction can be performed on the initial image data according to the perspective relationship model. Specifically, other points in the initial image data are subjected to transmission transformation according to the same transmission relationship as the key feature points, so as to obtain the corrected initial image data.

[0131] In this embodiment, a homography matrix is ​​constructed to describe the transmission transformation relationship of the image data before and after distortion correction, and then the initial image data is subjected to distortion correction, thereby providing an accurate data basis for the subsequent formulation of cooking parameters and improving the accuracy of the cooking parameters.

[0132] In order to explain the cooking device control method proposed in the present application more clearly, the following is a detailed description of the embodiment:

[0133] by Figure 3 Take the steam oven 1 in the figure as an example. If the user wants to use the steam oven 1 to cook sweet potatoes, the user puts 3 pieces of sweet potatoes on a baking tray and puts the baking tray into the steam oven 1. Through the data feedback of the pressure sensor in the steam oven 1, it can be known that the user has placed the baking tray on the second layer. By looking up the table, it is known that the vertical area ratio coefficient P=1.05. The moment the user closes the door of the steam oven, the camera 4 located on the door turns on the white balance mode and stabilizes for 1 second. After 1 second, a picture of the interior of the steam oven 1 is taken and uploaded to the cooking control device (or controller, control system). After analysis by the cooking control device, the areas of each of the 9 areas pre-divided on the storage layer are obtained, and the food area of ​​the sweet potato is calculated by combining the horizontal area ratio coefficient k corresponding to each area. On the other hand, the cooking control device recognizes that there are 3 sweet potatoes on the storage layer, that is, the number of ingredients is 3, and then the average area of ​​the sweet potatoes is obtained. , and by looking up the table, we know that the area coefficient Q=1.1 and the number coefficient N=1.0. After weighting the area coefficient Q and the number coefficient N, we get L=1.06. According to the cooking mode (steaming or baking) selected by the user, the initial cooking time is pre-written in the cooking control device. = 40 minutes, so the final cooking time displayed on the operation screen 2 is = 42.5 minutes, and there is a time progress bar on the operation screen 2. If the user wants to correct the cooking time, he can adjust the cooking time to the target cooking time by dragging the slider. Among them, the baking tray is placed on the second layer, the cooking mode does not involve the opening of the lower heating tube, and the initial cooking temperature is pre-written in the cooking control device =200 degrees Celsius, by looking up the table, we know that the cooking temperature coefficient m=102%, so the adjusted cooking temperature The temperature is 204 degrees Celsius, which is displayed on the operation screen 2. The user clicks "Start cooking" on the operation screen 2 to enter the cooking state.

[0134] In another detailed embodiment, the user turns on the steam oven 1, places two plates on the steaming tray, and five steamed buns on each plate, and puts the steaming tray into the steam oven 1. Through the data feedback of the pressure sensor, it is known that the user has placed the steaming tray on the third layer. By looking up the table, it can be known that the vertical area ratio coefficient P=1. The moment the user closes the door of the steam oven, the camera 4 located on the door turns on the white balance mode and stabilizes for 1 second. After 1 second, a picture of the interior of the steam oven 1 is taken and uploaded to the cooking control device (or controller, control system). After analysis by the cooking control device, the areas of each of the 9 areas pre-divided by the storage layer are analyzed, and the food area of ​​the steamed buns is calculated in combination with the horizontal area ratio coefficient k corresponding to each area. On the other hand, the cooking control device recognizes that there are 10 steamed buns on the storage layer, that is, the number of ingredients is 10, and then calculates the average area of ​​the steamed buns After data processing by the cooking control device, the area coefficient Q of the steamed bun is obtained to be 1.1, and the number coefficient N is 1.1. After weighting the area coefficient Q and the number coefficient N, L=1.1 is obtained. According to the cooking mode selected by the user, the initial cooking time is pre-written in the cooking control device. = 20 minutes, so the final cooking time displayed on the operation screen 2 is = 22 minutes, and there is a time progress bar on the operation screen 2. If the user wants to modify the cooking time, he can adjust the cooking time to the target cooking time by dragging the slider. The initial cooking temperature is pre-written in the cooking control device. = 100 degrees Celsius, the cooking mode of steamed buns is steaming, so keep the initial cooking temperature unchanged, that is, the cooking temperature The temperature is 100 degrees Celsius, which is displayed on the operation screen 2. The user can enter the cooking state by clicking Start Cooking on the operation screen 2.

[0135] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0136] Based on the same inventive concept, the embodiment of the present application also provides a cooking device control apparatus for implementing the cooking device control method involved above. The implementation scheme for solving the problem provided by the apparatus is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more cooking device control apparatus embodiments provided below can refer to the limitations of the cooking device control method above, and will not be repeated here.

[0137] In an exemplary embodiment, Fig.12 As shown, a cooking equipment control device 500 is provided, comprising: a data acquisition module 510, an ingredient analysis module 520, a cooking parameter determination module 530 and a cooking module 540, wherein:

[0138] The data acquisition module 510 is used to acquire image data inside the cooking device and the position of the food in the cooking device when the food is put into the cooking device; the image data includes image information of the food.

[0139] The ingredient analysis module 520 is used to determine the area and number of ingredients according to the image data and the positions of the ingredients in the cooking device.

[0140] The cooking parameter determination module 530 is used to determine the cooking parameters of the cooking device according to the position of the ingredients in the cooking device, the number of ingredients and the area of ​​the ingredients.

[0141] The cooking module 540 is used to control the cooking equipment to work according to the cooking parameters.

[0142] In one embodiment, the ingredient analysis module 520 is also used to determine the number of ingredients and the area of ​​the ingredients in the image data based on the image data, determine the area ratio coefficient based on the position of the ingredients in the cooking equipment, and determine the area of ​​the ingredients based on the area of ​​the ingredients in the image data and the area ratio coefficient.

[0143] In one embodiment, the position includes the horizontal position of the food in the horizontal direction and the vertical position of the food in the vertical direction, the area ratio coefficient includes the vertical area ratio coefficient and the horizontal area ratio coefficient, and the food analysis module 520 is also used to determine the vertical area ratio coefficient according to the vertical position of the food in the cooking device, and determine the horizontal area ratio coefficient according to the horizontal position of the food in the cooking device.

[0144] In one embodiment, the cooking parameters include cooking temperature and cooking time. The cooking parameter determination module 530 is also used to determine the cooking temperature coefficient of the ingredients according to the position of the ingredients in the cooking device, determine the cooking time coefficient of the ingredients according to the number of ingredients and the area of ​​the ingredients, adjust the preset initial cooking temperature of the cooking device according to the cooking temperature coefficient, determine the cooking temperature of the cooking device, adjust the preset initial cooking time of the cooking device according to the cooking time coefficient, and determine the cooking time of the cooking device.

[0145] In one embodiment, the cooking parameter determination module 530 is also used to determine the area coefficient corresponding to the area of ​​the food according to the area range of the food, determine the number coefficient corresponding to the number of ingredients according to the number range of the food, and determine the cooking time coefficient of the food according to the number of ingredients, the area of ​​the food, the area coefficient and the number coefficient.

[0146] In one embodiment, the cooking device control apparatus 500 is further configured to receive a cooking time adjustment signal carrying a target cooking time, and adjust the cooking time of the cooking device to the target cooking time.

[0147] In one embodiment, the data acquisition module 510 is used to acquire initial image data inside the cooking device, perform distortion correction on the initial image data to obtain corrected initial image data, and preprocess the corrected initial image data to obtain image data inside the cooking device.

[0148] In one embodiment, the data acquisition module 510 is used to identify key feature points in the initial image data, determine the perspective relationship model of the initial image data based on the key feature points, and perform distortion correction on the initial image data according to the perspective relationship model to obtain the corrected initial image data.

[0149] Each module in the cooking equipment control device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0150] In one embodiment, the present application proposes a cooking control device for controlling a cooking device according to a cooking device control method in any one of the above cooking device control method embodiments, which will not be described in detail herein.

[0151] In one embodiment, the present application provides a cooking device 600, such as Fig.13 As shown, the cooking device 600 includes a cooking control device 610, a pressure sensor 620 and an image acquisition device 630 connected to the cooking control device 610. The pressure sensor 620 is used to acquire the position of food in the cooking device 600 and send it to the cooking control device 610. The image acquisition device 630 is used to acquire image data inside the cooking device 600 and send it to the cooking control device 610. Figure 3 Taking the steam oven 1 in the example, the pressure sensing device 620 is equivalent to the pressure sensor 3-2-5 in the steam oven 1, and the image acquisition device 630 is equivalent to the camera 4 in the steam oven 1.

[0152] In one embodiment, Fig.14 As shown, the cooking device 600 further includes a touch device 640 connected to the cooking control device 610, which is used to generate and send a cooking time adjustment signal carrying a target cooking time to the cooking control device 610 in response to a touch operation. Figure 3 Taking the steam oven 1 in the figure as an example, the touch device 640 is equivalent to the operation screen 2. While responding to the user's touch operation, the touch device 640 can also serve as a display device to present relevant cooking parameters to the user.

[0153] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.15 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as image data inside the cooking device. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a cooking device control method is implemented.

[0154] Those skilled in the art will understand that Fig.14 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0155] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps in any one of the above cooking device control method embodiments are implemented.

[0156] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in any one of the above cooking device control method embodiments are implemented.

[0157] In one embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps in any one of the above cooking device control method embodiments.

[0158] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0159] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0160] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0161] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A cooking equipment control method, characterized in that: The method comprises: When food is placed in a cooking device, image data of the interior of the cooking device and the position of the food in the cooking device are acquired; the image data includes image information of the food; Determine the area and number of ingredients according to the image data and the position of the ingredients in the cooking device; Determining cooking parameters of the cooking device according to the position of the food in the cooking device, the number of the food, and the area of ​​the food; The cooking device is controlled to operate according to the cooking parameters.

2. The method according to claim 1, characterized in that The step of determining the area and number of ingredients according to the image data and the positions of the ingredients in the cooking device comprises: Determine the number of ingredients and the area of ​​the ingredients in the image data according to the image data; Determining an area ratio coefficient according to the position of the food in the cooking device; The area of ​​the food is determined based on the area of ​​the food in the image data and the area ratio coefficient.

3. The method according to claim 2, characterized in that The position includes the horizontal position of the food in the horizontal direction and the vertical position in the vertical direction, and the area ratio coefficient includes the vertical area ratio coefficient and the horizontal area ratio coefficient; The determining of the area ratio coefficient according to the position of the food in the cooking device includes: Determining a vertical area ratio coefficient according to a vertical position of the food in the cooking device; A horizontal area ratio coefficient is determined according to the horizontal position of the food in the cooking device.

4. The method according to claim 1, characterized in that: The cooking parameters include cooking temperature and cooking time. The determining the cooking parameters of the cooking device according to the position of the food in the cooking device, the number of the food, and the area of ​​the food includes: determining a cooking temperature coefficient of the food according to a position of the food in the cooking device; Determining a cooking time coefficient of the ingredients according to the number of ingredients and the area of ​​the ingredients; According to the cooking temperature coefficient, adjusting a preset initial cooking temperature of the cooking device to determine a cooking temperature of the cooking device; According to the cooking time coefficient, the preset initial cooking time of the cooking device is adjusted to determine the cooking time of the cooking device.

5. The method according to claim 4, characterized in that The step of determining the cooking time coefficient of the ingredients according to the number of ingredients and the area of ​​the ingredients includes: Determining an area coefficient corresponding to the food area according to the food area interval in which the food area is located; Determining a number coefficient corresponding to the number of ingredients according to the number of ingredients interval in which the number of ingredients is located; The cooking time coefficient of the ingredients is determined according to the number of ingredients, the area of ​​the ingredients, the area coefficient and the number coefficient.

6. The method according to claim 4, characterized in that After adjusting the preset initial cooking time of the cooking device according to the cooking time coefficient and determining the cooking time of the cooking device, the method further includes: receiving a cooking time adjustment signal carrying a target cooking time; The cooking time of the cooking time is adjusted to the target cooking time.

7. The method according to any one of claims 1 to 6, characterized in that: The acquiring of image data inside the cooking device comprises: Acquiring initial image data of the interior of the cooking device; Performing distortion correction on the initial image data to obtain corrected initial image data; The corrected initial image data is preprocessed to obtain image data of the interior of the cooking device.

8. The method according to claim 7, characterized in that The step of performing distortion correction on the initial image data to obtain corrected initial image data includes: Identifying key feature points in the initial image data; Based on the key feature points, determining a perspective relationship model of the initial image data; According to the perspective relationship model, the initial image data is subjected to distortion correction to obtain corrected initial image data.

9. A cooking equipment control device, characterized in that: The device comprises: A data acquisition module, used to acquire image data inside the cooking device and the position of the food in the cooking device when food is placed in the cooking device; the image data includes image information of the food; An ingredient analysis module, used to determine the area and number of ingredients according to the image data and the position of the ingredients in the cooking device; a cooking parameter determination module, configured to determine the cooking parameters of the cooking device according to the position of the food in the cooking device, the number of the food and the area of ​​the food; The cooking module is used to control the cooking device to work according to the cooking parameters.

10. A cooking control device, characterized in that: The cooking control device is used to control the cooking device according to the cooking device control method according to any one of claims 1 to 8.

11. A cooking device, characterized in that: The cooking control device comprises a cooking control device, a pressure sensing device and an image acquisition device connected to the cooking control device. The pressure sensing device is used to acquire the position of food in the cooking device and send it to the cooking control device. The image acquisition device is used to acquire image data inside the cooking device and send it to the cooking control device.

12. The cooking device according to claim 11, characterized in that The cooking device further comprises a touch control device connected to the cooking control device, for generating and sending a cooking time adjustment signal carrying a target cooking time to the cooking control device in response to a touch control operation.

Images