Method and device for obtaining cooking data in cooking process and cooking equipment

Abstract

The invention discloses a method and a device for acquiring cooking data in a cooking process and cooking equipment. The method comprises the following steps: respectively acquiring a standard RGB image and a standard IR image in a standard cooking process; according to the convolutional neural network, identifying the time point of adding the food material, the added food material and the region of interest in the element steps in the standard RGB image, and according to the standard IR image, obtaining the temperature of the region of interest; and obtaining standard cooking data of a standard cooking process according to the time point of adding the food materials, the added food materials and the temperature of the region of interest, and establishing a corresponding relationship between the element steps and the standard cooking data. The food material adding time point, the added food material and the region of interest in the element steps in the standard RGB image can be identified, and the temperature of the region of interest is acquired according to the standard IR image, so that the standard cooking data of the standard cooking process is acquired, the food material adding time and the food material temperature can be digitalized, and quantitative indexes are formed.

Description

Technical Field

[0001] The present application relates to the field of cooking technology, and in particular to a method, device and cooking equipment for acquiring cooking data during a cooking process. Background Art

[0002] For chain restaurant brands, it is very difficult to ensure the uniformity of food quality between stores. Restaurant brands will formulate strict SOPs (Standard Operating Procedures) for each dish. However, the SOPs of general restaurant brands lack quantitative indicators. For example, the oil temperature is usually only a few degrees of oil temperature, and it is difficult to describe it with text to be particularly accurate. The time to add ingredients and the way to stir-fry also need to be digitized. Summary of the invention

[0003] The technical problem to be solved by the present application is to provide a method, device and cooking equipment for obtaining cooking data during the cooking process in view of the deficiencies in the prior art.

[0004] According to a first aspect of the present application, the present application provides a method for acquiring cooking data in a cooking process, comprising:

[0005] Acquire a standard RGB image and a standard IR image of a standard cooking process respectively, and divide the standard cooking process into a plurality of meta-steps, wherein the meta-steps at least include heating a pan, heating oil, sautéing and / or stir-frying;

[0006] Aligning the standard RGB image and the standard IR image;

[0007] identifying the time point of adding ingredients, the added ingredients and the region of interest in the meta-step in the standard RGB image according to the convolutional neural network, and acquiring the temperature of the region of interest according to the standard IR image, wherein the region of interest includes a pot and / or ingredients in the pot;

[0008] The standard cooking data of the standard cooking process is acquired according to the time point of adding the ingredients, the added ingredients and the temperature of the region of interest, and a corresponding relationship between the meta-steps and the standard cooking data is established.

[0009] In the above method, aligning the standard RGB image and the standard IR image comprises:

[0010] Input the standard RGB image and the standard IR image as single frames respectively;

[0011] After the standard RGB image and the standard IR image are aligned in time according to the timestamp, the standard RGB image and the standard IR image are aligned in pixels.

[0012] The above method, wherein the time point of adding ingredients, the added ingredients and the region of interest in the meta-step in the standard RGB image are identified according to a convolutional neural network, and the temperature of the region of interest is acquired according to the standard IR image, wherein the region of interest includes a pot and / or ingredients in the pot, comprises:

[0013] When there is no food in the pot, the region of interest includes the pot;

[0014] When the temperature of the pot rises and exceeds the first preset value, it is a step of heating the pot.

[0015] The above method, wherein the time point of adding ingredients, the added ingredients and the region of interest in the meta-step in the standard RGB image are identified according to a convolutional neural network, and the temperature of the region of interest is acquired according to the standard IR image, wherein the region of interest includes a pot and / or ingredients in the pot, comprises:

[0016] When there are ingredients in the pot, the region of interest includes the ingredients in the pot;

[0017] When the food in the pot is oil and the oil temperature exceeds the second preset value, it is the oil heating step.

[0018] The above method, wherein the time point of adding ingredients, the added ingredients and the region of interest in the meta-step in the standard RGB image are identified according to a convolutional neural network, and the temperature of the region of interest is acquired according to the standard IR image, wherein the region of interest includes a pot and / or ingredients in the pot, comprises:

[0019] When the ingredients in the pot are small ingredients and the temperature of the small ingredients exceeds the third preset value, it is the step of stir-frying.

[0020] The above method, wherein the time point of adding ingredients, the added ingredients and the region of interest in the meta-step in the standard RGB image are identified according to a convolutional neural network, and the temperature of the region of interest is acquired according to the standard IR image, wherein the region of interest includes a pot and / or ingredients in the pot, comprises:

[0021] When the area of ​​the food in the pot occupies an area of ​​the pot that is greater than a fourth preset value, and the food in the pot changes from being gathered together to being spread out or from being spread out to being gathered together, it is a stir-frying step.

[0022] In the above method, the stir-frying step further comprises:

[0023] The stir-frying range is obtained based on the ratio of the maximum and minimum coverage areas of the ingredients;

[0024] The number of stir-frying times is obtained according to the frequency of periodic changes in the coverage area of ​​the ingredients.

[0025] The above method further includes:

[0026] Also includes:

[0027] Respectively obtain an RGB image and an IR image of the chef during the cooking process of the chef, and align the RGB image and the IR image of the chef;

[0028] identifying the time point of adding ingredients, the added ingredients and the region of interest in the meta-step in the chef RGB image according to a convolutional neural network, and acquiring the temperature of the region of interest according to the chef IR image;

[0029] Acquire chef cooking data of the chef's cooking process according to the time point when the chef adds ingredients during the cooking process, the added ingredients and the temperature of the region of interest, and establish a corresponding relationship between the meta-step and the chef's cooking data;

[0030] The chef's cooking data and the standard cooking data are compared to obtain a difference between the chef's cooking data and the standard cooking data in the meta-step.

[0031] According to a second aspect of the present application, the present application provides a device for acquiring cooking data in a cooking process, comprising:

[0032] An image acquisition module, used to respectively acquire a standard RGB image and a standard IR image of a standard cooking process, and divide the standard cooking process into a plurality of elementary steps, wherein the elementary steps at least include heating a pan, heating oil, sautéing and / or stir-frying;

[0033] Aligning the standard RGB image and the standard IR image;

[0034] An image processing module, used for aligning the standard RGB image and the standard IR image;

[0035] a temperature processing module, for identifying the time point of adding ingredients, the added ingredients and the region of interest in the meta-step in the standard RGB image according to a convolutional neural network, and acquiring the temperature of the region of interest according to the standard IR image, wherein the region of interest includes a pot and / or ingredients in the pot;

[0036] The standard processing module is used to obtain standard cooking data of the standard cooking process according to the time point of adding ingredients, the added ingredients and the temperature of the area of ​​interest, and establish a corresponding relationship between the meta-steps and the standard cooking data.

[0037] According to a third aspect of the present application, the present application provides a cooking device, comprising the above-mentioned device for acquiring cooking data during the cooking process.

[0038] Due to the adoption of the above technical solution, the beneficial effects of this application are:

[0039] The method, device and cooking equipment for obtaining cooking data in a cooking process provided by an embodiment of the present application include: obtaining a standard RGB image and a standard IR image of a standard cooking process respectively; identifying the time point of adding ingredients, the added ingredients and the region of interest in a meta-step in the standard RGB image according to a convolutional neural network, and obtaining the temperature of the region of interest according to the standard IR image; obtaining standard cooking data of the standard cooking process according to the time point of adding ingredients, the added ingredients and the temperature of the region of interest, and establishing a correspondence between the meta-step and the standard cooking data. Since the present application can identify the time point of adding ingredients, the added ingredients and the region of interest in a meta-step in a standard RGB image, and obtain the temperature of the region of interest according to the standard IR image, thereby obtaining standard cooking data of the standard cooking process, the time of adding ingredients and the temperature of ingredients can be digitized to form quantitative indicators. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A flowchart of a method according to an embodiment of the present application in one implementation manner;

[0041] Figure 2 A flowchart of a method according to an embodiment of the present application in another implementation manner;

[0042] Figure 3 A flowchart of a method according to another embodiment of the present application;

[0043] Figure 4 A schematic diagram of a program module of an apparatus according to an embodiment of the present application in one implementation manner. DETAILED DESCRIPTION

[0044] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0045] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0046] The serial numbers assigned to the components in this article, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning.

[0047] In addition, the technical features and technical solutions described herein may be combined in any suitable manner in one or more embodiments. It is easy for a person skilled in the art to understand that the steps or operation sequence of the methods related to the embodiments provided herein may also be changed. Therefore, any sequence in the drawings and embodiments is only for illustrative purposes and does not imply a requirement to follow a certain sequence unless it is explicitly stated that a certain sequence is required.

[0048] like Figure 1 As shown, the method for obtaining cooking data in a cooking process provided in an embodiment of the present application, one implementation manner of which includes the following steps:

[0049] Step 101: Obtain a standard RGB image and a standard IR image of a standard cooking process, respectively. The standard cooking process is divided into multiple meta-steps, and the meta-steps at least include heating the pan, heating the oil, sautéing and / or stir-frying. Select the cooking process of an experienced and reputable chef as the standard cooking process, use an RGB camera to capture a standard RGB image of the standard cooking process, and use an IR camera to capture a standard IR image of the standard cooking process.

[0050] Step 102: Aligning the standard RGB image and the standard IR image. Aligning the standard RGB image and the standard IR image refers to aligning the standard RGB image and the standard IR image in terms of time and pixels.

[0051] In one embodiment, Figure 2 As shown, step 102 may include the following steps:

[0052] Step 1021: inputting a standard RGB image and a standard IR image as single frames respectively;

[0053] Step 1022: After aligning the standard RGB image and the standard IR image in time according to the timestamp, align the standard RGB image and the standard IR image in pixels.

[0054] Step 103: Identify the time point of adding ingredients, the added ingredients, and the region of interest in the meta-step in the standard RGB image according to the convolutional neural network, and obtain the temperature of the region of interest according to the standard IR image, where the region of interest includes the pot and / or the ingredients in the pot.

[0055] In one embodiment, a single-frame standard IR image and a standard RGB image can be segmented to find the key temperature of the corresponding meta-step. For example, for the hot pot stage, the highest temperature of the pot can be extracted; for the hot oil and stir-fry stage, the oil temperature can be extracted; for the stir-fry stage, the temperature of the ingredients can be extracted, and then the time and temperature curves of each meta-step can be summarized.

[0056] In one implementation, step 103 may include the following steps:

[0057] When there is no food in the pot, the region of interest includes the pot;

[0058] When the temperature of the pot rises and exceeds a first preset value, it is a step of heating the pot, wherein the first preset value is an empirical value and can be preset as needed.

[0059] In one implementation, step 103 may further include the following steps:

[0060] When there are ingredients in the pot, the region of interest includes the ingredients in the pot;

[0061] When the food in the pot is oil and the oil temperature exceeds the second preset value, it is the oil heating step. The second preset value is also an empirical value and can be preset as needed.

[0062] In one implementation, step 103 may further include the following steps:

[0063] When the ingredients in the pot are small ingredients and the temperature of the small ingredients exceeds the third preset value, it is the step of sautéing. The third preset value is also an empirical value and can be pre-set as needed. The third preset value is also an empirical value and can be pre-set as needed.

[0064] In one implementation, step 103 may further include the following steps:

[0065] When the area of ​​the food in the pot occupies more than the fourth preset value, and the food in the pot changes from being gathered to being spread out or from being spread out to being gathered, it is a stir-frying step. The fourth preset value is also an empirical value and can be preset as needed.

[0066] For the stir-frying step, the amplitude and number of stir-frying required in the standard cooking process can be finally obtained according to the gathering / spreading ratio and frequency of the extracted food.

[0067] The stir-frying step may also include:

[0068] The stir-frying range is obtained based on the ratio of the maximum and minimum coverage areas of the ingredients;

[0069] The number of stir-frying times is obtained according to the frequency of periodic changes in the coverage area of ​​the ingredients.

[0070] Step 104: Obtain the cooking standard of the standard cooking process according to the time point of adding the ingredients, the added ingredients and the temperature of the region of interest, and establish a corresponding relationship between the meta-steps and the cooking standard.

[0071] like Figure 3 As shown, another implementation of the method for obtaining cooking data in a cooking process provided in an embodiment of the present application may include the following steps:

[0072] Step 301: Obtain a standard RGB image and a standard IR image of a standard cooking process, respectively. The standard cooking process is divided into multiple meta-steps, and the meta-steps at least include heating the pan, heating the oil, sautéing and / or stir-frying. Select the cooking process of an experienced and reputable chef as the standard cooking process, use an RGB camera to capture a standard RGB image of the standard cooking process, and use an IR camera to capture a standard IR image of the standard cooking process.

[0073] Step 302: Aligning the standard RGB image and the standard IR image. Aligning the standard RGB image and the standard IR image refers to aligning the standard RGB image and the standard IR image in terms of time and pixels.

[0074] Step 303: Identify the time point of adding ingredients, the added ingredients and the region of interest in the meta-step in the standard RGB image according to the convolutional neural network, and obtain the temperature of the region of interest according to the standard IR image, where the region of interest includes the pot and / or the ingredients in the pot.

[0075] Step 304: Obtain the cooking standard of the standard cooking process according to the time point of adding the ingredients, the added ingredients and the temperature of the area of ​​interest, and establish a corresponding relationship between the meta-steps and the cooking standard.

[0076] Step 305: respectively obtain the chef's RGB image and the chef's IR image during the chef's cooking process, and align the chef's RGB image and the chef's IR image;

[0077] Step 306: Identify the time point of adding ingredients, the added ingredients, and the region of interest in the meta-step in the chef's RGB image using a convolutional neural network, and obtain the temperature of the region of interest based on the chef's IR image;

[0078] Step 307: Obtain the chef's cooking data of the chef's cooking process according to the time point when the ingredients are added during the cooking process, the added ingredients, and the temperature of the area of ​​interest, and establish a corresponding relationship between the meta-steps and the chef's cooking data;

[0079] Step 308: Compare the chef's cooking data with the cooking standard to obtain the difference between the chef's cooking data and the cooking standard in the meta-step.

[0080] like Figure 4 As shown, an apparatus for obtaining cooking data in a cooking process provided in an embodiment of the present application, one implementation manner thereof includes:

[0081] The image acquisition module 410 is used to respectively acquire a standard RGB image and a standard IR image of a standard cooking process, and divide the standard cooking process into a plurality of elementary steps, wherein the elementary steps at least include heating the pan, heating the oil, sautéing and / or stir-frying;

[0082] An image processing module 420, for aligning a standard RGB image and a standard IR image;

[0083] The temperature processing module 430 is used to identify the time point of adding ingredients, the added ingredients and the region of interest in the meta-step in the standard RGB image according to the convolutional neural network, and obtain the temperature of the region of interest according to the standard IR image, where the region of interest includes the pot and / or the ingredients in the pot;

[0084] The standard processing module 440 is used to obtain standard cooking data of the standard cooking process according to the time point of adding ingredients, the added ingredients and the temperature of the area of ​​interest, and establish a corresponding relationship between the meta-steps and the standard cooking data.

[0085] An embodiment of the present application also provides a cooking device, including the above-mentioned device for obtaining a temperature curve in a standard cooking process.

[0086] An embodiment of the present application also provides an electronic device, one implementation of which includes a memory and a processor.

[0087] Memory, used to store programs;

[0088] The processor is used to implement the above method by executing the program stored in the memory.

[0089] The above contents are further detailed descriptions of the present application in combination with specific implementation methods, and it cannot be determined that the specific implementation of the present application is limited to these descriptions. For ordinary technicians in the technical field to which the present application belongs, several simple deductions or substitutions can be made without departing from the concept of the present application.

Claims

1. A method for obtaining cooking data during a cooking process, characterized in that: include: Acquire a standard RGB image and a standard IR image of a standard cooking process respectively, and divide the standard cooking process into a plurality of meta-steps, wherein the meta-steps at least include heating a pan, heating oil, sautéing and / or stir-frying; Aligning the standard RGB image and the standard IR image; identifying the time point of adding ingredients, the added ingredients and the region of interest in the meta-step in the standard RGB image according to the convolutional neural network, and acquiring the temperature of the region of interest according to the standard IR image, wherein the region of interest includes a pot and / or ingredients in the pot; The standard cooking data of the standard cooking process is acquired according to the time point of adding the ingredients, the added ingredients and the temperature of the region of interest, and a corresponding relationship between the meta-steps and the standard cooking data is established.

2. The method according to claim 1, characterized in that The aligning the standard RGB image and the standard IR image comprises: Input the standard RGB image and the standard IR image as single frames respectively; After the standard RGB image and the standard IR image are aligned in time according to the timestamp, the standard RGB image and the standard IR image are aligned in pixels.

3. The method according to claim 2, characterized in that The step of identifying the time point of adding ingredients, the added ingredients, and the region of interest in the meta-step in the standard RGB image according to the convolutional neural network, and acquiring the temperature of the region of interest according to the standard IR image, wherein the region of interest includes a pot and / or ingredients in the pot, comprises: When there is no food in the pot, the region of interest includes the pot; When the temperature of the pot rises and exceeds the first preset value, it is a step of heating the pot.

4. The method according to claim 3, characterized in that The step of identifying the time point of adding ingredients, the added ingredients, and the region of interest in the meta-step in the standard RGB image according to the convolutional neural network, and acquiring the temperature of the region of interest according to the standard IR image, wherein the region of interest includes a pot and / or ingredients in the pot, comprises: When there are ingredients in the pot, the region of interest includes the ingredients in the pot; When the food in the pot is oil and the oil temperature exceeds the second preset value, it is the oil heating step.

5. The method according to claim 2, characterized in that The step of identifying the time point of adding ingredients, the added ingredients, and the region of interest in the meta-step in the standard RGB image according to the convolutional neural network, and acquiring the temperature of the region of interest according to the standard IR image, wherein the region of interest includes a pot and / or ingredients in the pot, comprises: When the ingredients in the pot are small ingredients and the temperature of the small ingredients exceeds the third preset value, it is the step of stir-frying.

6. The method according to claim 2, characterized in that The step of identifying the time point of adding ingredients, the added ingredients, and the region of interest in the meta-step in the standard RGB image according to the convolutional neural network, and acquiring the temperature of the region of interest according to the standard IR image, wherein the region of interest includes a pot and / or ingredients in the pot, comprises: When the area of ​​the food in the pot occupies an area of ​​the pot that is greater than a fourth preset value, and the food in the pot changes from being gathered together to being spread out or from being spread out to being gathered together, it is a stir-frying step.

7. The method according to claim 6, characterized in that The stir-frying step also includes: The stir-frying range is obtained based on the ratio of the maximum and minimum coverage areas of the ingredients; The number of stir-frying times is obtained according to the frequency of periodic changes in the coverage area of ​​the ingredients.

8. The method according to any one of claims 1 to 7, characterized in that Also includes: Respectively obtain an RGB image and an IR image of the chef during the cooking process of the chef, and align the RGB image and the IR image of the chef; identifying the time point of adding ingredients, the added ingredients and the region of interest in the meta-step in the chef RGB image according to a convolutional neural network, and acquiring the temperature of the region of interest according to the chef IR image; Acquire chef cooking data of the chef's cooking process according to the time point when the chef adds ingredients during the cooking process, the added ingredients and the temperature of the region of interest, and establish a corresponding relationship between the meta-step and the chef's cooking data; The chef's cooking data and the standard cooking data are compared to obtain a difference between the chef's cooking data and the standard cooking data in the meta-step.

9. A device for acquiring cooking data during a cooking process, characterized in that: include: An image acquisition module, used to respectively acquire a standard RGB image and a standard IR image of a standard cooking process, and divide the standard cooking process into a plurality of elementary steps, wherein the elementary steps at least include heating a pan, heating oil, sautéing and / or stir-frying; Aligning the standard RGB image and the standard IR image; An image processing module, used for aligning the standard RGB image and the standard IR image; a temperature processing module, for identifying the time point of adding ingredients, the added ingredients and the region of interest in the meta-step in the standard RGB image according to a convolutional neural network, and acquiring the temperature of the region of interest according to the standard IR image, wherein the region of interest includes a pot and / or ingredients in the pot; The standard processing module is used to obtain standard cooking data of the standard cooking process according to the time point of adding ingredients, the added ingredients and the temperature of the area of ​​interest, and establish a corresponding relationship between the meta-steps and the standard cooking data.

10. A cooking device, characterized in that: The method comprises the device for acquiring cooking data during a cooking process as claimed in claim 9.

Images