Cooking device for detecting fire hazard and control method thereof

Abstract

There is provided a cooking apparatus (100) comprising: a cooking chamber (230) for accommodating an object to be cooked; a camera (220) for capturing an image of the interior of the cooking chamber; an output interface (250); a memory (240) to store at least one instruction; and at least one processor (210). The at least one processor (210), by executing at least one instruction to: detect a fire hazard during cooking based on a captured image captured by the camera (220); if a fire hazard is detected, a fire hazard notification is output via an output interface (250).

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a cooking device for detecting fire hazards, a method for controlling the cooking device, and a computer-readable recording medium having a computer program recorded thereon, which, when executed by a computer, executes the method for controlling the cooking device. Background Art

[0002] Among all household appliances, cooking devices have the highest fire risk. During cooking using a cooking device, a fire may occur in the cooking device when oil with a low smoke point is selected as an ingredient. In addition, a fire may occur when an object that should not be placed inside a cooking device (e.g., a microwave oven) is accidentally placed.

[0003] In the case of cooking devices such as ovens, when the oven door is opened, fire alarms are often triggered unnecessarily in the house due to smoke generated during cooking. In addition, smoke generated during cooking can cover the food being cooked, making it difficult to inspect the interior of the oven and to check the status of the cooking process. Summary of the invention

[0004] Technical solutions to the problem

[0005] According to an embodiment of the present disclosure, a cooking device includes: a cooking chamber for accommodating a cooking object to be cooked; a camera configured to photograph the interior of the cooking chamber; an output interface; a memory configured to store at least one instruction; and at least one processor, wherein the at least one processor is configured to execute the at least one instruction to: detect a fire hazard during cooking based on a captured image obtained by the camera; and output a fire hazard notification through the output interface based on the detection of the fire hazard.

[0006] According to an embodiment of the present disclosure, a method for controlling a cooking device, wherein the cooking device includes a camera that photographs the interior of a cooking chamber that accommodates an object to be cooked, the method comprising: photographing the interior of the cooking chamber by using the camera; detecting a fire hazard during cooking based on a captured image obtained by photographing the interior of the cooking chamber; and outputting a fire hazard notification based on the detection of the fire hazard.

[0007] According to an embodiment of the present disclosure, a computer readable recording medium has recorded thereon a computer program which, when executed by a computer, performs a method for controlling a cooking device. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a diagram illustrating the operation of a cooking device according to an embodiment of the present disclosure.

[0009] Figure 2is a block diagram of a structure of a cooking device according to an embodiment of the present disclosure.

[0010] Figure 3 The structure of a cooking device according to an embodiment of the present disclosure is shown.

[0011] Figure 4 is a flowchart of a method of controlling a cooking device according to an embodiment of the present disclosure.

[0012] Figure 5 A monitoring area according to an embodiment of the present disclosure is shown.

[0013] Figure 6 is a flowchart of a method of controlling a cooking device according to an embodiment of the present disclosure.

[0014] Figure 7a is a flowchart illustrating a process of detecting fire hazards based on captured images according to an embodiment of the present disclosure.

[0015] Figure 7b is a diagram illustrating a process of comparing a monitoring area with fire-related colors in a color space according to an embodiment of the present disclosure.

[0016] Figure 8 A process of detecting smoke or flame based on color distribution according to an embodiment of the present disclosure is shown.

[0017] Fig. 9 is a block diagram of a structure of a cooking device according to an embodiment of the present disclosure.

[0018] Fig.10 is a flowchart of a process of detecting a fire hazard based on a captured image and a temperature detection value according to an embodiment of the present disclosure.

[0019] Fig.11 The process of identifying food from a captured image and determining a monitoring area according to an embodiment of the present disclosure is shown.

[0020] Fig.12 is a diagram illustrating a structure of recognizing ingredients by using a machine learning model according to an embodiment of the present disclosure.

[0021] Fig.13 A monitoring area according to a setup of an embodiment of the present disclosure is shown.

[0022] Fig.14 is a block diagram illustrating a process of applying one or more weights to a monitoring area according to an embodiment of the present disclosure.

[0023] Fig.15 is a flowchart of a process of selecting a monitoring area according to an embodiment of the present disclosure.

[0024] Fig.16 is a diagram illustrating shrinkage of food according to an embodiment of the present disclosure.

[0025] Fig.17 is a flowchart of a process of determining a monitoring area in consideration of shrinkage of food according to an embodiment of the present disclosure.

[0026] Fig.18 is a block diagram of a structure of a cooking device according to an embodiment of the present disclosure.

[0027] Fig.19 A cooking device, a user device, and a server according to an embodiment of the present disclosure are shown.

[0028] Fig. 20 is a diagram illustrating a process of obtaining food information from a user device according to an embodiment of the present disclosure.

[0029] Fig.21 is a diagram illustrating a process of obtaining food information from a user device according to an embodiment of the present disclosure.

[0030] Fig. 22 is a diagram illustrating a process of receiving food information through an input interface of a cooking device according to an embodiment of the present disclosure.

[0031] Fig.23 A process of setting a monitoring area based on color data of a captured image according to an embodiment of the present disclosure is shown.

[0032] Fig.24 is a flow chart of a process of detecting fire hazards according to an embodiment of the present disclosure.

[0033] Fig.25 is an example of a situation where a flame is detected in a monitoring area according to an embodiment of the present disclosure.

[0034] Fig.26 is a diagram illustrating a process of outputting a fire hazard according to an embodiment of the present disclosure.

[0035] Fig. 27 is a diagram illustrating a process of outputting a fire hazard through a user device according to an embodiment of the present disclosure.

[0036] Fig.28 is a block diagram of a structure of a cooking device according to an embodiment of the present disclosure.

[0037] Fig.29 is a diagram illustrating a process of monitoring a cooking operation of a cooking device according to an embodiment of the present disclosure.

[0038] Fig.30 is a block diagram of a structure of a cooking device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0039] The various embodiments and terms used herein are not intended to limit the technical features mentioned herein to specific embodiments, and it should be understood that the present disclosure covers all changes, equivalents and / or substitutions that do not depart from the spirit and technical scope.

[0040] Regarding the description of the drawings, like reference numbers indicate similar or related elements.

[0041] Unless the relevant context clearly dictates otherwise, the singular form of a noun corresponding to an item may include one item or a plurality of items.

[0042] The expressions "A or B", "at least one of A and B", "at least one of A or B", and "one or more of A and / or B", "A, B or C", "at least one of A, B and C", and "at least one of A, B or C" may include any one of the listed items and all of at least one combination of these items.

[0043] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0044] Terms such as "first", "second", "primary" or "secondary" used herein may refer to various elements regardless of order and / or importance, and do not limit the corresponding elements. These terms may be used to distinguish one element from another.

[0045] When an element (e.g., a first element) is mentioned as being "operably" or "communicably" "coupled to" or "connected to" another element (e.g., a second element) with or without the term "operably" or "communicably", this may mean that the element may be directly (e.g., by wire) or wirelessly coupled to the other element, or another element (e.g., a third element) may be present in between.

[0046] The terms such as “including”, “having” and “comprising” used herein are intended to indicate the presence of the features, numbers, steps, actions, components, parts or combinations thereof disclosed in the present specification, but are not intended to exclude the possibility that one or more other features, numbers, steps, actions, components, parts or combinations thereof may exist or may be added.

[0047] When a component is referred to as being "connected to," "coupled to," "supported by," or "contacting" another component, this may include not only a case where the components are directly connected, coupled, supported, or contacting each other, but also a case where the components are indirectly connected, coupled, supported, or contacting each other via a third component.

[0048] When a component is referred to as being "on" another component, this includes not only a case where the component is in contact with the other component but also a case where the other component exists between the two components.

[0049] The operating principles of embodiments of the present disclosure and various embodiments of the present disclosure will now be described more fully with reference to the accompanying drawings.

[0050] Figure 1 is a diagram illustrating the operation of a cooking device according to an embodiment of the present disclosure.

[0051] According to an embodiment of the present disclosure, a camera may be provided inside or adjacent to the cooking device 100, the camera being configured to photograph the interior of the cooking device 100. In one or more examples, a fire hazard may be detected based on a captured image 110 obtained by photographing the interior. In one or more examples, the cooking device 100 may include a camera that photographs the interior. The camera may detect a fire risk of the cooking device 100 based on the captured image 110 obtained by photographing the interior of the cooking device. When a fire risk is detected, the cooking device 100 may output a fire hazard notification 120. The cooking device 100 may output the fire hazard notification 120 through an output interface of the cooking device 100, or may output the fire hazard notification 120 through an external device. The fire hazard notification may be an auditory notification or a visual notification (e.g., a text or symbol output to a display).

[0052] The cooking device 100 may include special sensors for detecting fire risks, such as CO 2 A cooking device 100 may include a camera, a temperature sensor, a smoke sensor, or any other suitable sensor known to a person of ordinary skill in the art. According to an embodiment of the present disclosure, the cooking device 100 may detect a fire hazard by using a camera inside the cooking device 100 without including one of the aforementioned sensors for determining a fire. The cooking device 100 may perform additional operations such as monitoring a cooking process, recommending a recipe, and first hazard detection by using an internal camera. According to an embodiment of the present disclosure, the cooking device 100 with a camera may detect a fire hazard without a special sensor and inform the user of the fire hazard.

[0053] Figure 2 is a block diagram of a structure of a cooking device according to an embodiment of the present disclosure.

[0054] The cooking device 100 according to an embodiment of the present disclosure includes various types of cooking devices 100 that cook in a high temperature environment or cook by heating food. The cooking device 100 can be implemented in the form of, for example, an oven, a microwave oven, an air fryer, a smart cooker, a toaster, or any other suitable cooking device known to those of ordinary skill in the art.

[0055] According to an embodiment of the present disclosure, the cooking apparatus 100 may include a processor 210 , a camera 220 , a cooking chamber 230 , a memory 240 , and an output interface 250 .

[0056] According to an embodiment, the processor 210 may control all operations of the cooking device 100. The processor 210 may be implemented as one or more processors. The processor 210 may perform a specific operation by executing instructions or commands stored in the memory 240. The processor 210 may control the operations of components included in the cooking device 100. The processor 210 may include at least one of a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), or a combination thereof.

[0057] The camera 220 performs photoelectric conversion on the incident light to generate an electrical image signal. The camera 220 may include at least one lens, a lens driving unit, and an image sensor. The camera 220 may be configured to shoot the interior of the cooking chamber 230. For example, the camera 220 may be disposed on the ceiling of the cooking chamber, on the door 310 of the cooking chamber, or on the lateral side of the cooking chamber. The processor 220 may include one or more cameras 220. The camera 220 creates captured image data and outputs the captured image data to the processor 210.

[0058] The processor 210 may control the photographing operation of the camera 220 according to the operation mode. According to an embodiment of the present disclosure, when the cooking device 100 is performing a cooking operation, the processor 210 may control the camera 220 to photograph the interior of the cooking chamber. The processor 210 may start the cooking operation based on a user input requesting to start cooking, and may start the photographing operation of the camera 220 according to the request to start cooking. The photographing may be automatically performed based on a determination that one or more predetermined conditions (e.g., the internal heat reaches a predetermined temperature, a predetermined amount of time has passed since the start of cooking) are satisfied. In one or more examples, the photographing may be performed based on an input from a user.

[0059] The captured image may include a still image or a moving image. According to an embodiment of the present disclosure, the captured image may correspond to a real-time moving image obtained by photographing the interior of the cooking chamber while a cooking operation is being performed. According to an embodiment of the present disclosure, the captured image may correspond to a real-time moving image obtained by photographing the interior of the cooking chamber at regular time intervals while a cooking operation is being performed. According to an embodiment of the present disclosure, the captured image may correspond to a still image or a moving image obtained by photographing the interior of the cooking chamber based on a user input.

[0060] The captured image captured by the camera 220 may be an image compressed in a format such as H.264 or JPEG. When the captured image is a compressed image, the processor 210 creates a captured image in a format such as YUV or RGB through a decoding process. The processor 210 obtains color data of the captured image from the decoded captured image.

[0061] The cooking chamber 230 corresponds to a cooking space for accommodating ingredients. The cooking chamber 230 includes a space formed by a partition wall isolated from the outside. The cooking chamber 230 may include a tray or a rack on which ingredients can be placed. The cooking chamber 230 may be insulated by a heat insulating member to block internal heat. According to an embodiment of the present disclosure, the cooking device 100 may output heat from a heating device to the cooking chamber 230 to perform a cooking operation inside the cooking chamber 230. According to an embodiment of the present disclosure, the cooking device 100 may output microwaves from a microwave output device to the cooking chamber 230 to perform a cooking operation inside the cooking chamber 230.

[0062] According to an embodiment of the present disclosure, the cooking chamber 230 may be opened or blocked from the outside by an openable door. According to an embodiment of the present disclosure, the cooking chamber 230 may correspond to a drawer-type basket and may be opened or blocked from the outside by an insertion or withdrawal operation of the drawer-type basket.

[0063] The processor 210 may detect a fire hazard inside the cooking chamber based on the captured image. The processor 210 may detect the fire hazard based on the color data of the captured image. The processor 210 may detect the fire hazard by comparing a predefined fire-related color with the color data of the captured image. The predefined fire-related color may be a color corresponding to smoke or flame. The predefined fire-related color may be defined as a predetermined color coordinate region in a color space. The predefined fire-related color may include a smoke color region associated with smoke and a flame color region associated with flame.

[0064] The processor 210 may identify at least one monitoring area based on the color data of each pixel of the captured image. The at least one monitoring area may be a predetermined area in the captured image and may include one or more pixels. The processor 210 may identify an area having a color with a low similarity to a fire-related color as a monitoring area. The processor 210 monitors the color change of the monitoring area. When the similarity between the color data of the monitoring area and the fire-related color is equal to or greater than a reference value, the processor 210 may determine that a fire hazard has occurred.

[0065] According to an embodiment of the present disclosure, the processor 210 may previously obtain food information about food disposed inside the cooking chamber, and may select a monitoring area of ​​the captured image based on the obtained food information. The processor 210 may select an area having a lower similarity to a fire-related color as a monitoring area based on the food information. The processor 210 may change a weight of a monitoring area or a reference value regarding a similarity to a fire-related color according to the food information.

[0066] According to an embodiment of the present disclosure, the processor 210 can select a monitoring area based on the color data of the captured image without obtaining information about the food in advance. The processor 210 can select an area with a low similarity to the fire-related color as the monitoring area based on the color data of the captured image.

[0067] The memory 240 stores various information, data, instructions, programs, etc. required for the operation of the cooking device 100. The memory 240 may include one or both of a volatile memory and a non-volatile memory, or a combination thereof. The memory 240 may include at least one type of storage medium selected from the following: a flash memory type, a hard disk type, a multimedia card micro, a card-type memory (e.g., a secure digital (SD) or extreme digital (XD) memory), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable ROM (EEPROM), a programmable ROM (PROM), a magnetic memory, a magnetic disk, and an optical disk. The memory 240 may correspond to a network storage device or a cloud server that performs a storage function on the Internet.

[0068] The output interface 250 may output information and data related to the operation of the cooking device 100. The output interface 250 may include, for example, a display, a speaker, and a light emitting diode (LED).

[0069] When a fire hazard is detected, the output interface 250 outputs a fire hazard notification. The fire hazard notification may include at least one of visual information or audio information. According to an embodiment of the present disclosure, the fire hazard notification may include a message displayed via a display. According to an embodiment of the present disclosure, the fire hazard notification may include a voice message output via a speaker.

[0070] According to an embodiment of the present disclosure, the fire hazard notification may include at least one of information indicating that a fire hazard has been detected, the type of the fire hazard, response guidance information, or cooking device operation change information. The type of the fire hazard may include information such as smoke detection, flame detection, or temperature spike. The response guidance information may include, for example, a request to open a door, a request to cut off power, or a request to prohibit contact. The cooking device operation change information may include information such as door unlocking, cooking operation interruption, automatic power cutoff, or steam discharge from a cooking chamber.

[0071] Figure 3 The structure of the cooking device 100 according to the embodiment of the present disclosure is shown.

[0072] According to an embodiment of the present disclosure, the cooking device 100 may be opened or blocked from the outside by opening or closing the door 310. The cooking chamber 230 may accommodate food in an inner space. The cooking chamber 230 may include a tray 320 on which food can be placed.

[0073] The camera 220 may be disposed on the ceiling of the cooking chamber 230 to photograph a cooking process of ingredients disposed inside the cooking chamber 230 .

[0074] Figure 4 is a flowchart of a method of controlling a cooking device according to an embodiment of the present disclosure.

[0075] The method of controlling a cooking device according to an embodiment of the present disclosure can be performed by various types of cooking devices. In the present disclosure, an embodiment in which the cooking device 100 according to an embodiment of the present disclosure performs a cooking device control method will be mainly described. However, the cooking device control method according to an embodiment of the present disclosure is not limited to being performed by the cooking device 100 of the present disclosure.

[0076] In operation S402, the cooking device 100 creates a captured image by photographing the interior of the cooking chamber using the camera 220 inside the cooking chamber. The captured image may be captured during the cooking process. The cooking device 100 may start photographing the interior of the cooking chamber while starting the cooking operation. For example, when the cooking operation ends or when the door 310 is opened, the cooking device 100 may end photographing the interior of the cooking chamber. The captured image may include a still image or a moving image.

[0077] Next, in operation S404, the cooking device 100 detects a fire hazard based on the captured image. The cooking device 100 may determine the fire hazard based on the color data of the captured image. The cooking device 100 may identify at least one monitoring area based on the color data of the captured image, and may determine whether a fire has occurred based on the similarity between the color data of the at least one monitoring area and a predefined fire-related color. The cooking device 100 may calculate the similarity by comparing the predefined fire-related color with the color data of the at least one monitoring area. The predefined fire-related colors may include a smoke-related color corresponding to smoke and a flame-related color corresponding to flame. The similarity may be determined based on the color coordinate distance between the color coordinate area of ​​the fire-related color and the monitoring area in the color space.

[0078] When it is determined in operation S404 that there is a fire hazard, in operation S406, the cooking device 100 outputs a fire hazard notification. The fire hazard notification may include at least one of visual information or audio information. The fire hazard notification may include a message displayed through a display, a voice message output through a speaker, etc. According to an embodiment of the present disclosure, the fire hazard notification may include at least one of information indicating that a fire hazard has been detected, the type of the fire hazard, response guidance information, or cooking device operation change information.

[0079] When a fire hazard is not detected in operation S404, the cooking apparatus 100 may perform operations S404 and S406 based on the next captured image.

[0080] According to an embodiment of the present disclosure, when the captured image is a moving picture, operations S404 and S406 may be performed at predetermined frame intervals. For example, after starting a cooking operation, the cooking device 100 may determine a fire hazard by performing operations S404 and S406 at intervals of 1 second.

[0081] According to an embodiment of the present disclosure, when the captured image is a still image, the cooking apparatus 100 may determine a fire hazard by performing operations S404 and S406 whenever the still image is captured.

[0082] Figure 5 A monitoring area according to an embodiment of the present disclosure is shown.

[0083] According to an embodiment of the present disclosure, the cooking device 100 may determine the first monitoring area 520a, the second monitoring area 520b, the third monitoring area 520c, and the fourth monitoring area 520d, respectively, based on the color data of the captured image 510. The first monitoring area 520a, the second monitoring area 520b, the third monitoring area 520c, and the fourth monitoring area 520d may be defined as areas having colors other than fire-related colors, respectively. The cooking device 100 may extract the color data of the first monitoring area 520a, the second monitoring area 520b, the third monitoring area 520c, and the fourth monitoring area 520d, respectively, from the captured image captured during the cooking operation, and compare the color data of the first monitoring area 520a, the second monitoring area 520b, the third monitoring area 520c, and the fourth monitoring area 520d with the fire-related colors, respectively, to detect fire hazards.

[0084] The cooking device 100 may determine the monitoring area by using a captured image captured before the start of the cooking operation. When the cooking operation starts, the color of the food changes, so the cooking device 100 may determine the monitoring area by using the captured image captured before the start of the cooking operation. The cooking device 100 may determine the monitoring area based on subsequent captured images and perform an operation of detecting a fire hazard.

[0085] exist Figure 5 In the example of , the first region 530 corresponds to raw chicken and is different from the flame-related color corresponding to the flame and is similar to the smoke-related color corresponding to the smoke. The second region 532 is a red region in the raw chicken and is similar to the fire-related color and different from the smoke-related color. The third region 534 corresponds to the tray and is different from the flame-related color and the smoke-related color.

[0086] According to an embodiment of the present disclosure, the cooking device 100 may set an area with a different color associated with flame and a different color associated with smoke as a monitoring area. Figure 5 In the example of FIG. 1 , the cooking device 100 may select the first monitoring area 520a and the fourth monitoring area 520d having different colors associated with flames and different colors associated with smoke as monitoring areas, but not select the second monitoring area 520b and the third monitoring area 520c as monitoring areas.

[0087] According to an embodiment of the present disclosure, the cooking device 100 may select an area with a different color associated with flame as a flame monitoring area, and may set an area with a different color associated with smoke as a smoke monitoring area. The cooking device 100 may detect flames based on the color data of the flame monitoring area. The cooking device 100 may detect smoke based on the color data of the smoke monitoring area. Figure 5In the example, the cooking device 100 may select the first monitoring area 520a, the second monitoring area 520b, the third monitoring area 520c, and the fourth monitoring area 520d as flame monitoring areas, and may select the first monitoring area 520a and the fourth monitoring area 520d as smoke monitoring areas.

[0088] The first monitoring area 520a, the second monitoring area 520b, the third monitoring area 520c, and the fourth monitoring area 520d may respectively correspond to an area including one or more pixels.

[0089] According to an embodiment of the present disclosure, the first monitoring area 520a, the second monitoring area 520b, the third monitoring area 520c and the fourth monitoring area 520d may be defined in the form of rectangular blocks. However, as understood by those skilled in the art, the monitoring area may be of any suitable shape.

[0090] According to an embodiment of the present disclosure, the first monitoring area 520a, the second monitoring area 520b, the third monitoring area 520c and the fourth monitoring area 520d can be defined respectively according to the edge of the specific object. For example, the first monitoring area 520a, the second monitoring area 520b, the third monitoring area 520c and the fourth monitoring area 520d can be defined as areas corresponding to food.

[0091] The first monitoring area 520a, the second monitoring area 520b, the third monitoring area 520c, and the fourth monitoring area 520d may include a plurality of monitoring areas 520a, 520b, 520c, and 520d. According to an embodiment of the present disclosure, the cooking device 100 may set the plurality of monitoring areas 520a, 520b, 520c, and 520d by selecting different color areas different from the fire-related colors from the captured image 510. Figure 5 In the example of , the first monitoring area 520a may be a color corresponding to the tray, and the second monitoring area 520b may be a color corresponding to the chicken as an ingredient. Therefore, the cooking device 100 may select areas with different colors in the captured image 510 as the first monitoring area 520a and the second monitoring area 520b.

[0092] According to an embodiment of the present disclosure, the cooking device 100 may set a plurality of monitoring areas in the captured image, the plurality of monitoring areas including monitoring areas 520b and 520c corresponding to areas corresponding to the food, and monitoring areas 520a and 520d corresponding to the cooking device 100. Since flames are generally generated in the food rather than in another object such as a tray, the cooking device 100 may set at least one area corresponding to the food as the monitoring areas 520b and 520c. Since the color of the cooking device 100 is known to the cooking device 100 and does not change during the cooking process, the cooking device 100 may set at least one area corresponding to the color of the cooking device 100 as the monitoring areas 520a and 520d. The at least one area corresponding to the color of the cooking device 100 may include an area corresponding to a detachable tray used in the cooking device 100.

[0093] According to an embodiment of the present disclosure, the first monitoring area 520a, the second monitoring area 520b, the third monitoring area 520c, and the fourth monitoring area 520d may be changed respectively during the cooking process.

[0094] Figure 6 is a flowchart of a method of controlling a cooking device according to an embodiment of the present disclosure.

[0095] According to an embodiment of the present disclosure, the cooking apparatus 100 may photograph the inside of the cooking chamber before the cooking operation starts, and may set a monitoring area based on a captured image corresponding to a result of the photographing.

[0096] In operation S602, the cooking apparatus 100 creates a captured image by photographing the inside of the cooking chamber.

[0097] Next, in operation S604, the cooking apparatus 100 sets a monitoring area based on the captured image.

[0098] When the monitoring area is determined, the cooking apparatus 100 repeats operations S402 and S404 and detects a fire hazard based on the captured image.

[0099] Figure 7a is a flowchart illustrating a process of detecting fire hazards based on captured images according to an embodiment of the present disclosure.

[0100] Figure 7b is a diagram illustrating a process of comparing a monitoring area with fire-related colors in a color space according to an embodiment of the present disclosure.

[0101] According to an embodiment of the present disclosure, the cooking device 100 determines whether there is a fire hazard based on the color data of the monitoring area. Figure 7aDetails of operation S404 according to an embodiment of the present disclosure are described.

[0102] In operation S702, the cooking device 100 identifies color data of the monitoring area. When the monitoring area may include one pixel, the color data of the monitoring area corresponds to the color data of the one pixel. When the monitoring area may include multiple pixels, the cooking device 100 may identify an average value of the color data of the multiple pixels as the color data of the monitoring area.

[0103] Color data can be defined as color coordinates in a predetermined color space. Figure 7b As shown, color data can be defined in an RGB color space. The cooking device 100 can decode the captured image to obtain an RGB image and obtain the RGB color coordinates of each pixel. The color data of the monitoring area can be defined as a first monitoring area color coordinate 720a and a second monitoring area color coordinate 720b on the color space.

[0104] The cooking device 100 may define a fire color region corresponding to a fire-related color in a color space. The fire color region may include a flame color region 712 (e.g., a red or orange-related color) and a smoke color region 714 (e.g., a black, white, or gray-related color). The flame color region 712 and the smoke color region 714 may be defined as predetermined color coordinate regions on the color space. The flame color region 712 and the smoke color region 714 may be previously defined and stored in the memory 240.

[0105] The color data of the monitoring area may be defined in a color space. When there are two monitoring areas, the first monitoring area color coordinate 720a and the second monitoring area color coordinate 720b may be predetermined according to one or more settings. The cooking device 100 may identify the first monitoring area color coordinate 720a corresponding to the average value of the color data of the first monitoring area. The cooking device 100 may also identify the second monitoring area color coordinate 720b corresponding to the average value of the color data of the second monitoring area.

[0106] Next, in operation S704, the cooking apparatus 100 calculates a similarity between the color data of the monitoring area and the flame-related color area. The monitoring area in operation S704 may correspond to the flame monitoring area.

[0107] refer to Figure 7b , the cooking device 100 calculates the distance between the color coordinates 720a of the first monitoring area and the flame color area 712 in the color space. According to an embodiment of the present disclosure, the cooking device 100 can calculate the distance between the color coordinates 720a of the first monitoring area and the flame color area 712 in the color space. Figure 7bThe distance between the first monitoring area color coordinate 720a and the center of the flame color area 712 is calculated by the method defined as D2 in FIG. According to an embodiment of the present disclosure, the cooking device 100 can also be based on Figure 7b The method defined as D1 in the above is used to calculate the distance 730a between the first monitoring area color coordinate 720a and the boundary of the flame color area 712. When the distance 730a between the first monitoring area color coordinate 720a and the boundary of the flame color area 712 is calculated according to the method D2, the cooking device 100 can obtain the distance 730a between the first monitoring area color coordinate 720a and the boundary point of the flame color area 712 closest to the first monitoring area color coordinate 720a.

[0108] Similarly, the cooking device 100 may calculate the distance between the second monitoring area color coordinate 720 b and the flame color area 712 in the color space.

[0109] This disclosure will mainly describe the Figure 7b The method of D2 calculates the distance 730a between the first monitoring area color coordinate 720a and the boundary of the flame color area 712 and the distance 730b between the second monitoring area color coordinate 720b and the boundary of the flame color area 712 to calculate the similarity. However, the embodiment is not limited to these configurations, and does not exclude the use of the method according to Figure 7b The method D1 calculates the distances between the first monitoring region color coordinate 720 a and the second monitoring region color coordinate 720 b and the center of the flame color region 712 to calculate the similarity.

[0110] The cooking device 100 can calculate the similarity between the color data of the monitoring area and the flame-related color based on the distances 730a and 730b on the color space. The similarity can be defined as the distances 730a and 730b themselves, or can be defined as a value calculated based on the distances 730a and 730b.

[0111] Next, in operation S706, the cooking device 100 determines whether the color similarity is equal to or greater than a first reference value. When the color similarity is equal to or greater than the first reference value, in operation S708, the cooking device 100 determines that a flame has been detected. When it is determined that a flame has been detected, in operation S716, the cooking device 100 determines that a fire hazard has been detected.

[0112] In operation S710, the cooking apparatus 100 calculates a similarity between color data of the monitoring area and a smoke-related color. In operation S710, the monitoring area in operation S710 may correspond to a smoke monitoring area.

[0113] refer to Figure 7b, the cooking device 100 calculates the distance between the color coordinates 720a of the first monitoring area and the smoke color area 714 in the color space. According to an embodiment of the present disclosure, the cooking device 100 can calculate the distance between the color coordinates 720a of the first monitoring area and the smoke color area 714 in the color space. Figure 7b The distance between the color coordinates 720a of the first monitoring area and the center of the smoke color area 714 is calculated by the method defined as D2 in FIG. According to an embodiment of the present disclosure, the cooking device 100 can also be based on Figure 7b The distance 732a between the first monitoring area color coordinate 720a and the boundary of the smoke color area 714 is calculated by the method defined as D1 in FIG. Figure 7b When method D2 is used to calculate the distance 732a between the first monitoring area color coordinate 720a and the boundary of the smoke color area 714, the cooking device 100 obtains the distance 732a between the first monitoring area color coordinate 720a and the boundary point of the smoke color area 714 closest to the first monitoring area color coordinate 720a.

[0114] Similarly, the cooking device 100 may calculate the distance between the second monitoring area color coordinate 720 b and the smoke color area 714 in the color space.

[0115] This disclosure will mainly describe the Figure 7b The embodiment of calculating the similarity by calculating the distance 732a between the first monitoring area color coordinate 720a and the boundary of the smoke color area 714 and the distance 732b between the second monitoring area color coordinate 720b and the boundary of the smoke color area 714. However, the embodiment is not limited to these configurations, and does not exclude the use of the method according to Figure 7b The method D1 calculates the distance between the first monitoring area color coordinate 720a and the second monitoring area color coordinate 720b and the center of the smoke color area 714 to calculate the similarity.

[0116] The cooking device 100 can calculate the similarity between the color data of the monitoring area and the smoke-related color based on the distances 732a and 732b on the color space. The similarity can be defined as the distances 732a and 732b themselves, or can be defined as a value calculated based on the distances 732a and 732b.

[0117] Next, in operation S712, the cooking device 100 determines whether the color similarity is equal to or greater than a second reference value. The second reference value is the same as or different from the first reference value described above. When the color similarity is equal to or greater than the second reference value, in operation S714, the cooking device 100 determines that smoke has been detected. When it is determined that smoke is detected, in operation S716, the cooking device 100 determines that a fire hazard has been detected.

[0118] When a fire hazard is detected, the cooking device 100 outputs a fire hazard notification in operation S406. The multimedia device 100 may output the type of fire. For example, the cooking device 100 may output information indicating that a flame has been detected, information indicating that smoke has been detected, or information indicating that a flame and smoke have been detected.

[0119] Figure 8 A process of detecting smoke or flame based on color distribution according to an embodiment of the present disclosure is shown.

[0120] According to an embodiment of the present disclosure, when the color distribution 810 of the monitoring area corresponding to the smoke-related color is further considered, the cooking device 100 determines whether smoke has been detected. Since the cooking chamber 230 of the cooking device 100 is a closed space, when a fire generates smoke, the interior of the cooking chamber 230 of the cooking device 100 may be filled with smoke. Therefore, when a fire generates smoke, the entire area of ​​the captured image may change to a color close to the smoke-related color. The cooking device 100 can detect smoke by using a plurality of monitoring areas, and when the similarity with the smoke-related color is high (e.g., equal to or greater than a second reference value) in a predetermined percentage of the plurality of monitoring areas, it is determined that smoke has been detected.

[0121] According to an embodiment of the present disclosure, the cooking device 100 may determine whether a flame has been detected based on the color distribution 820 of the monitoring area corresponding to the flame-related color in consideration. When a flame is generated in the cooking chamber 230 of the cooking device 100, the flame may be detected or seen occasionally. The cooking device 100 may detect a flame by using a plurality of monitoring areas, and when the similarity with the flame-related color in a reference number or more of the plurality of monitoring areas is high (e.g., equal to or greater than a first reference value), it may be determined that a flame has been detected and that smoke has been detected. The reference numeral may correspond to a number of 1 or greater, for example.

[0122] Fig. 9 is a block diagram of a structure of a cooking device according to an embodiment of the present disclosure.

[0123] According to an embodiment of the present disclosure, the cooking device 100 may include a processor 210, a camera 220, a cooking chamber 230, a memory 240, an output interface 250, a temperature sensor 910, and a heating module 920. Fig. 9 Main description and Figure 2 The cooking device 100 is different.

[0124] The temperature sensor 910 may detect the temperature of the cooking chamber 230 of the cooking device 100. The temperature sensor 910 may measure the temperature and convert the temperature into an electrical signal. The temperature sensor 910 may be installed inside the cooking chamber 230. The temperature sensor 910 outputs the measured temperature detection value to the processor 210.

[0125] The heating module 920 may control the internal temperature of the cooking chamber 230. The heating module 920 may include, for example, an electric heater or a gas burner. The processor 210 may control the internal temperature of the cooking chamber 230 to a set temperature by controlling the heating module 920. Based on the temperature detection value detected by the temperature sensor 910, the processor 210 may increase the internal temperature of the cooking chamber 230 by using the heating module 920 or stop heating the heating module 920.

[0126] The processor 210 may determine the fire hazard based on the temperature detection value. When the fire hazard is detected based on the captured image, the processor 210 may re-determine and verify the fire hazard by additionally considering the temperature detection value.

[0127] When detecting a fire hazard based on a captured image, the processor 210 may determine whether the temperature detection value exceeds a reference temperature. The reference temperature is a predetermined value determined in advance.

[0128] According to an embodiment of the present disclosure, the reference temperature may be changed according to the cooking progress time. The cooking device 100 may control the internal temperature of the cooking chamber 230 by controlling the heating module 920 according to a predetermined cooking process. When the temperature detection value is higher than the target temperature of the temperature control, the cooking device 100 determines that there is a fire risk. The reference temperature may be set to a value higher than the target temperature of the temperature control. The cooking process may be determined based on detection of one or more ingredients or user input indicating an item to be cooked.

[0129] According to an embodiment of the present disclosure, the reference temperature may vary according to one or more ingredients used in the cooking process. The reference temperature may be set differently according to the ignition point of the ingredients. For example, for an ingredient with a lower ignition point, the reference temperature may be set lower, and for an ingredient with a higher ignition point, the reference temperature may be set higher.

[0130] Fig.10 is a flowchart of a process of detecting a fire hazard based on a captured image and a temperature detection value according to an embodiment of the present disclosure.

[0131] According to an embodiment of the present disclosure, when detecting a fire hazard based on a captured image in operation S404, the cooking apparatus 100 obtains a temperature detection value from the temperature sensor 910 in operation S1002. The temperature sensor 910 detects the internal temperature of the cooking chamber 230 to generate a temperature detection value.

[0132] In operation S1004, the cooking device 100 determines whether the temperature detection value is equal to or greater than the reference temperature. When the temperature detection value is equal to or greater than the reference temperature, in operation S1006, the cooking device 100 determines that there is a fire risk. When the temperature detection value is less than the reference temperature, the cooking device 100 returns to operation S402 to obtain the next captured image.

[0133] Fig.11 The process of identifying food from a captured image and determining a monitoring area according to an embodiment of the present disclosure is shown.

[0134] According to an embodiment of the present disclosure, the cooking apparatus 100 may recognize food from the captured image 1110 and determine a monitoring area according to the recognition result of the food.

[0135] The cooking device 100 may identify the food from the captured image 1110 (1120). The cooking device 100 may identify the food by performing object recognition on the captured image 1110. The cooking device 100 may perform food recognition to identify at least one of the type of food, the color of food, the position of food, or the edge of food. The cooking device 100 may identify the food from the captured image 1110 to create food recognition result 1122.

[0136] exist Fig.11 In the example of , the captured image 1110 is an image obtained by photographing the interior of the cooking chamber 230 while a pizza is being cooked. The cooking apparatus 100 recognizes bell peppers, mushrooms, dough, flour, and olives from the captured image 1110 by performing ingredient recognition on the captured image 1110. The cooking apparatus 100 also recognizes a decorative material or a tray of the cooking chamber 230. Fig.11 In the example of , cooking device 100 identifies a baking pan from captured image 1110 .

[0137] Fig.12 is a diagram illustrating a structure of recognizing ingredients by using a machine learning model according to an embodiment of the present disclosure.

[0138] According to an embodiment of the present disclosure, the cooking device 100 can identify the food information and the food position from the captured image 1110 by using the machine learning model 1210. The food information may include the type of food and the color of the food. The food position indicates the area corresponding to the food in the captured image 1110. The food position may include edge information corresponding to the food.

[0139] The machine learning model 1210 may be a model that performs machine learning through training data, the training data including a plurality of captured images as input data and a plurality of food information and a plurality of food positions as output data. The machine learning model 1210 may use algorithms such as convolutional neural networks (CNNs), region-based convolutional neural networks (R-CNNs), you only look once (YOLO), or single shot detectors (SSDs), or any other neural network models known to those of ordinary skill in the art.

[0140] According to an embodiment of the present disclosure, the cooking apparatus 100 may use the machine learning model 1210 according to an On-Device method. The memory 240 may store instructions of the machine learning model 1210, and the processor 210 may execute the machine learning model 1210 by executing the instructions of the machine learning model 1210 stored in the memory 240.

[0141] According to an embodiment of the present disclosure, the cooking device 100 can obtain an ingredient recognition result by using a machine learning model 1210 executed in an external server. The cooking device 100 can send a captured image 1110 to the external server and request ingredient recognition. The external server can input the captured image 1110 to the machine learning model 1210 to obtain ingredient information and ingredient location. The external server can send the obtained ingredient information and the obtained ingredient location to the cooking device 100.

[0142] By returning a reference Fig.11 Next, the operation of food identification 1120 will be described.

[0143] The cooking device 100 may calculate the color similarity with the fire-related color based on the food type and food color recognized from the captured image 1110 (1130). Figure 7a and Figure 7b As described above, the color similarity may be calculated based on the distance between the coordinate region of the fire-related color in the color space and the color coordinates of the recognized food color.

[0144] According to an embodiment of the present disclosure, the cooking apparatus 100 may calculate the color similarity between the recognized food color and the smoke-related color. The cooking apparatus 100 calculates the color similarity between the recognized food color and the flame-related color.

[0145] The cooking device 100 can calculate the color similarity to create a color similarity calculation result 1132. The color similarity calculation result 1132 can be defined as low, similar to smoke, very similar to smoke, similar to flame, very similar to flame, or any other suitable similarity category.

[0146] Next, the cooking apparatus 100 may select a monitoring area based on the color similarity calculation result 1132 ( 1140 ).

[0147] According to an embodiment of the present disclosure, the cooking device 100 may select a food area having a lower color similarity to a fire-related color as a monitoring area. The cooking device 100 may select a food area having a lower color similarity to a fire-related color and a smoke-related color as a monitoring area. Fig.11 In the example of , the cooking device 100 may select areas corresponding to bell peppers, olives, and grill pans that have a lower similarity to fire-related colors as monitoring areas.

[0148] According to an embodiment of the present disclosure, the cooking device 100 may select a food area having a lower color similarity to a smoke-related color as a smoke monitoring area, and may select a food area having a lower color similarity to a flame-related color as a flame monitoring area. Fig.11 In the example of , the cooking device 100 may select the areas corresponding to the bell peppers, olives, and grill pans that have a lower similarity to the smoke-related colors as the smoke monitoring areas. The cooking device 100 may select the areas corresponding to the bell peppers, olives, and grill pans that have a lower similarity to the flame-related colors as the flame monitoring areas.

[0149] According to an embodiment of the present disclosure, the cooking device 100 may set all regions corresponding to the food selected as the monitoring region as the monitoring region. In this case, all pixels corresponding to the food selected as the monitoring region may be set as the monitoring region.

[0150] According to an embodiment of the present disclosure, the cooking device 100 may set one or more areas corresponding to the food selected as the monitoring area as the monitoring area. For example, the cooking device 100 may set a rectangular block including some pixels among the pixels corresponding to the bell pepper as the monitoring area. For example, the cooking device 100 may set the pixels of the central area of ​​each bell pepper except the edge area of ​​each bell pepper among the pixels corresponding to the bell pepper as the monitoring area. As understood by those of ordinary skill in the art, the monitoring area may be any suitable shape other than a rectangular block.

[0151] Fig.13 A monitoring area according to a setup of an embodiment of the present disclosure is shown.

[0152] According to an embodiment of the present disclosure, the cooking device 100 may define a pixel area corresponding to food having a low color similarity to a fire-related color as a monitoring area. The cooking device 100 may create a mask in which pixels corresponding to the monitoring area have a value of 1 and the remaining pixels have a value of 0. The cooking device 100 may extract the monitoring area by using the mask.

[0153] According to an embodiment of the present disclosure, a mask for a smoke monitoring area and a mask for a flame monitoring area may be created separately.

[0154] The cooking apparatus 100 may determine the monitoring area before starting the cooking operation and may create a mask corresponding to the monitoring area. The cooking apparatus 100 stores the mask corresponding to the monitoring area in the memory 240 and extracts color data of the monitoring area by using the mask.

[0155] Fig.14 is a block diagram illustrating a process of applying one or more weights to a monitoring area according to an embodiment of the present disclosure.

[0156] According to an embodiment of the present disclosure, the cooking apparatus 100 may recognize an ingredient ( 1120 ), calculate a color similarity between the recognized ingredient and a fire-related color ( 1130 ), and apply a weight to each pixel region based on the color similarity.

[0157] The cooking device 100 may identify the food from the captured image 1110 (1120). The cooking device 100 may identify the food by performing object recognition on the captured image 1110. The cooking device 100 may perform food recognition to identify the type of food, the color of food, the location of food, and the edge of food. The cooking device 100 may identify the food from the captured image 1110 to create food recognition result 1122.

[0158] The cooking device 100 may calculate the color similarity with the fire-related color based on the food type and food color recognized from the captured image 1110 (1130). Figure 7a and Figure 7b As described above, the color similarity may be calculated based on the distance between the coordinate region of the fire-related color in the color space and the color coordinates of the recognized food color.

[0159] According to an embodiment of the present disclosure, the cooking apparatus 100 may calculate the color similarity between the recognized food color and the smoke-related color. The cooking apparatus 100 may calculate the color similarity between the recognized food color and the flame-related color.

[0160] The cooking device 100 may calculate color similarity to generate a color similarity calculation result 1132. The color similarity calculation result 1132 may be defined as low, similar to smoke, very similar to smoke, similar to flame, very similar to flame, or any other suitable color similarity category.

[0161] Next, the cooking device 100 may determine a weight for each pixel area based on the color similarity calculation result 1132 (1410). In the color similarity calculation result 1132, the cooking device 100 may assign a higher weight to ingredients with lower color similarity, and assign a lower weight to ingredients with higher color similarity. The cooking device 100 may create a weight map according to the color similarity, and obtain a fire-related index related to the color similarity based on the weight map. The cooking device 100 may determine the color similarity of the color data of each pixel to the fire-related color. The cooking device 100 may apply a weight to the color similarity value of each pixel by multiplying the color similarity value of each pixel by the weight map. The cooking device 100 may define the sum of the values ​​obtained by applying weights to the corresponding color similarity values ​​of the pixels as the fire-related index. When the fire-related index is equal to or greater than the reference value, the cooking device 100 may determine that a fire hazard has been detected.

[0162] exist Fig.14 In the example of , bell peppers, olives, and baking sheets are determined to have lower color similarity and therefore have higher weights. Mushrooms are determined to have higher color similarity to smoke and have lower weights. Dough and flour are determined to have very high color similarity to smoke and have a lowest weight defined. The lowest weight value may be, for example, 0.

[0163] According to an embodiment of the present disclosure, the cooking apparatus 100 may differently set a flame weight for a flame-related color and a smoke weight for a smoke-related color.

[0164] According to an embodiment of the present disclosure, the cooking device 100 may calculate the similarity of the color associated with the flame, and calculate the flame weight based on the similarity of the color associated with the flame. As a result of the calculation of the similarity of the color associated with the flame, the cooking device 100 may assign a higher weight to the ingredients with a lower color similarity and assign a lower weight to the ingredients with a higher color similarity. The cooking device 100 may generate a flame weight map according to the color similarity, and obtain a flame related index related to the color similarity of the flame based on the flame weight map. The cooking device 100 may determine the color similarity of the color data of each pixel to the flame related color. The cooking device 100 may apply the flame weight to the color similarity value of each pixel. The cooking device 100 may apply the flame weight by multiplying the color similarity value of each pixel by the flame weight map. The cooking device 100 may define the sum of the values ​​obtained by applying the flame weight to the corresponding color similarity values ​​of the pixels as the flame related index. In one or more examples, when the flame related index is equal to or greater than the reference value, the cooking device 100 determines that a fire hazard has been detected.

[0165] According to an embodiment of the present disclosure, the cooking device 100 may calculate the similarity with the smoke-related color, and calculate the smoke weight based on the similarity with the smoke-related color. As a result of the calculation of the similarity with the smoke-related color, the cooking device 100 may assign a higher weight to the ingredients with a lower color similarity and assign a lower weight to the ingredients with a higher color similarity. The cooking device 100 may establish a smoke weight map according to the color similarity, and obtain a smoke-related index related to the color similarity of the smoke based on the smoke weight map. The cooking device 100 may determine the color similarity between the color data of each pixel and the smoke-related color. The cooking device 100 may apply the smoke weight to the color similarity value of each pixel. The cooking device 100 may apply the smoke weight by multiplying the color similarity value of each pixel by the smoke weight map. The cooking device 100 may define the sum of the values ​​obtained by applying the smoke weight to the corresponding color similarity values ​​of the pixels as the smoke-related index. When the smoke-related index is equal to or greater than the reference value, the cooking device 100 determines that a fire hazard has been detected.

[0166] Fig.15 is a flowchart of a process of selecting a monitoring area according to an embodiment of the present disclosure.

[0167] According to an embodiment of the present disclosure, the cooking device 100 may select a monitoring area based on the color change of the food during the cooking process. During the cooking process, when the food is boiled, the food may change color. For example, beef or pork may have a red color before cooking, and by being cooked by the cooking device 100, it may change to a color close to white. In this case, the color of the beef or pork in the cooking state has a high color similarity with the smoke. Accordingly, even if the color similarity with the fire-related color is low before cooking and the color similarity with the fire-related color is high after cooking, the cooking device 100 may not select the food as the monitoring area, thereby reducing the risk of erroneously detecting smoke or fire hazards.

[0168] In operation S1502, the cooking device 100 may identify the type of ingredients from the captured image. As described above, the cooking device 100 may identify the type of ingredients from the captured image by using an object recognition algorithm. The cooking device 100 may identify the type of ingredients from the captured image by using a machine learning model.

[0169] When a plurality of types of ingredients are recognized from the captured image, the cooking apparatus 100 may perform operations S1504, S1506, S1508, and S1510 for each type of ingredient.

[0170] In operation S1504, the cooking device 100 determines whether the color similarity between the recognized food and the fire-related color is equal to or greater than a reference value. According to an embodiment of the present disclosure, the cooking device 100 may calculate the color similarity between the flame-related color and the color data of the food area of ​​the captured image. According to an embodiment of the present disclosure, the cooking device 100 may calculate the color similarity between the smoke-related color and the color data of the food area of ​​the captured image.

[0171] When the color similarity is equal to or greater than the reference value, the cooking apparatus 100 does not select the food area as the monitoring area in operation S1508.

[0172] When the color similarity is less than the reference value, in operation S1506, the cooking device 100 determines whether the color similarity of the food material to the flame-related color becomes higher during the cooking process. For example, in the case of beef or pork, the color similarity to the fire-related color is low before the start of cooking, but after cooking, the color similarity to the smoke-related color is high. The cooking device 100 may have previously stored information about food materials whose color similarity to the fire-related color is highly changed during the cooking process. For example, the cooking device 100 may store information about food materials whose color similarity to the flame-related color is highly changed during the cooking process and information about food materials whose color similarity to the smoke-related color is highly changed during the cooking process in the memory 240. By using the information stored in the memory 240, the cooking device 100 determines whether the color similarity of the food material to the fire-related color is high while the food material is being cooked.

[0173] When it is determined in operation S1506 that the color similarity between the food and the fire-related color becomes higher while the food is being cooked, the cooking apparatus 100 does not select the area corresponding to the food as the monitoring area in operation S1508. When it is determined in operation S1506 that the color similarity between the food and the fire-related color does not become higher while the food is being cooked, the cooking apparatus 100 selects the area corresponding to the food as the monitoring area in operation S1510.

[0174] When the monitoring area is determined in operation S604, the cooking apparatus 100 creates a captured image by photographing the inside of the cooking chamber in operation S402.

[0175] Fig.16 is a diagram illustrating shrinkage of food according to an embodiment of the present disclosure.

[0176] According to an embodiment of the present disclosure, the cooking apparatus 100 may determine a monitoring area in consideration of shrinkage of food.

[0177] During the cooking process and boiling, ingredients can shrink as water is removed. For example, meat shrinks during cooking. Fig.16 As shown, the edge of the meat moves from a first edge 1610 prior to cooking to a second edge 1612 due to shrinkage 1614 during the cooking process. As another example, certain plants, such as spinach, shrink during the cooking process.

[0178] According to an embodiment of the present disclosure, the monitoring area is determined in consideration of the change of the food during the cooking process. In order to prevent the monitoring area from changing from the area corresponding to the food to the area not corresponding to the food due to the shrinkage of the food during the cooking process, the cooking device 100 may exclude the edge area of ​​the food from the monitoring area and determine the center area of ​​the food as the monitoring area.

[0179] Fig.17 is a flowchart of a process of determining a monitoring area in consideration of shrinkage of food according to an embodiment of the present disclosure.

[0180] First, in operation S1502, the cooking device 100 recognizes the type of ingredients from the captured image. As described above, the cooking device 100 can recognize the type of ingredients from the captured image by using an object recognition algorithm. The cooking device 100 can recognize the type of ingredients from the captured image by using a machine learning model.

[0181] When a plurality of types of ingredients are recognized from the captured image, the cooking apparatus 100 may perform operations S1504, S1702, S1704, S1706, and S1708 for each type of ingredient.

[0182] In operation S1504, the cooking device 100 determines whether the color similarity between the recognized food and the fire-related color is equal to or greater than a reference value. According to an embodiment of the present disclosure, the cooking device 100 may calculate the color similarity between the flame-related color and the color data of the food area of ​​the captured image. According to an embodiment of the present disclosure, the cooking device 100 may calculate the color similarity between the smoke-related color and the color data of the food area of ​​the captured image.

[0183] When the color similarity is equal to or greater than the reference value, the cooking apparatus 100 does not select the food area as the monitoring area in operation S1708.

[0184] When the color similarity is less than the reference value, in operation S1702, the cooking device 100 determines whether the identified food shrinks during the cooking process. For example, meat or vegetables shrink as they are cooked during the cooking process. The cooking device 100 may previously store information about food that shrinks during the cooking process. For example, the cooking device 100 stores information about food that shrinks in the memory 240, and determines whether the identified food is food that shrinks during the cooking process by using the information stored in the memory 240.

[0185] When it is determined in operation S1702 that the identified food is food that shrinks during the cooking process, in operation S1704, the cooking apparatus 100 excludes the edge area from the area corresponding to the identified food and selects the center portion of the food area as the monitoring area. Fig.16A detailed example of selecting a monitoring area for food that shrinks during a cooking process is described. When food is expected to shrink from first edge 1610 to second edge 1614 during a cooking process, cooking device 100 excludes an edge area between first edge 1610 and second edge 1614 from the monitoring area, and sets a central area of ​​second edge 1614 as the monitoring area.

[0186] According to an embodiment of the present disclosure, when the food material is a food material that shrinks during the cooking process, the cooking device 100 may previously store information about the shrinkage rate. For example, the cooking device 100 may previously store information about the shrinkage rate of the food material that shrinks during the cooking process according to three categories of high, medium, and low. As understood by those of ordinary skill in the art, the number of categories may be set to any suitable number. The cooking device 100 may adjust the edge area of ​​the monitoring area to be excluded from the food material area according to the shrinkage rate. When the shrinkage rate is high, the cooking device 100 sets the ratio of the edge area to the entire food material area (as a first ratio) to high. When the shrinkage rate is medium, the cooking device 100 sets the ratio of the edge area to the entire food material area (as a second ratio) to medium. When the shrinkage rate is low, the cooking device 100 sets the ratio of the edge area to the entire food material area (as a third ratio) to low. The first ratio, the second ratio, and the third ratio may be pre-specified.

[0187] When it is determined in operation S1702 that the food material does not shrink during the cooking process, in operation S1706, the cooking device 100 selects the entire food material area as the monitoring area. However, according to an embodiment of the present disclosure, even in the case of food material that does not shrink, the monitoring area may exclude the edge area in the food material area. The edge area set in operation S1706 may be set to be narrower than the edge area excluded for food material that shrinks in operation S1704.

[0188] When the monitoring area is determined in operation S604, the cooking apparatus 100 creates a captured image by photographing the inside of the cooking chamber in operation S402.

[0189] Fig.18 is a block diagram of a structure of a cooking device according to an embodiment of the present disclosure.

[0190] According to an embodiment of the present disclosure, the cooking device 100 may obtain the food information based on external input or user input. The cooking device 100 may obtain the food information through the communication module 1810 or the input interface 1820.

[0191] According to an embodiment of the present disclosure, the cooking device 100 may include a processor 210, a camera 220, a cooking chamber 230, a memory 240, an output interface 250, a communication module 1810, and an input interface 1820. Now, reference will be made to Fig.18 Main description and Figure 2 The cooking device 100 is different.

[0192] The communication module 1810 may communicate with an external device such as a server, a mobile device, or a user device in a wired or wireless manner. The communication module 1810 may access an access point (AP) device to send / receive Wi-Fi signals. The processor 210 may control the sending / receiving operation of the communication module 1810.

[0193] The communication module 1810 may include a wireless communication module (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (e.g., a local area network (LAN) communication module or a power line communication module). The communication module 1810 may perform short-range communication and may use, for example, Bluetooth, Bluetooth Low Energy (BLE), near field communication, WLAN (Wi-Fi), Zigbee, Infrared Data Association (IrDA) communication, Wi-Fi Direct (WFD), Ultra Wideband (UWB), or Ant+ communication. For example, the communication module 1810 may perform long-range communication, such as communicating with an external device through a traditional cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN).

[0194] For example, the communication module 1810 may use mobile communication and may exchange wireless signals with at least one of a base station, an external terminal, or a server on a mobile communication network.

[0195] According to an embodiment of the present disclosure, the communication module 1810 is connected to an AP in a house through Wi-Fi communication. The communication module 1810 can communicate with an external device through the AP.

[0196] The input interface 1820 receives user input and may include keys, a touch pad, a touch screen, a dial, buttons, and the like.

[0197] Fig.19 A cooking device, a user device, and a server according to an embodiment of the present disclosure are shown.

[0198] According to an embodiment of the present disclosure, the cooking device 100 communicates with the user device 1910 and the server 1920 through the communication module 1810. The cooking device 100 may be connected to another home appliance, the user device 1910, or the server 1920 through the network NET.

[0199] The server 1920 may manage user account information and information of the cooking device 100 connected to the user account. For example, a user may access the server 1920 through the user device 1910 and create a user account. The user account may be identified by an ID and a password set by the user. The server 1920 may register the cooking device 100 in the user account according to a predetermined process. For example, the server 1920 may register the cooking device 100 by connecting identification information (e.g., a serial number or a MAC address) of the cooking device 100 to the user account.

[0200] The user device 1910 may include: a communication module configured to communicate with the cooking device 100 and the server 1920; a user interface for receiving user input or outputting information to the user; at least one processor for controlling the operation of the user device 1910; and a memory for storing a program for controlling the operation of the user device 1910.

[0201] In one or more examples, the user device 1910 may be carried by the user or placed in the user's house or office. The user device 1910 may include, for example, a personal computer, a terminal, a portable phone, a smart phone, a handheld device, a wearable device, etc., but the embodiments of the present disclosure are not limited thereto.

[0202] In one or more examples, a program (e.g., an application) for controlling the cooking device 100 may be stored in a memory of the user device 1910. The user device 1910 may be sold with or without the application for controlling the cooking device 100 installed therein. When the user device 1910 is not sold with the application for controlling the cooking device 100, the user may download the application from an external server that provides the application and install the downloaded application on the user device 1910.

[0203] In one or more examples, the user may control the cooking device 100 by using an application installed on the user device 100. For example, when the user executes the application installed on the user device 1910, identification information of the cooking device 100 connected to the same user account as the user device 1910 may appear on the application execution window. The user may perform desired control on the cooking device 100 through the application execution window. When the user inputs a control command for the cooking device 100 through the application execution window, the user device 1910 may directly send the control command to the cooking device 100 through the network, or may send the control command to the cooking device 100 via the server 1920.

[0204] The network NET may include a wired network and a wireless network. A wired network may include a cable network or a telephone network, and a wireless network may include any network that sends and receives signals via radio waves. A wired network and a wireless network may be connected to each other.

[0205] The network NET may include a wide area network (WAN) such as the Internet, a local area network (LAN) formed around an AP, and a wireless personal area network (WPAN) that does not pass through an AP. The WPAN may include, but is not limited to, Bluetooth™ (IEEE 802.15.1), Zigbee (IEEE 802.15.4), Wi-Fi Direct, Near Field Communication (NFC), and Z-Wave.

[0206] The AP may connect a LAN to which the cooking apparatus 100 and the user device 1910 are connected to a WAN to which the server 1920 is connected. The cooking apparatus 100 or the user device 1910 may be connected to the server 1930 through the WAN.

[0207] The AP may communicate with the cooking device 100 and the user device 1910 by using wireless communication such as Wi-Fi™ (IEEE 802.11), and may access the WAN via wired communication.

[0208] The cooking device 100 may transmit information about operation or status to the server 1920 through the network NET. For example, the cooking device 100 may transmit information about operation or status to the server 1920 via Wi-Fi™ (IEEE 802.11). When the cooking device 100 does not have a Wi-Fi communication module, the cooking device 100 may transmit information about operation or status to the server 1920 through another home appliance having a Wi-Fi communication module. For example, when the cooking device 100 transmits information about operation or status to another home appliance through a WPAN (e.g., Bluetooth Low Energy (BLE) communication), the other home appliance may transmit information about operation or status of the cooking device 100 to the server 1920. The cooking device 100 may provide the server 1920 with information about operation or status of the cooking device 100 according to the user's previous approval. The information transmission to the server 1920 may be performed when a request is received from the server 1920, may be performed when a specific event occurs in the cooking device 100, or may be performed periodically or in real time.

[0209] When receiving the operation or status information from the cooking device 100, the server 1920 may update the previously stored information related to the cooking device 100. The server 1920 may transmit the information about the operation or status of the cooking device 100 to the user device 1910 through the network NET. When receiving a request from the user device 1910, the server 1920 may transmit the information about the operation or status of the cooking device 100 to the user device 1910. For example, when the user executes an application connected to the server 1920 from the user device 1910, the user device 1910 may request the server 1920 for the information about the operation or status of the cooking device 100, and receive the information about the operation or status of the cooking device 100. When receiving the operation or status information of the cooking device 100 from the cooking device 100, the server 1920 may transmit the information about the operation or status of the cooking device 100 to the user device 1910 in real time. For example, when the server 1920 receives information indicating that the operation of the cooking device 100 has been completed from the cooking device 100, the server 1920 may transmit the information indicating that the operation of the cooking device 100 has been completed to the user device 1910 in real time through the application installed in the user device 1910. The server 1920 may periodically transmit information on the operation or state of the cooking device 100 to the user device 1910. The user device 1910 may transmit information on the operation or state of the cooking device 100 to the user by displaying the information on the operation or state of the cooking device 100 on the application execution window.

[0210] The cooking device 100 may obtain various information from the server 1920 and provide the obtained information to the user. For example, the cooking device 100 may obtain information such as recipes and weather from the server 1920 and output the obtained information through the output interface 250. The cooking device 100 may receive a file for updating pre-installed software or data related to the pre-installed software from the server 1920 and update the pre-installed software or data related to the pre-installed software based on the received file.

[0211] The cooking device 100 may operate according to the control command received from the server 1920. For example, when the cooking device 100 obtains a previous approval from the user, the cooking device 100 may operate according to the control command received from the server 1920, the previous approval being configured to act according to the control command from the server 1920 even without user input. The control command received from the server 1920 may include, but is not limited to, a control command input by the user through the user device 1920 or a control command created by the server 1920 based on preset conditions.

[0212] Fig. 20is a diagram illustrating a process of obtaining food information from a user device according to an embodiment of the present disclosure.

[0213] According to an embodiment of the present disclosure, the user device 1910 may store and execute a first application for controlling the cooking device 100. The first application performs various operations, such as controlling the cooking device 100, providing recipe information, and inputting ingredient information. The user device 1910 may receive the ingredient information through the first application and send the ingredient information to the cooking device 100.

[0214] The first application may operate in cooperation with the server 1920 while being executed in the user device 1910. For example, the first application may perform information input and output operations, data processing operations, etc. through the user device 1910, and may perform communication with the cooking device 100, information search operations, etc. through the server 1920. Accordingly, the operations performed by the first application in the present disclosure may be performed by the user device 1910 or the server 1920.

[0215] According to an embodiment of the present disclosure, in operation 2010, the user device 1910 may provide recipe information through a first application. The recipe information may include ingredient information 2012 of the recipe. When the user selects a recipe (operation 2014), in operation 2020, the first application asks whether to continue cooking in the cooking device 100 by using the recipe. When the user selects (operation 222) the cooking device 100 to cook using the recipe through the first application, the user device 1910 may send the ingredient information 2012 included in the recipe to the cooking device 100. The user device 1910 may send the ingredient information to the cooking device 100 via the server 1920.

[0216] According to an embodiment of the present disclosure, the user device 1910 may send recipe information selected by the user to the cooking device 100. According to an embodiment of the present disclosure, the first application may send a control signal for controlling the operation of the cooking device 100 to the cooking device 100 according to the recipe information selected by the user. The cooking device 100 may set a cooking operation based on the recipe information received from the first application. For example, the cooking device 100 may perform operations such as temperature setting, cooking time setting, air blowing, and steaming based on the recipe information.

[0217] According to an embodiment of the present disclosure, the user device 1910 may directly receive ingredient information from the user through the first application. The first application may provide a menu in which the user may directly specify ingredients without selecting recipe information. When the user directly specifies and inputs ingredients, the first application sends information about the ingredients input by the user to the cooking device 100.

[0218] Fig.21is a diagram illustrating a process of obtaining food information from a user device according to an embodiment of the present disclosure.

[0219] According to an embodiment of the present disclosure, the first application of the user device 1910 can capture information related to the food by using a camera and obtain the food information from the captured image. According to an embodiment of the present disclosure, the first application can obtain the food information by using a visual code such as a barcode or a QR code.

[0220] In operation 2110, the first application may provide a visual code shooting menu 2112 for obtaining meal or convenience food information by using a visual code. When the user selects the visual code shooting menu 2112 (operation 2116), in operation 2120, the first application may execute a shooting mode configured to shoot a visual code 2122 by using a camera. The first application may identify a visual code from a captured image captured by the camera. The first application may obtain meal / convenience food information corresponding to the identified visual code. The meal / convenience food information may include recipes, ingredient information, or any other suitable information related to food or a cooking process.

[0221] According to an embodiment of the present disclosure, the first application may request the meal / convenience food information corresponding to the visual code from the server 1920, and may receive the meal / convenience food information corresponding to the visual code from the server 1920. According to an embodiment of the present disclosure, the first application stores the meal / convenience food information corresponding to the visual code in the memory of the user device 1910. The first application obtains the meal / convenience food information corresponding to the visual code by using the meal / convenience food information stored in the user device 1910.

[0222] The first application may send the meal / convenience food information to the cooking device 100. When the user requests to send the meal / convenience food information to the cooking device 100, the first application may send the meal / convenience food information to the cooking device 100. The first application may send the recipe, ingredient information, etc. included in the meal / convenience food information to the cooking device 100.

[0223] When the set meal / convenience food information is transmitted to the cooking device 100, the first application may transmit a control signal to the cooking device 100, the control signal being used to control the operation of the cooking device 100 according to the set meal / convenience food information. The cooking device 100 may set a cooking operation based on the recipe information included in the set meal / convenience food information received from the user device 1910. For example, the cooking device 100 may perform operations such as temperature setting, cooking time setting, air blowing, and steaming based on the recipe information included in the set meal / convenience food information.

[0224] Fig. 22is a diagram illustrating a process of receiving food information through an input interface of a cooking device according to an embodiment of the present disclosure.

[0225] According to an embodiment of the present disclosure, the cooking apparatus 100 may obtain food information through the input interface 1820 of the cooking apparatus 100 .

[0226] In operation 2210, the cooking apparatus 100 may provide a menu for inputting ingredient information before starting a cooking operation. The display 252 of the output interface 250 may output a message for commanding selection of ingredients.

[0227] In operation S2220, the cooking device 100 may receive a user input for selecting an ingredient. According to an embodiment of the present disclosure, the input interface 1820 may include a dial 1822, and the user may select one of a plurality of candidate ingredients by rotating the dial 1822 (operation 2222). The display 252 of the output interface 250 outputs information about the currently selected ingredient.

[0228] In operation S2230, the cooking apparatus 100 may complete the ingredient input and obtain the ingredient information. The cooking apparatus 100 may set the monitoring area based on the ingredient information obtained from the user input. The display 252 of the output interface 250 may output a message indicating that the ingredient selection has been completed.

[0229] Fig.23 A process of setting a monitoring area based on color data of a captured image according to an embodiment of the present disclosure is shown.

[0230] According to an embodiment of the present disclosure, the cooking apparatus 100 may determine a monitoring area based on color data of a captured image without recognizing ingredients.

[0231] The cooking device 100 may identify color data of each pixel from the captured image 2310. Fig.23 In the foregoing, for ease of description, a case is described where the color data of three regions are uniform in the captured image. However, as understood by those of ordinary skill in the art, any number of regions may be used. According to an embodiment of the present disclosure, the cooking device 100 may identify the color data of each pixel and determine the similarity with a fire-related color. According to an embodiment of the present disclosure, the cooking device 100 may identify the color data of each region including a plurality of pixels and determine the similarity with a fire-related color. For example, the cooking device 100 may identify color data in units of 3*3 blocks and determine the similarity with a fire-related color.

[0232] For ease of description, the embodiment of the present disclosure in which the cooking device 100 recognizes the color data of each pixel and determines the similarity with the fire-related color will be mainly described. However, this is only for ease of description, and the present disclosure does not exclude an embodiment in which the color data of each area including a plurality of pixels is recognized and the similarity with the fire-related color is determined.

[0233] The cooking device 100 may determine the similarity with the fire-related color for each pixel's color data. The cooking device 100 may calculate the similarity with the flame-related color and the similarity with the smoke-related color for each pixel's color data. Fig.23 The example corresponds to the case where the ingredient is raw poultry (e.g., chicken). Fig.23 In the example of , the first area 2312 of the raw chicken is an area corresponding to the skin color, where the similarity with the flame-related color is determined to be low and the similarity with the smoke-related color is determined to be high. The second area 2314 of the raw chicken is an area corresponding to the red color, where the similarity with the flame-related color is determined to be high and the similarity with the smoke-related color is determined to be low. The third area 2316 is a black area corresponding to the tray of the cooking device 100, where the similarity with the flame-related color is determined to be low and the similarity with the smoke-related color is determined to be low.

[0234] The cooking device 100 may select a monitoring area based on the similarity to the flame-related color and the similarity to the smoke-related color. The cooking device 100 may set pixels with a lower similarity to the flame-related color and a higher similarity to the smoke-related color as the monitoring area. The cooking device 100 may set pixels with a higher similarity to the flame-related color and a lower similarity to the smoke-related color as the monitoring area. The cooking device 100 may set pixels with a lower similarity to the flame-related color and a lower similarity to the smoke-related color as the flame monitoring area and the smoke monitoring area.

[0235] The cooking apparatus 100 may select a part or all of the area selected as the flame monitoring area as the flame monitoring area.

[0236] According to an embodiment of the present disclosure, when the cooking apparatus 100 selects a portion among the flame monitoring areas, the cooking apparatus 100 may select the flame monitoring areas at predetermined intervals within the pixels selected as the flame monitoring areas.

[0237] According to an embodiment of the present disclosure, when the pixels selected as the flame monitoring area have multiple colors, the cooking device 100 can select the flame monitoring area to include pixels of different colors. The cooking device 100 can divide the pixels of the flame monitoring area into multiple color groups and select at least one flame monitoring area for each color group in the multiple color groups.

[0238] The cooking apparatus 100 may select a part or all of the area selected as the smoke monitoring area as the smoke monitoring area.

[0239] According to an embodiment of the present disclosure, when the cooking apparatus 100 selects a portion of the smoke monitoring areas, the cooking apparatus 100 may select the smoke monitoring areas at predetermined intervals within the pixels selected as the smoke monitoring areas.

[0240] According to an embodiment of the present disclosure, when the pixels selected as the smoke monitoring area have multiple colors, the cooking device 100 can select the smoke monitoring area to include pixels of different colors. The cooking device 100 can divide the pixels of the smoke monitoring area into multiple color groups and select at least one smoke monitoring area for each color group in the multiple color groups.

[0241] Fig.24 is a flow chart of a process of detecting fire hazards according to an embodiment of the present disclosure.

[0242] According to an embodiment of the present disclosure, when the cooking device 100 selects a monitoring area based on the color data of the captured image 2310 without obtaining food information, the cooking device 100 may move the monitoring area during the process of detecting fire hazards. When the cooking device 100 does not obtain food information, the cooking device 100 does not have information about how the color data of the food changes during cooking. When the cooking device 100 does not obtain food information, the cooking device 100 may predict the shape change, contraction, expansion, or any other known change of the food during the cooking process. According to an embodiment of the present disclosure, when the cooking device 100 does not obtain food information and detects a fire-related color from the monitoring area, the cooking device 100 moves the monitoring area and monitors the fire-related color again.

[0243] Will refer to Fig.24 A case where a monitoring area 2410 is set in a captured image is described.

[0244] In operation S2402, the cooking apparatus 100 determines whether there are pixels having a high similarity to the fire-related color in the monitoring area 2410. The cooking apparatus 100 determines whether the similarity between the color data of each pixel in the monitoring area 2410 and the flame-related color or the smoke-related color is greater than or equal to a reference value.

[0245] When it is determined in operation S2402 that the similarity between the color data of the monitoring area 2410 and the fire-related color is less than the reference value, the cooking apparatus 100 may perform operation S2402 by using a next captured image.

[0246] When a color having a higher similarity to the fire-related color is detected in operation S2402, the cooking device 100 moves the monitoring area in operation S2404. When moving the monitoring area, the cooking device 100 selects a new monitoring area by selecting pixels having a color having a lower similarity to the fire-related color. Fig.24 In the example of FIG. 24 , the cooking apparatus 100 changes the monitoring area from the existing monitoring area 2410 to the new monitoring area 2412 (operation 2414 ).

[0247] When the existing monitoring area 2410 is a flame monitoring area, the cooking device 100 selects an area with a lower similarity to the flame-related color as the new monitoring area 2412. When the existing monitoring area 2410 is a smoke monitoring area, the cooking device 100 selects an area with a lower similarity to the smoke-related color as the new monitoring area 2412.

[0248] In operation S2406, the cooking device 100 determines whether the similarity between the color data of the new monitoring area 2412 and the fire-related color is equal to or greater than a reference value. When the similarity between the color data of the new monitoring area 2412 and the fire-related color is changed to be equal to or greater than the reference value, in operation S2408, the cooking device 100 determines that a fire hazard has been detected.

[0249] When it is determined in operation S2406 that the similarity between the color data of the new monitoring area 2412 and the fire-related color is less than the reference value, the cooking apparatus 100 performs operation S2406 by using the next captured image.

[0250] Fig.25 is a flow chart of a process of detecting a fire hazard based on a color change around an edge and a shape change of the edge according to an embodiment of the present disclosure.

[0251] According to an embodiment of the present disclosure, when fire-related colors are detected based on color data of the captured image 2310, the cooking device 100 detects fire hazards by determining whether color changes around the edge and shape changes of the edge correspond to fire characteristics based on characteristics of flames or smoke.

[0252] Fig.25 is an example of a situation where a flame is detected in monitoring area 2510.

[0253] In operation S2502, the cooking apparatus 100 determines whether there are pixels having a high similarity to the fire-related color in the monitoring area 2510. The cooking apparatus 100 determines whether the similarity of color data of each pixel in the monitoring area 2510 to the flame-related color or the smoke-related color is greater than or equal to a reference value.

[0254] When it is determined in operation S2502 that the similarity between the color data of the monitoring area 2510 and the fire-related color is less than the reference value, the cooking apparatus 100 may perform operation S2502 using a next captured image.

[0255] When it is determined in operation S2502 that the similarity between the color data of the monitoring area 2510 and the fire-related color is equal to or greater than the reference value, in operation S2504, the cooking apparatus 100 detects a peripheral color change or an edge shape change of the edge.

[0256] For example, when a monitoring area 2510 having a color similarity to a flame-related color equal to or greater than a reference value is detected from the monitoring area 2510, the cooking apparatus 100 monitors a shape change of an area having a color having a higher similarity to the flame-related color.

[0257] For example, when a monitoring area 2510 having a color similarity to a smoke-related color equal to or greater than a reference value is detected from the monitoring area 2510 , the cooking apparatus 100 monitors a color change around an edge of the food.

[0258] In operation S2506, the cooking apparatus 100 determines whether the edge periphery color change or edge shape change corresponds to the flame characteristic.

[0259] For example, the cooking device 100 determines whether the shape change of the area having a color with a high similarity to the flame-related color in the monitoring area 2510 corresponds to the flame. When the shape change corresponds to the flame, the cooking device 100 determines that the edge shape change in the monitoring area 2510 corresponds to the flame characteristic.

[0260] For example, when the color similarity between the monitoring area 2510 and the smoke-related color is equal to or greater than the reference value, the cooking device 100 determines whether the color around the edge of the food has changed evenly. When the color around the edge of the food has changed evenly, the cooking device 100 determines that the color change around the edge corresponds to the fire characteristic.

[0261] When it is determined in operation S2506 that the color change around the edge corresponds to the fire characteristic, the cooking apparatus 100 determines in operation S2508 that a fire hazard has been detected. When it is determined in operation S2506 that the color change around the edge does not correspond to the flame characteristic, the cooking apparatus 100 performs operation S2502 by using the next captured image.

[0262] Now refer to Figure 26 to Figure 29 An embodiment of outputting a fire hazard notification is described.

[0263] Fig.26 is a diagram illustrating a process of outputting a fire hazard according to an embodiment of the present disclosure.

[0264] According to an embodiment of the present disclosure, when the cooking device 100 detects a fire hazard, the cooking device 100 may output the fire hazard through the output interface 250. For example, the cooking device 100 may display a fire hazard message through a display. For example, the cooking device 100 may output the fire hazard message as an audio signal through a speaker.

[0265] According to an embodiment of the present disclosure, the fire hazard notification may include at least one of information indicating that a fire hazard has been detected, the type of the fire hazard, response guidance information, or cooking device operation change information. The type of the fire hazard may include information such as smoke detection, flame detection, or temperature spike. The response guidance information may include, for example, a request to open a door, a request to cut off power, or a request to prohibit contact. The cooking device operation change information may include information such as door unlocking, cooking operation interruption, automatic power cutoff, or steam discharge from a cooking chamber.

[0266] In operation 2610, the cooking device 100 may output information indicating that a fire hazard has occurred. The display 252 of the interface 250 displays a message indicating that a fire hazard has occurred. The cooking device 100 may output a fire alarm sound or fire hazard voice information through a speaker of the output interface 250.

[0267] In operation 2620, the cooking apparatus 100 may output the type of fire hazard. For example, the cooking apparatus 100 may display a message indicating that a flame has been detected or a message indicating that smoke has been detected through the display 252 of the output interface 250. The cooking apparatus 100 may output a voice message indicating that a flame has been detected or a voice message indicating that smoke has been detected through the speaker of the output interface 250.

[0268] In operation 2630, the cooking device 100 may output cooking device operation change information for fire hazards. For example, the cooking device 100 may display a message indicating that the door has been unlocked, a message indicating that cooking is stopped, and a message indicating that power is cut off through the display 252 of the output interface 250. The cooking device 100 may output a voice message indicating that the door is unlocked, a voice message indicating that cooking is stopped, and a voice message indicating that power is cut off through the speaker of the output interface 250.

[0269] The sequence of operations 2610, 2620, and 2630 may be performed continuously or non-continuously. The cooking apparatus 100 may perform at least one of operations 2610, 2620, and 2630 or a combination thereof.

[0270] Fig. 27is a diagram illustrating a process of outputting a fire hazard through a user device according to an embodiment of the present disclosure.

[0271] According to an embodiment of the present disclosure, when a fire hazard is detected, the cooking device 100 may output fire hazard information through an external device of the user device 1910 or any other suitable device. When a fire hazard is detected, the cooking device 100 may determine that a fire hazard event has occurred. When a fire hazard event has occurred, the cooking device 100 may generate fire hazard information. The fire hazard information may include at least one of information indicating that a fire hazard has been detected, the type of fire hazard, response guidance information, or cooking device operation change information. The cooking device 100 may send the fire hazard information to the user device 1910. The cooking device 100 may also send an output request for a fire hazard notification, account information for registering the cooking device 100, identification information (e.g., MAC address) of the cooking device 100, etc. together with the fire hazard information to the user device 1910.

[0272] As referenced above Fig.19 As described above, the cooking device 100 can communicate with the user device 1910 through the server 1920. The cooking device 100 can send the fire hazard information to the server 1920 through the communication module 1810. The server 1920 sends the fire hazard information to the user device 1910 registered with the same account as the cooking device 100.

[0273] The user device 1910 may receive and output the fire hazard information. The user device 1910 may display the fire hazard information 2720 through the first application. When the first application is not running, the user device 1910 may display the fire hazard information 2720 through a pop-up message provided by the first application. The user device 1910 may output vibration or sound while displaying the fire hazard information 2720.

[0274] According to an embodiment of the present disclosure, the first application may display a real-time captured image obtained by photographing the inside of the cooking chamber 230 together with the fire hazard information 2720. The cooking device 100 may transmit the real-time captured image to the server 1920, and the server 1920 may stream the real-time captured image to the cooking device 100.

[0275] According to an embodiment of the present disclosure, the first application may display a captured image 2710 in which a fire hazard is detected. The first application may obtain a still image or a moving image in which a fire hazard is detected from the captured image. The first application may display the still image or the moving image in which a fire hazard is detected together with the fire hazard information 2720.

[0276] Fig.28is a block diagram of a structure of a cooking device according to an embodiment of the present disclosure.

[0277] According to an embodiment of the present disclosure, the cooking apparatus 100 may circulate air inside the cooking chamber 230 or discharge steam into the cooking chamber 230. When a fire hazard is detected, the cooking apparatus 100 may prevent a fire by circulating air inside the cooking chamber 230 or discharging steam into the cooking chamber 230.

[0278] When the camera 220 photographs the inside of the cooking chamber 230, the cooking apparatus 100 can prevent the inside of the cooking chamber 230 from being properly photographed due to smoke. For example, when smoke is detected inside the cooking chamber 230, the cooking apparatus 100 removes the smoke to enable monitoring of the cooking chamber 230.

[0279] According to an embodiment of the present disclosure, the cooking device 100 may include a processor 210, a camera 220, a cooking chamber 230, a memory 240, an output interface 250, a ventilation fan 2810, and a steam exhaust module 2820. Fig.28 Main description and Figure 2 The cooking device 100 is different.

[0280] The ventilation fan 2810 can discharge the air inside the cooking chamber 230 to the outside. The ventilation fan 2810 may include fan-shaped blades for circulating air, a motor, and a driving circuit for circulating air. The ventilation fan 2810 may also include a predetermined filter. The processor 210 may output a driving signal for driving the ventilation fan 2810 to the ventilation fan 2810. The ventilation fan 2810 may operate under the control of the processor 210. The ventilation fan 2810 may operate at one or more intensity levels. The processor 210 may adjust the intensity of the ventilation fan 2810.

[0281] The steam discharge module 2820 discharges steam into the cooking chamber 230. The steam discharge module 2820 may generate steam by heating water using a heating device, and discharge the steam into the cooking chamber 230. The steam discharge module 2820 may discharge steam having one or more intensity levels. The processor 210 may generate a control signal for controlling the steam discharge module 2820, and output the control signal to the steam discharge module 2820. The processor 210 may control the steam discharge module 2820 to adjust the temperature or intensity of the steam of the steam discharge module 2820.

[0282] According to an embodiment of the present disclosure, when a fire hazard is detected, the processor 210 may operate the ventilation fan 2810 and the steam exhaust module 2820. When a fire hazard is detected and steam is exhausted into the cooking chamber 230, the processor 210 may output a message indicating that a fire hazard is detected and steam is exhausted through the output interface 250 or the user device 1910. When a fire hazard is not detected after the fire hazard is detected and the steam is exhausted, the processor 210 may control the steam exhaust module 2820 to stop exhausting steam. When a fire hazard is detected and the ventilation fan 2810 is operated, the processor 210 may output a message indicating that a fire hazard is detected and the ventilation fan 2810 is operated through the output interface 250 or the user device 1910. When a fire hazard is detected and no fire hazard is detected after the ventilation fan 2810 is operated, the processor 210 may control the ventilation fan 2810 to stop its operation.

[0283] According to an embodiment of the present disclosure, when flame is detected, the processor 210 may control the steam discharge module 2820 to discharge steam to the cooking chamber 230 .

[0284] According to an embodiment of the present disclosure, when smoke is detected, the processor 210 may operate the ventilation fan 2810. The processor 210 may adjust the intensity of the ventilation fan 2810 according to the intensity of the smoke. The processor 210 may determine the concentration of the smoke, and when it is determined that the smoke concentration is high, the intensity of the ventilation fan 2810 is set to high, and when it is determined that the smoke concentration is low, the intensity of the ventilation fan 2810 is set to low. The processor 210 may determine the concentration of the smoke based on the color data of the captured image. When the similarity between the color data of the captured image and the smoke-related color is high, the processor 210 may determine that the smoke concentration is high. When the similarity between the color data of the captured image and the smoke-related color is low, the processor 210 may determine that the smoke concentration is low.

[0285] According to an embodiment of the present disclosure, when smoke is detected, the processor 210 may discharge smoke through the steam discharge module 2820. The processor 210 may adjust the steam discharge amount of the steam discharge module 2820.

[0286] According to an embodiment of the present disclosure, when smoke is detected, the processor 210 may operate both the ventilation fan 2810 and the steam exhaust module 2820. When smoke is detected, the processor 210 may determine whether to operate only one of the ventilation fan 2810 and the steam exhaust module 2820 or both of them according to the concentration of the smoke. For example, when smoke is detected, when the smoke concentration is low, the processor 210 operates only the ventilation fan 2810, and when the smoke concentration is high, the processor 210 operates both the ventilation fan 2810 and the steam exhaust module 2820.

[0287] Fig.29 is a diagram illustrating a process of monitoring a cooking operation of a cooking device according to an embodiment of the present disclosure.

[0288] According to an embodiment of the present disclosure, the cooking device 100 transmits a captured image to the user device 1910. The user device 1910 displays the captured image through the first application. The first application may provide information (eg, operation mode, temperature, etc.) of the cooking device 100.

[0289] According to an embodiment of the present disclosure, when smoke is generated inside the cooking chamber 230 while the first application is monitoring the inside of the cooking chamber 230, the cooking device 100, the server 1920, or the user device 1910 may control the cooking device 100 to remove smoke from the cooking chamber 230. While the first application is displaying a captured image obtained by photographing the inside of the cooking chamber 230 of the cooking device 100 as in operation 2910, smoke may be generated inside the cooking chamber 230 in operation 2920. In this case, according to an embodiment of the present disclosure, the cooking device 100 removes smoke from the cooking chamber 230 by using the ventilation fan 2910 or the steam exhaust module 2820. Due to the removal of the smoke, in operation 2930, the first application may provide a captured image from which the smoke has been removed.

[0290] According to an embodiment of the present disclosure, the first application may have a menu for requesting the cooking device 100 to remove smoke. In response to the smoke removal request, the first application sends the smoke removal request to the cooking device 100. The cooking device 100 receives the smoke removal request and removes smoke inside the cooking chamber 230 by controlling the ventilation fan 2810 or the steam exhaust module 2820.

[0291] Fig.30 is a block diagram of a structure of a cooking device according to an embodiment of the present disclosure.

[0292] The cooking device 3000 according to an embodiment of the present disclosure may include a sensor 3010, an output interface 3020, an input interface 3030, a memory 3040, a communication interface 3050, a cooking module 3060, a camera 3070, a power module 3080, and a processor 3090. The cooking device 3000 may be Fig.30 Various combinations of components shown are configured and are not Fig.30 All components shown are required.

[0293] Fig.30 The cooking device 3000 corresponds to the above reference Figure 2 , Fig. 9 , Fig.18 and Fig.28 The cooking device 100 described herein. The camera 3070 corresponds to Figure 2The camera 220. The memory 3040 corresponds to Figure 2 The communication interface 3050 corresponds to the memory 240. Fig.18 The communication module 1810. The processor 3090 corresponds to Figure 2 The temperature sensor 3011 corresponds to the processor 210. Fig. 9 The output interface 3020 corresponds to the temperature sensor 910. Figure 2 The output interface 250 corresponds to the input interface 3030. Fig.18 The cooking chamber 3061 corresponds to the input interface 1820. Figure 2 The cooking chamber 230. The ventilation fan 3062 corresponds to Fig.28 The ventilation fan 2810. The steam exhaust module 3063 corresponds to Fig.28 Steam exhaust module 2820.

[0294] The sensor 3010 may include various types of sensors, such as a temperature sensor 3011 and a smoke sensor 3012 .

[0295] The output interface 3020 may include one or both of a display 3021 and a speaker 3022 , or a combination thereof. The output interface 3020 outputs various notifications, messages, information, etc. created by the processor 3090 .

[0296] The input interface 3030 may include a key 3031 , a touch pad 3032 , a dial 3033 , etc. The input interface 3030 receives a user input and sends the user input to the processor 3090 .

[0297] The memory 3040 stores various information, data, instructions, programs, etc. required for the operation of the cooking device 3000. The memory 3040 may include one or both of a volatile memory and a non-volatile memory, or a combination thereof. The memory 3040 may include at least one type of storage medium selected from the following: a flash memory type, a hard disk type, a multimedia card micro, a card-type memory (e.g., a secure digital (SD) or extreme digital (XD) memory), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable ROM (EEPROM), a programmable ROM (PROM), a magnetic memory, a magnetic disk, and an optical disk. The cooking device 3000 may operate a network storage device or a cloud server that performs a storage function on the Internet.

[0298] The communication module 3050 may include at least one of a short-range communication interface 3052 or a mobile communication interface 3054, or a combination thereof. The communication module 3050 may include at least one antenna for wirelessly communicating with other devices.

[0299] The short-range communication interface 3052 may include, but is not limited to, a Bluetooth communication interface, a BLE communication interface, an NFC interface, a WLAN (or Wi-Fi) communication interface, a Zigbee communication interface, an IrDA communication interface, a WFD communication interface, a UWB communication interface, an Ant+ communication interface, and a microwave (uWave) communication interface.

[0300] The mobile communication interface 3054 may exchange wireless signals with at least one selected from a base station, an external terminal, and a server on a mobile communication network. Here, examples of wireless signals may include voice call signals, video call signals, and various types of data exchanged according to text / multimedia messages.

[0301] The cooking module 3060 may include a cooking chamber 3061, a ventilation fan 3062, a steam exhaust module 3063, a door 3064, a heating module 3065, etc. The cooking chamber 3061 accommodates food. The ventilation fan 3062 circulates air inside the cooking chamber 3061. The steam exhaust module 3063 exhausts steam into the cooking chamber 3061. The door 3064 opens / closes the cooking chamber 3061. The heating module 3065 supplies heat to the cooking chamber 3061 to adjust the internal temperature of the cooking chamber 3061.

[0302] Camera 3070 photographs the interior of cooking chamber 3061 .

[0303] The power module 3080 supplies power to the cooking device 3000. The power module 3080 may include a battery, a power driving circuit, a converter, a conversion circuit, etc. The power module 3080 is connected to an external power source to receive power.

[0304] The processor 3090 may control all operations of the cooking apparatus 3000. The processor 3090 may control components of the cooking apparatus 3000 by executing a program stored in the memory 3040.

[0305] According to an embodiment of the present disclosure, the processor 3090 may include a separate NPU for performing operations of a machine learning model. The processor 3090 may also include a central processing unit (CPU), a graphics processing unit (GPU), and the like.

[0306] Processor 3090 can perform operations of the cooking device 3000, such as control of operating modes, determination and control of travel routes, obstacle identification, control of cleaning operations, location identification, communication with an external server, monitoring of remaining battery capacity, and control of battery charging operations.

[0307] The machine-readable storage medium may be provided as a non-transitory storage medium. A "non-transitory storage medium" is a tangible device and simply means that it does not contain signals (e.g., electromagnetic waves). The term does not distinguish between a case where data is semi-permanently stored in a storage medium and a case where data is temporarily stored in a storage medium. For example, a non-transitory recording medium may include a buffer that temporarily stores data.

[0308] According to an embodiment of the present disclosure, the method can be provided by including the method according to various disclosed embodiments in a computer program product. The computer program product (which is a commodity) can be traded between a seller and a buyer. The computer program product is distributed in the form of a device-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)), or can be distributed online (e.g., downloaded or uploaded) directly through an application store or between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable application) can be at least temporarily stored in a device-readable storage medium (e.g., a memory of a manufacturer's server, a service of an application store, or a relay server), or can be temporarily generated.

[0309] According to aspects of the present disclosure, the cooking device 100 may include: a cooking chamber 230 configured to accommodate an object to be cooked; a camera 220 configured to photograph the interior of the cooking chamber; an output interface 250; a memory 240 configured to store at least one instruction; and at least one processor 210. The at least one processor 210 is configured to execute at least one instruction to: detect a fire hazard during cooking based on a captured image obtained by the camera 220, and output a fire hazard notification through the output interface 250 based on the detected fire hazard.

[0310] According to an embodiment of the present disclosure, at least one processor 210 may also be configured to execute at least one instruction to: identify at least one monitoring area from a captured image based on color data of each pixel of the captured image, and determine whether a fire hazard is detected based on the similarity between the color data of the at least one monitoring area and a predetermined fire-related color.

[0311] According to an embodiment of the present disclosure, at least one processor 210 may also be configured to execute at least one instruction to: identify the type and color of ingredients present in the cooking chamber from the captured image, and based on the identified type and color of ingredients, determine an area corresponding to ingredients that do not correspond to fire-related colors as a monitoring area.

[0312] According to an embodiment of the present disclosure, at least one processor 210 may further be configured to execute at least one instruction to: determine an area corresponding to the following ingredients as a monitoring area: the color before cooking the ingredients and the color after cooking the ingredients do not correspond to a fire-related color.

[0313] According to an embodiment of the present disclosure, at least one processor 210 may further be configured to execute at least one instruction to: identify at least one area that does not correspond to a fire-related color from a captured image, and identify the at least one area that does not correspond to a fire-related color as at least one monitoring area.

[0314] According to an embodiment of the present disclosure, at least one processor 210 may also be configured to execute at least one instruction to: based on a determination that a degree of similarity between color data of a first monitoring area among at least one monitoring area and a fire-related color is equal to or greater than a reference value, change the first monitoring area to another area having a lower degree of similarity to the fire-related color; and based on a determination that a fire hazard has been detected, determine that the degree of similarity between the changed first monitoring area and the fire-related color is changed to a reference value or a greater value.

[0315] According to an embodiment of the present disclosure, at least one processor 210 may also be configured to execute at least one instruction to: calculate the distance between a first coordinate of color data of at least one monitoring area defined in a color space and a second coordinate of a fire-related color defined in the color space, and when the distance is lower than a reference value, detect the presence of a fire hazard.

[0316] According to an embodiment of the present disclosure, a reference value may be determined according to the type of food recognized from a captured image.

[0317] According to an embodiment of the present disclosure, the fire-related colors may include a smoke-related color corresponding to smoke and a flame-related color corresponding to flame.

[0318] According to an embodiment of the present disclosure, the cooking device 100 may further include a temperature sensor 910 configured to measure the internal temperature of the cooking chamber 230, and at least one processor 210 may further be configured to execute at least one instruction to: repeat the determination of the fire hazard based on the temperature detection value of the temperature sensor 910 based on the fire hazard detected during cooking according to the captured image.

[0319] According to an embodiment of the present disclosure, the cooking device 100 may further include an input interface 1820 and a heating module 920, wherein the heating module 920 is configured to heat the interior of the cooking chamber, and the at least one processor 210 may further be configured to execute at least one instruction to: receive user input for selecting a set temperature through the input interface 1820; perform temperature control by controlling the heating module 920 based on the temperature detection value of the temperature sensor 910 and the set temperature; detect fire hazards during the cooking process based on a captured image; and detect the occurrence of a fire based on a determination that the temperature detection value is higher than a predicted temperature predicted by temperature control.

[0320] According to an embodiment of the present disclosure, the cooking device 100 may further include a communication module 1810, and the at least one processor 210 is further configured to execute at least one instruction to: based on detecting a fire hazard, send a fire hazard notification to at least one external device through the communication module 1810.

[0321] According to an embodiment of the present disclosure, the cooking device 100 may further include: a ventilation fan 2810 configured to circulate internal air to the outside; and a steam exhaust module 2820 configured to supply steam to the interior of the cooking chamber 230, and at least one processor 210 may further be configured to execute at least one instruction to: based on the detection of a fire hazard, perform at least one of an operation for controlling the ventilation fan 2810 to exhaust smoke from the cooking chamber 210 or an operation for controlling the steam exhaust module 2820 to supply steam to the interior of the cooking chamber 230.

[0322] According to an embodiment of the present disclosure, the cooking device 100 may further include: a ventilation fan 2810 configured to circulate internal air to the outside; and a steam exhaust module 2820 configured to supply steam to the interior of the cooking chamber 230, and at least one processor 210 may further be configured to execute at least one instruction to: based on the smoke detected from the captured image, perform at least one of an operation for controlling the ventilation fan 2810 to exhaust smoke from the cooking chamber 230 or an operation for controlling the steam exhaust module 2820 to supply steam to the interior of the cooking chamber 230.

[0323] According to an embodiment of the present disclosure, a method for controlling a cooking device, the cooking device including a camera for photographing the interior of a cooking chamber accommodating an object to be cooked, the method may include: photographing the interior of the cooking chamber by using the camera; detecting a fire hazard during cooking based on a captured image obtained by photographing the interior of the cooking chamber; and outputting a fire hazard notification based on the detection of the fire hazard.

[0324] According to another aspect of the present disclosure, a non-transitory computer-readable recording medium has recorded thereon a computer program which, when executed by a computer, performs a method for controlling a cooking device.

Claims

1. A cooking device (100), include: a cooking chamber (230) configured to accommodate an object to be cooked; A camera (220) configured to photograph the interior of the cooking chamber; Output interface (250); A memory (240) configured to store at least one instruction; as well as At least one processor (210), wherein the at least one processor (210) is configured to execute the at least one instruction to: detecting a fire hazard during cooking based on a captured image obtained by the camera (220); as well as Based on the detection of the fire hazard, a fire hazard notification is output through the output interface (250).

2. The cooking device (100) according to claim 1, in, The at least one processor (210) is further configured to execute the at least one instruction to: identifying at least one monitoring area from the captured image based on color data of each pixel of the captured image; as well as Based on the similarity between the color data of the at least one monitoring area and a predetermined fire-related color, it is determined whether the fire hazard is detected.

3. The cooking device (100) according to claim 2, in, The at least one processor (210) is further configured to execute the at least one instruction to: identifying the type and color of food present in the cooking chamber (230) from the captured image; and Based on the identified types and colors of the food materials, an area corresponding to the food materials that do not correspond to the fire-related color is determined as the monitoring area.

4. The cooking device (100) according to claim 3, in, The at least one processor (210) is further configured to execute the at least one instruction to determine, as the monitoring area, an area corresponding to the following food: the color of the food before cooking the food and the color of the food after cooking the food do not correspond to the fire-related color.

5. The cooking device (100) according to any one of claims 2 to 4, in, The at least one processor (210) is further configured to execute the at least one instruction to: identifying at least one region from the captured image that does not correspond to the fire-related color; and The at least one area not corresponding to the fire-related color is identified as at least one monitoring area.

6. The cooking device (100) according to claim 5, in, The at least one processor (210) is further configured to execute the at least one instruction to: Based on a determination that color data of a first monitoring area among the at least one monitoring area has a similarity to the fire-related color and the similarity is equal to or greater than a reference value, changing the first monitoring area to another area having a lower similarity to the fire-related color; as well as Based on the determination that the similarity between the changed first monitoring area and the fire-related color is changed to the reference value or more, it is determined that a fire hazard has been detected.

7. The cooking device (100) according to any one of claims 2 to 6, in, The at least one processor (210) is further configured to execute the at least one instruction to: calculating a distance between a first coordinate of color data of the at least one monitoring area defined on a color space and a second coordinate of the fire-related color defined on the color space; as well as When the distance is lower than a reference value, a fire hazard is detected.

8. The cooking device (100) according to any one of claims 2, 3, 6 and 7, in, The reference value is determined according to the type of food recognized from the captured image.

9. The cooking device (100) according to any one of claims 2 to 8, in, The fire-related colors include smoke-related colors corresponding to smoke and flame-related colors corresponding to flames.

10. The cooking device (100) according to any one of claims 1 to 9, in, The cooking device (100) further comprises a temperature sensor (910), wherein the temperature sensor (910) is configured to measure the internal temperature of the cooking chamber (230); and The at least one processor (210) is further configured to execute the at least one instruction to: based on detecting a fire hazard during cooking according to the captured image, repeat the determination of the fire hazard based on the temperature detection value of the temperature sensor (910).

11. The cooking device (100) according to claim 10, in, The cooking device (100) further comprises: Input interface (1820); and a heating module (920) configured to heat the interior of the cooking chamber, and The at least one processor (210) is further configured to execute the at least one instruction to: receiving a user input for selecting a set temperature through the input interface (1820); performing temperature control by controlling the heating module (920) based on the set temperature and the temperature detection value of the temperature sensor (910); and Based on the captured image, a fire hazard during cooking is detected, and based on a determination that the temperature detection value is higher than a predicted temperature predicted by the temperature control, occurrence of a fire is detected.

12. The cooking device (100) according to any one of claims 1 to 11, in, The cooking device (100) further comprises a communication module (1810), and The at least one processor (210) is further configured to execute the at least one instruction to: based on detecting the fire hazard, send the fire hazard notification to at least one external device through the communication module (1810).

13. The cooking device (100) according to any one of claims 1 to 12, further comprising: include: a ventilation fan (2810) configured to circulate interior air to the exterior; as well as a steam exhaust module (2820) configured to supply steam to the interior of the cooking chamber (230), and Wherein, the at least one processor (210) is also configured to execute the at least one instruction to: based on the detection of the fire hazard, perform at least one of an operation for controlling the ventilation fan (2810) to discharge smoke from the cooking chamber (230) or an operation for controlling the steam discharge module (2820) to supply steam to the interior of the cooking chamber (230).

14. The cooking device (100) according to any one of claims 1 to 13, further comprising: include: a ventilation fan (2810) configured to circulate interior air to the exterior; as well as a steam exhaust module (2820) configured to supply steam to the interior of the cooking chamber (230), and The at least one processor (210) is also configured to execute the at least one instruction to: based on detecting smoke from the captured image, perform at least one of an operation for controlling the ventilation fan (2810) to discharge smoke from the cooking chamber (230) or an operation for controlling the steam discharge module (2820) to supply steam to the interior of the cooking chamber (230).

15. A method of controlling a cooking device, in, The cooking device includes a camera that photographs the interior of a cooking chamber that accommodates an object to be cooked, and the method includes: photographing the interior of the cooking chamber by using the camera; detecting a fire hazard during cooking based on a captured image obtained by photographing the interior of the cooking chamber; and Based on the detection of the fire hazard, a fire hazard notification is output.

Images