Cooking apparatus and control method therefor

Abstract

A cooking apparatus and a control method therefor are disclosed. In particular, a cooking apparatus according to the present disclosure comprises a cooking chamber, a camera, a display, a memory, and a processor, wherein the memory stores previous cooking details, and the processor controls the camera to capture an image of an object to be cooked when the object to be cooked is put into the cooking chamber, controls the display to display a UI recommending registration of a cooking method according to the previous cooking details as a user menu for the object to be cooked, on the basis of the similarity between a feature point extracted from the image obtained by capturing the object to be cooked and a feature point of the object to be cooked included in the previous cooking details being greater than or equal to a first reference value, and stores the user menu including the feature points and the cooking method according to the previous cooking details in the memory when the UI is selected.

Description

Cooking device and control method thereof

[0001] The present disclosure relates to a cooking device and a method for controlling the same.

[0002] Thanks to advancements in electronic technology, various types of cooking devices are being developed and distributed.

[0003] After placing the food to be cooked into the cooking device, the user can cook the food by directly setting the cooking time, cooking temperature, etc.

[0004] The foods frequently cooked may vary depending on each user's lifestyle and preferences. In such cases, users may experience the inconvenience of having to set their own cooking methods for these frequently prepared foods every time they cook. Therefore, there is a growing need for technology that can efficiently recognize foods and recommend optimal cooking methods by accumulating data on foods that users continuously cook.

[0005] According to at least one embodiment of the present disclosure, a cooking device comprises a cooking chamber, a camera, a display, a memory, and a processor, wherein the memory stores a previous cooking history, and the processor controls the camera to photograph the cooking subject when a cooking subject is introduced into the cooking chamber, and controls the display to display a UI that recommends registering a cooking method according to the previous cooking history as a user menu for the cooking subject based on the similarity between a feature point extracted from the image of the cooking subject and a feature point of the cooking subject included in the previous cooking history being greater than or equal to a first reference value, and when the UI is selected, stores the user menu including the feature point and the cooking method according to the previous cooking history in the memory.

[0006] The processor controls the display to display a UI that can register a cooking method input for the cooking target as a user menu based on the similarity between a feature point extracted from the captured image and a feature point of the cooking target included in the previous cooking history being less than the first reference value, and when the UI that can register the cooking method as a user menu is selected, the feature point extracted from the cooking target and the input cooking method can be stored in the memory as a user menu.

[0007] The apparatus further includes at least one cooking module for cooking the cooking target introduced into the cooking chamber, and the processor can control the at least one cooking module to cook the cooking target using a cooking method registered in the pre-registered user menu based on the similarity between a feature point extracted from a captured image of the cooking target and a feature point included in a pre-registered user menu being greater than or equal to a second reference value.

[0008] The apparatus further includes at least one cooking module for cooking the cooking target introduced into the cooking chamber, wherein the memory stores information regarding a basic menu including a feature point set for each cooking target and a plurality of standard cooking methods, and the processor controls the display to display identification information for at least one pre-registered user menu corresponding to the cooking target and identification information for at least one basic menu corresponding to the cooking target on a single screen based on a feature point extracted from a captured image of the cooking target, and controls the at least one cooking module to cook the cooking target using a cooking method corresponding to the selected identification information based on the selection of one of the identification information displayed on the display.

[0009] The processor can update the previously registered user menu based on the new cooking method, based on the identification of a user menu similar to the cooking target among the previously registered user menus and the input of a new cooking method different from the cooking method registered in the identified user menu.

[0010] The processor controls the display to display a list of pre-registered user menus based on the fact that no user menu similar to the cooking target is identified among the pre-registered user menus and a cooking method is entered for the cooking target, and when one user menu among the list is selected, the selected user menu can be updated based on the entered cooking method.

[0011] The processor controls the display to display a list of pre-registered user menus and a UI for registering a new user menu on one screen based on the input of a cooking method for the cooking target, and based on the selection of the UI for registering a new user menu, stores the name of the cooking target received from the user and the input cooking method as the new user menu in the memory, and when one of the user menus in the list is selected, the selected user menu can be updated based on the input cooking method.

[0012] The processor can update the feature points for the cooking target and the cooking method entered for the cooking target in the previous cooking history based on the fact that the recommendation of the UI is not selected.

[0013] According to at least one embodiment of the present disclosure, a control method for a cooking device comprises the steps of: photographing a cooking target when the cooking target is introduced into the cooking chamber; displaying a UI that recommends registering a cooking method according to the previous cooking history as a user menu for the cooking target based on the similarity between a feature point extracted from the image of the cooking target and a feature point of the cooking target included in the previous cooking history being greater than or equal to a first reference value; and saving the user menu including the feature point and the cooking method according to the previous cooking history when the UI is selected.

[0014] Based on the fact that the similarity between the feature point extracted from the above-mentioned captured image and the feature point of the cooking target included in the above-mentioned previous cooking history is less than the above-mentioned first reference value, the method may further include the step of displaying a UI that can register a cooking method input for the above-mentioned cooking target to the user menu, and when the UI that can register the above-mentioned cooking method to the user menu is selected, the method may further include the step of saving the feature point extracted from the above-mentioned cooking target and the above-mentioned cooking method to the user menu.

[0015] Based on the fact that the similarity between a feature point extracted from a captured image of the cooking target and a feature point included in a pre-registered user menu is greater than or equal to a second threshold value, the method may further include the step of cooking the cooking target using a cooking method registered in the pre-registered user menu.

[0016] The method may further include the step of displaying identification information for at least one pre-registered user menu corresponding to the cooking target and identification information for at least one basic menu corresponding to the cooking target on a single screen based on feature points extracted from a captured image of the cooking target, and the step of cooking the cooking target using a cooking method corresponding to the selected identification information based on the selection of one of the displayed identification information.

[0017] The method may further include a step of updating the previously registered user menu based on the new cooking method, based on the identification of a user menu similar to the cooking target among the previously registered user menus and the input of a new cooking method different from the cooking method registered as the identified user menu.

[0018] The method may further include the step of displaying a list of previously registered user menus based on the fact that no user menu similar to the cooking target is identified among the previously registered user menus and a cooking method is entered for the cooking target, and, when one user menu is selected from the list, the step of updating the selected user menu based on the entered cooking method.

[0019] Based on the input of a cooking method for the above-mentioned cooking target, the method may further include the steps of displaying a list of previously registered user menus and a UI for registering a new user menu on a single screen; based on the selection of the UI for registering a new user menu, setting the name of the above-mentioned cooking target and the input cooking method received from the user as the new user menu; and when one user menu among the list is selected, updating the selected user menu based on the input cooking method.

[0020] FIG. 1 is a schematic diagram showing a cooking apparatus according to at least one embodiment of the present disclosure.

[0021] FIG. 2 is a block diagram illustrating the configuration of a cooking device according to at least one embodiment of the present disclosure.

[0022] FIGS. 3 to 5 are drawings for illustrating cases where registration is recommended in a user menu according to at least one embodiment of the present disclosure.

[0023] FIG. 6 is a drawing for visually explaining a plurality of cooking targets stored in a user menu and previous cooking history according to at least one embodiment of the present disclosure.

[0024] FIG. 7 is a drawing showing a case where a user registers a new user menu according to at least one embodiment of the present disclosure.

[0025] FIGS. 8 and 9 are drawings illustrating a case in which a user menu is updated when a similar cooking target is introduced in a cooking device according to at least one embodiment of the present disclosure.

[0026] FIGS. 10 to 12 are drawings illustrating a case in which a user menu is updated when a dissimilar cooking target is introduced in a cooking device according to at least one embodiment of the present disclosure.

[0027] FIG. 13 is a flowchart illustrating a control method for a cooking device according to at least one embodiment of the present disclosure.

[0028] FIG. 14 is a flowchart for explaining in detail a method for recommending user menu registration in a cooking device according to at least one embodiment of the present disclosure.

[0029] FIG. 15 is a flowchart illustrating a method in which a cooking device according to at least one embodiment of the present disclosure recognizes a cooking target and cooks it.

[0030] FIG. 16 is a flowchart illustrating the operation when a cooking object having dissimilar feature points is introduced into a cooking device according to at least one embodiment of the present disclosure.

[0031] FIG. 17 is a drawing for explaining the operation of a cooking device and a server according to at least one embodiment of the present disclosure.

[0032] FIG. 18 is a drawing for explaining the operation of a cooking device, a server, and a terminal device according to at least one embodiment of the present disclosure.

[0033] The terms used in the embodiments of this disclosure have been selected to be as widely used and general as possible, taking into account their functions within this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant explanatory section of this disclosure. Therefore, terms used in this disclosure should be defined not merely by their names, but based on their meanings and the overall content of this disclosure.

[0034] In this specification, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of such features (e.g., numerical values, functions, operations, or components such as parts) and do not exclude the presence of additional features.

[0035] The expression "at least one of A or / and B" should be understood as representing either "A" or "B" or "A and B".

[0036] Expressions such as "first," "second," "first," or "second" used in this specification may modify various components regardless of order and / or importance, and are used only to distinguish one component from another and do not limit said components.

[0037] Where it is stated that a component (e.g., Component 1) is "(operatively or communicatively) coupled with / to" or "connected to" another component (e.g., Component 2), it should be understood that the component may be directly connected to the other component or connected through the other component (e.g., Component 3).

[0038] The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "consisting of" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0039] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Additionally, a plurality of "modules" or a plurality of "parts" may be integrated into at least one module and implemented by at least one processor (not shown), except for a "module" or "part" that needs to be implemented in specific hardware.

[0040] In this specification, the term "user" may refer to a person using a cooking device or a device using a cooking device (e.g., an artificial intelligence electronic device).

[0041] An embodiment of the present disclosure will be described in more detail below with reference to the attached drawings.

[0042] FIG. 1 is a schematic diagram illustrating a cooking device according to at least one embodiment of the present disclosure. In the present disclosure, the cooking device (100) is an electronic device configured to cook food. The cooking device (100) can perform various cooking operations by applying high-temperature heat generated using electricity or gas to the food being cooked. Examples of cooking devices may include an oven, an induction cooker, an electric range, an electric grill, an air fryer, a gas range, etc. Recently, there are also cases where various cooking methods performed by these various types of cooking devices are implemented to be performed in a single device. The cooking device described in the various embodiments of the present disclosure may be implemented as one of the cooking devices of the various examples described above, including such smart cooking devices. However, it is not limited thereto and may be applied to other types of cooking devices other than the examples described above.

[0043] According to one embodiment, the cooking device (100) may include a cooking chamber (110), a camera (120), and a display (140).

[0044] The cooking device (100) can cook food based on user input related to the operation of the cooking device. The user can place the food to be cooked into the cooking chamber (110) and input a cooking method (e.g., cooking mode, cooking temperature, cooking time). The cooking device (100) can cook food according to the cooking temperature, cooking time, and cooking mode corresponding to the cooking method. In the present disclosure, the food to be cooked refers to various types of cooking materials such as meat, vegetables, fish, and liquids.

[0045] The cooking chamber (110) may be a space capable of performing cooking while accommodating a cooking object. The cooking chamber (110) may include a driving device capable of rotating and moving the cooking object.

[0046] The cooking chamber (110) can be opened and closed by a cover provided in the cooking device (100), and can form a closed space when the cover is closed. However, it is not limited thereto, and depending on the type or form of the cooking device (100), it may be implemented as an open cooking chamber (110) without a cover.

[0047] In the various embodiments of the present disclosure, for convenience of explanation, a single cooking chamber (110) has been illustrated and described, but the cooking chamber (110) may be divided into multiple different areas. In this case, a separate cover may be formed for each cooking chamber, or the structure may be manufactured so that multiple cooking chambers are opened and closed at once by a single cover.

[0048] Cooking modules corresponding to various cooking modes may be arranged within the cooking chamber (110). A cooking mode may refer to a cooking style of food. The cooking device (100) may determine the method of heat transfer or the method of controlling internal airflow according to the cooking mode. Cooking modes may include, for example, convection, bake, variable broil, steam bake, steam roast, convection vegetable, air fry, and air sous vide. Examples of cooking modules corresponding to each cooking mode will be described again in the following section. The cooking device (100) may raise the internal temperature to a cooking temperature using, for example, a heater among the cooking modules mounted on the cooking device, and then maintain it for a preset cooking time. The cooking temperature may be the internal temperature of the cooking chamber (110). The cooking time may refer to the time from when the cooking device (100) starts operating until when it ends.

[0049] The cooking device (100) can count the time elapsed from the time cooking begins. When the counted time reaches a preset cooking time, the cooking device (100) can terminate the operation of the cooking device (100).

[0050] According to one embodiment, the cooking device (100) can photograph a cooking target located in the cooking chamber (110) using a camera (120). The camera (120) may be positioned to photograph the interior of the cooking chamber (110). FIG. 1 shows the camera (120) positioned on the upper side of the interior of the cooking chamber (110). Accordingly, the image obtained through the camera (120) may be a top-view image or an image taken at an angle close to a top-view image. Alternatively, the camera (120) may be positioned on the cover side that can open and close the cooking chamber (110), so that it can photograph the cooking target located inside the cooking chamber (110) from the front when the cover is closed.

[0051] The camera (120) is not required to be used as a single unit, but may be provided in multiple units to photograph the cooking target from various locations and may be distributed and placed in various locations such as the cooking chamber (110) and the cover. If the camera (120) is placed inside the cooking chamber, the camera (120) may be damaged due to the heat inside the cooking chamber. Therefore, a portion of the wall or cover inside the cooking chamber may be made of a transparent material (e.g., glass), and the camera (120) may be placed on the outside of the transparent material. According to various embodiments of the present disclosure, in addition to the camera (120), various sensors may be added, such as a weight sensing sensor to detect the weight of the cooking target and a UV sensor to detect whether the cooking target has been placed inside the cooking chamber (110). The display (140) is configured to display various screens related to the operation of the cooking device (100). The display (140) may be configured as a touch screen that allows the user to input touch operations.

[0052] FIG. 2 is a block diagram illustrating the configuration of a cooking device according to at least one embodiment of the present disclosure.

[0053] Referring to FIG. 2, the cooking device (100) may include a cooking chamber (110), a camera (120), a processor (130), a display (140), a memory (150), and at least one cooking module (160) (hereinafter referred to as the cooking module (160)). However, such a configuration is exemplary, and it is understood that in carrying out the present disclosure, new configurations may be added or some configurations may be omitted in addition to such configurations.

[0054] A processor (130) can control the overall operation of the cooking device (100). Specifically, the processor (130) is connected to each component of the cooking device (100) to control the overall operation of the cooking device (100). The processor (130) may include one or more of a CPU (Central Processing Unit), GPU (Graphics Processing Unit), APU (Accelerated Processing Unit), MIC (Many Integrated Core), DSP (Digital Signal Processor), NPU (Neural Processing Unit), hardware accelerator, or machine learning accelerator. The processor (130) can control one or any combination of other components of the cooking device (100) and can perform operations or data processing related to communication. The processor (130) can execute one or more programs or instructions stored in the memory (150) of the cooking device (100). For example, the processor (130) can perform the method according to various embodiments of the present disclosure by executing one or more instructions stored in memory (150).

[0055] The processor (130) may be implemented as a single-core processor including one core, or as one or more multicore processors including multiple cores (e.g., homogeneous multicore or heterogeneous multicore). When the processor (130) is implemented as a multicore processor, each of the multiple cores included in the multicore processor may include internal processor memory such as cache memory or on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. Additionally, each of the multiple cores included in the multicore processor (or some of the multiple cores) may independently read and execute program instructions for implementing a method according to one embodiment of the present disclosure, or all (or some) of the multiple cores may be linked together to read and execute program instructions for implementing a method according to one embodiment of the present disclosure.

[0056] A processor may refer to a system-on-chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, GPU, APU, MIC, DSP, NPU, hardware accelerator, or machine learning accelerator, but the embodiments of the present disclosure are not limited thereto.

[0057] A camera (120) is provided inside the cooking room (110) and can photograph the inside of the cooking room (110). The camera (120) can photograph food contained inside the cooking room (110). The camera (120) can acquire an image corresponding to the subject by using an image sensor to convert light emitted or reflected and transmitted from the subject into an electrical signal.

[0058] The memory (150) is configured to store data, programs, instructions, etc., necessary for various embodiments. Depending on the purpose of data storage, the memory (150) may be implemented in the form of a memory embedded in the cooking device (100) or in the form of a memory that can be attached to and detached from the cooking device (100). For example, data for operating the cooking device (100) may be stored in the memory embedded in the cooking device (100), and data for the expansion function of the cooking device (100) may be stored in the memory that can be attached to and detached from the cooking device (100). Meanwhile, the memory embedded in the cooking device (100) may be implemented as at least one of volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), etc.), non-volatile memory (e.g., OTPROM (one-time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash, etc.), hard drive, or solid state drive (SSD). Additionally, the memory that is detachable from the cooking device (100) may be implemented in the form of a memory card (e.g., CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.), external memory connectable to a USB port (e.g., USB memory), etc. there is.

[0059] In FIG. 2, the memory (120) is shown as being separate from the processor (130), but at least one memory may be implemented as an internal memory of the processor (130).

[0060] The cooking module (160) may be configured to cook a cooking target located inside the cooking chamber (110). Specifically, the cooking module (160) may include an upper heating element, a lower heating element, a convection heating element, a grill heating element, etc. Additionally, the cooking module (160) may include a steam generating module that provides steam into the cooking chamber (110) to perform steaming functions, etc. Furthermore, the cooking module (160) may include a rotisserie module capable of cooking while rotating the cooking target 360 degrees. Alternatively, the cooking module (160) may further include a microwave oven module for providing microwaves. The location and number of the cooking modules (160) may be determined in various ways. Multiple cooking modules (160) may be distributed at various locations within the cooking chamber.

[0061] After a cooking target is introduced into the cooking chamber (110), the processor (130) performs cooking by driving at least one of a plurality of cooking modules (160) based on the cooking method when a cooking method is selected through a display (140) or other interface (e.g., a button). If a cooking method for a cooking target is pre-registered in memory (150), the processor (130) can control the display (140) to display the registered cooking method. Such cooking methods may be stored in memory (150) in advance by the manufacturer of the cooking device (100) before product launch, or may be newly added to or changed in memory (150) during an occasional or periodic update process. In the present disclosure, such cooking methods are referred to as standard cooking methods, and a menu matching a cooking target with a standard cooking method is referred to as a basic menu. The standard cooking method may include information regarding the cooking temperature, cooking time, and the type of cooking module to be driven during cooking for each cooking target. The cooking temperature does not need to be set to remain constant throughout the cooking time, but may be set to change at regular intervals. For example, a series of cooking sequences such as operating at 180 degrees for 30 minutes in air fryer mode, followed by operating at 150 degrees for 10 minutes in grill mode, may be registered as a cooking method.

[0062] In addition to these standard cooking methods, the processor (130) may also store cooking methods entered by the user in memory (150). In this disclosure, such cooking methods are referred to as user menus. That is, for the same cooking target, the user may enter their own cooking method without using the standard cooking method and register it. Cooking methods included in user menus may also include various information such as the type of cooking module to be used, cooking time, cooking temperature, and cooking order, just like standard cooking methods.

[0063] The processor (130) can save a new user menu to memory (150) or update the user menu stored in memory (150) according to various events. This will be explained in detail below.

[0064] FIG. 3 is a diagram illustrating the operation of a cooking device according to at least one embodiment of the present disclosure. Specifically, FIG. 3 illustrates the operation of a cooking device that recommends registering a cooking method entered by a user as a user menu. A user menu may refer to a menu that includes cooking targets frequently cooked by the user. If a user has cooked multiple identical cooking targets by placing them into the cooking device (100), the cooking method that is repeatedly entered for the cooking target can be stored in the cooking device (100) as a user menu. In this case, when the user puts the same cooking target in again the next time, the user can cook by immediately selecting the user menu they have registered without having to select the cooking time, cooking order, cooking mode, etc., one by one again.

[0065] The cooking device (100) can determine whether the cooking target introduced into the cooking chamber (110) corresponds to a user menu, and if it corresponds to a user menu, it can perform cooking according to the cooking method included in the user menu.

[0066] When a new cooking target is introduced into the cooking room (110), the processor (130) can control the camera (120) to photograph the new cooking target. The new cooking target does not refer to a cooking target that has never been cooked before, but may refer to cooking ingredients introduced into the cooking room (110) to perform cooking anew. That is, the new cooking target may represent a cooking target introduced into the cooking room (110). Hereinafter, the new cooking target may also be referred to as a cooking target. The processor (130) can identify what the cooking target is from the image of the new cooking target captured through the camera (120). At this time, the processor (130) may use an artificial intelligence model trained to determine what the cooking target is.

[0067] Specifically, the memory (150) may store a first artificial intelligence model trained to analyze an image and determine what the cooking target included in the image is. The first artificial intelligence model may refer to a basic menu recognition model (Food Recognition Model).

[0068] The basic menu may include a plurality of general basic cooking targets that can be cooked in the cooking device (100). For example, the cooking targets of the basic menu may include pasta, pizza, steak, etc. Information regarding the plurality of basic cooking targets may be stored in the memory (150). Specifically, characteristic points and standard cooking methods for the basic cooking targets may be stored.

[0069] The processor (130) can input an image of a cooking target captured through the camera (120) into the first artificial intelligence model.

[0070] Although only the camera (120) is shown in FIG. 1, the cooking device (100) may further include a configuration capable of obtaining information about the cooking target. For example, it may include a scale capable of measuring the weight of the cooking target to obtain information about the capacity from the weight of the cooking target. Additionally, it may include an infrared thermometer capable of measuring the temperature of the material and a probe core thermometer capable of measuring the core temperature of the material to obtain temperature information about the cooking target. If such configurations are additionally provided, the processor (130) may input various information, such as weight information and temperature information about the cooking target, in addition to the image, into the first artificial intelligence model.

[0071] The first artificial intelligence model can extract feature points from images or information regarding an input cooking target. For example, it can extract feature points such as boundaries, patterns, shapes, colors, and volumes of the cooking target. The first artificial intelligence model can output embedding vectors corresponding to the feature points extracted from the cooking target. The embedding vector may also be referred to as a feature vector. The embedding vector can be displayed at various locations in a three-dimensional spatial coordinate system. The embedding vectors for each of a plurality of similar cooking targets can be grouped within a certain distance range in the three-dimensional spatial coordinate system.

[0072] The processor (130) can identify whether there is similarity between feature points of each cooking target based on the distance between feature vectors. The processor (130) can identify a cooking target as similar to a cooking target of the basic menu if it is identified as a feature vector within a certain distance range when compared with feature vectors corresponding to a cooking target of the basic menu. The processor (130) can identify multiple cooking targets as similar if the embedding vectors for each of the multiple cooking targets are displayed in a dense cluster within a close distance in a three-dimensional spatial coordinate system. That is, the processor (130) can calculate the density of each cooking target and determine that the higher the density, the higher the similarity.

[0073] Meanwhile, the processor (130) can identify whether the cooking targets are similar by considering not only the feature points of the cooking targets but also the cooking method for the cooking targets. Specifically, if the cooking methods of the cooking targets are identical or similar, the processor (130) can add the similarity according to the cooking method (e.g., various cooking setting values ​​such as cooking mode, temperature, time, etc.) to the similarity calculated through the feature vectors corresponding to the cooking targets.

[0074] For example, the processor (130) may add 10% similarity when the cooking targets are cooked in the same cooking mode. Additionally, if the difference in cooking temperature for the cooking targets is less than 5°C, the similarity may be added by 10%, and if the difference in cooking temperature is less than 10°C, the similarity may be added by 5%. As described above, the processor (130) may add similarity of various magnitudes depending on the difference in cooking time for the cooking targets. In this case, even if the cooking targets are judged to be different menus based only on their feature points, the similarity of the cooking method is compared together to determine that they are the same menu, thereby improving the ability to identify the cooking targets. Consequently, the first artificial intelligence model can output information about what the cooking target is that has been introduced into the cooking chamber (110). The processor (130) can control the display (140) to display information about the output cooking target.

[0075] The memory (150) may store a second artificial intelligence model trained to determine the cooking target of a user menu that is most similar to the cooking target introduced into the cooking chamber (110). The second artificial intelligence model may refer to a Custom Food Recognition Model.

[0076] The processor (130) can input images of the cooking target captured through the camera (120) and a user menu database into the second artificial intelligence model. In addition, information about the cooking target, such as the capacity and temperature of the cooking target, can be input into the second artificial intelligence model.

[0077] The user menu can be organized into at least one database and stored in memory (150). Each user menu may include information and feature points regarding the name, image, cooking method, and cooking target registered by the user.

[0078] The second artificial intelligence model can identify the cooking target of the user menu that is most similar to the cooking target introduced into the cooking room (110) by comparing the feature points of the cooking target introduced into the cooking room (110) with the feature points of the cooking target included in the user menu database. At this time, the second artificial intelligence model can identify the cooking target of the user menu that is most similar to the cooking target introduced into the cooking room (110) by comparing the cooking methods of the cooking targets together.

[0079] Meanwhile, the second artificial intelligence model can identify cooking targets by prioritizing comparisons of main menus, taking into account cooking frequency and recent registration status. Main menus may be cooking targets with high cooking frequency, as they are frequently cooked by the user, or cooking targets recently registered in the user menu. Since main menus are highly likely to be cooked by the user, the time required to identify cooking targets can be reduced by prioritizing comparisons of new cooking targets added by the user with those in the main menus.

[0080] As a result, the second artificial intelligence model can output a cooking target of a user menu corresponding to a cooking target introduced into the cooking room (110). The second artificial intelligence model can output one or more cooking targets of a user menu based on the similarity of feature vectors. The processor (130) can control the display (140) to display information about the output cooking target.

[0081] The first artificial intelligence model and the second artificial intelligence model described above may each be stored in memory (150) in a learned state and may be stored in memory (150) as a combined artificial intelligence model.

[0082] Meanwhile, the processor (130) may fail to identify a new cooking target introduced into the cooking chamber (110). Specifically, if the user introduces a completely new cooking target that is not included in the basic menu or user menu, the feature points detected by the first and second artificial intelligence models may not reach a confidence range. In this case, the processor (130) may control the display (140) to display a message indicating that the basic menu or user menu cannot be found.

[0083] If the identification of a new cooking target by the first and second artificial intelligence models fails, the user must directly input the cooking method. The user can directly input information regarding the cooking mode, cooking temperature, and cooking time through the display (140) or other interfaces. Specifically, the user can input information regarding the cooking method through the touch screen of the display (140), or through physical input devices such as dials or buttons. Additionally, information regarding the cooking method can be input through voice recognition or motion recognition methods. Alternatively, if the cooking device (100) is linked with a terminal device such as a mobile phone or tablet PC used by the user, the user may input the cooking method using an interface (touch screen, buttons, etc.) provided on the terminal device. This will be explained in more detail in the following section.

[0084] When a cooking target that has been cooked by the user more than a preset number of times is identified, the processor (130) may inquire whether the user wants to create a user menu for that cooking target. In the present disclosure, this operation may be described as "recommendation."

[0085] Referring to FIG. 3, the processor (130) can control the display (140) to display information about a cooking method received from a user. Specifically, information about a cooking mode (310), a cooking temperature (320), and a cooking time (330) may be displayed. Although FIG. 3 only shows information about the cooking mode (310), the cooking temperature (320), and the cooking time (330), it is not necessarily limited thereto, and information about various cooking methods, such as the core temperature of the ingredients, may be displayed on the display (140). The arrangement of cooking methods is also not limited to FIG. 3.

[0086] Referring to FIG. 3, when a user inputs information about a cooking method and starts cooking, the processor (130) can control the display (140) to display a UI (340) indicating that cooking is in progress. Referring to FIG. 3, the UI (340) indicating that cooking is in progress shows a cooking mode, cooking time, cooking temperature, cooking stop ('OFF'), and cooking information save ('Save Recipe'), but is not necessarily limited to this, and the UI configuration and arrangement may be changed.

[0087] The processor (130) can control the display (140) to display a UI menu recommending that a cooking method according to a previous cooking history be registered as a user menu for the cooking target if a feature point extracted from a captured image of the cooking target corresponds to a cooking target included in a previous cooking history. Meanwhile, in the present disclosure, the expression “UI menu” may be otherwise referred to as UI.

[0088] At this time, the fact that a feature point extracted from the captured image corresponds to a cooking target included in the previous cooking history may indicate that the similarity between the feature point extracted from the captured image and the feature point of the cooking target included in the previous cooking history is greater than or equal to a preset value (e.g., 90%). At this time, the preset value may be referred to as a first reference value.

[0089] That is, if the similarity between the feature points extracted from the video of the cooking target and the feature points of the cooking target included in the previous cooking history is greater than or equal to the first threshold value, a UI recommending that the cooking method according to the previous cooking history be registered as a user menu for the cooking target can be displayed.

[0090] Conversely, the fact that feature points extracted from the captured image do not match the cooking target included in the previous cooking history may indicate that the similarity between the feature points extracted from the captured image and the feature points of the cooking target included in the previous cooking history is less than a preset value.

[0091] The previous cooking history may include information about cooking targets that were previously introduced into the cooking room (110). Specifically, it may include characteristic points of cooking targets that were previously introduced into the cooking room (110) but are not registered in the user menu, and cooking methods for the cooking targets. The previous cooking history may be stored in memory (150).

[0092] The previous cooking history may contain a number of information about cooking targets having similar characteristic points. For example, a user may cook the same cooking target multiple times in the cooking device (100). Although the detailed appearance of the cooking target introduced into the cooking device (100) may differ each time the user cooks, the characteristic points extracted from the cooking target may be similar to each other. In this case, if the user has not registered the cooking target in the user menu, the processor (130) may store information about the cooking target in the memory (150) by including it in the previous cooking history each time cooking is performed.

[0093] Meanwhile, if a cooking target having similar characteristics to the previous cooking history is included more than a preset number of times, the processor (130) can control the display (140) to display a UI menu recommending that a cooking method according to the previous cooking history be registered as a user menu for the cooking target.

[0094] According to one embodiment of the present disclosure, when a user cooks a cooking target that is identical to a previously cooked cooking target but is not separately registered in the user menu five or more times, the processor (130) may display a UI menu recommending that the cooking target be registered in the user menu. For example, if the number of times a user has put 'frozen pizza' into the cooking device (100) and cooked it is four times and the user has not registered 'frozen pizza' in the user menu, the processor (130) may store previous cooking history information, including feature points and cooking method information for 'frozen pizza', in the memory (150) for each of the four previous cookings. Subsequently, when the user cooks 'frozen pizza' again, since the feature points extracted from the put-in 'frozen pizza' match the feature points of the 'frozen pizza' stored in the previous cooking history and similar feature points have been cooked five or more times, the processor (130) may control the display (140) to display a UI menu recommending that the 'frozen pizza' be registered in the user menu. Referring to FIG. 3, the processor (130) can control the display (140) to display a UI menu (350) recommending registration to the user menu in a popup form on top of the UI (340) showing the cooking screen. However, the processor (130) can control the display (140) to change the UI (340) showing that cooking is in progress to the UI menu (350) recommending registration to the user menu.

[0095] In addition, the configuration of the UI menu (350) shown in Fig. 3 is not limited to that of the UI menu shown in Fig. 3, and other UI menus containing phrases recommending registration as a user menu may also be displayed.

[0096] FIG. 4 is a drawing for explaining the process of registering a cooking target to a user menu in a cooking device according to at least one embodiment of the present disclosure.

[0097] When a UI menu (350) is selected, the processor (130) can store a user menu containing characteristic points of a cooking target and a cooking method in memory (150).

[0098] Referring to FIG. 4, when a user wants to register a cooking target in the user menu, the user can touch the 'Save' menu (410) within the UI. When the processor (130) receives a signal corresponding to the user's touch, it can control the display (140) to display a UI menu (420) where the name of the cooking target can be entered.

[0099] FIG. 4 illustrates a keyboard shape in a UI menu (420) where the name of the cooking target can be entered, but it is not necessarily limited to this, and an image representing a recording button may also be displayed to receive voice input from the user. Alternatively, the name may be received using a separately provided button.

[0100] When the processor (130) receives input of the name of the cooking target from the user through the UI menu (420), it can store the feature points and cooking information along with the name of the cooking target received from the user in the memory (150) as a user menu.

[0101] Referring to FIG. 4, the user menu database stores information (430) including an image captured by a camera (120), a feature vector, a cooking mode, a cooking time, a cooking temperature, and whether it is preheated, along with the name 'Lemon Rice'. However, it is not limited to this, and various information about the cooking target can be stored.

[0102] FIG. 5 is a drawing for explaining a case in which a cooking target is not registered in a user menu in a cooking device according to at least one embodiment of the present disclosure.

[0103] If the processor (130) does not select a menu recommended for registration as a user menu, it can add the characteristic points and cooking method for the cooking target to the previous cooking history and store them in memory (150).

[0104] Referring to FIG. 5, if the user does not register a cooking target in the user menu, the user can touch the 'Cancel' portion (510) of the touchpad. When the processor (130) receives a signal corresponding to the user's touch, it can store information about the cooking target in memory (150) by including it in the previous cooking history. Additionally, the processor (130) can control the display (140) to display a UI (340) indicating that cooking is in progress.

[0105] FIG. 6 is a drawing for visually explaining a plurality of cooking targets stored in a user menu and previous cooking history according to at least one embodiment of the present disclosure.

[0106] Each point (620) on the graph can be a feature point of a cooking target. Points (620) are positioned on the graph according to feature vectors extracted from cooking targets. As cooking targets become more similar to each other, the distance between feature vectors becomes closer, forming clusters. Clusters labeled 'Pasta' (630) or 'White Pasta' (640) on the graph represent clusters of cooking targets registered as 'Pasta' and 'White Pasta' in the user menu, respectively. Additionally, clusters labeled 'Unknown_01' (650) represent cooking targets that are not registered in the user menu and are saved as previous cooking history. If a user registers a cooking target with a feature vector close to the 'Unknown_01' cluster (650) in the user menu, the information of the 'Unknown_01' cluster can be registered in the user menu with the name entered by the user.

[0107] The processor (130) can recommend to the user that the 'Unknown_01' cluster (650) be registered in the user menu. Specifically, if the feature vectors of the 'Unknown_01' cluster (650) are dense, the processor (130) can recommend to the user that the user be registered in the user menu. The higher the density of multiple feature vectors, the more frequently the user has cooked a cooking target with high similarity, so the cooking target can be registered in the user menu.

[0108] FIG. 7 is a drawing showing a case where a user registers a new user menu in a cooking device according to at least one embodiment of the present disclosure.

[0109] Feature points extracted from a new cooking target introduced by the user may not match cooking targets included in the previous cooking history. That is, the similarity between feature points extracted from an image capturing the cooking target and feature points of a cooking target included in the previous cooking history may be less than a preset value (or a first reference value). Alternatively, feature points extracted from the new cooking target may match cooking targets included in the previous cooking history, but the number of times the cooking target has been cooked may be less than a preset number.

[0110] For example, if a user cooks 'frozen pizza' 4 times, the feature points for the 'frozen pizza' introduced each time are stored in the previous cooking history and match the feature points of the newly introduced 'frozen pizza', but the number of times 'frozen pizza' has been cooked may be less than 5 times.

[0111] Alternatively, information regarding the cooking target may not be stored in the user menu database. Specifically, information regarding the cooking target may not be stored because the user has not registered the cooking target in the user menu.

[0112] In such cases, the recommended action described in the above-described embodiment is impossible. In this embodiment, even in such situations, the user can create a cooking method they have selected as a user menu.

[0113] The processor (130) can control the display (140) to display a UI menu that can register a cooking method input for a cooking target as a user menu if the feature point extracted from the captured image does not match a cooking target included in the previous cooking history.

[0114] Specifically, after the cooking target is input and a cooking method is entered and a cooking start command is entered, the processor (130) can control the display (140) to display a UI menu (340) indicating that cooking is in progress. The UI menu (340) may additionally provide a menu for determining whether to register information about the cooking method entered by the user as a user menu. Referring to FIG. 7, the UI (340) indicating that cooking is in progress may include a UI menu (710) corresponding to the registration of a user menu called 'Save Recipe'.

[0115] When the processor (130) receives a touch signal from the user for a UI menu (710) corresponding to the registration of a user menu, it can control the display (140) to display a UI menu (720) corresponding to the registration of a new user menu. Referring to FIG. 7, the UI menu (720) corresponding to the registration of a new user menu is shown as 'New Recipe', but it is not necessarily limited to this and may be displayed with other text.

[0116] When the processor (130) receives a touch signal from the user for a UI menu (720) corresponding to the new registration of a user menu, it can control the display (140) to display a UI menu (420) for which the name of the cooking target can be entered.

[0117] When a registration menu is selected, the processor (130) can store the feature points extracted from the cooking target and the input cooking method in the memory (150) as a user menu.

[0118] When the processor (130) receives input of the name of the cooking target from the user through the UI menu (420), it can store the feature points and cooking information along with the name of the cooking target received from the user as a user menu in memory (150). The method of storing information of the cooking target in the user menu database has been described above, so it is omitted.

[0119] FIG. 8 is a drawing showing a case where a user menu is updated when a cooking target similar to a cooking target registered in the user menu according to at least one embodiment of the present disclosure is introduced.

[0120] In one or more embodiments, the processor (130) may update the previously registered user menu based on the new cooking method, based on the identification of a user menu similar to the cooking target among the previously registered user menus and the input of a new cooking method different from the cooking method registered as the identified user menu.

[0121] When a new cooking target is introduced into the cooking chamber (110), the processor (130) can control the display (140) to recognize the cooking target and display the result.

[0122] The processor (130) can control the display (140) to display identification information for at least one user menu corresponding to the new cooking target and identification information for at least one basic menu corresponding to the new cooking target on one screen based on feature points extracted from a captured image of the new cooking target.

[0123] Specifically, the processor (130) can identify at least one cooking target of a basic menu and at least one cooking target of a user menu that are similar to the new cooking target from the first and second artificial intelligence models. The processor (130) can control the display (140) to display identification information of the identified multiple cooking targets on a single screen.

[0124] The identification information of the cooking target may include an image, icon, name, or a combination thereof of the cooking target that can identify the user cooking target. Referring to FIG. 8, a display (140) is shown in which identification information (810) for the cooking target of the user menu and identification information (820) for the cooking target of the basic menu are displayed on one screen.

[0125] Referring to FIG. 8, one cooking target from the user menu and one cooking target from the basic menu are displayed, but this is not necessarily limited to this. For example, the user may set it so that only one of the cooking targets from the user menu or the cooking targets from the basic menu is displayed. Additionally, one or more cooking targets from the user menu may be displayed on the display (140). In this case, the processor (130) may control the display (140) to display the next cooking target from the user menu when the user presses a scroll button or touches the screen and drags in one direction while one cooking target from the user menu and one cooking target from the basic menu are displayed on one screen. That is, information regarding multiple cooking targets may be divided and displayed on multiple pages. The number of cooking targets displayed within one screen may vary depending on the size of the display (140), the screen layout, etc.

[0126] In FIG. 8, the processor (130) can control the display (140) to display two cooking targets on one screen.

[0127] When one of the identification information displayed on the display (140) is selected, the processor (130) can control at least one cooking module (160) to cook a new cooking target using a cooking method corresponding to the selected identification information.

[0128] Referring to FIG. 8, when a user touches the display (140) screen corresponding to the identification information (810) of the user menu, the processor (130) can control the display (140) to display the cooking method of the cooking target stored in the user menu. When the processor (130) receives a signal corresponding to the start of cooking from the user, the processor (130) can control the cooking module (160) to cook the cooking target using the cooking method stored in the user menu.

[0129] When a user touches the display (140) screen corresponding to the identification information (820) of the basic menu, the processor (130) can control the display (140) to display the cooking method of the cooking target stored in the basic menu. Additionally, when the processor (130) receives a signal corresponding to the start of cooking from the user, the processor (130) can control the cooking module (160) to cook the cooking target using the cooking method stored in the basic menu.

[0130] The user may arbitrarily change the cooking method while the user menu is selected. For example, referring to FIG. 8, when the user changes the cooking time (830), the processor (130) controls the display (140) to display the changed cooking time, and when it receives a signal from the user to start cooking, it can control the cooking module (160) to cook the target using the changed cooking method. The processor (130) can control the display (140) to display the changed cooking time (840) on the UI menu (340) indicating that cooking is in progress. The change in cooking method may be applied only to the user menu, and the standard cooking method included in the basic menu may be set so that it is not possible to change it, but it is not necessarily limited to this.

[0131] FIG. 9 is a drawing showing that a user menu is updated according to a novel cooking method according to at least one embodiment of the present disclosure.

[0132] The processor (130) can update the user menu according to the new cooking method when a cooking target having similar characteristics to a cooking target registered in the user menu is introduced into the cooking chamber (110) and a new cooking method is input for the cooking target. When a new cooking target having similar characteristics to 'Pasta' registered in the user menu is introduced into the cooking chamber (110), the processor (130) can display identification information for 'Pasta' on the display (140). The processor (130) receives touch input for 'Pasta' from the user and can display the cooking method for 'Pasta' (cooking mode: Convection, cooking time: 30 minutes, cooking temperature: 160 degrees) on the display (140). At this time, if the user changes the cooking time to 13 minutes, the processor (130) can display the changed cooking time on the display (140) and control the cooking module (160) to cook according to the changed cooking method. Additionally, the processor (130) can update the changed cooking method in the user menu. The processor (130) can update the 'Pasta' in the user menu by additionally storing an image of a new cooking target, feature points extracted from the image, and the changed cooking method (910).

[0133] FIG. 10 is a drawing showing a display (140) that displays a list of cooking targets registered in the user menu when a new cooking target that is dissimilar to the cooking target of the user menu according to at least one embodiment of the present disclosure is introduced.

[0134] In one or more embodiments, the processor (130) displays a list of registered user menus based on the fact that no user menu similar to the cooking target is identified among the registered user menus and a cooking method is entered for the cooking target, and when one user menu among the list is selected, the selected user menu can be updated based on the entered cooking method.

[0135] The processor (130) can control the display (140) to display a list of cooking targets registered in the user menu when a cooking target having a characteristic point similar to that of a cooking target registered in the user menu is introduced into the cooking room (110) and a new cooking method is input for the cooking target.

[0136] Specifically, the processor (130) may fail to identify a new cooking target introduced into the cooking room (110) when the new cooking target having characteristic points dissimilar to the cooking target of the user menu is introduced into the cooking room (110). For example, the first artificial intelligence model may not be able to output a result value regarding what the new cooking target is, and the second artificial intelligence model may not be able to output a result value because the new cooking target is not a cooking target included in the user menu. Additionally, the first and second artificial intelligence models may output result values ​​different from the cooking target introduced by the user.

[0137] At this time, if the user directly inputs a new cooking method, the processor (130) can control the cooking module (160) to cook the target according to the cooking method received from the user, and control the display (140) to display a UI menu (340) indicating that cooking is in progress.

[0138] If a user wishes to register a new cooking target to the user menu and there is a cooking target to be registered to the user menu, the cooking method for the new cooking target can be saved to the cooking target registered in the user menu.

[0139] At this time, the processor (130) can control the display (140) to display a list (1020) of cooking targets registered as a user menu.

[0140] Referring to FIG. 10, when the processor (130) receives an input signal (1010) corresponding to the registration of a user menu from the user, it can control the display (140) to display a cooking target (1020) registered in the user menu. The cooking target of the user menu displayed on the display (140) may be a cooking target of the user menu having the feature points most similar to the new cooking target. Alternatively, a list of cooking targets registered in the user menu may be displayed on the display (140). The list of cooking targets registered in the user menu may be displayed in the order in which they are registered in the user menu, or in order of similarity of feature points with the new cooking target.

[0141] Meanwhile, the processor (130) can control the display (140) to display a list (1020) of cooking targets registered as user menus and a UI menu (1030) for new registration on a single screen.

[0142] If there is no cooking target in the user menu that the user wishes to register, the user may register a new cooking target in the user menu. FIG. 10 is illustrated as displaying a cooking target (1020) registered in the user menu and a UI menu (1030) for new registration on a single screen, but is not necessarily limited thereto. A list of cooking targets registered in the user menu may be displayed in a scrollable form, and a UI menu for new registration may be displayed at the end of the scroll.

[0143] FIG. 11 is a diagram showing that in a cooking device according to at least one embodiment of the present disclosure, one cooking target is selected from a list of cooking targets in a user menu and the user menu is updated.

[0144] When one of the cooking targets in the list of cooking targets registered in the user menu is selected, the processor (130) can update the user menu registered for the selected cooking target based on a new cooking method.

[0145] Specifically, when the processor (130) receives an input signal (1110) corresponding to the registration of a cooking target in the user menu from the user, the processor (130) can add information about the new cooking target to the user menu. For example, referring to FIG. 11, the processor (130) can add and store the name, captured image, feature points, cooking mode, cooking time, cooking temperature, and whether preheating is required for the new cooking target 'Pasta' in the database of the user menu.

[0146] FIG. 12 is a diagram showing that a UI menu for new registration according to at least one embodiment of the present disclosure is selected, and a new cooking target is updated in the user menu.

[0147] When a UI menu for new registration of a user menu is selected, the processor (130) can store the name of the cooking target and the new cooking method received from the user in the memory (150) as a user menu.

[0148] Specifically, when the processor (130) receives an input signal (1210) corresponding to the new registration of a user menu from the user, the processor receives the name of the new cooking target from the user and can store the information of the received new cooking target in the user menu. Referring to FIG. 12, the processor (130) can store the name of the new cooking target ('White Pasta'), a captured image, feature points, a cooking mode, a cooking time, a cooking temperature, and whether preheating is required in the database of the user menu.

[0149] FIG. 13 is a flowchart illustrating a control method of a cooking device (100) according to at least one embodiment of the present disclosure.

[0150] Referring to FIG. 13, the cooking device (100) can photograph the cooking target that is introduced (S1310).

[0151] The cooking device (100) may display a UI menu recommending that a cooking method according to a previous cooking history be registered as a user menu if a feature point extracted from a captured image corresponds to a cooking target included in a previous cooking history stored in the cooking device (100) (S1320). The previous cooking history may include information about cooking targets that were previously introduced into the cooking device (100). Specifically, it may include feature points and cooking methods of cooking targets that were previously introduced into the cooking device (100) but were not registered in the user menu. The cooking device (100) may extract feature points from a captured image of a cooking target using at least one artificial intelligence model, and may display a UI menu recommending that a cooking target be registered as a user menu if the extracted feature point corresponds to a cooking target included in a previous cooking history stored in the cooking device (100).

[0152] When a UI menu is selected, the cooking device (100) can store a user menu including feature points of a cooking target and a cooking method (S1330). The cooking device (100) can receive a name for a cooking target from the user and store the name, captured image, feature points, cooking mode, cooking temperature, cooking time, preheating status, etc., for the cooking target.

[0153] FIG. 14 is a flowchart for explaining in detail a method of recommending registration as a user menu in a cooking device according to at least one embodiment of the present disclosure.

[0154] Referring to FIG. 14, the cooking device (100) can photograph the cooking target that is introduced (S1410).

[0155] Additionally, if a feature point extracted from a captured video corresponds to a cooking target that has been cooked more than a preset number of times in the previous cooking history (S1420), the cooking device (100) may display a UI menu (or UI) recommending that a cooking method according to the previous cooking history be registered as a user menu (S1430). For example, if a feature point for a similar cooking target that was previously introduced into the cooking device (100) is stored in the previous cooking history five or more times, and a feature point extracted from a newly introduced cooking target is similar to a feature point of a cooking target stored in the previous cooking history, the cooking device (100) may display a UI menu recommending that the cooking target be registered as a user menu.

[0156] On the other hand, if the feature points extracted from the captured video do not match the cooking target that has been cooked more than a preset number of times in the previous cooking history (S1420), the cooking device (100) may display a UI that allows the cooking method input for the cooking target to be registered as a user menu (S1440). If the feature points extracted from the cooking target introduced by the user do not match the cooking target included in the previous cooking history, the cooking device (100) may display a UI menu indicating that cooking is in progress instead of a UI menu recommending that the cooking target be registered as a user menu. At this time, the UI menu indicating that cooking is in progress may include a UI that allows the user to register it as a user menu.

[0157] When a displayed menu is selected by the user, the cooking device (100) can receive input regarding the name of the cooking target from the user (S1450). When the cooking device (100) displays a UI menu recommending that a cooking method based on previous cooking history be registered as a user menu, and the user selects to register it as a user menu, the cooking device (100) can display a UI menu that allows the user to input the name of the cooking target and receive the name of the cooking target from the user. In addition, when the cooking device (100) displays a UI that allows the user to newly register a cooking method entered for the cooking target as a user menu, the user can likewise receive the name of the cooking target from the user through a UI menu that allows the user to input the name.

[0158] The cooking device (100) can store information about a cooking target, including the name of the received cooking target, in a user menu (S1460). Specifically, the cooking device (100) can store the name of the cooking target, a captured image, a cooking mode, a cooking time, a cooking temperature, etc., in a user menu. The information about the cooking target may include the name of the received cooking target, a feature point, and a cooking method.

[0159] The cooking device (100) can store feature points extracted from a cooking target and cooking methods in memory as user menus according to a selected UI. For example, if a UI recommending that a cooking method based on previous cooking history be registered as a user menu for the cooking target is selected, the cooking device (100) can store a user menu containing feature points extracted from the cooking target and cooking methods based on previous cooking history in memory. Alternatively, if a UI allowing a cooking method input for the cooking target to be registered as a user menu is selected, the cooking device (100) can store feature points extracted from the cooking target and input cooking methods in memory.

[0160] FIG. 15 is a flowchart illustrating a method of recognizing a cooking target in a cooking device according to at least one embodiment of the present disclosure and cooking the cooking target using a pre-stored cooking method.

[0161] The cooking device (100) can photograph the newly introduced cooking target (S1510).

[0162] Additionally, the cooking device (100) can recognize a cooking target corresponding to a basic menu and a user menu (S1520). Specifically, the cooking device (100) can recognize at least one basic menu corresponding to a new cooking target based on information about a basic menu that includes feature points set for each stored cooking target and a plurality of standard cooking methods, and feature points extracted from a captured image of a new cooking target. The cooking device (100) can recognize at least one basic menu corresponding to a new cooking target by inputting the information about the basic menu and the feature points of the new cooking target into a first artificial intelligence model.

[0163] Additionally, the cooking device (100) can recognize at least one user menu corresponding to the new cooking target by inputting the feature points of the new cooking target into a second artificial intelligence model. The cooking device (100) can display at least one recognized basic menu and at least one user menu on a single screen (S1530). Specifically, it can display identification information for at least one basic menu and identification information for at least one user menu. At this time, the identification information may include an image, icon, name, or a combination thereof of the cooking target that can identify the cooking target.

[0164] When the cooking device (100) receives input from the user corresponding to the selection of identification information for the basic menu, it can cook a new cooking target using the cooking method of the basic menu (S1540). The cooking device (100) can display information regarding the cooking method of the basic menu. When the cooking device (100) receives a signal from the user corresponding to the start of cooking, it can cook the cooking target using the displayed cooking method. On the other hand, if the cooking device (100) receives input from the user for a changed cooking method, it can cook the new cooking target using the changed cooking method.

[0165] When the cooking device (100) receives input from the user corresponding to the selection of identification information for the user menu, the cooking device (100) can cook a new cooking target using the cooking method of the user menu (S1550). At this time, just as when the user selects the basic menu, the cooking device (100) can cook a new cooking target using the modified cooking method received from the user.

[0166] That is, the cooking device (100) displays identification information for at least one pre-registered user menu corresponding to the cooking target and identification information for at least one basic menu corresponding to the cooking target on one screen based on feature points extracted from a captured image of the cooking target, and can cook the cooking target using a cooking method corresponding to the selected identification information based on the selection of one of the displayed identification information.

[0167] Meanwhile, in one or more embodiments, the cooking device (100) can cook the cooking target using a cooking method registered in a pre-registered user menu based on the fact that the similarity between a feature point extracted from a captured image of the cooking target and a feature point included in a pre-registered user menu is greater than or equal to a preset value (e.g., 90%).

[0168] In this case, the preset value may be referred to as a second reference value. The second reference value may be used to distinguish it from the first reference value used for comparison with the cooking target included in the aforementioned previous cooking history. The first reference value and the second reference value may be different values, or they may be the same value.

[0169] FIG. 16 is a flowchart showing a case where a cooking target having a feature point dissimilar to a cooking target registered as a user menu according to at least one embodiment of the present disclosure is introduced.

[0170] A new cooking target having characteristic points that are dissimilar to the cooking target registered in the user menu may be introduced into the cooking device (100) (S1610). In this case, the cooking device (100) may fail to recognize the cooking target corresponding to the basic menu and the user menu (S1620). For example, the first artificial intelligence model may not be able to output a result value regarding what the new cooking target is, and the second artificial intelligence model may not be able to output a result value because the new cooking target is not a cooking target included in the user menu. Additionally, the first and second artificial intelligence models may output result values ​​different from the cooking target introduced by the user.

[0171] The user may directly input a cooking method and choose whether to save the newly inputted cooking method (S1630). If the user wishes to register a new cooking target in the user menu and there is a cooking target to be registered in the user menu, the cooking method for the new cooking target can be saved as a cooking target registered in the user menu. The cooking device (100) may perform cooking of the new cooking target using the cooking method received from the user and display a UI menu (340) indicating that cooking is in progress. When the cooking device (100) receives an input signal corresponding to registration in the user menu from the user, it may display a list of one or more cooking targets stored in the user menu (S1640). At this time, the cooking device (100) may display a UI menu for new registration along with a list of one or more cooking targets stored in the user menu on a single screen.

[0172] When the user selects one of the cooking targets in the user menu (S1650), the cooking device (100) can update the cooking method for the selected cooking target (S1660). The cooking device (100) can add a new cooking method received from the user as a cooking method for the cooking target in the user menu.

[0173] If the user selects a UI menu for new registration without selecting one of the cooking targets in the user menu (S1650), the cooking device (100) can update the new cooking target in the user menu (S1670). The cooking device (100) can add the name of the new cooking target and the new cooking method received from the user to the user menu.

[0174] That is, in one or more embodiments, the cooking device (100) can update the previously registered user menu based on the new cooking method, based on the identification of a user menu similar to the cooking target among the previously registered user menus and the input of a new cooking method different from the cooking method registered as the identified user menu.

[0175] In one or more embodiments, the cooking device (100) displays a list of pre-registered user menus based on the fact that a user menu similar to the cooking target is not identified among the pre-registered user menus and a cooking method is entered for the cooking target, and when one user menu among the list is selected, the selected user menu can be updated based on the entered cooking method.

[0176] In one or more embodiments, the cooking device (100) displays a list of pre-registered user menus and a UI for registering a new user menu on one screen based on the input of a cooking method for a cooking target, and based on the selection of the UI for registering a new user menu, saves the name of the cooking target received from the user and the input cooking method as a new user menu, and when one user menu from the list is selected, the selected user menu can be updated based on the input cooking method.

[0177] Each of the flowcharts in FIGS. 13 to 16 above may be performed by a cooking device (100) having the configuration of FIG. 2, but is not necessarily limited thereto and may be performed by a cooking device (100) with various configurations added or modified.

[0178] The various embodiments described above may be performed by the cooking device (100) alone, but are not necessarily limited thereto and may be performed together with a server or terminal device.

[0179] FIG. 17 is a drawing for explaining the operation of a cooking device and a server according to at least one embodiment of the present disclosure.

[0180] Referring to FIG. 17, if the cooking device (100) further includes a communication unit, it can communicate with a server (1700). The server (1700) can store an artificial intelligence model that recognizes a new cooking target introduced into the cooking device (100).

[0181] The cooking device (100) can transmit a captured image of a cooking target placed in the cooking chamber to a server (1700). In this case, not only the captured image but also information about the cooking method entered by the user may be transmitted together.

[0182] The server (1700) can recognize what a new cooking target is by inputting it into an artificial intelligence model. Additionally, the server (1700) can store a list of cooking targets and cooking methods for the cooking targets. The server (1700) can transmit the cooking target recognized through the artificial intelligence model and the cooking method for the corresponding cooking target to the cooking device (100). The cooking device (100) can display the transmitted cooking target and perform cooking of the new cooking target according to the cooking method for the cooking target. That is, the user menu described in the various embodiments above can be directly stored in the server (1700). If the user of the cooking device (100) permits it, the server (1700) may share the user menu registered by that user with other users. If another user downloads the cooking method and uses it on their cooking device, the server (1700) may pay a reward to the user who registered the user menu based on the number of times, etc.

[0183] FIG. 18 is a diagram showing a case where a cooking device (100) according to at least one embodiment of the present disclosure operates together with a server and a terminal device.

[0184] Referring to FIG. 18, the server (1700) can receive a captured image of a new cooking target from the cooking device (100) and recognize what the new cooking target is. The server (1700) can transmit information about one or more recognized cooking targets to the terminal device (1800).

[0185] The terminal device (1800) can be an electronic device used by the user. Specifically, the terminal device (1800) can be a PC, laptop PC, mobile phone, tablet PC, kiosk, home network device, etc.

[0186] The user can control the cooking device (100) using their terminal device (1800), and can view a screen identical or similar to the screen provided by the cooking device (100) through their terminal device (1800).

[0187] The terminal device (1800) can display information about one or more cooking targets that have been transmitted. The user can select one of the cooking targets displayed on the terminal device (1800). The terminal device (1800) can display a cooking method for the selected cooking target and receive a modified cooking method from the user. The terminal device (1800) can transmit an existing cooking method or a modified cooking method to the cooking device (100) via the server (1700). The cooking device (100) can perform cooking of the new cooking target according to the transmitted cooking method. Additionally, the cooking device (100) can transmit information about the cooking progress to the terminal device (1800) via the server (1700). For example, information regarding the remaining cooking time and the core temperature of the cooking target can be transmitted. The terminal device (1800) can display the transmitted information. The user can check how much cooking has progressed through the displayed information. The user's terminal device (1800) may be equipped with an input means (e.g., keyboard, soft keyboard, etc.) that is easier to operate than the cooking device (100), so that user menu registration may be more advantageous.

[0188] The various embodiments described above may be implemented individually, but are not necessarily limited thereto, and may be implemented together in combination with at least one other embodiment, either partially or wholly.

[0189] The methods according to the various embodiments of the present disclosure described above can be implemented by software upgrade or hardware upgrade of the existing cooking device (100).

[0190] Meanwhile, according to a specific example of the present disclosure, the control method according to the various embodiments described above may be implemented as software comprising instructions stored on a non-transitory machine-readable storage media that can be read by various machines (e.g., computers), such as a cooking device (100).

[0191] Specifically, a program for performing a control method may be provided in a state stored on a non-transient computer-readable recording medium, comprising: a step of photographing a cooking target when a cooking target is introduced into a cooking room; a step of displaying a UI that recommends registering a cooking method according to a previous cooking history as a user menu for a cooking target based on the fact that the similarity between a feature point extracted from the image of the cooking target and a feature point of the cooking target included in the previous cooking history is greater than or equal to a first reference value; and a step of saving a user menu including the feature point and the cooking method according to the previous cooking history when the UI is selected.

[0192] When stored software or instructions are executed by the processor (130), the processor (130) may perform operations according to the various embodiments described above, either directly or by using other components. Instructions may include code generated or executed by a compiler or an interpreter. Here, 'non-transient' means only that the storage medium does not contain a signal and is tangible, and does not distinguish whether data is stored semi-permanently or temporarily in the storage medium.

[0193] Additionally, according to one or more embodiments of the present disclosure, the method according to the various embodiments described above may be provided as a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed online through an online store as well as the non-transient readable recording medium described above. In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created in a storage medium such as the memory (150) of a manufacturer's server, an application store's server, or a relay server.

[0194] Additionally, each component (e.g., module or program) according to the various embodiments described above may be composed of a single or multiple entities, and some of the aforementioned sub-components may be omitted, or other sub-components may be further included in the various embodiments. Generally or additionally, some components (e.g., module or program) may be integrated into a single entity to perform the functions performed by each of the respective components prior to integration in the same or similar manner. The operations performed by the module, program, or other components according to the various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations added.

[0195] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure.

Claims

1. In a cooking device, cuisine; camera; display; Memory; and Includes a processor; The above memory stores previous cooking history, and The above processor is, When a cooking target is introduced into the above cooking chamber, the camera is controlled to photograph the cooking target, and Control the display to display a UI that recommends registering a cooking method according to the previous cooking history as a user menu for the cooking target, based on the fact that the similarity between a feature point extracted from a video of the cooking target and a feature point of the cooking target included in the previous cooking history is greater than or equal to a first threshold value. A cooking device that, when the above UI is selected, stores the user menu, which includes the above feature point and the cooking method according to the above previous cooking history, in the memory.

2. In Paragraph 1, The above processor is, Based on the fact that the similarity between a feature point extracted from the above-mentioned captured image and a feature point of a cooking target included in the above-mentioned previous cooking history is less than the above-mentioned first reference value, the display is controlled to display a UI that can register a cooking method input for the above-mentioned cooking target to the above-mentioned user menu, and A cooking device that, when a UI capable of registering the above cooking method as a user menu is selected, stores feature points extracted from the above cooking target and the above input cooking method as a user menu in the memory.

3. In Paragraph 1, It further includes at least one cooking module for cooking the cooking target introduced into the cooking chamber; The above processor is, A cooking device that controls at least one cooking module to cook the cooking target using a cooking method registered in the pre-registered user menu, based on the fact that the similarity between a feature point extracted from a captured image of the cooking target and a feature point included in a pre-registered user menu is greater than or equal to a second reference value.

4. In Paragraph 1, It further includes at least one cooking module for cooking the cooking target introduced into the cooking chamber, and The above memory stores information about a basic menu including feature points set for each cooking target and a plurality of standard cooking methods, and The above processor is, Based on feature points extracted from a captured image of the cooking target, the display is controlled to display identification information for at least one pre-registered user menu corresponding to the cooking target and identification information for at least one basic menu corresponding to the cooking target on a single screen. A cooking device that controls at least one cooking module to cook a cooking target using a cooking method corresponding to the selected identification information, based on the selection of one of the identification information displayed on the above display.

5. In Paragraph 1, The above processor is, A cooking device that identifies a user menu similar to the cooking target among previously registered user menus, and updates the previously registered user menu based on the new cooking method, based on the input of a new cooking method different from the cooking method registered in the identified user menu.

6. In Paragraph 1, The above processor is, Based on the fact that among the previously registered user menus, no user menu similar to the cooking target is identified and a cooking method is entered for the cooking target, the display is controlled to display a list of previously registered user menus. A cooking device that updates the selected user menu based on the input cooking method when one of the above-mentioned lists is selected.

7. In Paragraph 1, The above processor is, Based on the input of a cooking method for the above-mentioned cooking target, the display is controlled to display a list of previously registered user menus and a UI for registering new user menus on a single screen, and Based on the selection of the UI for registering the new user menu, the name of the cooking target and the input cooking method received from the user are stored in the memory as the new user menu, and A cooking device that updates the selected user menu based on the input cooking method when one of the above-mentioned lists is selected.

8. In Paragraph 1, The above processor is, A cooking device that updates the feature points for the cooking target and the cooking method entered for the cooking target to the previous cooking history based on the fact that the recommendation of the above UI is not selected.

9. In a method for controlling a cooking device, A step of photographing the cooking target when the cooking target is introduced into the cooking chamber, A step of displaying a UI that recommends registering a cooking method according to the previous cooking history as a user menu for the cooking target, based on the fact that the similarity between a feature point extracted from a video of the cooking target and a feature point of the cooking target included in the previous cooking history is greater than or equal to a first threshold value; and A control method comprising the step of saving the user menu including the feature point and the cooking method according to the previous cooking history when the above UI is selected.

10. In Paragraph 9, A step of displaying a UI that allows a cooking method input for a cooking target to be registered to the user menu based on the fact that the similarity between a feature point extracted from the above-mentioned captured image and a feature point of a cooking target included in the above-mentioned previous cooking history is less than the above-mentioned first reference value; and A control method further comprising the step of saving feature points extracted from the cooking target and the input cooking method to the user menu when a UI capable of registering the above cooking method to the user menu is selected.

11. In Paragraph 9, A control method further comprising the step of cooking the cooking target using a cooking method registered in the previously registered user menu, based on the fact that the similarity between a feature point extracted from a captured image of the cooking target and a feature point included in a previously registered user menu is greater than or equal to a second reference value.

12. In Paragraph 9, A step of displaying identification information for at least one pre-registered user menu corresponding to the cooking target and identification information for at least one basic menu corresponding to the cooking target on a single screen based on feature points extracted from a captured image of the cooking target; and A control method further comprising the step of cooking the cooking target using a cooking method corresponding to the selected identification information based on the selection of one of the identification information displayed above.

13. In Paragraph 9, A control method further comprising the step of identifying a user menu similar to the cooking target among previously registered user menus, and, based on inputting a new cooking method different from the cooking method registered as the identified user menu, updating the previously registered user menu based on the new cooking method.

14. In Paragraph 9, A step of displaying a list of previously registered user menus based on the fact that among the previously registered user menus, no user menu similar to the cooking target is identified and a cooking method is entered for the cooking target; and A control method further comprising the step of updating the selected user menu based on the input cooking method when one of the above-mentioned user menus is selected.

15. A step of displaying a list of previously registered user menus and a UI for registering new user menus on a single screen based on the input of a cooking method for the above-mentioned cooking target; Based on the selection of the UI for registering the new user menu, the step of making the name of the cooking target and the input cooking method received from the user into the new user menu; and A control method further comprising the step of updating the selected user menu based on the input cooking method when one of the above-mentioned user menus is selected.

Images