Electronic device for identifying type of container and method for controlling same

Abstract

An electronic device is disclosed. The electronic device includes: a microwave generating device; a thermal imaging camera; one or more processors; and a memory configured to store commands. When the commands are executed by the one or more processors. the electronic device may control the microwave generating device to output microwaves to a cavity which is an internal space of the electronic device, obtain a first temperature pattern on the basis of a plurality of first thermal images obtained by the thermal imaging camera when the microwaves are output, obtain a second temperature pattern on the basis of a plurality of second thermal images obtained by the thermal imaging camera when the microwaves are not output, obtain temperature pattern information including information on whether the microwaves are output, the first temperature pattern, and the second temperature pattern, identify the type of container by inputting the temperature pattern information to a first neural network model, and output a first use notification for the electronic device on the basis of the type of container.

Description

Electronic device for identifying the type of container and its control method

[0001] The present disclosure relates to an electronic device and a method for controlling the same, and more particularly, to an electronic device for identifying a type of container and a method for controlling the same.

[0002] Advances in electronic technology have led to the development of a diverse range of electronic devices. In particular, these devices can enhance user convenience.

[0003] However, cooking devices may rely on the user's discretion in applying information about unusable containers or food.

[0004] Accordingly, using inappropriate containers may result in fire or hazardous substances being generated inside the cooking device.

[0005] An electronic device capable of identifying the type of container and a method of controlling the same are provided.

[0006] According to one embodiment of the present disclosure for achieving the above object, an electronic device includes a microwave generating device, a thermal imaging camera, one or more processors, and a memory configured to store a command, wherein when the command is executed by the one or more processors, the electronic device controls the microwave generating device to output microwaves into a cavity, which is an internal space of the electronic device, and when the microwaves are output, a first temperature pattern is acquired based on a plurality of first thermal image images acquired through the thermal imaging camera, and when the microwaves are not output, a second temperature pattern is acquired based on a plurality of second thermal image images acquired through the thermal imaging camera, and temperature pattern information including information on whether the microwaves are output, the first temperature pattern, and the second temperature pattern is acquired, and the temperature pattern information is input into a first neural network model to identify a type of a container, and a first usage notification for the electronic device is output based on the type of the container.

[0007] And, the one or more processors execute the command so that the electronic device obtains the temperature pattern information at preset time intervals, inputs the temperature pattern information obtained at the preset time intervals into the first neural network model to obtain a plurality of probability values ​​each corresponding to the types of the plurality of containers, and if the largest probability value among the plurality of probability values ​​is greater than or equal to the preset value, the first type corresponding to the largest probability value can be identified as the type of the container.

[0008] In addition, the one or more processors can execute the command so that the electronic device outputs a notification indicating that the container is dangerous and controls the microwave generating device to stop outputting the microwave if the type of the container is a preset type corresponding to one or more hazardous substances among one or more preset types, and can control the microwave generating device to maintain outputting the microwave if the type of the container is not the preset type.

[0009] And, further comprising a display, wherein the one or more processors can control the display to execute the command so that the electronic device identifies a second type of recommended container corresponding to an operation mode of the electronic device, and if the type of the container is different from the second type, indicates the second type.

[0010] In addition, the device further includes a user interface, wherein the one or more processors execute the command so that when a user command for controlling the electronic device is received through the user interface, the electronic device can control the microwave generating device to output the microwave to the cavity at a preset output power level for a preset time to identify the type of the container before performing an operation corresponding to the user command.

[0011] And, the one or more processors can execute the command so that the electronic device can identify the type of the container for the preset period of time, and if the type of the container is not a preset type corresponding to one or more hazardous substances among the one or more preset types, control the microwave generating device based on an operation mode of the electronic device corresponding to the user command.

[0012] Additionally, the first neural network model can be trained based on first input data including a plurality of first temperature patterns and first output data including a plurality of container types.

[0013] The first input data may further include at least one of a plurality of ambient temperatures, a plurality of operating modes, or a plurality of output power levels.

[0014] The one or more processors may execute the instructions so that the electronic device obtains the temperature pattern information from a second neural network model, and the second neural network model may be trained based on second input data including a plurality of thermal images and second output data including a plurality of second temperature patterns.

[0015] And, further comprising a heater, wherein said one or more processors execute said instructions so that said electronic device can output a second usage notification for said heater when the type of said container is a second type that cannot use said heater.

[0016] Meanwhile, according to one embodiment of the present disclosure, a method for controlling an electronic device includes the steps of controlling a microwave generating device included in the electronic device to output microwaves into a cavity, which is an internal space of the electronic device, when the microwaves are output, obtaining a first temperature pattern based on a plurality of first thermal image images acquired through a thermal imaging camera included in the electronic device, when the microwaves are not output, obtaining a second temperature pattern based on a plurality of second thermal image images acquired through the thermal imaging camera, obtaining temperature pattern information including information on whether the microwaves are output, the first temperature pattern, and the second temperature pattern, inputting the temperature pattern information into a first neural network model to identify a type of a container, and outputting a first usage notification for the electronic device based on the type of the container.

[0017] And, the step of acquiring the temperature pattern information acquires the temperature pattern information at preset time intervals, and the step of identifying the type of the container inputs the temperature pattern information acquired at the preset time intervals into the first neural network model to acquire a plurality of probability values ​​each corresponding to the types of the plurality of containers, and if the largest probability value among the plurality of probability values ​​is greater than or equal to a preset value, the first type corresponding to the largest probability value can be identified as the type of the container.

[0018] In addition, the step of outputting the usage notification may include, if the type of the container is a preset type corresponding to one or more hazardous substances among one or more preset types, outputting a notification indicating that the container is hazardous and controlling the microwave generating device to stop outputting the microwave, and if the type of the container is not a preset type corresponding to the hazardous substance, controlling the microwave generating device to maintain outputting the microwave.

[0019] And, the method may further include a step of identifying a second type of recommended container corresponding to the operation mode of the electronic device, and a step of indicating the second type when the type of the container is different from the second type.

[0020] In addition, the step of outputting the microwave may include, when a user command for controlling the electronic device is received through a user interface, controlling the microwave generating device to output the microwave to the cavity for a preset time at a preset output power level to identify the type of the container before an operation corresponding to the user command.

[0021] And, the step of identifying the type of the container may further include identifying the type of the container for the preset time, and the control method may further include a step of controlling the microwave generating device based on an operation mode of the electronic device corresponding to the user command if the type of the container is not a preset type corresponding to one or more hazardous substances among one or more preset types.

[0022] The first neural network model can be trained based on first input data including a plurality of first temperature patterns and first output data including a plurality of container types.

[0023] The first input data may further include at least one of a plurality of ambient temperatures, a plurality of operating modes, or a plurality of output power levels.

[0024] The above control method may further include a step of outputting a second usage notification for the heater when the type of the container is a second type that cannot use a heater.

[0025] Meanwhile, according to one embodiment of the present disclosure, a non-transitory computer-readable recording medium storing a program for executing an operating method of an electronic device includes a step of controlling a microwave generating device included in the electronic device to output microwaves into a cavity, which is an internal space of the electronic device, a step of obtaining a first temperature pattern based on a plurality of first thermal image images acquired through a thermal imaging camera included in the electronic device when the microwaves are output, a step of obtaining a second temperature pattern based on a plurality of second thermal image images acquired through the thermal imaging camera when the microwaves are not output, a step of obtaining temperature pattern information including information on whether the microwaves are output, the first temperature pattern, and the second temperature pattern, a step of inputting the temperature pattern information into a first neural network model to identify a type of a container, and a step of outputting a first usage notification for the electronic device based on the type of the container.

[0026] The above and other aspects, features and advantages of specific embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings.

[0027] FIG. 1 is a drawing for explaining an electronic system according to one embodiment of the present disclosure.

[0028] FIG. 2 is a block diagram showing the configuration of an electronic device according to an embodiment of the present disclosure.

[0029] FIG. 3 is a block diagram showing a detailed configuration of an electronic device according to an embodiment of the present disclosure.

[0030] FIGS. 4 and 5 are drawings for explaining a temperature pattern according to control of a microwave generating device according to an embodiment of the present disclosure.

[0031] FIGS. 6 and 7 are drawings illustrating a method for analyzing a thermal image according to an embodiment of the present disclosure.

[0032] FIGS. 8 and 9 are diagrams for explaining temperature patterns according to the type of container and output of a neural network model according to the temperature pattern according to one embodiment of the present disclosure.

[0033] FIG. 10 is a drawing for explaining an operation of outputting a notification indicating that a container is a hazardous material according to one embodiment of the present disclosure.

[0034] FIG. 11 is a drawing for explaining an operation for guiding the efficiency of a container according to one embodiment of the present disclosure.

[0035] FIG. 12 is a drawing for explaining an operation of guiding the temperature of a container according to one embodiment of the present disclosure.

[0036] FIG. 13 is a drawing for explaining the operation according to the type of glass according to one embodiment of the present disclosure.

[0037] FIG. 14 is a flowchart for explaining a control method of an electronic device according to an embodiment of the present disclosure.

[0038] The embodiments described in this disclosure and the configurations illustrated in the drawings are merely examples of embodiments, and various modifications may be made without departing from the scope and spirit of the present disclosure.

[0039] The exemplary embodiments of the present disclosure are susceptible to various modifications. Accordingly, exemplary embodiments are illustrated in the drawings and described in detail in the detailed description. However, it should be understood that the present disclosure is not limited to the exemplary embodiments, but encompasses all modifications, equivalents, and alternatives without departing from the technical spirit of the present disclosure.

[0040] The purpose of the present disclosure is to provide an electronic device and a control method thereof that identify the type of a container based on temperature pattern information inside the electronic device and output a notification to a user based on the type of the container.

[0041] It should be understood that the various embodiments and terms used in this document are not intended to limit the technical features described in this document to specific embodiments, but rather to include various modifications, equivalents, or substitutes of the embodiments.

[0042] In connection with the description of the drawings, similar reference numerals may be used for similar or related components.

[0043] The singular form of a noun corresponding to an item may include one or more items, unless the context clearly indicates otherwise.

[0044] In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" may include any one of the items listed together in that phrase, or all possible combinations thereof.

[0045] Terms such as "first," "second," or "first" or "second" may be used to distinguish one component from another, but do not limit the components in any other respect (e.g., importance or order).

[0046] When a component (e.g., a first component) is referred to as being “coupled” or “connected” to another component (e.g., a second component), with or without the terms “functionally” or “communicatively,” it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0047] The terms “include” or “have” are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in this document, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0048] When a component is said to be “connected,” “coupled,” “supported,” or “in contact with” another component, this includes not only cases where the components are directly connected, coupled, supported, or in contact, but also cases where the components are indirectly connected, coupled, supported, or in contact through a third component.

[0049] When we say that a component is “on” another component, this includes not only cases where the component is in contact with the other component, but also cases where there is another component between the two components.

[0050] The term “and / or” includes any combination of a plurality of related described elements or any one of a plurality of related described elements.

[0051] The operating principle and embodiments of the present invention will be described with reference to the attached drawings below.

[0052] FIG. 1 is a drawing for explaining an electronic system according to one embodiment of the present disclosure.

[0053] The home appliance (10) may include a communication module capable of communicating with another home appliance, a user device (2), or a server (3), a user interface for receiving user input or outputting information to a user, at least one processor for controlling the operation of the home appliance (10), and at least one memory in which a program for controlling the operation of the home appliance (10) is stored.

[0054] The home appliance (10) may be at least one of various types of home appliances. For example, the home appliance (10) may include, but is not limited to, at least one of a refrigerator (11), a dishwasher (12), an electric range (13), an electric oven (14), an air conditioner (15), a clothes manager (16), a washing machine (17), a dryer (18), and a microwave oven (19), and may include various types of home appliances such as, for example, a cleaning robot, a vacuum cleaner, and a television. In addition, the home appliances mentioned above are merely examples, and in addition to the home appliances mentioned above, a device that is connected to another home appliance, a user device (2), or a server (3) and can perform the operations described below may be included in the home appliance (10) according to an embodiment.

[0055] The server (3) may include a communication module capable of communicating with another server, a home appliance (10), or a user device (2), at least one processor capable of processing data received from another server, a home appliance (10), or a user device (2), and at least one memory capable of storing a program for processing data or processed data. The server (3) may be implemented as various computing devices such as a workstation, a cloud, a data drive, or a data station. The server (3) may be implemented as one or more servers that are physically or logically separated based on function, detailed configuration of function, or data, and may transmit and receive data through communication between each server and process the transmitted and received data.

[0056] The server (3) can perform functions such as managing user accounts, registering home appliances (10) by linking them to user accounts, and managing or controlling registered home appliances (10). For example, a user can access the server (3) through a user device (2) and create a user account. The user account can be identified by an ID and password set by the user. The server (3) can register home appliances (10) to the user account according to a set procedure. For example, the server (3) can link identification information (e.g., serial number or MAC address) of the home appliance (10) to the user account, thereby registering, managing, and controlling the home appliance (10). The user device (2) can include a communication module capable of communicating with the home appliance (10) or the server (3), a user interface for receiving user input or outputting information to the user, at least one processor for controlling the operation of the user device (2), and at least one memory in which a program for controlling the operation of the user device (2) is stored.

[0057] The user device (2) may be carried by the user or placed in the user's home or office, etc. The user device (2) may include, but is not limited to, a personal computer, a terminal, a portable telephone, a smart phone, a handheld device, a wearable device, etc.

[0058] A program for controlling a home appliance (10), for example, an application, may be stored in the memory of the user device (2). The application may be sold installed in the user device (2) or downloaded and installed from an external server.

[0059] A user can access a server (3) by executing an application installed on a user device (2), create a user account, and communicate with the server (3) based on the logged-in user account to register a home appliance (10).

[0060] For example, when the home appliance (10) is operated so that the home appliance (10) can be connected to the server (3) according to the procedure guided by the application installed on the user device (2), the home appliance (10) can be registered in the user account by registering the identification information (e.g., serial number or MAC address) of the home appliance (10) in the corresponding user account on the server (3).

[0061] A user can control a home appliance (10) using an application installed on the user device (2). For example, when a user logs into a user account using an application installed on the user device (2), a home appliance (10) registered to the user account appears, and when a control command for the home appliance (10) is input, the control command can be transmitted to the home appliance (10) via the server (3).

[0062] A network can include both wired and wireless networks. Wired networks include cable networks or telephone networks, while wireless networks can include any network that transmits and receives signals via radio waves. Wired and wireless networks can be interconnected.

[0063] A network may include a wide area network (WAN) such as the Internet, a local area network (LAN) formed around an access point (AP), and a short-range wireless network that does not use an access point (AP). Short-range wireless networks may include, but are not limited to, Bluetooth™ (IEEE 802.15.1), Zigbee (IEEE 802.15.4), Wi-Fi Direct, Near Field Communication (NFC), Z-Wave, etc.

[0064] An access point (AP) can connect a home appliance (10) or a user device (2) to a wide area network (WAN) to which a server (3) is connected. The home appliance (10) or the user device (2) can be connected to the server (3) via the wide area network (WAN).

[0065] The access point (AP) can communicate with a home appliance (10) or user device (2) using wireless communication such as Wi-Fi (Wi-Fi™, IEEE 802.11), Bluetooth (Bluetooth™, IEEE 802.15.1), or Zigbee (IEEE 802.15.4), and can connect to a wide area network (WAN) using wired communication, but is not limited thereto.

[0066] According to various embodiments, the home appliance (10) may be directly connected to the user device (2) or server (3) without going through an access point (AP).

[0067] The home appliance (10) can be connected to a user device (2) or a server (3) via a long-distance wireless network or a short-distance wireless network.

[0068] For example, the home appliance (10) can be connected to the user device (2) via a short-range wireless network (e.g., Wi-Fi Direct).

[0069] As another example, the home appliance (10) may be connected to a user device (2) or a server (3) via a wide area network (WAN) using a long-distance wireless network (e.g., a cellular communication module).

[0070] As another example, a home appliance (10) can connect to a wide area network (WAN) using wired communication and be connected to a user device (2) or a server (3) through the wide area network (WAN).

[0071] If the home appliance (10) can connect to a wide area network (WAN) using wired communication, it can also function as an access relay. Accordingly, the home appliance (10) can connect other home appliances to the wide area network (WAN) to which the server (3) is connected. In addition, other home appliances can connect the home appliance (10) to the wide area network (WAN) to which the server (3) is connected.

[0072] A home appliance (10) can transmit information about its operation or status to another home appliance, a user device (2), or a server (3) via a network. For example, the home appliance (10) can transmit information about its operation or status to another home appliance, a user device (2), or a server (3) when a request is received from a server (3), when a specific event occurs in the home appliance (10), or periodically or in real time. When information about its operation or status is received from the home appliance (10), the server (3) can update the information about the operation or status of the home appliance (10) that has been stored therein, and transmit the updated information about the operation and status of the home appliance (10) to the user device (2) via a network. Here, updating information can include various operations that change existing information, such as an operation of adding new information to existing information, an operation of replacing existing information with new information, etc.

[0073] The home appliance (10) can obtain various information from other home appliances, user devices (2), or servers (3), and provide the obtained information to the user. For example, the home appliance (10) can obtain information related to the functions of the home appliance (10) (e.g., cooking methods, washing instructions, etc.) and various environmental information (e.g., weather, temperature, humidity, etc.) from the server (3), and output the obtained information through a user interface.

[0074] The home appliance (10) can operate according to a control command received from another home appliance, a user device (2), or a server (3). For example, if the home appliance (10) has obtained prior approval from the user to operate according to a control command from the server (3) even without user input, the home appliance (10) can operate according to a control command received from the server (3). Here, the control command received from the server (3) may include, but is not limited to, a control command input by the user through the user device (2) or a control command based on preset conditions.

[0075] The user device (2) can transmit information about the user to the home appliance (10) or the server (3) via the communication module. For example, the user device (2) can transmit information about the user's location, the user's health status, the user's preferences, the user's schedule, etc. to the server (3). The user device (2) can transmit information about the user to the server (3) with the user's prior consent.

[0076] The home appliance (10), user device (2), or server (3) may determine a control command using technology such as artificial intelligence. For example, the server (3) may receive information regarding the operation or status of the home appliance (10) or information regarding the user of the user device (2), process the information using technology such as artificial intelligence, and transmit the processing result or control command to the home appliance (10) or user device (2) based on the processing result.

[0077] Hereinafter, among home appliances (10), home appliances such as an electric range (13), an electric oven (14), and a microwave oven (19) that can cook food contained in a container are described as electronic devices (100).

[0078] FIG. 2 is a block diagram showing the configuration of an electronic device (100) according to one embodiment of the present disclosure.

[0079] The electronic device (100) is a device that cooks food contained in a container, and may be a device that identifies the type of container. For example, the electronic device (100) is a device that automatically cooks food contained in a container, and may be an oven, microwave oven, air fryer, oven range, deep fryer, etc. However, the present invention is not limited thereto, and the electronic device (100) may be any device that can identify the type of container.

[0080] According to FIG. 2, the electronic device (100) includes a microwave generating device (110), a thermal imaging camera (120), a memory (130), and a processor (140).

[0081] The microwave generating device (110) may be a device that outputs microwaves. For example, the microwave generating device (110) may include a magnetron (MGT) that generates microwaves of 2.45 GHz. However, the present invention is not limited thereto, and the microwave generating device (110) may be any device that can rotate polarized water molecules through electromagnetic waves and generate heat through collisions with other water molecules.

[0082] A thermal imaging camera (120) can obtain a thermal image by visualizing infrared radiation emitted by a subject. For example, the thermal imaging camera (120) can detect and image the radiant heat emitted by a hot object. In this case, the processor (140) can obtain a temperature pattern based on the thermal image.

[0083] However, this is not limited to this, and the thermal imaging camera (120) can obtain a thermal image by visualizing infrared rays emitted by a subject, and can also obtain a temperature pattern from the thermal image. In this case, the processor (140) can obtain the temperature pattern provided by the thermal imaging camera (120).

[0084] Memory (130) may refer to hardware that stores information such as data in an electrical or magnetic form so that a processor (140) or the like can access it. To this end, memory (130) may be implemented as at least one piece of hardware from among non-volatile memory, volatile memory, flash memory, hard disk drive (HDD), solid state drive (SSD), RAM, ROM, etc.

[0085] The memory (130) may store at least one instruction for the operation of the electronic device (100) or the processor (140). Here, the instruction is a code unit that instructs the operation of the electronic device (100) or the processor (140), and may be written in machine language, which is a language that a computer can understand. The memory (130) may also store a plurality of instructions for performing the work of the electronic device (100) or the processor (140) as an instruction set.

[0086] The memory (130) may store data in bit or byte units that can represent characters, numbers, images, etc. For example, a neural network module, etc. may be stored in the memory (130).

[0087] The memory (130) is accessed by the processor (140), and the processor (140) can read, write, modify, delete, or update instructions, instruction sets, or data.

[0088] The processor (140) controls the overall operation of the electronic device (100). The processor (140) is connected to each component of the electronic device (100) and can control the overall operation of the electronic device (100). For example, the processor (140) is connected to components such as a microwave generator (110), a thermal imaging camera (120), a memory (130), a user interface, etc. and can control the operation of the electronic device (100).

[0089] At least one processor may include one or more of a CPU, a GPU (Graphics Processing Unit), an APU (Accelerated Processing Unit), a MIC (Many Integrated Core), an NPU (Neural Processing Unit), a hardware accelerator, or a machine learning accelerator. The at least one processor may control one or any combination of other components of the electronic device (100) and perform operations related to communication or data processing. The at least one processor may execute one or more programs or instructions stored in the memory (130). For example, the at least one processor may perform a method according to an embodiment of the present disclosure by executing one or more instructions stored in the memory (130).

[0090] When a method according to an embodiment of the present disclosure includes multiple operations, the multiple operations may be performed by a single processor or by multiple processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor and the third operation may be performed by the second processor (e.g., an artificial intelligence-dedicated processor).

[0091] At least one processor may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicores or heterogeneous multicores). When at least one processor is implemented as a multicore processor, each of the multiple cores included in the multicore processor may include internal processor memory, such as cache memory or on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to an embodiment of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to an embodiment of the present disclosure.

[0092] When a method according to an embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one core among the plurality of cores included in a multi-core processor, or may be performed by the plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.

[0093] In embodiments of the present disclosure, at least one processor may mean a system on a chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, an NPU, a hardware accelerator, or a machine learning accelerator, but embodiments of the present disclosure are not limited thereto. However, for convenience of explanation, the operation of the electronic device (100) is described below using the expression processor (140).

[0094] The processor (140) controls the microwave generating device (110) to output microwaves into a cavity, which is an internal space of the electronic device (100), obtains a first temperature pattern based on a plurality of first thermal image images acquired through the thermal imaging camera (120) while outputting microwaves, obtains a second temperature pattern based on a plurality of second thermal image images acquired through the thermal imaging camera (120) while not outputting microwaves, obtains temperature pattern information including information on whether microwaves are output, the first temperature pattern, and the second temperature pattern, inputs the temperature pattern information into a neural network model to identify the type of a container, and outputs a usage notification for the electronic device (100) based on the type of the container.

[0095] The processor (140) may acquire a plurality of first thermal image images through the thermal imaging camera (120) while outputting microwaves, acquire a plurality of second thermal image images through the thermal imaging camera (120) while not outputting microwaves, and acquire temperature pattern information including a first temperature pattern of the container acquired based on the plurality of first thermal image images and a second temperature pattern of the container acquired based on the plurality of second thermal image images. For example, the processor (140) may output microwaves for 30 seconds, not output microwaves for 30 seconds immediately thereafter, and repeat this for 5 minutes. The processor (140) may acquire six first thermal image images through the thermal imaging camera (120) at 5-second intervals while outputting microwaves for 30 seconds, and acquire six second thermal image images through the thermal imaging camera (120) at 5-second intervals while not outputting microwaves for 30 seconds immediately thereafter. The processor (140) can identify a first temperature pattern in which the temperature increases from six first thermal image images, identify a second temperature pattern in which the temperature decreases from six second thermal image images, and obtain temperature pattern information for one minute. Here, the temperature pattern information can include a first temperature pattern while microwaves are output and a second temperature pattern while microwaves are not output. The temperature pattern information can include not only the first temperature pattern and the second temperature pattern, but also information that the first temperature pattern is a temperature pattern while microwaves are output and the second temperature pattern is a temperature pattern while microwaves are not output.

[0096] In the above, it has been described that the processor (140) acquires temperature pattern information for 1 minute, but this is not limited thereto, and the time for acquiring the temperature pattern information may vary. In addition, when the microwave generator (110) is turned on / off periodically, the processor (140) may acquire temperature pattern information during the time it takes for the microwave generator (110) to be turned on / off once. For example, when the processor (140) turns on the microwave generator (110) for 8 seconds and immediately turns off the microwave generator (110) for 22 seconds, the processor (140) may acquire temperature pattern information for 30 seconds. The processor (140) may identify the time it takes for the microwave generator (110) to be turned on / off once as the minimum time for acquiring temperature pattern information.

[0097] However, the present invention is not limited thereto, and the processor (140) may identify a time longer than the time for turning on / off the microwave generator (110) once as the time for acquiring temperature pattern information. For example, the processor (140) may identify a time for turning on / off the microwave generator (110) twice as the time for acquiring temperature pattern information. The processor (140) may also operate adaptively. For example, the processor (140) may acquire first temperature pattern information during the time for turning on / off the microwave generator (110) once, and input the first temperature pattern information into a neural network model to identify the type of container. However, if the type of the container is not clearly identified using the first temperature pattern information, the processor (140) may obtain second temperature pattern information while the microwave generator (110) is turned on / off once and then turned on / off once more, and input the second temperature pattern information into a neural network model to identify the type of the container.

[0098] Temperature pattern information may also include temperature patterns when microwaves are not output at specific cycles.

[0099] The temperature pattern information may store only one temperature pattern. In this case, the temperature pattern information may include information about one temperature pattern and a time period during which microwaves are output and a time period during which microwaves are not output in the temperature pattern. For example, the processor (140) may acquire six thermal image images through the thermal imaging camera (120) at 5-second intervals while outputting microwaves for 30 seconds, and may acquire six thermal image images through the thermal imaging camera (120) at 5-second intervals while not outputting microwaves for 30 seconds immediately thereafter. The processor (140) may identify a temperature pattern from the 12 thermal image images and acquire temperature pattern information including the temperature pattern and microwave output information at each point in time of the temperature pattern.

[0100] The processor (140) obtains temperature pattern information at preset time intervals, inputs the temperature pattern information obtained at preset time intervals into a neural network model, obtains multiple probability values ​​corresponding to the types of multiple containers, and if the largest probability value among the multiple probability values ​​is greater than or equal to a preset value, the type corresponding to the largest probability value can be identified as the type of the container.

[0101] For example, the processor (140) can input the temperature pattern information acquired during the first minute into the neural network model to obtain a probability value of 0.2 when the container is ceramic, a probability value of 0.1 when the container is paper, and a probability value of 0.7 when the container is steel. Here, the processor (140) can identify the container as steel if the largest probability value of 0.7 is greater than or equal to the preset probability value of 0.7. Since the preset probability value is 0.8, the processor (140) can also input the temperature pattern information acquired during the minute immediately following the first minute into the neural network model to identify the type of the container if the largest probability value of 0.7 is less than the preset probability value of 0.8. In this case, since the second temperature pattern information is a state in which the container has been heated through a microwave for a total of 2 minutes, it can be easier to identify the type of the container than the temperature pattern information for the first minute.

[0102] If the type of the container is a preset type corresponding to a hazardous material, the processor (140) can control the microwave generating device (110) to output a notification indicating that the container is a hazardous material and to stop the output of microwaves, and if the type of the container is not a preset type corresponding to a hazardous material, the processor (140) can control the microwave generating device (110) to maintain the output of microwaves.

[0103] For example, if the container type is steel, the processor (140) may output a notification indicating that the container is unsuitable for use due to the risk of sparks occurring if the container is continuously heated, and control the microwave generating device (110) to stop the microwave output. If the container type is a non-hazardous container such as ceramic, the processor (140) may control the microwave generating device (110) to maintain the microwave output. Here, the non-hazardous container may be a container suitable for use with the electronic device (100). Through this operation, the problem of dangers such as fire occurring even when a user uses a container unsuitable for the electronic device (100) can be solved.

[0104] The electronic device (100) further includes a display, and the processor (140) can identify a type of recommended container corresponding to an operation mode of the electronic device (100), and control the display to display the recommended container when the type of the container is different from the type of the recommended container.

[0105] For example, if a pot or heating fan is used in the warming operation mode, the heat applied to the food is taken away by the container, which reduces efficiency, and the processor (140) may control the display to recommend a container that is more suitable for warming.

[0106] However, it is not limited thereto, and the processor (140) may also control the display to identify the type of the container and display the operation mode of the electronic device (100) corresponding to the type of the container.

[0107] The electronic device (100) further includes a user interface, and when a user command for controlling the electronic device (100) is received through the user interface, the processor (140) can control the microwave generating device (110) to output microwaves to the cavity at a preset output power level for a preset time to identify the type of the container before an operation corresponding to the user command.

[0108] For example, when a defrosting command is received, the processor (140) may control the microwave generator (110) to output microwaves at 180 W for 20 seconds to identify the type of container before a defrosting operation corresponding to the defrosting command. Here, 20 seconds and 180 W may be unrelated to the defrosting operation. The processor (140) may control the microwave generator (110) to output microwaves at 180 W for 20 seconds to identify the type of container regardless of which operation command is received. However, the present invention is not limited thereto, and the minimum time and minimum output for identifying the type of container before an operation corresponding to a user command may be varied in any number of ways.

[0109] The processor (140) can identify the type of the container for a preset period of time, and if the type of the container is not a preset type corresponding to a hazardous material, control the microwave generating device (110) based on the operation mode of the electronic device (100) corresponding to the user command. In the example described above, when a heating command is received, the processor (140) can control the microwave generating device (110) to output microwaves at 180 W for 20 seconds before starting the heating operation to identify the type of the container, and if the type of the container is not a preset type corresponding to a hazardous material, start the heating operation. When a heating command is received, the processor (140) controls the microwave generator (110) to output microwaves at 180 W for 20 seconds before starting the heating operation to identify the type of the container, and if the type of the container is a preset type corresponding to a hazardous material, outputs a notification indicating that the container is a hazardous material, and controls the microwave generator (110) to stop outputting microwaves.

[0110] The processor (140) may obtain a first temperature pattern of the container based on a plurality of first thermal image images, obtain a second temperature pattern of the container based on a plurality of second thermal image images, obtain temperature pattern information of the container including information on whether microwaves are output, the first temperature pattern of the container, and the second temperature pattern of the container, and input the temperature pattern information of the container into a neural network model to identify the type of the container.

[0111] The electronic device (100) further includes a heater, and the processor (140) can output a usage notification for the heater to the user when the type of the container is one that cannot use the heater.

[0112] For example, when a user command to use an oven function is received, the processor (140) controls the microwave generating device (110) to output microwaves into a cavity, which is an internal space of the electronic device (100), obtains a first temperature pattern of the container based on a plurality of first thermal image images, obtains a second temperature pattern of the container based on a plurality of second thermal image images, obtains temperature pattern information including information on whether microwaves are output, the first temperature pattern of the container, and the second temperature pattern of the container, inputs the temperature pattern information into a neural network model to identify the type of the container, and if the type of the container is not a heat-resistant glass that is difficult to withstand high temperatures or a container covered with vinyl, outputs a notification that a fire may occur in the container, and stops the operation of the heater.

[0113] Meanwhile, functions related to artificial intelligence according to the present disclosure can be operated through a processor (140) and a memory (130).

[0114] The processor (140) may be composed of one or more processors. In this case, the one or more processors may be a processor such as a CPU, an AP, a DSP, a graphics-only processor such as a GPU, a VPU (Vision Processing Unit), or an artificial intelligence-only processor such as an NPU.

[0115] One or more processors are controlled to process input data according to predefined operation rules or artificial intelligence models stored in memory (130). If one or more processors are dedicated artificial intelligence processors, the dedicated artificial intelligence processors may be designed with a hardware structure specialized for processing artificial intelligence models. The predefined operation rules or artificial intelligence models are characterized by being created through learning.

[0116] Here, "created through learning" means that a predefined set of operating rules or an artificial intelligence model is created to perform a desired characteristic (or purpose) by learning an artificial intelligence model using a learning algorithm and a plurality of learning data. This learning may be performed on the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and / or system. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0117] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and performs neural network operations by calculating the results of previous layers and the multiple weights. The multiple weights of the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated during the learning process to reduce or minimize the loss or cost values ​​obtained by the artificial intelligence model.

[0118] Artificial neural networks may include deep neural networks (DNNs), such as, but not limited to, convolutional neural networks (CNNs), deep neural networks (DNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), generative adversarial networks (GANs), or deep Q-networks.

[0119] A neural network model for identifying a container type may be a model that has learned temperature pattern information and the container type. Here, the temperature pattern information may include at least one temperature pattern and whether or not a microwave output corresponds to each of the at least one temperature patterns. During the learning process, the input data of the neural network model may be temperature pattern information, and the output data may be the container type.

[0120] However, this is not limited thereto, and the data for training the neural network model may be of any variety. For example, during the training process, the input data of the neural network model may further include not only temperature pattern information, but also at least one of the ambient temperature of the electronic device (100), the operating mode of the electronic device (100), or the output power level of the microwave generator (110). During the training process, the output data of the neural network model may be the type of container.

[0121] FIG. 3 is a block diagram showing a detailed configuration of an electronic device (100) according to an embodiment of the present disclosure. The electronic device (100) may include a microwave generating device (110), a thermal imaging camera (120), a memory (130), and a processor (140). In addition, according to FIG. 3, the electronic device (100) may further include a user interface (150), a display (160), a heater (170), a communication interface (180), a microphone (190), and a speaker (195). For components illustrated in FIG. 3 that overlap with those illustrated in FIG. 2, a detailed description thereof will be omitted.

[0122] The user interface (150) may be implemented with buttons, a touch pad, a mouse, a keyboard, etc., or may be implemented with a touch screen capable of performing both display and operation input functions. Here, the buttons may be various types of buttons, such as mechanical buttons, touch pads, wheels, etc., formed on any area of ​​the front, side, or back of the main body of the electronic device (100).

[0123] The display (160) is a configuration that displays an image and can be implemented as a display of various forms such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, a PDP (Plasma Display Panel), etc. The display (160) may also include a driving circuit, a backlight unit, etc. that can be implemented as a form such as an a-si TFT, an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc. Meanwhile, the display (160) may be implemented as a touch screen combined with a touch sensor, a flexible display, a 3D display, etc.

[0124] The heater (170) may be configured to cook food under the control of the processor (140). For example, the heater (170) may include at least one of a heater for applying heat to the food or a steamer used in the cooking process. The heater (170) may also be implemented in a form that includes a heater and a fan for circulating the heat of the heater. However, the present invention is not limited thereto, and the heater (170) may have any configuration as long as it can cook food.

[0125] The communication interface (180) is a configuration that performs communication with various types of external devices according to various types of communication methods. For example, the electronic device (100) can perform communication with each of the user device (2) and the server (3) through the communication interface (180).

[0126] The communication interface (180) may include a Wi-Fi module, a Bluetooth module, an infrared communication module, a wireless communication module, etc. Here, each communication module may be implemented in the form of at least one hardware chip.

[0127] Wi-Fi and Bluetooth modules communicate via Wi-Fi and Bluetooth, respectively. When using a Wi-Fi or Bluetooth module, connection information, such as the SSID and session key, is first transmitted and received. This information is then used to establish a communication connection before various other information can be transmitted and received. Infrared communication modules use infrared data association (IrDA) technology, which wirelessly transmits data over short distances using infrared light, which lies between visible light and millimeter waves.

[0128] In addition to the above-described communication method, the wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), 5G (5th Generation), etc.

[0129] The communication interface (180) may include a wired communication interface such as HDMI, DP, Thunderbolt, USB, RGB, D-SUB, DVI, etc.

[0130] In addition, the communication interface (180) may include at least one of a LAN (Local Area Network) module, an Ethernet module, or a wired communication module that performs communication using a pair cable, a coaxial cable, or an optical fiber cable.

[0131] The microphone (190) is configured to receive sound and convert it into an audio signal. The microphone (190) is electrically connected to the processor (140) and can receive sound under the control of the processor (140).

[0132] For example, the microphone (190) may be formed as an integrated unit integrated into the upper side, front side, side side, etc. of the electronic device (100). The microphone (190) may also be provided in a remote control separate from the electronic device (100). In this case, the remote control may receive sound through the microphone (190) and provide the received sound to the electronic device (100).

[0133] The microphone (190) may include various configurations such as a microphone that collects analog sound, an amplifier circuit that amplifies the collected sound, an A / D conversion circuit that samples the amplified sound and converts it into a digital signal, and a filter circuit that removes noise components from the converted digital signal.

[0134] Meanwhile, the microphone (190) may be implemented in the form of a sound sensor, and any method may be used as long as it has a configuration capable of collecting sound.

[0135] The processor (140) may also receive user commands through a microphone (190).

[0136] The speaker (195) is a component that outputs various audio data processed by the processor (140) as well as various notification sounds and voice messages.

[0137] The processor (140) can control the speaker (195) to output a sound that guides the use of the electronic device (100).

[0138] As described above, the electronic device (100) can identify the type of container based on temperature pattern information inside the electronic device (100) and output a notification to the user, thereby improving user convenience.

[0139] Hereinafter, the operation of the electronic device (100) will be described in more detail with reference to FIGS. 4 to 13. For convenience of explanation, individual embodiments are described in FIGS. 4 to 13. However, the individual embodiments of FIGS. 4 to 13 may be implemented in any combination.

[0140] FIGS. 4 and 5 are drawings for explaining a temperature pattern according to the control of a microwave generating device (110) according to one embodiment of the present disclosure.

[0141] The processor (140) can control the microwave generating device (110) based on a PWM (pulse width modulation) control signal. For example, the processor (140) can provide a PWM control signal as illustrated in FIG. 4 to the microwave generating device (110) to turn the microwave generating device (110) on or off.

[0142] The microwave generator (110) can be turned on in a high level section (410) and turned off in a low level section (420) in the PWM control signal.

[0143] In Fig. 4, the duty cycle is illustrated as 50% for convenience of explanation, but is not limited thereto, and the duty cycle may vary as desired. For example, the duty cycle may be determined based on the output power level of the microwave generator (110). For example, if the output power level of the microwave generator (110) is 450 W, the waveform may be at a high level for 18 seconds and at a low level for 12 seconds, and this waveform may be repeated. If the output power level of the microwave generator (110) is 180 W, the waveform may be at a high level for 8 seconds and at a low level for 22 seconds, and this waveform may be repeated.

[0144] The processor (140) can obtain temperature pattern information based on the PWM control signal and information obtained through the thermal imaging camera (120). Here, the temperature pattern information can include microwave output information and a temperature pattern of a container placed within a cavity.

[0145] For convenience of explanation, Fig. 5 assumes that the output power level of the microwave generator (110) is 180 W, and that the PWM control signal is at a high level for 8 seconds and a low level for 22 seconds, and shows the temperature pattern of the container accordingly.

[0146] The processor (140) can control the microwave generating device (110) based on a PWM control signal that is at a high level for 8 seconds and at a low level for 22 seconds. Accordingly, the microwave generating device (110) is turned on for 8 seconds and then turned off for 22 seconds, and this operation can be repeated.

[0147] The processor (140) can acquire a plurality of first thermal image images through the thermal imaging camera (120) during 8 seconds when microwaves are output. The processor (140) can acquire a temperature pattern while microwaves are output from the plurality of first thermal image images, and the temperature pattern may be a state in which the temperature increases, as shown by the dotted circle in FIG. 5. The processor (140) can acquire a plurality of second thermal image images through the thermal imaging camera (120) during 22 seconds when microwaves are not output. The processor (140) can acquire a temperature pattern while microwaves are not output from the plurality of second thermal image images, and the temperature pattern may be a state in which the temperature decreases, as shown by the solid circle in FIG. 5.

[0148] FIGS. 6 and 7 are drawings illustrating a method for analyzing a thermal image according to an embodiment of the present disclosure.

[0149] The processor (140) can identify a container within a cavity based on information acquired from the thermal imaging camera (120). For example, the processor (140) can acquire a plurality of thermal image images (610, 620, 630) through the thermal imaging camera (120) at preset time intervals, as illustrated in FIG. 6. Here, the plurality of thermal image images may be, for example, in a state where the temperature changes due to microwaves, and since the temperature changes of the container and an area outside the container (e.g., the air or the floor where the container is not raised, etc.) are different, the degree of temperature change may vary depending on the area, as illustrated in FIG. 6.

[0150] The processor (140) can identify a container based on a plurality of thermal images and identify a temperature pattern of the container. For example, the processor (140) can preprocess a plurality of thermal images through an Erosion algorithm and identify the container through a K-means clustering algorithm. For example, as illustrated in FIG. 7, the processor (140) normalizes a thermal image at time t and a thermal image at time t+1, respectively, obtains a temperature difference image from the normalized thermal image at time t and the normalized thermal image at time t+1, performs pixel clustering on the thermal image at time t+1 to obtain a clustered image, and identifies an area of ​​the container based on the temperature difference image and the clustered image. At this time, since the container and the food inside the container may have similar temperature patterns, the processor (140) can also identify the entire area including the container and the food inside the container.

[0151] However, the present invention is not limited thereto, and the processor (140) may identify the temperature pattern of a container within a plurality of thermal image images in any number of different ways. For example, the processor (140) may identify the temperature pattern of a container from a plurality of thermal image images through a neural network model. In this case, the input data of the neural network model during the learning process may be a plurality of thermal image images, and the output data may be the temperature pattern of the container.

[0152] FIGS. 8 and 9 are diagrams for explaining temperature patterns according to the type of container and output of a neural network model according to the temperature pattern according to one embodiment of the present disclosure.

[0153] For convenience of explanation, Fig. 8 assumes that the output power level of the microwave generator (110) is 450 W, and that the PWM control signal is at a high level for 18 seconds and a low level for 12 seconds, and shows the temperature patterns of various containers accordingly.

[0154] The processor (140) may acquire a plurality of first thermal image images through the thermal imaging camera (120) while outputting microwaves, acquire a plurality of second thermal image images through the thermal imaging camera (120) while not outputting microwaves, and acquire temperature pattern information including a first temperature pattern of the container acquired based on the plurality of first thermal image images and a second temperature pattern of the container acquired based on the plurality of second thermal image images. For example, as illustrated in FIG. 8, the processor (140) may acquire a plurality of first thermal image images through the thermal imaging camera (120) during a section 810 while outputting microwaves, acquire a plurality of second thermal image images through the thermal imaging camera (120) during a section 820 while not outputting microwaves, and acquire temperature pattern information including a first temperature pattern of the container acquired based on the plurality of first thermal image images and a second temperature pattern of the container acquired based on the plurality of second thermal image images.

[0155] Temperature patterns may vary depending on the container type. For example, as illustrated in FIG. 8, the temperature of paper rises rapidly in the 810 section. As illustrated in FIG. 9, the processor (140) inputs temperature pattern information of the paper into a neural network model to identify the container type as paper and provide a notification indicating a risk of fire.

[0156] As illustrated in FIG. 8, the temperature of the risk-free container and steel increases at a relatively slower rate than that of the paper in the 810 section, and the processor (140) inputs the temperature pattern information of the risk-free container into the neural network model to identify that the type of the container is a risk-free container, and inputs the temperature pattern information of the steel into the neural network model to identify that the type of the container is steel.

[0157] However, the neural network model may not be precise, in which case the processor (140) may not be able to clearly distinguish between a safe container and steel. For example, the processor (140) may input the temperature pattern information of steel into the neural network model, and as a result, obtain a probability of 0.4 that it is a safe container, a probability of 0.4 that it is steel, and a probability of 0.2 that it is paper. The type of the container may not be clearly identified with only the temperature pattern information using the sections 810 and 820. In this case, as illustrated in FIG. 8, the processor (140) may additionally acquire a plurality of first thermal image images through the thermal imaging camera (120) during the section 830 that outputs microwaves, additionally acquire a plurality of second thermal image images through the thermal imaging camera (120) during the section 840 that does not output microwaves, and update the temperature pattern information based on the additionally acquired plurality of first thermal image images and the additionally acquired plurality of second thermal image images. The processor (140) can re-identify the type of container through updated temperature pattern information and output a hazard warning to the user before the steel sparks.

[0158] The neural network model can learn temperature patterns according to the type of each container. Once learning is complete, the neural network model can receive temperature pattern information and output probability values ​​for each container type, as illustrated in FIG. 9. If the highest probability value among multiple probability values ​​is greater than or equal to a preset value, the processor (140) can identify the type corresponding to the highest probability value as the container type. However, the present invention is not limited thereto, and the neural network model can also output the type of the container based on the temperature pattern information.

[0159] Meanwhile, in Fig. 8, it is described that temperature pattern information is acquired based on a plurality of thermal image images during the 810 section where microwaves are output and the 820 section where microwaves are not output, but it is not limited thereto. For example, the processor (140) may acquire temperature pattern information based on a plurality of thermal image images during the 810 section where microwaves are output, and input the temperature pattern information into a neural network model to identify the type of container.

[0160] FIG. 10 is a drawing for explaining an operation of outputting a notification indicating that a container is a hazardous material according to one embodiment of the present disclosure.

[0161] When a user command for controlling the electronic device (100) is received through the user interface (150), the processor (140) may control the microwave generating device (110) to output microwaves to the cavity at a preset output power level for a preset time to identify the type of container before an operation corresponding to the user command. The processor (140) may control the microwave generating device (110) to output microwaves to the cavity to identify the type of container, regardless of the operation mode corresponding to the user command. In this case, the processor (140) may control the display (160) to display a notification (1010) that says, “Measuring temperature & hazardous materials...”, as illustrated in the upper part of FIG. 10.

[0162] The processor (140) can identify the type of the container for a preset period of time, and if the type of the container is not a preset type corresponding to a hazardous material, control the microwave generating device based on the operating mode of the electronic device (100) corresponding to the user command.

[0163] As shown in the lower part of FIG. 10, if the type of the container is a preset type corresponding to a hazardous material, the processor (140) can control the microwave generating device (110) to provide a notification (1020) indicating that the container is a hazardous material, such as “Danger! Check the container.”, and stop the output of the microwave.

[0164] In Fig. 10, for convenience of explanation, the processor (140) is described as outputting microwaves for a preset period of time regardless of the operation mode corresponding to the user command, but is not limited thereto. For example, the processor (140) may output microwaves for a preset period of time regardless of the operation mode corresponding to the user command, but may extend the time for outputting microwaves if the container is not identified while outputting microwaves.

[0165] FIG. 11 is a drawing for explaining an operation for guiding the efficiency of a container according to one embodiment of the present disclosure.

[0166] The processor (140) can identify the type of recommended container corresponding to the operation mode of the electronic device (100), as illustrated in FIG. 11, and, if the type of the identified container is the same as the type of the recommended container, provide a notification (1110) informing that the temperature and efficiency of the container are present.

[0167] The processor (140) may also control the display (150) to display the recommended container if the type of the identified container is different from the type of the recommended container.

[0168] FIG. 12 is a drawing for explaining an operation of guiding the temperature of a container according to one embodiment of the present disclosure.

[0169] After completing operation in an operation mode corresponding to a user command, the processor (140) may provide a notification regarding the temperature of the container. For example, after completing operation in an operation mode corresponding to a user command, the processor (140) may provide a notification (1210) such as "Container: 40°C." This operation may prevent the user from being burned.

[0170] However, the present invention is not limited thereto, and the processor (140) may provide a notification regarding the temperature of the container even before completing an operation in an operation mode corresponding to a user command. For example, after performing an operation to identify the type of the container, if the type of the container is a preset type corresponding to a hazardous material, the processor (140) may output a notification indicating that the container is a hazardous material along with the temperature of the container.

[0171] FIG. 13 is a drawing for explaining the operation according to the type of glass according to one embodiment of the present disclosure.

[0172] The processor (140) can also distinguish between heat-resistant glass, tempered glass, and general glass, even if they are the same glass. This operation is possible because the neural network model learns the difference in temperature change from the temperature pattern of each glass.

[0173] The processor (140) can identify that the container cannot be used in a microwave oven or oven if the container is made of glass. The processor (140) can restrict the use of the microwave generator (110) and the heater (170) if the container is made of glass.

[0174] The processor (140) can identify that the container is suitable for use in a microwave oven or oven if the container is made of heat-resistant glass. If the container is made of heat-resistant glass, the processor (140) can control the microwave generator (110) or heater (170) to perform cooking.

[0175] FIG. 14 is a flowchart for explaining a control method of an electronic device according to an embodiment of the present disclosure.

[0176] First, a microwave generating device included in an electronic device is controlled to output microwaves into a cavity, which is an internal space of the electronic device (S1410). Then, a first temperature pattern is acquired based on a plurality of first thermal image images acquired through a thermal imaging camera included in the electronic device while outputting microwaves (S1420). Then, a second temperature pattern is acquired based on a plurality of second thermal image images acquired through the thermal imaging camera while not outputting microwaves (S1430). Then, temperature pattern information including information on whether microwaves are output, the first temperature pattern, and the second temperature pattern is acquired (S1440). Then, the temperature pattern information is input into a neural network model to identify the type of the container (S1450). Then, a usage notification for the electronic device is output based on the type of the container (S1460).

[0177] And, the step of obtaining temperature pattern information (S1440) obtains temperature pattern information at preset time intervals, and the step of identifying (S1450) inputs the temperature pattern information obtained at preset time intervals into a neural network model to obtain multiple probability values ​​corresponding to the types of multiple containers, and if the largest probability value among the multiple probability values ​​is greater than or equal to the preset value, the type corresponding to the largest probability value can be identified as the type of the container.

[0178] In addition, the step of outputting a usage notification (S1460) outputs a notification indicating that the container is a hazardous material if the type of the container is a preset type corresponding to a hazardous material, and controls the microwave generating device to stop microwave output, and if the type of the container is not a preset type corresponding to a hazardous material, controls the microwave generating device to maintain microwave output.

[0179] In addition, the method may further include a step of identifying a type of recommended container corresponding to an operation mode of the electronic device and a step of displaying the recommended container when the type of the container is different from the type of the recommended container.

[0180] In addition, the step of outputting microwaves (S1410) can output microwaves to the cavity by controlling the microwave generating device for a preset time at a preset output power level to identify the type of the container before an operation corresponding to the user command is performed when a user command for controlling the electronic device is received.

[0181] And, the identifying step (S1450) may further include a step of identifying the type of the container for a preset time, and the control method may further include a step of controlling the microwave generating device based on the operation mode of the electronic device corresponding to the user command if the type of the container is not a preset type corresponding to a hazardous material.

[0182] In addition, the step of obtaining a first temperature pattern (S1420) obtains a first temperature pattern of the container based on a plurality of first thermal image images, the step of obtaining a second temperature pattern (S1430) obtains a second temperature pattern of the container based on a plurality of second thermal image images, the step of obtaining temperature pattern information (S1440) obtains temperature pattern information of the container including information on whether microwaves are output, the first temperature pattern of the container, and the second temperature pattern of the container, and the step of identifying (S1450) inputs the temperature pattern information of the container into a neural network model to identify the type of the container.

[0183] In addition, if the type of the container is one that cannot use a heater included in the electronic device, a step of outputting a usage notification for the heater may be further included.

[0184] According to various embodiments of the present disclosure as described above, an electronic device can identify the type of a container based on temperature pattern information inside the electronic device and output a notification to the user, thereby improving user convenience.

[0185] Meanwhile, according to a temporary example of the present disclosure, the various embodiments described above can be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device is a device that can call instructions stored from the storage medium and operate according to the called instructions, and may include an electronic device (e.g., electronic device (A)) according to the disclosed embodiments. When an instruction is executed by a processor, the processor can perform a function corresponding to the instruction directly or by using other components under the control of the processor. The instruction may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not contain a signal and is tangible, but does not distinguish between data being stored semi-permanently or temporarily in the storage medium.

[0186] Furthermore, according to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0187] Furthermore, according to one embodiment of the present disclosure, the various embodiments described above may be implemented in a computer-readable recording medium or a similar device using software, hardware, or a combination thereof. In some cases, the embodiments described herein may be implemented by the processor itself. In a software implementation, embodiments such as the described procedures and functions may be implemented as separate software. Each software may perform one or more functions and operations described herein.

[0188] Meanwhile, computer instructions for performing processing operations of a device according to the various embodiments described above may be stored in a non-transitory computer-readable medium. The computer instructions stored in such a non-transitory computer-readable medium, when executed by the processor of the device, cause the device to perform processing operations of the device according to the various embodiments described above. A non-transitory computer-readable medium refers to a medium that permanently stores data and can be read by the device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Examples of non-transitory computer-readable media may include, but are not limited to, a CD, DVD, hard disk, Blu-ray disk, USB, memory card, or ROM.

[0189] Additionally, each of the components (e.g., modules or programs) according to the various embodiments described above may be composed of a single or multiple entities, or other sub-components may be further included in the various embodiments. Some components (e.g., modules or programs) may be integrated into a single entity and perform the same or similar functions performed by each respective component prior to integration. Operations performed by modules, programs, or other components according to various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, or other operations may be added.

[0190] While preferred embodiments of the present disclosure have been illustrated and described, the present disclosure is not limited to the above-described embodiments, and various modifications may be made by those skilled in the art without departing from the spirit of the present disclosure.

Claims

1. In electronic devices, Microwave generating device; thermal imaging camera; one or more processors; and A memory configured to store commands; When the above instructions are executed by the one or more processors, the electronic device, Controlling the microwave generating device to output microwaves into a cavity, which is an internal space of the electronic device, When the above microwave is output, a first temperature pattern is obtained based on a plurality of first thermal images obtained through the thermal imaging camera, If the above microwave is not output, a second temperature pattern is acquired based on a plurality of second thermal images acquired through the thermal imaging camera, Obtaining temperature pattern information including information on whether the microwave is output or not, the first temperature pattern, and the second temperature pattern, The above temperature pattern information is input into the first neural network model to identify the type of container, An electronic device that outputs a first usage notification for the electronic device based on the type of the container.

2. In paragraph 1, The one or more processors execute the instructions so that the electronic device, Obtain the above temperature pattern information at preset time intervals, The temperature pattern information acquired at the above preset time interval is input into the first neural network model to acquire multiple probability values ​​corresponding to each type of multiple containers, An electronic device that identifies the first type corresponding to the largest probability value as the type of the container if the largest probability value among the plurality of probability values ​​is greater than or equal to a preset value.

3. In paragraph 1, The one or more processors execute the instructions so that the electronic device, If the type of the container is a preset type corresponding to one or more hazardous substances among one or more preset types, a notification is output to indicate that the container is dangerous, and the microwave generating device is controlled to stop outputting the microwave. An electronic device that controls the microwave generating device to maintain the output of the microwave if the type of the container is not the preset type.

4. In paragraph 1, including display; The one or more processors execute the instructions so that the electronic device, Identify a second type of recommended container corresponding to the operating mode of the electronic device, An electronic device that controls the display to indicate the second type when the type of the container is different from the second type.

5. In paragraph 1, further including a user interface; The one or more processors execute the instructions so that the electronic device, An electronic device, wherein when a user command for controlling the electronic device is received through the user interface, the electronic device controls the microwave generating device for a preset time at a preset output power level to output the microwaves into the cavity before an operation corresponding to the user command to identify the type of the container.

6. In paragraph 5, The one or more processors execute the instructions so that the electronic device, Identify the type of the container during the above preset time, An electronic device that controls the microwave generating device based on an operation mode of the electronic device corresponding to the user command, if the type of the container is not a preset type corresponding to one or more hazardous substances among one or more preset types.

7. In paragraph 1, The above first neural network model is, An electronic device, wherein learning is performed based on first input data including a plurality of first temperature patterns and first output data including a plurality of types of containers.

8. In paragraph 7, The above first input data is, An electronic device further comprising at least one of a plurality of ambient temperatures, a plurality of operating modes, or a plurality of output power levels.

9. In paragraph 7, The one or more processors execute the instructions so that the electronic device, Obtain the temperature pattern information from the second neural network model, The above second neural network model is, An electronic device, wherein learning is performed based on second input data comprising a plurality of thermal images and second output data comprising a plurality of second temperature patterns.

10. In paragraph 1, Including heater; The one or more processors execute the instructions so that the electronic device, An electronic device that outputs a second usage notification for the heater when the type of the container is a second type that cannot use the heater.

11. In a method for controlling an electronic device, A step of controlling a microwave generating device included in the electronic device to output microwaves into a cavity, which is an internal space of the electronic device; A step of obtaining a first temperature pattern based on a plurality of first thermal imaging images obtained through a thermal imaging camera included in the electronic device while the above microwave is output; If the above microwave is not output, a step of obtaining a second temperature pattern based on a plurality of second thermal images obtained through the thermal imaging camera; A step of obtaining temperature pattern information including information on whether the microwave is output, the first temperature pattern, and the second temperature pattern; A step of inputting the above temperature pattern information into a first neural network model to identify the type of container; and A control method comprising: a step of outputting a first usage notification for the electronic device based on the type of the container; 12. In paragraph 10, The step of obtaining the above temperature pattern information is: Obtain the above temperature pattern information at preset time intervals, The step of identifying the type of the above container is: The temperature pattern information acquired at the above preset time interval is input into the first neural network model to acquire multiple probability values ​​corresponding to each type of multiple containers, A control method for identifying the first type corresponding to the largest probability value as the type of the container if the largest probability value among the above plurality of probability values ​​is greater than or equal to a preset value.

13. In paragraph 11, The steps for outputting the above usage notification are: If the type of the container is a preset type corresponding to one or more hazardous substances among one or more preset types, a notification is output to indicate that the container is dangerous, and the microwave generating device is controlled to stop outputting the microwave. A control method for controlling the microwave generating device to maintain the output of the microwave if the type of the container is not a preset type corresponding to the hazardous material.

14. In paragraph 11, A step of identifying a second type of recommended container corresponding to the operation mode of the electronic device; and A control method further comprising: a step of indicating the second type when the type of the container is different from the second type.

15. In paragraph 11, The step of outputting the above microwave is: A control method for controlling the microwave generating device to output microwaves into the cavity at a preset output power level for a preset time to identify the type of the container before an operation corresponding to the user command is performed, when a user command for controlling the electronic device is received through a user interface.

Images