Electronic device for mapping identification information and custom information and control method thereof

Abstract

An electronic device and a control method thereof are provided. The electronic device includes a communication interface and at least one processor. The at least one processor obtains identification information of an object. The at least one processor controls the communication interface to receive operation related information of the household appliance. The at least one processor maps setting information selected from the obtained operation-related information of the home appliance to the obtained identification information of the object.

Description

Technical Field

[0001] The present disclosure relates to an electronic device for mapping identification information and custom information and a control method thereof. Background Art

[0002] With the development of electronic technology, various types of electronic devices are being used. Recently used electronic devices may be newly developed electronic devices. Alternatively, recently used electronic devices may be devices that perform mechanical operations, but can be classified as electronic devices when electronic components are included therein.

[0003] For example, electronic devices may include household appliances that facilitate family life, such as refrigerators, washing machines, dryers, air conditioners, televisions, microwave ovens, electric stoves, and airdressers. Electronic devices may include desktop computers, laptop computers, smart phones, tablet computers, etc. that perform various functions. In addition, electronic devices may include devices that store and quickly process large amounts of data, such as servers.

[0004] In the past, electronic devices usually performed functions independently of each other. However, with the development of communication technology, various types of electronic devices are increasingly connected to each other in a communicative manner to perform functions in cooperation with each other. Electronic devices connected to each other in a communicative manner can perform functions while sharing roles according to the amount of data to be processed, and can quickly output results by processing data in parallel. In addition, a central electronic device can store a large amount of various data, and a terminal electronic device can obtain necessary data from the central electronic device and perform functions when necessary. Summary of the invention

[0005] An electronic device according to one or more embodiments of the present disclosure may include a communication interface and at least one processor. The at least one processor may obtain identification information of an object. The at least one processor may control the communication interface to receive operation-related information of a household appliance. The at least one processor may map setting information selected from the obtained operation-related information of the household appliance to the obtained identification information of the object.

[0006] According to one or more embodiments of the present disclosure, a control method of an electronic device may obtain identification information of an object. The control method may include receiving operation-related information of a household appliance. The control method may include mapping setting information selected from the obtained operation-related information of the household appliance to the obtained identification information of the object. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 is a diagram illustrating a system according to one or more embodiments of the present disclosure;

[0008] Figure 2is a block diagram showing a configuration of an electronic device according to one or more embodiments of the present disclosure;

[0009] Figure 3 is a detailed block diagram showing a configuration of an electronic device according to one or more embodiments of the present disclosure;

[0010] Figures 4a to 4c is a diagram for describing a process of mapping identification information and setting information to each other according to one or more embodiments of the present disclosure;

[0011] Figure 5a to Figure 5c is a diagram showing a process of obtaining identification information and operating a home appliance according to one or more embodiments of the present disclosure;

[0012] Figure 6 is a diagram illustrating a process of mapping identification information and setting information of multiple objects to each other according to one or more embodiments of the present disclosure;

[0013] Figure 7 is a diagram illustrating a process of mapping voice information to identification information according to one or more embodiments of the present disclosure;

[0014] Figure 8 is a diagram illustrating a process of operating a home appliance using setting information according to one or more embodiments of the present disclosure;

[0015] Figures 9a to 9c is a diagram showing common information according to one or more embodiments of the present disclosure;

[0016] Fig.10 is a diagram illustrating a process of operating a home appliance using a captured image according to one or more embodiments of the present disclosure;

[0017] Fig.11 is a diagram illustrating a process of operating a home appliance by photographing an object according to one or more embodiments of the present disclosure;

[0018] Figures 12a to 12c is a diagram showing operation-related information of a home appliance according to one or more embodiments of the present disclosure;

[0019] Fig.13a and Fig.13b is a diagram illustrating a process of estimating the composition of clothing and providing a notification to a user according to one or more embodiments of the present disclosure;

[0020] Fig.14a and Fig.14b is a diagram illustrating a process of estimating ingredients of food and providing notification to a user according to one or more embodiments of the present disclosure;

[0021] Fig.15 is a flowchart of a method for controlling an electronic device according to one or more embodiments of the present disclosure;

[0022] Fig.16 is a flowchart of a process of registering setting information according to one or more embodiments of the present disclosure; and

[0023] Fig.17 and Fig.18 is a flowchart of a process of operating a home appliance according to setting information according to one or more embodiments of the present disclosure. DETAILED DESCRIPTION

[0024] Hereinafter, one or more embodiments will be described in more detail with reference to the accompanying drawings. Various modifications may be made to the embodiments set forth herein. Example embodiments may be shown in the accompanying drawings and described in detail in the specific embodiments. However, the embodiments shown in the accompanying drawings are only intended to help understand the one or more embodiments. Therefore, the technical scope of the present disclosure is not limited by the embodiments shown in the accompanying drawings, and should be understood to include all equivalents or substitutes within the spirit and scope of the present disclosure.

[0025] Various components may be described using terms including ordinal numbers such as first and second, but the components are not limited to the terms. The terms are used only to distinguish components from each other.

[0026] It should be understood that the terms "include" and / or "comprising" when used herein specify the presence of the claimed features, integers, steps, operations, elements, components, or combinations thereof, but do not exclude the presence or addition of one or more features, integers, steps, operations, elements, components, or combinations thereof. When a component is referred to as being "coupled to" or "connected to" another component, it should be understood that the component can be directly coupled to or connected to the other component, however, another component may be interposed therebetween. Conversely, when a component is referred to as being "directly coupled to" or "directly connected to" another component, it should be understood that no component is interposed therebetween.

[0027] Meanwhile, the term "module" or "unit" when used in this article to describe a component should be understood to perform at least one function or operation. In addition, a "module" or "unit" can perform a function or operation by hardware, software, or a combination of hardware and software. Multiple "modules" or multiple "units" can be integrated into at least one module, excluding a "module" or "unit" that should be executed by certain hardware or at least one processor. As used herein, unless the context clearly indicates otherwise, a singular expression is intended to also include the plural form.

[0028] In the following description of the present disclosure, unless the previous operation should be performed before the next operation logically and temporally, the order of operations should be understood as non-restrictive. That is, except for this exception, even if the operation described as the next operation is performed before the operation described as the previous operation, the subject matter of the present disclosure will not change, and the scope of the present disclosure should be defined independently of the order of operations. As used herein, the expression "A or B" should not only be defined as selectively referring to A or B, but also including both A and B. As used herein, the term "comprising" should be understood to include other components in addition to the listed components.

[0029] In this specification, only the necessary components required to describe the present disclosure are described, and components irrelevant to the subject matter of the present disclosure are not described. In addition, the term "comprising" should not be interpreted as an exclusive meaning of only including the components described herein, but should be interpreted as a non-exclusive meaning of including other components.

[0030] In the following description of the present disclosure, when it is determined that the relevant well-known functions or components will obscure the subject matter of the present disclosure due to unnecessary details, the well-known functions or components will be briefly described or not described. Meanwhile, the embodiments can be implemented or operated independently, but can also be implemented or operated in combination.

[0031] Figure 1 is a diagram for describing a system according to an embodiment of the present disclosure.

[0032] refer to Figure 1 , the system may include a terminal device 100-1, a central electronic device 100-2, and a home appliance 300. For example, the terminal device 100-1 may include a smart phone, a tablet PC, a laptop computer, a navigation device, a wearable device, etc. The central electronic device 100-2 may include a server, a cloud, a hub device, etc. The server and the cloud may be embodied as separate electronic devices, and the hub device may be understood as one or more electronic devices in the home appliance 300 that are set as a main device. The home appliance 300 may include a clothing care machine 300-1, a dryer 300-2, a washing machine 300-3, an electric stove 300-4, a refrigerator 300-5, etc. The above examples are merely examples, and the present disclosure is not limited thereto.

[0033] The system may obtain the identification information of the object and obtain the setting information of the home appliance 300. In addition, the system may map the setting information of the home appliance 300 to the obtained identification information.

[0034] For example, the central electronic device 100 - 2 may perform a mapping process.

[0035] The terminal device 100-1 may obtain identification information of an object. Examples of objects may include clothes, food, and the like. The identification information may be implemented as an image or text including information about the object. The identification information may include a barcode, a quick response (QR) code, PDF417, CODE 16K, a data matrix, a maxi code, a color code, a three-dimensional (3D) code, and the like. The identification information may also include an image of the object. The identification information may be displayed on an area of ​​the object. The terminal device 100-1 may capture an image of the identification information of the object using a camera and obtain the identification information from the captured image. The terminal device 100-1 may send the obtained identification information to the central electronic device 100-2. For example, the terminal device 100-1 may capture an image of a barcode displayed (or attached) on the clothes and obtain the identification information of the clothes from the captured image. Alternatively, the terminal device 100-1 may capture an image of the clothes and identify the clothes from the captured image. Here, the image of the clothes may be the identification information.

[0036] The central electronic device 100-2 may receive operation-related information from each of the home appliances 300. For example, the central electronic device 100-2 may receive information such as fabric type, cleaning process, operation time, etc. from the clothing care machine 300-1.

[0037] The user may select setting information in the home appliance 300. For example, the user may select a standard course in the clothing care machine 300-1. Alternatively, the user may select various types of setting information for cleaning objects in the clothing care machine 300-1. That is, the home appliance 300 may identify the setting information selected by the user and send the identified setting information to the central electronic device 100-2. Alternatively, the user may select setting information of the home appliance 300 in the terminal device 100-1. The central electronic device 100-2 may send operation-related information obtained from each of the home appliances 300 to the terminal device 100-1. The terminal device 100-1 may obtain operation-related information of each of the home appliances 300 through the central electronic device 100-2. The user may select setting information in the terminal device 100-1. The terminal device 100-1 may identify the setting information selected by the user and send the identified setting information to the central electronic device 100-2.

[0038] The central electronic device 100-2 may obtain identification information of the object from the terminal device 100-1. The central electronic device 100-2 may obtain setting information selected by the user from the terminal device 100-1 or the home appliance 300. The central electronic device 100-2 may map the setting information to the obtained identification information of the object. The central electronic device 100-2 may send mapping information to the terminal device 100-1, wherein the identification information and the setting information are mapped to each other.

[0039] After the identification information and the setting information are mapped to each other, the central electronic device 100-2 can send a control command to the home appliance 300 when the identification information is obtained to perform the operation of the setting information mapped to the identification information. The terminal device 100-1 can capture an image of the identification information of the object and obtain the identification information from the captured image. In addition, the terminal device 100-1 can send the obtained identification information to the central electronic device 100-2. The central electronic device 100-2 can receive the identification information of the object from the terminal device 100-1 to obtain the identification information. The central electronic device 100-2 can identify the setting information mapped to the obtained identification information. The central electronic device 100-2 can send a control command related to the identified setting information to the home appliance 300. For example, the central electronic device 100-2 can identify that the setting information mapped to the obtained identification information is information for operating the clothing care machine 300-1 according to the standard process. The central electronic device 100-2 can send a command to the clothing care machine 300-1 for operating the clothing care machine 300-1 according to the standard process. The clothing care machine 300-1 may perform operations according to a control command transmitted thereto.

[0040] Alternatively, the terminal device 100-1 may identify the setting information mapped to the identification information. As described above, the terminal device 100-1 may obtain mapping information in which the identification information and the setting information are mapped to each other from the central electronic device 100-2. When the terminal device 100-1 obtains the identification information of the object, the terminal device 100-1 may identify the setting information mapped to the obtained identification information. The terminal device 100-1 may send a control command to the home appliance 300 to perform the operation of the identified setting information. For example, the terminal device 100-1 may directly send the control command to the home appliance 300, or send the control command to the home appliance 300 through the central electronic device 100-2.

[0041] Alternatively, for example, the terminal device 100 - 1 may perform a mapping process.

[0042] The terminal device 100-1 may capture an image of identification information of an object using a camera and obtain identification information from the captured image. The user may select setting information in the home appliance 300. The home appliance 300 may identify the setting information selected by the user and send the identified setting information to the central electronic device 100-2. The central electronic device 100-2 may send the obtained setting information to the terminal device 100-1. The terminal device 100-1 may obtain the setting information set by the user from the central electronic device 100-2. The terminal device 100-1 may map the setting information to the obtained identification information of the object.

[0043] Alternatively, the user can select setting information in the terminal device 100-1. First, the central electronic device 100-2 can obtain operation-related information from each of the home appliances 300. In addition, the central electronic device 100-2 can send the obtained operation-related information to the terminal device 100-1. The terminal device 100-1 can obtain the operation-related information of each of the home appliances 300 through the central electronic device 100-2. When the identification information of the object is received, the terminal device 100-1 can display the operation-related information of the home appliance 300. The user can select setting information from the displayed operation-related information. The terminal device 100-1 can map the setting information selected by the user to the obtained identification information.

[0044] After the identification information and the setting information are mapped to each other, the terminal device 100-1 can send a control command to the home appliance 300 when the identification information is obtained to perform the operation of the setting information mapped to the identification information. For example, the terminal device 100-1 can directly send the control command to the home appliance 300, or send the control command to the home appliance 300 through the central electronic device 100-2. The home appliance 300 can perform an operation according to the control command sent thereto.

[0045] The configuration of the electronic device will be described below.

[0046] Figure 2 is a block diagram illustrating a configuration of an electronic device according to one or more embodiments of the present disclosure.

[0047] refer to Figure 2 , the electronic device 100 may include a communication interface 110 and a processor 120 .

[0048] For example, the electronic device 100 may be a central electronic device.

[0049] The communication interface 110 can communicate with an external device. For example, the external device may include a terminal device and a household appliance. The communication interface 110 can send information to a terminal device or a household appliance and receive information from the terminal device or the household appliance. For example, the communication interface 110 can receive identification information of an object and setting information of a household appliance selected by a user in the terminal device 100-1 from a terminal device, and send operation-related information of the household appliance to the terminal device. In addition, the communication interface 110 can receive operation-related information from a household appliance and send a control command for executing the operation of setting information to the household appliance. For example, the communication interface 110 can communicate with an external device through at least one communication method of Wi-Fi, Wi-Fi Direct, Bluetooth, Zigbee, the third generation (3G), the third generation partnership project (3GPP), or long term evolution (LTE). The communication interface 110 can be referred to as a communication device, a communicator, a communication module, a transceiver, etc.

[0050] The electronic device 100 may include one or more processors 120. The processor 120 may control components of the electronic device 100. For example, the processor 120 may control the communication interface 110 to transmit and receive data to and from an external device.

[0051] The processor 120 may obtain identification information of the object. For example, the processor 120 may control the communication interface 110 to obtain identification information of the object from the terminal device. The processor 120 may control the communication interface 110 to receive operation-related information of the household appliance. The processor 120 may map setting information selected from the obtained operation-related information of the household appliance to the obtained identification information of the object. The processor 120 may set time information for the mapped setting information. For example, the time information may be preset time information or time information entered by the user. The time information may be expiration time information. That is, the processor 120 may delete the mapped setting information at a time point corresponding to the set time information.

[0052] The selected setting information may include at least one setting information. The processor 120 may map at least one selected setting information to identification information. The processor 120 may obtain input information related to the object. For example, the input information related to the object may include an image of the object, voice information of the user, etc. The input information related to the object may be input through a terminal device. The processor 120 may obtain the input information related to the object from the terminal device through the communication interface 110. The processor 120 may map the obtained input information related to the object to identification information. In other words, the processor 120 may map one or more setting information selected by the user and the input information related to the object to one identification information.

[0053] Alternatively, the processor 120 may obtain multiple pieces of identification information from multiple objects. For example, the processor 120 may obtain identification information of a T-shirt, identification information of a skirt, identification information of pants, and identification information of a dress. In addition, the processor 120 may obtain selected setting information. The processor 120 may map the selected setting information to the multiple pieces of identification information of the multiple objects. The selected setting information mapped to each piece of identification information of the multiple pieces of identification information of the multiple objects may be the same information. Therefore, the same setting information mapped to the multiple pieces of identification information of the multiple objects may be referred to as public information.

[0054] After mapping the setting information and the like to the identification information, when the processor 120 obtains the identification information, an operation related to the mapped setting information can be performed. For example, the processor 120 can obtain the identification information of the object. When the obtained identification information includes the mapped setting information, the processor 120 can control the communication interface 110 to send a control command for performing an operation corresponding to the mapped setting information to the household appliance. Alternatively, the processor 120 can obtain multiple identification information from multiple objects. When the processor 120 obtains multiple identification information of the multiple objects, the processor 120 can identify the public information mapped to the multiple identification information of the obtained multiple objects. When there is public information mapped to the multiple identification information, the processor 120 can output the identified public information. If there is no public information mapped to the multiple identification information, the processor 120 can output a message asking whether to register the public information.

[0055] The electronic device 100 may be a terminal device.

[0056] When the electronic device 100 is a terminal device, the communication interface 110 can send information to the central electronic device or the home appliance and receive information from the central electronic device or the home appliance. For example, the communication interface 110 can receive information related to the operation of the home appliance and setting information selected in the home appliance from the central electronic device, and send a control command for executing the operation of the setting information to the central electronic device or the home appliance.

[0057] The processor 120 may obtain identification information of the object. The processor 120 may map setting information selected from the obtained operation-related information of the home appliance to the obtained identification information of the object. The processor 120 may set time information for the mapped setting information. The processor 120 may delete the mapped setting information at a time point corresponding to the set time information.

[0058] The selected setting information may include at least one setting information. The processor 120 may map the at least one selected setting information to the identification information. Alternatively, the processor 120 may obtain input information related to the object and map the obtained input information related to the object to the identification information. That is, the processor 120 may map one or more setting information selected by the user and the input information related to the object to one identification information.

[0059] Alternatively, the processor 120 may obtain multiple pieces of identification information from multiple objects. In addition, the processor 120 may obtain the selected setting information. The processor 120 may map the selected setting information to the multiple pieces of identification information of the multiple objects. The selected setting information mapped to each piece of identification information in the multiple pieces of identification information of the multiple objects may be the same information.

[0060] After mapping the setting information and the like to the identification information, when the processor 120 obtains the identification information, operations related to the mapped setting information can be performed. For example, the processor 120 can obtain the identification information of the object. When the obtained identification information includes the setting information mapped thereto, the processor 120 can control the communication interface 110 to send a control command for performing an operation corresponding to the mapped setting information to the home appliance. The control command can be sent directly to the home appliance, or sent to the home appliance through a central electronic device. Alternatively, the processor 120 can obtain multiple identification information from multiple objects. When the processor 120 obtains multiple identification information of the multiple objects, the processor 120 can identify the public information mapped to the multiple identification information of the multiple objects obtained. When there is mapped public information, the processor 120 can output the identified public information. If there is no mapped public information, the processor 120 can output a message asking whether to register the public information.

[0061] Figure 3 is a detailed block diagram showing a configuration of an electronic device according to an embodiment of the present disclosure.

[0062] refer to Figure 3 , the electronic device 100 may include a communication interface 110, a processor 120, an input interface 130, a camera 140, a microphone 150, a speaker 160, a display 170, a memory 180, and a sensor 190. The communication interface 110 may be as described above with reference to Figure 2 described.

[0063] A control command may be received from a user through the input interface 130. For example, a control command related to the function of the camera 140 or the microphone 150, a selection command related to a home appliance or operation-related information of the home appliance, an input command related to user information, etc. may be input through the input interface 130. For example, the input interface 130 may include a keyboard, a button, a keypad, a touch pad, or a touch screen. The input interface 130 may be referred to as an input device, an inputter, an input module, etc.

[0064] The camera 140 may capture an image of identification information of an object or an image of an object. The processor 120 may recognize the object or the identification information of the object from the captured image. Alternatively, the captured image of the object may be input information related to the object. The processor 120 may map the captured image of the object to the identification information. For example, the camera 140 may include an RGB camera, a depth camera, a wide-angle camera, a telephoto camera, etc.

[0065] The sound of the surrounding environment or the user's voice can be input through the microphone 150. The processor 120 can map the input sound or voice to the identification information. Alternatively, the processor 120 can recognize the control command based on the user's input voice and perform a control operation related to the recognized control command. For example, the microphone 150 can include a general microphone, a surround microphone, a directional microphone, etc.

[0066] The speaker 160 may output a sound signal. For example, when the identification information is obtained while mapping the sound (or voice) to the identification information, the speaker 160 may output the mapped sound under the control of the processor 120. In addition, the speaker 160 may output information about a command input by a user, warning-related information, operation-related information, notification information, etc. in the form of voice or notification sound.

[0067] The display 170 may output the data processed by the processor 120 in the form of an image. The display 170 may display a user interface (UI) for mapping setting information, etc. to identification information. In addition, the display 170 may display setting information, etc. mapped to the identification information. When multiple pieces of information are mapped to the identification information, the display 170 may display the mapped multiple pieces of information. For example, the multiple pieces of information displayed on the display 170 may include an image of an object. When the image of the displayed object is selected, the processor 120 may send a command to the home appliance to perform an operation corresponding to the setting information mapped to the displayed image. Alternatively, the display 170 may display notification information, etc. For example, the display 170 may be embodied as a liquid crystal display (LCD), an organic light emitting diode (OLED), a flexible display, a touch screen, etc. When the display 170 is embodied as a touch screen, the electronic device 100 may receive a control command through the touch screen.

[0068] The memory 180 may store data, algorithms, etc. for executing the functions of the electronic device 100, and programs, commands, etc. to be executed by the electronic device 100. Alternatively, the memory 180 may store information about home appliances, operation-related information of home appliances, identification information, setting information mapped to the identification information, etc. The algorithm or data stored in the memory 180 may be loaded into the processor 120 to map the identification information and the setting information to each other under the control of the processor 120 and to perform functions related to the mapped setting information. For example, the memory 180 may be implemented as a memory type such as a read-only memory (ROM), a random access memory (RAM), a hard disk drive (HDD), a solid-state drive (SSD), or a memory card.

[0069] The sensor 190 may detect the surrounding environment of the electronic device 100. Examples of the sensor 190 may include an angle sensor, an acceleration sensor, a gravity sensor, a gyro sensor, a geomagnetic sensor, a direction sensor, an infrared sensor, an ultrasonic sensor, a time-of-flight (ToF) sensor, a light wave detection and ranging (LiDAR) sensor, a laser sensor, a motion recognition sensor, a thermal sensor, an image sensor, a tracking sensor, a proximity sensor, an illumination sensor, a voltmeter, an ammeter, a barometer, a hygrometer, a thermometer, a touch sensor, and the like.

[0070] The electronic device 100 may include all or some of the above components. The configuration of the electronic device 100 has been described above. The process of mapping the identification information and the setting information to each other and the process of operating the home appliance based on the mapped setting information performed by the electronic device 100 will be described below. As described above, the electronic device 100 may be a terminal device or a central electronic device. The following will mainly describe an example in which the electronic device 100 is a terminal device.

[0071] Figures 4a to 4c is a diagram for describing a process of mapping identification information and setting information to each other according to one or more embodiments of the present disclosure.

[0072] Figure 4a An electronic device 100 is shown for capturing identification information 1 displayed on an object. The object may be clothing, food, etc. The identification information may include a barcode, a quick response (QR) code, PDF417, CODE 16K, data matrix, maxi code, color code, three-dimensional (3D) code, etc., and is displayed on an area of ​​the object.

[0073] The electronic device 100 may use a camera to capture the identification information 1. Alternatively, the electronic device 100 may include a scanner and scan the identification information 1 using the scanner. The electronic device 100 may obtain the identification information 1 using a camera or a scanner. When obtaining the identification information 1, the electronic device 100 may receive information to be mapped to the identification information 1. The information to be mapped to the identification information 1 may be obtained from the home appliance 300 or the electronic device 100.

[0074] Figure 4b An embodiment of obtaining information to be mapped to identification information 1 from a home appliance 300 is shown. The user can capture identification information 1 of the clothes 10 as an object, and set (or select) operation-related information of the home appliance 300 in the home appliance 300. For example, when the home appliance 300 is a clothing care machine, the user can select an item for cleaning the clothes 10. The user can set a cleaning process (e.g., a standard process), an operation time, etc. The home appliance 300 can send the operation information set by the user to the electronic device 100. The home appliance 300 can directly send the setting information to the electronic device 100, or send the setting information to the electronic device 100 through a central electronic device. When obtaining the setting information, the electronic device 100 can map the setting information to the identification information from the home appliance 300.

[0075] Alternatively, the setting information may be input from the electronic device 100 .

[0076] Figure 4c It is shown that the electronic device 100 displays operation-related information of the home appliance 300. The central electronic device can receive operation-related information from each home appliance 300. For example, the central electronic device can receive information such as a cleaning process and an operation time from a clothing care machine. The central electronic device can send the received operation-related information of each home appliance 300 to the electronic device 100. When the electronic device 100 obtains the identification information 1 of the clothing 10, the electronic device 100 can display a UI for setting the operation of the home appliance 300.

[0077] For example, the electronic device 100 may display a list of home appliances 300, and when a home appliance 300 is selected from the list, operation-related information of the home appliance 300 is displayed. For example, the electronic device 100 may display a list of refrigerators, electric stoves, microwave ovens, washing machines, dryers, clothing care machines, etc., and when a clothing care machine is selected, a UI for setting a cleaning process, an operation time, etc. is displayed.

[0078] Alternatively, the electronic device 100 may display a list of home appliances 300 related to the object corresponding to the obtained identification information, and display operation-related information of the home appliance 300 when the home appliance 300 is selected from the list. For example, when the object corresponding to the obtained identification information is clothing 10, the electronic device 100 may display a list of washing machines, dryers, and clothing care machines related to the clothing 10, and display a UI for setting a cleaning process, operation time, etc. when the clothing care machine is selected. The electronic device 100 may map the operation information selected (or set) by the user from the displayed operation-related information of the home appliance 300 to the obtained identification information.

[0079] For example, the electronic device 100 may be a central electronic device, the object may be clothing 10, and the home appliance 300 may be a clothing care machine.

[0080] When the electronic device 100 is a central electronic device, the identification information 1 of the clothing 10 can be obtained from the terminal device. When the user selects to perform a 39-minute standard cleaning process on the clothing care machine, the clothing care machine can send the setting information selected by the user to the electronic device 100. The electronic device 100 can map the selected setting information obtained from the clothing care machine to the identification information 1 of the clothing 10.

[0081] Alternatively, the electronic device 100 may receive operation-related information from various types of household appliances 300 including a clothing care machine, and send the received operation-related information to the terminal device. When the identification information 1 of the clothing 10 is obtained, the terminal device may display a UI for setting the operation of the household appliance 300 based on the received operation-related information of the household appliance 300. When the user selects the clothing care machine to operate for 39 minutes according to the standard cleaning process through the terminal device, the terminal device may send the setting information of the clothing care machine selected by the user to the electronic device 100. The electronic device 100 may obtain the setting information selected by the user and map the setting information to the identification information 1 of the clothing 10.

[0082] Figure 5a to Figure 5c is a diagram illustrating a process of obtaining identification information and operating a home appliance according to one or more embodiments of the present disclosure.

[0083] Figure 5a The electronic device 100 is shown for capturing identification information 2 displayed on a frozen pizza 20 as an object. Figures 4a to 4c As described above, the electronic device 100 can map the 3-minute thawing process to be performed by the microwave oven to the identification information 2 of the frozen pizza 20. After mapping the identification information 2 with the setting information of the microwave oven, the electronic device 100 can obtain the identification information 2 of the frozen pizza 20 as an object, such as Figure 5aAs described above, the electronic device 100 may capture the identification information 2 using a camera and scan the identification information 2 using a scanner.

[0084] Figure 5b and Figure 5c One or more embodiments of performing operations corresponding to the mapped setting information are shown. One or more pieces of information may be mapped to identification information 2. Figure 5b and Figure 5c An example of mapping one piece of information to identification information 2 is described, and one or more embodiments of mapping multiple pieces of information to identification information will be described below.

[0085] When the identification information 2 is obtained, the electronic device 100 can identify the setting information mapped thereto. In addition, the electronic device 100 can send a command to the home appliance 300 for executing an operation corresponding to the mapped setting information. For example, when the identification information 2 of the frozen pizza 20 is obtained, the electronic device 100 can send a control command to the microwave oven to execute a 3-minute thawing process mapped to the identification information 2 of the frozen pizza 20. The electronic device 100 can send the control command to the microwave oven and display the following information: Figure 5b The microwave oven can receive the control command from the terminal device and perform the operation of the 3-minute defrosting process, such as Figure 5c shown.

[0086] For example, the electronic device 100 may be a central electronic device, the object may be a frozen pizza 20, and the home appliance 300 may be a microwave oven.

[0087] When the electronic device 100 is a central electronic device, the identification information 2 of the frozen pizza 20 can be obtained from the terminal device. The electronic device 100 can identify the setting information of the 3-minute thawing process of the microwave oven mapped to the identification information 2 of the frozen pizza 20. The electronic device 100 can send a control command for executing the 3-minute thawing process to the microwave oven. In addition, the electronic device 100 can send status notification information to the terminal device. The terminal device can display the status notification message, and the microwave oven can perform the operation of the 3-minute thawing process.

[0088] An example in which one piece of information is mapped to one piece of identification information has been described above. An example in which one piece of information is mapped to a plurality of pieces of identification information or a plurality of pieces of information is mapped to one piece of identification information will be described.

[0089] Figure 6 is a diagram illustrating a process of mapping identification information and setting information of multiple objects to each other according to one or more embodiments of the present disclosure.

[0090] refer to Figure 6, the electronic device 100 may obtain identification information of a plurality of objects. For example, the electronic device 100 may capture (or scan) identification information 1-1 of the shirt 10-1, capture identification information 1-2 of the dress 10-2, and capture identification information 1-3 of the skirt 10-3. After obtaining identification information 1-1 of the shirt 10-1, identification information 1-2 of the dress 10-2, and identification information 1-3 of the skirt 10-3, the electronic device 100 may receive information to be mapped to each of the identification information 1-1, 1-2, and 1-3. The information to be mapped may be obtained by the electronic device 100 from the home appliance 300.

[0091] For example, the home appliance 300 may be a clothing care machine, and the user may select items for cleaning the shirt 10-1, the dress 10-2, and the skirt 10-3 in the clothing care machine. The user may set a standard course, an operation time, and the like. The clothing care machine may transmit operation information set by the user to the electronic device 100. When obtaining the setting information from the clothing care machine, the electronic device 100 may map the information set in the clothing care machine (e.g., the standard course and the operation time) to the identification information 1-1 of the shirt 10-1, the identification information 1-2 of the dress 10-2, and the identification information 1-3 of the skirt 10-3.

[0092] Alternatively, the user may input setting information through the electronic device 100. The electronic device 100 may obtain operation-related information of the home appliance 300 through the central electronic device. When the electronic device 100 obtains the identification information 1-1 of the shirt 10-1, the identification information 1-2 of the dress 10-2, and the identification information 1-3 of the skirt 10-3, the electronic device 100 may display a UI for setting the operation of the home appliance 300. The electronic device 100 may be provided with information (i.e., standard process, operation time, etc.) of the clothing care machine selected through the displayed UI. The electronic device 100 may map information such as the standard process and operation time of the clothing care machine selected by the user from the displayed operation-related information of the home appliance 300 to the identification information 1-1 of the shirt 10-1, the identification information 1-2 of the dress 10-2, and the identification information 1-3 of the skirt 10-3. The information mapped to each of the identification information 1-1 of the shirt 10-1, the identification information 1-2 of the dress 10-2, and the identification information 1-3 of the skirt 10-3 may be information such as the standard process and operation time of the same clothing care machine. The same information mapped to the pieces of identification information of the multiple objects may be referred to as common information.

[0093] The electronic device 100 may be a central electronic device. When the electronic device 100 is a central electronic device, identification information 1-1 of the shirt 10-1, identification information 1-2 of the dress 10-2, and identification information 1-3 of the skirt 10-3 may be obtained from the terminal device. In addition, the electronic device 100 may obtain information such as a standard process and operation time of the clothing care machine set by the user from the clothing care machine or the terminal device. The electronic device 100 may map the obtained information such as a standard process and operation time of the clothing care machine to the identification information 1-1 of the shirt 10-1, the identification information 1-2 of the dress 10-2, and the identification information 1-3 of the skirt 10-3.

[0094] It has been described above that the information mapped to the identification information is an example of setting information related to the operation of the home appliance 300. The information mapped to the identification information may be input information (eg, custom information) input by the user.

[0095] Figure 7 is a diagram illustrating a process of mapping voice information to identification information according to one or more embodiments of the present disclosure.

[0096] refer to Figure 7 , the electronic device 100 can obtain identification information 1-1 of the object. After obtaining the identification information 1-1, the user can input personal input information instead of setting information of the home appliance 300. For example, the input information may include information such as text, image, or sound (including voice).

[0097] For example, the electronic device 100 may obtain the identification information 1-1 of the shirt 10-1. The electronic device 100 may perform a voice input function according to the user's selection and receive a voice input from the user. For example, a voice sentence "a gift from my mother on my 20th birthday" may be input from the user to the electronic device 100. When the input of the voice sentence is completed, the electronic device 100 may map the information of the input voice to the identification information 1-1 of the shirt 10-1.

[0098] Alternatively, the electronic device 100 may be a central electronic device. In this case, the electronic device 100 may obtain the identification information 1-1 of the shirt 10-1 from the terminal device and obtain the information of the input voice. The electronic device 100 may map the obtained voice information to the identification information 1-1.

[0099] When the electronic device 100 maps various types of information to identification information of an object and obtains the identification information of the object, the electronic device 100 may perform an operation corresponding to the mapped information. The identification information of the object may include one or more pieces of identification information, and one or more pieces of information may be provided thereto.

[0100] Figure 8 is a diagram illustrating a process of operating a home appliance using setting information according to one or more embodiments of the present disclosure.

[0101] refer to Figure 8 , the electronic device 100 can obtain the identification information of the object in a state where the setting information is mapped to the identification information of the object. For example, the electronic device 100 can obtain the identification information 1-1 of the shirt 10-1, the identification information 1-2 of the dress 10-2, and the identification information 1-3 of the skirt 10-3. Figure 6 As described above, the electronic device 100 may map the same setting information to the identification information 1-1 of the shirt 10-1, the identification information 1-2 of the dress 10-2, and the identification information 1-3 of the skirt 10-3. Therefore, after mapping the same setting information to the identification information 1-1, 1-2, and 1-3, the electronic device 100 may output the same information mapped to the identification information 1-1, 1-2, and 1-3 when obtaining the identification information 1-1 of the shirt 10-1, the identification information 1-2 of the dress 10-2, and the identification information 1-3 of the skirt 10-3. When multiple pieces of the same information are mapped to the identification information 1-1, 1-2, and 1-3, the electronic device 100 may output the multiple pieces of the same information. For example, Figure 8 As shown, the electronic device 100 can display the same information mapped to the identification information 1-1, 1-2 and 1-3 in the form of a list. The electronic device 100 can classify and display multiple pieces of information according to registration time, usage history (e.g., usage frequency, recent usage history, etc.) and items (e.g., type of home appliance, type of input information, etc.).

[0102] Information may be selected by the user from the displayed multiple pieces of information. For example, when recording 4 is selected, the electronic device 100 may output the sound information of recording 4. When a washing process is selected, the electronic device 100 may send a control command related to the operation corresponding to the selected washing process to the home appliance 300, i.e., the washing machine. When receiving the control command, the washing machine may perform a washing operation according to the information about the washing process.

[0103] For example, the electronic device 100 may be a central electronic device. In this case, the electronic device 100 may obtain identification information, setting information, etc. from a terminal device or a household appliance, and map the setting information to the identification information. Afterwards, when the terminal device obtains the identification information 1-1 of the shirt 10-1, the identification information 1-2 of the dress 10-2, and the identification information 1-3 of the skirt 10-3, the electronic device 100 may obtain the identification information 1-1, 1-2, and 1-3 from the terminal device. The electronic device 100 may identify the same information mapped to the identification information 1-1, 1-2, and 1-3 based on the mapped information. When multiple pieces of the same information are mapped to the identification information, the electronic device 100 may output the multiple pieces of the same information. For example, the electronic device 100 may send the multiple pieces of information to the terminal device.

[0104] The terminal device may classify and display the multiple pieces of information according to a specific standard, and send the information selected from the multiple pieces of information to the electronic device 100. The electronic device 100 may send the selected information to the terminal device or the household appliance. For example, when recording 4 is selected, the electronic device 100 may send the sound information of recording 4 to the terminal device. When a washing process is selected, the electronic device 100 may send a control command related to the operation corresponding to the selected washing process to the washing machine. In addition, the electronic device 100 may send status-related information to the terminal device. The terminal device may display status-related information.

[0105] An example has been described above in which a plurality of pieces of identification information are obtained simultaneously or sequentially, and the same setting information mapped to the plurality of pieces of identification information is common information. However, common information may be mapped to the plurality of pieces of identification information separately.

[0106] Figures 9a to 9c is a diagram illustrating common information according to one or more embodiments of the present disclosure.

[0107] Figure 9a The setting information mapped to the identification information 1 - 1 of the shirt 10 - 1 is shown. Figure 9b The setting information mapped to the identification information 1 - 2 of the dress 10 - 2 is shown. Fig.9c The setting information of the identification information 1-3 mapped to the skirt 10-3 is shown. Figures 9a to 9c .

[0108] For example, "washing machine: standard" (7) and "clothing care machine: women's shirt process" can be mapped as setting information to identification information 1-1 of shirt 10-1. "Drying machine: synthetic fiber", "washing machine: standard" (7) and "clothing care machine: dress process" can be mapped as setting information to identification information 1-2 of dress 10-2. "Clothing care machine: skirt process", "dryer: standard" and "washing machine: standard" (7) can be mapped as setting information to identification information 1-3 of skirt 10-3. The setting information mapped to identification information 1-1 of object 10-1, the setting information mapped to identification information 1-2 of object 10-2, and the setting information mapped to identification information 1-3 of object 10-3 may be information registered at different times. Alternatively, the mapped setting information may be information registered at the same time.

[0109] In this case, as referenced above Figure 8 As described above, the electronic device 100 can simultaneously or sequentially obtain the identification information 1-1 of the shirt 10-1, the identification information 1-2 of the dress 10-2, and the identification information 1-3 of the skirt 10-3. The electronic device 100 can identify the setting information mapped to each of the identification information 1-1, 1-2, and 1-3. The electronic device 100 can identify that the setting information "Washing Machine: Standard" (7) in the mapped setting information is the same information mapped to the identification information 1-1, 1-2, and 1-3. The electronic device 100 can output the setting information "Washing Machine: Standard" (7), which is the same information mapped to the identification information 1-1, 1-2, and 1-3. For example, the electronic device 100 can display the setting information "Washing Machine: Standard" (7) and send a control command to the washing machine for operating the washing machine according to the standard washing process.

[0110] When there are multiple identical pieces of information, the electronic device 100 can classify and output the multiple identical pieces of information according to a specific standard, as shown in the above reference. Figure 8 When one of the plurality of identical pieces of information is selected, the electronic device 100 may perform an operation related to the selected piece of information.

[0111] For example, the electronic device 100 may be a central electronic device. In this case, the electronic device 100 may obtain identification information 1-1, 1-2, and 1-3 of a plurality of objects from a terminal device, and identify the same information mapped to the identification information 1-1, 1-2, and 1-3. In addition, the electronic device 100 may output the identified same information. For example, the electronic device 100 may send the identified same information to the terminal device, and the terminal device may display the received same information.

[0112] Although an example of obtaining identification information of an object again and performing an operation corresponding to the setting information has been described above, the electronic device 100 may perform an operation corresponding to the setting information based on an image.

[0113] Fig.10 is a diagram illustrating a process of operating a home appliance using a captured image according to one or more embodiments of the present disclosure. Fig.11 is a diagram showing a process of operating a home appliance by photographing an object according to one or more embodiments of the present disclosure. Fig.10 and Fig.11 .

[0114] As described above, the information input by the user may be mapped to the identification information. The information input by the user may include information such as text, images, and sounds (including voice). The electronic device 100 may obtain the identification information during the mapping of the identification information and the setting information. In addition, the electronic device 100 may obtain an image obtained by photographing the object. The electronic device 100 may obtain setting information related to the operation of the home appliance 300. The electronic device 100 may map the image of the object and the setting information related to the operation of the home appliance 300 to the identification information. For example, when the setting information and the image are obtained, the electronic device 100 may output a message asking whether to map both the image and the setting information to the identification information. When the user allows both the image and the setting information to be mapped to the identification information, the electronic device 100 may map the image and the setting information to the identification information. When the user refuses to map both the image and the setting information to the identification information, the electronic device 100 may map the image as the first information to the identification information, and map the setting information as the second information to the identification information.

[0115] After mapping the information to the identification information, when the electronic device 100 obtains the identification information of the object, the electronic device 100 may output one or more pieces of information mapped to the obtained identification information. Fig.10 As shown, when the information mapped to the identification information includes an image, the electronic device 100 may output the image as the image item 3. When the user selects the image item 3, the electronic device 100 may perform an operation related to (mapped to) the selected image item 3.

[0116] When only the image is mapped to the identification information, the electronic device 100 can enlarge and output the selected image. When both the image and the setting information of the home appliance 300 are mapped to the identification information, the electronic device 100 can send a command to the home appliance 300 to perform an operation corresponding to the mapped setting information.

[0117] For example, the electronic device 100 may obtain identification information of clothes, and obtain an image of the clothes and washing setting information of a washing machine. The electronic device 100 may map both the image of the clothes and the washing setting information to the identification information according to the user's selection. After mapping the information, when the electronic device 100 obtains the identification information of the clothes, the electronic device 100 may display an image item 3 of the clothes. When the image item 3 of the clothes is selected, the electronic device 100 may send a command to the washing machine to perform an operation corresponding to the washing machine setting information.

[0118] When both the image of the object and the setting information of the home appliance 300 are mapped to the identification information during mapping, the electronic device 100 can photograph the object and operate the home appliance 300 .

[0119] For example, Fig.11 As shown, the electronic device 100 can obtain an image of the clothes 10 by photographing the clothes 10. The electronic device 100 can search for the mapped clothes image information that matches the obtained image of the clothes 10. When there is no mapped clothes image information that matches the obtained image of the clothes 10, the electronic device 100 can perform a general camera function. When there is clothes image information that matches the obtained image of the clothes 10, the electronic device 100 can identify the washing setting information of the washing machine mapped together with the image of the clothes 10. When the washing setting information of the washing machine is identified, the electronic device 100 can send a command to the washing machine for performing an operation related to the identified washing setting information.

[0120] As described above, for example, the electronic device 100 may be a central electronic device, and the central electronic device may obtain identification information, images, setting information, etc. from a terminal device or a home appliance and perform a process similar to the above process.

[0121] An example of mapping the setting information of a home appliance and / or the information input by a user to the identification information of an object and performing a corresponding operation by recognizing the identification information has been described above. An example of the electronic device 100 estimating the composition of an object using a neural network model will be described below.

[0122] Figures 12a to 12c is a diagram illustrating operation-related information of a home appliance according to one or more embodiments of the present disclosure. Fig.13a and Fig.13b is a diagram showing a process of estimating the composition of clothing and providing notification to a user according to one or more embodiments of the present disclosure; Figures 12a to 13b .

[0123] like Figures 12a to 12cAs shown, the home appliances may each include operation-related information. The home appliances related to clothes may include a clothing care machine, a washing machine, a dryer, etc. The clothing care machine may include a blouse course, a wool / knit course, etc. as a cleaning course, the washing machine may include a standard course, a shirt course, etc. as a washing course, and the dryer may include a shirt course, a towel course, etc. as a drying course. The home appliances related to clothes may include various types of operation information to perform optimal operations according to the composition (or material or fabric) of the clothes without damaging the clothes.

[0124] Therefore, the operation information of the household appliance related to clothes may include information about the composition of the clothes. The electronic device 100 may train a neural network model (or artificial intelligence model) 70 based on the operation information mapped to the identification information, and the neural network model (or artificial intelligence model) 70 recognizes information about the composition of the object. Alternatively, the operation information of the household appliance may not include sufficient information about the composition of the object. The electronic device 100 may collect information about the composition of various types of objects related to the operation information mapped to the identification information, and train the neural network model 70 based on the operation information and the collected information about the composition of the object.

[0125] For example, Figures 12a to 12c As shown, when the operation information of each household appliance includes information about the composition of clothes, the electronic device 100 can train a neural network model 70 for identifying information about the composition of clothes based on the operation information. When the operation information of each household appliance does not include information about the material of the object (the composition of clothes), the electronic device 100 can collect information about the composition of clothes related to the operation information of each household appliance from the manufacturer's homepage of each household appliance, a website related to each household appliance, etc., and train a neural network model 70 for identifying information about the composition of clothes (object) based on the operation information and the collected information about the composition of clothes. The electronic device 100 can use the neural network model 70 trained based on the identification information of the object to identify information about the composition of the object.

[0126] For example, Fig.13aAs shown, the first information, the second information, and the third information mapped to the identification information of the garment A may be clothing care machine: blouse process, washing machine: wool / underwear process, and dryer: wool process, respectively. The electronic device 100 may estimate that the composition of the garment A may be cotton, polyester, rayon, linen, or wool by using the neural network model 70 trained based on the first information mapped to the identification information of the garment A. The electronic device 100 may estimate that the composition of the garment A may be wool by using the neural network model 70 trained based on the mapped second information. The electronic device 100 may estimate that the composition of the garment A may be wool by using the neural network model 70 trained based on the mapped third information. Therefore, the electronic device 100 may identify that the composition of the garment A is wool by using the trained neural network model 70. For ease of description, the above process has been divided into several parts to describe it, and therefore, these parts of the above process may be performed simultaneously (or in parallel or in combination) by the trained neural network model 70.

[0127] As described above, the electronic device 100 can simultaneously or sequentially obtain multiple pieces of identification information of multiple objects, and map the setting information of the home appliance as common information to the obtained multiple pieces of identification information. In this case, the electronic device 100 can analyze the components of each object by using the trained neural network model 70, and output a notification message to the user when there is a problem with the setting information to be mapped to the components of each object analyzed.

[0128] For example, Fig.13b As shown, the electronic device 100 can obtain identification information of the first garment (garment 1), identification information of the second garment (garment 2), identification information of the third garment (garment 3), and identification information of the fourth garment (garment 4), and map the standard process information of the washing machine as common information to the obtained identification information. When the identification information of each of the first to fourth garments is obtained, the electronic device 100 can identify the composition of each of the first to fourth garments by using the trained neural network model 70. For example, the electronic device 100 can identify the composition of the first garment as synthetic fiber, the composition of the second garment as cotton, the composition of the third garment as wool, and the composition of the fourth garment as cotton. In addition, the electronic device 100 can store information indicating that clothes made of wool may be damaged when washed according to the standard process of the washing machine.

[0129] That is, when the user maps the standard process information of the washing machine as public information to the identification information of the first to fourth clothes, the electronic device 100 can recognize that the composition of the third clothes is wool and cannot be washed according to the standard process of the washing machine. When it is determined that the washing setting information to be mapped is inappropriate based on the recognized composition of the clothes, the electronic device 100 can output a notification message such as Fig.13b That is, when the identification information is obtained from each of the multiple objects, the electronic device 100 can identify the component information of each of the multiple objects. In addition, the electronic device 100 can output a notification message based on the identified component information of each of the multiple objects and the setting information to be mapped.

[0130] Fig.14a and Fig.14b is a diagram illustrating a process of estimating ingredients of food and providing a notification to a user according to one or more embodiments of the present disclosure.

[0131] Users can use home appliances to make food based on various types of ingredients and recipes. For example, users can make bread based on ingredients such as flour, yeast, and red beans and a bread recipe by performing a baking process at 170°C for 5 minutes in an oven.

[0132] Some users are allergic to specific ingredients. For example, when a user is allergic to nuts, the user should not eat food containing nuts, but it is not easy to check all food ingredients. The electronic device 100 of the present disclosure can output a notification message to the user by using the trained neural network model 70 that recognizes the object ingredients.

[0133] like Fig.14a As shown, the electronic device 100 can train a neural network model 70 for identifying ingredient information of an object based on the operation information mapped to the identification information. For example, the electronic device 100 can map the setting information of an oven (e.g., 170°C, five minutes, and bake) to the identification information of food ingredients such as flour, yeast, and red beans. In addition, the electronic device 100 can train a neural network model 70 for identifying ingredient information based on the identification information of food ingredients, etc. As described above, when sufficient ingredient information about an object is not included in the operation information of a home appliance, the electronic device 100 can collect ingredient information of various types of objects related to the operation information mapped to the identification information, and train the neural network model 70 based on the operation information and the collected object ingredient information.

[0134] The electronic device 100 can recognize the composition of the object by using the trained neural network model 70. For example, Fig.14bAs shown, the electronic device 100 may obtain identification information of flour, identification information of sauce X, and identification information of powder A. The electronic device 100 may identify the ingredients of flour as wheat, the ingredients of sauce X as grapes, and the ingredients of powder A as nuts by using a neural network model trained based on the identification information. The electronic device 100 may store information indicating that nuts are allergic ingredients.

[0135] That is, when the user maps the cooking process information of the cooking home appliance as common information to the identification information of flour, sauce X, and powder A, the electronic device 100 may output a notification message that “powder A contains nuts as an allergic ingredient”.

[0136] Functions related to artificial intelligence according to the present disclosure may be performed by the processor 120 and the memory 180 of the electronic device 100 .

[0137] The processor 120 may include one or more processors. In this case, the one or more processors may include but are not limited to at least one of a central processing unit (CPU), a graphics processing unit (GPU), or a neural processing unit (NPU).

[0138] The CPU is a general-purpose processor that can perform artificial intelligence operations as well as general operations, and can efficiently execute complex programs through a multi-layer cache structure. The CPU is conducive to a serial processing method in which the previous calculation result and the next calculation result can be organically connected to each other through sequential calculations. The general-purpose processor is not limited to the above examples unless it is specified as the above-mentioned CPU.

[0139] A GPU is a processor configured to perform a large number of calculations (such as floating-point operations for graphics processing), in which cores are integrated on a large scale to perform large-scale calculations in parallel. Specifically, a GPU may have advantages over a CPU in parallel processing methods such as convolution operations. Alternatively, a GPU may be used as an auxiliary processor to supplement the functions of a CPU. A processor configured to perform a large number of calculations is not limited to the above examples unless it is specified as the above-mentioned GPU.

[0140] NPU is a processor dedicated to AI calculations using AI neural networks, and each layer of the AI ​​neural network can be implemented by hardware (such as silicon). In this case, the NPU is specially designed according to the specifications required by the company, so its degree of freedom is lower than that of the CPU or GPU, but the NPU is able to effectively handle AI operations. The NPU, which is a processor dedicated to AI intelligent operations, can be embodied in various forms, such as a tensor processing unit (TPU), an intelligent processing unit (IPU), and a visual processing unit (VPU). The AI ​​processor is not limited to the above examples unless it is specified as the above-mentioned NPU.

[0141] The one or more processors may be implemented as a system on chip (SoC). In this case, in addition to the one or more processors, the SoC may also include a memory and a network interface, such as a bus for data communication between the processor and the memory.

[0142] When the SoC included in the electronic device 100 includes a plurality of processors, the electronic device 100 may perform AI-related operations (e.g., operations related to training an AI model or interference) by using some of the plurality of processors. For example, the electronic device may perform AI-related operations by using at least one of a GPU, an NPU, a VPU, a TPU, or a hardware accelerator dedicated to AI-related operations (such as convolution operations and matrix multiplication operations) among the plurality of processors. However, the above description is only an example, and a general-purpose processor (e.g., a CPU) may be used to process AI-related operations.

[0143] In addition, the electronic device can perform AI-related operations by using multiple cores (e.g., dual cores, quad cores, etc.) included in one processor. Specifically, the electronic device can perform AI-related operations such as convolution operations and matrix multiplication operations in parallel by using multiple cores included in the processor.

[0144] The one or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in the memory. The predefined operation rules or artificial intelligence models can be formed by learning.

[0145] Here, providing an AI model or predefined operation rules by learning can be understood to mean obtaining predefined operation rules or AI corresponding to the required features by applying a learning algorithm to multiple training data. Such training can be performed by a device executing the AI ​​according to the present disclosure, or by a separate server and / or system.

[0146] The artificial intelligence model may include multiple neural network layers. At least one weight value is assigned to at least one layer, and the operation of the layer is performed by the operation of the previous layer and the result of at least one defined operation. Examples of neural networks disclosed herein include convolutional neural networks (CNNs), deep neural networks (DNNs), recurrent neural networks (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), deep Q networks, and Transformers, but are not limited thereto unless otherwise specified.

[0147] A learning algorithm is a method of training a target device (e.g., a robot) using multiple pieces of training data so that the target device can make decisions or predictions on its own. Examples of learning algorithms may include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited thereto unless otherwise specified.

[0148] An example of mapping identification information and setting information to each other and performing an operation corresponding to the mapped setting information has been described above. A control method of an electronic device will be described below.

[0149] Fig.15 is a flowchart of a control method of an electronic device according to one or more embodiments of the present disclosure.

[0150] refer to Fig.15 , the electronic device may obtain identification information of an object (S1510). Examples of the object may include clothes, food, etc. The identification information may include a barcode, a quick response (QR) code, PDF417, CODE 16K, a data matrix, a maxi code, a color code, a 3D code, etc., and is displayed on an area of ​​the object. The electronic device may obtain identification information of one object, and may sequentially obtain multiple pieces of identification information of multiple objects.

[0151] The electronic device may receive operation-related information of the home appliance (S1520). For example, the operation-related information of the home appliance may include setting information, process information, time information, etc. related to the operation of the home appliance. Alternatively, the operation-related information may include information related to the composition of the object.

[0152] The electronic device may map the selected setting information to the obtained identification information of the object (S1530). The electronic device may map the setting information selected by the user from the operation-related information of the home appliance to the identification information. For example, when the object is clothes, the electronic device may map the standard process setting information of the washing machine to the identification information of the clothes. When the object is frozen food, the electronic device may map the thawing process setting information of the microwave oven to the identification information of the frozen food.

[0153] One or more pieces of setting information may be selected. The electronic device may map the selected one or more pieces of setting information to identification information. Alternatively, the electronic device may obtain information input by the user after obtaining the identification information. The electronic device may map the obtained input information to the identification information. For example, the input information obtained from the user may include text, images, sound information, etc. When the electronic device obtains the identification information of each of the multiple objects, the selected setting information may be mapped as common information to the identification information of each of the multiple objects.

[0154] After mapping the setting information and the like to the identification information, the electronic device can obtain the identification information of the object. When the identification information of the object is obtained after mapping the setting information to the identification information, the electronic device can send a command to the household appliance for performing an operation corresponding to the mapped setting information. After mapping multiple pieces of setting information to the identification information, the electronic device can classify and output the mapped multiple pieces of setting information. For example, the electronic device can classify and output the multiple pieces of setting information according to the registration time, usage history (e.g., usage frequency, recent usage record, etc.) and items (e.g., type of household appliance, type of input information, etc.). When the electronic device includes a display, the electronic device can display the multiple pieces of setting information on the display.

[0155] When the identification information of multiple objects is obtained, the electronic device can identify the common information mapped to the identification information of the multiple objects and output the identified common information. When multiple pieces of common information are identified, the electronic device can classify and output the multiple pieces of common information. When no common information is mapped to the identification information of multiple objects, the electronic device can output a message asking whether to register the common information.

[0156] The electronic device may map both the image of the object and the setting information of the home appliance to the identification information, and transmit a command for performing an operation related to the mapped setting information of the home appliance to the home appliance when the image of the object is selected.

[0157] In addition, the electronic device may train a neural network model for identifying object components based on identification information, setting information, or collected relevant information. The electronic device may identify the components of an object by using a neural network model trained based on identification information or setting information mapped to the identification information. When identification information of multiple objects is obtained, the electronic device may identify the components of each of the multiple objects by using a trained neural network model. The electronic device may output a notification message to the user based on the identified object components.

[0158] Fig.16 is a flowchart of a process of registering setting information according to one or more embodiments of the present disclosure.

[0159] refer to Fig.16 , the electronic device may receive identification information (S1610). For example, the electronic device may scan the identification information using a scanner. Alternatively, the electronic device may capture the identification information using a camera.

[0160] The electronic device may determine whether the input of the identification information is completed (S1620). When the input of the identification information is not completed, the electronic device may receive another piece of identification information (S1610). When the input of the identification information is completed, the electronic device may receive setting information to be registered (S1630). For example, the setting information may be operation-related information of the electronic device. Alternatively, the setting information may be information input by a user. The information input by the user may include text, images, sound information, etc.

[0161] The electronic device may register the setting information (S1640). For example, the electronic device may map the received setting information to identification information. The process in which the electronic device maps the setting information to the identification information may be referred to as a registration process.

[0162] The electronic device may register a list of identification information in a database (or memory) and add the setting information to the list of setting information (S1650). For example, the identification information may include a barcode, a QR code, etc. The electronic device may store the identification information of the object in the database. In addition, the electronic device may store the setting information mapped to the identification information in the list of setting information. The electronic device may store mapping information of the identification information and the setting information.

[0163] Through the above-described process, the electronic device can register the setting information by mapping the setting information to the identification information, and manage the identification information and the setting information.

[0164] Fig.17 and Fig.18 is a flowchart of a process of operating a home appliance according to setting information according to one or more embodiments of the present disclosure. Fig.17 and Fig.18 . Fig.17 and Fig.18 is a flowchart of a process performed by an electronic device after registering setting information by mapping the setting information to identification information.

[0165] refer to Fig.17 and Fig.18 , the electronic device may obtain identification information (S1710). For example, the electronic device may scan the identification information using a scanner. Alternatively, the electronic device may capture the identification information using a camera.

[0166] The electronic device may determine whether the input of the identification information is finished (S1720). When the input of the identification information is not finished, the electronic device may receive another piece of identification information (S1710). When the input of the identification information is finished, the electronic device may extract the setting information mapped to the identification information. In this case, when multiple pieces of identification information are received, the electronic device may extract the common information included in the multiple pieces of identification information (S1730). For example, the common information may be understood as the same setting information mapped to the multiple pieces of identification information.

[0167] The electronic device may determine whether the information list is empty (S1740). That is, the electronic device may determine whether there is public information mapped to the multiple pieces of identification information. When the information list includes information (when public information exists), the electronic device may display the public information. When multiple pieces of public information exist, the electronic device may display a list of public information. Next, the electronic device may execute the setting information according to the user's selection (S1750). That is, the electronic device may send a command to the home appliance for executing an operation corresponding to the selected setting information.

[0168] When the information list does not include information (when there is no public information), the electronic device may check whether to register the setting information (S1810). For example, the electronic device may output a message asking whether to register the setting information. The electronic device may determine whether to register the setting information additionally (S1820). That is, when the user refuses to register the setting information, the electronic device may end the registration process. When the user agrees to register the setting information, the electronic device may receive the setting information to be registered (S1830). For example, the setting information may be operation-related information of the electronic device. Alternatively, the setting information may be information input by the user. The information input by the user may include text, images, sound information, etc.

[0169] The electronic device may register the setting information (S1840). Thereafter, the electronic device may register a list of identification information in a database (or memory) and add the setting information to the list of setting information (S1850). The registration process of the setting information may be the same as that described above with reference to Fig.16 The process described is essentially the same.

[0170] The effects of the present disclosure are not limited to the above-described effects, and other effects not described herein will be apparent to those of ordinary skill in the art from the above description.

[0171] The control method of the electronic device according to one or more embodiments described above may be provided by a computer program product. The computer program product may be a software (S / W) program, or may include a non-transitory computer-readable medium storing the S / W program.

[0172] A non-transitory computer-readable medium should be understood as a medium that stores data semi-permanently and can be read by a machine, rather than a medium that stores data for a short period of time (such as a register, cache, or memory). Specifically, the various applications or programs described above can be provided by being stored in a non-transitory computer-readable recording medium (such as a CD, DVD, hard disk, Blu-ray disc, USB, memory card, ROM, etc.).

[0173] Although the embodiments of the present disclosure have been illustrated and described herein, the present disclosure is not limited thereto, and various modifications may be made thereto by those skilled in the art without departing from the gist of the present disclosure as claimed in the appended claims. These modifications should not be understood to depart from the scope and spirit of the present disclosure.

Claims

1. An electronic device, comprising: Communication interface; as well as At least one processor configured to: Get the object's identification information. controlling the communication interface to obtain operation-related information of the household appliance, and Setting information selected from the operation-related information of the home appliance is mapped to the identification information of the object.

2. The electronic device according to claim 1 in, The selected setting information includes at least one setting information, and Wherein, the at least one processor is further configured to: obtaining input information associated with the object, and The input information related to the object and the at least one setting information are mapped to identification information of the object.

3. The electronic device according to claim 1, wherein: The at least one processor is further configured to obtain a plurality of pieces of identification information of a plurality of objects including the object, and map the selected setting information as common information to the plurality of pieces of identification information of the plurality of objects.

4. The electronic device according to claim 3, wherein: The at least one processor is further configured to identify common information of the plurality of pieces of identification information mapped to the plurality of objects, and output the identified common information.

5. The electronic device according to claim 4, wherein: The at least one processor is further configured to output a message asking whether to register public information based on the absence of public information mapped to the pieces of identification information of the plurality of objects.

6. The electronic device according to claim 1, wherein: The at least one processor is further configured to: based on the identification information including the mapped setting information, control the communication interface to send a command for executing an operation corresponding to the mapped setting information to the home appliance.

7. The electronic device according to claim 2, further comprising a display, in, The at least one processor is further configured to control the display to display the mapped setting information and the mapped input information.

8. The electronic device according to claim 7 in, The input information related to the object includes an image of the object, and Wherein, the at least one processor is further configured to: Based on selecting the image of the object from the mapped input information and the mapped setting information, the communication interface is controlled to transmit a command for performing an operation corresponding to the mapped setting information to the home appliance.

9. The electronic device according to claim 2, further comprising a camera, in, The at least one processor is further configured to: control the camera to obtain identification information of the object.

10. The electronic device according to claim 9 in, The input information related to the object includes a first image of the object, and Wherein, the at least one processor is further configured to: controlling the camera to capture a second image of the object, and Based on the matching of the second image with the first image, the communication interface is controlled to send a command for performing an operation corresponding to the mapped setting information.

11. The electronic device according to claim 2, further comprising a display, in, The at least one processor is further configured to control the display to display the mapped setting information and the mapped input information.

12. The electronic device according to claim 1, wherein: The at least one processor is further configured to delete the mapped setting information a predetermined amount of time after mapping the mapped setting information to the identification information.

13. The electronic device according to claim 2 in, The operation-related information of the household appliance includes ingredient information, and Wherein, the at least one processor is further configured to: train a neural network model to recognize the ingredient information based on the operation-related information of the household appliance and the mapped setting information and the mapped input information.

14. The electronic device according to claim 13, wherein: The at least one processor is further configured to: identify composition information of the object using a trained neural network model based on the identification information of the object.

15. A method for controlling an electronic device, the method comprising: Obtain identification information of the object; Receive information related to the operation of household appliances; as well as Setting information selected from the operation-related information of the home appliance is mapped to the identification information of the object.

Images