Electronic device and control method thereof

Abstract

This electronic device comprises: a microphone for receiving noise generated by an external home appliance; a camera for acquiring an image of the external home appliance; a memory for storing a virtual image of the external home appliance; a display; and a processor for controlling the display such that the virtual image of the external home appliance is output on the display, controlling, on the basis of the virtual image of the external home appliance, which is selected by a user, and the image of the external home appliance, which is acquired by the camera, the display such that a noise measurement start screen for measuring noise of the external home appliance is output on the display, and measuring, on the basis that a noise measurement start command is input by the user, the external home appliance noise, which is input into the microphone.

Description

Electronic devices and methods for controlling electronic devices

[0001] The disclosed invention relates to an electronic device for measuring noise generated from a home appliance.

[0002] Homes use a variety of home appliances, including refrigerators, air conditioners, clothes dryers, and display devices. These various appliances can generate noise during operation.

[0003] In order to determine whether there is a problem with the operation of a home appliance, it is necessary to measure the noise generated during the operation of the home appliance, and the noise generated during the operation of the home appliance can be measured using various electronic devices.

[0004] Ordinary users of home appliances transmit noise measured during the operation of the home appliance to experts in home appliances via servers, etc., and the experts in home appliances analyze the noise and inform the ordinary users of the home appliances of the results of their judgment on whether the home appliance is operating abnormally.

[0005] However, while experts in home appliances know the appropriate noise measurement locations to measure the noise generated during the operation of home appliances, general users of home appliances often do not know the appropriate noise measurement locations, making it difficult to accurately determine whether there is a problem with the operation of home appliances.

[0006] The present disclosure can provide an electronic device and a control method for an electrical device capable of measuring noise generated from an external home appliance at an appropriate noise measurement location.

[0007] An electronic device according to the invention may include a microphone for receiving noise generated from an external home appliance; a camera for acquiring an image of the external home appliance; a memory for storing a virtual image of the external home appliance; a display; and a processor for controlling the display so that the virtual image of the external home appliance is output to the display, and for controlling the display so that a noise measurement start screen for measuring noise of the external home appliance is output to the display based on a virtual image of the external home appliance selected by a user and an image of the external home appliance acquired by the camera, and for measuring noise of the external home appliance input to the microphone based on a noise measurement start command input by the user.

[0008] A method for controlling an electronic device according to the invention comprises: a microphone for receiving noise generated from an external home appliance; a camera for acquiring an image of the external home appliance; a memory for storing a virtual image of the external home appliance; and a display; wherein the method comprises: controlling the display so that the virtual image of the external home appliance is output to the display; controlling the display so that a noise measurement start screen for measuring noise of the external home appliance is output to the display based on a virtual image of the external home appliance selected by a user and an image of the external home appliance acquired by the camera; and measuring noise of the external home appliance input to the microphone based on a noise measurement start command input by the user.

[0009] According to one aspect of the disclosed invention, noise generated from an external home appliance can be measured at an appropriate noise measurement location.

[0010] According to one aspect of the disclosed invention, a noise measurement location for measuring noise generated from an external home appliance can be determined using a camera of an electronic device, thereby allowing more accurate measurement of the noise to be measured.

[0011] According to one aspect of the disclosed invention, a camera of an electronic device can be used to display a notification when a noise measurement location is not an appropriate location, thereby allowing a user of the electronic device to measure noise at an appropriate location.

[0012] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure belongs from the description below.

[0013] Figure 1 is a control block diagram of an electronic device according to one embodiment.

[0014] FIG. 2 illustrates a home appliance noise measurement system according to one embodiment.

[0015] FIG. 3 is a flowchart illustrating a method for measuring noise generated from an external electrical appliance of an electronic device according to one embodiment.

[0016] FIG. 4 illustrates an example in which an electronic device outputs a virtual image of an external home appliance according to one embodiment.

[0017] FIG. 5 illustrates an example of an electronic device outputting an image of an external home appliance according to one embodiment.

[0018] FIG. 6 illustrates an example of an electronic device outputting a noise measurement start screen for measuring noise of an external home appliance according to one embodiment.

[0019] FIG. 7 illustrates an example of an electronic device outputting a notification indicating that it is not in an appropriate location to measure noise from an external home appliance, according to one embodiment.

[0020] FIG. 8 illustrates an example in which an electronic device outputs a virtual image of an external home appliance according to another embodiment than FIG. 4.

[0021] FIG. 9 illustrates an example of an electronic device outputting a noise measurement start screen for measuring noise of an external home appliance according to another embodiment than FIG. 6.

[0022] It should be understood that the various embodiments and terms used in this document are not intended to limit the technical features described in this document to specific embodiments, but rather to include various modifications, equivalents, or substitutes of the embodiments.

[0023] In connection with the description of the drawings, similar reference numerals may be used for similar or related components.

[0024] The singular form of a noun corresponding to an item may include one or more of said items, unless the relevant context clearly indicates otherwise.

[0025] In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" may include any one of the items listed together in that phrase, or all possible combinations thereof.

[0026] The term "and / or" includes any combination of a plurality of related described elements or any one of a plurality of related described elements.

[0027] The terms "part," "module," and "member" may be implemented in hardware or software. Depending on the embodiments, multiple "parts," "modules," or "members" may be implemented as a single component, or a single "part," "module," or "member" may include multiple components.

[0028] Terms such as "first," "second," or "first" or "second" may be used simply to distinguish one component from another and do not qualify the components in any other respect (e.g., importance or order).

[0029] When a component (e.g., a first component) is referred to as being "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0030] The terms "include" or "have" are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in this document, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0031] When a component is said to be “connected,” “coupled,” “supported,” or “in contact with” another component, this includes not only cases where the components are directly connected, coupled, supported, or in contact, but also cases where the components are indirectly connected, coupled, supported, or in contact through a third component.

[0032] When we say that a component is "on" another component, this includes not only cases where the component is in contact with the other component, but also cases where there is another component between the two components.

[0033] Meanwhile, the terms "front", "back", "left", "right", "upper", "lower", etc. used in the following description are defined based on the drawing, but the shape and position of each component are not limited by the above terms. For example, the front side may be defined as the +X side, and the rear side may be defined as the -X side. For example, based on the drawing, the right side may be defined as the +Y side, and the left side may be defined as the -Y side. For example, based on the drawing, the upper side may be defined as the +Z side, and the lower side may be defined as the -Z side.

[0034] Hereinafter, embodiments according to the present invention will be described in detail with reference to the attached drawings.

[0035] Figure 1 is a control block diagram of an electronic device according to one embodiment.

[0036] Referring to FIG. 1, an electronic device (1) may include a microphone (10) that receives sound generated from the outside of the electronic device (1), a user interface (20) that obtains a user's input or displays various information related to the operation of the electronic device (1), a camera (30) that photographs the outside of the electronic device (1) to obtain an image, a communication unit (40) that communicates with an external device, and / or a control unit (50) that controls the microphone (10), the user interface (20), the camera (30), and / or the communication unit (40).

[0037] The microphone (10) can receive sound generated from outside the electronic device (1). The sound generated from outside the electronic device (1) may include the voice of the user of the electronic device (1), noise generated from an external home appliance (2, see FIG. 2) located outside the electronic device (1), and sound output from a home appliance (2) located outside the electronic device (1).

[0038] The microphone (10) can transmit sound input from the outside of the electronic device (1) to the control unit (50).

[0039] The user interface (20) may include an input unit (21) for obtaining user input and / or a display (22) for displaying information.

[0040] The input unit (21) can convert sensory information received from the user into an electrical signal.

[0041] The input unit (21) may include buttons, dials and / or touch pads provided at various locations of the electronic device (1).

[0042] For example, the input unit (21) may include a push button, a touch button, a touch pad, a touch screen, a dial, a stick-type operating device, and / or a trackball. If the input unit (21) is implemented as a touch screen, the input unit (21) may be provided as an integral part with the display (22).

[0043] The input unit (21) can convert sensory information received from the user into an electrical signal and transmit the converted electrical signal to the control unit (50).

[0044] The display (22) can output various information related to the operation of the electronic device (1) to the user by generating sensory information. The display (22) can include a light emitting diode (LED) panel, an organic light emitting diode (OLED) panel, and / or a liquid crystal display (LCD) panel, or a touch screen.

[0045] The camera (30) may include all components for acquiring an image of the outside of the electronic device (1). The camera (30) may be installed at various locations of the electronic device (1) to acquire an image of the outside of the electronic device (1). For example, the camera (30) may be installed at the front of the electronic device (1) to acquire an image of the space corresponding to the front of the electronic device (1). As another example, the camera (30) may be installed at the rear of the electronic device (1) to acquire an image of the space corresponding to the rear of the electronic device (1).

[0046] The camera (30) can transmit the acquired image of the outside of the electronic device (1) to the control unit (50).

[0047] The communication unit (40) may include at least one of a short-range communication module or a long-range communication module.

[0048] The communication unit (40) can transmit data to or receive data from an external device. The external device may include a server (3, see FIG. 2) and / or an external home appliance (2). This will be described later with reference to FIG. 2.

[0049] The control unit (50) can process user input received from the input unit (21). For example, the control unit (50) can process the user input received from the input unit (21) and recognize it as a user input for performing an operation of the microphone (10), a user input for performing an operation of the camera (30), a user input for performing an operation of the display (22), and / or a user input for performing an operation of the communication unit (40).

[0050] The control unit (50) can process the sound transmitted from the microphone (10) to identify the type of sound. For example, the control unit (50) can process the sound transmitted from the microphone (10) to recognize the sound generated from outside the electronic device (1) as the user's voice, noise generated from the external home appliance (2), and / or voice output from the external home appliance (2).

[0051] The control unit (50) can process an image of the outside of the electronic device (1) received from the camera (30) to obtain various information. For example, the control unit (50) can recognize an object (e.g., an external home appliance (2)) located outside the electronic device (1) by processing an image of the outside of the electronic device (1) received from the camera (30).

[0052] The control unit (50) can control the communication unit (40) so that the communication unit (40) establishes communication with an external device.

[0053] For example, the control unit (50) can control the communication unit (40) to transmit an image of the outside of the electronic device (1) received from the camera (30) to the server (3, see FIG. 2). The server (3, see FIG. 2) can recognize an object (e.g., an external home appliance (2)) located outside the electronic device (1) by processing the image of the outside of the electronic device (1) received from the electronic device (1). The server (3, see FIG. 2) can transmit information about an object (e.g., an external home appliance (2)) located outside the electronic device (1) to the electronic device (1).

[0054] Information about an object located outside of an electronic device (1) may include at least one of whether an object located outside of the electronic device (e.g., an external home appliance (2)) is recognized, the type of the object located outside of the electronic device (e.g., an external home appliance (2)), and the distance between the electronic device (1) and the object located outside of the electronic device (1) (e.g., an external home appliance (2)).

[0055] The control unit (50) may include hardware such as a CPU, Micom, or memory, and software such as a control program. For example, the control unit (50) may include at least one memory (52) that stores data in the form of a program, an algorithm for controlling the operation of components within the electronic device (1), and at least one processor (51) that performs the operations described above and the operations to be described below using the data stored in the at least one memory (52). The memory (52) and the processor (51) may each be implemented as separate chips. The processor (51) may include one or more processor chips or one or more processing cores. The memory (52) may include one or more memory chips or one or more memory blocks. In addition, the memory (52) and the processor (51) may be implemented as a single chip.

[0056] FIG. 2 illustrates a home appliance noise measurement system according to one embodiment.

[0057] Referring to FIG. 2, it may include an electronic device (1), at least one external home appliance (2) and / or a server (3).

[0058] The electronic device (1) can communicate with an external home appliance (2) or server (3). The electronic device (1) can receive user input or output information to the user.

[0059] The electronic device (1) can be carried by the user or placed in the user's home or office, etc. The electronic device (1) may include, but is not limited to, a personal computer, a terminal, a portable telephone, a smart phone, a handheld device, a wearable device, etc.

[0060] The electronic device (1) may store a program, i.e., an application, for controlling an external home appliance (2). The application may be sold installed on the electronic device (1) or downloaded and installed from an external server. The application may include an application for measuring noise generated by the external home appliance (2).

[0061] A user can access a server (3) by executing an application installed on an electronic device (1), create a user account, and communicate with the server (3) based on the logged-in user account to register an external home appliance (2).

[0062] For example, when the external home appliance (2) is operated so that the external home appliance (2) can be connected to the server (3) according to the procedure guided by the application installed on the electronic device (1), the external home appliance (2) can be registered in the user account by registering the identification information (e.g., serial number or MAC address) of the external home appliance (2) in the corresponding user account on the server (3).

[0063] The electronic device (1) can download a virtual image of an external home appliance (2) registered to a user account from a server (3) and store the downloaded virtual image of the external home appliance (2) in memory (52).

[0064] A user can control an external home appliance (2) using an application installed on an electronic device (1). For example, when a user logs into a user account using an application installed on an electronic device (1), an external home appliance (2) registered to the user account appears, and when a control command for the external home appliance (2) is input, the control command can be transmitted to the external home appliance (2) via a server (3).

[0065] In external home appliance (2), the outside may mean the outside of the electronic device (1).

[0066] The external home appliance (2) may include a communication unit that communicates with another external home appliance, electronic device (2), or server (3). The external home appliance (2) may include a user interface that receives user input or outputs information to the user. The external home appliance (2) may include a processor for controlling the operation of the external home appliance (2), and may include a memory that stores a program for controlling the operation of the external home appliance (2).

[0067] The external home appliance (2) may be at least one of various types of home appliances. For example, the external home appliance (2) may include a refrigerator (2a), a clothes treatment device (2b), an air conditioner (2c), etc., as illustrated. However, the external home appliance (2) may include not only the home appliances illustrated in FIG. 2, but also appliances commonly used in the home. For example, the external home appliance (2) may include at least one of a dishwasher, an electric range, an electric oven, a shoe care device, a microwave oven, a cleaning robot, a vacuum cleaner, a television, and a clothes care device.

[0068] The server (3) can communicate with an electronic device (1), another server, or an external home appliance (2). The server (3) can process data received from the electronic device (1), another server, or an external home appliance (2), and can store a program for processing the data or the processed data. The server (3) can be implemented as various computing devices such as a workstation, a cloud, a data drive, a data station, etc. The server (3) can be implemented as one or more servers that are physically or logically separated based on function, detailed configuration of function, or data, and can transmit and receive data and process the transmitted and received data through communication between each server.

[0069] The server (3) can perform functions such as managing user accounts, registering external home appliances (2) by linking them to user accounts, and managing or controlling the registered external home appliances (2). For example, a user can access the server (3) through an electronic device (1) and create a user account. The user account can be identified by an ID and password set by the user. The server (3) can register an external home appliance (2) to the user account according to a set procedure. For example, the server (3) can link identification information (e.g., serial number or MAC address) of the external home appliance (2) to the user account, thereby registering, managing, and controlling the external home appliance (2).

[0070] A network can include both wired and wireless networks. Wired networks include cable networks or telephone networks, while wireless networks can include any network that transmits and receives signals via radio waves. Wired and wireless networks can be interconnected.

[0071] The network may include a wide area network (WAN) such as the Internet, a local area network (LAN) formed around an Access Point (AP), and / or a short-range wireless network that does not pass through an Access Point (AP). Short-range wireless networks may include, but are not limited to, Bluetooth (IEEE 802.15.1), Zigbee (IEEE 802.15.4), Wi-Fi Direct, Near Field Communication (NFC), Z-Wave, etc., for example.

[0072] An access point (AP) can connect an external home appliance (2) or electronic device (1) to a wide area network (WAN) to which a server (3) is connected. The external home appliance (2) or electronic device (1) can be connected to the server (3) via the wide area network (WAN).

[0073] The access point (AP) can communicate with external home appliances (2) or electronic devices (1) using wireless communication such as Wi-Fi (IEEE 802.11), Bluetooth (IEEE 802.15.1), or Zigbee (IEEE 802.15.4), and can connect to a wide area network (WAN) using wired communication, but is not limited thereto.

[0074] According to various embodiments, the external home appliance (2) may be directly connected to the electronic device (1) or server (3) without going through an access point (AP).

[0075] An external home appliance (2) can be connected to an electronic device (1) or a server (3) via a long-distance wireless network or a short-distance wireless network.

[0076] For example, an external home appliance (2) can be connected to an electronic device (1) via a short-range wireless network (e.g., Wi-Fi Direct).

[0077] As another example, an external home appliance (2) may be connected to an electronic device (1) or a server (3) via a wide area network (WAN) using a long-distance wireless network (e.g., a cellular communication module).

[0078] As another example, an external home appliance (2) can be connected to a wide area network (WAN) using wired communication and connected to an electronic device (1) or a server (3) through the wide area network (WAN).

[0079] If the external home appliance (2) can connect to a wide area network (WAN) using wired communication, it can also function as an access relay. Accordingly, the external home appliance (2) can connect other home appliances to the wide area network (WAN) to which the server (3) is connected. In addition, other home appliances can connect the external home appliance (2) to the wide area network (WAN) to which the server (3) is connected.

[0080] An external home appliance (2) can transmit information about its operation or status to another home appliance, electronic device (1), or server (3) via a network. For example, the external home appliance (2) can transmit information about its operation or status to another home appliance, electronic device (1), or server (3) when a request is received from the server (3), when a specific event occurs in the external home appliance (2), or periodically or in real time. When information about its operation or status is received from the external home appliance (2), the server (3) can update the information about the operation or status of the external home appliance (2) that has been stored, and transmit the updated information about the operation and status of the external home appliance (2) to the electronic device (1) via a network. Here, updating information can include various operations that change existing information, such as an operation of adding new information to existing information, an operation of replacing existing information with new information, etc.

[0081] An electronic device (1) can transmit information about a user to an external home appliance (2) or server (3) via a communication unit (40). For example, the electronic device (1) can transmit information about the user's location, health status, preferences, schedule, etc. to the server (3). The electronic device (1) can transmit information about the user to the server (3) with the user's prior consent.

[0082] An external home appliance (2), electronic device (1), or server (3) can determine a control command using technology such as artificial intelligence. For example, the server (3) can receive information regarding the operation or status of an external home appliance (2) or information regarding a user of an electronic device (1), process the information using technology such as artificial intelligence, and transmit the processing result or control command to the external home appliance (2) or electronic device (1) based on the processing result.

[0083] FIG. 3 is a flowchart illustrating a method for measuring noise generated from an external electrical appliance of an electronic device according to one embodiment.

[0084] FIG. 4 illustrates an example in which an electronic device outputs a virtual image of an external home appliance according to one embodiment.

[0085] FIG. 5 illustrates an example of an electronic device outputting an image of an external home appliance according to one embodiment.

[0086] FIG. 6 illustrates an example of an electronic device outputting a noise measurement start screen for measuring noise of an external home appliance according to one embodiment.

[0087] FIG. 7 illustrates an example of an electronic device outputting a notification indicating that it is not in an appropriate location to measure noise from an external home appliance, according to one embodiment.

[0088] Referring to FIG. 3, the electronic device (1) can output a virtual image of an external home appliance (2) to a display (22) (S1).

[0089] Referring to FIG. 4, the virtual image of the external home appliance (2) may include a full virtual image (VI) corresponding to the full shape of the external home appliance (2) registered through the application.

[0090] The virtual image of the external home appliance (2) may include virtual images in the front, rear, upper, lower and side directions of the external home appliance (2).

[0091] However, for convenience of explanation, the following description assumes that the virtual image of the external home appliance (2) is a virtual image in front of the external home appliance (2).

[0092] In one embodiment, the processor (51) can control the display (22) so that a virtual image of an external home appliance (2) is output to the display (22).

[0093] For example, the processor (51) can control the display (22) to output a virtual image of an external home appliance (2) to the display (22) based on a command to execute an application.

[0094] In one embodiment, the memory (52) can store a virtual image of an external home appliance (2). For example, the memory (52) can download and store a virtual image corresponding to an external home appliance (2) registered in the application from the server (3).

[0095] The virtual image of the external home appliance (2) may include a first virtual area image (VI-1) and / or a second virtual area image (VI-2) occupying a predetermined area. The areas corresponding to each of the first virtual area image (VI-1) and the second virtual area image (VI-2) occupying the predetermined area may include areas corresponding to images that can be acquired by the camera (30) at an appropriate noise measurement location for measuring noise generated from the external home appliance (2).

[0096] Referring to FIGS. 3 and 4, a user can select a virtual image of an external home appliance (2) (S2).

[0097] For example, the processor (51) can control the display (22) to output a first interface (U1) that can select a first virtual area image (VI-1) and / or a second interface (U2) that can select a second virtual area image (VI-2) to the display (22).

[0098] When the input unit (21) is configured as a touch screen, the user can generate an input signal for selecting the first virtual area image (VI-1) by pressing an area corresponding to the first interface (U1). In addition, the user can generate an input signal for selecting the second virtual area image (VI-2) by pressing an area corresponding to the second interface (U2). However, the embodiment for selecting a virtual image according to the present disclosure is not limited thereto, and an input signal for selecting a virtual image can be generated according to various embodiments. For example, the user can generate an input signal for selecting a virtual image by inputting a voice for selecting the first virtual area image (VI-1) or the second virtual area image (VI-2) through a speaker.

[0099] Referring to FIGS. 3 and 5, the electronic device (1) can output an image of an external home appliance (2) acquired by the camera (30) through the display (22) (S3). The image of the external home appliance (2) acquired by the camera (30) may include an image acquired by the camera (30) photographing the external home appliance (2) located outside the electronic device (1).

[0100] The image of the external home appliance (2) acquired by the camera (30) may include an image (RI) corresponding to the overall shape of the external home appliance (2), a first area image (RI-1) occupying an area corresponding to the first virtual area image (VI-1) of the external home appliance (2), and / or a second area image (RI-2) occupying an area corresponding to the second virtual area image (VI-2) of the external home appliance (2).

[0101] In one embodiment, the processor (51) can output an image of the external home appliance (2) acquired by the camera (30) through the display (22) based on the user's selection of a virtual image of the external home appliance (2) (S3).

[0102] For example, the processor (51) can control the display (22) so that the image of the external home appliance (2) acquired by the camera (30) is output to the display (22) based on the user's selection of an area corresponding to the first virtual area image (VI-1) among the virtual images of the external home appliance (2).

[0103] In various embodiments, the processor (51) may control the display (22) to output a noise measurement start screen for measuring noise of the external home appliance (2) based on a virtual image of the external home appliance (2) selected by the user and an image of the external home appliance (2) acquired by the camera (30). Referring to FIG. 6, the noise measurement start screen may include a third interface (U3) for providing a notification indicating that the location is an appropriate location for measuring noise of the external home appliance (2) and / or a fourth interface (U4) for the user to generate a noise measurement start command for measuring noise generated from the external home appliance (2).

[0104] The processor (51) can measure the noise of an external home appliance (2) input to the microphone (10) based on a noise measurement start command generated by the user selecting the fourth interface (U4).

[0105] Referring to FIG. 3, in various embodiments, the processor (51) may control the display (22) to output a noise measurement start screen for measuring noise of the external home appliance (2) to the display (22) based on whether the image of the external home appliance (2) acquired by the camera (30) matches the virtual image of the external home appliance (2) selected by the user (example of S4) (S5).

[0106] In one embodiment, when the user selects the first virtual area image (VI-1), the processor (51) may control the display (22) to output the noise measurement start screen in FIG. 6 to the display (22) based on the fact that the first virtual area image (VI-1) matches the image of the external home appliance (2) acquired by the camera (30). In this case, the image of the external home appliance (2) acquired by the camera (30) may be a first area image (RI-1) that occupies an area corresponding to the first virtual area image (VI-1) of the external home appliance (2).

[0107] Referring to FIG. 6, in one embodiment, the processor (51) can control the display (22) to output a noise measurement start screen for measuring noise of the external home appliance (2) on the display (22) based on the matching of the image of the external home appliance (2) in the boundary area (R) for recognizing the image of the external home appliance (2) corresponding to the first virtual area image (VI-1) and the image of the external home appliance (2).

[0108] For example, the processor (51) may control the display (22) so that the image of the external home appliance (2) and the boundary area (R) acquired by the camera (30) are output to the display (22) based on the user's selection of the first virtual area image (VI-1), and may control the display (22) so that the noise measurement start screen for measuring the noise of the external home appliance (2) is output to the display (22) based on the image of the external home appliance (2) in the boundary area (R) being the first area image (RI-1).

[0109] Referring to FIGS. 3 and 6, in one embodiment, the processor (51) can measure noise of an external home appliance (2) input to a microphone (10) based on a noise measurement start command input from a user (example of S6) (S7).

[0110] For example, when a user selects a first virtual area image (VI-1), the processor (51) controls the display so that a noise start screen is output to the display (22) based on the image of the external home appliance (2) in the boundary area (R) being the first area image (RI-1), and can measure the noise of the external home appliance (2) input to the microphone (10) for a predetermined first noise measurement time based on a noise measurement start command input from the user.

[0111] The processor (51) can store information about noise measured during a predetermined first noise measurement time in the memory (52). The information about noise measured during the predetermined first noise measurement time can include the size of noise measured during the predetermined first noise measurement time, the period at which noise is generated, and / or a history of noise measurement during the predetermined first noise measurement time based on selection of the first virtual area image (VI-1).

[0112] Referring to FIGS. 3 and 7, in one embodiment, the processor (51) may control the display (22) to output a notification indicating that the location is not an appropriate location for measuring noise of the external home appliance (2) based on the fact that the image of the external home appliance (2) acquired by the camera (30) during the noise measurement time does not match the virtual image of the external home appliance (2) selected by the user (NO in S9).

[0113] For example, when a user selects a first virtual area image (VI-1), the processor (51) can control the display (22) to output an interface (U5) to the display (22) that provides a notification indicating that the image of the external home appliance (2) in the boundary area (R) during the first noise measurement time is not in an appropriate position for measuring noise based on the image not matching the first virtual area image (VI-1).

[0114] The processor (51) can stop the measurement of noise based on the fact that the image of the external home appliance (2) in the boundary area (R) during the first noise measurement time does not match the first virtual area image (VI-1).

[0115] The processor (51) controls the display (22) so that an interface (U5) that provides a notification indicating that the external home appliance (2) in the boundary area (R) is not in an appropriate position for measuring noise is output to the display (22) based on the fact that the image of the external home appliance (2) in the boundary area (R) does not match the first virtual area image (VI-1), and then controls the display (22) so that a noise measurement start screen for measuring noise of the external home appliance (2) is output to the display (22) based on the fact that the image of the external home appliance (2) in the boundary area (R) matches the first virtual area image (VI-1).

[0116] FIG. 8 illustrates an example in which an electronic device outputs a virtual image of an external home appliance according to another embodiment than FIG. 4.

[0117] FIG. 9 illustrates an example of an electronic device outputting a noise measurement start screen for measuring noise of an external home appliance according to another embodiment than FIG. 6.

[0118] Referring to FIG. 8, the processor (51) can control the display (22) so that a second virtual area image (VI-2) is output to the display (22) based on the noise of the external home appliance (2) measured during a predetermined first noise measurement time.

[0119] For example, the processor (51) can control the display (22) so that a second interface (U2) capable of selecting a second virtual area image (VI-2) is output to the display (22) after measuring the noise of the external home appliance (2) for a predetermined first noise measurement time.

[0120] The processor (51) can control the display (22) so that the second interface (U2), excluding the first interface (U1, see FIG. 4) that can select the first virtual area image (VI-1), after measuring the noise of the external home appliance (2) for a predetermined first noise measurement time, is output to the display (22).

[0121] Referring to FIG. 5, in one embodiment, the processor (51) can control the display (22) so that an image of an external home appliance (2) acquired by the camera (30) is output to the display (22) based on the user's selection of the second virtual area image (VI-2).

[0122] Referring to FIG. 9, in one embodiment, the processor (51) can control the display (22) to output a noise measurement start screen to the display (22) based on whether the second virtual area image (VI-2) selected by the user matches the image of the external home appliance (2) acquired by the camera (30).

[0123] For example, when a user selects a second virtual area image (VI-2), the processor (51) controls the display (22) so that the image of the external home appliance (2) acquired by the camera (30) and the boundary area (R) for recognizing the second virtual area image (VI-2) are output to the display (22), and the display (22) can be controlled so that a noise start screen is output to the display (22) based on the image of the external home appliance (2) in the boundary area (R) being the second area image (RI-2).

[0124] In one embodiment, the processor (51) can measure noise from an external home appliance (2) input to the microphone (10) for a predetermined second noise measurement time based on a noise measurement start command input from a user.

[0125] For example, when a user selects a second virtual area image (VI-2), the processor (51) controls the display so that a noise start screen is output to the display (22) based on the image of the external home appliance (2) in the boundary area (R) being the second area image (RI-2), and can measure the noise of the external home appliance (2) input to the microphone (10) for a predetermined second noise measurement time based on a noise measurement start command input from the user.

[0126] The processor (51) can store information about noise measured during a predetermined second noise measurement time in the memory (52). The information about noise measured during the predetermined second noise measurement time can include the size of noise measured during the predetermined second noise measurement time, the period at which noise is generated, and / or a history of noise measurement during the predetermined second noise measurement time based on selection of the second virtual area image (VI-2).

[0127] In one embodiment, the processor (51) can control the communication unit (40) to transmit information about noise measured during a predetermined first noise measurement time and / or information about noise measured during a predetermined second noise measurement time to an external home appliance (2) and / or a server (3).

[0128] For example, the processor (51) controls the communication unit (40) to transmit information about noise measured during a predetermined first noise measurement time and / or information about noise measured during a predetermined second noise measurement time to the server (3), and an expert in home appliances can use the information about noise measured during a predetermined first noise measurement time and / or information about noise measured during a predetermined second noise measurement time received through the server (3) to determine whether there is an abnormality in the operation of the external home appliance (2).

[0129] An electronic device according to one embodiment of the present disclosure may include a microphone for receiving noise generated from an external home appliance; a camera for acquiring an image of the external home appliance; a memory for storing a virtual image of the external home appliance; a display; and a processor for controlling the display so that the virtual image of the external home appliance is output to the display, and for controlling the display so that a noise measurement start screen for measuring noise of the external home appliance is output to the display based on a virtual image of the external home appliance selected by a user and an image of the external home appliance acquired by the camera, and for measuring noise of the external home appliance input to the microphone based on a noise measurement start command input from the user.

[0130] The virtual image of the external home appliance includes a first virtual area image and a second virtual area image occupying a predetermined area, and the processor can control the display so that the image of the external home appliance acquired by the camera is output to the display based on the user's selection of the first virtual area image.

[0131] The processor can measure the noise of the external home appliance input to the microphone for a predetermined first noise measurement time based on a noise measurement start command input from the user.

[0132] The processor may control the display to output a notification indicating that the location is not an appropriate location for measuring noise of the external home appliance based on the fact that the first virtual area image selected by the user and the image of the external home appliance do not match during the first predetermined noise measurement time.

[0133] The processor may control the display to output the second virtual area image to the display based on measuring the noise of the external home appliance during the first predetermined noise measurement time, and may control the display to output the image of the external home appliance acquired by the camera to the display based on the user selecting the second virtual area image.

[0134] The processor may control the display so that the noise measurement start screen is output to the display based on whether the second virtual area image selected by the user matches the image of the external home appliance acquired by the camera.

[0135] The processor can measure the noise of the external home appliance input to the microphone for a predetermined second noise measurement time based on a noise measurement start command input from the user.

[0136] The electronic device further includes a communication unit that communicates with an external device including a server and the external home appliance; and the processor can control the communication unit to transmit information about noise measured during the first predetermined noise measurement time and information about noise measured during the second predetermined noise measurement time to the external device.

[0137] The processor may control the display so that an image of the external home appliance acquired by the camera and a boundary area for recognizing an image of the external home appliance corresponding to the first virtual area image are output to the display based on the user's selection of the first virtual area image, and may control the display so that a noise measurement start screen for measuring noise of the external home appliance is output to the display based on the coincidence of the boundary area and the image of the external home appliance corresponding to the first virtual area image.

[0138] A method for controlling an electronic device according to one embodiment of the present disclosure may include: a microphone for receiving noise generated from an external home appliance; a camera for acquiring an image of the external home appliance; a memory for storing a virtual image of the external home appliance; and a display, the method comprising: controlling the display so that the virtual image of the external home appliance is output to the display; controlling the display so that a noise measurement start screen for measuring noise of the external home appliance is output to the display based on a virtual image of the external home appliance selected by a user and an image of the external home appliance acquired by the camera; and measuring noise of the external home appliance input to the microphone based on a noise measurement start command input by the user.

[0139] The virtual image of the external home appliance may further include a first virtual area image and a second virtual area image occupying a predetermined area, and controlling the display so that the image of the external home appliance acquired by the camera is output to the display based on the user's selection of the first virtual area image.

[0140] Controlling the display so that the noise measurement start screen is output to the display based on the virtual image of the external home appliance selected by the user and the image of the external home appliance acquired by the camera; may include controlling the display so that the noise measurement start screen is output to the display based on the first virtual area image selected by the user matching the image of the external home appliance acquired by the camera.

[0141] Measuring the noise of the external home appliance input to the microphone based on the noise measurement start command being input from the user; may include measuring the noise of the external home appliance input to the microphone for a predetermined first noise measurement time based on the noise measurement start command being input from the user.

[0142] The method may further include controlling the display so that a notification indicating that the location is not appropriate for measuring noise of the external home appliance is output on the display based on the fact that the first virtual area image selected by the user and the image of the external home appliance do not match during the first noise measurement time set in advance.

[0143] The method may further include controlling the display to output the second virtual area image to the display based on measuring the noise of the external home appliance during the first noise measurement time set in advance; and controlling the display to output the image of the external home appliance acquired by the camera to the display based on the user selecting the second virtual area image.

[0144] The method may further include controlling the display so that the noise measurement start screen is output to the display based on whether the second virtual area image selected by the user matches the image of the external home appliance acquired by the camera.

[0145] It may further include measuring the noise of the external home appliance input to the microphone for a predetermined second noise measurement time based on the noise measurement start command being input from the user.

[0146] The electronic device may further include a communication unit that communicates with an external device including a server and the external home appliance; and may further include controlling the communication unit to transmit information about noise measured during the first predetermined noise measurement time and information about noise measured during the second predetermined noise measurement time to the external device.

[0147] Controlling the display so that the image of the external home appliance acquired by the camera is output to the display based on the user's selection of the first virtual area image; includes controlling the display so that the image of the external home appliance acquired by the camera and a boundary area for recognizing the image of the external home appliance corresponding to the first virtual area image are output to the display based on the user's selection of the first virtual area image; and controlling the display so that the noise measurement start screen is output to the display based on the virtual image of the external home appliance selected by the user and the image of the external home appliance acquired by the camera; may include controlling the display so that the noise measurement start screen for measuring noise of the external home appliance is output to the display based on the coincidence of the boundary area and the image of the external home appliance corresponding to the first virtual area image.

[0148] Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium storing computer-executable instructions. The instructions may be stored in the form of program code, and when executed by a processor, may generate program modules to perform the operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

[0149] Computer-readable storage media include all types of storage media that store instructions that can be deciphered by a computer. Examples include read-only memory (ROM), random access memory (RAM), magnetic tape, magnetic disks, flash memory, and optical data storage devices.

[0150] Additionally, a computer-readable recording medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is permanently stored in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0151] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable recording medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated on a machine-readable recording medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0152] The disclosed embodiments have been described with reference to the attached drawings as described above. Those skilled in the art will understand that the present invention can be implemented in forms other than the disclosed embodiments without altering the technical spirit or essential features of the present invention. The disclosed embodiments are illustrative and should not be construed as limiting.

Claims

1. A microphone that receives noise generated from external electrical appliances; A camera for acquiring images of the above external home appliance; A memory for storing a virtual image of the external home appliance; display; and Controlling the display so that a virtual image of the external home appliance is output to the display; Controlling the display so that a noise measurement start screen for measuring noise of the external home appliance is output to the display based on a virtual image of the external home appliance selected by the user and an image of the external home appliance acquired from the camera; An electronic device comprising a processor for measuring noise of the external home appliance input to the microphone based on a noise measurement start command input from the user.

2. In paragraph 1, The virtual image of the above external home appliance is, Includes a first virtual area image and a second virtual area image occupying a predetermined area, The above processor, An electronic device that controls the display so that an image of the external home appliance acquired by the camera is output to the display based on the user's selection of the first virtual area image.

3. In paragraph 2, The above processor, An electronic device that controls the display so that the noise measurement start screen is output to the display based on whether the first virtual area image selected by the user matches the image of the external home appliance acquired by the camera.

4. In paragraph 3, The above processor, An electronic device that measures the noise of the external home appliance input to the microphone for a predetermined first noise measurement time based on a noise measurement start command input from the user.

5. In paragraph 4, The above processor, An electronic device that controls the display so that a notification indicating that the location is not appropriate for measuring noise of the external home appliance is output on the display based on the fact that the image of the first virtual area selected by the user and the image of the external home appliance do not match during the first noise measurement time set in advance.

6. In paragraph 4, The above processor, Controlling the display so that the second virtual area image is output to the display based on the noise of the external home appliance measured during the first noise measurement time set in advance; An electronic device that controls the display so that an image of the external home appliance acquired by the camera is output to the display based on the user's selection of the second virtual area image.

7. In paragraph 6, The above processor, An electronic device that controls the display so that the noise measurement start screen is output to the display based on whether the second virtual area image selected by the user matches the image of the external home appliance acquired by the camera.

8. In paragraph 7, The above processor, An electronic device that measures the noise of the external home appliance input to the microphone for a predetermined second noise measurement time based on a noise measurement start command input from the user.

9. In paragraph 8, Further comprising a communication unit for communicating with an external device including a server and the external home appliance; The above processor, An electronic device that controls the communication unit to transmit information about noise measured during the first predetermined noise measurement time and information about noise measured during the second predetermined noise measurement time to the external device.

10. In paragraph 2, The above processor, Controlling the display so that an image of the external home appliance acquired by the camera and a boundary area for recognizing an image of the external home appliance corresponding to the first virtual area image are output to the display based on the user's selection of the first virtual area image; An electronic device that controls the display so that a noise measurement start screen for measuring noise of the external home appliance is output to the display based on the matching of the image of the first virtual area and the image of the external home appliance in the boundary area.

11. A method for controlling an electronic device including a microphone for receiving noise generated from an external home appliance, a camera for obtaining an image of the external home appliance, a memory for storing a virtual image of the external home appliance, and a display, Controlling the display so that a virtual image of the external home appliance is output to the display; Controlling the display so that a noise measurement start screen for measuring noise of the external home appliance is output to the display based on a virtual image of the external home appliance selected by the user and an image of the external home appliance acquired from the camera; and A method for controlling an electronic device, comprising: measuring noise of the external home appliance input to the microphone based on a noise measurement start command input from the user; 12. In paragraph 11, The virtual image of the above external home appliance is, Includes a first virtual area image and a second virtual area image occupying a predetermined area, A method for controlling an electronic device, further comprising: controlling the display so that an image of the external home appliance acquired by the camera is output to the display based on the user's selection of the first virtual area image.

13. In paragraph 12, Controlling the display so that the noise measurement start screen is output to the display based on the virtual image of the external home appliance selected from the user and the image of the external home appliance acquired from the camera; A method for controlling an electronic device, comprising: controlling the display so that the noise measurement start screen is output to the display based on whether the first virtual area image selected by the user matches the image of the external home appliance acquired by the camera; 14. In paragraph 13, Measuring the noise of the external home appliance input to the microphone based on the noise measurement start command input from the user; A method for controlling an electronic device, comprising: measuring noise of the external home appliance input to the microphone for a predetermined first noise measurement time based on a noise measurement start command input from the user.

15. In paragraph 14, A method for controlling an electronic device, further comprising: controlling the display so that a notification indicating that the location is not appropriate for measuring noise of the external home appliance is output on the display based on the fact that the first virtual area image selected by the user and the image of the external home appliance do not match during the first noise measurement time set in advance.

Images