Electronic device, operation method thereof, and storage medium

Abstract

An electronic device is disclosed. The electronic device comprises: at least one processor including a processing circuit; and a memory storing instructions and including one or more storage media, wherein the instructions, when executed individually or collectively by the at least one processor, cause the electronic device to: obtain first location information in a map corresponding to each of at least one Internet of Things (IoT) device; identify a pointing direction of a user wearing a wearable device, on the basis of sensing data obtained through an inertial measurement unit (IMU) sensor included in the wearable device; identify a first IoT device corresponding to the pointing direction of the user from among the at least one IoT device, on the basis of at least one of the pointing direction of the user, the first location information, and second location information corresponding to the user in the map; and control an operation of the identified first IoT device.

Description

Electronic device and its method of operation and storage medium

[0001] The present disclosure relates to an electronic device, a method of operation thereof, and a storage medium, and more specifically, to an electronic device for identifying a device existing in the direction of a user's pointing, a method of operation thereof, and a storage medium.

[0002] With the advancement of electronic technology, various types of electronic devices are being developed and distributed. With the development of the Internet of Things, home appliances (e.g., air conditioners, dehumidifiers, etc.) can be controlled remotely in real time, and home appliances can be controlled within a single system.

[0003] Meanwhile, users can use various functions by wearing a wearable device such as a smart ring or a smart watch. Accordingly, users can control the home appliance they wish to control among the home appliances present in the home using the wearable device.

[0004] An electronic device according to one embodiment of the present disclosure comprises at least one processor including a processing circuit and a memory including one or more storage media for storing instructions, wherein the instructions, when executed individually or collectively by the at least one processor, can enable the electronic device to obtain first location information within a map corresponding to each of at least one IoT (Internet of Things) device.

[0005] According to one embodiment, the instructions may enable the electronic device to identify the pointing direction of a user wearing the wearable device based on sensing data obtained through an Inertial Measurement Unit (IMU) sensor included in the wearable device.

[0006] According to one embodiment, the instructions may enable the electronic device to identify a first IoT device corresponding to the user's pointing direction among the at least one IoT device, based on at least one of the user's pointing direction, the first location information, and the second location information corresponding to the user within the map.

[0007] According to one embodiment, the instructions may enable the electronic device to control the operation of the identified first IoT device.

[0008] A method of operating an electronic device according to one embodiment of the present disclosure may include the step of obtaining first location information within a map corresponding to each of at least one IoT (Internet of Things) device.

[0009] According to one embodiment, the operation method may include the step of identifying the pointing direction of a user wearing the wearable device based on sensing data obtained through an Inertial Measurement Unit (IMU) sensor included in the wearable device.

[0010] According to one embodiment, the operation method may include the step of identifying a first IoT device corresponding to the user's pointing direction among the at least one IoT device, based on at least one of the user's pointing direction, the first location information, and the second location information corresponding to the user within the map.

[0011] According to one embodiment, the operation method may include the step of controlling the operation of the identified first IoT device.

[0012] In a storage medium for storing computer-readable instructions according to one embodiment of the present disclosure, the instructions, when executed by at least one processor of an electronic device, can cause the electronic device to obtain first location information within a map corresponding to each of at least one IoT (Internet of Things) device.

[0013] According to one embodiment, the instructions may enable the electronic device to identify the pointing direction of a user wearing the wearable device based on sensing data obtained through an Inertial Measurement Unit (IMU) sensor included in the wearable device.

[0014] According to one embodiment, the instructions may enable the electronic device to identify a first IoT device corresponding to the user's pointing direction among the at least one IoT device, based on at least one of the user's pointing direction, the first location information, and the second location information corresponding to the user within the map.

[0015] According to one embodiment, the instructions may enable the electronic device to control the operation of the identified first IoT device.

[0016] FIGS. 1a and FIGS. 1b are drawings for schematically illustrating an example of use of an electronic device according to one embodiment.

[0017] FIG. 2 is a block diagram showing the configuration of an electronic device according to one embodiment.

[0018] FIG. 3 is a flowchart illustrating a method of operation of an electronic device according to one embodiment.

[0019] FIG. 4a is a flowchart illustrating the operation of identifying a first IoT device according to one embodiment.

[0020] FIGS. 4b and FIGS. 4c are drawings for explaining the operation of identifying a first IoT device according to one embodiment.

[0021] FIG. 5a is a flowchart illustrating the operation of identifying a first IoT device according to one embodiment.

[0022] FIGS. 5b and FIGS. 5c are drawings for explaining the operation of identifying a first IoT device according to one embodiment.

[0023] FIG. 6 is a flowchart illustrating a method for transmitting a control signal to an identified first IoT device according to one embodiment.

[0024] FIG. 7 is a diagram illustrating map information according to one embodiment.

[0025] FIG. 8a is a block diagram showing the configuration of a first device according to one embodiment.

[0026] FIG. 8b is a block diagram showing the configuration of a server according to one embodiment.

[0027] FIG. 8c is a block diagram showing the configuration of a second device according to one embodiment.

[0028] FIG. 9 is a block diagram showing the detailed configuration of an electronic device according to one embodiment.

[0029] The present disclosure will be described in detail below with reference to the attached drawings.

[0030] The terms used in this specification will be briefly explained, and the present disclosure will be described in detail.

[0031] The terms used in the embodiments of this disclosure have been selected to be as widely used as possible, taking into account their functions within this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant explanatory section of this disclosure. Therefore, terms used in this disclosure should be defined not merely by their names, but based on their meanings and the overall content of this disclosure.

[0032] In this specification, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of such features (e.g., numerical values, functions, operations, or components such as parts) and do not exclude the presence of additional features.

[0033] The expression "at least one of A or / and B" should be understood as representing either "A" or "B" or "A and B".

[0034] Expressions such as "first," "second," "first," or "second" used in this specification may modify various components regardless of order and / or importance, and are used only to distinguish one component from another and do not limit said components.

[0035] Where it is stated that a component (e.g., Component 1) is "(operatively or communicatively) coupled with / to" or "connected to" another component (e.g., Component 2), it should be understood that the component may be directly connected to the other component or connected through the other component (e.g., Component 3).

[0036] The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "consisting of" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0037] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Additionally, a plurality of "modules" or a plurality of "parts" may be integrated into at least one module and implemented by at least one processor (not shown), except for a "module" or "part" that needs to be implemented in specific hardware.

[0038] Furthermore, in this specification, the term "signal" includes not only electrical signals but also signals in the form of sound waves, and in the case of electrical signals, it may be a digital signal as well as an analog signal. For example, the expression "audio signal" (or "noise signal") refers to a sound wave (or radio wave) signal if the signal is outside the electronic device, and an electrical signal if it is inside the electronic device, depending on the location of the signal. Additionally, the signal processing, etc. within the electronic device described below may be a digital signal processing method, an analog signal processing method, or a signal processing method that uses a combination of analog and digital methods.

[0039] And in this specification, the term "filter" refers to removing a specific component (e.g., a specific frequency range or a specific pattern), and the filter may be a digital filter or an analog filter.

[0040] FIGS. 1a and FIGS. 1b are drawings for schematically illustrating an example of use of an electronic device according to one embodiment.

[0041] According to FIGS. 1a and 1b, in one embodiment, the electronic device (10-1 or 10-2) of the present invention may be implemented as a smart ring (10-1, Smart Ring) or a smart watch (10-2, Smart Watch). In one example, the smart ring (10-1) is a ring-type electronic device that can be worn by a user (1) and can detect the motion (or gesture) of the user (1) or the pointing direction of the user (1) (or the finger of the user (1)). In one example, the smart watch (10-2) is a watch-type electronic device worn by the user (1) and can detect the motion of the user (1) or the pointing direction corresponding to the wrist of the user (1). However, it is not limited thereto, and in one example, the smart watch (10-2) can also detect the pointing direction of the finger of the user (1).

[0042] According to one example, an electronic device (10-1 or 10-2) can identify an IoT (Internet of Things) device (20) corresponding to the pointing direction of the user (1). According to one example, the IoT device (20) may be a device existing within the space where the electronic device is located and may be in a state of being directly connected to the electronic device (10-1 or 10-2). Alternatively, according to one example, the IoT device (20) may communicate indirectly with the electronic device (10-1 or 10-2) through a server.

[0043] According to one example, an electronic device (10-1 or 10-2) can perform a control operation on an IoT device (20) corresponding to the pointing direction of the user (1). For example, when the electronic device (10-1 or 10-2) receives a command (e.g., a gesture) from the user (1) to perform an operation related to the identified IoT device (20), it can transmit a control signal to the IoT device (20) based thereon.

[0044] According to one example, although not illustrated in FIG. 1a and FIG. 1b, the electronic device of the present invention may be implemented as a server. According to one example, the electronic device of the present invention may receive information related to the user's pointing direction from a smart ring or smart watch and identify an IoT device (20) based thereon. Of course, the electronic device may transmit a control signal to the IoT device (20) to control the identified IoT device (20).

[0045] FIG. 2 is a block diagram showing the configuration of an electronic device according to one embodiment.

[0046] According to FIG. 2, the electronic device (100) may include at least one processor (110) and memory (120).

[0047] According to one example, the electronic device (100) may be implemented as a wearable device. For example, the electronic device (100) may be implemented as a smart ring or smart watch that a user can wear. Alternatively, the electronic device (100) may be implemented as a server. However, it is not limited thereto, and the electronic device (100) may be implemented as a different type of electronic device (100) capable of computing, processing, or storing data.

[0048] At least one processor (110) (hereinafter, processor) is electrically connected to memory (120) to control the overall operation of the electronic device (100). The processor (110) may be composed of one or more processors. Specifically, the processor (110) may perform the operation of the electronic device (100) according to various embodiments of the present disclosure by executing at least one instruction stored in memory (120).

[0049] According to one embodiment, the processor (110) may be implemented as a digital signal processor (DSP) that processes digital video signals, a microprocessor, a Graphics Processing Unit (GPU), an Artificial Intelligence (AI) processor, a Neural Processing Unit (NPU), or a Time Controller (TCON). However, it is not limited thereto, and may include or be defined by one or more of a central processing unit (CPU), a Micro Controller Unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a communication processor (CP), or an ARM processor. Additionally, the processor (110) may be implemented as a System on Chip (SoC) or Large Scale Integration (LSI) with a built-in processing algorithm, or may be implemented in the form of an Application Specific Integrated Circuit (ASIC) or Field Programmable Gate Array (FPGA).

[0050] The memory (120) can store data necessary for various embodiments. Depending on the purpose of data storage, the memory (120) may be implemented in the form of a memory embedded in the electronic device (100) or in the form of a memory that can be attached to and detached from the electronic device (100). For example, data for operating the electronic device (100) may be stored in a memory embedded in the electronic device (100), and data for the expansion function of the electronic device (100) may be stored in a memory that can be attached to and detached from the electronic device (100).

[0051] Meanwhile, the memory embedded in the electronic device (100) may be implemented as at least one of volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), etc.), non-volatile memory (e.g., OTPROM (one-time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash, etc.), hard drive, or solid state drive (SSD). Additionally, the memory that is detachable from the electronic device (100) may be implemented in the form of a memory card (e.g., CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.), external memory connectable to a USB port (e.g., USB memory), etc. there is.

[0052] According to one embodiment, the processor (110) may obtain first location information within a map corresponding to each of at least one Internet of Things (IoT) device. According to one example, at least one IoT device may be an IoT device within a space (or a first space) where the electronic device (100) is located, and may be a different type of IoT device including a TV, air conditioner, air purifier, humidifier, speaker, microwave oven, smart switch, light, and dehumidifier. According to one example, at least one IoT device present in a home may communicate with the electronic device (100). For example, at least one IoT device may transmit and receive data while connected to the electronic device (100).

[0053] According to one example, the map may be information about the space where the electronic device (100) is located. According to one example, the information about the map may include information about the shape of the map and information about the location within the map of at least one IoT device located in the first space. According to one example, if the electronic device (100) is implemented as a wearable device, the electronic device (100) may obtain information about the map from an external server. Alternatively, according to one example, if the electronic device (100) is implemented as a server, information about the shape of the map may be stored in memory (120), and the electronic device (100) may obtain information about the location of each IoT device from at least one IoT device. This will be explained in detail through FIG. 7.

[0054] According to one example, the first location information may be information about a location within a map corresponding to each of at least one IoT device. According to one example, the first location information may be coordinate information corresponding to each of the IoT devices. According to one example, the coordinate information may be 2D (Dimensional) information, but it is obvious that it may also be 3D information.

[0055] According to one embodiment, the processor (110) can identify the pointing direction of a user wearing a wearable device based on sensing data obtained through an inertial measurement unit (IMU) sensor included in the wearable device.

[0056] According to one example, the wearable device may include an IMU sensor. According to one example, when an electronic device (100) is implemented as a wearable device, the electronic device (100) may acquire sensing data from the IMU sensor. According to one example, the IMU sensor may sense at least one of data related to the acceleration of the electronic device (100) corresponding to each of a plurality of axes or data related to the strength of a magnetic field. According to one example, when the processor (110) acquires sensing data through the IMU sensor, it may identify the pointing direction of a user wearing the wearable device based on the data.

[0057] According to one example, if the electronic device (100) is implemented as a server, the electronic device (100) can acquire sensing data from a wearable device through a communication interface (e.g., the communication interface (160) of FIG. 9). According to one example, the electronic device (100) can identify the user's pointing direction based on the sensing data acquired through the communication interface.

[0058] According to one example, the user's pointing direction may be the direction pointed by the hand (or finger) wearing the wearable device. According to one example, the user's pointing direction may include direction information corresponding to at least one axis. For example, the user's pointing direction may be a direction calculated based on acceleration values ​​corresponding to each of two axes (or X-axis and Y-axis) horizontal to the ground. Or, for example, the user's pointing direction may be a direction calculated based on acceleration values ​​corresponding to each of two axes horizontal to the ground and acceleration values ​​corresponding to an axis perpendicular to the ground. Accordingly, the information regarding the user's pointing direction may be 2D information, but is not limited thereto, and the information regarding the user's pointing direction may, of course, be 3D information. This will be described later.

[0059] According to one embodiment, the processor (110) can identify a first IoT device corresponding to the user's pointing direction among at least one IoT device based on at least one of the user's pointing direction, first location information, and second location information corresponding to the user in the map.

[0060] According to one example, the processor (110) may obtain second location information corresponding to a user. According to one example, the second location information may be coordinate information corresponding to the user's location within a map. According to one example, it may be obtained based on at least one of a Bluetooth Low Energy (BLE) beacon or a presence sensor located within a space (or a first space) corresponding to the map.

[0061] According to one example, the BLE beacon may be a wireless communication device capable of automatically recognizing an electronic device (100) in close proximity and transmitting and receiving necessary data. According to one example, the occupancy sensor may be a sensor that detects motion in the sensing area of ​​the occupancy sensor. According to one example, the motion may be motion corresponding to a user (e.g., the user's movement). According to one example, the occupancy sensor may divide the space where the occupancy sensor is located into multiple areas and track motion detected in the multiple areas in real time to obtain information related to the user's location or the user's movement. According to one example, the BLE beacon may obtain second location information of the user by communicating with a wearable device worn by the user. According to one example, the occupancy sensor may transmit the acquired sensing data to an electronic device.

[0062] According to one example, the processor (110) may obtain sensing data related to the user's location from a BLE beacon or occupancy sensor through a communication interface, and obtain second location information corresponding to the user from the obtained sensing data. Alternatively, according to one example, the processor (110) may obtain second location information from a BLE beacon or occupancy sensor.

[0063] According to one example, when the processor (110) obtains second location information corresponding to the user's location within the map and the user's pointing direction, it can identify a first IoT device corresponding to the user's pointing direction within the map. For example, the processor (110) can identify an IoT device that is included within a preset angle range from the user's pointing direction based on the second location information corresponding to the user's location. If there is an IoT device included within the preset angle range, the processor (110) can identify the included IoT device as the first IoT device.

[0064] For example, the pre-set angle range may be changed based on the locations of the IoT device and the user. For instance, if the locations of the IoT device and the user are relatively far apart, the angle range may be relatively larger to accurately identify the IoT device. Alternatively, the area within the map corresponding to the IoT device may be relatively wider. This will be described later.

[0065]

[0066] According to one embodiment, the processor (110) can control the operation of the identified first IoT device. According to one example, when the first IoT device is identified, the processor (110) can transmit information to the first IoT device through a communication interface to acquire control of the identified IoT device. According to one example, the processor (110) can perform a control operation related to the first IoT device based on a user's command (or gesture).

[0067] FIG. 3 is a flowchart for explaining the operation method of an electronic device (e.g., the electronic device (100) of FIG. 2) according to one embodiment.

[0068] According to FIG. 3, according to one embodiment, the operation method may include an operation (S310) of obtaining first location information (e.g., first location information of FIG. 2) within a map corresponding to each of at least one IoT (Internet of Things) device (e.g., at least one IoT device of FIG. 2).

[0069] According to one example, an electronic device (e.g., the electronic device (100) of FIG. 2) may obtain first location information within a map of at least one IoT device in the space where the electronic device is located. For example, if the electronic device is implemented as a wearable device, the electronic device may obtain map information (e.g., information about the map of FIG. 2) including the first location information from an external server. Alternatively, for example, if the electronic device is implemented as a server, the electronic device may obtain map information by obtaining location information corresponding to each of at least one IoT device from at least one IoT device.

[0070] According to one embodiment, the operation method may include an operation (S320) of identifying the pointing direction (e.g., the pointing direction of FIG. 2) of a user wearing a wearable device based on sensing data obtained through an IMU (Inertial Measurement Unit) sensor (e.g., the IMU sensor of FIG. 2) included in the wearable device.

[0071] According to one example, an electronic device can acquire sensing data through an IMU sensor included in a wearable device. According to one example, the electronic device can identify a pointing direction corresponding to the direction of the hand or finger of the user wearing the wearable device based on acceleration values ​​corresponding to each of a plurality of axial directions acquired through the IMU sensor.

[0072] According to one embodiment, the operation method may include an operation (S330) of identifying a first IoT device (e.g., the first IoT device of FIG. 2) corresponding to the user's pointing direction among at least one IoT device, based on at least one of the user's pointing direction, first location information, and second location information corresponding to the user in the map (e.g., the second location information of FIG. 2).

[0073] According to one example, the electronic device may obtain second location information corresponding to the user from an external device (e.g., a BLE beacon or occupancy sensor of FIG. 2). According to one example, the electronic device may identify a first IoT device corresponding to the user's pointing direction among at least one IoT device located within a map based on the second location information and the user's pointing direction.

[0074] According to one embodiment, the operation method may include an operation (S340) for controlling the operation of the identified first IoT device.

[0075] According to one example, when a first IoT device is identified, the electronic device may transmit information to the first IoT device through a communication interface (e.g., the communication interface of FIG. 2) to acquire control of the identified IoT device. When control of the first IoT device is acquired, the electronic device may perform an operation to control the first IoT device based on a user's command or gesture. For example, when a user command related to the operation of the first IoT device is received, the electronic device may generate a corresponding control signal and transmit it to the first IoT device.

[0076] According to the example described above, the electronic device (100) can identify the IoT device that the user points to among a plurality of IoT devices existing in the space where the user is located, and control the identified IoT device. In this case, the electronic device (100) can identify the device that the user points to by considering information obtained through an IMU sensor equipped in a wearable device, the user's location information, and the location of the IoT device within the home, thereby improving the accuracy of identification and increasing the user's satisfaction.

[0077] FIG. 4a is a flowchart illustrating an operation for identifying a first IoT device (e.g., the first IoT device of FIG. 2) according to one embodiment. FIG. 4b and FIG. 4c are drawings illustrating an operation for identifying a first IoT device according to one embodiment.

[0078] According to FIGS. 4a, 4b, and 4c, according to one embodiment, the method of operation may include, when a second IoT device corresponding to a first angle range based on a pointing direction (e.g., the pointing direction of FIG. 2) is identified among at least one IoT device (e.g., at least one IoT device of FIG. 2), an operation (S410) of obtaining information about a second area in which a first area corresponding to the identified second IoT device is expanded to a first size.

[0079] According to one example, the first location information corresponding to at least one IoT device may include information about an area corresponding to the IoT device existing within a map (401 or 402, e.g., the map of FIG. 2). For example, the location information corresponding to 'device A' shown in FIG. 4b may include coordinate information for an area (411) corresponding to 'device A' within the map (401). Alternatively, the location information corresponding to 'device B' may include coordinate information for an area (421) corresponding to 'device B' within the map (401).

[0080] However, this is not limited thereto, and according to one example, the first location information may include not only information about the area corresponding to each corresponding IoT device, but also coordinate information within the map (401) corresponding to the center of the IoT device.

[0081] According to one example, an electronic device (e.g., the electronic device (100) of FIG. 2) can identify a second IoT device corresponding to a first angle range (42) based on the identified information when the second location information of the user (40) (e.g., the second location information of FIG. 2) and the pointing direction (41) of the user (40) are identified. According to one example, referring to FIG. 4c, the first angle range (42) may be an angle range set based on the pointing direction (41) corresponding to the user (40, or the user's location). According to one example, the first angle range (42) may be an angle range set regardless of the height of the pointing direction (41). According to one example, the first angle range (42) may be changed based on user input.

[0082] Alternatively, depending on one example, the first angle range (42) may be changed based on the characteristics of the space within the map where the user (40) is located. For example, if the user (40) is located in a relatively large space (e.g., a living room), the first angle range (42) may be larger than when the user (40) is located in a relatively small space. When the user (40) is located in a large space such as a living room, the distance between the IoT device and the user (40) is relatively far, so the first angle range (42) may be relatively larger to identify the device that the user (40) is pointing to.

[0083] According to one example, the second IoT device corresponding to the first angle range (42) may be an IoT device in which the entire area corresponding to the second IoT device is included within the first angle range (42).

[0084] According to one example, when a second IoT device is identified, the electronic device may expand the area corresponding to the identified second IoT device to a first size. For example, when 'Device A' is identified as a device corresponding to the first angle range (42), the electronic device may identify a second area (412) that is expanded from the first area (411) and corresponds to the identified 'Device A'. Alternatively, for example, when 'Device B' is identified as a device corresponding to the first angle range (42), the electronic device may identify a fourth area (422) that is expanded from the third area (421) and corresponds to the identified 'Device B'.

[0085] According to one example, the electronic device may calculate a first size based on map location information corresponding to a second IoT device and second location information corresponding to a user (40). According to one example, the first size may be a value calculated based on the map distance between the second IoT device and the user (40). According to one example, the first size may be a ratio. For example, if the map distance between the second IoT device and the user (40) is within a specified first distance range, a ratio (e.g., 1.4) corresponding to the first distance range may be identified as the first size.

[0086] According to one example, the electronic device may expand the area corresponding to each of the identified multiple IoT devices when multiple second IoT devices are identified. In this case, according to one example, each expanded area may be an area expanded such that each area (412 and 422) is adjacent to each other, as shown in the right-hand drawing of FIG. 4b.

[0087] However, this is not limited thereto, and according to one example, the electronic device may expand the area corresponding to each of at least one IoT device located within the map, regardless of the angle range corresponding to the pointing direction. For example, when the electronic device receives an action (or trigger action) for a user (40) to point to one of the IoT devices, it expands the size of the area for each IoT device located within the map by a preset size, thereby enabling the user (40) to better identify the IoT device being pointed to.

[0088] According to one embodiment, the method of operation may include an operation (S420) of identifying a first IoT device corresponding to the pointing direction (41) of the user (40) based on information regarding the acquired second area. According to one example, when information regarding the second area is acquired, the electronic device may identify the first IoT device based on information regarding the map (402) in which the area corresponding to each IoT device is expanded, rather than information regarding the existing map (401). According to one example, after the second area (412) corresponding to 'A device' is identified, if the identified pointing direction (41) of the user (40) is identified as corresponding to the second area (412), the electronic device may identify 'A device' as the first IoT device.

[0089] According to the example described above, the electronic device expands the area of ​​an IoT device within a preset angle range from the pointing direction (41) of the user (40), and is able to identify the IoT device that the user (40) is pointing to based on the expanded area. Accordingly, the user (40) can accurately point to the IoT device they wish to control, and the recognition rate of the IoT device can be increased.

[0090] FIG. 5a is a flowchart illustrating an operation for identifying a first IoT device (e.g., the first IoT device of FIG. 2) according to one embodiment. FIG. 5b and FIG. 5c are drawings illustrating an operation for identifying a first IoT device according to one embodiment.

[0091] According to FIGS. 5A, 5B, and 5C, according to one embodiment, the operation method may include an operation (S510) of obtaining a first angle (51) corresponding to a first axis (5, axis) of a pointing direction (50, e.g., pointing direction (50) of FIG. 2) based on acquired sensing data (e.g., sensing data of FIG. 2).

[0092] According to one example, an electronic device (e.g., the electronic device (100) of FIG. 2) may acquire sensing data through a sensor (e.g., the IMU sensor of FIG. 2) included in a wearable device. According to one example, the sensing data may include sensing data related to the magnitude of acceleration (or magnetic field strength) corresponding to each of a plurality of axes (e.g., X-axis, Y-axis, and Z-axis). According to one example, when the acceleration corresponding to each of the plurality of axes is identified based on the acquired sensing data, the electronic device may acquire a first angle (51) corresponding to one of the plurality of axes based on the identified value.

[0093] According to one example, the first axis (5) may be an axis perpendicular to the ground (or a Z-axis). According to one example, the first angle (51) may be the angle formed by the first axis and the user's pointing direction (50), with reference to FIG. 5c. According to one example, the electronic device may calculate the first angle (51) based on acquired sensing data.

[0094] According to one embodiment, the operation method may include an operation (S520) of identifying an IoT device corresponding to the second angle range among at least one IoT device when it is identified that the identified first angle (51) is included in the second angle range with respect to the first axis (5).

[0095] According to one example, when a first angle (51) is identified, the electronic device may determine whether the first angle (51) belongs to any one of a plurality of angle ranges (52, 53 and 54). According to one example, the plurality of angle ranges (52, 53 and 54) may be angle ranges set relative to the first axis (5). For example, the second angle range may be an angle range that is 60 degrees or more and less than 120 degrees counterclockwise from the first axis (5). Or, for example, the second angle range may be an angle range that is 120 degrees or more and less than 180 degrees from the first axis (5).

[0096] According to one example, the electronic device may obtain information about an IoT device (e.g., at least one IoT device of FIG. 2) corresponding to each of a plurality of angular ranges (52, 53, and 54). According to one example, map information (500, e.g., information about the map of FIG. 2) may include 3D (Dimensional) information about a first space (e.g., the first space of FIG. 2) where the electronic device is located, as illustrated in FIG. 5b, and may include 3D coordinate information corresponding to each of at least one IoT device located in the first space. According to one example, the electronic device may identify an IoT device corresponding to each of the plurality of angular ranges (52, 53, and 54) based on a coordinate value corresponding to a first axis (5, or Z-axis) of each of at least one IoT device.

[0097] For example, in the case of an IoT device (e.g., 'LIGHT 4') where the coordinate value corresponding to the first axis (5) is greater than or equal to a specified first value (e.g., 7), it may be included in the IoT device corresponding to the angle range (52) at a relatively higher position among the plurality of angle ranges (52, 53, and 54). Alternatively, for example, in the case of an IoT device (e.g., 'AIR') where the coordinate value corresponding to the first axis (5) is less than or equal to a specified second value (e.g., 3), it may be included in the IoT device corresponding to the angle range (54) at a relatively lower position among the plurality of angle ranges (52, 53, and 54). According to one example, the electronic device may group at least one IoT device existing in the map according to the plurality of angle ranges (52, 53, and 54) based on the coordinate value corresponding to the first axis (5) of each of at least one IoT device existing in the map (e.g., the map of FIG. 2).

[0098] According to one example, the electronic device can identify an IoT device corresponding to an angle range that includes an identified first angle (51) based on information about an IoT device corresponding to each of a plurality of angle ranges (52, 53 and 54).

[0099] According to one example, if the electronic device identifies that the identified first angle (51) is included in the second angle range with respect to the first axis (5), it can identify an IoT device corresponding to the second angle range.

[0100] According to one embodiment, the operation method may include an operation (S530) of identifying a first IoT device among IoT devices corresponding to a second angle range.

[0101] According to one example, if an IoT device corresponding to a second angle range is identified, the electronic device can identify a first IoT device among them. For example, the electronic device can identify an IoT device corresponding to the user's pointing direction (50) among the IoT devices corresponding to the second angle range as the first IoT device.

[0102] According to one embodiment, the second angle range may be identified based on the posture of a user wearing a wearable device. According to one example, the angle range corresponding to when the user is standing and when the user is sitting may be different. According to one example, the electronic device may identify the user's posture through a sensor included in the wearable device (e.g., a motion detection sensor, etc.), and if it is identified as the first posture among a plurality of postures, it may identify the angle range corresponding to the identified first posture. The electronic device may identify an IoT device corresponding to the identified angle range and identify a first IoT device among the identified IoT devices.

[0103] According to the example described above, the electronic device can identify the device that the user is pointing to by taking into account the height of the IoT devices present in the home. When there are multiple IoT devices corresponding to the user's pointing direction (50) (for example, when a robot vacuum cleaner is located under a light), it may be difficult to identify the device corresponding to the user's pointing direction (50) among the robot vacuum cleaner and the light on the 2D map information. However, in the present invention, since the device corresponding to the user's pointing direction (50) can be identified among the IoT devices at a height (or Z-axis angle range) corresponding to the user's pointing direction (50), a more accurate identification of the device becomes possible.

[0104] Returning to FIG. 2, according to one embodiment, the processor (110) can identify the pointing direction of a user based on a trigger action. According to one example, the processor (110) can identify the pointing direction of a user wearing a wearable device when a trigger action for identifying at least one IoT device is identified. According to one example, the trigger action may be a different type of pre-set action, including a tap action by the user with a finger.

[0105] According to one example, the trigger action may be a preliminary action (or start gesture) for identifying the IoT device that the user intends to control among at least one IoT device. For example, the processor (110) may identify the user's pointing direction after the trigger action is identified and identify the first IoT device based thereon. However, it is not limited thereto.

[0106] According to one example, the wearable device may include a motion detection sensor that detects the motion of a user. According to one example, when the electronic device is implemented as a wearable device, the processor (110) may identify a trigger action based on sensing data obtained through the motion detection sensor. According to one example, when the electronic device is implemented as a server, the processor (110) may receive sensing data corresponding to the motion detection sensor from the wearable device through a communication interface and may identify a trigger action of the user based on the received sensing data. According to one embodiment, if the processor (110) does not identify an IoT device corresponding to the user's pointing direction among at least one IoT device, the processor (110) may provide a User Interface (UI) that guides the user to change the pointing direction. According to one example, the processor (110) may provide a UI that causes the user to change their pointing direction if the IoT device is not identified within a preset angle range corresponding to the user's pointing direction (e.g., a first angle range in FIG. 4a or a second angle range in FIG. 5a). For example, the UI may provide text that guides the user to change their pointing direction, or provide an alarm (e.g., a vibration alarm or a sound alarm).

[0107] Alternatively, according to one example, the processor (110) may provide a UI that guides the user to select one of the multiple IoT devices when multiple IoT devices are identified within a preset angle range corresponding to the user's pointing direction. For example, if the electronic device is implemented as a smart watch, the processor (110) may provide a UI containing text that allows the user to select one of the multiple devices through the display of the watch.

[0108] According to one example, when the electronic device is implemented as a Smart Ring, the processor (110) may provide information that a first device is selected when the user performs a first number of tap actions, and a second device is selected when the user performs a second number of tap actions, through a user terminal (e.g., a smartphone) that is connected to the Smart Ring. However, this is not limited thereto, and according to one example, when the electronic device is implemented as a Smart Ring, the processor (110) may guide the user to select one of the devices by providing a vibration alarm for a preset number of times or a vibration alarm for a preset time.

[0109] According to one embodiment, the processor (110) may provide a UI for verifying an IoT device. According to one example, when an IoT device corresponding to the user's pointing direction is identified, the processor (110) may provide a UI for selecting whether to select the identified IoT device along with information about the identified IoT device. For example, the UI may provide a UI that includes information such as 'Would you like to select the first device?'

[0110] FIG. 6 is a flowchart illustrating a method for transmitting a control signal according to one embodiment to an identified first IoT device (e.g., the first IoT device of FIG. 2).

[0111] According to FIG. 6, according to one embodiment, the operation method may include an operation (S610) of identifying a first function of the first IoT device corresponding to the first gesture when a first gesture of a user is identified after the first IoT device is identified. According to one example, the first gesture may be a gesture for causing the first IoT device to perform a first function that can be performed after the first IoT device is identified.

[0112] According to one example, an electronic device (e.g., the electronic device (100) of FIG. 2) can identify a user's gesture. According to one example, a wearable device may include a motion detection sensor that detects motion of a user wearing the device. According to one example, the electronic device can acquire sensing data related to the user's motion through the motion detection sensor and identify a first gesture of the user based on the acquired sensing data.

[0113] According to one example, a memory (e.g., memory (120) of FIG. 2) may store information regarding different types of gesture-specific functions corresponding to the first IoT device. According to one example, when the first gesture of a user is identified, the electronic device may identify a first function corresponding to the first gesture among a plurality of functions corresponding to an air conditioner based on the information stored in the memory.

[0114] According to one embodiment, the operation method may include an operation (S620) of transmitting a control signal to a first IoT device to perform an identified first function through a communication interface (e.g., the communication interface (160) of FIG. 9).

[0115] According to one example, a case can be assumed where an air conditioner is identified as a first IoT device. When the air conditioner is identified as the first IoT device, the electronic device may provide information (e.g., a vibration notification) to the user indicating that the IoT device corresponding to the user's pointing direction has been selected. After the air conditioner is identified as the first IoT device, if a user's gesture of clenching a fist is detected, the electronic device may transmit a control signal to turn on the air conditioner through a communication interface. The air conditioner may perform a turn-on operation based on the received control signal.

[0116] According to one example, if a user raises their hand gesture after the air conditioner is turned on, the electronic device may transmit a control signal to the air conditioner through a communication interface to raise the desired temperature of the air conditioner. Alternatively, according to one example, if a user taps their finger a preset number of times, the electronic device may transmit information related to the termination of air conditioner control to the air conditioner through a communication interface.

[0117] According to one example, the electronic device may identify a trigger action as a gesture different from the first gesture. According to one example, the trigger action may be a preliminary action (or start gesture) for identifying the IoT device that the user wishes to control among at least one IoT device. According to one example, when the trigger action is identified, the electronic device may identify the user's pointing direction and identify the first IoT device based thereon. Subsequently, when the first gesture is identified, the electronic device may transmit a control signal to the first IoT device to perform a first function corresponding to the first gesture.

[0118] For example, a case can be assumed where the trigger action (or start gesture) is the action of folding a fist and then opening it again. When the trigger action is identified, the electronic device can identify the user's pointing direction. The electronic device can identify a first IoT device corresponding to the user's pointing direction. After the first IoT device is identified, when a first gesture for controlling the first IoT device is identified, a control signal corresponding to the first gesture can be transmitted to the first IoT device. Subsequently, when a gesture for terminating the control operation for the first IoT device (e.g., performing a tap action once, or a gesture for terminating the control operation) is identified, the electronic device can terminate the control operation for the first IoT device. After the control operation for the first IoT device is terminated, when a gesture for terminating the action of identifying the user's pointing direction is identified (e.g., folding a fist and then opening it again, or a pointing termination gesture), the electronic device can terminate the action of identifying the user's pointing direction.

[0119] According to one example, an electronic device may control an identified IoT device based on a selection action (or selection gesture) for selecting an identified IoT device among at least one IoT device, as a gesture different from a first gesture. For example, a case may be assumed where a user pointing direction is identified and a first IoT device corresponding thereto is identified. When a user gesture for selecting the first IoT device (e.g., performing a tap action twice) is identified, the electronic device selects the first IoT device and becomes able to control the selected first IoT device. Even after the first IoT device is identified, the electronic device becomes able to control the identified first IoT device only when a separate user selection action is identified. Accordingly, even if the user's pointing direction changes due to the user's hand movements, the user can easily select the identified first IoT device.

[0120] According to one example, a selection gesture may be a single action with a trigger action. According to one example, when a selection gesture (e.g., performing a tap action twice) is identified, an electronic device may select an IoT device corresponding to the user's pointing direction at the time the selection gesture is identified, even in the absence of a separate trigger action. In this case, the electronic device may continuously track the user's pointing direction based on sensing data, and may continuously identify an IoT device corresponding to the user's pointing direction. When a selection gesture is identified in the absence of a separate trigger action, the electronic device may control the action of the IoT device corresponding to the user's pointing direction at the time the selection gesture is identified (or at the current time). Accordingly, even without a separate trigger action, the user can select and control the IoT device they wish to control with only a simple action. FIG. 7 is a diagram for explaining map information (e.g., information regarding the map of FIG. 2) according to one embodiment.

[0121] Referring to FIG. 7, according to one embodiment, an electronic device (e.g., the electronic device (100) of FIG. 2) may obtain map information (700). According to one example, the map information (700) may be map information (700) for a first space (e.g., the first space of FIG. 2) where the electronic device is located. According to one example, the map information (700) may include information about at least one IoT device (e.g., at least one IoT device of FIG. 2) located in the first space. For example, the map information (700) may include location information within the map (e.g., the first location information of FIG. 2) corresponding to each of the at least one IoT device.

[0122] In one example, the map information (700) may be stored in memory (e.g., memory (120) of FIG. 2), but is not limited thereto. In one example, the electronic device may obtain the map information (700) from an external server through a communication interface (e.g., communication interface (160) of FIG. 9).

[0123] According to one example, when the second location information corresponding to the user (70) and the pointing direction (71) of the user (70) are identified, the electronic device can identify the first IoT device (e.g., the first IoT device of FIG. 2) using map information (700) together with the identified second location information and pointing direction (71).

[0124] For example, as illustrated in FIG. 7, a case may be assumed where a user (70) is located in a first sub-space included in a first space. Based on second location information and the pointing direction (71) of the user (70), the electronic device can identify one device (7-1) corresponding to the pointing direction (71) of the user (70) among a plurality of IoT devices (7-1 and 7-2) existing in the first sub-space. The electronic device provides information to the user (70) indicating that the identified device (7-1) has been selected, and subsequently, can remotely control the device (7-1) based on the user's (70's) command (e.g., gesture).

[0125] Alternatively, for example, if there are multiple devices corresponding to the pointing direction (71) of the user (70), unlike as illustrated in FIG. 7, the electronic device may identify one of the devices based on the location of the subspace where the user (70) is present. For example, depending on the location of the user (70), there may be cases where not only the device existing in the subspace where the user (70) is present, but also the device existing in the subspace where the user (70) is not present corresponds to the pointing direction (71) of the user (70). The electronic device may identify the device corresponding to the pointing direction (71) of the user (70) among the devices existing within the same subspace as the user (70) as the first IoT device based on the location of the subspace where the user (70) is present.

[0126] According to one example, as illustrated in FIGS. 5a to 5c, the map information (700) may be 3D information. According to one example, the first location information corresponding to at least one IoT device, the second location information corresponding to a user (70), and the pointing direction (71) of the user (70) may each be 3D information. When the electronic device identifies the first IoT device using the pointing direction (71), it may identify the first IoT device based on the Z-axis information of the pointing direction (71) (e.g., the first axis in FIG. 5a) and the Z-axis information of the IoT device.

[0127] FIG. 8a is a block diagram showing the configuration of a first device according to one embodiment.

[0128] As illustrated in FIG. 8a, according to one embodiment, a first device (810, e.g., the electronic device (100) of FIG. 2) may include an angle calculation module (811), a gesture recognition module (812), a second device control module (813), a sensing data collection module (814), a notification information display module (815), and a communication module (816). According to one example, the first device may be a wearable device (e.g., the wearable device of FIG. 2).

[0129] According to one example, the angle calculation module (811) can calculate an angle related to the user's pointing direction (e.g., the pointing direction of FIG. 2) based on sensing data obtained through an IMU sensor (e.g., the IMU sensor of FIG. 2). For example, the angle calculation module (811) can calculate an angle corresponding to each of a plurality of axes (e.g., a plurality of axes of FIG. 2).

[0130] According to one example, the gesture recognition module (812) can recognize a gesture of a user wearing the first device (810). According to one example, the first device (810) may include a motion detection sensor that detects the user's motion, and according to one example, the gesture recognition module (812) can recognize the user's gesture based on the acquired sensing data.

[0131] According to one example, the second device (e.g., the second device of FIG. 8c) control module (813) may be a module that generates a control signal for controlling the second device. According to one example, the second device may be a first IoT device (e.g., the first IoT device of FIG. 2) corresponding to the user's pointing direction. According to one example, the second device control module (813) may generate a control signal based on the user's command (or gesture) for controlling the first IoT device when such command is identified.

[0132] According to one example, the sensing data collection module (814) may collect sensing data for identifying a user’s gesture or for identifying a user’s pointing direction. For example, the sensing data collection module (814) may collect sensing data from an IMU sensor. Or, for example, the sensing data collection module (814) may collect sensing data from a motion detection sensor.

[0133] According to one example, the notification information display module (815) can provide a notification to the user. For example, the notification information display module (815) can display text-type information through a display. Alternatively, the notification information display module (815) may provide a vibration-type notification to the user through a vibration sensor.

[0134] In one example, the communication module (816) may transmit a control signal generated through the second device control module (813) to the second device (e.g., the second device of FIG. 8c). Alternatively, in one example, the communication module (816) may receive information of a different type from a server (e.g., the server of FIG. 8b).

[0135] According to one example, when the electronic device of FIG. 2 is implemented as a wearable device, the electronic device may include a plurality of modules included in the first device (810).

[0136] FIG. 8b is a block diagram showing the configuration of a server according to one embodiment.

[0137] As illustrated in FIG. 8b, according to one embodiment, the server (820) may include a second device (e.g., the second device in FIG. 8c) location data collection module (821) and a user location data collection module (822).

[0138] According to one example, the second device location data collection module (821) can collect information about the location of a second device (e.g., at least one IoT device of FIG. 2) existing within a first space (e.g., the first space of FIG. 2) where an electronic device (e.g., the electronic device (100) of FIG. 2) is located. For example, the second device location data collection module (821) can obtain data about each location from each of the at least one IoT device existing within the first space through a communication interface (e.g., the communication interface (160) of FIG. 9).

[0139] According to one example, the user location data collection module (822) may collect data regarding the user's location within a first space. According to one example, the user location data collection module (822) may obtain the data based on at least one of a Bluetooth Low Energy (BLE) beacon or a presence sensor located within a space corresponding to a map (or, the first space). According to one example, the user location data collection module (822) may collect the user's location data based on sensing data received from at least one of the beacon or presence sensor.

[0140] FIG. 8c is a block diagram showing the configuration of a second device according to one embodiment.

[0141] As illustrated in FIG. 8c, according to one embodiment, the second device (830) may include a signal output module (831) and a device control module (832).

[0142] According to one example, the second device (830) may be at least one IoT device of FIG. 2. Alternatively, the second device may be an identified first IoT device (e.g., the first IoT device of FIG. 2).

[0143] According to one example, the signal output module (831) may be a module that outputs a signal related to the location of the second device (830) within the space where the second device (830) is located (e.g., the first space of FIG. 2).

[0144] According to one example, the device control module (832) may be a module that controls the operation of the second device (830). According to one example, when the second device (830) receives a first control signal related to a user gesture from the first device (e.g., the first device (810) of FIG. 8a), the device control module (832) may control the second device (830) based on the received first control signal.

[0145] FIG. 9 is a block diagram showing the detailed configuration of an electronic device according to one embodiment.

[0146] According to FIG. 9, the electronic device (100') may include at least one processor (110), memory (120), display (130), at least one sensor (140), user interface (150), communication interface (160), speaker (170), and microphone (180). A detailed description of configurations shown in FIG. 9 that overlap with configurations shown in FIG. 2 will be omitted.

[0147] The display (130) may be implemented as a display including a self-emissive element or as a display including a non-emissive element and a backlight. For example, it may be implemented as various types of displays such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, an LED (Light Emitting Diodes), a micro LED, a Mini LED, a PDP (Plasma Display Panel), a QD (Quantum dot) display, or a QLED (Quantum dot light-emitting diodes). The display (130) may also include a driving circuit, a backlight unit, etc., which can be implemented in the form of an a-si TFT, an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc. Meanwhile, the display (130) may be implemented as a touch screen combined with a touch sensor, a flexible display, a rollable display, a 3D display, a display in which a plurality of display modules are physically connected, etc. The processor (110) can control the display (130) to output an output image obtained according to the various embodiments described above. Here, the output image may be a high-resolution image of 4K or 8K or higher. According to one embodiment, the output image may be a game image.

[0148] According to one embodiment, the display (130) may include a plurality of haptic elements. The haptic elements may be implemented as motors to provide haptic feedback (e.g., vibration feedback) to a user, but are not limited thereto. According to one example, the display (130) may include a predetermined number of haptic elements. For example, the display (130) may include a predetermined number of haptic elements corresponding to a predetermined number of sub-regions of the display, but is not limited thereto, and it is obvious that the display may include a number of haptic elements different from the number of sub-regions corresponding to the display.

[0149] At least one sensor (140, hereinafter referred to as the sensor) may include a plurality of sensors of various types. The sensor (140) may measure physical quantities or detect the operating state of an electronic device (100') and convert the measured or detected information into an electrical signal. The sensor (140) may include a camera, and the camera may include a lens that focuses visible light or other optical signals received by being reflected by an object onto an image sensor, and an image sensor capable of detecting visible light or other optical signals. Here, the image sensor may include a 2D pixel array divided into a plurality of pixels. Alternatively, at least one sensor (140) may include a temperature sensor or an infrared sensor.

[0150] The user interface (150) is a configuration for the electronic device (100') to perform interaction with the user. For example, the user interface (150) may include at least one of a touch sensor, a motion sensor, a button, a jog dial, a switch, a microphone, or a speaker, but is not limited thereto.

[0151] The communication interface (160) can input and output various types of data. For example, the communication interface (160) can transmit and receive various types of data to and from an external device (e.g., source device), an external storage medium (e.g., USB memory), an external server (e.g., web hard drive) through communication methods such as AP-based Wi-Fi (Wi-Fi, Wireless LAN network), Bluetooth, Zigbee, wired / wireless LAN (Local Area Network), WAN (Wide Area Network), Ethernet, IEEE 1394, HDMI (High-Definition Multimedia Interface), USB (Universal Serial Bus), MHL (Mobile High-Definition Link), AES / EBU (Audio Engineering Society / European Broadcasting Union), Optical, Coaxial, etc.

[0152] According to one example, the communication interface (160) may include a Bluetooth Low Energy (BLE) module. BLE refers to Bluetooth technology capable of transmitting and receiving low-power, low-capacity data in a 2.4 GHz frequency band with a range of about 10 m. However, it is not limited thereto, and the communication interface (160) may include a Wi-Fi communication module. That is, the communication interface (160) may include at least one of a Bluetooth Low Energy (BLE) module or a Wi-Fi communication module.

[0153] According to one embodiment, the speaker (170) may be composed of a tweeter for reproducing high-frequency sound, a midrange for reproducing mid-frequency sound, a woofer for reproducing low-frequency sound, a subwoofer for reproducing ultra-low-frequency sound, an enclosure for controlling resonance, and a crossover network for dividing the frequency of an electrical signal input to the speaker into bands.

[0154]

[0155] According to one embodiment, the speaker (170) can output an acoustic signal to the outside of the electronic device (100'). The speaker (170) can output multimedia playback, recording playback, various notification sounds, voice messages, etc. The electronic device (100') may include an audio output device such as the speaker (170), but may include an output device such as an audio output terminal. In particular, the speaker (170) can provide acquired information, information processed or produced based on the acquired information, response results to user voice or operation results, etc., in the form of voice.

[0156] The microphone (180) may refer to a module that acquires sound and converts it into an electrical signal, and may be a condenser microphone, ribbon microphone, moving coil microphone, piezoelectric element microphone, carbon microphone, or MEMS (Micro Electro Mechanical System) microphone. Additionally, it may be implemented in omnidirectional, bidirectional, unidirectional, subcardioid, supercardioid, or hypercardioid modes. According to one embodiment, the electronic device (100') may include a microphone (180) and an inner microphone, and the microphone (180) may be a microphone located relatively outside the body. According to one example, the electronic device (100') may acquire an audio signal including external noise through the microphone (180). According to one embodiment, the microphone (180) may be positioned in a direction opposite to the direction in which the speaker (170) emits sound.

[0157] According to the example described above, the electronic device (100') can identify the IoT device that the user points to among a plurality of IoT devices existing in the space where the user is located, and control the identified IoT device. In this case, the electronic device (100') can identify the device that the user points to by considering information obtained through an IMU sensor equipped in a wearable device, the user's location information, and the location of the IoT device within the home, thereby improving the accuracy of identification and increasing the user's satisfaction.

[0158] Meanwhile, according to the exemplary embodiments of the present disclosure, the various embodiments described above may be implemented as software comprising instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include a display device (e.g., a display device (A)) according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions. When instructions are executed by a processor, the processor may perform a function corresponding to the instructions directly or by using other components under the control of the processor. Instructions may include code provided or executed by a compiler or an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory" means only that the storage medium does not contain a signal and is tangible, and does not distinguish whether data is stored semi-permanently or temporarily in the storage medium.

[0159] Additionally, according to one embodiment, the method according to the various embodiments described above may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed online in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or provided on a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0160] Additionally, each component (e.g., module or program) according to the various embodiments described above may be composed of a single or multiple entities, and some of the aforementioned sub-components may be omitted, or other sub-components may be further included in the various embodiments. Generally or additionally, some components (e.g., module or program) may be integrated into a single entity to perform the functions performed by each of the respective components prior to integration in the same or similar manner. The operations performed by the module, program, or other components according to the various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations added.

[0161] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure.

Claims

1. In an electronic device, At least one processor including a processing circuit; and Memory that stores instructions and includes one or more storage media; and When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Acquire first location information within a map corresponding to each of at least one IoT (Internet of Things) device, and Based on sensing data acquired through an IMU (Inertial Measurement Unit) sensor included in a wearable device, the pointing direction of a user wearing the wearable device is identified, and Based on at least one of the user’s pointing direction, the first location information, and the second location information corresponding to the user within the map, the first IoT device corresponding to the user’s pointing direction among the at least one IoT device is identified, and An electronic device that controls the operation of the first IoT device identified above.

2. In Paragraph 1, The above first location information is, It includes information on areas corresponding to IoT devices within the above map, and The above instructions cause the electronic device, Among the above at least one IoT device, if a second IoT device corresponding to a first angle range based on the pointing direction is identified, information regarding a second area in which a first area corresponding to the identified second IoT device is expanded to a first size is obtained, and An electronic device that identifies a first IoT device corresponding to the user's pointing direction based on information regarding the second area obtained above.

3. In Paragraph 2, The above instructions cause the electronic device, An electronic device that calculates the first size based on location information corresponding to the second IoT device and second location information corresponding to the user.

4. In Paragraph 1, The above instructions cause the electronic device, Based on the above-mentioned acquired sensing data, a first angle corresponding to the first axis of the pointing direction is acquired, and If the first angle identified above is identified as being included in the second angle range based on the first axis, the IoT device corresponding to the second angle range among the at least one IoT device is identified, and An electronic device that identifies the first IoT device among the IoT devices corresponding to the second angle range.

5. In Paragraph 4, The above first location information is, Includes 3D (Dimensional) coordinate information, The above first axis is, It is an axis perpendicular to the ground, and The above instructions cause the electronic device, An electronic device that identifies an IoT device corresponding to the second angle range based on coordinate information corresponding to the first axis among coordinate information corresponding to each of the at least one IoT device.

6. In Paragraph 4, The above second angle range is, An electronic device identified based on the posture of a user wearing the above-mentioned wearable device.

7. In Paragraph 1, The second location information corresponding to the above user is, An electronic device obtained based on at least one of a BLE (Bluetooth Low Energy) beacon or a presence sensor located within a space corresponding to the map above.

8. In Paragraph 1, The above instructions cause the electronic device, An electronic device that identifies the pointing direction of a user wearing the wearable device when a trigger operation for controlling any one of the at least one IoT device is identified.

9. In Paragraph 1, The above instructions cause the electronic device, An electronic device that provides a User Interface (UI) that guides the user to change the pointing direction if, among the at least one IoT device, the IoT device corresponding to the user's pointing direction is not identified.

10. In Paragraph 1, The above instructions cause the electronic device, An electronic device that provides a guide UI for verifying an identified IoT device when an IoT device corresponding to the pointing direction of the user is identified.

11. In Paragraph 1, In addition to a communication interface; The above instructions cause the electronic device, After the first IoT device is identified, if the first gesture of the user is identified, the first function of the first IoT device corresponding to the first gesture is identified, and An electronic device that transmits a control signal to the first IoT device to perform the identified first function through the communication interface.

12. In a method of operating an electronic device, A step of obtaining first location information within a map corresponding to each of at least one IoT (Internet of Things) device; A step of identifying the pointing direction of a user wearing a wearable device based on sensing data acquired through an IMU (Inertial Measurement Unit) sensor included in the wearable device; A step of identifying a first IoT device corresponding to the user's pointing direction among the at least one IoT device based on at least one of the user's pointing direction, the first location information, and the second location information corresponding to the user within the map; and A method of operation comprising the step of controlling the operation of the first IoT device identified above.

13. In Paragraph 12, The above first location information is, It includes information on areas corresponding to IoT devices within the above map, and The step of identifying the first IoT device is, Among the at least one IoT device, if a second IoT device corresponding to a first angle range based on the pointing direction is identified, a step of obtaining information about a second area in which a first area corresponding to the identified second IoT device is expanded to a first size; and A method of operation comprising the step of identifying a first IoT device corresponding to the user's pointing direction based on information regarding the second area obtained above.

14. In Paragraph 13, The step of obtaining information regarding the second area above is, A method of operation comprising the step of calculating the first size based on location information corresponding to the second IoT device and second location information corresponding to the user.

15. In a storage medium storing computer-readable instructions, the instructions, when executed by at least one processor of an electronic device, cause the electronic device, Acquire first location information within a map corresponding to each of at least one IoT (Internet of Things) device, and Based on sensing data acquired through an IMU (Inertial Measurement Unit) sensor included in a wearable device, the pointing direction of a user wearing the wearable device is identified, and Based on at least one of the user’s pointing direction, the first location information, and the second location information corresponding to the user within the map, the first IoT device corresponding to the user’s pointing direction among the at least one IoT device is identified, and A storage medium that causes to control the operation of the first IoT device identified above.

Images