Location inference method of IoT device, server and electronic device supporting the method

Abstract

A server is disclosed, the server comprising a communication circuit, a memory, and a processor. The processor is configured to store identification information and location information of at least one external device in the memory, receive identification information of an Internet of Things (IoT) device near the at least one external device from the at least one external device using the communication circuit, and store the identification information of the IoT device in the memory as a database in association with the identification information and location information of the at least one external device.

Description

Technical Field

[0001] The present disclosure generally relates to position inference techniques performed by Internet of Things (IoT) devices. Background Art

[0002] IoT devices are being introduced to create an IoT environment in which there is a smart communication infrastructure in which multiple smart devices (e.g., homes, lights, speakers) are equipped with communication functions. Therefore, various operating platforms for the IoT environment are being studied. For example, currently voice recognition services may be combined with the IoT environment to support the operation of IoT devices that can respond to user verbal commands.

[0003] The above information is provided as background information only to assist in understanding the present disclosure. No determination has been made, and no assertion is made, as to whether any of the above may be applicable as prior art with respect to the present disclosure. Summary of the invention

[0004] Technical issues

[0005] When processing user verbal commands (also known as user speech input) related to the operation of an IoT device, the location information of the IoT device needs to be included in the user speech input so that the IoT device user speech input corresponding to the IoT device is clearly identified. That is, when issuing a verbal command, the user may have to manually specify the IoT device that should process the command at a specific location. In another scenario, when a new IoT device is intended to be registered in an IoT environment, the IoT device and its location need to be clearly identified so that when a user speech input for operating the new IoT device is generated later, the user speech input can be quickly processed by the appropriate IoT device. For this reason, during the registration process, the location information of the new IoT device is traditionally manually entered. Therefore, it may be inconvenient to mention or enter the location information of the corresponding IoT device whenever an IoT device is operated or registered.

[0006] Solution to the problem

[0007] According to one aspect of the present disclosure, a server supporting operation of an Internet of Things (IoT) environment may include a communication circuit, a memory, and a processor electrically connected to the communication circuit and the memory. The processor may be configured to store identification information and location information of at least one external device in the memory, receive identification information of an IoT device adjacent to the at least one external device from the at least one external device using the communication circuit, and store the identification information of the IoT device in association with the identification information and location information of the at least one external device in the memory as a database.

[0008] According to another aspect of the present disclosure, an electronic device supporting operation of an IoT environment may include a communication circuit and a processor electrically connected to the communication circuit. The processor may be configured to perform scanning of a specified signal using the communication circuit, identify at least one IoT device by detecting a specified signal sent from at least one IoT device adjacent to or near the electronic device during scanning of the specified signal, send a request for identification information to the identified at least one IoT device using the communication circuit to receive the identification information, send the identification information of the at least one IoT device to an external device using the communication circuit, and further send data or a signal for requesting the external device to store the identification information of the at least one IoT device associated with the identification information and location information of the electronic device.

[0009] According to another aspect of the present disclosure, an IoT device location inference method of a server supporting IoT environment operation may include storing identification information and location information of at least one external device in a memory, receiving identification information of an IoT device adjacent to or near the at least one external device from the at least one external device using a communication circuit, and storing the identification information of the IoT device in the memory as a database in association with the identification information and location information of the at least one external device.

[0010] Other aspects, advantages, and salient features of the disclosure will become apparent to those skilled in the art from the following detailed description, which, taken in conjunction with the annexed drawings, discloses various embodiments of the disclosure.

[0011] Advantageous Effects of the Invention

[0012] According to an embodiment, when a user is not required to specify location information of a desired IoT device when registering an IoT device, user convenience can be achieved.

[0013] According to the embodiment, even if the IoT device is not designated by the user, the IoT device can be clearly identified through registration of the IoT device.

[0014] Furthermore, various other effects and advantages directly or indirectly understood through the present disclosure can be provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent through the following description in conjunction with the accompanying drawings, in which:

[0016] Figure 1 is a diagram showing an example of an IoT environment according to one embodiment;

[0017] Figure 2 is a diagram showing components in an IoT environment according to an embodiment;

[0018] Figure 3a is a diagram showing a registration process of an IoT device according to an embodiment;

[0019] Figure 3b is a diagram showing a registration example of an IoT device according to an embodiment;

[0020] Figure 4a is a diagram illustrating processing of a user utterance according to an embodiment;

[0021] Figure 4b is a diagram showing a processing example of a user utterance according to an embodiment;

[0022] Figure 4c is a diagram showing a processing example of another user utterance according to another embodiment;

[0023] Figure 5 is a diagram showing a user interface according to an embodiment;

[0024] Figure 6 is a diagram showing an example of location information of an IoT device updated according to an embodiment;

[0025] Figure 7 is a diagram illustrating an example of a process of receiving a user utterance by a plurality of electronic devices according to an embodiment; and

[0026] Figure 8 is a diagram illustrating electronic devices in a network environment according to an embodiment. DETAILED DESCRIPTION

[0027] Hereinafter, various embodiments of the present disclosure will be described with reference to the accompanying drawings. However, those skilled in the art will recognize that various modifications, equivalents and / or substitutions may be made to the various embodiments described herein without departing from the scope and spirit of the present disclosure.

[0028] An aspect of the present disclosure is to at least solve the above-mentioned problems and / or disadvantages and to at least provide the following advantages. Therefore, one aspect of the present disclosure is to provide a method for inferring the position of an IoT device, which uses the position information of an electronic device that receives a user speech input to infer the position of the IoT device associated with the user speech input. An electronic device and a server supporting the electronic device are also disclosed.

[0029] Figure 1 is a diagram showing an example of an IoT environment according to an embodiment. Figure 2 is a diagram illustrating components of an IoT environment according to an embodiment.

[0030] Reference Figure 1 and Figure 2According to the embodiment, the IoT environment 1000 can be created at a specific location 10 (e.g., smart home, smart grid, smart factory, etc.) where at least one IoT device 200 is located. The IoT environment 1000 can support the operation of at least one IoT device 200. For example, the IoT environment 1000 can support centralized monitoring or control based on user speech input to at least one IoT device 200. In this regard, the IoT environment 1000 may include at least one electronic device 100, at least one IoT device 200, an intelligent server 300, and a network 400. In one embodiment, the IoT environment 1000 may omit at least one of the above components or may also include another component not shown. For example, the IoT environment 1000 may also include a mobile communication terminal (not shown) owned by a user (e.g., a smart phone, etc.). In this example, the mobile communication terminal may include a dedicated application for accessing the IoT environment 1000. According to this embodiment, the mobile communication terminal may execute a dedicated application under the control of the user to provide monitoring information about at least one IoT device 200 in the IoT environment 1000, or perform information synchronization between at least one or more IoT devices 200. Alternatively, the IoT environment 1000 may further include an account server 500 that manages user account information of various users (e.g., Samsung account information, etc.). According to an exemplary embodiment, the account server 500 may authenticate the user account information input by the user on the mobile communication terminal, and then may support the registration process of at least one IoT device 200.

[0031] According to an embodiment, at least one electronic device 100, at least one IoT device 200, and the smart server 300 may interact with each other via a network 400 (e.g., a LAN, a WAN, the Internet, a telephone network, an ad hoc network, etc.). For example, when a new IoT device 200 is located in the location 10 for the first time or initially, the at least one electronic device 100 may recognize the existence of a new IoT device 200_1 (e.g., an IoT device that has not been previously registered in the IoT environment 1000). The identification may be performed by the at least one electronic device 100 using a communication circuit installed therein to perform wired or wireless communication to access the network 400. Therefore, the at least one electronic device 100 may send information associated with the registration of the new IoT device 200_1 to the smart server 300. Alternatively, the at least one electronic device 100 may receive a user speech input located in the location 10 associated with the operation of the at least one IoT device 200 (e.g., an IoT device that was previously registered in the IoT environment 1000), and may access the network 400 by performing wired or wireless communication. Therefore, at least one electronic device 100 may send voice data of user speech input to the smart server 300. In response, the smart server 300 may use the information received from the at least one electronic device 100 to perform a registration process of a new IoT device 200_1, or may process the user speech input based on the voice data. In these operations, the smart server 300 may refer to the location information of the specific electronic device 100 that provides the information or voice data. For example, when registering a new IoT device 200_1 in the IoT environment 1000, the smart server 300 may register the new IoT device 200_1 in association with the location information of the specific electronic device 100. Alternatively, when processing the user speech input, the smart server 300 may identify the IoT device 200 associated with the location information of the specific electronic device 100 to identify the IoT device 200 corresponding to the user speech input. Hereinafter, an example of inferring the location of the IoT device 200 based on the location information of at least one electronic device 100 and the functional operation of the components in the IoT environment 1000 that implement it will be described.

[0032] In an embodiment, at least one electronic device 100 may operate as a hub device or an artificial intelligence (AI) device that relays information between a user and an intelligent server 300 (or at least one IoT device 200). To this end, at least one electronic device 100 may be physically or logically located within a location 10 where an IoT environment 1000 is created. According to an embodiment, location information about at least one electronic device 100 in location 10 may be input into the above-mentioned mobile communication terminal owned by the user. For example, the mobile communication terminal may output a specified user interface by executing a dedicated application capable of accessing the IoT environment 1000 under the control of the user, and may receive identification information and location information about at least one electronic device 100 located in location 10, obtained from the user through the user interface. The mobile communication terminal may communicate with the intelligent server 300, and send the received identification information and location information of at least one electronic device 100 to the intelligent server 300.

[0033] In an embodiment, the first electronic device 100_1 arranged in the first space 11 of at least one space in the location 10 may support registration of a new IoT device 200_1 to also be located in the first space 11. For example, the first electronic device 100_1 may identify the presence of the new IoT device 200_1 by performing a signal scan to detect a signal transmitted from the new IoT device 200_1, and may obtain information from the identified new IoT device 200_1 and transmit the information to the smart server 300. In this example, the information obtained from the new IoT device 200_1 may be referred to as information for a registration process of the new IoT device 200_1 for the IoT environment 1000 or the smart server 300. Alternatively or additionally, the second electronic device 100_2 located in the second space 12 of the location 10 may receive a user speech input 20 associated with an operation of the IoT device 200_2 located in the second space 12. The second electronic device 100_2 may transmit voice data of the user speech input 20 to the smart server 300 to process the user speech input 20. Regarding the above operations, at least one electronic device 100 may include a microphone 110, a first communication circuit 120, a first memory 130, a first processor 140, and a speaker 150. In one embodiment, at least one electronic device 100 may omit at least one of the above components or may further include another component not shown. For example, at least one electronic device 100 may further include a battery for supplying power to the above components and / or a display for outputting various contents. Alternatively, at least one electronic device 100 may further include a display for outputting a reference to the above components. Figure 8 One or more components of electronic device 801 are described.

[0034] The microphone 110 may receive voice signals corresponding to various user speech inputs. In one embodiment, the microphone 110 may operate in a normally open state to receive voice signals. Alternatively, the microphone 110 may operate after being activated, wherein the activation is caused by a user input to a hardware button of the electronic device 100. According to an embodiment, the microphone 110 may be implemented as one or more microphones, and at least a portion of the microphones may be exposed to the outside of the electronic device 100 to better detect voice signals.

[0035] The first communication circuit 120 may support communication between the electronic device 100 and at least one external device (e.g., at least one IoT device 200, the smart server 300, etc.). For example, the first communication circuit 120 may transmit or receive data or signals associated with the operation of the IoT environment 1000 by performing wired or wireless communication with at least one external device according to a protocol for accessing the network 400.

[0036] The first memory 130 may store at least one command associated with functional operation control of a component of the electronic device 100. Alternatively and / or additionally, the first memory 130 may store at least one data associated with the operation of a component of the electronic device 100. For example, the first memory 130 may store identification information of the electronic device 100 (e.g., model information, version information, operating country information, OS information, etc.) and / or may store location information of the electronic device 100 located in the location 10 (e.g., information about the space in which the electronic device 100 is located within the location 10). Alternatively and / or additionally, the first memory 130 may store query data to be output through the speaker 150 during the registration process of the IoT device 200.

[0037] The first processor 140 may be implemented with at least one of a central processing unit, an application processor, or a communication processor, and may control components of the electronic device 100. For example, the first processor 140 may be electrically or operably connected to a component of the electronic device 100 to transmit at least one command associated with a functional operation to the component or perform various arithmetic operations, data processing, etc. The first processor 140 may include a microprocessor or any suitable type of processing circuit, such as one or more general-purpose processors (e.g., ARM-based processors), digital signal processors (DSPs), programmable logic devices (PLDs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), graphics processing units (GPUs), video card controllers, etc. In addition, when a general-purpose computer accesses the code, it will be recognized that in order to implement the processing shown here, the execution of the code converts the general-purpose computer into a special-purpose computer to perform the processing shown here. Certain functions and steps provided in the figure may be implemented with hardware, software, or a combination of both, and may be executed in whole or in part within the programming instructions of a computer. Any claim element herein shall not be subject to 35 U.S.C. §112 (f) unless the phrase "means" is used to expressly recite the element. Furthermore, the skilled artisan understands and appreciates that a "processor" or "microprocessor" in the claimed disclosure may be hardware. The appended claims are to be construed consistent with 35 USC §101 pursuant to the broadest reasonable interpretation.

[0038] In an embodiment, when a user speech input is received through the microphone 110, the first processor 140 may control the first communication circuit 120 to send the voice data of the user speech input to the intelligent server 300. Alternatively and / or additionally, the first processor 140 may also control the first communication circuit 120 to perform signal scanning in real time or periodically. The first processor 140 may identify the presence of a new IoT device 200_1 based on signal scanning by detecting a specific signal sent from a new IoT device 200_1 adjacent to or near the electronic device 100_1. The first processor 140 may send a request for first identification information (e.g., model information, version information, operating country information, OS information, etc.) to the identified new IoT device 200_1 to obtain the first identification information, and may send the obtained first identification information to the intelligent server 300 to register the new IoT device 200_1. According to an embodiment, in the operation of sending the first identification information of the new IoT device 200_1, the first processor 140 may also send the second identification information (for example, model information, version information, operating country information, OS information, etc.) and the location information of the electronic device 100_1 to the intelligent server 300.

[0039] In another embodiment, the second identification information and location information of at least one electronic device 100 may be provided to the intelligent server 300 in advance (for example, when the at least one electronic device 100 is first located at the location 10). In this way, the transmission of the second identification information and location information of the at least one electronic device 100 is independent of the transmission of the first identification information of the new IoT device 200_1. For example, when the at least one electronic device 100 is located at the location 10, the user may input the second identification information about the at least one electronic device 100 and the location information of the at least one electronic device 100 in the location 10 into the mobile communication terminal owned by the user, and the mobile communication terminal may send the received second identification information and the received location information to the intelligent server 300. Therefore, when the first processor 140 sends the first identification information of the new IoT device 200_1, the information about the at least one electronic device 100 located in the location 10 (for example, the second identification information and location information) may have been stored in the intelligent server 300.

[0040] According to an embodiment, when the first identification information of the new IoT device 200_1 is transmitted, the first processor 140 may transmit data (e.g., a message) or a signal requesting the smart server 300 to create an association between the first identification information and the second identification information and the location information of the electronic device 100. Doing so may enable the smart server 300 to identify the specific IoT device 200 and the corresponding electronic device 100 associated with the network. Therefore, when the electronic device 100 later transmits voice data of the user's speech input to the smart server 300, the smart server 300 may use the location information of the identified electronic device 100 to infer the location of the identified IoT device 200.

[0041] The speaker 150 may output designated voice data. For example, when the presence of a new IoT device 200_1 adjacent to or near the electronic device 100 is identified through signal scanning, the speaker 150 may output query data in the form of voice output, where the voice output is generated using information stored in the first memory 130. Alternatively, the speaker 150 may output voice data provided from the smart server 300 to the electronic device 100. An example of such voice data is a response to data or a signal sent from the electronic device 100.

[0042] In an embodiment, at least one IoT device 200 may be various IoT devices, such as security devices, industrial devices, or sensors. In short, at least one IoT device 200 may be any device based on IoT technology. According to an embodiment, at least one IoT device 200 may be registered in the IoT environment 1000 or the intelligent server 300 through a series of registration processes described later, and during the registration process, at the request of the electronic device 100, its first identification information (e.g., model information, version information, operating country information, OS information, etc.) may be sent to the electronic device 100. Alternatively and / or in addition, at least one IoT device 200 may send various operation information (e.g., status information, function information, etc.) to the intelligent server 300, at least one electronic device 100, or the user's mobile communication terminal in batches or in real time.

[0043] In an embodiment, the smart server 300 may jointly manage at least one IoT device 200 registered in the IoT environment 1000 or the smart server 300. For example, the smart server 300 may perform a registration process on a new IoT device 200_1 located on the IoT environment 1000 (i.e., within the location 10 where the IoT environment 1000 is created) by interacting with at least one electronic device 100. Alternatively and / or additionally, the smart server 300 may receive voice data of a user's speech input associated with the operation of at least one registered IoT device 200 from at least one electronic device 100, and may process the user's speech input by analyzing the voice data. To this end, the smart server 300 may include a second communication circuit 310, a second memory 320, and a second processor 330.

[0044] The second communication circuit 310 can access the network 400 by performing wired or wireless communication to send or receive data or signals between the smart server 300 and at least one external device (e.g., at least one electronic device 100 and / or at least one IoT device 200).

[0045] The second memory 320 may store information received from at least one electronic device 100 or at least one IoT device 200. For example, the second memory 320 may store various operation information (e.g., status information, function information, etc.) sent from at least one IoT device 200. Alternatively and / or additionally, the second memory 320 may store second identification information (e.g., model information, version information, operating country information, OS information, etc.) and corresponding location information of at least one electronic device 100, which are sent from at least one electronic device 100 or from a mobile communication terminal owned by a user. According to one embodiment, when a specific electronic device 100 is located on the IoT environment 1000 for the first time (i.e., within the location 10 where the IoT environment 1000 is created), the second identification information and location information of the electronic device 100 may be input by the user into the mobile communication terminal, and then may be provided to the intelligent server 300 from the mobile communication terminal. Alternatively, whenever a particular electronic device 100 receives user speech input or recognizes the existence of an adjacent new IoT device 200_1, the second identification information and location information of the electronic device 100 can be provided from the electronic device 100 to the intelligent server 300 together with the voice data of the user speech input and / or the first identification information obtained from the new IoT device 2001.

[0046] The second processor 330 may be implemented with at least one of a central processing unit, an application processor, or a communication processor and may control the components of the intelligent server 300. For example, the second processor 330 may be electrically or operably connected to the components of the intelligent server 300 to transmit at least one command associated with a functional operation to the components or perform various arithmetic operations, data processing, etc. The second processor 330 may include a microprocessor or any suitable type of processing circuit, such as one or more general-purpose processors (e.g., ARM-based processors), digital signal processors (DSPs), programmable logic devices (PLDs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), graphics processing units (GPUs), graphics card controllers, etc. In addition, when a general-purpose computer accesses the code, it will be recognized that in order to implement the processing shown here, the execution of the code converts the general-purpose computer into a special-purpose computer to perform the processing shown here. Certain functions and steps provided in the figure may be implemented with hardware, software, or a combination of both, and may be executed in whole or in part within the programming instructions of a computer. No claim element herein may be construed under 35 U.S.C. §112 (f) unless the element is explicitly recited using the phrase "means for". Furthermore, the skilled artisan understands and appreciates that a "processor" or "microprocessor" in the claimed disclosure may be hardware. The appended claims are to be construed consistent with 35 USC §101 pursuant to the broadest reasonable interpretation.

[0047] In an embodiment, the second processor 330 may establish the database 322 using the information stored in the second memory 320. For example, when the second processor 330 receives the first identification information of a new IoT device 200_1 adjacent to or near the specific electronic device 100_1 (or existing in the same space of the location 10) from the electronic device 100_1, the second processor 330 may store the first identification information of the new IoT device 200_1 in the database 322 in association with the second identification information and location information of the electronic device 100_1 previously stored in the second memory 320. In this regard, since the electronic device 100_1 obtains the first identification information by recognizing the presence of the new IoT device 200_1 based on the signal scan, the second processor 330 may infer that the electronic device 100_1 and the new IoT device 200_1 exist in the same space. Based on the inference, the second processor 330 may determine that the location of the new IoT device 200_1 is adjacent to or near the user's mobile communication terminal, and may store the association between the location of the new IoT device 200_1 and the previously stored location information of the electronic device 100_1 in the second memory 320 so that the location information of the electronic device 100_1 may be used as the location information of the new IoT device 200_1. That is, when the second processor 330 receives the first identification information of the new IoT device 200_1 adjacent to or near the electronic device 100_1 from the electronic device 100_1, the second processor 330 may store the second identification information and the location information of the electronic device 100_1 provided together with the first identification information in the database 322 in association with the first identification information.

[0048] According to one embodiment, in an operation of storing the first identification information of the new IoT device 200_1 in association with the second identification information and location information of the electronic device 100_1, the second processor 330 may modify and store at least a portion of the first identification information. For example, the second processor 330 may combine the model information of the new IoT device 200_1 included in the first identification information with the location information of the electronic device 100_1, and may then store the result of the combination. In a more specific example, when the model information of the new IoT device 200_1 represents an air purifier and the location information of the electronic device 100_1 represents a living room that is part of location 10, when location 10 is a smart home, the second processor 330 may change the model information of the new IoT device 200_1 to "air purifier for the living room", and may store the changed information in the database 322.

[0049] According to an embodiment, the second processor 330 may analyze and process voice data of user speech input received from the electronic device 100, in particular, the electronic device 100_1. For example, the user speech input may include a command to operate the IoT device 200_1 adjacent to or near the electronic device 100_1 (or present in the same space in the location 10). With respect to analyzing and processing the voice data of the user speech input, the second processor 330 may include an automatic speech recognition (ASR) module 331 and a natural language understanding (NLU) module 332. According to different embodiments, the ASR module 331 and the NLU module 332 may be independent of each other or may be at least partially integrated.

[0050] The ASR module 331 can recognize the voice data received from the electronic device 100_1 and can convert the voice data into text data. For example, the ASR module 331 can perform such conversion using an acoustic model including information about speech or pronunciation or a language model including at least one unit phoneme information and a combination of unit phonemes.

[0051] The NLU module 332 may receive text data from the ASR module 331, and may derive the intent of the user's utterance input associated with the voice data based on the text data. For example, the NLU module 332 may divide the text data into grammatical units (e.g., words, phrases, or morphemes), and may analyze the grammatical elements or language features of each unit to determine the meaning of the text data, and thus may derive the intent of the user's utterance input associated with the voice data.

[0052] According to an embodiment, the second processor 330 may identify the IoT device 200 corresponding to the intent of the user utterance input derived by the NLU module 332 by referring to the database 322 built in the second memory 320. For example, the second processor 330 may identify the specific electronic device 100 that sends the voice data using the database 322 based on the second identification information received together with the voice data of the user utterance input, and may determine that the IoT device 200 stored in association with the identified specific electronic device 100 is the IoT device 200 corresponding to the intent of the derived user utterance input. In addition, since the identified specific electronic device 100 is associated with the IoT device 200 corresponding to the intent of the user utterance input, the second processor 330 may infer that the specific electronic device 100 and the IoT device 200 are adjacent to each other or exist in the same space to determine that the location of the IoT device 200 is the same as or similar to the location of the specific electronic device 100. In other words, the second processor 330 may infer that the location information of the IoT device 200 may correspond to the location information of the specific electronic device 100. In an embodiment, the second processor 330 may generate an operation sequence or a control signal and send the operation sequence or the control signal to the IoT device 200 so that the IoT device 200 whose position is inferred performs an operation corresponding to the derived intention of the user speech input.

[0053] As described above, the second processor 330 may associate information of the electronic device 100_1 located at position 10 (e.g., second identification information and location information) with information of a new IoT device 200_1 adjacent to or near the electronic device 100_1 (e.g., first identification information). These information and associations may be stored in the database 322. Furthermore, when a user speech input associated with an operation of the new IoT device 200_1 occurs later, the second processor 330 may determine the location of the new IoT device 200_1 by referring to the location information of the electronic device 100_1 stored in association with the new IoT device 200_1 in the database 322. Therefore, even if there are no parameters indicating identification information or location information of the IoT device 200_1 in the user speech input, the second processor 330 may identify the IoT device 200_1 associated with the electronic device 100_1 providing voice data of the user speech input to determine the IoT device 200_1, and may send an operation sequence or control signal corresponding to the intention of the user speech input. The above operations may also be applied to Figure 1 The IoT device 200_2 and the electronic device 100_2 are shown. For example, when the electronic device 100_2 records the user speech input 20, by performing the above operations, the smart server 300 can infer that the IoT device 200_2 is the intended target of the user speech input 20.

[0054] Figure 3a is a diagram illustrating a registration process of an IoT device according to an embodiment. Figure 3b is a diagram showing a registration example of an IoT device according to an embodiment.

[0055] Reference Figure 3a and Figure 3b , in operation 301, located at Figure 1 The electronic device 100 in the partial space 11 in the location 10 (e.g., smart home, smart grid, smart factory, etc.) can perform a signal scan, wherein a Figure 1 or Figure 2 IoT environment 1000. For example, the electronic device 100_1 Figure 2 The first processor 140 can use the electronic device 100_1 Figure 2 The first communication circuit 120 performs signal scanning. The signal scanning can be performed based on a schedule, in real time, or periodically.

[0056] In operation 303, when a power voltage is applied thereto, the IoT device 200_1 (e.g., an IoT device that has not been previously registered in the IoT environment 1000) is positioned adjacent to or near the electronic device 100_1 (or located in the space 11 where the electronic device 100_1 is located) may transmit a designated signal (e.g., Bluetooth Low Energy (BLE), ZigBee, etc.) in real time or periodically. According to an embodiment, it can be understood that the signal transmission of the IoT device 200_1 is an operation of transmitting a signal to the electronic device 100_1 so that the electronic device 100_1 recognizes the presence of the IoT device 200_1.

[0057] In operations 305 and 307, the first processor 140 of the electronic device 100_1 may identify an IoT device 200_1 that is adjacent or in the same space 11, and may send a request for information associated with the IoT device 200_1 from the identified IoT device 200_1. For example, when performing a signal scan using the first communication circuit 120, the first processor 140 may identify the presence of the IoT device 200_1 by receiving a signal sent from the IoT device 200_1, and may send a request for first identification information (e.g., model information, version information, operating country information, OS information, etc.) of the IoT device 200_1 to collect information of the identified IoT device 200_1.

[0058] In operations 309 and 311, the first processor 140 of the electronic device 100_1 may receive the first identification information from the IoT device 200 and may ask the user to register (or set up) the IoT device 200_1. For example, the first processor 140 may use a processor installed in the electronic device 100_1 to register (or set up) the IoT device 200_1. Figure 2The speaker 150 outputs the query data in the form of voice output. In this operation, the first processor 140 may include the first identification information and the second identification information of the electronic device 100_1 and at least a portion of the location information in the query data. For example, the first processor 140 may include at least a portion of the first identification information of the IoT device 200_1 received from the IoT device 200_1 (e.g., model information A) and the location information of the electronic device 100_1 stored in the first memory 130 (e.g., living room) in the query data, and then may output the query data as a voice output (e.g., "A new device 'A' has been identified. Do you want to register the location information as the living room?"). In another example, when the location information of the electronic device 100_1 does not exist in the first memory 130, the first processor 140 may request the pre-stored location information of the electronic device 100_1 (e.g., stored in and provided from the user's mobile communication terminal) to the intelligent server 300 to obtain the location information of the electronic device 100_1. Then, the first processor 140 may include the obtained location information of the electronic device 100_1 and at least a portion of the first identification information (eg, model information A) of the IoT device 200_1 in the query data and output the query data.

[0059] When a response speech input for approving the registration of the IoT device 200_1 is received from the user after the query data is output, in operation 313, the first processor 140 of the electronic device 100 may send the first identification information and the second identification information (e.g., model information, version information, operating country information, OS information, etc.) of the IoT device 200_1 received from the IoT device 200_1 and the location information of the electronic device 100 to the smart server 300, which jointly manages at least one IoT device 200 in the IoT environment 1000. According to one embodiment, the first processor 140 may exclude the transmission of at least one of the second identification information and the location information of the electronic device 100_1. When the electronic device 100_1 is located in the IoT environment 1000 for the first time (or in the partial space 11 of the location 10 where the IoT environment 1000 is created), this may occur when the second identification information and / or the location information of the electronic device 100_1 is previously input into the mobile communication terminal of the user. The second identification information and / or location information of the electronic device 100_1 may be sent from the mobile communication terminal to the smart server 300. Therefore, when the IoT device 200_1 is registered, the smart server 300 may have stored the second identification information and / or location information of the electronic device 100_1 in the smart server 300. Figure 2 in the second memory 320.

[0060] In operation 315, the smart server 300 Figure 2The second processor 330 may store the second identification information and location information of the electronic device 100_1 in association with the first identification information of the IoT device 200_1 (e.g., storing both in a mapping or table form). For example, the second processor 330 may store the second identification information and location information of the electronic device 100_1, which are provided together with the first identification information of the IoT device 200_1 or have been stored in the second memory 320. According to one embodiment, Figure 2 The second processor 330 may simply store the second identification information and location information of the electronic device 100_1 in association with the first identification information of the IoT device 200_1 in the database 322 of the IoT device 200_1, and may refer to the associated location information of the electronic device 100_1 when determining the location of the IoT. Alternatively, the second processor 330 may generate separate location information of the IoT device 200_1 based on the location information of the electronic device 100_1, and may store the first identification information and location information of the IoT device 200_1 in association with the second identification information and location information of the electronic device 100_1.

[0061] According to one embodiment, the registration process of the IoT device 200_1 may include an authentication process for the account registered by the IoT device 200_1. In this regard, in operation 317, the mobile communication terminal owned by the user may request a specified authorization code from the account server 500 that jointly manages user account information associated with the operation of the IoT environment 1000. For example, the mobile communication terminal may output a specified user interface (e.g., an execution screen of a dedicated application for accessing the IoT environment 1000) in response to user control and may send the user account information (e.g., Samsung account information, etc.) input by the user to the user interface, and to the account server 500 to request the specified authorization code.

[0062] In operation 319, the account server 500 may issue an authorization code based on the authentication of the user account information provided from the mobile communication terminal. For example, the account server 500 may determine the validity of the user account information by referring to the publication history of the user account information, whether the user account information is authentic, the validity period of the user account information, etc. When it is determined (or authenticated) that the user account information is valid, the account server 500 may generate an authorization code. The account server 500 may send the generated authorization code to the mobile communication terminal, and may request that the authorization code be sent to the IoT device 200_1 as the IoT device being registered.

[0063] In operation 321, the IoT device 200_1 may receive an authorization code from the mobile communication terminal or the account server 500, may send the authorization code to the smart server 300, and complete the registration of the IoT environment 1000 (or register in the smart server 300). According to an embodiment, when the smart server 300 approves the registration of the IoT device 200_1, communication between the smart server 300 and the IoT device 200_1 may be established.

[0064] Figure 4a is a diagram illustrating processing of a user utterance according to an embodiment. Figure 4b and Figure 4c is a diagram illustrating examples of processing various user utterances according to certain embodiments.

[0065] Reference Figure 4a , Figure 4b and Figure 4c In operation 401, at the creation Figure 1 or 2 IoT environments 1000 Figure 1 The electronic device 100 in a location 10 (e.g., a smart home, a smart grid, a smart factory, etc.) can receive user speech input (e.g., Figure 4b to Figure 4c As shown, “clean it from here” or “turn off the light”) to operate at least one IoT device 200 registered in the IoT environment 1000 (or the smart server 300). For example, the first processor ( Figure 2 140) can be connected via the installed microphone ( Figure 2 The microphone 110 receives a user speech input for operating at least one IoT device 200, which is adjacent to or near the electronic device 100 or exists in the same space (e.g., space 11). According to an embodiment, the user speech input received through the microphone 110 may not include a parameter for identifying or locating the desired IoT device 200. Alternatively, the user speech input may include a parameter indicating the location of the desired IoT device 200 (e.g., "Turn off the lights in the living room.").

[0066] In operation 403, the first processor 140 of the electronic device 100 may use Figure 2 The first communication circuit 120 in the embodiment sends the voice data input by the user's speech to the intelligent server 300, and the intelligent server 300 uses Figure 2 The first communication circuit 120 in the IoT environment 1000 jointly manages at least one IoT device 200 in the IoT environment 1000. According to an embodiment, in order to support the intelligent server 300 to recognize the voice data with high reliability, the first processor 140 can pre-process the voice data before transmission (for example, echo cancellation, background noise suppression, volume control, equalization, endpoint detection, etc.).

[0067] In operation 405, the second processor ( Figure 2 The second processor 330 may analyze the voice data received from the electronic device 100. For example, the second processor 330 may convert the voice data into text data using an acoustic model or a language model. The second processor 330 may then derive the intention of the user's speech input associated with the voice data by determining the meaning of the text data as described above.

[0068] In operation 407, the second processor 330 of the smart server 300 may identify the IoT device 200 corresponding to the derived intention of the user speech input, and may determine the location of the IoT device 200. In this operation, the second processor 330 may Figure 2 The second memory 320 refers to Figure 2 The database 322 is established by a registration process of at least one IoT device 200 for the IoT environment 1000 (or in the smart server 300). According to an embodiment, the database 322 can be established by associating the second identification information (e.g., model information, version information, operating country information, OS information, etc.) and location information of the electronic device 100 supporting the registration of the corresponding IoT device 200 with the first identification information (e.g., model information, version information, operating country information, OS information, etc.) of at least one IoT device 200 to be registered in the IoT environment 1000. According to another embodiment, the database 322 can be constructed by associating the second identification information and location information of the electronic device 100 with the first identification information and location information of the IoT device 200 (e.g., separate location information of the IoT device 200 generated based on the location information of the electronic device 100).

[0069] According to an embodiment, the second processor 330 may determine that the electronic device 100 and the user that send the voice data exist in, for example, the space 11, and may identify the electronic device 100 that sends the voice data in the database 322. In addition, the second processor 330 may determine that the user speech input is an input associated with the operation of the IoT device 200, which also exists in, for example, the space 11 adjacent to the user or the electronic device 100, and may identify that the IoT device 200 stored in association with the electronic device 100 identified in the database 322 is the IoT device 200 corresponding to the intention of the user speech input. As described above, the database 322 stored in the second memory 320 of the intelligent server 300 includes the location information of the electronic device 100 stored in association with the identified IoT device 200. Alternatively, the database 322 includes the location information of the IoT device 200 that is the same as or corresponds to the location information of the electronic device 100. Therefore, the second processor 330 may identify and / or locate the IoT device 200 with reference to the database 322.

[0070] According to an embodiment, the electronic device 100 that transmits the voice data of the user's speech input and the IoT device 200 corresponding to the intention of the user's speech input may not be associated with each other. For example, when the IoT device 200 corresponding to the intention of the user's speech input is Figure 4b When the mobile IoT device 200 shown is registered, the electronic device 100 supporting the registration of the mobile IoT device 200 may be different from the electronic device 100 that sends the voice data, and therefore, the electronic device 100 that sends the voice data and the IoT device 200 corresponding to the intention of the user's speech input may not be associated with each other. In this regard, when the second processor 330 receives the voice data of the user's speech input from the specific electronic device 100, the second processor 330 may identify the attributes (e.g., functions) of the IoT device 200 stored in association with the specific electronic device 100 by referring to the database 322. When the attributes of the IoT device 200 associated with the specific electronic device 100 do not correspond to the derived intention of the user's speech input, the second processor 330 may identify the attributes of at least one IoT device 200 associated with another electronic device 100 in the database 322. In an embodiment, when the attribute of the IoT device 200 stored in association with the other electronic device 100 corresponds to the intention of the user's speech input, the second processor 330 may generate voice type feedback data including the location information of the other electronic device (e.g., "The expected device is currently in the kitchen. Is your intention referring to that device?"). Figure 4b As shown, the second processor 330 may determine that the user speech input includes the word "clean". The second processor 330 may then determine the attribute of the IoT device 200 corresponding to the word "clean". A sweeping robot may have an attribute corresponding to "clean", but a smart light bulb may not have an attribute corresponding to "clean". The second processor 330 may then identify the IoT device 200 based on the attribute. The second processor 330 may then send feedback data to the specific electronic device 100 that sent the voice data of the user speech input. The second processor 330 may identify the IoT device 200 corresponding to the intention of the user speech input based on the user response speech provided from the specific electronic device 100 that sent the voice data of the user speech input, and may determine the location of the identified IoT device 200 based on the location information of the electronic device 100 stored in association with the identified IoT device 200 or the location information of the identified IoT device 200 itself.

[0071] In operation 409 , the second processor 330 may generate an operation sequence or a control signal and send the operation sequence or the control signal to the corresponding IoT device 200 , so that the IoT device 200 that determines the location performs an operation corresponding to the intention of the user speech input.

[0072] According to an embodiment, the above operations 405 to 409 may be performed by one of the at least one electronic device 100 instead of the smart server. In this regard, the at least one electronic device 100 may include a plurality of electronic devices 100 located in each space in the location 10. One of the plurality of electronic devices 100 may serve as a master electronic device 100 to jointly manage another at least one electronic device 100. In an embodiment, the master electronic device 100 may perform at least some functional operations of the above smart server 300. For example, the master electronic device 100 may be configured to perform the above-described operations by using Figure 2 The ASR module 331 and Figure 2 The NLU module 332 of the main electronic device 100 analyzes the voice data of the user speech input sent from at least another electronic device 100, so that the main electronic device 100 can derive the intention of the user speech input. In addition, the main electronic device 100 can identify the specific electronic device 100 that sends the voice data of the user speech input and the IoT device 200 corresponding to the intention of the user speech input by using a database established in the same or similar manner as the database 322 of the intelligent server 300. In addition, the main electronic device 100 can determine the location of the IoT device 200 corresponding to the intention of the user speech input based on the location information of the electronic device 100 stored in association with the corresponding IoT device 200 in the database. Alternatively, the main electronic device 100 can determine the location of the IoT device 200 based on the location information of the IoT device 200 itself (for example, location information generated based on the location information of the electronic device 100 stored in association with the corresponding IoT device 200).

[0073] Figure 5 is a diagram illustrating a user interface according to an embodiment.

[0074] Reference Figure 5 According to an embodiment, the mobile communication terminal 600 owned by the user may execute a function for accessing the Figure 1 or Figure 2 The dedicated application of the IoT environment 1000 may be configured to output a user interface 610 of the dedicated application. According to an embodiment, the user interface 610 may include a main screen of the dedicated application and various other screens that can be navigated from the main screen.

[0075] In an embodiment, the mobile communication terminal 600 may output a user interface 610 including user interface (UI) elements of at least one electronic device 100 and at least one IoT device 200 in the list. These UI elements may be based on information input from the user. Figure 2 These UI elements can be based on information about at least one electronic device 100 in the Figure 2Information of at least one IoT device 200 provided by the intelligent server 300.

[0076] According to an embodiment, at least one electronic device 100 and at least one IoT device 200 may be grouped based on location and displayed on the user interface 610. For example, location information 611 associated with at least one electronic device 100 or at least one IoT device 200 may be displayed as an upper portion on the user interface 610; identification information 613 of at least one electronic device 100 and / or at least one IoT device 200 associated with location information 611 may be displayed as a lower portion corresponding to the upper portion. In a more specific example, when the electronic device 100 is located as Figure 1 In the case of "living room" which is part of the location 10 of the smart server 300, the location information corresponding to the "living room" may be displayed as the upper part; and the "speaker" which is the identification information of the electronic device 100 may be displayed as the lower part. In an embodiment, the location information displayed as the upper part may be stored in the smart server 300. Figure 2 The location information of the electronic device 100 in the database 322 or the location information of the IoT device 200 itself, or the location information of the electronic device 100 stored in association with the IoT device 200. According to an embodiment, the identification information of at least one electronic device 100 and / or at least one IoT device 200 displayed at the bottom may be displayed in combination with the location information of the upper part (e.g., a living room speaker).

[0077] According to an embodiment, the user interface 610 may be updated according to the registration of the new IoT device 200_1 in the IoT environment 1000 or the smart server 300. For example, when the new IoT device 200_1 is located in the partial space within the location 10, when the electronic device 100 located in the partial space recognizes the new IoT device 200_1, the electronic device 100 and the new IoT device 200_1 may be associated with each other, and then the association information may be stored in the database 322 of the smart server 300. Subsequently, the mobile communication terminal may update the user interface 610 by receiving information about the new IoT device 200_1 registered in the IoT environment 1000 from the smart server 300. For example, when the new IoT device 200_1 is located in the "living room" of the location 10 and is registered in the IoT environment 1000, "cleaner", which is the identification information 615 of the new IoT device 200_1, may be displayed at the lower part of the user interface 610, and the label "living room" is displayed at the corresponding upper part. As described above, the identification information of the new IoT device 200_1 may be displayed in combination with the location information shown in the upper portion (eg, living room cleaner).

[0078] Figure 6is a diagram illustrating an example of updated location information of an IoT device according to an embodiment.

[0079] Reference Figure 6 According to the embodiment, the IoT device 200 may be capable of Figure 1 A mobile IoT device 200 moving in various spaces of location 10, wherein a Figure 1 or Figure 2 IoT environment 1000. In this case, the information associated with the mobile IoT device 200 may be included in Figure 1 or Figure 2 The intelligent server 300 Figure 2 For example, the mobile IoT device 200 can be associated with the first electronic device 100_1 in the registered space 15 supporting the first IoT environment 1000 (or in the smart server 300), and the result of the association can then be stored in the database 322. Then, the mobile IoT device 200 can be associated with the second electronic device 1002 in another space 16 adjacent to the IoT device 200 after it moves, and the association result can then be stored. In this regard, the mobile IoT device 200 can send a designated signal (e.g., BLE or ZigBee) at the beginning of the movement, and the second electronic device 100×2 that detects the designated signal through signal scanning can obtain the first identification information of the mobile IoT device 200 and send the first identification information to the smart server 300. In an embodiment, the second processor 330 of Figure 3. The smart server 300 Figure 2 The information storage state of the mobile IoT device 200 (or the association state with the corresponding electronic device 100) corresponding to the first identification information sent by the second electronic device 100_2 can be identified by referring to the database 322. When the mobile IoT device 200 is not associated with the second electronic device 100_2, the second processor 330 can store the second identification information and location information of the second electronic device 100_2 in association with the first identification information of the mobile IoT device 200. In an embodiment, in the operation of the second processor 330 associating the mobile IoT device 200 with the second electronic device 100_2 and then storing the association result, the second processor 330 can generate the location information of the mobile IoT device 200 so that it is the same as or corresponds to the location information of the second electronic device 100_2, and can further associate and store the generated location information of the mobile IoT device 200.

[0080] Figure 7 is a diagram illustrating an example of a process in which a plurality of electronic devices receive a user utterance according to an embodiment.

[0081] Reference Figure 7, the intelligent server 300 can receive voice data of user speech input from multiple electronic devices 100_3 and 100_4. In this regard, in the space 17, the user speech input issued by the user adjacent to or near the third electronic device 100_3 can also be received by the fourth electronic device 100_4 in another space 18 adjacent to or near the third electronic device 100_3. In this case, each of the third electronic device 100_3 and the fourth electronic device 100_4 can send the received voice data of the user speech input to the intelligent server 300. According to one embodiment, when the same or similar voice data segments are received within a specified time range, the intelligent server 300 Figure 2 The second processor 330 may identify the signal strength or time stamp information of each voice data segment. The second processor 330 may analyze the voice data having a relatively high signal strength and whose time stamp information is relatively early. Other voice data may be discarded.

[0082] According to the above-described embodiments, a server supporting operation of an Internet of Things (IoT) environment may include a communication circuit, a memory, and a processor electrically connected to the communication circuit and the memory.

[0083] According to an embodiment, the processor can store identification information and location information of at least one external device in a memory, can use a communication circuit to receive identification information of an IoT device adjacent to the at least one external device from the at least one external device, and can store the identification information of the IoT device associated with the identification information and location information of the at least one external device as a database in the memory.

[0084] According to an embodiment, when at least one external device is initially located in an IoT environment, identification information and location information of at least one external device may be sent from the at least one external device, or, when identification information of an IoT device is sent from the at least one external device, identification information and location information of the at least one external device may be sent from the at least one external device.

[0085] According to an embodiment, the processor may include an automatic speech recognition (ASR) module and a natural language understanding (NLU) module associated with the recognition of voice data. The processor may receive voice data of a user's speech input associated with the operation of the IoT device from at least one external device using a communication circuit, and may recognize the voice data using the ASR module and the NLU module.

[0086] According to an embodiment, the processor can derive a command or intention included in the user speech input from the recognized voice data, and can determine that the IoT device is associated with the derived command or intention based on identification information of the IoT device associated with at least one external device providing the voice data stored in the database.

[0087] According to an embodiment, the processor may determine the location of the IoT device associated with the derived command or intention based on location information of at least one external device providing voice data included in the database.

[0088] According to an embodiment, the processor may use the communication circuit to send a control signal associated with the derived command or intent to the IoT device, the location of which is determined.

[0089] According to an embodiment, the at least one external device may include a plurality of external devices. The processor may identify the attributes of an IoT device associated with at least one external device providing voice data stored in a database, and when the attributes of the IoT device cannot correspond to the derived command or intention, the processor may identify the attributes of at least one other IoT device associated with at least one other external device stored in the database.

[0090] According to an embodiment, when an attribute of at least one other IoT device corresponds to the derived command or intention, the processor may generate voice-type feedback data including location information of at least one other external device, and may send the feedback data to at least one external device providing voice data using a communication circuit.

[0091] According to an embodiment, the at least one external device may include a plurality of external devices. When receiving the same or similar voice data from a plurality of external devices within a specified time range, the processor may identify the voice data having the highest signal strength and whose time stamp information is earliest.

[0092] According to the above-described embodiments, an electronic device supporting operation of an IoT environment may include a communication circuit and a processor electrically connected to the communication circuit.

[0093] According to an embodiment, the processor may perform scanning of a specified signal using a communication circuit, may identify at least one IoT device by detecting a specified signal sent from at least one IoT device adjacent to the electronic device during the signal scan, may send a request for identification information to the identified at least one IoT device to receive the identification information using the communication circuit, may send the identification information of the at least one IoT device to an external device using the communication circuit, and may further send data or a signal for requesting the external device to store the identification information of the at least one IoT device associated with the identification information and location information of the electronic device.

[0094] According to an embodiment, when the electronic device is initially located in the IoT environment, identification information and location information of the electronic device may be transmitted to the external device.

[0095] According to an embodiment, identification information and location information of the electronic device may be transmitted to an external device together with identification information of at least one IoT device.

[0096] According to an embodiment, the electronic device may further include at least one microphone. The processor may use the at least one microphone to receive user speech input associated with the operation of at least one IoT device, and may use the communication circuit to send voice data of the user speech input to an external device.

[0097] According to an embodiment, the electronic device may further include a memory and at least one speaker. The processor may store voice type query data associated with registration of at least one IoT device for the IoT environment in the memory, and may output the query data using at least one speaker when at least one IoT device is identified.

[0098] According to an embodiment, the processor may include location information of the electronic device and identification information of at least one IoT device in the query data.

[0099] According to an embodiment, the designated signal may be at least one of a Bluetooth Low Energy (BLE) signal and a ZigBee signal.

[0100] According to the above-mentioned embodiment, the IoT device location inference method of the server supporting IoT environment operation may include storing identification information and location information of at least one external device in a memory, receiving identification information of an IoT device adjacent to the at least one external device from the at least one external device using a communication circuit, and storing the identification information of the IoT device associated with the identification information and location information of the at least one external device as a database in the memory.

[0101] According to an embodiment, the IoT device position inference method may further include receiving voice data of a user's utterance input associated with the operation of the IoT device from at least one external device using a communication circuit and recognizing the voice data.

[0102] According to an embodiment, recognition of voice data may include deriving a command or intent included in a user speech input, and determining that an IoT device is associated with the derived command or intent based on identification information of the IoT device associated with at least one external device providing voice data stored in a database.

[0103] According to an embodiment, recognition of voice data may include deriving a command or intent included in a user speech input, and determining a location of an IoT device associated with the derived command or intent based on location information of at least one external device providing voice data included in a database.

[0104] Figure 88 is a block diagram illustrating an electronic device 801 in a network environment 800 according to various embodiments. Figure 8 , the electronic device 801 in the network environment 800 may communicate with the electronic device 802 via the first network 898 (e.g., a short-range wireless communication network), or communicate with the electronic device 804 or the server 808 via the second network 899 (e.g., a long-range wireless communication network). According to an embodiment, the electronic device 801 may communicate with the electronic device 804 via the server 808. According to an embodiment, the electronic device 801 may include a processor 820, a memory 830, an input device 850, a sound output device 855, a display device 860, an audio module 870, a sensor module 876, an interface 877, a haptic module 879, a camera module 880, a power management module 888, a battery 889, a communication module 890, a user identification module (SIM) 896, or an antenna module 897. In some embodiments, at least one of the components (e.g., the display device 860 or the camera module 880) may be omitted from the electronic device 801, or one or more other components may be added to the electronic device 801. In some embodiments, some of the components may be implemented as a single integrated circuit. For example, the sensor module 876 (eg, a fingerprint sensor, an iris sensor, or an illumination sensor) may be implemented as embedded in the display device 860 (eg, a display).

[0105] The processor 820 may run, for example, software (e.g., program 840) to control at least one other component (e.g., hardware component or software component) of the electronic device 801 connected to the processor 820, and may perform various data processing or calculations. According to one embodiment, as at least part of the data processing or calculation, the processor 820 may load a command or data received from another component (e.g., sensor module 876 or communication module 890) into a volatile memory 832, process the command or data stored in the volatile memory 832, and store the result data in a non-volatile memory 834. According to an embodiment, the processor 820 may include a main processor 821 (e.g., a central processing unit (CPU) or an application processor (AP)) and an auxiliary processor 823 (e.g., a graphics processing unit (GPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that is independent of or combined with the main processor 821 in operation. Additionally or alternatively, the auxiliary processor 823 may be adapted to consume less power than the main processor 821, or adapted to be specifically used for a specified function. The auxiliary processor 823 may be implemented separately from the main processor 821 , or as part of the main processor 821 .

[0106] When the main processor 821 is in an inactive (e.g., sleep) state, the auxiliary processor 823 may control at least some of the functions or states related to at least one component (e.g., display device 860, sensor module 876, or communication module 890) among the components of the electronic device 801 (not the main processor 821), or when the main processor 821 is in an active state (e.g., running an application), the auxiliary processor 823 may control at least some of the functions or states related to at least one component (e.g., display device 860, sensor module 876, or communication module 890) among the components of the electronic device 801 together with the main processor 821. According to an embodiment, the auxiliary processor 823 (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., camera module 880 or communication module 890) that is functionally related to the auxiliary processor 823.

[0107] The memory 830 may store various data used by at least one component of the electronic device 801 (e.g., the processor 820 or the sensor module 876). The various data may include, for example, software (e.g., the program 840) and input data or output data for commands related thereto. The memory 830 may include a volatile memory 832 or a non-volatile memory 834.

[0108] The program 840 may be stored as software in the memory 830 , and may include, for example, an operating system (OS) 842 , middleware 844 , or an application 846 .

[0109] The input device 850 may receive commands or data from outside the electronic device 801 (e.g., a user) to be used by other components of the electronic device 801 (e.g., the processor 820). The input device 850 may include, for example, a microphone, a mouse, a keyboard, or a digital pen (e.g., a stylus).

[0110] The sound output device 855 can output a sound signal to the outside of the electronic device 801. The sound output device 855 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as playing multimedia or playing records, and the receiver can be used for incoming calls. Depending on the embodiment, the receiver can be implemented as a separate part from the speaker, or as part of the speaker.

[0111] The display device 860 may visually provide information to the outside of the electronic device 801 (e.g., a user). The display device 860 may include, for example, a display, a holographic device, or a projector, and a control circuit for controlling a corresponding one of the display, the holographic device, and the projector. According to an embodiment, the display device 860 may include a touch circuit adapted to detect a touch or a sensor circuit (e.g., a pressure sensor) adapted to measure the strength of a force caused by a touch.

[0112] The audio module 870 can convert sound into an electrical signal, or vice versa. According to an embodiment, the audio module 870 can obtain sound via the input device 850, or output sound via the sound output device 855 or an earphone of an external electronic device (e.g., electronic device 802) directly (e.g., wired) or wirelessly connected to the electronic device 801.

[0113] The sensor module 876 may detect an operating state (e.g., power or temperature) of the electronic device 801 or an environmental state (e.g., a state of a user) outside the electronic device 801, and then generate an electrical signal or data value corresponding to the detected state. According to an embodiment, the sensor module 876 may include, for example, a gesture sensor, a gyroscope sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illumination sensor.

[0114] The interface 877 may support one or more specific protocols to be used to connect the electronic device 801 directly (e.g., wired) or wirelessly to an external electronic device (e.g., the electronic device 802). According to an embodiment, the interface 877 may include, for example, a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0115] The connection end 878 may include a connector, wherein the electronic device 801 can be physically connected to an external electronic device (e.g., the electronic device 802) via the connector. According to an embodiment, the connection end 878 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0116] The haptic module 879 may convert the electrical signal into mechanical stimulation (eg, vibration or motion) or electrical stimulation that can be recognized by the user via his sense of touch or kinesthetic sense. According to an embodiment, the haptic module 879 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0117] The camera module 880 may capture a still image or a moving image. According to an embodiment, the camera module 880 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0118] The power management module 888 may manage power supply to the electronic device 801. According to an embodiment, the power management module 888 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0119] The battery 889 may power at least one component of the electronic device 801. According to an embodiment, the battery 889 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0120] The communication module 890 may support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 801 and an external electronic device (e.g., electronic device 802, electronic device 804, or server 808), and perform communication via the established communication channel. The communication module 890 may include one or more communication processors capable of operating independently from the processor 820 (e.g., an application processor (AP)) and supporting direct (e.g., wired) communication or wireless communication. According to an embodiment, the communication module 890 may include a wireless communication module 892 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 894 (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). A corresponding one of these communication modules can communicate with an external electronic device via a first network 898 (e.g., a short-range communication network such as Bluetooth, Wireless Fidelity (Wi-Fi) Direct, or Infrared Data Association (IrDA)) or a second network 899 (e.g., a long-range communication network such as a cellular network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules can be implemented as a single component (e.g., a single chip), or these various types of communication modules can be implemented as multiple components separated from each other (e.g., multiple chips). The wireless communication module 892 can identify and authenticate the electronic device 801 in a communication network (such as the first network 898 or the second network 899) using user information (e.g., an International Mobile Subscriber Identity (IMSI)) stored in the user identification module 896.

[0121] The antenna module 897 may transmit a signal or power to the outside of the electronic device 801 (e.g., an external electronic device) or receive a signal or power from the outside of the electronic device 801 (e.g., an external electronic device). According to an embodiment, the antenna module 897 may include an antenna including a radiating element, and the radiating element is formed of a conductive material or a conductive pattern formed in a substrate (e.g., a PCB) or formed on a substrate. According to an embodiment, the antenna module 897 may include a plurality of antennas. In this case, at least one antenna suitable for a communication scheme used in a communication network (such as a first network 898 or a second network 899) may be selected from the plurality of antennas by, for example, the communication module 890 (e.g., the wireless communication module 892). Subsequently, a signal or power may be transmitted or received between the communication module 890 and the external electronic device via the selected at least one antenna. According to an embodiment, another component (e.g., a radio frequency integrated circuit (RFIC)) other than the radiating element may be additionally formed as a part of the antenna module 897.

[0122] At least some of the above components may be connected to each other via an inter-peripheral communication scheme (e.g., a bus, a general purpose input output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)) and communicatively transmit signals (e.g., commands or data) therebetween.

[0123] According to an embodiment, a command or data may be sent or received between the electronic device 801 and the external electronic device 804 via the server 808 connected to the second network 899. Each of the electronic device 802 and the electronic device 804 may be a device of the same type as the electronic device 801, or a device of a different type from the electronic device 801. According to an embodiment, all or some operations to be executed in the electronic device 801 may be executed in one or more of the external electronic device 802, the external electronic device 804, or the server 808. For example, if the electronic device 801 should automatically execute a function or service or should execute a function or service in response to a request from a user or another device, the electronic device 801 may request the one or more external electronic devices to execute at least part of the function or service instead of executing the function or service, or the electronic device 801 may request the one or more external electronic devices to execute at least part of the function or service in addition to executing the function or service. The one or more external electronic devices that receive the request may execute the requested at least part of the function or service, or execute another function or another service related to the request, and transmit the result of the execution to the electronic device 801. The electronic device 801 may provide the result as at least a partial reply to the request with or without further processing the result. To this end, for example, cloud computing technology, distributed computing technology, or client-server computing technology may be used.

[0124] The electronic device according to various embodiments may be one of various types of electronic devices. The electronic device may include, for example, a portable communication device (e.g., a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a household appliance. According to an embodiment of the present disclosure, the electronic device is not limited to those electronic devices described above.

[0125] It should be understood that the various embodiments of the present disclosure and the terms used therein are not intended to limit the technical features set forth herein to specific embodiments, but include various changes, equivalent forms or alternative forms for corresponding embodiments. For the description of the accompanying drawings, similar reference numerals may be used to refer to similar or related elements. It will be understood that the nouns in the singular form corresponding to the term may include one or more things unless the relevant context clearly indicates otherwise. As used herein, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C" and "at least one of A, B or C" may include any one or all possible combinations of the items listed together with the corresponding one of the multiple phrases. As used herein, terms such as "1st" and "2nd" or "first" and "second" may be used to simply distinguish the corresponding component from another component, and do not limit the component in other aspects (e.g., importance or order). It will be understood that if an element (e.g., a first element) is referred to as being “combined with another element (e.g., the second element)”, “combined to another element (e.g., the second element)”, “connected with another element (e.g., the second element)”, or “connected to another element (e.g., the second element)” when the terms “operably” or “communicatively” are used or when the terms “operably” or “communicatively” are not used, it means that the element may be directly (e.g., wired) connected to the other element, wirelessly connected to the other element, or connected to the other element via a third element.

[0126] As used herein, the term "module" may include units implemented in hardware, software, or firmware, and may be used interchangeably with other terms (e.g., "logic," "logic block," "portion," or "circuit"). A module may be a single integrated component adapted to perform one or more functions or a minimum unit or portion of the single integrated component. For example, according to an embodiment, a module may be implemented in the form of an application specific integrated circuit (ASIC).

[0127] The various embodiments described herein may be implemented as software (e.g., program 840) including one or more instructions stored in a storage medium (e.g., internal memory 836 or external memory 838) that can be read by a machine (e.g., electronic device 801). For example, under the control of a processor, a processor (e.g., processor 820) of the machine (e.g., electronic device 801) may call at least one of the one or more instructions stored in the storage medium and execute the at least one instruction with or without the use of one or more other components. This enables the machine to operate to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be run by an interpreter. A machine-readable storage medium may be provided in the form of a non-transitory storage medium. Among them, the term "non-transitory" only means that the storage medium is a tangible device and does not include a signal (e.g., an electromagnetic wave), but the term does not distinguish between data being semi-permanently stored in a storage medium and data being temporarily stored in a storage medium.

[0128] According to an embodiment, the method according to various embodiments of the present disclosure may be included and provided in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be released in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)), or may be downloaded via an application store (e.g., Play Store TM ) The computer program product may be published (e.g., downloaded or uploaded) online, or the computer program product may be distributed (e.g., downloaded or uploaded) directly between two user devices (e.g., smart phones). If published online, at least part of the computer program product may be temporarily generated, or at least part of the computer program product may be at least temporarily stored in a machine-readable storage medium (such as a memory of a manufacturer's server, an application store's server, or a forwarding server).

[0129] According to various embodiments, each component (e.g., module or program) in the above-mentioned components may include a single entity or multiple entities. According to various embodiments, one or more components in the above-mentioned components may be omitted, or one or more other components may be added. Alternatively or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, according to various embodiments, the integrated component may still perform the one or more functions of each component in the multiple components in the same or similar manner as a corresponding component in the multiple components before integration. According to various embodiments, the operations performed by a module, program or another component may be performed sequentially, in parallel, repeatedly or in a heuristic manner, or one or more operations in the operations may be run or omitted in different orders, or one or more other operations may be added.

[0130] According to an embodiment, when a user is not required to specify location information of a desired IoT device when registering the IoT device, user convenience can be achieved.

[0131] According to the embodiment, even if the user does not specify the IoT device, the IoT device can be clearly identified through registration of the IoT device.

[0132] In addition, various other effects and advantages directly or indirectly understood through the present disclosure can also be provided.

[0133] Some of the above embodiments of the present disclosure may be implemented in hardware, firmware, or by executing software or computer code that may be stored in a recording medium, such as a CDROM, a digital versatile disc (DVD), a tape, a RAM, a floppy disk, a hard disk, or a magneto-optical disk, or a computer code downloaded over a network that is initially stored on a remote recording medium or a non-transitory machine-readable medium and stored on a local recording medium, so that the methods described herein may be presented via such software that is stored on a recording medium using a general-purpose computer or a dedicated processor or programmable or dedicated hardware (such as an ASIC or FPGA). As will be understood in the art, a computer, processor, microprocessor controller, or programmable hardware includes memory components, such as RAM, ROM, flash memory, and the like. A computer that may store or receive software or computer code that, when accessed and executed by a computer, processor, or hardware, implements the processing methods described herein.

[0134] While the present disclosure has been shown and described with reference to an embodiment thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents.

Claims

1. A server, comprising: Communication circuits; a memory storing the identification information of the IoT device in association with the location information of the first external device; as well as a processor electrically connected to the communication circuit and the memory, Wherein, the processor is configured to: receiving, by the communication circuit, from a second external device, second identification information of the second external device and voice data of a user's speech input associated with an operation of the IoT device; recognizing the speech data based on automatic speech recognition and natural language understanding; recognizing a command from the recognized voice data; identifying, based on an attribute of the IoT device, that the IoT device is associated with the intent of the identified command; determining the location information of the IoT device based on the location information of the first external device; and Based on the location information of the second external device corresponding to the location information associated with the IoT device, a control signal associated with the identified command is sent to the IoT device.

2. The server according to claim 1, wherein: When the first external device is connected to the server, sending identification information and location information of the first external device from the first external device; or When the identification information of the IoT device is transmitted from the first external device, the identification information and the location information of the first external device are transmitted from the first external device.

3. The server according to claim 1, wherein: The processor is further configured to: When the attribute of the IoT device does not correspond to the identified command, an attribute of at least one other IoT device associated with at least one other external device is identified.

4. The server according to claim 3, wherein: The processor is further configured to: When the attribute of the at least one other IoT device corresponds to the recognized command, generating voice-type feedback data including location information of the at least one other external device; as well as The feedback data is sent to the second external device using the communication circuit.

5. The server according to claim 1, wherein: The processor is configured to: When a plurality of identical or similar voice data segments are received from a plurality of external devices including the second external device within a specified time range, the voice data with the highest signal strength and the earliest timestamp information is identified.

Images