Processing voice commands based on device topology

Abstract

The present application relates to processing voice commands based on device topology. Exemplary aspects of the present disclosure relate to processing voice commands or utterances. For example, data representing a voice utterance may be received. A device topology representation may be accessed. The device topology representation may define a plurality of smart devices associated with one or more buildings. The device topology representation may further define a location of each of the plurality of devices within the associated building. A transcription of the voice utterance may be determined based at least in part on the device topology representation. One or more selected devices and one or more actions to be performed by the one or more selected devices may be determined based at least in part on the determined transcription and the device topology representation.

Description

[0001] Description of the case

[0002] This application is a divisional application of Chinese invention patent application No. 201710924487.6, filed on September 30, 2017. Technical Field

[0003] The present disclosure relates generally to interpreting speech utterances and, more particularly, to interpreting speech utterances based on a topology of a device. Background Art

[0004] Smart, multi-sensing network-connected devices continue to proliferate, such as speakers, home alarms, door locks, cameras, lighting systems, treadmills, weight scales, smart beds, irrigation systems, garage door openers, home appliances, baby monitors, fire alarms, etc. These smart devices may, for example, be located within the confines of a building, and a user may wish to interact with such smart devices. To this end, the user may use a control device to wirelessly communicate with the smart device, for example, via a network.

[0005] In some cases, a user may wish to communicate with one or more smart devices by providing voice commands or utterances to a control device. Conventional voice command processing techniques use various suitable automatic speech recognition, natural language processing, and / or natural language understanding techniques to process and interpret voice utterances and control smart devices based on the voice utterances. However, such conventional techniques are generally unable to interpret complex voice utterances and may require requests for more information from the user. In particular, such conventional techniques generally lack a full understanding of the devices within a smart home network and the relationships between these devices within the network. Summary of the invention

[0006] Aspects and advantages of embodiments of the present disclosure will be set forth in part in the following description, or may be learned from the description, or may be learned through practice of the embodiments.

[0007] An example aspect of the present disclosure relates to a computer-implemented method for processing a speech utterance. The method includes receiving, by one or more computing devices, data representing a speech utterance. The speech utterance is provided by a user to a control device. The method also includes accessing, by the one or more computing devices, a device topology representation. The device topology representation defines a plurality of smart devices associated with one or more buildings. The device topology representation also defines the location of each of the plurality of devices within the associated building. The method also includes determining, by the one or more computing devices, a transcription of the speech utterance based at least in part on the device topology representation. The method also includes determining, by the one or more computing devices, one or more selected devices and one or more actions to be performed by the one or more selected devices based at least in part on the determined transcription and the device topology representation.

[0008] Other example aspects of the present disclosure relate to systems, devices, tangible, non-transitory computer-readable media, user interfaces, memory devices, and electronic devices for processing voice commands.

[0009] These and other features, aspects and advantages of various embodiments will become better understood with reference to the following description and appended claims.The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the relevant principles. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] A detailed discussion of embodiments for those of ordinary skill in the art is set forth in the specification, which refers to the accompanying drawings, in which:

[0011] Figure 1 depicts an overview of an example system for processing voice commands according to an example embodiment of the present disclosure;

[0012] Figure 2 depicts an example user interface for implementing device topology representation according to an example embodiment of the present disclosure;

[0013] Figure 3 depicts a block diagram of an example device topology representation according to an example embodiment of the present disclosure;

[0014] Figure 4 A flowchart depicting an example method of processing voice commands according to an example embodiment of the present disclosure;

[0015] Figure 5 A flowchart depicting an example method of generating a personalized language model according to an example embodiment of the present disclosure;

[0016] Figure 6 A flow chart depicting an example method of determining a transcription of a speech utterance according to an example embodiment of the present disclosure;

[0017] Figure 7 A flowchart depicting an example method of interpreting a transcription of a speech utterance according to an example embodiment of the present disclosure;

[0018] Figure 8 An overview of an example system according to an example embodiment of the present disclosure is depicted; Specific embodiments

[0019] Reference will now be made in detail to the embodiments, one or more examples of which are illustrated in the accompanying drawings. Each example is provided by way of illustration of the embodiments without limiting the present disclosure. In fact, it will be apparent to those skilled in the art that various modifications and variations may be made to the embodiments without departing from the scope or spirit of the present disclosure. For example, a feature illustrated or described as part of one embodiment may be used together with another embodiment to produce yet another embodiment. Therefore, it is intended that aspects of the present disclosure encompass these modifications and variations.

[0020] Example aspects of the present disclosure relate to controlling a smart device based on voice commands from a user. For example, a voice utterance may be received from a user of a control device. A voice utterance may be a command spoken by a user of a user device that is close to a device configured to control one or more smart devices. A smart device may be an intelligent, multi-sensing, network-connected device configured to communicate with a central server, a cloud computing system, and / or one or more additional smart devices to provide any of a variety of useful smart home services. For example, a smart device may be or may otherwise be associated with an air conditioning system, a lighting device, a home theater and entertainment system, a security system, an automatic door lock system, a thermostat device, a home automation system, a speaker, a camera device, a treadmill, a weight scale, a smart bed, an irrigation system, a garage door opener, a home appliance, a baby monitor, a fire alarm, and / or other suitable devices or systems.

[0021] A user who wishes to control one or more smart devices can construct a device topology representation. The device topology representation can be constructed using a variety of suitable techniques and can be any suitable format and / or structure capable of representing the topology and organization of one or more smart devices within one or more buildings. The device topology representation can define and / or organize one or more buildings associated with the user. For example, the building associated with the user can be a home, business, vehicle, office, and / or any other suitable building associated with the user. The user can organize each building into one or more rooms (e.g., bedroom, living room, kitchen), and can assign one or more smart devices to the one or more rooms. For example, a user can assign one or more smart lights to the first room of a particular building, and one or more door lock systems to the second room of the building. Then, the user can assign one or more different smart devices to one or more rooms in the second building.

[0022] A user may assign one or more identities to a smart device in the device topology representation. The identity may be selected by the user and associated with a corresponding smart device within the device topology representation. The identity(ies) may include a nickname and / or alias (e.g., couch lamp, front door lock, bedroom speaker, etc.) selected by the user for the smart device. In this manner, when providing a voice utterance for controlling a smart device, the identity may be a name or alias of the corresponding smart device that the user may use.

[0023] The device topology representation may further specify one or more device properties associated with the corresponding device. The device properties may be associated with one or more actions that the smart device configuration performs. For example, the device properties associated with the lighting device may include a turn-on capability, a turn-off capability, a dimming capability, and / or other suitable properties associated with the lighting device. As another example, an automatic door lock device may include a lock capability, an unlock capability, and / or other suitable capabilities.

[0024] The user may provide voice utterances to the control device. The voice utterances may be any suitable voice utterances that specify the intention to control one or more smart devices. For example, the user may provide voice utterances with the intention of changing the state of the smart device (e.g., turning the smart device on or off, increasing the volume of the smart device, etc.), or with the intention of receiving information associated with the smart device (e.g., information related to the current temperature, etc.). The control device may be any suitable device capable of receiving voice utterances from the user, and, for example, communicating with one or more smart devices or a central server via a network. The control device may be a device to which the user speaks when providing the voice utterances. In some embodiments, the control device may be a user device associated with the user, such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a wearable computing device, and / or other suitable user devices. In some embodiments, the control device may be a smart device configured to receive voice utterances. For example, the control device may be a smart device specified in the device topology representation.

[0025] The speech utterance can be interpreted to determine the control command intended by the user for one or more smart devices. In this regard, speech recognition can be performed on the voice speech based at least in part on the device topology representation. For example, data representing the speech utterance can be provided to one or more language models to determine the transcription of the speech utterance. In some embodiments, the general language model can be biased toward the device topology representation to determine the text transcription of the speech utterance. The general language model can include multiple common phrases, and a probability estimate can be associated with each phrase. Each phrase can contain a sequence of one or more words. The probability estimate can specify the predicted occurrence of the corresponding phrase. It should be understood that various suitable general language models can be used without departing from the scope of the present disclosure.

[0026] The general language model may be biased toward the device topology representation to generate a personalized language model. In particular, the general language model may be biased by increasing the probability estimates associated with one or more words or phrases associated with the device topology representation. For example, the one or more words or phrases may include a room name, a building name, a device identifier, an item associated with a device attribute, etc. specified in the device topology representation. In this way, the personalized language model may estimate a higher probability of occurrence of a word or phrase included in the device topology representation.

[0027] Data representing the speech utterance may be provided to a personalized language model to determine a text transcription of the speech utterance based on the personalized language model. In some embodiments, the data representing the speech utterance may be provided to a general language model to determine a first transcription of the speech utterance based on the general language model. A first confidence score representing a confidence in the accuracy of the first transcription may be determined for the first transcription. Then, the data representing the speech utterance may be provided to the personalized language model to determine a second transcription of the speech utterance based on the personalized language model. A second confidence score representing a confidence in the accuracy of the second transcription may be determined for the second transcription. In some embodiments, the transcription with the higher confidence score may be used. In some embodiments, the data representing the speech utterance may be further provided to one or more additional language models, such as one or more specialized language models associated with various suitable applications. In this way, the additional language model may be used to determine one or more additional transcriptions, each with an associated confidence score. The transcription with the highest confidence score may be used.

[0028] Once the transcription of the speech utterance is determined, one or more devices to be controlled (e.g., selected devices) and one or more actions to be performed by the selected devices can be determined. For example, a semantic interpretation of the transcription can be determined using one or more semantic interpretation techniques. The semantic interpretation can provide a representation of the conceptual meaning of the transcription. In this way, the interpretation of the transcription can provide an interpretation of the user's intent when performing the speech utterance. For example, if the transcription of the speech utterance from the user states "turn on the lights," the semantic interpretation of the transcription can attempt to determine the action to be performed (e.g., start one or more light devices), as well as one or more selected devices to perform the action (e.g., one or more appropriate lighting devices intended to be controlled by the user).

[0029] The interpretation may be determined based at least in part on the device topology representation. For example, the device topology representation may be accessed to determine the one or more selected devices and / or actions to be performed. In this manner, the device topology representation may be accessed and compared to the transcription to determine a match between one or more items associated with the transcription and one or more items associated with the device topology representation. For example, if the transcription reads "turn on the couch light," the device topology representation may be accessed to determine whether there is a specified device in the device topology representation that has a device identification of "couch light."

[0030] As noted, one or more actions to be performed by the selected device(s) may also be determined. For example, the actions may be determined based on the interpretation of the transcription. In some embodiments, the actions may be determined based at least in part on device properties associated with the selected device(s) specified in the device topology representation. In this manner, the identified actions may be actions that the selected device is capable of performing.

[0031] In some cases, a direct match between a device identification specified in a device topology and a transcribed item may not be determined. For example, when performing a voice utterance, a user may not specify a device identification, but may give a more informal instruction. As another example, multiple buildings specified in a device topology representation may include one or more rooms and / or devices with the same or similar names or identifications. For example, a transcription that reads "turn on the lights in the bedroom" may not include a reference to a specific device identification, making it impossible to determine a direct match. In this case, the device topology representation may be accessed to infer one or more selected devices. For example, the device topology representation may be accessed to identify a room labeled "bedroom" in a building specified in the device topology representation. If such a "bedroom" is identified, one or more appropriate devices located within the bedroom may be identified. For example, if the device topology representation specifies one or more lighting devices within a bedroom, at least one of the one or more lighting devices may be inferred as a selected device.

[0032] In the case where the device identification is not specified in the transcription, the one or more selected devices and / or the action to be performed can be determined based at least in part on the device attributes specified in the device topology representation. For example, the device attributes specified in the device topology representation can be compared to the transcription to determine one or more devices that can perform the action specified in the transcription. In this way, the selected device can be inferred based at least in part on the corresponding device attributes.

[0033] In some embodiments, the selected device and / or the action to be performed may be determined based at least in part on contextual data such as the user's location. For example, in response to receiving a voice utterance from a user, the user's location may be determined. A variety of suitable location determination techniques may be used to determine the location. The location determination technique may be determined based at least in part on the control device to which the user provides the voice utterance. For example, the control device is a smart device specified in the device topology representation, and the user's location may be mapped to a building and / or room to which the control device is assigned in the device topology representation. If the control device is a user device not specified in the device topology representation, one or more location determination techniques may be used to determine the user's location, such as techniques using wireless access points or short-range beacon devices associated with one or more smart devices, and / or other suitable location determination techniques. The user's location may then be mapped to one or more buildings and / or rooms specified in the device topology representation.

[0034] In this manner, the user location may be used as contextual data in determining one or more selected devices. For example, in response to receiving a transcription that reads "turn on the lights," the user location may be determined and mapped to one or more rooms and / or buildings specified in the device topology representation to determine the building and room in which the user is located. One or more light devices may be identified within such buildings and rooms to determine the selected device that the user wishes to control. In this manner, in this case, it may be inferred that the light that the user wishes to control is the light in the room in which the user is currently located.

[0035] After determining the one or more selected devices to be controlled and the one or more actions to be performed by the selected devices, one or more control commands can be generated and provided to the (one or more) selected devices, respectively. The one or more control commands may include instructions for causing the (one or more) selected devices to perform specified actions. Once the appropriate control command is received, the (one or more) selected devices may perform appropriate actions according to the (one or more) received control commands. In some embodiments, if one or more selected devices and / or one or more actions to be performed cannot be determined based on the received voice utterance, a prompt may be provided to the user (e.g., in a user interface displayed by a control device or other suitable device), requesting the user to clarify the voice utterance and / or manually select one or more actions to be performed by one or more smart devices.

[0036] In this way, the example aspects of the present disclosure can be used to improve the efficiency and accuracy of voice command processing for smart home and / or Internet of Things environments. In particular, organizing smart devices relative to the topology of one or more buildings where the smart devices are located can allow for a more robust interpretation of voice utterances, and can allow a natural "conversational" communication experience between the user and the control device. Considering contextual data such as the user's location in conjunction with the device topology can provide a more powerful ability to infer the meaning of the voice utterances provided by the user. In addition, this voice command processing technology can allow fewer communication rounds between the user and the system by facilitating the ability to accurately and more quickly interpret a wider range of voice utterances. Therefore, the number of operations performed by the smart device can be reduced, for example, receiving and processing repeated voice utterances, or outputting requests for more information, and processing replies can be avoided, and the power consumption of the smart device can therefore be reduced. Although the present disclosure relates to processing voice commands in smart home and / or Internet of Things environments, it should be understood that the example aspects of the present disclosure can be extended to various other suitable applications and environments without departing from the scope of the present disclosure. For example, the example aspects of the present disclosure can be used to process and interpret voice commands for controlling various other suitable non-Internet of Things devices.

[0037] Referring now to the accompanying drawings, example aspects of the present disclosure will be described in more detail. For example, Figure 1 An overview of an example system 100 for processing voice commands according to an example embodiment of the present disclosure is depicted. System 100 includes a control device 102 and a server 104. Control device 102 and server 104 may communicate, for example, via a network. Control device 102 may be any suitable device configured to monitor and receive voice utterances from a user. As noted, a voice utterance may be a sentence or phrase spoken by a user with the intent to control one or more smart devices. Control device 102 may be a user device associated with a user. In some embodiments, control device 102 may be a smart device. A smart device may be a network-connected device configured to communicate with a central server, a cloud computing system, and / or one or more additional smart devices to provide a variety of useful smart home and / or Internet of Things services. The control device may be dynamically configured based on a voice utterance provided by a user. For example, the control device may be a device (e.g., a user device, a smart device, or other device) that receives a voice utterance from a user.

[0038] Once a speech utterance is received from a user, the control device 102 may provide data representing the speech utterance to the server 104. The server 104 includes a biaser 106, a transcriber 108, an interpreter 110, and a command generator 112. The data representing the speech utterance may be provided to the transcriber 108. The transcriber 108 may be configured to determine one or more text transcriptions of the speech utterance based at least in part on language model data 114. The language model data 114 may include one or more language models for the recorded speech utterance. In particular, the language model data 114 may include a general language model specifying probability estimates for a plurality of common phrases. The language model data 114 may also include one or more specialized language models, such as, for example, a personalized language model. The personalized language model may be generated by the biaser 106. In particular, the personalized language model may be generated by biasing the general language model toward the device topology representation 116.

[0039] The device topology representation 116 may be constructed by a user to define an organizational topology of one or more smart devices associated with the user. Figure 2 An example user interface 120 for defining a device topology representation is depicted. Specifically, Figure 2 A user device 122 is depicted that is configured to display the user interface 120. It should be understood that Figure 2 The user interface 120 depicted in is for illustrative purposes only, and various other suitable user interfaces may be used to implement the device topology representation without departing from the scope of the present disclosure. The user device 122 may be any suitable user device, such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a wearable computing device, etc. In some embodiments, the user device 122 may be a smart device or other device. In some embodiments, the user may need to be present at a certain location to define the device topology representation for that location. For example, in such an embodiment, a user who wishes to define a device topology representation for a particular building will need to be located at that building to implement the device topology representation. In this way, various suitable location determination techniques may be used to determine the user's location, and the user may be allowed to define a device topology representation for a building located at or near the user's location.

[0040] The user interface 120 may include one or more user interface elements 124 to facilitate a user to implement a device topology representation 126. The device topology representation 126 may correspond to Figure 1 and Figure 3. As shown, the one or more user interface elements 124 include interface elements that allow a user to add buildings, add rooms, and add devices to the device topology representation 126. In this manner, a user can interact with the user interface elements 124 as needed to define and organize the device topology representation 126. For example, a user can select an "add device" interface element to add a new smart device to the device topology representation 126. In this manner, a user may be able to add a new device, add one or more device identifiers to be associated with the device, add one or more device properties to be associated with the device, and the like. Similarly, if a user selects an "add room" interface element, the user may be able to add a new room to a particular building, add a room identifier associated with the room, and the like.

[0041] In some implementations, the device topology representation 126 may be an interactive user interface element such that a user may interact with the device topology representation 126 as displayed within the user interface 120. For example, a user may select (e.g., via touch input, or other suitable input) a particular building, room, or device of the device topology representation 126 to facilitate presentation of more information associated with the selected building, room, or device.

[0042] Figure 3 A block diagram of an example device topology representation 116 is depicted according to an example embodiment of the present disclosure. The device topology representation 116 defines a topology of multiple smart devices within multiple buildings associated with a user. In particular, the buildings include a main residence 152, a beach house 154, and an office 156 associated with the user. Each building can be organized into one or more rooms within the building. For example, Figure 3 A living room 158, a bedroom 160, and a guest room 162 are depicted as being associated with a main residence 152. Each room may be associated with one or more smart devices within a smart home and / or an Internet of Things associated with a user. In particular, the living room 158 is associated with a lighting device having a device identification of "couch lamp", a thermostat device having a device identification of "thermostat downstairs), and a speaker device having a device identification of "living room speaker". The bedroom 160 is associated with a ceiling fan device having a device identification of "ceiling fan", one or more lighting devices having a device identification of "ceiling light", and a speaker device having a device identification of "speaker". The guest room 162 is associated with a thermostat device having a device identification of "thermostat upstairs", a lighting device having a device identification of "desk lamp", and a speaker device having a device identification of "speaker". A designated device may be a single device or a group of devices. For example, a device identification of "ceiling light" may correspond to a single lighting device or a group of lighting devices constituting an overhead ceiling light in a bedroom.

[0043] As noted, a user may define one or more buildings and one or more rooms within the device topology representation. A user may assign an identifier to each room and building. A user may assign one or more smart devices to each room to represent the topology of devices within the rooms of an actual building. A user may further assign one or more device identifiers to each of the one or more smart devices. In some embodiments, if a user does not assign a device identifier to a device, a default device identifier may be automatically assigned to the device. The device topology representation 116 may also specify one or more device attributes for each device. Device attributes may be associated with one or more actions that the device can perform. For example, a light device may have an "on" attribute, an "off" attribute, a "dim" attribute, etc. A ceiling fan device may have an "on" attribute, an "off" attribute, an "increase speed" attribute, a "decrease speed" attribute, etc. In some embodiments, a user may input device attributes into the device topology representation 116. In some embodiments, device attributes may be automatically added to the device topology representation 116.

[0044] When the user wishes to reflect one or more changes to, for example, the device topology, the user can update or modify the device topology representation. For example, the user can add devices, remove devices, move devices between rooms and / or buildings, add buildings, remove buildings, add rooms, remove rooms, etc., as the user desires. In this way, the user can organize the device topology to provide an accurate and dynamic topology of devices within an actual building.

[0045] The device topology representation 116 can be used to bias the general language model. Figure 1 , the biaser 106 can be configured to access the device topology representation 116 and the general language model to bias the general language model toward the device topology representation 116. For example, the biaser can identify one or more terms or phrases specified in the device topology representation 116 (e.g., device identification, room name, building name, device type (e.g., lighting device, speaker device), device attributes, etc.). The biaser 106 can increase the probability estimates of those terms specified by the general language model, and can generate a personalized language model such that the identified terms or phrases are assigned increased probability estimates. In this manner, a personalized language model can be generated to reflect the increased likelihood that the speech utterance provided by the user will include terms or phrases associated with the device topology representation 116.

[0046] As noted, the transcriber 108 may determine one or more text transcriptions of the speech utterance based at least in part on the language model data 114. It should be appreciated that the transcriber 108 may determine the text transcription of the speech utterance using a variety of suitable speech recognition techniques. In some embodiments, the transcriber 108 may determine the transcription of the speech utterance based on the personalized language model generated by the biaser 106. In some embodiments, the transcriber 108 may determine one or more additional transcriptions of the speech utterance using one or more additional language models (e.g., a general language model and / or one or more other suitable language models associated with the language model data 114). In such embodiments, a confidence score may be determined for each transcription, each transcription specifying a confidence of the transcriber 108 such that the corresponding transcription is an accurate transcription of the speech utterance. A transcription may then be selected from the plurality of transcriptions based at least in part on the confidence score. In some embodiments, if the selected transcription is a transcription determined using the personalized language model, the selected transcription may be provided to the interpreter 110.

[0047] In some embodiments, a device topology representation may be selected for use based at least in part on the location of the user. For example, it may be determined that the user is located within a particular building specified in the device topology representation. Then, the device topology representation that specifies the building may be accessed for use when processing a speech utterance provided by the user. In such an embodiment, the accessed device topology representation may then be used to determine a transcription of the speech utterance provided by the user. In this way, the transcription may then be determined based at least in part on the location of the user. For example, after determining the location of the user, the corresponding device topology representation may be accessed, and one or more items associated with the device topology representation may be used to bias a general language model. In some embodiments, a personalized language model for a building and / or device topology representation may have been previously generated, in which case, in response to determining the user location and / or receiving a speech utterance from the user, an appropriate personalized language model may be accessed and used to determine a transcription of the speech utterance. According to an exemplary aspect of the present disclosure, such a device topology representation corresponding to the user's location may then be used to interpret the speech utterance.

[0048] The determined transcription may be provided to the interpreter 110. The interpreter 110 may be configured to determine the user's intent when providing the voice utterance to the control device 102. In particular, the interpreter 110 may be configured to determine one or more selected devices 118 and one or more actions to be performed by the selected devices 118 based at least in part on the transcription. For example, in some embodiments, the interpreter 110 may perform a semantic interpretation technique on the transcription to determine a logical representation of the conceptual meaning of the transcription. In this way, the interpreter 110 may determine the role of each word in the transcription by identifying semantic relationships between the words.

[0049] The interpreter 110 may determine the one or more selected devices 118 and / or actions to be performed based at least in part on the device topology representation 116. For example, the interpreter 110 may compare the transcription with the device topology representation 116 to determine the selected device 118 and / or the action to be performed. In particular, the interpreter 110 may compare the transcription with the device identification specified in the device topology representation 116 to determine a correlation between one or more items of the transcription and the one or more device identifications. For example, if the transcription reads "Increase the volume of the living room speakers", the item "living room speakers" may be associated with the device identification "living room speakers" of the device topology representation 116. In this way, the speaker device with the device identification "living room speakers" may be selected as the selected device based on this correlation.

[0050] In some embodiments, the interpreter 110 may be configured to infer the one or more selected devices. For example, if an exact match between the device identification and the transcription cannot be found, the interpreter may be configured to infer the one or more selected devices. As another example, if two or more devices specified in the device topology representation have the same or similar device identifications, the interpreter 110 may not be able to determine which device the user wants to control. For example, the device topology representation 116 includes speaker devices in the bedroom 160 and the guest room 162 with the same device identification "speaker". In this case, the interpreter 110 may not be able to distinguish which speaker device the user wants to control. The interpreter 110 may then be configured to infer the selected device based at least in part on other information associated with the device topology representation 116 and / or the user.

[0051] As an example, if there is no direct match between the transcription and the device identification but the transcription includes one or more items that are sufficiently similar to the device identification specified in the device topology representation 116, the interpreter 110 can be configured to infer the selected device. In this case, the interpreter 110 can associate the sufficiently similar (one or more) items with the device identification and can infer the selected device based on the similarity. For example, if the transcription reads "Turn on the sofa bed light", the interpreter 110 can recognize the device identification "sofa light" and can determine that the phrase "sofa bed light" is sufficiently similar to the device identification "sofa light". The interpreter 110 can then select the lighting device with the identification "sofa light" as the selected device.

[0052] In some embodiments, the interpreter 110 may infer the selected devices based at least in part on the device attributes specified in the device topology representation 116. The device attributes may specify the type of each device (e.g., lighting device, speaker device, etc.), one or more capabilities or actions that each device can perform (e.g., turn on, turn off, increase volume, decrease volume, increase speed, decrease speed, etc.), and / or other suitable attributes or characteristics associated with the device. In this manner, the transcript may be compared to the device attributes specified in the device topology representation to determine the one or more selected devices. For example, the interpreter 110 may compare one or more items in the transcript to the one or more device attributes of the device to determine relevance. For example, if the transcript reads "increase temperature," the interpreter 110 may attempt to identify one or more devices with suitable attributes associated with the action of increasing temperature. In this manner, the interpreter 110 may be able to narrow down the appropriate devices to "upstairs thermostat" and "downstairs thermostat."

[0053] In some embodiments, the interpreter 110 may determine the one or more selected devices based at least in part on the location of the user when the voice utterance is provided to the control device 102. As noted, the control device 102 may be a smart device or a user device to which the voice utterance is provided. The interpreter 110 may determine the location of the control device 102 and correlate the location of the control device 102 with the location of the user. For example, where the control device 102 is a smart device specified in the device topology representation 116, the interpreter 110 may access the device topology representation to determine the location (e.g., the building and room specified in the device topology representation 116). The interpreter 110 may then correlate the user location with the location of the control device 102 to which the user provided the voice command.

[0054] In the case where the control device 102 is not specified in the device topology representation 116 (e.g., when the control device 102 is a user device or other suitable control device), one or more alternative location determination techniques may be used. For example, in some embodiments, one or more wireless (e.g., Wi-Fi) access points may be used to locate the control device 102. The location of the control device 102 may be determined based at least in part on the known locations of the one or more access points and one or more signals communicated between the control device 102 and the one or more access points. More specifically, in some embodiments, the location of the user device may be determined based at least in part on a round trip time (RTT) calculation associated with one or more signals communicated between the user device and one or more access points. For example, the user device may initiate a location lookup process by sending a signal to a plurality of access points. The access point may send one or more response signals to the user device. The response signal may include data indicating a time delay (e.g., a processing delay) between the corresponding access point receiving the start signal and sending the response signal. The user device may calculate the flight time associated with the start signal and / or the response signal based at least in part on the time when the user device sends the start signal, the time when the user device receives the response signal, and / or the time delay associated with the access point. If the user device receives response signals from three or more access points, the user device can apply trilateration techniques to the determined distances and positions to locate the user device itself relative to the access points. For example, the user device can determine that it is located at a specific location in a certain area (e.g., within a specific room).

[0055] Similarly, in some embodiments, one or more beacon devices (e.g., Bluetooth beacon devices) can be used to determine the location of the control device 102. For example, the location of the control device can be determined at least in part based on the known location of the beacon device and one or more signals communicated between the beacon device and the control device 102. It should be understood that various other appropriate location determination techniques can be used without departing from the scope of the present disclosure. More specifically, when determining the location of the user device, the user device can broadcast one or more signals that can be received by two or more beacon devices located in the building where the user is located. The beacon device can have a known location within the building. In some embodiments, the beacon device can be implemented in one or more smart devices defined in the device topology representation 116. In response to receiving the one or more signals, two or more beacon devices can broadcast one or more identification signals that identify the beacon devices respectively. The identification signal can include the location of the corresponding beacon device. Then, the location of the user device can be determined at least in part based on the beacon location specified in the received identification signal.

[0056] It should be understood that various other location determination techniques may be used without departing from the scope of the present disclosure. For example, location determination techniques may be implemented using various suitable communication signals such as Bluetooth, Bluetooth Low Energy, ZigBee, near field communication, Wi-Fi Direct, or other technologies. In alternative embodiments, the user's location may be determined at least in part by a positioning system associated with the user device (e.g., GPS or other positioning system) and / or network-based (e.g., WiFi, cellular network, etc.) location determination.

[0057] In some embodiments, the location of the control device 102 can be determined relative to the device topology representation 116. For example, in embodiments where wireless access points and / or beacon devices are used to determine the location of the control device 102, the locations of the access points and / or beacon devices can be mapped to one or more rooms within the device topology representation 116. In this manner, the location of the control device 102 relative to the device topology representation 116 can then be determined based at least in part on the locations of the access points and / or beacon devices within the device topology representation 116.

[0058] As noted, the interpreter 110 can determine the selected device based at least in part on the location of the control device when the user provided the voice utterance. The interpreter 110 can infer that the selected device will be located in the same room where the user is located. As an example, if the transcription reads "turn on the light", the interpreter 110 can determine that the user is located in a bedroom and can infer that the lighting device the user is referring to is a light device associated with a bedroom in the device topology representation 116 (e.g., "ceiling light").

[0059] The interpreter 110 may interpret the transcription using one or more of the above-described interpretation techniques.For example, in various cases, the selected device may be determined based at least in part on one or more of a device identification, a device attribute, or a user location.

[0060] The interpreter 110 may further determine one or more selected actions to be performed. For example, the interpreter 110 may analyze the transcription to determine the one or more actions. For example, the interpreter 110 may analyze a transcription that reads "turn on the light" and may determine that the phrase "turn on (turn on)" corresponds to an action of starting or turning on a device. In some embodiments, the interpreter 110 may determine the action to be performed based at least in part on device properties specified in the device topology representation 116. For example, once the selected device has been determined, the interpreter 110 may compare the transcription to the device properties of the selected device to determine a correlation between the transcription and the device properties.

[0061] Once the selected device(s) and the action(s) to be performed have been determined, one or more control commands may be determined to instruct the selected device(s) to perform the appropriate action(s). In particular, data representing the selected device(s) and the action(s) to be performed may be provided to the command generator 112. The command generator 112 may determine one or more appropriate control commands for the selected device(s). As noted, the control command(s) may be any appropriate control command for communicating with the selected device(s). In particular, the control command(s) may be determined based at least in part on one or more communication protocols associated with the selected device(s). In this manner, based at least in part on the communication protocol(s) of the selected device(s), the control command(s) may be in any appropriate format or structure. The command generator 112 may provide the control command(s) to the selected device(s) so that the selected device(s) may execute the control command(s) and perform the appropriate action(s).

[0062] Although the system 100 is depicted as having a client-server architecture, it will be appreciated that one or more of the functions of the server 104 (e.g., one or more of the biaser 106, the transcriber 108, the interpreter 110, or the command generator 112) may be performed by one or more other suitable devices, such as, for example, the control device 102. In this manner, it will be appreciated that the control device 102 may include one or more of the following: the biaser 106, the transcriber 108, the interpreter 110, or the command generator 112. .

[0063] Figure 4 A flowchart of an example method (200) for processing voice commands according to an exemplary embodiment of the present disclosure is depicted. The method (200) may be implemented by one or more computing devices, such as Figure 8 One or more computing devices among the computing devices shown. In addition, Figure 4 The steps performed in a specific order are depicted for the purpose of illustration and discussion. One of ordinary skill in the art using the disclosure provided herein will appreciate that the steps of any method discussed herein may be adjusted, rearranged, expanded, omitted or modified in various ways without departing from the scope of the present disclosure.

[0064] At (202), the method (300) may include receiving data representing a voice utterance from a user. As noted, a user may provide a voice utterance to a control device in order to control one or more smart devices.

[0065] At (204), method (200) may include accessing a device topology representation associated with a user. As noted, a user may set or construct a device topology representation that defines a topology of one or more buildings associated with the user. For example, a user may organize a building into rooms, and devices may be assigned to rooms. A user may assign device identifiers to devices. The device topology representation may further include one or more device attributes for the device. The device attributes may specify device type, capabilities, etc. In this manner, the device topology representation may allow a user to organize a smart home or Internet of Things by the location of the devices.

[0066] At (206), method (200) may include determining a transcription of the speech utterance based at least in part on the device topology representation. For example, the transcription may be determined based at least in part on one or more language models. For example, in some embodiments, the transcription may be determined by biasing a general language model toward one or more items specified in the device topology representation (e.g., device identification, room name, building name, device attributes, etc.). In this manner, a personalized language model may be generated with increased probability estimates associated with these items of the device topology representation. Data representing the speech utterance may be provided to the personalized language model to determine a transcription of the speech utterance. In some embodiments, data representing the speech utterance may be provided to one or more additional language models to determine a transcription.

[0067] At (208), method (200) may include determining an interpretation of the transcription based at least in part on the device topology representation. For example, determining the interpretation may include determining one or more selected devices and one or more actions to be performed by the selected devices based at least in part on the transcription and the device topology representation. The one or more selected devices may be devices specified in the device topology representation, and the one or more actions to be performed may be actions associated with one or more device attributes associated with the selected devices.

[0068] At (210), the method (200) may include generating one or more control commands for the one or more selected devices based at least in part on the interpretation. The one or more control commands may include instructions for the corresponding selected device to perform an appropriate action. The control command may be any suitable control command that can be understood by the associated smart device. At (212), the method (200) may include providing the one or more control commands to the selected device so that the selected device can execute the control command and perform the appropriate action.

[0069] Figure 5A flowchart of an exemplary method (300) for generating a personalized language model according to an exemplary embodiment of the present disclosure is depicted. The method (300) may be implemented by one or more computing devices, such as Figure 8 In some embodiments, the method (300) may be performed by Figure 1 The bias device 106 depicted in FIG.

[0070] At (302), the method (300) may include accessing a common language model. As noted, the common language model may include a plurality of common phrases. A phrase may be a sequence of one or more words. The common language model may also include a probability estimate associated with each phrase, representing an estimate of the probability of occurrence of the associated phrase.

[0071] At (304), the method (300) may include identifying one or more biasing terms or phrases from the device topology representation. For example, the one or more biasing terms or phrases may include device identification, room identification, building identification, device attributes, and / or other suitable terms or phrases associated with the device topology representation.

[0072] At (306), the method (300) may include biasing the general language model based at least in part on the one or more bias terms. For example, biasing the general language model may include adjusting one or more probability estimates of the general language model based at least in part on the bias terms. In particular, biasing the general language model may include increasing the probability estimates of the bias terms to increase the estimated probability of occurrence of the bias terms. At (308), the method (300) may include generating a personalized language model based at least in part on the bias. For example, the personalized language model may be generated based on the general language model. The personalized language model may include the same set of phrases as the general language model, but may include probability estimates biased toward the bias terms.

[0073] Figure 6 A flowchart of an exemplary method (400) for determining a text transcription of a speech utterance according to an exemplary embodiment of the present disclosure is depicted. The method (400) may be implemented by one or more computing devices, such as Figure 8 In some embodiments, the method (300) may be performed by Figure 1 The transcriber 108 depicted in FIG.

[0074] At (402), the method (400) may include providing data representing a speech utterance to a general language model. At (404), the method (400) may include determining a first transcription of the speech utterance based at least in part on the general language model. The first transcription may be determined using a variety of suitable speech recognition techniques. At (406), the method (400) may include determining a first confidence score for the first transcription. The first confidence score may represent an estimate of the accuracy of the first transcription of the speech utterance.

[0075] At (408), method (400) may include providing data representing the speech utterance to a personalized language model. The personalized language model may be determined based at least in part on a topology of the smart home or the Internet of Things (e.g., specified in the device topology representation). For example, the personalized language model may be Figure 5 In (410), the method (400) may include determining a second transcription of the speech utterance based at least in part on the personalized language model. The second transcription may be determined using a variety of suitable speech recognition techniques. In (412), the method (400) may include determining a second confidence score for the second transcription. The second confidence score may represent an estimate of the accuracy of the second transcription of the speech utterance.

[0076] At (414), method (400) may include selecting a transcription to be interpreted based at least in part on the confidence scores. For example, a transcription with a higher confidence score may be determined to be a more accurate transcription of the speech utterance and may be selected as the transcription to be interpreted. In some embodiments, one or more additional language models may be used to determine one or more additional transcriptions. Additional confidence scores may be determined for the corresponding additional transcriptions. The transcription to be interpreted may be selected based at least in part on each confidence score.

[0077] Figure 7 A flowchart of an exemplary method (500) for interpreting a transcription of a speech utterance according to an exemplary embodiment of the present disclosure is depicted. The method (500) may be implemented by one or more computing devices, such as Figure 8 In some embodiments, the method (300) may be performed by Figure 1 The interpreter 110 depicted in FIG.

[0078] At (502), the method (500) may include comparing the transcription to one or more device identifications specified in the device topology representation. At (504), the method (500) may include determining whether a selected device can be determined based at least in part on the device identification. For example, if a correlation is generated between the device identification and the transcription, the selected device can be determined. If one or more device identifications match one or more items in the transcription, and / or if one or more device identifications are sufficiently similar to one or more items in the transcription, a correlation can be generated. In some embodiments, if a correlation is generated between multiple devices specified in the device topology representation and the transcription, the selected device may not be determined. For example, if multiple devices have the same or similar device identifications, the selected device may not be determined. If the selected device can be determined, then at (506), the method (500) may include determining the selected device and an action to be performed by the selected device.

[0079] If the selected device cannot be determined, then at (508), method (500) may include comparing the transcript to one or more device attributes specified in the device topology representation. As noted, the device attributes may represent a device type and / or one or more capabilities of the device. In this manner, the device attributes may be compared to one or more items in the transcript to determine a correlation between the one or more items and the device attributes. The correlation between the transcript and the one or more device attributes may provide evidence regarding the selected device. For example, if a correlation is found, it may be determined that only devices having the correlated device attributes should be selected as the selected device. In this manner, the correlation may help narrow the range of devices when determining the selected device.

[0080] At (510), the method (500) may include determining whether the selected device can be determined. In particular, determining whether the selected device can be determined may include determining whether the selected device can be determined based at least in part on a comparison of the transcription with the one or more device attributes and / or the one or more device identifications. If the selected device can be determined, the method (500) may return to (506) and the one or more actions to be performed may be determined.

[0081] If the selected device cannot be determined, then at (512), the method (500) may include determining the location of the user relative to the device topology representation. As noted, in the case where the control device is a device specified within the organizational topology of the device topology representation, the user's location may correspond to the location of the control device in the device topology representation. In the case where the control device is not specified in the device topology representation, the user's location may be determined based on various suitable location determination techniques (e.g., techniques using wireless access points and / or beacon devices). The user's location so determined may then be mapped to the buildings and rooms specified in the device topology representation.

[0082] At (514), method (500) may include comparing the relative user location to a device topology specified in the device topology representation. For example, the user's location in a building may be compared to the devices specified in the device topology representation that are associated with the location. In this manner, it may be inferred that the selected device is a device located in the room in which the user is located. For example, if the transcription reads "turn on the lights," the devices located in the room in which the user is located may be analyzed to identify one or more lighting devices associated with the room. In this manner, it may be inferred that the device selected as the selected device should be a lighting device located in the room in which the user is currently located.

[0083] At (516), method (500) may include determining whether the selected device can be determined. In particular, determining whether the selected device can be determined may include determining whether the selected device can be determined based at least in part on a comparison of the transcription to the one or more device attributes, the one or more device identifications, and / or a relative user location. If the selected device can be determined, method (500) may include returning to (506), and an action to be performed may be determined. If the selected device cannot be determined, at (518), method (500) may include requesting more information from the user. For example, a prompt may be presented to the user in a user interface of a control device or other suitable device, asking the user for more information about the voice command.

[0084] It should be understood that in various cases, the selected device and / or action to be performed may be determined based at least in part on one or more of a device identification, a device attribute, or a relative user location. In this manner, various data associated with the user and / or device topology representation may be accessed and / or determined to determine or infer the one or more selected devices associated with the speech utterance. Depending on the particular organization of the transcription and device topology representation, various suitable combinations of the factors referenced above may be used to determine the selected device and / or action to be performed. In this manner, it should be understood that various aspects of the device topology representation and / or user location may be used in various ways to determine the selected device and / or action to be performed.

[0085] Figure 8 An exemplary computing system 700 is depicted that can be used to implement methods and systems according to exemplary aspects of the present disclosure. The system 700 can be implemented using a client-server architecture including a server 710 communicating with one or more client devices 730 via a network 740. The system 700 can be implemented using other suitable architectures, such as a single computing device.

[0086] The system 700 includes a server 710, such as a network server. The server 710 can be implemented using any suitable computing device. The server 710 can have one or more processors 712 and one or more memory devices 714. The server 710 can also include a network interface for communicating with one or more client devices 730 via a network 740. The network interface can include any suitable components for connecting to one or more networks, including, for example, a transmitter, a receiver, a port, a controller, an antenna, or other suitable components.

[0087] The one or more processors 712 may include any suitable processing device, such as a microprocessor, a microcontroller, an integrated circuit, a logic device, or other suitable processing device. The one or more memory devices 714 may include one or more computer-readable media, including but not limited to non-transitory computer-readable media, RAM, ROM, a hard drive, a flash drive, or other memory devices. The one or more memory devices 714 may store information accessible to the one or more processors 712, including computer-readable instructions 716 executable by the one or more processors 712. The instructions 716 may be any set of instructions that, when executed by the one or more processors 712, cause the one or more processors 712 to perform operations. For example, the instructions 716 may be executed by the one or more processors 712 to implement reference Figure 1 The biaser 106, transcriber 108, interpreter 110 or command generator 112 are described.

[0088] like Figure 8 As shown, the one or more memory devices 714 can also store data 718 that can be retrieved, manipulated, created or stored by the one or more processors 712. The data 718 can include, for example, language model data, device topology structure representation data, and other data. The data 718 can be stored in one or more databases. The one or more databases can be connected to the server 710 via a high-bandwidth LAN or WAN, or can also be connected to the server 710 via a network 740. The one or more databases can be split so that they are located in multiple places.

[0089] The server 710 may exchange data with one or more client devices 730 via a network 740. Any number of client devices 730 may be connected to the server 710 via a network 740. One or more client devices 730 may be smart devices configured to provide home automation services, smart home services, Internet of Things services, and / or other suitable services. Smart devices may include or may be otherwise associated with the following devices: air conditioning systems, lighting devices, home theater and entertainment systems, security systems, automatic door lock systems, thermostat devices, home automation systems, sound speakers, camera devices, treadmills, weight scales, smart beds, irrigation systems, garage door openers, home appliances, baby monitors, fire alarms, and / or other suitable devices or systems. One or more of the client devices may also be user computing devices, such as general-purpose computers, special-purpose computers, laptops, desktop computers, mobile devices, navigation systems, smart phones, tablet computers, wearable computing devices, displays with one or more processors, or other suitable computing devices.

[0090] Similar to the server 710, the client device 730 may include one or more processors 732 and a memory 734. The one or more processors 732 may include one or more central processing units (CPUs), graphics processing units (GPUs) dedicated to efficiently rendering images or performing other specialized calculations, and / or other processing devices. The memory 734 may include one or more computer-readable media and may store information accessible to the one or more processors 732, including instructions 736 and data 738 that may be executed by the one or more processors 732. For example, the memory 734 may store instructions 736 for implementing a user interface.

[0091] Figure 8The client device 730 may include various input / output devices 737 for providing and receiving information from the user, such as a touch screen, a touch pad, data input keys, a speaker, and / or a microphone suitable for speech recognition. For example, according to an exemplary aspect of the present disclosure, the client device 730 may have a display device 735 for presenting a user interface.

[0092] The client device 730 may also include a network interface for communicating with one or more remote computing devices (e.g., server 710) over the network 740. The network interface may include any suitable component for connecting to one or more networks, including, for example, a transmitter, receiver, port, controller, antenna, or other suitable component.

[0093] The network 740 may be any type of communication network, such as a local area network (e.g., an intranet), a wide area network (e.g., the Internet), a cellular network, or some combination thereof. The network 740 may also include a direct connection between the client device 730 and the server 710. In general, the communication between the server 710 and the client device 730 may be carried over a network interface using any type of wired and / or wireless connection using a variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, secure HTTP, SSL).

[0094] An exemplary aspect of the present disclosure relates to a computer-implemented method for processing a speech utterance. The method may include receiving, by one or more computing devices, data representing the speech utterance. The speech utterance may be provided by a user to a control device. The method may include accessing, by one or more computing devices, a device topology representation. The device topology representation may define a plurality of smart devices associated with one or more buildings. The device topology representation may further define the location of each of the plurality of devices and the control device within the associated building. The method may include determining, by the one or more computing devices, one or more selected devices and one or more actions to be performed by the one or more selected devices based at least in part on the data representing the speech utterance and further at least in part on the location of the control device defined by the device topology representation within the associated building.

[0095] In some embodiments, determining, by the one or more computing devices, the one or more selected devices may include: identifying, by the one or more computing devices, a subset of the multiple devices that share the same location as the control device; and determining, by the one or more computing devices, the one or more selected devices from the subset of the identified multiple devices.

[0096] In some embodiments, the device topology representation may define one or more rooms in each of the one or more buildings. The location of each of the plurality of devices and control devices defined by the device topology representation may correspond to at least one of the one or more rooms, respectively.

[0097] In some embodiments, determining, by the one or more computing devices, the one or more selected devices may include: identifying, by the one or more computing devices, a subset of the multiple devices that are in the same room as the control device; and determining, by the one or more computing devices, the one or more selected devices from the subset of the identified multiple devices.

[0098] In some embodiments, determining, by the one or more computing devices, the one or more selected devices from a subset of the identified multiple devices may include: identifying, by the one or more computing devices, one or more device attributes referenced by the voice utterance; and identifying, by the one or more computing devices, which one of the subset of the multiple devices has the one or more device attributes.

[0099] In some embodiments, the control device is one of the plurality of devices.

[0100] In some embodiments, the device topology representation may specify one or more device identifications and one or more device attributes for each of the plurality of devices.

[0101] In some embodiments, determining, by the one or more computing devices, the one or more selected devices and the one or more actions may include determining, by the one or more computing devices, the one or more selected devices and the one or more actions based at least in part on at least one of the one or more device identifiers specified in the device topology representation.

[0102] In some embodiments, determining, by the one or more computing devices, the one or more selected devices and the one or more actions may include determining, by the one or more computing devices, the one or more selected devices and the one or more actions based at least in part on at least one of the one or more device attributes specified in the device topology representation.

[0103] In some embodiments, the method may further include determining, by the one or more computing devices, one or more control commands based at least in part on the one or more selected devices and the one or more actions, the one or more control commands respectively instructing the one or more selected devices to perform one or more actions. In some embodiments, the method may further include providing, by the one or more computing devices, the one or more control commands to the one or more selected devices.

[0104] In some embodiments, the one or more computing devices executing the method may consist of a control device.

[0105] Another exemplary aspect of the present disclosure relates to a computing system comprising one or more processors and one or more memory devices. The one or more memory devices may store computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations. The operations may include: receiving data representing a voice utterance. The voice utterance may be provided by a user to a control device. The operations may include accessing a device topology representation. The device topology representation may define a plurality of smart devices associated with one or more buildings. The device topology representation may further define the location of each of the plurality of devices and control devices within the associated building. The operations may include determining one or more selected devices and one or more actions to be performed by the one or more selected devices based at least in part on the data representing the voice utterance and further at least in part on the location of the control device defined by the device topology representation within the associated building.

[0106] In some embodiments, determining the one or more selected devices may include: identifying a subset of a plurality of devices that share a common location with the control device; and determining the one or more selected devices from the identified subset of the plurality of devices.

[0107] In some embodiments, the device topology representation may define one or more rooms within each of the one or more buildings. The location of each of the plurality of devices and control devices defined by the device topology representation may correspond to at least one of the one or more rooms, respectively.

[0108] In some embodiments, determining the one or more selected devices may include: identifying a subset of a plurality of devices that are in the same room as the control device; and determining the one or more selected devices from the identified subset of the plurality of devices.

[0109] In some embodiments, determining the one or more selected devices from a subset of the identified multiple devices may include: identifying one or more device attributes referenced by the voice utterance; and identifying which of the subset of the multiple devices has the one or more device attributes.

[0110] In some embodiments, the operation may further include: determining one or more control commands based at least in part on the one or more selected devices and the one or more actions, the one or more control commands respectively instructing the one or more selected devices to perform the one or more actions; and providing the one or more control commands to the one or more selected devices.

[0111] In some embodiments, the computing system may include a control device, and the control device may include the one or more processors and the one or more memory devices.

[0112] Another exemplary aspect of the present disclosure relates to a computer-implemented method. The method may include obtaining, by one or more computing devices, a device topology representation that describes a corresponding position of each of a plurality of devices included in the device topology representation. The plurality of devices may include one or more control devices. The method may include obtaining, by the one or more computing devices, a voice utterance. The method may include identifying, by the one or more computing devices, a first control device among the one or more control devices to which a user provides a voice utterance. The method may include determining, by the one or more computing devices, a subset of a plurality of devices described by the device topology representation as having the same position as the first control device to which the voice utterance is provided. The method may include determining, by the one or more computing devices, one or more selected devices from a subset of a plurality of devices based, at least in part, on data representing the voice utterance.

[0113] In some embodiments, the device topology representation may specify a particular room for each of the plurality of devices. In some embodiments, determining, by the one or more computing devices, a subset of the plurality of devices may include: identifying, by the one or more computing devices, the subset of the plurality of devices, for which the device topology representation specifies the same room as the first control device to which the voice utterance was provided.

[0114] In some embodiments, the one or more computing devices may consist of a first control device, such that the first control device performs a computer-implemented method. In some embodiments, the multiple devices may include multiple control devices. In some embodiments, the multiple control devices may be associated with multiple locations, respectively. For example, the multiple control devices may be associated with multiple rooms, respectively.

[0115] The technology discussed herein refers to servers, databases, software applications, and other computer-based systems, as well as the actions and information sent to and from these systems. It will be appreciated by those skilled in the art that the inherent flexibility of computer-based systems allows for various possible configurations, combinations, and divisions between and among components for tasks and functions. For example, the server processes discussed herein can be implemented using a single server or multiple servers that work in combination. Databases and applications can be implemented on a single system or distributed on multiple systems. Distributed components can operate sequentially or in parallel.

[0116] Although the subject matter of the present invention has been described in detail with respect to specific exemplary embodiments of the subject matter of the present invention, it should be understood that those skilled in the art can easily produce substitutions, changes and equivalents to these embodiments after knowing and understanding the foregoing. Therefore, the scope of the present disclosure is by way of example rather than limitation, and the present disclosure does not exclude such modifications, changes and / or additions to the subject matter of the present invention, which will be apparent to those of ordinary skill in the art.

Claims

1. A computer-implemented method for processing a speech utterance, the method comprising: receiving, by one or more computing devices, data indicative of a speech utterance; A device topology representation is accessed by the one or more computing devices, the device topology representation defining: a plurality of smart devices associated with one or more structures, a respective location of each of the plurality of smart devices within the associated structure, and one or more device properties of each of the plurality of smart devices, the one or more device properties being associated with one or more corresponding actions that each of the plurality of smart devices is configured to perform; identifying, by the one or more computing devices, a first control device in the one or more devices to which the voice utterance is provided by a user; Determining, by the one or more computing devices, one or more selected smart devices from the plurality of smart devices, wherein determining the one or more selected smart devices is based on: said data indicative of a speech utterance, the one or more device attributes of each of the plurality of smart devices, and The one or more selected intelligent devices have the same position in the device topology representation as the first control device, and wherein one or more device identifiers correspond to one or more of the selected smart devices and the device identifiers are not included in the data indicative of the voice utterance; generating, by the one or more computing devices, one or more control commands for the one or more selected smart devices based at least in part on the data indicative of the voice utterance; and The one or more control commands are provided by the one or more computing devices to the one or more selected smart devices.

2. The computer-implemented method of processing speech utterances of claim 1, wherein: The device topology representation defines a plurality of rooms, and wherein a corresponding location of each of the plurality of smart devices is a corresponding one of the plurality of rooms.

3. The computer-implemented method of processing speech utterances of claim 1, wherein: The one or more computing devices include the first control device.

4. The computer-implemented method of processing speech utterances of claim 1, wherein: The one or more computing devices are comprised of the first control device.

5. The computer-implemented method of processing speech utterances of claim 1, wherein: The smart device determining the one or more selections includes: identifying, by the one or more computing devices, a subset of the plurality of smart devices, the device topology representation designating for the subset of the plurality of devices a same room as the first control device to which the voice utterance was provided; and The one or more selected smart devices are determined, by the one or more computing devices, from the subset of the plurality of smart devices based at least in part on the data indicative of a speech utterance.

6. The computer-implemented method of processing speech utterances of claim 1, wherein: Generating one or more control commands for the one or more selected smart devices based at least in part on the data indicative of the voice utterance includes: determining one or more actions to be performed by the one or more selected smart devices based at least in part on the transcription of the speech utterance; and The one or more control commands are generated based at least in part on the one or more selected smart devices and the one or more actions.

7. A computer-implemented method of processing speech utterances according to any one of claims 1 to 6, wherein: The one or more selected smart devices are one or more lighting devices.

8. A computing system comprising: one or more processors; as well as one or more memory devices storing computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: receiving data indicative of a speech utterance; Accessing a device topology representation, the device topology representation defines: a plurality of smart devices associated with one or more structures, a respective location of each of the plurality of smart devices within the associated structure, and one or more device properties of each of the plurality of smart devices, the one or more device properties being associated with one or more corresponding actions that each of the plurality of smart devices is configured to perform; identifying a first control device among the one or more devices to which the voice utterance is provided by a user; Determine one or more selected smart devices from the plurality of smart devices, wherein determining the one or more selected smart devices is based on: said data indicative of a speech utterance, the one or more device attributes of each of the plurality of smart devices, and The one or more selected intelligent devices have the same position in the device topology representation as the first control device, and wherein one or more device identifiers correspond to one or more of the selected smart devices and the device identifiers are not included in the data indicative of the voice utterance; generating one or more control commands for the one or more selected smart devices based at least in part on the data indicative of the voice utterance; and The one or more control commands are provided to the one or more selected smart devices.

9. The computing system of claim 8, wherein: The device topology representation defines a plurality of rooms, and wherein a corresponding location of each of the plurality of smart devices is a corresponding one of the plurality of rooms.

10. The computing system of claim 8, wherein: The computing system includes the first control device.

11. The computing system of claim 8, wherein: The computing system is composed of the first control device.

12. The computing system of claim 8, wherein: The smart device determining the one or more selections includes: identifying a subset of the plurality of smart devices, the device topology indicating a room designated for the subset of the plurality of devices to be the same as the first control device to which the voice utterance was provided; and The one or more selected smart devices are determined from the subset of the plurality of smart devices based at least in part on the data indicative of a voice utterance.

13. The computing system of claim 8, wherein: Generating one or more control commands for the one or more selected smart devices based at least in part on the data indicative of the voice utterance includes: determining one or more actions to be performed by the one or more selected smart devices based at least in part on the transcription of the speech utterance; and The one or more control commands are generated based at least in part on the one or more selected smart devices and the one or more actions.

14. The computing system of any one of claims 8 to 13, wherein: The one or more selected smart devices are one or more lighting devices.

15. One or more non-transitory computer-readable storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: receiving data indicative of a speech utterance; Accessing a device topology representation, the device topology representation defines: a plurality of smart devices associated with one or more structures, a respective location of each of the plurality of smart devices within the associated structure, and one or more device properties of each of the plurality of smart devices, the one or more device properties being associated with one or more corresponding actions that each of the plurality of smart devices is configured to perform; identifying a first control device among the one or more devices to which the voice utterance is provided by a user; Determine one or more selected smart devices from the plurality of smart devices, wherein determining the one or more selected smart devices is based on: said data indicative of a speech utterance, the one or more device attributes of each of the plurality of smart devices, and The one or more selected intelligent devices have the same position in the device topology representation as the first control device, and wherein one or more device identifiers correspond to one or more of the selected smart devices and the device identifiers are not included in the data indicative of the voice utterance; generating one or more control commands for the one or more selected smart devices based at least in part on the data indicative of the voice utterance; and The one or more control commands are provided to the one or more selected smart devices.

Images