Method for associating a transient account with a voice-enabled device

Abstract

The present invention discloses a system and method for temporarily associating a user account with a voice-enabled device. The voice-enabled device that can be associated with a default account can be located in an environment, such as a hotel room. A user may wish to utilize the voice-enabled device to perform one or more actions. In examples, some or all of the actions may require the use of system resources that are not available to the default account. In these examples, the user can provide contact information for sending a message to the user's personal device to obtain user account information. Upon receiving such information, the user account can be temporarily associated with the voice-enabled device and can be disassociated from the voice-enabled device when a time period has passed and / or a disassociation event occurs.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Patent Application No. 16 / 001,763, filed on June 6, 2018, and U.S. Patent Application No. 16 / 001,823, filed on June 6, 2018, both of which are incorporated herein by reference in their entirety. Background Art

[0003] People currently use voice-enabled devices in their homes. It may be desirable for voice-enabled devices to have other uses. This article describes technical improvements and solutions to technical problems that can be used to improve the usability of voice-enabled devices for users. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The following detailed description is made in conjunction with the accompanying drawings. In the drawings, the leftmost digit of a reference number indicates the drawing in which the reference number first appears. The use of the same reference number in different drawings indicates similar or identical items. The systems shown in the drawings are not drawn to scale, and the components in the drawings may not be drawn to scale relative to each other.

[0005] Figure 1A A schematic diagram of an example environment for transient account association with a voice-enabled device is shown.

[0006] Figure 1B Shows Figure 1A 00100 00101 0010102 0010103 0010203 0010304 0010305 0010406 ​​0010307 0010408 00105 0010609 00105 0010608 00105 0010609 0010308

[0007] Figure 2 A flow diagram of an example process for temporarily associating an account with a voice-enabled device is shown.

[0008] Figure 3 A flow chart of an example process for utilizing a voice-enabled device that has been temporarily associated with an account is shown.

[0009] Figure 4 A flow diagram of an example process for temporarily re-associating an account with a voice-enabled device using contact information is shown.

[0010] Figure 5 A flow diagram of an example process for temporarily re-associating an account with a voice-enabled device using a voice model is shown.

[0011] Figure 6 An example environment for temporary account association with one or more voice-enabled devices is shown.

[0012] Figure 7A flow chart of an example process for temporary account association with a voice-enabled device is shown.

[0013] Figure 8 A flow chart of another example process for temporary account association with a voice-enabled device is shown.

[0014] Fig. 9 A flow chart of another example process for temporary account association with a voice-enabled device is shown.

[0015] Fig.10 A flow diagram of an example process for disassociating an account from a voice-enabled device is shown.

[0016] Fig.11 A flow chart illustrating another example process for disassociating an account from a voice-enabled device is shown.

[0017] Fig.12 A conceptual diagram showing components of a speech processing system for processing audio data provided by one or more devices.

[0018] Fig.13 A conceptual diagram of components of a voice processing system for establishing a temporary account association with a voice-enabled device is shown. DETAILED DESCRIPTION

[0019] This paper describes the system and method for establishing a temporary account association with a voice-enabled device. Take a hotel or other environment where a user may temporarily live as an example. The hotel may include many rooms and / or suites, and some or all of these rooms may be equipped with one or more voice-enabled devices. The voice-enabled device may be any audio that can be captured (such as capturing audio from a person who speaks), generates audio data to be processed, and performs the operation associated with the audio and / or confirms that a certain action has been performed by another device and / or system. Such equipment may include, for example, voice assistant, television, door lock and / or window lock, light bulb, clock, alarm, telephone, kitchenware, etc. The voice-enabled device in the hotel may be associated with one or more user accounts associated with the hotel.

[0020] When a guest is in a hotel room, the guest may wish to interact with the voice-enabled device, such as saying "What's the weather outside," "Play song A," "Turn off the lights," "Order a pizza," "Buy product A," etc. The microphone of the voice-enabled device may capture audio representing the user's speech, and may generate corresponding audio data to be sent, for example, to a remote system. The remote system may determine the intent corresponding to the user's speech, such as by generating intent data. For example, the intent may be to output audio indicating the weather at and / or around the location of the voice-enabled device, to output audio corresponding to song A, to stop a light bulb located in the hotel room from emitting light, etc.

[0021] In some examples, the hotel's user account (which may be described as a default user account) may have access to and / or be permitted to utilize one or more resources to perform an action associated with the intent. For example, the default user account may have access to a weather application. In other examples, the default user account may not have access to and / or be permitted to utilize resources to perform the action. For example, the default user account may not have access to a music resource to output a requested song, or may not have access to a resource representing a payment instrument to purchase a product or order a pizza. In these examples, the hotel's default user account may not be used to perform the action requested by the user.

[0022] In order to improve the experience of using voice-enabled devices in hotels or other environments where users temporarily live, the present invention can use technology to establish a temporary association between a personal user account and a voice-enabled device. For example, audio data representing user speech can be received by a remote system, and can be analyzed to determine that the resource used to implement the intention associated with the user speech is inaccessible and / or the default account associated with the voice-enabled device is not authorized to access the resource. In these examples, instruction data can be generated by a remote system and can be sent to the voice-enabled device. The instruction data can cause the voice-enabled device to output a request for information, such as via one or more speakers of the voice-enabled device, to establish a temporary association between a personal user account associated with the user and the voice-enabled device. The request can represent a request for contact information associated with a personal user account. The contact information can be, for example, a telephone number associated with a personal user account, and / or other contact information, such as an email address and / or a social media account identifier.

[0023] The user may then provide another user utterance including the contact information to the voice-enabled device. Audio representing the user utterance may be captured by a microphone of the voice-enabled device, and corresponding audio data may be generated and sent to, for example, a remote system. The remote system may perform automatic speech recognition on the audio data to generate corresponding text data representing the user utterance. The text data may include a text representation of the contact information provided by the user. The remote system may utilize the contact information to send a message to a personal device associated with the contact information, which message will be used to temporarily associate the personal user account with the voice-enabled device. For example, the message may be a short message service (SMS) message that includes an optional portion such as a web page link, which, when selected by a user of the personal device, may allow receipt of account access information, such as a username and / or password associated with the personal user account.

[0024] In some examples, when selected, the optional portion may cause a website application, such as a website access and / or browsing application, to open and display a web page that allows the user to enter account access information. In other examples, when selected, the optional portion of the message may cause an application associated with the user account, such as a companion application, to be accessed. The companion application may have access to the account access information and may be used to obtain the account access information without requiring the user to enter the information into, for example, a website application. Data corresponding to the account access information may be received from the personal device and may be used by the remote system to determine which user account is associated with the voice-enabled device. The remote system may then temporarily associate the user account with a device identifier associated with the voice-enabled device.

[0025] Once a user account is associated with a voice-enabled device, subsequent user utterances may be captured by the microphone of the voice-enabled device, and corresponding audio data may be generated and sent to a remote system for processing. An intent associated with such user utterance may be determined, and resources associated with the active account may be used to perform one or more actions associated with the intent, such as playing a particular song, ordering a product, requesting personal account information, performing one or more implicit actions based on the user utterance, and the like.

[0026] Additionally or alternatively, the present invention may provide techniques to disassociate a user account from a voice-enabled device, such as in the case where the voice-enabled device is located in an environment where the user is temporarily residing. Using the examples provided above, when a user enters account access information, and / or when a user account is associated with a voice-enabled device, a website application or another application may be caused to display one or more fields that may allow a user of a personal device to enter check-out information. For example, a user may be prompted to indicate a check-out date and / or check-out time for the user's stay at a hotel. This information may be used to determine a time period for maintaining an association between a user account and a voice-enabled device.

[0027] Additionally or alternatively, the time period can be based on data received from one or more systems associated with the environment, such as room scheduling data. Additionally or alternatively, the time period can be based at least in part on the occurrence of a disassociation event, which can be, for example, an indication that the user has checked out from the environment, a user request to disassociate an account is received, and / or an indication that the user is temporarily away from the residence during the time period is received. By doing so, when requested and / or desired, a user account can be temporarily associated with a voice-enabled device and can be disassociated to prevent others from inadvertently using personal account resources and information.

[0028] The present disclosure provides an overall understanding of the principles of structure, function, manufacture, and use of the systems and methods disclosed herein. One or more examples of the present disclosure are shown in the accompanying drawings. It will be understood by those of ordinary skill in the art that the systems and methods specifically described herein and shown in the accompanying drawings are non-limiting embodiments. Features shown or described in conjunction with one embodiment may be combined with features of other embodiments, including as between systems and methods. Such modifications and variations are intended to be included within the scope of the appended claims.

[0029] Further details are described below with reference to several example embodiments.

[0030] Figure 1A A schematic diagram of an example system 100 for establishing a temporary account association with a voice-enabled device is shown. The system 100 may include, for example, one or more voice-enabled devices 102, which may be located in one or more rooms and / or suites associated with an environment in which a person may temporarily reside. The system may additionally include a personal device 106, which may be a device associated with a user located within the environment. The system 100 may additionally include one or more components that may be remote from the voice-enabled device 102 and / or the personal device 106. These components may include, for example, a managed attribute voice widget 160, a customer authentication component 162, a device verification web application 164, a data encryption component 166, a notification component 168, an association management component 170, a voice widget web application 172, a policy retrieval component 174, one or more managed attribute settings 176, one or more domain voice widgets 178, an agent 180, and an OPF 182. Each of these components will be described in detail below by one or more examples.

[0031] For example, audio data may be generated via one or more microphones associated with the voice-enabled device 102. The audio data may correspond to user utterances for explicitly or implicitly performing an action. The audio data may be received by a remote system associated with the voice-enabled device 102, where automatic speech recognition and / or natural language understanding techniques may be utilized to determine the intention of performing an action. The remote system may determine that the resources to be used to perform the action are not available for use by the default user account associated with the voice-enabled device. For example, a user utterance may be "buy Chinese food for delivery", and the resources and / or domain voice widgets for implementing the intention of ordering Chinese food for delivery may not be available for use by the default user account associated with the voice-enabled device 102. In these examples, it may be desirable to temporarily associate the user account of the user with the voice-enabled device 102, such as when the user account is associated with a resource to be used to implement a given intention.

[0032] In these examples, audio may be output via a speaker of the voice-enabled device 102. The audio may represent a request for contact information to begin the process of temporarily associating a user account with a device identification of the voice-enabled device 102. The microphone of the voice-enabled device 102 may capture audio corresponding to user speech representing the contact information, and corresponding audio data may be generated. The audio data may be processed using ASR and / or NLU techniques. The managed attribute voice widget 160 may receive intent data indicating an intent to initiate a temporary account association based at least in part on the contact information provided. A request to "seal" or otherwise encrypt the device identification associated with the voice-enabled device 102 may be sent to the data encryption component 166, which may seal or otherwise encrypt the device identification. Data representing the encrypted device identification may be received by the managed attribute voice widget 160.

[0033] In an example, the managed attribute voice widget 160 may additionally request a code pair from the customer authentication component 162. The code pair may include and / or represent a uniform resource locator (URL) for inputting information and / or disassociating a sealed device identification. The managed attribute voice widget 160 may receive the code pair and may send the code pair, the device identification, and the contact information to the notification component 168. The notification component 168 may identify, determine, and / or generate a message that may include a URL link and / or other optional parts. The message may be sent to the personal device 106.

[0034] The display of the personal device 106 may display the link and / or other optional portions. Once the link and / or other optional portions are selected, the voice widget web application 172 may be launched and may cause a website to be displayed to provide authentication information associated with the user account that will be temporarily associated. Additionally or alternatively, the website may allow the user to provide information indicating when the temporary association between the user account and the device identification should be terminated. In an example, the voice widget web application 172 may call the data encryption component 166 to unseal and / or decrypt the device identification associated with the voice-enabled device 102.

[0035] The voice widget web application 172 may provide the user account identification and / or disassociation information to the association management component 170. The client authentication component 162 may provide the device identification to the association management component 170. Using the user account identification and the device identification, the association management component 170 may associate the user account with the device identification. For example, when the system 100 receives subsequent audio data, the audio data may include data indicating the device identification associated with the voice-enabled device 102. The association management component 170 may be used to identify and / or determine which user account is associated with the device identification in a given time period. If the user account is associated with the device identification, the association management component 170 may provide such indications to other components of the system 100, which may indicate which domain voice widgets 178 and / or resources can be used to perform actions associated with the user utterance.

[0036] Additionally or alternatively, in examples, an application associated with the voice-enabled device 102 can be installed on the personal device 106. In these examples, the device verification web application 164 can verify the device identification associated with the application and can extract or otherwise receive the user account credentials from the personal device 106, such as without requiring the user to enter such credentials. The association management component 170 can utilize this information to associate the user account with the device identification.

[0037] One or more domain voice widgets 178 associated with a user account may be identified and / or determined, or described as "whitelisted" domain voice widgets 178. These domain voice widgets 178 may be used to process audio data representing user speech when the user account is associated with a device identity of a voice-enabled device 102. Additionally or alternatively, managed attribute settings 176 may be received from one or more devices associated with managed attributes. These settings 176 may include enabling or disabling of temporary account associations, indications of which domain voice widgets 178 are allowed, one or more rules for associating and / or disassociating device identities with user accounts, and the like. A policy retrieval component 174 may receive the settings 176 and may utilize those settings to access a domain voice widget 178 and / or process a request to temporarily associate a user account with a device identity.

[0038] Data and / or instructions may be sent via agent 180 Figure 1A 1 and a voice-enabled device, the proxy 180 may format the data and / or utilize one or more application programming interfaces (APIs) to facilitate the communication. Similarly, data and / or instructions may be sent between the components described in FIG. 1 and the personal device 106 via the OPF 182, which may format the data and / or utilize one or more APIs to facilitate the communication.

[0039] Figure 1BA schematic diagram of additional components of an example system 100 for establishing a temporary account association with a voice-enabled device is shown. System 100 may include, for example, one or more voice-enabled devices 102, which may be located in one or more rooms and / or suites associated with environment 104, such as, for example, a hotel, a wedding venue, a study, a classroom, a car, and / or any other room or group of rooms where a person may temporarily reside. System 100 may additionally include personal device 106, which may be a device associated with a user located within environment 104. Personal device 106 may be, for example, a mobile phone, a personal assistant device, a laptop, a tablet, a watch, and / or any other electronic device capable of sending and receiving information.

[0040] The speech-enabled device 102 may include components such as, for example, one or more processors 108, one or more network interfaces 110, a memory 112, one or more microphones 114, one or more speakers 116, and / or one or more displays 118. The microphone 114 may be configured to capture audio, such as user speech, and generate corresponding audio data. The speaker 116 may be configured to output audio, such as in response to receiving audio data and / or instruction data from a remote system 120 via a network 122. The display 118 may be configured to visually present information to a user.

[0041] The personal device 106 may include components such as, for example, one or more processors 124, one or more network interfaces 126, memory 128, one or more microphones 130, one or more speakers 132, and / or one or more displays 134. The microphone 130, speaker 132, and / or display 134 may operate in the same or similar manner as the microphone 114, speaker 116, and / or display 118 of the voice-enabled device 102. The memory 128 of the personal device 106 may include one or more components such as, for example, a messaging application 136, a website application 138, and / or a voice-enabled device application 140. Each of these components will be described in detail below along with the components of the remote system 120.

[0042] The remote system 120 may be configured to receive and send data between the remote system 120, the speech-enabled device 102, and / or the personal device 106, for example. The remote system 120 may include components such as one or more processors 142, one or more network interfaces 144, and a memory 146. The components of the memory 146 may include, for example, a user registry 148 including one or more resources 150, an automatic speech recognition (ASR) component 152, a natural language understanding (NLU) component 154, one or more speech models 156, a text-to-speech (TTS) component 158, a device association component 174, and / or a reference Figure 1AIt should be understood that although Figure 1B Remote system 120 is shown with memory 146 and each of the components described herein, but remote system 120 may describe multiple systems with one or more memories. Each of the components described herein with reference to remote system 120 may be associated with their own system, and / or some or all of these components may be associated with a single system. The components of memory 146 are described in detail below.

[0043] The user registry component 148 may be configured to identify, determine and / or generate associations between users, user profiles, user accounts and / or devices. For example, one or more associations between a user profile and a user account may be identified, determined and / or generated by the user registry 148. The user registry 148 may additionally store information indicating one or more applications and / or resources 150 that are accessible and / or enabled by a given user profile and / or user account. It should also be understood that a user account may be associated with one or more user profiles. It should also be understood that the terms "user account" and / or "user profile" may be used to describe a set of data and / or functions associated with a given account identifier and / or profile identifier. For example, data identified, determined and / or generated when using some or all of the system 100 may be stored or otherwise associated with an account identifier and / or profile identifier. The data associated with a user account and / or profile may include, for example, account / profile access information, historical usage data, device association data and / or preference data. Additionally, as described herein, resources 150 and / or domain voice widget 168 may include one or more applications, skills, capabilities, functions, and / or capabilities of remote system 120 and / or one or more other devices and / or systems that have been authorized and / or otherwise available for use when audio data representing user utterances associated with a given account identifier and / or profile identifier is received. Some non-limiting examples of resources 150 may include functionality associated with playing music, accessing playlists and / or audio file databases, shopping, controlling accessory devices (such as light bulbs and / or locks), accessing contact information associated with a contact, and / or sending / or receiving messages.

[0044] The ASR component 152 may be configured to receive audio data that may represent human speech, such as user speech, and generate text data corresponding to the audio data. The text data may include words corresponding to the human speech. The NLU component 154 may be configured to determine one or more intents associated with the human speech based at least in part on the text data. Fig.12ASR component 152 and NLU component 154 are described in further detail. For purposes of illustration, ASR component 152 and NLU component 154 may be utilized to determine one or more intents to perform an action utilizing one or more of resources 150 and / or an intent to temporarily associate a user account with voice-enabled device 102.

[0045] For example, a user may speak a user utterance to the voice-enabled device 102. The microphone 114 of the voice-enabled device 102 may capture the user utterance and may generate corresponding audio data. The audio data may be sent to the remote system 120 via the network 122. The ASR component 152 may process the audio data and generate corresponding text data. The NLU component 154 may utilize the text data to determine intent data representing the user's intent (in these examples, the intent is to perform an action that utilizes one or more of the resources 150) and / or the intent to temporarily associate the user account with the voice-enabled device 102. For example, the user utterance may be "play song A". The remote system 120 may utilize the NLU component 154 to determine that the resource 150 is not available for performing an action to output audio corresponding to song A. For example, the voice-enabled device 102 may be associated with a default user account associated with the environment 104. The default user account may not have access to the resource 150 to output song A. For example, the default user account may not be associated with a music service and / or database that allows song A to be played. In these examples, the voice widget 160 of the remote system 120 may be utilized to initiate a process of temporarily associating a personal user account associated with the user with the voice-enabled device.

[0046] The voice widget 160 may receive the intent data from the NLU component 154, and may utilize the intent data to identify, determine, and / or generate instruction data corresponding to the intent data. For example, the voice widget 160 may be invoked and used to generate instruction data that, when sent to the voice-enabled device 102 and enabled by the voice-enabled device, results in outputting audio, such as via the speaker 116, that represents a request for information to temporarily associate a user account with the voice-enabled device 102. The TTS component 158 ​​may be used to generate audio data that will be used by the voice-enabled device 102 to output the audio as indicated by the instruction data generated by the voice widget 160. For example, the output audio may be "We cannot perform this operation, please provide your phone number to link your account to this device." The example audio provided above is provided by way of example only, and not by way of limitation. It should be understood that although the example of contact information is provided as a phone number, the contact information may additionally or alternatively include, for example, an email address and / or a social media account identifier.

[0047] The voice widget 160 described herein may include a voice-enabled web component that can be run in the remote system 120. The voice widget 160 can receive voice-initiated requests and respond. The voice widget 160 can define the lifecycle events of the skill as experienced by the user, the way to handle voice requests from the user and / or the callback of events occurring on the device interacting with the user. A given voice widget may be able to handle certain intents. For example, the NLU component 154 can generate intent data of an indication diagram and a payload associated with the intent. The voice widget can be associated with the intent, and the intent data including the payload can therefore be sent to the voice widget and received by the voice widget. The voice widget can process the intent by analyzing the payload and generating data representing instructions and / or instructions for performing an action. The instructions and / or instructions can be sent to other components of the system to perform the action.

[0048] The user may then speak a second user utterance that includes the requested contact information, in this example a phone number. The microphone 114 of the speech-enabled device 102 may capture audio corresponding to the second user utterance and may generate corresponding audio data. The audio data may be sent to the remote system 120 where the ASR component 152 may generate corresponding text data representing the contact information.

[0049] The text data may be provided to a notification component 168, which may be configured to identify, determine, and / or generate a message to be sent to a personal device 106 associated with the contact information. In an example, the message may be an SMS message. In other examples, the message may be an email, a phone call, a social media message (such as a direct message), or another form of message that the personal device 106 may receive.

[0050] The notification component 168 may generate the message and / or a portion of the message. For example, when selected, an optional portion associated with the message (such as a web link) may help collect account access information, such as a username and / or password associated with a personal user account. The notification component 168 may additionally or alternatively utilize the customer authentication component 162 to generate a code pair that associates the associated device identification with the voice-enabled device 102, the user-provided contact information, and / or the optional portion of the message. The data encryption component 166 may be used to encrypt the device identification, contact information, and / or the message. The notification component 168 of the remote system 120 may be used to send the message to the personal device 106, where the messaging application 136 of the personal device 106 may be used to receive and / or display the message.

[0051] In some examples, when selected, the selectable portion may cause a website application 138 (such as a website access and / or browsing application) of the memory 128 of the personal device 106 to open and display a web page that allows the user to enter account access information. In other examples, when selected, the selectable portion of the message may cause a voice-enabled device application 140 (such as a companion application) associated with the user account to be accessed. The voice-enabled device application 140 may have access to the account access information and may be used to obtain the account access information without requiring the user to enter the information into, for example, the website application 138. Data corresponding to the account access information may be received by the remote system 120 and from the personal device 106, and may be used by the remote system 120 to determine which user account is to be associated with the voice-enabled device 102.

[0052] The notification component 168 may also be configured to send one or more messages associated with the association and / or disassociation of a user account with a voice-enabled device 102. For example, a message may be sent to a personal device 106 that causes and / or allows the website application 138 to display a web page that allows the user of the personal device 106 to enter information associated with their stay in the environment 104. This information may include, for example, a checkout time and / or a checkout day for checking out from the environment 104. This information may be used to determine a time period for which the association between the user account and the voice-enabled device 102 will be maintained and / or when the user account should be disassociated from the voice-enabled device 102. By providing this functionality, the user is able to control when potential personal information and / or functionality associated with a voice account is available to the voice-enabled device 102, and when this information and these functionality should not be available.

[0053] The notification component 168 and / or other components of the remote system 120 may additionally or alternatively be configured to request schedule data from one or more devices and / or systems associated with the environment 104. The schedule data may indicate a time period during which the user remains in the environment, which may be used to identify and / or determine a time period to maintain an association between the user account and the voice-enabled device 102.

[0054] The association management component 170 may be configured to identify and / or determine an association between a user account and a voice-enabled device 102. For example, after receiving additional audio data from a voice-enabled device 102, the association management component 170 may be used to determine that the user account is validly associated with a device identification associated with the voice-enabled device 102. This information may be used to determine how to process the audio data and / or which resources 150 may be used to perform one or more actions associated with the audio data.

[0055] The association management component 170 may additionally or alternatively be configured to store data associated with a time period during which an account will be associated with the voice-enabled device 102. The association management component 170 may additionally or alternatively be configured to store disassociation event data indicating one or more disassociation events, which disassociate the user account from the voice-enabled device 102 when one of the disassociation events occurs. A disassociation event may include receiving an indication that a user has checked out of the environment 104. Additionally or alternatively, the remote system 120 may receive additional audio data representing another user utterance, and the remote system 120 may determine that the intent associated with the user utterance is to disassociate the user account from the voice-enabled device 102. Receiving the audio data and / or determining that the user utterance corresponds to an intent to disassociate may be considered a disassociation event. A disassociation event may also include a temporary disassociation event. For example, the voice-enabled device 102 and / or the remote system 120 may be configured to receive an indication that a lock and / or a key card reader associated with the environment has been used to leave a room. Based at least in part on the indication, the association management component 170 can be configured to temporarily disassociate the user account from the voice-enabled device 102, such as until the lock and / or key card reader indicates that the user has entered the room.

[0056] The association management component 170 may be further configured to determine when one or more events occur that cause a change in the time period for maintaining the association between the user account and the voice-enabled device 102. For example, the remote system 120 may receive an indication that the time period will be extended or shortened. For example, a user may decide to check out of a hotel early and / or the user may decide to extend the stay. In other examples, an indication may be received from a system associated with the environment indicating that the time period has been extended or shortened. Indications of such events may be sent to and received by the remote system 120 and may be used to identify, determine, and / or generate a second time period for maintaining the association between the user account and the voice-enabled device 102.

[0057] The device association component 184 may be configured to identify, determine, generate and / or store device association data indicating an association between the voice-enabled device 102 and one or more other devices within the environment 104. For example, the environment 104 may include one or more "smart" accessory devices configured to communicate with the voice-enabled device 102. Examples of such auxiliary devices may include light bulbs, televisions, plugs, sockets, locks, and / or appliances. The device association component 184 may be configured to store device association data indicating which of these accessory devices are associated with the voice-enabled device 102. In an example, once a user account is associated with the voice-enabled device 102, the user is able to provide user speech, for example, via the voice-enabled device 102 to control the functionality of the accessory device.

[0058] Additionally or alternatively, in an example, a user may visit the same or another environment 104 having the same voice-enabled device 102 and / or another voice-enabled device 102. The repeat visitor may wish to re-establish an association between a user account associated with the user and the voice-enabled device 102. The account may be re-established in one or more of a variety of ways. For example, a user may provide a user utterance captured by a microphone of the voice-enabled device 102. Corresponding audio data may be generated and sent to a remote system 120 for processing. When it is determined that the resources to be used to perform an action associated with the user utterance are not available for use by a default account associated with the voice-enabled device 102, an audio may be output by the voice-enabled device 102 requesting the user to provide contact information to initiate a process of associating the user's personal account with the voice-enabled device 102. The user may provide contact information, such as a telephone number.

[0059] The provided phone number may be cross-referenced with phone numbers that have been previously provided to associate an account with the voice-enabled device 102. In this example, assuming that an account has been previously associated with the voice-enabled device 102 using the contact information, the remote system 120 may identify the account associated with the contact information without sending a message to the user's personal device 106 and / or requiring the user to enter account identification information. In these examples, a message or other notification may be sent to the personal device 106 and / or the voice-enabled device 102 indicating that the account has been associated with the voice-enabled device 102. In doing so, if an account has been mistakenly or maliciously associated with a voice-enabled device 102, the user associated with the account may be notified so that corrective action may be taken.

[0060] Additionally or alternatively, repeat visitors may utilize one or more voice models 156 to associate an account with a voice-enabled device 102. The voice model 156 may include, for example, one or more reference voice signatures. The reference voice signature may indicate characteristics associated with audio data, such as amplitude and / or frequency corresponding to the audio data. Additionally or alternatively, frequency estimation, hidden Markov models, Gaussian mixture models, pattern matching algorithms, neural networks, matrix representations, and / or vector quantization may be utilized to identify, determine, and / or generate a reference voice signature. A user may provide user utterances that may be captured by a microphone of the voice-enabled device 102. Corresponding audio data may be generated and sent to the remote system 120. The user may be identified based at least in part on the voice model 156. Identification of the user may include, for example, determining that a voice signature of the audio data matches or most closely matches one of the reference voice signatures stored and / or accessed by the voice model 156. A user identification associated with the user may be associated with a user account. Having received an identification of a user account and a device identification associated with the voice-enabled device 102 from which the audio data was received, the remote system 120 may then temporarily associate the user account with the voice-enabled device 102. In these examples, a message or other notification may be sent to the personal device 106 and / or the voice-enabled device 102 indicating that the account has been associated with the voice-enabled device 102. In doing so, if an account has been mistakenly or maliciously associated with a voice-enabled device 102, the user associated with the account may be notified so that corrective action may be taken.

[0061] As used herein, a processor such as processor 108, 124 and / or 142 may include multiple processors and / or processors with multiple cores. In addition, the processor may include one or more different types of cores. For example, the processor may include an application processor unit, a graphics processing unit, etc. In a specific implementation, the processor may include a microcontroller and / or a microprocessor. Processor 108, 124 and / or 142 may include a graphics processing unit (GPU), a microprocessor, a digital signal processor, or other processing units or components known in the art. Alternatively or additionally, the functions described herein may be performed at least in part by one or more hardware logic components. For example, but not limited to, the exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), etc. In addition, each of processors 108, 124 and / or 142 may have its own local memory, which may also store program components, program data, and / or one or more operating systems.

[0062] The memory 112, 128, and / or 146 may include volatile and non-volatile memory, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program components, or other data. Such memory 112, 128, and / or 146 includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, cassettes, magnetic tape, magnetic disk storage or other magnetic storage, RAID storage systems, or any other medium that can be used to store the desired information and can be accessed by the computing device. The memory 112, 128, and / or 146 may be implemented as a computer-readable storage medium ("CRSM"), which may be any available physical medium that the processor 108, 124, and / or 142 can access to execute the instructions stored on the memory 112, 128, and / or 146. In a basic implementation, the CRSM may include random access memory ("RAM") and flash memory. In other implementations, the CRSM may include, but is not limited to, read-only memory ("ROM"), electrically erasable programmable read-only memory ("EEPROM"), or any other tangible medium that can be used to store the desired information and that can be accessed by a processor.

[0063] In addition, the functional components may be stored in the corresponding memory, or the same functions may be implemented in hardware, firmware, application specific integrated circuits, field programmable gate arrays, or as a system on chip (SoC) instead. In addition, although not shown, each corresponding memory discussed herein (such as memory 112, 128, and / or 146) may include at least one operating system (OS) component configured to manage hardware resource devices such as network interfaces, I / O devices of the corresponding devices, etc., and provide various services to applications or components executed on the processor. Such OS components may implement a variant of the FreeBSD operating system such as that released by the FreeBSD project; other UNIX or UNIX-like variants; a variant of the Linux operating system such as that released by Linus Torvalds; the FireOS operating system of Amazon.com Inc. in Seattle, Washington, USA; the Windows operating system of Microsoft Corporation in Redmond, Washington, USA; such as LynxOS released by Lynx Software Technologies, Inc. in San Jose, California; such as an embedded operating system (Enea OSE) released by ENEA AB in Sweden; and the like.

[0064] The network interfaces 110, 126, and / or 144 may enable messages between components and / or devices shown in the system 100 and / or with one or more other remote systems and other networked devices. Such network interfaces 110, 126, and / or 144 may include one or more network interface controllers (NICs) or other types of transceiver devices to send and receive messages over the network 122.

[0065] For example, each of the network interfaces 110, 126, and / or 144 may include a personal area network (PAN) component to enable the transmission of messages via one or more short-range wireless messaging channels. For example, the PAN component may enable messages that conform to at least one of the following standards IEEE 802.15.4 (ZigBee), IEEE 802.15.1 (Bluetooth), IEEE 802.11 (WiFi), or any other PAN messaging protocol. In addition, each of the network interfaces 110, 126, and / or 144 may include a wide area network (WAN) component to enable the transmission of messages via a wide area network.

[0066] In some cases, the remote system 120 may be local to an environment associated with the voice-enabled device 102. For example, the remote system 120 may be located within the voice-enabled device 102. In some cases, some or all of the functionality of the remote system 120 may be performed by the voice-enabled device 102.

[0067] Figures 2 to 5 Various processes for establishing a temporary account association with a voice-enabled device are shown. The processes described herein are shown as a collection of boxes in a logical flow diagram that represents a series of operations, some or all of which may be implemented in hardware, software, or a combination thereof. In the context of software, a box may represent computer-executable instructions stored on one or more computer-readable media that, when executed by one or more processors, program the processors to perform the described operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform specific functions or implement specific data types. Unless otherwise specified, the order in which the boxes are described should not be construed as limiting. Any number of the described boxes may be combined in any order and / or in parallel to implement the process or an alternative process, and not all boxes need to be executed. For purposes of discussion, the processes are described with reference to the environment, architecture, and system described in the examples herein, such as, for example, with reference to Figure 1A , Figure 1B , Figure 6 , Fig.12 and Fig.13 Those described, although these processes can be implemented in a variety of other environments, architectures, and systems.

[0068] Figure 2A flow chart of an example process 200 for establishing a temporary account association with a voice-enabled device is shown. The order in which the operations or steps are described is not intended to be construed as a limitation, and any number of the described operations may be combined in any order and / or in parallel to implement process 200. Figure 2 As shown, examples have been provided of which device and / or system can perform each of the operations depicted. However, it should be understood that one or more of these operations can be performed by Figure 2 The devices and / or systems specifically shown may be implemented by different devices and / or systems. Figure 2 Voice-enabled devices can be used with Figure 1B The voice-enabled device 102 described in is the same or similar. Figure 2 The remote system can be connected with Figure 1B The remote system 120 described in . Figure 2 of personal devices can be connected to Figure 1B The personal device 106 described in .

[0069] At block 202, process 200 may include generating audio data representing a user utterance. For example, a user may speak a user utterance to a speech-enabled device. A microphone of the speech-enabled device may capture the user utterance and may generate corresponding audio data. The audio data may be sent to a remote system via a network.

[0070] At box 204, process 200 may include determining that a resource is not available for use by a default user account associated with a voice-enabled device. For example, an ASR component of a remote system may process the audio data and generate corresponding text data. An NLU component of a remote system may utilize the text data to determine intent data representing the user's intent (in these examples, the intent is to perform an action that utilizes one or more system resources) and / or the intent to temporarily associate a user account with a voice-enabled device. For example, a user utterance may be "play song A." Using the NLU component, the remote system may determine that a resource is available for performing an action that outputs audio corresponding to song A. For example, a voice-enabled device may be associated with a default user account associated with an environment. The default user account may not have access to a resource to output song A. For example, the default user account may not be associated with a music service and / or database that allows song A to be played. In these examples, a voice widget of a remote system may be utilized to initiate a process of temporarily associating a personal user account associated with a user with the voice-enabled device.

[0071] At box 206, process 200 may include generating an instruction and audio data requesting contact information. The instruction and the audio data may be sent to a voice-enabled device. For example, the voice widget of the remote system described above may receive intent data from the NLU component, and may utilize the intent data to identify, determine, and / or generate instructions corresponding to the intent data. For example, the voice widget may be called and used to generate the instruction, which, when sent to the voice-enabled device and enabled by the voice-enabled device, causes audio to be output, such as via a speaker, representing a request for information to temporarily associate a user account with the voice-enabled device. The TTS component of the remote system may be used to generate audio data that will be used by the voice-enabled device to output audio as indicated by the instruction generated by the voice widget.

[0072] At box 208, process 200 may include outputting audio corresponding to the audio data based at least in part on the instruction. For example, the output audio may be "We can't perform this operation, please provide your phone number to link your account to this device." The example audio provided above is provided by way of example only, not by way of limitation. It should be understood that although the example of contact information is provided as a phone number, the contact information may additionally or alternatively include, for example, an email address and / or a social media account identifier. In addition, although the request for contact information has been described as being provided in an audio manner through a speaker of a voice-enabled device, it should be understood that the audio may be provided by a speaker other than a speaker of a voice-enabled device, and / or the request may be presented in a manner other than hearing or in a manner that supplements hearing. For example, in the case where the voice-enabled device includes a display and / or transmits a message to a device having a display, the request may be presented in a visual manner.

[0073] At block 210, process 200 may include generating audio data representing contact information. For example, a user may provide a second user utterance including contact information. A microphone of a speech-enabled device may capture audio corresponding to the user utterance and may generate corresponding audio data. The audio data may be sent to a remote system for processing. For example, an ASR component of the remote system may generate text data corresponding to the audio data and representing the contact information.

[0074] At block 212, process 200 may include generating a code pair that associates the contact information with a device identification that is associated with the device to which the contact information corresponds. The generation of the code pair may be performed by a code pair component of the remote system. The code pair component may generate a code pair that associates the device identification associated with the voice-enabled device, the contact information provided by the user, and / or an optional portion of the message described in more detail with reference to block 214. The code pair component may be used to encrypt the device identification, the contact information, and / or the message.

[0075] At box 214, process 200 may include generating a message associated with a link or other optional portion for account credential input. The message may be sent to a personal device associated with the contact information, such as by utilizing a code pair. For example, text data from the ASR component may be provided to a notification component of a remote system, which may be configured to identify, determine, and / or generate a message to be sent to a personal device associated with the contact information. In an example, the message may be an SMS message. In other examples, the message may be an email, a phone call, a social media message (such as a private message), or another form of message that a personal device may receive.

[0076] The notification component may generate the message and / or a portion of the message. For example, an optional portion such as a web link may be generated in association with the message that, when selected, may facilitate collecting account access information, such as a username and / or password associated with a personal user account. The notification component of the remote system may be used to send the message to a personal device, where a messaging application of the personal device may be used to receive and / or display the message.

[0077] At box 216, process 200 may include receiving input selecting the link and / or optional portion. In some examples, when selected, the optional portion may cause a website application of a personal device, such as a website access and / or browsing application, to open and display a web page that allows the user to enter account access information. In other examples, when selected, the optional portion of the message may cause a voice-enabled device application (such as a companion application) associated with the user account to be accessed. The voice-enabled device application may have access to the account access information and may be used to obtain the account access information without the user having to enter the information into, for example, a website application. Input data corresponding to the input may be generated, and the input data may be used to open a web application that may display a web page that allows entry of account credentials associated with the user account at box 218. Input data corresponding to the account credentials may be generated, and the input data may be sent to a remote system.

[0078] At block 220, process 200 may include associating a user account corresponding to the account credentials with the voice-enabled device. For example, upon receiving additional audio data from the voice-enabled device, an association management component of the remote system may be utilized to determine that the user account is effectively associated with a device identification associated with the voice-enabled device. This information may be used to determine how to process the audio data and / or which resources may be used to perform one or more actions associated with the audio data.

[0079] At box 222, process 200 may include outputting audio confirming that the account is associated with the voice-enabled device. For example, the remote system may generate instruction data and / or audio data based at least in part on the association of the user account with the voice-enabled device. The instruction data and / or audio data may be sent to the voice-enabled device, and audio corresponding to the audio data may be output via a speaker of the voice-enabled device. By doing so, a confirmation may be provided to the user indicating that his / her user account has been successfully associated with the voice-enabled device.

[0080] At box 224, based at least in part on the user account being associated with the voice-enabled device, process 200 may include causing a web page to be displayed on a personal device. The web page may allow input of an indication of when the user account should be disassociated from the device identification associated with the voice-enabled device. Input data corresponding to the input may be generated, and the input data may be sent to a remote system. For example, a message may be sent to a personal device that causes and / or allows a website application to display a web page that allows the user of the personal device to input an indication associated with their stay in the environment. The indication may include, for example, a checkout time and / or a checkout day for checking out from the environment. This information may be used to determine a time period for maintaining the association between the user account and the voice-enabled device, and when the user account should be disassociated from the voice-enabled device. By providing this functionality, a user can control when potential personal information and / or functionality is available for use by the voice-enabled device 102, and when the information and functionality should be unavailable.

[0081] The notification component and / or other components of the remote system may be configured to request scheduling data from one or more devices and / or systems associated with the environment. The scheduling data may indicate a time period during which the user remains in the environment, which may be used to identify and / or determine a time period to maintain an association between the user account and the voice-enabled device.

[0082] At box 226, process 200 may include receiving disassociation time data. For example, the association management component of the remote system may be configured to store data associated with the time period during which the account will be associated with the voice-enabled device. The association management component may additionally or alternatively be configured to store disassociation event data indicating one or more disassociation events, which disassociate the user account from the voice-enabled device when one of the disassociation events occurs. The disassociation event may include receiving an indication that the user has checked out from the environment. Additionally or alternatively, the remote system may receive additional audio data representing another user's utterance, and the remote system may determine that the intention associated with the user's utterance is to disassociate the user account from the voice-enabled device. Receiving the audio data and / or determining that the user's utterance corresponds to an intention to disassociate may be considered a disassociation event. The disassociation event may also include a temporary disassociation event. For example, the voice-enabled device and / or the remote system may be configured to receive an indication that a lock and / or a key card reader associated with the environment has been used to leave the room. Based at least in part on the indication, the association management component can be configured to temporarily disassociate the user account from the voice-enabled device, such as until the lock and / or key card reader indicates that the user has entered the room.

[0083] The association management component may be further configured to determine when one or more events that cause a change in the time period for maintaining the association between the user account and the voice-enabled device occur. For example, the remote system may receive an indication indicating whether the time period will be extended or shortened. For example, a user may decide to check out of a hotel early and / or the user may decide to extend the stay. In other examples, an indication indicating whether the time period has been extended or shortened may be received from a system associated with the environment. Indications of such events may be sent to and received by the remote system and may be used to identify, determine, and / or generate a second time period for maintaining the association between the user account and the voice-enabled device.

[0084] At block 228, process 200 may include disassociating the user account from the voice-enabled device based at least in part on the disassociation time data. Disassociating the user account from the voice-enabled device may be based at least in part on at least one of the time period passing and / or the occurrence of a disassociation event.

[0085] Figure 3 A flow chart of an example process 300 for utilizing a voice-enabled device that has been temporarily associated with an account is shown. The order in which the operations or steps are described is not intended to be construed as a limitation, and any number of the described operations may be combined in any order and / or in parallel to implement the process 300. Figure 3As shown, examples have been provided of which device and / or system can perform each of the operations depicted. However, it should be understood that one or more of these operations can be performed by Figure 3 The devices and / or systems specifically shown may be implemented by different devices and / or systems. Figure 3 Voice-enabled devices can be used with Figure 1B The voice-enabled device 102 described in is the same or similar. Figure 3 The remote system can be connected with Figure 1B The remote system 120 described in . Figure 3 of personal devices can be connected to Figure 1B The personal device 106 described in .

[0086] At block 302, process 300 may include generating audio data representing a user utterance. For example, a user may speak a user utterance to a speech-enabled device. A microphone of the speech-enabled device may capture the user utterance and may generate corresponding audio data. The audio data may be sent to a remote system via a network.

[0087] At block 304, process 300 may include generating intent data representing the intent associated with the user utterance. Fig.12 The ASR technology described in more detail below processes the audio data to generate text data representing the user's speech. Fig.12 The NLU technology described in more detail can utilize this text data to generate intent data. For example, the user utterance can be "Play Song A". The ASR component of the remote system can generate text data representing the text "Play Song A". The NLU component of the remote system can generate intent data representing the "Play" intent and an associated value of "Song A" that identifies the content to be played. The intent data can be sent along with the device identification of the voice-enabled device to a voice widget configured to process the intent determined by the NLU component. In the example where the voice-enabled device is temporarily associated with a user account, an association management component that stores data indicating the temporary association between the device identification and the user account can be used to determine that resources associated with the user account are available to serve the intent.

[0088] At block 306, process 300 may include identifying active account data based at least in part on an association between a user account and a voice-enabled device. For example, an association management component may be queried and / or utilized to determine whether an active account associated with a device identification of a voice-enabled device is a default account or a temporarily associated user account. In examples where the device identification is already associated with a user account, valid account data from the association management component may be used to determine that an available resource is a resource associated with the temporarily associated user account.

[0089] At block 308, process 300 may include accessing resources associated with the user account and the intent. Using the example provided above, the user account associated with the voice-enabled device may be associated with access and / or permission to use one or more resources, such as music services and / or functions, that provide the ability to play "Song A." The one or more resources may be accessed and may be used to perform the requested action.

[0090] At block 310, process 300 may include generating instructions to perform an operation using the resource. For example, the voice widget may utilize the resource to generate instructions for an action to be performed. Using the examples provided herein, the instructions may be to cause the voice-enabled device to output audio corresponding to song A.

[0091] At block 312, process 300 may include a voice-enabled device receiving the instruction. Audio data corresponding to the instruction may also be received.

[0092] At box 314, process 300 may include performing the action and / or outputting audio confirming that the action has been performed. For example, certain actions will be performed by a voice-enabled device. These actions can be, for example, outputting audio corresponding to a song, setting a timer, displaying a video on a display of a voice-enabled device, etc. In these examples, the instruction data can cause the voice-enabled device to perform the action. In other examples, the action will be performed by a device other than the voice-enabled device. These actions can be, for example, ordering a product, operating an appliance, operating a light bulb, etc. In these examples, the instruction data can cause one or more other devices to perform the action, and the voice-enabled device can be used to output audio confirming that the operation has been performed. It should be understood that the execution of the action may include the implicit execution of the action by the voice-enabled device and / or the remote system.

[0093] At block 316, process 300 may include the personal device receiving a notification of the action. The notification may be used to confirm the performance of the action and to confirm the purpose of associating the user account with the voice-enabled device. If the user account should not be associated with the voice-enabled device, the notification may be used to provide an opportunity to disassociate the user account from the voice-enabled device using the personal device. The sending of the notification may be optional and may be determined by one or more user preferences associated with the user account.

[0094] Figure 4 A flow chart of an example process 400 for temporarily reassociating an account with a voice-enabled device using contact information is shown. The order in which the operations or steps are described is not intended to be construed as a limitation, and any number of the described operations may be combined in any order and / or in parallel to implement the process 400. Figure 4 As shown, examples have been provided of which device and / or system can perform each of the operations depicted. However, it should be understood that one or more of these operations can be performed by Figure 4 The devices and / or systems specifically shown may be implemented by different devices and / or systems. Figure 4 Voice-enabled devices can be used with Figure 1B The voice-enabled device 102 described in is the same or similar. Figure 4 The remote system can be connected with Figure 1B The remote system 120 described in . Figure 4 of personal devices can be connected to Figure 1B The personal device 106 described in .

[0095] At block 402, process 400 may include generating audio data representing a user utterance. For example, a user may speak a user utterance to a speech-enabled device. A microphone of the speech-enabled device may capture the user utterance and may generate corresponding audio data. The audio data may be sent to a remote system via a network.

[0096] At box 404, process 400 may include determining that a resource is not available for use by a default user account associated with a voice-enabled device. For example, an ASR component of a remote system may process the audio data and generate corresponding text data. An NLU component of a remote system may utilize the text data to determine intent data representing the user's intent (in these examples, the intent is to perform an action that utilizes one or more system resources) and / or the intent to temporarily associate a user account with a voice-enabled device. For example, a user utterance may be "play song A." Using the NLU component, the remote system may determine that a resource is available for performing an action that outputs audio corresponding to song A. For example, a voice-enabled device may be associated with a default user account associated with an environment. The default user account may not have access to resources to output song A. For example, the default user account may not be associated with a music service and / or database that allows song A to be played. In these examples, a voice widget of a remote system may be utilized to initiate a process of temporarily associating a personal user account associated with a user with the voice-enabled device.

[0097] At box 406, process 400 may include generating audio data of instructions and requesting contact information. The instructions and the audio data may be sent to a voice-enabled device. For example, the voice widget of the remote system described above may receive intent data from the NLU component, and may utilize the intent data to identify, determine, and / or generate instructions corresponding to the intent data. For example, the voice widget may be called and used to generate instructions that, when sent to a voice-enabled device and enabled by the voice-enabled device, result in outputting audio, such as via a speaker, that represents a request for information to temporarily associate a user account with the voice-enabled device. The TTS component of the remote system may be used to generate audio data that will be used by the voice-enabled device to output audio as indicated by the instructions generated by the voice widget.

[0098] At box 408, process 400 may include outputting audio corresponding to the audio data based at least in part on the instruction. For example, the output audio may be "We can't perform this operation, please provide your phone number to link your account to this device." The example audio provided above is provided by way of example only, not by way of limitation. It should be understood that although the example of contact information is provided as a phone number, the contact information may additionally or alternatively include, for example, an email address and / or a social media account identifier. In addition, although the request for contact information has been described as being provided in an audio manner through a speaker of a voice-enabled device, it should be understood that the audio may be provided by a speaker other than the speaker of the voice-enabled device, and / or the request may be presented in a manner other than hearing or in a manner that supplements hearing. For example, in the case where the voice-enabled device includes a display and / or transmits a message to a device having a display, the request may be presented in a visual manner.

[0099] At block 410, process 400 may include generating audio data representing contact information. For example, a user may provide a second user utterance including contact information. A microphone of a speech-enabled device may capture audio corresponding to the user utterance and may generate corresponding audio data. The audio data may be sent to a remote system for processing. For example, an ASR component of the remote system may generate text data corresponding to the audio data and representing the contact information.

[0100] At box 412, process 400 may include determining a user account associated with the contact information. For example, in the case where the user is a repeat user or has previously undergone a process of temporarily associating a user account with a voice-enabled device, process 400 does not need to include obtaining account credential information from the user. Instead, data associating a user account with contact information may already be stored in the association management component of the remote system. Thus, when a repeat user provides his / her contact information, the remote system may query the association management component and identify the user account to be associated with the voice-enabled device without requesting such information from the user or personal device. It should be noted that process 400 may be used in the case where the user visits the same environment or a different environment and / or if the user stays in the same room or a different room of the environment. In other words, process 400 may be used to temporarily associate a user account with any voice-enabled device associated with and / or available to the remote system.

[0101] At block 414, process 400 may include generating a message requesting confirmation of intent to associate the user account with the voice-enabled device. This operation may be optional and may be provided as a means of ensuring that the user is authorized to associate the user account with the voice-enabled device. The message may be sent to the personal device.

[0102] At block 416, process 400 may include receiving input at the personal device confirming an intent to associate the user account with the voice-enabled device. For example, the message may request the user to provide input confirming that the user associated with the personal device and contact information for communicating with the personal device wishes to temporarily re-associate the user account with the voice-enabled device.

[0103] At box 418, process 400 may include associating the user account with the voice-enabled device based at least in part on receiving input data from the personal device indicating that the user has confirmed the intention to associate the user account with the voice-enabled device. At box 420, process 400 may include outputting audio confirming the association between the user account and the voice-enabled device. Outputting the audio may be based at least in part on instruction data received from the remote system, which instructs the voice-enabled device to output audio confirming the association. Thereafter, the user may provide subsequent user utterances representing commands to cause the voice-enabled device and / or one or more other devices and / or systems to perform an action.

[0104] Figure 5 A flow chart of an example process 500 for temporarily reassociating an account with a voice-enabled device using a voice model is shown. The order in which the operations or steps are described is not intended to be construed as a limitation, and any number of the described operations may be combined in any order and / or in parallel to implement the process 500. Figure 5 As shown, examples have been provided of which device and / or system can perform each of the operations depicted. However, it should be understood that one or more of these operations can be performed by Figure 5 The devices and / or systems specifically shown may be implemented by different devices and / or systems. Figure 5 Voice-enabled devices can be used with Figure 1B The voice-enabled device 102 described in is the same or similar. Figure 5 The remote system can be connected with Figure 1B The remote system 120 described in . Figure 5 of personal devices can be connected to Figure 1B The personal device 106 described in .

[0105] At block 502, process 500 may include generating audio data representing a user utterance. For example, a user may speak a user utterance to a speech-enabled device. A microphone of the speech-enabled device may capture the user utterance and may generate corresponding audio data. The audio data may be sent to a remote system via a network.

[0106] At block 504, process 500 may include determining a user account associated with a user based at least in part on one or more speech models. For example, identification of the user may include determining that a speech signature of the audio data matches or most closely matches a reference speech signature stored and / or accessed by the speech model. A user identification associated with the user may be associated with the user account.

[0107] At block 506, process 500 may include generating instructions and audio data requesting confirmation of intent from the user of the voice-enabled device. For example, the audio data may include an indicator of a user account that the remote system has determined to be associated with the user.

[0108] At box 508, process 500 may include outputting audio requesting confirmation of the intent based at least in part on the instruction. The output audio may include an indicator of the user account that the remote system determines to be associated with the user. Additionally or alternatively, the output audio may include a request to the user to provide an indication of the intent to confirm that the user account will be associated with the voice-enabled device. For example, the output audio may include "Do you want to link Rob's account to this device?" The user may then provide another user utterance to the voice-enabled device in response to the confirmation request.

[0109] At block 510, process 500 may include generating audio data representing a response to the intent confirmation request. The audio data may be sent to a remote system for processing. At block 512, process 500 may include analyzing the audio data to determine an intent to associate a user account with a voice-enabled device. The analysis may include using a voice model to confirm that the audio data corresponding to the response also matches or best matches the user identification.

[0110] At block 514, process 500 may include temporarily associating a user account with a voice-enabled device. In these examples, a message or other notification may be sent to the personal device at block 518 and / or to the voice-enabled device at block 516 indicating that the account has been associated with the voice-enabled device. By doing so, if an account is mistakenly or maliciously associated with a voice-enabled device, the user associated with the account may be notified so that corrective action can be taken.

[0111] Figure 6 An example environment 600 for temporary account association with one or more voice-enabled devices is shown. The environment 600 may include Figure 1B The environment 600 may include the same or similar devices and / or systems as the environment 100 of FIG. 104. For example, the environment 600 may include the environment 104, in which the voice-enabled device 102 and the personal device 106 associated with the user may be located. The environment 600 may additionally include one or more accessory devices 602.

[0112] The device association component of the remote system may be configured to identify, determine, generate and / or store device association data indicating an association between a voice-enabled device 102 and one or more other accessory devices 602 within the environment 104. For example, the environment 104 may include one or more "smart" accessory devices 602 that are configured to communicate with the voice-enabled device 102. Examples of such accessory devices 602 may include light bulbs, televisions, plugs, sockets, locks, and / or appliances. The device association component may be configured to store device association data indicating which of these accessory devices are associated with the voice-enabled device 102. In an example, once a user account is associated with the voice-enabled device 102, the user may be able to provide user speech, such as via the voice-enabled device 102, to control the functionality of the accessory device 602.

[0113] For example, a user may say "turn on the lights", "turn on the smart plug", "turn down the TV volume", etc. The microphone of the voice-enabled device 102 may capture such user utterances and may generate corresponding audio data. The audio data may be sent to the remote system for processing. In addition, the association management component of the remote system may be used to determine the active account currently associated with the voice-enabled device, and may utilize resources associated with the active account to service the request represented by the user utterance.

[0114] In an example, once a user account has been temporarily associated with a device identifier of a voice-enabled device, the user account may have access to a first set of resources, while a default account may have access to a second set of resources. Some of the resources in the first set of resources may be the same as the resources in the second set of resources, and / or some of the resources in the first set of resources may be different from the resources in the second set of resources. In these examples, account arbitration may be performed by a remote system to determine which user account to use to process and respond to user utterances. In some examples, account arbitration may include giving priority to a temporarily associated user account over a default account, unless the temporarily associated user account does not have access to resources for processing requests (such as ordering room service). In other examples, account arbitration may include giving priority to a default account over a temporarily associated user account, unless the default account does not have access to resources for processing requests.

[0115] Additionally or alternatively, some user accounts may be associated with their own voice-enabled devices. For example, some users may have voice-enabled devices associated with a remote system at home and / or in the office. The user account may be associated with a personal voice-enabled device. In these examples, a user request such as "turn off the lights" may indicate an intention to turn off the lights in the environment or to turn off the lights at home. The remote system may perform action arbitration to prioritize one or more actions over other actions. For example, when a user account is temporarily associated with a voice-enabled device in an environment, the remote system may prioritize actions associated with the environment. In other examples, the remote system may prioritize actions associated with home and / or office devices. In other examples, when a request corresponds to an action that can be performed by one or more devices, the remote system may request additional information from the user from the voice-enabled device from which the audio data was received before taking action, such as "Do you want me to turn off the lights in this room or home?"

[0116] Figures 7 to 11 Various processes for establishing a temporary account association with a voice-enabled device are shown. The processes described herein are shown as a collection of boxes in a logical flow diagram that represents a series of operations, some or all of which may be implemented in hardware, software, or a combination thereof. In the context of software, a box may represent computer-executable instructions stored on one or more computer-readable media that, when executed by one or more processors, program the processors to perform the described operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform specific functions or implement specific data types. Unless otherwise specified, the order in which the boxes are described should not be construed as limiting. Any number of the described boxes may be combined in any order and / or in parallel to implement the process or an alternative process, and not all boxes need to be executed. For purposes of discussion, the processes are described with reference to the environment, architecture, and system described in the examples herein, such as, for example, with reference to Figures 1A to 6 , Fig.12 and Fig.13 Those described, although these processes can be implemented in a variety of other environments, architectures, and systems.

[0117] Figure 7 A flow chart of an example process 700 for temporary account association with a voice-enabled device is shown. The order in which the operations or steps are described is not intended to be construed as a limitation, and any number of the described operations may be combined in any order and / or in parallel to implement process 700.

[0118] At block 702, process 700 may include receiving first audio data representing a first user utterance from a voice-enabled device, the first user utterance requesting performance of an action, the voice-enabled device being associated with a first account. For example, a user may speak a user utterance to the voice-enabled device. A microphone of the voice-enabled device may capture the user utterance and may generate corresponding audio data. The audio data may be sent to a remote system via a network, and the remote system may receive the audio data.

[0119] At box 704, process 700 may include determining that the resource for performing the action is not associated with the first account. For example, an ASR component of the remote system may process the audio data and generate corresponding text data. The NLU component of the remote system may use the text data to determine intent data representing the user's intent (in these examples, the intent is to perform an action that utilizes one or more system resources) and / or the intent to temporarily associate the user account with the voice-enabled device. For example, the user utterance may be "play song A". Using the NLU component, the remote system may determine that a resource can be used to perform an action of outputting audio corresponding to song A. For example, the voice-enabled device may be associated with a default user account associated with the environment. The default user account may not have access to resources to output song A. For example, the default user account may not be associated with a music service and / or database that allows song A to be played. In these examples, a voice widget of the remote system may be used to initiate a process of temporarily associating a personal user account associated with the user with the voice-enabled device.

[0120] At box 706, process 700 may include sending an instruction to the voice-enabled device to output audio via a speaker of the voice-enabled device, the audio representing a request for a phone number. For example, the remote system may generate instruction data and audio data requesting contact information. The instruction data and the audio data may be sent to the voice-enabled device. For example, the voice widget of the above-mentioned remote system may receive intent data from the NLU component, and the intent data may be used to identify, determine and / or generate instruction data corresponding to the intent data. For example, the voice widget may be called and used to generate instruction data, which, when sent to the voice-enabled device and enabled by the voice-enabled device, causes the instruction data to output audio such as via a speaker, the audio representing a request for information to temporarily associate a user account with the voice-enabled device. The TTS component of the remote system may be used to generate audio data that will be used by the voice-enabled device to output the audio as indicated by the instruction data generated by the voice widget.

[0121] The audio corresponding to the audio data may be output based at least in part on the instruction data. For example, the output audio may be "We cannot perform this operation, please provide your phone number to link your account to this device." The example audio provided above is provided by way of example only and not as a limitation. It should be understood that although the example of contact information is provided as a phone number, the contact information may additionally or alternatively include, for example, an email address and / or a social media account identifier. In addition, although the request for contact information has been described as being provided in an audio manner through a speaker of a voice-enabled device, it should be understood that the audio may be provided by a speaker other than a speaker of a voice-enabled device, and / or the request may be presented in a manner other than hearing or in a manner that supplements hearing. For example, in the case where the voice-enabled device includes a display and / or transmits a message to a device having a display, the request may be presented in a visual manner.

[0122] At block 708, process 700 may include receiving second audio data from a speech-enabled device, the second audio data representing a second user utterance including a telephone number. For example, a user may provide a second user utterance including contact information, here including a telephone number. A microphone of the speech-enabled device may capture audio corresponding to the user utterance, and may generate corresponding audio data. The audio data may be sent to a remote system for processing. For example, an ASR component of the remote system may generate text data corresponding to the audio data and representing the contact information.

[0123] At block 710, process 700 may include receiving a device identification of the voice-enabled device from the voice-enabled device. For example, data representing the device identification may be sent from the voice-enabled device to the remote system along with the second audio data. In other examples, the device identification may be determined by the remote system based at least in part on information associated with the voice-enabled device.

[0124] At block 712, process 700 may include generating a message associated with a second request to provide authentication information associated with a second account. For example, a code pair may be generated that associates contact information with a device identifier associated with a device corresponding to the contact information. The generation of the code pair may be performed by a code pair component of a remote system. The code pair component may generate a code pair that associates a device identifier associated with a voice-enabled device, contact information provided by a user, and / or an optional portion of a message described in more detail below. The code pair component may be used to encrypt the device identifier, the contact information, and / or the message.

[0125] The message may be generated and may include a link or other optional portion for account credential entry. The notification component may utilize an authentication link component of the remote system to generate the message and / or a portion of the message. For example, the authentication link component may be used to identify, determine, and / or generate an optional portion of the message (such as a web page link) that, when selected, may facilitate collection of account access information, such as a username and / or password associated with an individual user account.

[0126] At box 714, process 700 may include sending the message to a personal device associated with the phone number. A notification component of the remote system may be used to send the message to the personal device, where a message application of the personal device may be used to receive and / or display the message. The message may be sent to the personal device associated with the contact information, such as by utilizing a code pair. For example, text data from the ASR component may be provided to a notification component of the remote system, which may be configured to identify, determine, and / or generate a message to be sent to the personal device associated with the contact information. In an example, the message may be an SMS message. In other examples, the message may be an email, a phone call, a social media message (such as a private message), or another form of message that a personal device may receive.

[0127] At block 716, process 700 may include receiving authentication data from the personal device that identifies the second account and authenticates access to the second account. For example, input data corresponding to input provided by a user and / or input recognized by an application accessed from the personal device may represent account credentials and may be generated and sent to a remote system.

[0128] At block 718, process 700 may include associating an account identifier corresponding to the second account with a device identifier of the voice-enabled device. For example, after receiving additional audio data from the voice-enabled device, an association management component of the remote system may be utilized to determine that the account identifier is effectively associated with the device identifier associated with the voice-enabled device. This information may be used to determine how to process the audio data and / or which resources may be used to perform one or more actions associated with the audio data.

[0129] Process 700 may additionally or alternatively include sending a second message to the personal device requesting an indication of when to disassociate the user account from the device identification. Process 700 may also include receiving the indication from the personal device and determining a time period from the indication. Process 700 may also include disassociating the account identification from the device identification after the time period has elapsed.

[0130] Process 700 may additionally or alternatively include receiving additional audio data from a voice-enabled device, the additional audio data representing another user utterance for performance of the action. Process 700 may also include: when the audio data is received during a predetermined time period, associating the audio data with a user account; and determining that a resource for performing the action is associated with the user account. Process 700 may also include generating a second instruction to perform the action using the resource, and sending the second instruction to the voice-enabled device.

[0131] Process 700 may additionally or alternatively include: receiving additional audio data representing another user utterance performing the action from a second voice-enabled device associated with a second environment. The second environment may be associated with another default account. Process 700 may also include: determining that there are no resources for performing the action in the other default account; and sending an instruction to output audio to the second voice-enabled device, which requests contact information from the user. Process 700 may also include receiving audio data representing another user utterance including contact information from the second voice-enabled device. Process 700 may also include: identifying a user account based at least in part on the contact information; and receiving a second device identifier associated with the second voice-enabled device. Process 700 may also include associating the account identifier with the second device identifier. In this way, repeat visitors to an environment with a voice-enabled device associated with a remote system can temporarily associate a user account with a voice-enabled device without using a personal device.

[0132] Figure 8 A flow chart of another example process 800 for temporary account association with a voice-enabled device is shown. The order in which the operations or steps are described is not intended to be construed as a limitation, and any number of the described operations may be combined in any order and / or in parallel to implement process 800.

[0133] At block 802, process 800 may include receiving audio data representing a user utterance including contact information from a speech-enabled device. For example, a user may provide a user utterance including contact information. A microphone of the speech-enabled device may capture audio corresponding to the user utterance and may generate corresponding audio data. The audio data may be sent to a remote system for processing. For example, an ASR component of the remote system may generate text data corresponding to the audio data and representing the contact information.

[0134] At block 804, process 800 may include receiving a device identification of the voice-enabled device from the voice-enabled device. For example, data representing the device identification may be sent from the voice-enabled device to a remote system along with the audio data. In other examples, the device identification may be determined by the remote system based at least in part on information associated with the voice-enabled device.

[0135] At block 806, process 800 may include generating an encrypted device identification. For example, a request to "seal" or otherwise encrypt a device identification associated with a voice-enabled device may be sent to a data encryption component of a remote system, which may seal or otherwise encrypt the device identification. Data representing the encrypted device identification may be received by a managed attribute voice widget for further processing.

[0136] At block 808, process 800 may include generating a code pair associated with the optional link and including the encrypted device identification. For example, the managed attribute voice widget may request the code pair from the client authentication component of the remote system. The code pair may include and / or represent a URL for inputting information and / or disassociating the sealed device identification. The managed attribute voice widget may receive the code pair and may send the code pair, the device identification, and the contact information to the notification component.

[0137] At block 810, process 800 may include generating a message associated with a request to provide authentication information associated with a user account via an optional link. For example, the code pair may associate contact information with a device identifier associated with a device to which the contact information corresponds.

[0138] The message may be generated and may be associated with a link or other optional portion for account credential entry. The notification component may utilize an authentication link component of the remote system to generate the message and / or a portion of the message. For example, the authentication link component may be used to identify, determine, and / or generate an optional portion of the message (such as a web link) that, when selected, may facilitate collection of account access information, such as a username and / or password associated with an individual user account.

[0139] At box 812, process 800 may include sending the message to a personal device associated with the contact information. For example, a notification component of the remote system may be used to send the message to the personal device, where a message application of the personal device may be used to receive and / or display the message. The message may be sent to the personal device associated with the contact information, such as by utilizing a code. For example, text data from the ASR component may be provided to a notification component of the remote system, which may be configured to identify, determine, and / or generate a message to be sent to the personal device associated with the contact information. In an example, the message may be an SMS message. In other examples, the message may be an email, a phone call, a social media message (such as a private message), or another form of message that a personal device may receive.

[0140] At block 814, process 800 may include receiving authentication data from the personal device that authenticates access to the user account via the optional link. For example, input data corresponding to input provided by the user and / or input recognized by an application accessed from the personal device may represent account credentials and may be generated and sent to the remote system.

[0141] At block 816, process 800 may include associating an account identifier of the user account with a device identifier of the voice-enabled device based at least in part on receiving the authentication data. For example, after receiving the additional audio data from the voice-enabled device, an association management component of the remote system may be utilized to determine that the user account is effectively associated with the device identifier associated with the voice-enabled device. This information may be used to determine how to process the audio data and / or which resources may be used to perform one or more actions associated with the audio data.

[0142] Process 800 may additionally or alternatively include sending a second message to the personal device requesting an indication of when to disassociate the user account from the device identification. Process 800 may also include receiving the indication from the personal device and determining a time period from the indication. Process 800 may also include disassociating the account identification from the device identification after the time period has elapsed.

[0143] Process 800 may additionally or alternatively include receiving additional audio data from the voice-enabled device, the additional audio data representing another user utterance that causes the voice-enabled device to perform the action. Process 800 may also include: when the audio data is received during the time period, associating the audio data with the user account; and determining that a resource for performing the action is associated with the user account. Process 800 may also include generating a second instruction to perform the action using the resource, and sending the second instruction to the voice-enabled device.

[0144] Process 800 may additionally or alternatively include: receiving additional audio data representing another user utterance performing the action from a second voice-enabled device associated with a second environment. The second environment may be associated with another default account. Process 800 may also include: determining that there are no resources in the other default account for performing the action; and sending an instruction to output audio to the second voice-enabled device, which requests contact information from the user. Process 800 may also include receiving audio data representing another user utterance including contact information from the second voice-enabled device. Process 800 may also include: identifying a user account based at least in part on the contact information; and receiving a second device identifier associated with the second voice-enabled device. Process 800 may also include associating the account identifier with the second device identifier. In this way, a repeat visitor to an environment having a voice-enabled device associated with a remote system can temporarily associate a user account with a voice-enabled device without using a personal device.

[0145] Process 800 may additionally or alternatively include: receiving additional audio data representing another user utterance performing an action from a second voice-enabled device associated with a second environment. Process 800 may also include: identifying a user associated with the user utterance based at least in part on the voice model; and identifying a user account based at least in part on the identified user. Process 800 may also include: receiving a second device identification of the second voice-enabled device from the second voice-enabled device; and associating the account identification with the second device identification.

[0146] Process 800 may additionally or alternatively include: identifying an accessory device associated with the voice-enabled device located in the environment; and identifying a second device identification associated with the accessory device. Process 800 may also include associating the account identification with the second device identification based at least in part on associating the account identification with the first device identification of the voice-enabled device.

[0147] Process 800 may additionally or alternatively include: receiving additional audio data from the voice-enabled device, the additional audio data representing a request to associate an account identification with the voice-enabled device; and sending instructions to the voice-enabled device to cause the voice-enabled device to output a second request to provide contact information via a speaker of the voice-enabled device.

[0148] Process 800 may additionally or alternatively include: determining a time period during which the account identification is to be associated with the device identification; and receiving second audio data during the time period, the second audio data representing a second user utterance requesting performance of the action. Process 800 may also include: determining that a resource for performing the action is not associated with the first user account; and determining that the resource is available for use via a default user account associated with the voice-enabled device. Process 800 may also include selecting a default user account based at least in part on determining that the resource is not associated with the first user account and is associated with the default user account.

[0149] Fig. 9 A flow chart of another example process 900 for temporary account association with a voice-enabled device is shown. The order in which the operations or steps are described is not intended to be construed as a limitation, and any number of the described operations may be combined in any order and / or in parallel to implement process 900.

[0150] At block 902, process 900 may include receiving first audio data representing a first user utterance from a voice-enabled device, the first user utterance indicating a request to perform an action, the voice-enabled device being associated with a default account. For example, a user may speak a user utterance to the voice-enabled device. A microphone of the voice-enabled device may capture the user utterance and may generate corresponding audio data. The audio data may be sent to a remote system via a network, and the remote system may receive the audio data.

[0151] At box 904, process 900 may include determining that the resource for performing the action is not associated with the account. For example, the ASR component of the remote system may process the audio data and generate corresponding text data. The NLU component of the remote system may use the text data to determine intent data representing the user's intent (in these examples, the intent is to perform an action that utilizes one or more system resources) and / or the intent to temporarily associate the account identifier with the voice-enabled device. For example, the user utterance may be "play song A". Using the NLU component, the remote system may determine that resources can be used to perform an action of outputting audio corresponding to song A. For example, the voice-enabled device may be associated with a default user account associated with the environment. The default user account may not have access to resources to output song A. For example, the default user account may not be associated with a music service and / or database that allows song A to be played. In these examples, the voice widget of the remote system may be used to initiate a process of temporarily associating a personal user account associated with the user with the voice-enabled device.

[0152] At block 906, process 900 may include identifying, based at least in part on the audio data, an account identifier associated with the user who uttered the first user utterance, the account identifier having previously been associated with a device identifier of at least one of the speech-enabled device or another speech-enabled device. For example, an instruction to output audio may be sent to the speech-enabled device that requests contact information from the user. Audio data representing another user utterance including contact information may be received from the speech-enabled device, and the contact information may be used to identify the account identifier. Additionally or alternatively, a user identifier associated with the user utterance may be determined based at least in part on a speech model. The account identifier may be identified based at least in part on determining the user identifier.

[0153] At block 908, process 900 may include receiving confirmation data that confirms the intention to associate the account identification with the device identification. For example, a message may be sent to a personal device associated with the user account. The message may request confirmation of the intention to associate the account identification with the device identification.

[0154] At block 910, process 900 may include associating the account identification with the device identification of the voice-enabled device based at least in part on receiving the confirmation data. For example, after receiving the additional audio data from the voice-enabled device, an association management component of the remote system may be utilized to determine that the user account is effectively associated with the device identification associated with the voice-enabled device. This information may be used to determine how to process the audio data and / or which resources may be used to perform one or more actions associated with the audio data.

[0155] Fig.10 A flow chart of an example process 1000 for disassociating an account from a voice-enabled device is shown. The order in which the operations or steps are described is not intended to be construed as a limitation, and any number of the described operations may be combined in any order and / or in parallel to implement process 1000.

[0156] At block 1002, process 1000 may include receiving a first indication indicating that a temporary association between a first account identifier of a user account and a device identifier of a speech-enabled device associated with an environment is established. For example, a user may speak a user utterance to a speech-enabled device. A microphone of the speech-enabled device may capture the user utterance and may generate corresponding audio data. The audio data may be sent to a remote system via a network, and the remote system may receive the audio data.

[0157] The ASR component of the remote system can process the audio data and generate corresponding text data. The NLU component of the remote system can use the text data to determine intent data representing the user's intent (in these examples, the intent is to perform an action that utilizes one or more system resources) and / or the intent to temporarily associate the user account with the voice-enabled device. For example, the user utterance can be "play song A". Using the NLU component, the remote system can determine that resources can be used to perform an action that outputs audio corresponding to song A. For example, the voice-enabled device can be associated with a default user account associated with the environment. The default user account may not have access to resources to output song A. For example, the default user account may not be associated with a music service and / or database that allows song A to be played. In these examples, the voice widget of the remote system can be used to initiate the process of temporarily associating a personal user account associated with the user with the voice-enabled device.

[0158] Audio may be output, such as "We can't perform that operation, please provide your phone number to link your account to this device." The user may provide a second user utterance including contact information, here including a phone number. The microphone of the voice-enabled device may capture audio corresponding to the user utterance, and may generate corresponding audio data. The audio data may be sent to a remote system for processing. For example, an ASR component of the remote system may generate text data corresponding to the audio data and representing the contact information.

[0159] Data representing the device identification can be sent from the voice-enabled device to the remote system along with the second audio data. In other examples, the device identification can be determined by the remote system based at least in part on information associated with the voice-enabled device. A code pair can then be generated that associates the contact information with the device identification associated with the device corresponding to the contact information.

[0160] A message may be generated and may include a link or other optional portion for account credential entry. The notification component may utilize an authentication link component of the remote system to generate the message and / or a portion of the message. For example, the authentication link component may be used to identify, determine, and / or generate an optional portion of the message (such as a web link) that, when selected, may facilitate collection of account access information, such as a username and / or password associated with a personal user account. Input data corresponding to input provided by the user and / or input recognized by an application accessed from a personal device may represent account credentials and may be generated and sent to the remote system.

[0161] After receiving the additional audio data from the voice-enabled device, an association management component of the remote system may be used to determine that the user account is effectively associated with the device identification associated with the voice-enabled device. This information may be used to determine how to process the audio data and / or which resources may be used to perform one or more actions associated with the audio data. The association between the user account and the device identification of the voice-enabled device may be stored, such as at a remote system, and may be used to service user requests via the voice-enabled device.

[0162] At box 1004, process 1000 may include determining that a disassociation event associated with a voice-enabled device has occurred, the disassociation event representing at least one of the past of the identified time period for maintaining a temporary association or receiving a second indication of a disassociation event from a system associated with the environment. For example, a message may be sent to a personal device that causes and / or allows a website application to display a web page that allows a user of the personal device to enter information associated with their stay in the environment. The information may include, for example, a checkout time and / or a checkout day for checking out from the environment. The information may be used to determine a time period during which the association between a user account and a voice-enabled device will be maintained, and when the user account should be disassociated from the voice-enabled device, such as when the time period has passed. By providing this functionality, users are able to control when potential personal information and / or functionality is available to a voice-enabled device, and when the information and functionality should be unavailable.

[0163] The notification component and / or other components of the remote system may be configured to request scheduling data from one or more devices and / or systems associated with the environment. The scheduling data may indicate a time period during which the user remains in the environment, which may be used to identify and / or determine a time period to maintain an association between the user account and the voice-enabled device.

[0164] Additionally or alternatively, the disassociation event may include receiving an indication that a user has checked out from an environment. Additionally or alternatively, the remote system may receive additional audio data representing another user's speech, and the remote system may determine that the intention associated with the user's speech is to disassociate the user account from the voice-enabled device. Receiving the audio data and / or determining that the user's speech corresponds to an intention to disassociate may be considered a disassociation event. Additionally or alternatively, the remote system may receive an indication from a system associated with the environment indicating that the user has checked out from the environment. Receiving the indication may be a disassociation event. The disassociation event may also include a temporary disassociation event. For example, the voice-enabled device and / or the remote system may be configured to receive an indication that a lock and / or a key card reader associated with the environment has been used to leave the room. Based at least in part on the indication, the association management component may be configured to temporarily disassociate the user account from the voice-enabled device, such as until the lock and / or the key card reader indicates that the user has entered the room.

[0165] The association management component may be further configured to determine when one or more events that cause a change in the time period for maintaining the association between the user account and the voice-enabled device occur. For example, the remote system may receive an indication indicating whether the time period will be extended or shortened. For example, a user may decide to check out of a hotel early and / or the user may decide to extend the stay. In other examples, an indication indicating whether the time period has been extended or shortened may be received from a system associated with the environment. Indications of such events may be sent to and received by the remote system and may be used to identify, determine, and / or generate a second time period for maintaining the association between the user account and the voice-enabled device.

[0166] At block 1006, process 1000 may include disassociating the first account identification from the device identification based at least in part on determining that a disassociation event has occurred. Disassociating the account identification from the device identification may result in one or more resources associated with the user account being unassociated with the device identification.

[0167] At block 1008, process 1000 may include associating a second account identification corresponding to the default account with the device identification. For example, an association management device of a remote system may indicate that the device identification associated with the voice-enabled device is associated with a default user account and / or another account instead of a user account. Additionally or alternatively, when processing subsequent voice commands, the association management device may not provide an indication of an active account associated with the device identification, thereby causing the system to associate the default account with the device identification for those subsequent voice commands.

[0168] Fig.11 A flow chart of another example process 1100 for disassociating an account from a voice-enabled device is shown. The order in which the operations or steps are described is not intended to be construed as a limitation, and any number of the described operations may be combined in any order and / or in parallel to implement the process 1100.

[0169] At block 1102, process 1100 may include receiving a first indication indicating that an association between a first account identifier of a user account and a device identifier of a voice-enabled device is established. For example, a user may speak a user utterance to a voice-enabled device. A microphone of the voice-enabled device may capture the user utterance and may generate corresponding audio data. The audio data may be sent to a remote system via a network, and the remote system may receive the audio data.

[0170] The ASR component of the remote system can process the audio data and generate corresponding text data. The NLU component of the remote system can use the text data to determine intent data representing the user's intent (in these examples, the intent is to perform an action that utilizes one or more system resources) and / or the intent to temporarily associate the user account with the voice-enabled device. For example, the user utterance can be "play song A". Using the NLU component, the remote system can determine that resources can be used to perform an action that outputs audio corresponding to song A. For example, the voice-enabled device can be associated with a default user account associated with the environment. The default user account may not have access to resources to output song A. For example, the default user account may not be associated with a music service and / or database that allows song A to be played. In these examples, the voice widget of the remote system can be used to initiate the process of temporarily associating a personal user account associated with the user with the voice-enabled device.

[0171] Audio may be output, such as "We can't perform that operation, please provide your phone number to link your account to this device." The user may provide a second user utterance including contact information, here including a phone number. The microphone of the voice-enabled device may capture audio corresponding to the user utterance, and may generate corresponding audio data. The audio data may be sent to a remote system for processing. For example, an ASR component of the remote system may generate text data corresponding to the audio data and representing the contact information.

[0172] Data representing the device identification can be sent from the voice-enabled device to the remote system along with the second audio data. In other examples, the device identification can be determined by the remote system based at least in part on information associated with the voice-enabled device. A code pair can then be generated that associates the contact information with the device identification associated with the device corresponding to the contact information.

[0173] A message may be generated and may include a link or other optional portion for account credential entry. The notification component may utilize an authentication link component of the remote system to generate the message and / or a portion of the message. For example, the authentication link component may be used to identify, determine, and / or generate an optional portion of the message (such as a web link) that, when selected, may facilitate collection of account access information, such as a username and / or password associated with a personal user account. Input data corresponding to input provided by the user and / or input recognized by an application accessed from a personal device may represent account credentials and may be generated and sent to the remote system.

[0174] After receiving the additional audio data from the voice-enabled device, an association management component of the remote system may be used to determine that the user account is effectively associated with the device identification associated with the voice-enabled device. This information may be used to determine how to process the audio data and / or which resources may be used to perform one or more actions associated with the audio data. The association between the user account and the device identification of the voice-enabled device may be stored, such as at a remote system, and may be used to service user requests via the voice-enabled device.

[0175] At box 1104, process 1100 may include determining that a disassociation event has occurred based at least in part on at least one of the passage of a time period or receipt of a second indication that a disassociation event has occurred. For example, a message may be sent to a personal device that causes and / or allows a website application to display a web page that allows a user of the personal device to enter information associated with their stay in the environment. The information may include, for example, a checkout time and / or a checkout day for checking out from the environment. The information may be used to determine a time period during which the association between a user account and a voice-enabled device will be maintained, and when the user account should be disassociated from the voice-enabled device, such as when the time period has passed. By providing this functionality, users are able to control when potential personal information and / or functionality is available to the voice-enabled device, and when the information and functionality should be unavailable.

[0176] The notification component and / or other components of the remote system may be configured to request scheduling data from one or more devices and / or systems associated with the environment. The scheduling data may indicate a time period during which the user remains in the environment, which may be used to identify and / or determine a time period to maintain an association between the user account and the voice-enabled device.

[0177] Additionally or alternatively, the disassociation event may include receiving an indication that a user has checked out from an environment. Additionally or alternatively, the remote system may receive additional audio data representing another user's speech, and the remote system may determine that the intention associated with the user's speech is to disassociate the user account from the voice-enabled device. Receiving the audio data and / or determining that the user's speech corresponds to an intention to disassociate may be considered a disassociation event. Additionally or alternatively, the remote system may receive an indication from a system associated with the environment indicating that the user has checked out from the environment. Receiving the indication may be a disassociation event. The disassociation event may also include a temporary disassociation event. For example, the voice-enabled device and / or the remote system may be configured to receive an indication that a lock and / or a key card reader associated with the environment has been used to leave the room. Based at least in part on the indication, the association management component may be configured to temporarily disassociate the user account from the voice-enabled device, such as until the lock and / or the key card reader indicates that the user has entered the room.

[0178] The association management component may be further configured to determine when one or more events that cause a change in the time period for maintaining the association between the user account and the voice-enabled device occur. For example, the remote system may receive an indication indicating whether the time period will be extended or shortened. For example, a user may decide to check out of a hotel early and / or the user may decide to extend the stay. In other examples, an indication indicating whether the time period has been extended or shortened may be received from a system associated with the environment. Indications of such events may be sent to and received by the remote system and may be used to identify, determine, and / or generate a second time period for maintaining the association between the user account and the voice-enabled device.

[0179] At block 1106, process 1100 may include disassociating the first account identification from the device identification based at least in part on determining that a disassociation event has occurred. Disassociating the account identification from the device identification may result in one or more resources associated with the user account being unassociated with the device identification.

[0180] At block 1108, process 1100 may include associating a second account identification corresponding to the default account with the device identification. For example, an association management device of a remote system may indicate that the device identification associated with the voice-enabled device is associated with a default user account and / or another account instead of a user account. Additionally or alternatively, when processing subsequent voice commands, the association management device may not provide an indication of an active account associated with the device identification, thereby causing the system to associate the default account with the device identification for those subsequent voice commands.

[0181] Process 1100 may additionally or alternatively include: receiving an indication from a personal device associated with the user account indicating whether the time period will be extended or shortened; and determining a second time period for maintaining the association between the account identifier and the device identifier based at least in part on the indication indicating whether the time period will be extended or shortened. In these examples, disassociating the account identifier from the device identifier may be based at least in part on the second time period passing. Additionally or alternatively, an indication may be received from a system associated with the environment indicating whether the time period will be extended or shortened.

[0182] Process 1100 may additionally or alternatively include: receiving an indication from an accessory device associated with an exit point of the environment indicating that a user associated with a user account has left the environment; and disassociating the account identification from the device identification based at least in part on receiving the indication. For example, the indication may indicate that a door has been opened, a lock has been unlocked, a door handle and / or knob has been turned, a key card has been swiped and / or placed into a key card reader, etc. Process 1100 may also include: receiving a second indication from the accessory device indicating that the user has re-entered the environment during the time period; and associating the account identification with the device identification based at least in part on the second indication.

[0183] Fig.12 A conceptual diagram illustrating how spoken utterances are processed to allow the system to capture and execute a user-spoken command, such as a spoken command that may be followed by a wake word or trigger expression (i.e., a predefined word or phrase used to "wake up" a device to cause it to begin sending audio data to a remote system such as system 120). The various components shown may be located on the same or different physical devices. Fig.12 The messaging between the various components shown in the figure can occur directly or over the network 122. An audio capture component such as the microphone 112 of the device 102 or another device captures audio 1200 corresponding to the spoken utterance. The device 102 then processes the audio data corresponding to the audio 1200 using the wake-up word detection module 1201 to determine whether a keyword (such as a wake-up word) is detected in the audio data. After detecting the wake-up word, the device 102 sends the audio data 1202 corresponding to the utterance to the remote system 120 including the ASR module 152. The audio data 1202 may be output from an optional acoustic front end (AFE) 1256 located on the device before transmission. In other cases, the audio data 1202 may be in a different form for processing by a remote AFE 1256 (such as an AFE 1256 located on the ASR module 152 of the remote system 120).

[0184] The wake-up word detection module 1201 works with other components of the user device (e.g., a microphone) to detect keywords in the audio 1200. For example, the device may convert the audio 1200 into audio data and process the audio data using the wake-up word detection module 1201 to determine whether a human voice is detected, and if so, whether the audio data including the human voice matches an audio signature and / or model corresponding to a particular keyword.

[0185] The user device may use various techniques to determine whether the audio data includes human voices. Some embodiments may apply voice activity detection (VAD) technology. Such techniques may determine whether there is human voice in the audio input based on various quantitative aspects of the audio input, such as: the spectral slope between one or more frames of the audio input; the energy level of the audio input in one or more spectral bands; the signal-to-noise ratio of the audio input in one or more spectral bands; or other quantitative aspects. In other embodiments, the user device may implement a finite classifier configured to distinguish human voices from background noise. The classifier may be implemented by techniques such as linear classifiers, support vector machines, and decision trees. In other embodiments, hidden Markov models (HMM) or Gaussian mixture models (GMM) techniques may be applied to compare the audio input with one or more acoustic models in a human voice storage device, which may include models corresponding to human voices, noise (such as ambient noise or background noise), or silence. Other techniques may also be used to determine whether there is human voice in the audio input.

[0186] Once a human voice is detected in the audio received by the user device (or separately from human voice detection), the user device may use the wake-up word detection module 1201 to perform wake-up word detection to determine when the user intends to speak a command to the user device. This process may also be referred to as keyword detection, where the wake-up word is a specific example of a keyword. Specifically, keyword detection may be performed without performing language analysis, text analysis, or semantic analysis. Instead, the incoming audio (or audio data) is analyzed to determine whether the specific characteristics of the audio match a preconfigured sound wave waveform, audio signature, or other data, thereby determining whether the incoming audio "matches" the stored audio data corresponding to the keyword.

[0187] Therefore, the wake-up word detection module 1201 can compare the audio data with the stored model or data to detect the wake-up word. One method for wake-up word detection is to apply a general large vocabulary continuous speech recognition (LVCSR) system to decode the audio signal while performing a wake-up word search in the generated word grid or confusion network. LVCSR decoding may require relatively high computing resources. Another method for wake-up word discovery is to establish a hidden Markov model (HMM) for each key wake-up word and non-wake-up word speech signal. Non-wake-up word speech includes other spoken words, background noise, etc. One or more HMMs can be established to model non-wake-up word speech characteristics, which are called filling models. Viterbi decoding is used to search for the best path in the decoding graph, and the decoding output is further processed to determine whether there are keywords. By merging a hybrid DNN-HMM decoding framework, the method can be extended to include discriminant information. In another embodiment, a wake-up word discovery system can be built directly on a deep neural network (DNN) / recursive neural network (RNN) structure without involving HMM. Such a system can estimate the posterior of the wake-up word with context information by stacking frames in a context window for DNN or using RNN. Subsequent a posteriori threshold adjustment or smoothing is applied to make the decision.Other techniques for wake-up word detection, such as those known in the art, may also be used.

[0188] Once the wake word is detected, the local device 102 may "wake up" and begin transmitting audio data 1202 corresponding to the input audio 1200 to the remote system 120 for speech processing. The audio data corresponding to the audio may be sent to the remote system 120 for routing to the recipient device, or it may be sent to the remote system 120 for speech processing to interpret the contained speech (for the purpose of enabling voice messages and / or for the purpose of executing commands in the speech). The audio data 1202 may include data corresponding to the wake word, or a portion of the audio data corresponding to the wake word may be removed by the local device 102 before being sent. In addition, the local device may "wake up" when speech / verbal audio above a threshold is detected, as described herein. After being received by the remote system 120, the ASR module 152 may convert the audio data 1202 into text. ASR transcribes the audio data into text data representing words of the speech contained in the audio data 1202. Other components may then use the text data for various purposes, such as executing system commands, inputting data, and the like. The spoken utterance in the audio data is input to a processor configured to perform ASR, which then interprets the utterance based on similarities between the utterance and pre-established language models 1254 stored in an ASR model knowledge base (ASR model storage device 1252). For example, the ASR process can compare the input audio data with models of sounds (e.g., sub-word units or phonemes) and sound sequences to identify words that match the sound sequence spoken in the utterance in the audio data.

[0189] Different ways (i.e., different hypotheses) that can be used to interpret a spoken utterance can each be assigned a probability or confidence score that represents the likelihood that a particular set of words matches the words spoken in the utterance. The confidence score can be based on many factors, including, for example, the similarity of the sounds in the utterance to a model of language sounds (e.g., an acoustic model 1253 stored in an ASR model storage device 1252), and the likelihood that a particular word matching the sounds will be included at a particular position in the sentence (e.g., using a language or grammar model). Thus, each potential textual interpretation (hypothesis) of the spoken utterance is associated with a confidence score. Based on the factors considered and the confidence scores assigned, the ASR process 152 outputs the most likely text identified in the audio data. The ASR process can also output multiple hypotheses in the form of a word grid or N-best list, each hypothesis corresponding to a confidence score or other score (e.g., probability score, etc.).

[0190] The one or more devices performing ASR processing may include an acoustic front end (AFE) 1256 and a speech recognition engine 1258. The acoustic front end (AFE) 1256 converts audio data from a microphone into data for processing by the speech recognition engine 1258. The speech recognition engine 1258 compares the speech recognition data with the acoustic model 1253, the language model 1254, and other data models for identifying information about the speech conveyed in the audio data. The AFE 1256 can reduce noise in the audio data and divide the digitized audio data into frames representing time intervals, and the AFE 1256 determines multiple values ​​(called features) representing the quality of the audio data based on these time intervals, and a collection of these values ​​(called feature vectors), which represents the characteristics / quality of the audio data within the frame. As is known in the art, many different features can be determined, and each feature represents a certain quality of the audio that may be useful for ASR processing. The AFE may process the audio data using a variety of methods, such as Mel-frequency cepstral coefficients (MFCC), perceptual linear prediction (PLP) techniques, neural network eigenvector techniques, linear discriminant analysis, semi-bound covariance matrices, or other methods known to those skilled in the art.

[0191] The speech recognition engine 1258 may reference information stored in the speech / model storage device (1252) to process the output from the AFE 1256. Alternatively, the device performing the ASR processing may receive front-end processed data (such as feature vectors) from another source other than the internal AFE. For example, the user device may process the audio data into feature vectors (e.g., using the AFE 1256 on the device) and transmit the information over the network to the server for ASR processing. The feature vectors may arrive at the remote system 120 encoded, in which case the feature vectors may be decoded before being processed by the processor executing the speech recognition engine 1258.

[0192] The speech recognition engine 1258 attempts to match the received feature vectors with the language phonemes and words as known in the stored acoustic model 1253 and language model 1254. The speech recognition engine 1258 calculates the recognition score of the feature vector based on the acoustic information and the language information. The acoustic information is used to calculate the acoustic score, which represents the likelihood that the expected sound represented by a set of feature vectors matches the language phonemes. The language information is used to adjust the acoustic score by considering which sounds and / or words are used together in context, thereby increasing the likelihood that the ASR process will output a grammatically meaningful speech result. The specific model used can be a general model, or it can be a model corresponding to a specific domain, such as music, banking, etc. For example, the user utterance can be "Alexa, play song A" or "Alexa, play movie B on TV." The wake-up detection module can identify the wake-up word (originally described as a trigger expression) "Alexa" in the user utterance, and can "wake up" based on identifying the wake-up word. The audio data corresponding to the user utterance may be sent to the remote system 120 where the speech recognition engine 1258 may recognize, determine and / or generate text data corresponding to the user utterance, here "play song A" or "play movie B on TV."

[0193] The speech recognition engine 1258 can use a variety of techniques to match feature vectors to phonemes, for example, using a hidden Markov model (HMM) to determine the probability that a feature vector can match a phoneme. The received sound can be represented as a path between HMM states, and multiple paths can represent multiple possible text matches for the same sound.

[0194] After ASR processing, the speech recognition engine 1258 may send the ASR results to other processing components, which may be local to the device performing the ASR and / or distributed throughout the network. For example, the ASR results in the form of a single textual representation of the speech, an N-best list including multiple hypotheses and corresponding scores, a word lattice, etc., may be sent to the remote system 120 for natural language understanding (NLU) processing, such as converting the text into commands for execution by the user device, the remote system 120, or another device (such as a server running a specific application such as a search engine, etc.).

[0195] The device (e.g., server 120) performing NLU processing 154 may include various components, including potentially dedicated processors, memory, storage, etc. Fig.12As shown, the NLU component 154 may include an identifier 1263, which includes a named entity recognition (NER) module 1262, which is used to identify portions of the query text corresponding to named entities that the system is able to recognize. A downstream process called named entity resolution links the text portions to specific entities known to the system. To perform named entity resolution, the system can utilize gazetteer information (1284a-1284n) stored in the entity library storage device 1282. The gazetteer information can be used for entity resolution, such as matching ASR results with different entities (such as voice-enabled devices, accessories, devices, etc.). Gazetteers can be linked to users (e.g., a particular gazetteer can be associated with a particular user's device association), can be linked to certain domains (e.g., music, shopping, etc.), and can be organized in various other ways.

[0196] Typically, the NLU process takes text input (such as text input processed from ASR 152 based on speech input audio 1200) and attempts to semantically interpret the text. That is, the NLU process determines the meaning behind the text based on individual words and then implements that meaning. The NLU process 154 interprets the text string to derive the intended or desired action from the user and the relevant information in the text that allows the device (e.g., device 102) to complete the action. For example, if the spoken speech is processed using ASR 152 and the text "Play Song A" is output, the NLU process can determine that the user intended to cause the audio corresponding to Song A to be output via the voice-enabled device.

[0197] NLU can process multiple text inputs related to the same utterance. For example, if ASR 152 outputs N text segments (as part of an N-best list), NLU can process all N outputs to obtain an NLU result.

[0198] As will be discussed further below, the NLU process can be configured to parse and tag to annotate text during NLU processing. For example, for the text "Play Song A", "Play" can be tagged as a command (for outputting audio), and "Song A" can be tagged as an identifier of the audio to be output.

[0199] In order to properly perform NLU processing of speech input, the NLU process 154 may be configured to determine the "domain" of the utterance in order to determine and narrow down which services provided by the endpoint device (e.g., remote system 120 or user device) may be relevant. For example, the endpoint device may provide services related to interaction with a phone service, a contact list service, a calendar / scheduling service, a music player service, etc. The words in a single text query may imply more than one service, and some services may be functionally related (e.g., a phone service and a calendar service may both use data in a contact list).

[0200] The named entity recognition (NER) module 1262 receives a query in the form of an ASR result and attempts to identify relevant grammatical and vocabulary information that can be used to understand the meaning. To this end, the NLU module 154 may initially identify potential domains that may be related to the received query. The NLU storage device 1273 includes a device database (1274a-1274n) that identifies domains associated with specific devices. For example, a user device may be associated with domains for music, phone, calendar, contact list, and device-specific messages, but not with video. In addition, the entity library may include database entries about specific services on specific devices, which may be indexed by device ID, user ID, or family ID, or some other indicator.

[0201] In NLU processing, a domain may represent a discrete set of activities with a common theme, such as "shopping", "music", "calendar", etc. As such, each domain may be associated with a specific recognizer 1263, a language model and / or grammar database (1276a-1276n), a specific set of intents / actions (1278a-1278n), and a specific personalized dictionary (1286). Each gazetteer (1284a-1284n) may include domain index vocabulary information associated with a specific user and / or device. For example, gazetteer A (1284a) includes domain index vocabulary information 1286aa through 1286an. A user's contact list vocabulary information may include the names of contacts. Since each user's contact list is likely to be different, this personalized information will improve entity resolution.

[0202] As described above, in traditional NLU processing, the query may be processed using rules, models, and information applicable to each identified domain. For example, if the query potentially involves both messages and, for example, music, the query may be processed substantially in parallel using the grammatical model and vocabulary information for messages, and the query will be processed using the grammatical model and vocabulary information for music. Responses to the query generated based on each set of models are scored, and the overall highest ranking result from all applied domains is typically selected as the correct result.

[0203] The intent classification (IC) module 1264 parses the query to determine one or more intents for each identified domain, where the intent corresponds to an action to be performed in response to the query. Each domain is associated with a word database (1278a-1278n) linked to the intent. For example, the music intent database may link words and phrases such as "quiet", "volume down" and "mute" to the "mute" intent. As another example, the timer intent database may link words and phrases such as "set", "start", "start" and "enable" to the "set timer" intent. At the same time, the voice message intent database may link words and phrases such as "send a message", "send a voice message", and "send the following". The IC module 1264 identifies potential intents for each identified domain by comparing the words in the query with the words and phrases in the intent database 1278. In some cases, the determination of the intent by the IC module 1264 is performed using a set of rules or templates that process the input text to identify matching intents.

[0204] In order to generate a response for a particular interpretation, the NER 1262 applies the grammatical model and vocabulary information associated with the corresponding domain to actually identify the mention of one or more entities in the query text. In this way, the NER 1262 identifies the "slots" or values ​​(i.e., specific words in the query text) that may be needed for subsequent command processing. Depending on the complexity of the NER 1262, it can also mark each slot according to types with different specificities (such as nouns, places, cities, artist names, song names, durations, timer numbers, etc.). Each grammatical model 1276 includes the names (i.e., nouns) of entities that are commonly seen in speech about a specific domain (i.e., general terms), while the vocabulary information 1286 from the gazetteer 1284 is personalized according to the user and / or device. For example, a grammatical model associated with the shopping domain may include a database of words that people often use when discussing shopping.

[0205] The intent identified by IC module 1264 is linked to a domain-specific grammar framework (included in 1276), filling "slots" or "fields" with values. Each slot / field corresponds to a portion of the query text that the system deems to correspond to an entity. To make parsing more flexible, these frameworks are typically not structured as sentences, but rather based on associating slots with grammar tags. For example, if "play song A" is an identified intent, one or more grammar (1276) frameworks may correspond to a sentence structure such as "play {song A}".

[0206] For example, before identifying named entities, the NER module 1262 may parse the query based on grammatical rules and / or models to identify words as subjects, objects, verbs, prepositions, etc. The identified verbs may be used by the IC module 1264 to identify intents, which are then used by the NER module 1262 to identify frameworks. Meanwhile, a framework for the "play song" intent may specify a list of slots / fields suitable for playing the identified "songs" as well as any object modifiers (e.g., specifying a music collection from which the song should be accessed), etc. The NER module 1262 then searches the corresponding fields in the domain-specific personalized dictionary, attempting to match words and phrases in the query that are marked as grammatical objects or object modifiers with words and phrases identified in the database.

[0207] This process includes semantic tagging, i.e. tagging words or word combinations according to their type / semantic meaning. Parsing can be performed using heuristic grammar rules, or NER models can be built using techniques such as Hidden Markov Models, Maximum Entropy Models, Log-Linear Models, Conditional Random Fields (CRFs), etc.

[0208] The framework linked to the intent is then used to determine which database fields should be searched to determine the meaning of these phrases, such as searching the user's gazetteer for similarity to the framework slots. If the search of the gazetteer does not use the gazetteer information to resolve the slots / fields, the NER module 1262 may search a database of common words associated with the domain (in the knowledge base 1272). Thus, for example, if the query is "play song A", then after failing to determine which song to play, the NER component 1262 may search the domain vocabulary for the phrase "song A". Alternatively, the common words may be checked before the gazetteer information, or both may be tried, potentially producing two different results.

[0209] The output data from the NLU processing (which may include marked text, commands, etc.) can then be sent to the command processor 1207. The destination command processor 1207 can be determined based on the NLU output. For example, if the NLU output includes a command to send a message, the destination command processor 1207 can be a messaging application configured to execute the message sending command, such as a messaging application located on the user's device or in a messaging appliance. If the NLU output includes a search request, the destination command processor 1207 may include a search engine processor configured to execute the search command, such as a search engine processor located on a search server. After generating the appropriate command based on the user's intent, the command processor 1207 can provide some or all of this information to a text-to-speech (TTS) engine 158. The TTS engine 158 can then generate an actual audio file for outputting the audio data determined by the command processor 1207 (e.g., "Account linked successfully" or "Try your request again"). After generating the file (or "audio data"), the TTS engine 158 can provide the data back to the remote system 120.

[0210] The NLU operation of the existing system can take the form of a multi-domain architecture. Each domain (which can include a set of intents and entity slots that define a larger concept such as music, books, etc., and components such as trained models, etc. for performing various NLU operations such as NER, IC, etc.) can be constructed separately and can be used by the NLU component 154 during runtime operation when performing NLU operations on text (such as text output from the ASR component 152). Each domain can have components that are specially configured to perform each step of the NLU operation.

[0211] For example, in an NLU system, the system may include a multi-domain architecture consisting of multiple domains for intents / commands executable by the system (or by other devices connected to the system), such as music, video, books, and information. The system may include multiple domain identifiers, where each domain may include its own identifier 1263. Each identifier may include various NLU components such as NER component 1262, IC module 1264, and other components, such as entity resolvers or other components.

[0212] For example, a messaging domain identifier 1263-A (Domain A) may have a NER component 1262-A that identifies which slots (i.e., portions of input text) may correspond to specific words associated with the domain. The words may correspond to entities such as (for the messaging domain) a recipient. The NER component 1262 may use a machine learning model, such as a domain-specific conditional random field (CRF), to identify portions corresponding to entities and to identify which type of entity corresponds to the portion of text. The messaging domain identifier 1263-A may also have its own intent classification (IC) component 1264-A that determines the intent of the text, assuming the text is within a prohibited domain. The IC component may use a model (such as a domain-specific maximum entropy classifier) ​​to identify the intent of the text, where the intent is the action that the user wants the system to perform. To this end, the remote system computing device 120 may include a model training component. The model training component may be used to train the classifier / machine learning model discussed above.

[0213] As described above, multiple devices may be employed in a single speech processing system. In such a multi-device system, each device may include different components for performing different aspects of speech processing. The multiple devices may include overlapping components. As shown herein, the components of the user device and remote system 120 are exemplary and may be located in separate devices, or may be included as a component of a larger device or system, in whole or in part, may be distributed on a network or on multiple devices connected via a network, etc.

[0214] Fig.13 A conceptual diagram of components of a speech processing system 120 for associating audio output commands with multiple devices is shown, including a command processor 1207 configured to generate commands for selected speech-enabled devices to use in response to user utterances. Fig.12 As used, the speech-enabled device may include, for example, reference Figure 1B The voice-enabled device 102 described above. Fig.12As shown, the speech processing system 120 can be coupled to a target determination component 1334 and provide the intent determined to be expressed in the user's speech to the target determination component 1334; the speech processing system includes an orchestration component 1324, which includes an ASR component 152 and an NLU component 154. In addition, the arbitration component 1330 can provide a ranked list of devices to the target determination component 1334, as well as a device indicator (e.g., IP address, device name, etc.) of one or more speech-enabled devices in the ranked list of devices. The target determination component 1334 can then perform techniques for determining a target device (e.g., a device that performs the requested operation) and provide various data to the command processor 1207. For example, the target determination component 1334 can provide the command processor 1207 with various device identifications of the speech-enabled device, the determined target device, the determined intent and / or command, etc.

[0215] The command processor 1207 and / or the NLU component 154 may determine the domain based on the intent, and based on the determination, route the request corresponding to the audio data to the appropriate domain voice widget, such as the domain voice widget 156 shown. The domain voice widget 156 may include any type of device or device group (e.g., a hardware device, a virtual device or partition, a server, etc.), and may receive text data and / or an intent associated with the audio signal, and may determine how to respond to the request. For example, an intent for the command "play song A" may be routed to the music domain voice widget 156, which controls the generation of instruction data for sending the instruction data to a voice-enabled device to output audio corresponding to a song.

[0216] Various types of domain voice widgets 156 may be used to determine which device to send a command to and / or which device to use in response to a user utterance, as well as an appropriate response and potential additional content (e.g., audio data). For example, domain voice widgets 156 may include: a third-party skill domain voice widget 156 that may handle intents associated with games, productivity, etc.; a music domain voice widget 156 that may handle intents associated with music playback requests (e.g., Amazon Music, Pandora, Spotify, iHeart, etc.); and / or an information domain voice widget 156 that may handle requests for information associated with the state of a particular device and / or content being utilized and / or output of particular device and / or device group information.

[0217] After the domain voice widget 156 generates the appropriate command (which may be described herein as instruction data) based on the user's intent and / or provides additional content such as audio data to be output by one of the voice-enabled devices, the voice widget 156 may provide this information back to the speech system 120, which in turn provides some or all of this information to a text-to-speech (TTS) engine 158. The TTS engine 158 then generates the actual audio file for outputting the second audio data determined by the domain voice widget 156. After generating the file (or "audio data"), the TTS engine 158 may provide this data back to the speech system 120.

[0218] The speech system 120 may then publish (i.e., write) some or all of this information to the event bus 1346. That is, the speech system 120 may provide information to the event bus 1346 regarding the initial request (e.g., voice, text, domain / intent, etc.), a response to be provided to the speech-enabled device, or any other information related to the interaction between the speech-enabled device and the speech processing system 120.

[0219] Within the speech processing system 120, one or more components or services may subscribe to the event bus 1346 to receive information about interactions between user devices and the speech processing system 120. For example, in the example shown, the device management component 1348 may subscribe to the event bus 1346 and may therefore monitor information about these interactions. In some examples, the monitoring information in the event bus 1346 may include messages between the various components of the speech processing system 120. For example, the target determination component 1334 may monitor the event bus 1346 to identify device state data about the speech-enabled device. In some examples, the event bus 1346 may "push" or send indications of events and / or device state data to the target determination component 1334. Additionally or alternatively, the event bus 1346 may be "pulled", wherein the target determination component 1334 sends a request to the event bus 1346 to provide an indication of the device state data of the speech-enabled device. The event bus 1346 can store an indication of the device state for the device, such as in a database (e.g., user registry 148), and use the stored indication of the device state to send device state data about the voice-enabled device to the targeting component 1334. Thus, to identify the device state data for the device, the targeting component 1334 can send a request to the event bus 1346 (e.g., an event component) to provide an indication of the device state data associated with the device, and receive the requested device state data from the event bus 1346.

[0220] The device management component 1348 is used to monitor information published to the event bus 1346 and identify events that can trigger actions. For example, the device management component 1348 can identify (e.g., via filtering) the following events: (i) from a voice-enabled device associated with an auxiliary device (e.g., having an auxiliary device in its environment, such as a television, a personal computing device, an accessory device, etc.), and (ii) associated with supplemental content (e.g., image data, video data, etc.). The device management component 1348 can refer to the user registry 148 to determine which voice-enabled devices are associated with the auxiliary device, and to determine the device type, state, and other capabilities of these auxiliary devices. For example, the device management component 1348 can determine from the information published to the event bus 1346 an identifier associated with the voice-enabled device that made the corresponding request or the voice-enabled device that was selected to respond to the user's speech or operate according to the user's speech. The device management component 1348 can use the identifier to identify the user account associated with the voice-enabled device from the user registry 148. The device management component 1348 can also determine whether any auxiliary devices are registered with the identified user account, as well as the capabilities of any such auxiliary devices, such as how the auxiliary devices are configured to communicate (e.g., via WiFi, short-range wireless connections, etc.), the type of content that the devices are capable of outputting (e.g., audio, video, still images, flash, etc.), etc.

[0221] The device management component 1348 may determine whether the identified specific event is associated with supplementary content. That is, the device management component 1348 may write instructions about which types of events and / or which main content or responses are associated with supplementary content to a data repository. In some cases, the speech processing system 120 may grant third-party developers access rights to allow developers to register supplementary content for output on auxiliary devices for specific events and / or main content. For example, if the voice-enabled device is to output that the weather will include thunder and lightning, the device management component 1348 may store instructions for supplementary content such as pictures / animations of thunder and lightning. For another example, if the voice-enabled device is outputting information about a specific fact (e.g., "the blue whale is the largest mammal on earth..."), the auxiliary device (such as a television) may be configured to provide supplementary content, such as a video or picture of a blue whale. For another example, if the voice-enabled device is outputting audio, the second device (such as a speaker) may be configured to also output the audio based at least in part on a user's speech representing a request to add the auxiliary device to the audio session. In these and other examples, device management component 1348 can store an association between a primary response or content (e.g., outputting information about the world's largest mammal) and corresponding supplemental content (e.g., audio data, image data, etc.). In some cases, device management component 1348 can also indicate which types of auxiliary devices are to output which supplemental content. For example, in this example, device management component 1348 can store the following indication: auxiliary devices of category type "tablet" are to output pictures of blue whales. In these and other cases, device management component 1348 can also store supplemental content in association with auxiliary device functionality (e.g., a device with speakers outputs a secondary audio track, a device with a screen outputs an image, etc.).

[0222] The device management component 1348 may also determine how to transmit responses and / or supplemental content (and / or information to obtain content) to the speech-enabled device and / or the auxiliary device. To make this determination, the device management component 1348 may determine the device type of the speech-enabled device and / or the auxiliary device, the capabilities of the device, etc., as potentially stored in the user registry 148. In some cases, the device management component 1348 may determine that a particular device is capable of communicating directly with the speech processing system 120 (e.g., via WiFi), and thus the device management component 1348 may provide the response and / or content to the auxiliary device directly over the network 122 (possibly via the speech system 120). As another example, the device management component 1348 may determine that a particular auxiliary device is not capable of communicating directly with the speech processing system 120, but is instead configured to communicate with speech-enabled devices in its environment over a short-range wireless network. In this way, the device management component 1348 can provide supplemental content (or information) to the speech system 120, which can then send the supplemental content (or information) to the speech-enabled device, which can send the information to the auxiliary device via the short-range network.

[0223] The computer-readable medium 146 may also include a user registry 148, which includes data about user profiles as described herein. The user registry 148 may be located as part of the speech processing system 120 or near the speech processing system, or may transmit messages with various components, such as through the network 122. The user registry 148 may include various information about various users, accounts, etc. that interact with the speech-enabled device and the speech processing system 120. For illustration, the user registry 148 may include data about devices associated with a specific individual user profile. Such data may include user or device identification (ID) and Internet Protocol (IP) address information for different devices, as well as names that users can use to reference devices. Other modifiers describing the device and descriptions of the type of object of the device may also be listed. In addition, the user registry 148 may store indications of associations between various speech-enabled devices and / or auxiliary devices, such as virtual clusters of devices, states of devices, and associations between devices and audio session queues. The user registry 148 may represent a cluster of devices and / or a single device that can receive commands and spread commands to each device and / or spread commands in a cluster. In some examples, a virtual cluster of devices may be represented as a single device that is determined to be capable or incapable (e.g., offline) of executing a command in a user utterance. A virtual cluster of devices may generally correspond to a stored grouping of devices, or a stored association between a set of devices.

[0224] In some examples, the device state of a device associated with a user account may indicate the current state of the device. In this way, the command processor 1207 and / or the domain voice widget 156 may determine the current device state of the voice-enabled device based on the stored device state in the user registry 148. Different from receiving the device state of the voice-enabled device, in the metadata, the device state may have been determined or received and stored in the user registry 148. In addition, the user registry 148 may provide indications of various permission levels based on the user. For example, the voice system 120 may perform speaker recognition on the audio signal to determine the identity of the speaker. For example, if the speaker is a child, the child profile may have permission constraints, for example, they cannot request to output audio via certain devices and / or cannot output certain audio on one or more devices. In contrast, the parent profile may be able to manage the output of audio without constraints.

[0225] In some examples, to determine the device state, the event bus 1346 may publish different events that indicate the device state to various entities or components that subscribe to the event bus 1346. For example, if a "play song A" event occurs for a voice-enabled device, the event bus 1346 may publish an indication of the event, and thus the device state of the voice-enabled device from which the audio data was received may be determined. Thus, indications of various device states may be provided to various components such as the target determination component 1334 via the event bus 1346. The event bus 1346 may also store and / or update the device state of the voice-enabled device in the user registry 148. Components of the speech processing system 120 may query the user registry 148 to determine the device state.

[0226] A particular user profile may include various data that may be used by the system 120. For example, a user profile may include information about which voice-enabled devices are associated with a user and / or user profile. The user profile may further indicate the IP address of each device associated with the user and / or user profile, the user ID of the device, an indication of the type of device, and the current device status of the device.

[0227] The embodiments of the present disclosure can be described based on the following items.

[0228] 1. A system comprising: one or more processors; and a computer-readable medium storing computer-executable instructions, which, when executed by the one or more processors, cause the one or more processors to perform the following operations: receiving first audio data from a voice-enabled device, the first audio data representing a first user utterance requesting performance of an action, the voice-enabled device being associated with a first account; determining that a resource for performing the action is not associated with the first account; sending an instruction to the voice-enabled device to output audio via a speaker of the voice-enabled device, the audio representing a request for a telephone number; receiving second audio data from the voice-enabled device, the second audio data representing a second user utterance including the telephone number; receiving a device identification of the voice-enabled device from the voice-enabled device; generating a message associated with a second request, the second request being to provide authentication information associated with a second account associated with a user who provided the first user utterance; sending the message to a personal device associated with the telephone number; receiving authentication data from the personal device, the authentication data identifying the second account and authenticating access to the second account; and associating an account identification corresponding to the second account with the device identification of the voice-enabled device.

[0229] 2. A system according to clause 1, wherein the message includes a first message, and the operation also includes: sending a second message associated with a third request to the personal device, the third request being an indication of when to disassociate the account identifier from the device; receiving the indication from the personal device; determining a time period from the indication; and disassociating the account identifier from the device identifier when the time period has passed.

[0230] 3. A system according to clause 1 or 2, wherein the instructions include a first instruction, and the operation further includes: receiving third audio data from the voice-enabled device, the third audio data representing a third user utterance requesting performance of the action; determining that the third audio data is received during a time period when the account identifier is associated with the device identifier; associating the third audio data with the account identifier; determining that the resource can be accessed via the second account; and causing the action to be performed.

[0231] 4. A system according to clauses 1, 2 or 3, wherein the device identifier includes a first device identifier, and the operation further includes: identifying an accessory device associated with an environment of the voice-enabled device; identifying a second device identifier associated with the accessory device; and associating the account identifier with the second device identifier based at least in part on associating the account identifier with the first device identifier.

[0232] 5. A method comprising: receiving audio data from a voice-enabled device, the audio data representing user speech including contact information; receiving a device identification of the voice-enabled device from the voice-enabled device; generating an encrypted device identification; generating a code pair associated with an optional link and including the encrypted device identification; generating a message associated with a request based at least in part on the code pair, the request being to provide authentication information associated with a user account; sending the message to a personal device associated with the contact information; receiving authentication data from the personal device, the authentication data authenticating access to the user account via the optional link; and associating the account identification of the user account with the device identification of the voice-enabled device based at least in part on receiving the authentication data.

[0233] 6. A method according to clause 5, wherein the message includes a first message, the request includes a first request, and the method further includes: sending a second message associated with a second request to the personal device, the second request being an indication of when to disassociate the account identifier from the device identifier; receiving the indication from the personal device; determining a time period based at least in part on the indication; and disassociating the account identifier from the device identifier based at least in part on the passage of the time period.

[0234] 7. A method according to clause 5 or 6, wherein the audio data includes first audio data, the user speech includes first user speech, and the method further includes: receiving second audio data from the voice-enabled device, the second audio data representing a second user speech requesting the voice-enabled device to perform an action; determining that the second audio data is received when the user account is associated with the device identifier; determining that a resource for performing the action is associated with the user account; and causing the action to be performed.

[0235] 8. A method according to clauses 5, 6 or 7, wherein receiving the authentication data includes receiving the authentication data based at least in part on: sending an instruction to an application residing on the personal device and associated with the voice-enabled device, the instruction requesting the authentication data as stored in a memory associated with the application; and receiving the authentication data from the application.

[0236] 9. The method according to clause 5, 6, 7 or 8 further includes: receiving preference data indicating preference settings via a default account associated with the voice-enabled device, the preferences implementing a temporary account identifier association, and wherein associating the account identifier with the device identifier is based at least in part on the preference data.

[0237] 10. A method according to clauses 5, 6, 7, 8 or 9, wherein the device identifier includes a first device identifier, and the method further includes: identifying an accessory device associated with an environment of the voice-enabled device; identifying a second device identifier associated with the accessory device; and associating the account identifier with the second device identifier based at least in part on associating the account identifier with the first device identifier.

[0238] 11. A method according to clauses 5, 6, 7, 8, 9 or 10, wherein the audio data includes first audio data, and the method further includes: receiving second audio data from the voice-enabled device, the second audio data representing a first request to associate the user account with the voice-enabled device; and sending an instruction to the voice-enabled device to cause the voice-enabled device to output audio via a speaker of the voice-enabled device, the audio representing a second request to provide contact information, wherein the first audio data is received at least in part based on the second request.

[0239] 12. A method according to clauses 5, 6, 7, 8, 9, 10 or 11, wherein the audio data includes first audio data, the user utterance includes a first user utterance, the user account includes a first user account, and the method further includes: determining a time period during which the account identifier will be associated with the device identifier; receiving second audio data during the time period, the second audio data representing a second user utterance requesting the voice-enabled device to perform the action; determining that a resource for performing the action is not associated with the first user account; determining that the resource is available via a default user account associated with the voice-enabled device; and selecting the default user account based at least in part on determining that the resource is not associated with the first user account and is associated with the default user account.

[0240] 13. A method comprising: receiving first audio data representing a first user utterance from a voice-enabled device, the first user utterance indicating a request to perform an action, the voice-enabled device being associated with a default account; determining that a resource for performing the action is not associated with the account; identifying an account identifier associated with a user who uttered the first user utterance based at least in part on the audio data, the account identifier having been previously associated with a device identifier of at least one of the voice-enabled device or another voice-enabled device; receiving confirmation data confirming an intention to associate the account identifier with the device identifier; and associating the account identifier with the device identifier of the voice-enabled device based at least in part on receiving the confirmation data.

[0241] 14. A method according to clause 13, wherein the request includes a first request, and the method further includes: sending a message associated with a second request to a personal device, the second request being an indication of when to disassociate the account identifier from the device identifier; receiving the indication from the personal device; determining a time period based at least in part on the indication; and disassociating the account identifier from the device identifier based at least in part on the passage of the time period.

[0242] 15. A method according to clause 13 or 14, wherein the audio data includes first audio data, the user speech includes first user speech, and the operation further includes: receiving second audio data from the voice-enabled device, the second audio data representing a second user speech requesting execution of an action; associating the second audio data with the account identifier based at least in part on receiving the second audio data when the account identifier is associated with the device identifier; determining that a resource for performing the action is associated with a user account corresponding to the account identifier; and causing the action to be performed.

[0243] 16. A method according to clauses 13, 14 or 15, wherein the audio data includes first audio data, the user utterance includes a first user utterance, and identifying the account identifier includes identifying the account identifier based at least in part on: sending an instruction to the voice-enabled device, the instruction being audio for outputting a request for contact information; receiving second audio data from the voice-enabled device, the second audio data representing a second user utterance including the contact information; and using the contact information to identify the account identifier.

[0244] 17. A method according to clauses 13, 14, 15 or 16, wherein identifying the account identifier includes identifying the account identifier based at least in part on the following operations: determining a user identifier associated with the user utterance based at least in part on a speech model; identifying the account identifier based at least in part on the user identifier.

[0245] 18. A system according to clauses 13, 14, 15, 16 or 17, wherein the device identifier includes a first device identifier, and the system further includes: identifying an accessory device associated with the device identifier; identifying a second device identifier associated with the accessory device; and associating the account identifier with the second device identifier based at least in part on associating the account identifier with the first device identifier.

[0246] 19. The method according to clauses 13, 14, 15, 16, 17 or 18 further includes: receiving preference data indicating preference settings via the default account, the preferences implementing a temporary account identifier association, and wherein associating the account identifier with the device identifier is based at least in part on the preference data.

[0247] 20. A method according to clauses 13, 14, 15, 16, 17, 18 or 19, wherein the audio data includes first audio data, the user utterance includes a first user utterance, the user account includes a first user account, and the method further includes: determining a time period during which the account identifier will be associated with the device identifier; receiving second audio data during the time period, the second audio data representing a second user utterance requesting execution of the action; determining that the resource for performing the action is not associated with the user account associated with the account identifier; determining that the resource is associated with the default account; and selecting the default account based at least in part on determining that the resource is not associated with the user account and is associated with the default account.

[0248] 21. A system comprising: one or more processors; and a computer-readable medium storing computer-executable instructions, which, when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising: receiving a first indication indicating that a temporary association is established between a first account identifier of a user account and a device identifier of a voice-enabled device associated with an environment; determining that a disassociation event associated with the voice-enabled device has occurred, the disassociation event indicating at least one of the passage of a time period for maintaining the temporary association or receipt of a second indication of the disassociation event from a system associated with the environment; disassociating the first account identifier from the device identifier based at least in part on determining that the disassociation event has occurred; and associating a second account identifier corresponding to a default account with the device identifier.

[0249] 22. According to the system of clause 21, the operation also includes: sending a message to a personal device associated with the user account, the message requesting a third indication of when to disassociate the first account identifier from the device identifier; and determining the time period from the third indication.

[0250] 23. According to the system of clause 21 or 22, the operation also includes: sending a request for checkout data to the system associated with the environment, the checkout data indicating a checkout date and a checkout time associated with the environment; receiving the checkout data from the system; and determining the time period based on the checkout data.

[0251] 24. A system according to clause 21, 22 or 23, wherein the disassociation event includes checking out from a hotel room, and the operation further includes: receiving a third indication from the system associated with the environment indicating that the environment has been checked out, wherein the disassociation event corresponds to the third indication.

[0252] 25. A method comprising: receiving a first indication indicating that an association between a first account identifier of a user account and a device identifier of a voice-enabled device has been established; determining that a disassociation event has occurred based at least in part on at least one of the passage of a time period or receipt of a second indication that the disassociation event has occurred; disassociating the first account identifier from the device identifier based at least in part on determining that the disassociation event has occurred; and associating a second account identifier corresponding to a default account with the device identifier.

[0253] 26. The method of clause 25 further comprising: sending a message to a personal device associated with the user account, the message requesting a third indication of when to disassociate the first account identifier from the device identifier; and determining the time period from the third indication.

[0254] 27. The method according to clause 25 or 26 also includes: sending request data to a remote system, the request data representing a request for checkout data, the checkout data indicating at least one of a date or a time when the first account identifier will be disassociated from the device identifier; receiving the checkout data from the remote system; and determining the time period based at least in part on the checkout data.

[0255] 28. The method of clause 25, 26 or 27, wherein the second indication is based at least in part on checking out from the environment in which the voice-enabled device is located.

[0256] 29. A method according to clauses 25, 26, 27 or 28, wherein the time period includes a first time period, and the method further includes: receiving a third indication from a personal device associated with the first account identifier, the third indication indicating whether the time period will be extended or shortened; and determining a second time period for maintaining the association between the first account identifier and the device identifier, the second time period being based at least in part on the third indication.

[0257] 30. A method according to clauses 25, 26, 27, 28 or 29, wherein the time period includes a first time period, and the method further includes: receiving a third indication from a remote system, the third indication indicating whether the time period has been extended or shortened; and determining a second time period for maintaining the association between the first account identifier and the device identifier, the second time period being based at least in part on the third indication.

[0258] 31. The method according to clauses 25, 26, 27, 28, 29 or 30 further includes: receiving audio data representing user speech from the voice-enabled device; determining intent data from text data corresponding to the audio data, the intent data indicating an intention to disassociate the user account from the device identifier; and wherein determining that the disassociation event has occurred includes determining that the disassociation event has occurred based at least in part on the intent data.

[0259] 32. A method according to clauses 25, 26, 27, 28, 29, 30 or 31, wherein the time period includes a first time period, and the method further includes: receiving input data from at least one of the voice-enabled device or a personal device associated with the user account, the input data indicating a request to temporarily disassociate the first account identifier from the device identifier during a second time period; disassociating the first account identifier from the device identifier during the second time period; and associating the first account identifier with the device identifier based at least in part on the passage of the second time period.

[0260] 33. A system comprising: one or more processors; and a computer-readable medium storing computer-executable instructions, which, when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising: receiving a first indication indicating that an association between a first account identifier of a user account and a device identifier of a voice-enabled device has been established; determining that a disassociation event has occurred based at least in part on at least one of the passage of a time period or receipt of a second indication that the disassociation event has occurred; disassociating the first account identifier from the device identifier based at least in part on determining that the disassociation event has occurred; and associating a second account identifier corresponding to a default account with the device identifier.

[0261] 34. According to the system of clause 33, the operation also includes: sending a message to a personal device associated with the user account, the message requesting a third indication of when to disassociate the first account identifier from the device identifier; and determining the time period from the third indication.

[0262] 35. According to the system of clause 33 or 34, the operation also includes: sending request data to a remote system, the request data representing a request for checkout data, the checkout data indicating at least one of a date or a time when the first account identifier will be disassociated from the device identifier; receiving the checkout data from the remote system; and determining the time period based at least in part on the checkout data.

[0263] 36. According to the system of clauses 33, 34 or 35, the operation also includes: generating a message based at least in part on disassociating the first account identifier from the device identifier, the message including an indication that the first account identifier has been disassociated from the device identifier; and sending the message to a personal device associated with the user account.

[0264] 37. A system according to clauses 33, 34, 35 or 36, wherein the operation further includes: receiving a third indication from an accessory device associated with an exit point of the environment in which the voice-enabled device is located, the third indication indicating that a user associated with the user has left the environment; disassociating the first account identifier from the device identifier based at least in part on receiving the third indication; receiving a fourth indication from the accessory device, the fourth indication indicating that the user has re-entered the environment during the time period; and associating the first account identifier with the device identifier based at least in part on the fourth indication.

[0265] 38. A system according to clauses 33, 34, 35, 36 or 37, wherein the time period includes a first time period, and the operation further includes: receiving a third indication from a remote system, the third indication indicating whether the time period has been extended or shortened; and determining a second time period for maintaining the association between the first account identifier and the device identifier, the second time period being based at least in part on the third indication.

[0266] 39. In a system according to clauses 33, 34, 35, 36, 37 or 38, the operation further includes: receiving audio data representing user speech from the voice-enabled device; determining intent data from text data corresponding to the audio data, the intent data indicating an intention to disassociate the first account identifier from the device identifier; and determining that the disassociation event has occurred based at least in part on the intent data.

[0267] 40. A system according to clauses 33, 34, 35, 36, 37, 38 or 39, wherein the time period includes a first time period, and the operation further includes: receiving input data from at least one of the voice-enabled device or a personal device associated with the user account, the input data indicating a request to temporarily disassociate the first account identifier from the device identifier during a second time period; disassociating the first account identifier from the device identifier during the second time period; and associating the first account identifier with the device identifier based at least in part on the passage of the second time period.

[0268] Although the foregoing invention has been described with reference to specific examples, it should be understood that the scope of the invention is not limited to these specific examples. Since other modifications and variations to adapt to specific operating requirements and environments will be apparent to those skilled in the art, the invention should not be considered to be limited to the examples selected for disclosure purposes, and the invention encompasses all variations and modifications that do not constitute a departure from the true spirit and scope of the invention.

[0269] Although the present application describes embodiments with specific structural features and / or method actions, it should be understood that the claims are not necessarily limited to the specific features or actions described. Instead, the specific features and actions are only examples of some embodiments falling within the scope of the claims of the present application.

Claims

1. A method for temporarily associating a user account with a voice-enabled device, the method comprising: receiving first audio data representing a first user utterance from a voice-enabled device to perform an action, the first user utterance indicating a first request, the voice-enabled device being associated with a default account; determining that a resource used to perform the action is not associated with the default account; identifying, based at least in part on the audio data, an account identifier associated with a user who uttered the first user utterance, the user account corresponding to the account identifier, the account identifier having been previously associated with a device identifier of at least one of the speech-enabled device or another speech-enabled device; receiving confirmation data confirming an intent to associate the account identifier with the device identifier; associating the account identification with the device identification of the voice-enabled device based at least in part on receiving the confirmation data; sending a message associated with the second request to the personal device, the message being an indication of when to disassociate the account identifier from the device identifier; receiving the indication from the personal device; determining a time period based at least in part on the indication; as well as The account identification is disassociated from the device identification based at least in part on the passage of the time period.

2. The method according to claim 1, further comprising: receiving second audio data from the voice-enabled device, the second audio data representing a second user utterance requesting performance of an action; associating the second audio data with the account identification based at least in part on receiving the second audio data when the account identification is associated with the device identification; determining that a resource for performing the action is associated with the user account; as well as Causes the action to be performed.

3. The method according to claim 1, further comprising: determining a time period during which the account identifier is to be associated with the device identifier; receiving second audio data during the time period, the second audio data representing a second user utterance requesting performance of the action; determining that the resource used to perform the action is not associated with the user account; determining that the resource is associated with the default account; as well as The default account is selected based at least in part on a determination that the resource is not associated with the user account and is associated with the default account.

4. A method for temporarily associating a user account with a voice-enabled device, the method comprising: receiving a first indication indicating that an association between a first account identifier of the user account and a device identifier of the voice-enabled device is established; sending a message associated with a second request to a personal device associated with the user account, the message being an indication of when to disassociate the first account identifier from the device identifier; receiving the indication from the personal device; determining a time period based at least in part on the indication; determining that the disassociation event has occurred based at least in part on the passage of the time period; disassociating the first account identification from the device identification based at least in part on determining that the disassociation event has occurred; as well as A second account identifier corresponding to the default account is associated with the device identifier.

5. The method of claim 4, wherein the time period comprises a first time period, and the method further comprises: receiving a third indication from the personal device, the third indication indicating that the time period is to be extended or shortened; as well as A second time period is determined for maintaining the association between the first account identification and the device identification, the second time period being based at least in part on the third indication.

6. The method according to claim 4, further comprising: receiving audio data representing user utterances from the speech-enabled device; determining intent data from text data corresponding to the audio data, the intent data indicating an intent to disassociate the user account from the device identification; and Wherein determining that the disassociation event has occurred comprises determining that the disassociation event has occurred based at least in part on the intent data.

7. The method of claim 4, wherein the time period comprises a first time period, and the method further comprises: receiving input data from at least one of the voice-enabled device or the personal device, the input data indicating a request to temporarily disassociate the first account identifier from the device identifier for a second time period; disassociating the first account identifier from the device identifier during the second time period; as well as The first account identification is associated with the device identification based at least in part on the second time period expiring.

8. A method for temporarily associating a user account with a voice-enabled device, comprising: receiving audio data from the voice-enabled device, the audio data representing a user utterance requesting that the voice-enabled device be associated with the user account; receiving, from the voice-enabled device, a device identification of the voice-enabled device; Generate an encrypted device ID; generating a code pair including the encrypted device identification; generating, based at least in part on the code pair, a first message associated with a first request to provide authentication information associated with the user account; sending the first message to a personal device associated with the user account; receiving authentication data from the personal device, the authentication data authenticating access to the user account; associating an account identifier of the user account with the device identifier of the voice-enabled device based at least in part on receiving the authentication data; sending a second message associated with a second request to the personal device, the second message being an indication of when to disassociate the account identifier from the device identifier; receiving the indication from the personal device; determining a time period based at least in part on the indication; as well as The account identification is disassociated from the device identification based at least in part on the passage of the time period.

9. The method of claim 8, wherein the audio data comprises first audio data, the user utterance comprises a first user utterance, and the method further comprises: receiving second audio data from the voice-enabled device, the second audio data representing a second user utterance requesting the voice-enabled device to perform an action; determining that the second audio data is received when the user account is associated with the device identification; determining that a resource for performing the action is associated with the user account; and Causes the action to be performed.

10. The method of claim 8, wherein receiving the authentication data comprises receiving the authentication data based at least in part on: sending an instruction to an application resident on the personal device and associated with the voice-enabled device, the instruction requesting the authentication data as stored in a memory associated with the application; and The authentication data is received from the application.

11. The method of claim 8, wherein the device identification comprises a first device identification, and the method further comprises: identifying an accessory device associated with an environment of the voice-enabled device; identifying a second device identifier associated with the accessory device; as well as The account identification is associated with the second device identification based at least in part on associating the account identification with the first device identification.

12. The method of claim 8, wherein the audio data comprises first audio data, and the method further comprises: receiving, from the voice-enabled device, second audio data representing a third request to associate the user account with the voice-enabled device; as well as Instructions are sent to the voice-enabled device to cause the voice-enabled device to output audio, via a speaker of the voice-enabled device, representing a fourth request to provide contact information, wherein the first audio data is received based at least in part on the fourth request.

13. The method of claim 9, wherein the audio data comprises first audio data, the user utterance comprises a first user utterance, the user account comprises a first user account, and the method further comprises: determining a time period during which the account identifier is to be associated with the device identifier; receiving second audio data during the time period, the second audio data representing a second user utterance requesting the voice-enabled device to perform the action; determining that a resource used to perform the action is not associated with the first user account; determining that the resource is available for use via a default user account associated with the voice-enabled device; as well as The default user account is selected based at least in part on a determination that the resource is not associated with the first user account and is associated with the default user account.

Images