Providing remote keyboard services

Abstract

To provide a remote keyboard service for a media device on a computing device without the need to launch an application or unlock the computing device.SOLUTION: The computing device receives a message indicating that a text input field on the other device is selected and presents a notification prompting the user to invoke a virtual keyboard on the computing device to provide text input to the selected text input field. User input selecting the notification is also received, and a virtual keyboard for providing text input to the other device is presented. Keyboard input may then be received from the user selecting a character and the character may be transmitted to the other device. The other device can then use the character as text input into the selected text input field.SELECTED DRAWING: None

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority to U.S. Provisional Patent Application No. 62 / 348,771, filed June 10, 2016, and U.S. Patent Application No. 15 / 272,010, filed September 21, 2016, the entire contents of which are incorporated herein by reference.

[0002] TECHNICAL FIELD This disclosure relates generally to coordinating services across different devices. [Background technology]

[0003] Streaming media devices are becoming increasingly popular. Streaming media devices allow users to browse, search, and watch media content on demand. However, the input mechanisms for these streaming media devices are often simple remote controls without keyboards, requiring users to search for and select individual text characters when entering a search query into a text input control (e.g., a text input box). This can make the process of providing text input to these streaming media devices cumbersome for users.

[0004] Some streaming media devices have corresponding applications that allow users to input text using a virtual keyboard on the user's handheld device (e.g., a smartphone, a tablet computer, etc.), but the user must install and launch the application before the virtual keyboard can be used to provide text input to the streaming media device. Summary of the Invention

[0005] The system described herein provides remote keyboard services for a media device on a separate computing device without requiring the launch of an application or the unlocking of the computing device. The computing device can receive a message indicating that a text entry field on the other device has been selected and receive a notification prompting a user to launch a virtual keyboard on the computing device to provide text entry into the selected text entry field. The computing device can receive user input selecting the notification and present a virtual keyboard to provide text entry to the other device. The computing device can receive keyboard input from the user selecting a character and send the character to the other device. The other device can then use the character as text entry into the selected text entry field.

[0006] Certain implementations provide at least the following advantages: A remote virtual keyboard can be provided by the user device without the need to launch a specific application. A remote virtual keyboard can be provided even when the user's device is locked. All user devices are made aware of the need for a keyboard, allowing the user to choose the user device that is most convenient for them.

[0007] The details of one or more implementations are set forth in the accompanying drawings and the description below. Other features, aspects, and potential advantages will become apparent from the description and drawings, and from the claims. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram of an example system 100 for providing remote keyboard services on a user device.

[0009] [Figure 2]FIG. 1 illustrates an exemplary graphical user interface for presenting remote keyboard notifications on a user device.

[0010] [Figure 3] FIG. 1 illustrates an exemplary graphical user interface for presenting a virtual keyboard on a user device.

[0011] [Figure 4] FIG. 1 illustrates an exemplary graphical user interface for presenting a virtual keyboard on a user device.

[0012] [Figure 5] FIG. 10 is a flow diagram of an example process for providing remote keyboard services on a user device.

[0013] [Figure 6] FIG. 10 is a flow diagram of an example process for requesting a remote keyboard service from a user device.

[0014] [Figure 7] FIG. 7 is a block diagram of an exemplary computing device capable of implementing the functions and processes of FIGS. 1-6.

[0015] Like reference symbols in the various drawings indicate like elements. DETAILED DESCRIPTION OF THE INVENTION

[0016] Overview

[0013] Exemplary methods, apparatuses, and computer programs for providing remote keyboard services on a user device are disclosed below. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of embodiments of the present invention. However, it will be apparent to those skilled in the art that embodiments of the present invention may be practiced without these specific details or with equivalent configurations. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring embodiments of the present invention.

[0017] 1 is a block diagram of an example system 100 for providing remote keyboard services on a user device. In some implementations, the system 100 can include user devices 102 (e.g., 102a, 102b, ... 102n). For example, the user device 102a can be a computing device such as a smartphone, a tablet computer, a television, a desktop computer, a laptop computer, a home device, an e-book device, a gaming device, a smartwatch, smart glasses, and / or other mobile or wearable devices, accessories and peripherals for these devices, or any combination thereof.

[0018] In some implementations, user devices 102 (e.g., user device 102a, user device 102b, ... user device 102n) can be configured with respective operating systems 104 that provide various services and functions. For example, operating systems 104 can be configured to receive and decode status advertisement messages (e.g., Bluetooth packets, Bluetooth LE packets, etc.) from different devices in the vicinity of user device 102. Status advertisement messages can include information describing the current state of other devices, including the availability of different services, data required for coordination of operations between different devices, and / or requests for information and / or services from other devices. For example, user device 102a can receive status advertisement messages from user devices 102b and / or 102n that describe the current state of these devices.

[0019] In some implementations, the user device 102 can receive status advertisement messages from the media device 106. For example, the media device 106 can be a computing device such as a streaming media player, a set-top box, a smart television, a laptop computer, or other computing device. A user of the user device 102 can interact with the media device 106 to browse and search for media items to play and / or other content to view. For example, the media device 106 can present a graphical user interface on a display (e.g., a television, a computer monitor, etc.) connected to the media device 106 that allows the user to browse, search, and select media items (e.g., movies, websites, images, photos, etc.) for the media device 106 to present on the display of the media device 106.

[0020] In some implementations, the media device 106 may request text input from the user. For example, the media device 106 may request text input from the user when searching for a media item. The media device 106 may request text input from the user when logging in to a subscription service that requires the user's login credentials. The operating system 108 of the media device 106 sometimes requests some text input. For example, the media device 106 may present prompts and text input controls on the display of the media device 106 for receiving network configuration input, login credentials, etc. Applications 110 (e.g., media content provider software applications, video applications, music applications, game applications, weather applications, etc.) sometimes present prompts and text input controls for receiving text input from the user. For example, the application 110 may provide a media item search function that allows the user to enter text that defines a search query.

[0021] In some implementations, a user can provide text input using remote control device 120. For example, media device 106 may be sold with or work with a remote control device 120 that includes a small number of directional and selection buttons for moving a cursor on media device 106 and selecting items on the display. Sometimes, remote control device 120 may have a touch interface, and remote control device 120 can receive touch input gestures (e.g., swipe up, swipe down, swipe left, swipe right, tap, etc.) for moving a cursor on media device 106 and selecting items on the display. When using remote control device 120 to provide text input to media device 106, a user typically must use remote control device 120 to move a cursor over an array of characters presented on a graphical user interface, selecting each character individually, until the appropriate string of characters is constructed. This process can be slow and cumbersome for the user and generally results in user frustration when the user has to repeatedly go through this hunt-and-peck process to find the media item that the user wants the media device 106 to play.

[0022] To address the above challenges associated with entering text using a remote control device 120, the user device 102 may be configured with a remote control application 104 (e.g., 104a, 104b) for remotely interacting with the media device 106. For example, the remote control application may control the media device 106 over a network 112. The network 112 may be, for example, a local area network, a wide area network, a local Wi-Fi network, an ad-hoc peer-to-peer network, etc. The network 112 may be implemented using Internet Protocol, Bluetooth, Bluetooth Low Energy, or any other networking protocol.

[0023] In some implementations, the remote control application 104 can be configured to provide functionality similar to that of the remote control device 120. For example, the remote control application 104 can present, on the display of the user device 102, directional controls for moving a cursor and selection controls for selecting items presented by the media device 106. However, the remote control application 104 can also provide a virtual keyboard that a user can use to provide text input to the media device 106. For example, the remote control application 104a can present a virtual keyboard on the touch-sensitive display of the user device 102a, and the user can type on the virtual keyboard to select characters and generate a string of characters corresponding to the requested text input. The user device 102a can then transmit the text input to the media device 106 over the network 112. The media device 105 can then use the text input to perform a search, configure the operating system 108, configure an application 110, or for any other purpose.

[0024] While providing text input to a virtual keyboard provided by the remote control application 104 is an improvement over entering text using the remote control device 120, it still places the burden on the user to unlock the user device 102a and launch the remote control application 104a before being able to access and use the virtual keyboard to provide text input to the media device 106. Thus, in some implementations, the operating system 104a of the user device 102a can provide a virtual keyboard service for providing text input to the media device 106. For example, the user device 102a may provide the virtual keyboard service without unlocking the user device 102a. The user device 102a can provide the virtual keyboard service without launching the remote control application 104a.

[0025] In some implementations, the media device 106 can broadcast a remote keyboard service request to connect to a remote keyboard service provided by the user device 102. For example, the remote keyboard service request can be (e.g., a portion of) a device state advertisement message indicating that a text input control has been selected on the media device 106. For example, when a user is interacting with the media device 106 (e.g., using the remote control device 120) and selects a text input field, the media device 106 can generate a state advertisement message indicating that the text input field has been selected. For example, if the application 110 is presenting a media item search graphical user interface, the user may select a text input control on the graphical user interface so that the user can provide a search query. In response to selecting the text input control, the application 110 can invoke an application programming interface (API) of the operating system 108 to request keyboard support for providing input to the selected text input control.

[0026] If the operating system 108 presents a configuration graphical user interface that requires text input, the user may select a text input control on the graphical user interface so that the user can provide the appropriate configuration string (e.g., network identifier, account credentials, etc.). In response to selecting the text input control, the operating system 108 may invoke an API of the operating system 108 to request keyboard support for providing input to the selected text input control. When the keyboard support API is invoked, the operating system 108 may generate a status advertisement message that includes data indicating that the text input control has been selected on the media device 106. For example, the status advertisement message may include an identifier of the media device 106, an identifier of the application 110, data indicating that the text input control has been selected (e.g., true / false), data describing the text input control or text input to be provided by the user, and / or text input that the user has already provided to the media device 106. The status advertisement message data may be encoded into and decoded from the status advertisement message using a status advertisement message schema such as that described in U.S. patent application Ser. No. 15 / 179,676, the entire contents of which are incorporated herein by reference.

[0027] In some implementations, the media device 106 may broadcast the generated status advertisement message. For example, the media device 106 may broadcast the generated status advertisement message using a wireless (e.g., peer-to-peer) technology such as Bluetooth, Bluetooth low energy, peer-to-peer Wi-Fi, or some other wireless technology. For example, the status advertisement message may be broadcast without selecting a specific recipient device. For example, any user device (e.g., user device 102) within range of a wireless signal transmitted by the media device 106 may receive and / or process the advertisement message generated by the media device 106. Alternatively, only user devices associated with the same user account (e.g., subscriber account, cloud service account, etc.) as the transmitting device may receive and decode the message from that transmitting device. Alternatively, the transmitting device and / or receiving device may not be associated with the same user and / or the same user account. For example, the user device 102 and the media device 106 may all be associated with different user accounts, or may not be associated with any user accounts at all. The user device 102 and the media device 106 may be different types of computing devices, from different manufacturers, with different configurations and / or different operating systems.

[0028] In some implementations, the user device 102 can receive a broadcasted state advertisement message. For example, the user devices 102 (e.g., user device 102a, user device 102b, ... user device 102n) within broadcast range of the media device 106 can receive the state advertisement message. In response to receiving the state advertisement message, the user device 102 can determine that a text input control on the media device 106 is currently selected based on the state data in the state advertisement message. When the user device 102 determines that a text input control is selected on the media device 106, the operating system 104 on the user device 102 presents a graphical notification on the respective device prompting the user to activate a virtual keyboard. For example, if a user has user device 102a (e.g., a smartphone), user device 102b (e.g., a laptop), and user device 102n (e.g., a tablet) all within broadcast range of media device 106, these devices can all determine that a text input control on media device 106 is currently selected based on the status advertisement message, and these user devices can all present a graphical notification prompting the user to activate a virtual keyboard on each of these user devices.

[0029] In some implementations, a user device 102 (e.g., user device 102a, user device 102b, ... user device 102n) can present a keyboard notification on the screen of the user device even when the screen of the user device is locked. For example, the user device 102 can present a remote keyboard notification when it receives a status advertisement message being broadcast from the media device 106.

[0030] Continuing with the above example, assuming the user has user device 102a within broadcast range of media device 106, user device 102a can present a remote keyboard notification on the display of user device 102a. After the user selects the notification on user device 102a (e.g., by touching, swiping, etc., the notification), user device 102a can establish a two-way communication session with media device 106 over communication network 112. For example, communication network 112 can include various networks, such as a data network, a wireless network, a telephone network, or any combination thereof. After user device 102a responds to the remote keyboard service request (e.g., embodied in a status advertisement message), media device 106 can broadcast another status advertisement message indicating that media device 106 is now connected to a remote keyboard. This allows other user devices (e.g., user device 102b, user device 102n, etc.) to dismiss (e.g., hide, remove from display) their corresponding keyboard notifications in response to receiving a status advertisement message indicating that media device 106 is connected to a remote keyboard.

[0031] In some implementations, the media device 106 can send information describing the requested text input to the user device 102a. For example, after establishing a two-way communication session over the network 112, the operating system 108 of the media device 106 can send information describing the text input provided by the user via a remote keyboard to the operating system 104a of the user device 102a. For example, the information may include a prompt for a text entry field (e.g., "Enter user name," "Enter password," "Enter search query," etc.). The information may include an identification or representation of the application 110 or the operating system 108. For example, the information may include an image, icon, or graphic representing the manufacturer of the application 110 or the operating system 108. The operating system 104a can then use this information to present text entry controls and / or a virtual keyboard on the display of the user device 102a, as described further below. Alternatively, this information can be obtained from a status advertisement message received by the user device 102a, as described above.

[0032] In some implementations, the operating system 104a can send virtual keyboard input to the media device 106. For example, the operating system 104a can detect when a user provides input that selects a character or other graphical element presented on the virtual keyboard. After detecting each input, the operating system 104a can send a message to the operating system 108 identifying the selected character or graphical element. For example, the operating system 104a can send each character or graphical element as they were selected so that the operating system 108 and / or the application 110 can respond appropriately to each keyboard input (e.g., by presenting search results, navigating between text input controls, etc.). For example, the operating system 108 can send the keyboard input to the application 110 so that the application 110 can respond to the keyboard input according to the programming of the application 110. In some implementations, the keyboard service provided by the user device 102a can be terminated after the user provides input indicating that the user has finished providing text input to the selected control. For example, when a user selects a graphical element on the virtual keyboard (e.g., an "enter" key, a "done" element, etc.), the keyboard service can be terminated and the virtual keyboard can be dismissed or hidden.

[0033] In some implementations, the operating system 104a may wait a certain period of time before terminating the remote virtual keyboard service. For example, even if the user has finished providing input to a text control presented by the media device 106, the operating system 104a may continue to present the virtual keyboard for a certain period of time (e.g., 5 seconds, 7 seconds, etc.) after the last input to allow the user to provide input to additional text controls presented by the media device 106. This may save the user the burden of having to re-launch the virtual keyboard if the media device 106 requires additional text input.

[0034] 2 illustrates an exemplary graphical user interface 200 for presenting a remote keyboard notification 202 on a user device 102a. For example, the notification 202 may be a graphical element presented by the operating system 104a on the display of the user device 102a in response to receiving a status advertisement message indicating that a text input control has been selected on the media device 106, as described above. The notification 202 may be presented on each user device 102 that receives the status advertisement message. The notification 202 may include, for example, a description 204 and / or an image 206 representing the media device 106 and / or the application 110. The notification 202 may include a prompt 208 that describes how to activate a virtual keyboard on the user device 102a. For example, a user may activate the virtual keyboard by selecting (e.g., long press, press touch, tap, swipe, etc.) the notification 202.

[0035] In some implementations, a user device 102 (e.g., user device 102a, user device 102b, ... user device 102n) can present a keyboard notification 202 on a GUI 200 of the user device 102a when the user device 102a is locked. For example, the GUI 200 can be a lock screen of the user device 102a. The GUI 200 can be a home screen of the user device 102a when the user device 102a is unlocked (e.g., a user has been authenticated as the user of the user device 102a).

[0036] 3 shows an exemplary graphical user interface 300 for presenting a virtual keyboard on a user device 102a. For example, the GUI 300 may be presented by the operating system 104a on the display of the user device 102a in response to a user selecting a notification 202, as described above with reference to the GUI 200 of FIG. 2. In some implementations, the user device 102a may present a graphical element 302 representing an application 110 and / or a media device 106. For example, the graphical element 302 may include a description 204 and / or an image 206 representing the media device 106. When the selected text input control is associated with an application 110, the graphical element 302 may include a graphical representation 304 (e.g., an image, an icon, a graphic, etc.) of the application 110.

[0037] In some implementations, the graphical element 302 may include a prompt 306 that describes a text input that a user must provide. For example, the prompt 306 may include instructions such as "enter your password" or "enter search term." In some implementations, the graphical element 302 may include prompt instructions 308 that describe how to invoke a virtual keyboard on the user device 102a. For example, the prompt 308 may include instructions to provide a tap gesture, a swipe gesture, a long press gesture, or a pressure touch gesture on the graphical element 302 to invoke the virtual keyboard.

[0038] In some implementations, the GUI 300 may include a virtual keyboard 310. For example, the virtual keyboard 310 may be presented on the GUI 300 in response to a user selecting a graphical element 302 to activate the virtual keyboard 310. For example, the virtual keyboard 310 may be presented in response to the user device 102a receiving an input specified by a prompt 208. The virtual keyboard may include, for example, a typical virtual keyboard layout for selecting characters. In some implementations, the GUI 300 may be presented without the graphical element 302. For example, the virtual keyboard 310 may be presented similar to the GUI 300 of FIG. 3, except without the graphical element 302.

[0039] In some implementations, the user device 102a can present the virtual keyboard 310 in response to detecting a movement. For example, the user device 102a can present the notification 202 on the GUI 200 of FIG. 2. When a user picks up the user device 102a to view the notification 202, the user device 102a can detect the movement of the user device 102a (e.g., using a built-in motion sensor, accelerometer, etc.), and the operating system 104a automatically presents the virtual keyboard 310 on the display of the user device 102a. For example, in response to detecting the movement of the user device 102a, the operating system 104a can present the GUI 300 of FIG. 3. This allows a user of the user device 102a to activate the virtual keyboard 310 simply by picking up the user device 102a.

[0040] In some implementations, the virtual keyboard 310 may include a text control 308. For example, when a user selects a character presented on the virtual keyboard 310, the selected character may be displayed on the text control 308. As described above, characters are also transmitted to the media device 106 when selected on the virtual keyboard 310, so that both the user device 102a and the media device 106 can present similar character strings based on the characters selected by the user. This allows the user to avoid having to divert their attention from the display of the user device 102a while entering text, but can still compare the text entered on the user device 102a with the text received by the media device 106 if necessary.

[0041] In some implementations, the virtual keyboard 310 may include graphical elements 314 and / or 312 for navigating between text controls (e.g., text entry boxes, text entry fields, etc.) presented on the display of the media device 106. For example, the operating system 108 or an application 110 on the media device 106 may request text entry into multiple text entry controls. The operating system 108 or application 110 may request, for example, a user's username and password. The user can select the graphical element 312 to select the next text entry control. The user can select the graphical element 314 to select the previous text entry control.

[0042] In some implementations, the virtual keyboard 310 may include a graphical element 316 that indicates that the user is finished providing text input. For example, the user may select the graphical element 316, indicating that the user is finished inputting into the current text control. When the user selects the graphical element 316, the user device 102a may send a message to the media device 106 causing the media device 106 to select another (e.g., next) text input control. When the user selects the graphical element 316, the user device 102a may send a message to the media device 106 to terminate the remote virtual keyboard service. Alternatively, the user may select the graphical element 318 to terminate the remote virtual keyboard service and dismiss the graphical element 302 and the virtual keyboard 310. However, in some implementations, as described above, the termination of the service may be delayed to allow for the selection of additional text input controls and the provision of additional text input on the media device 106.

[0043] 4 illustrates an exemplary graphical user interface 400 for presenting a virtual keyboard on a user device. For example, GUI 400 may be presented by operating system 104a on the display of user device 102a in response to a user selecting notification 202 of FIG. 2. For example, GUI 400 may be presented as an alternative to GUI 300 based on the functionality (or lack thereof) provided by various user devices 102. For example, if user device 102a is configured with press-touch input functionality, user device 102a may present GUI 300. If user device 102a is not configured with press-touch input functionality, user device 102a may present GUI 400.

[0044] In some implementations, the GUI 400 may include a graphical element 402 for presenting text input. For example, the graphical element 402 may be presented in response to a selection of the notification 202. The graphical element 402 may be presented adjacent to or abutting the notification 202. The graphical element 402 may include a prompt 406. For example, the prompt 406 may provide instructions to the user describing what type of information the user should provide in a selected text input control. The graphical element 402 may include a text input control 408. For example, the text input control 408 may present text input provided by the user using a virtual keyboard, as described above.

[0045] In some implementations, the GUI 400 may include a virtual keyboard 410. For example, the graphical element 410 may be presented on the GUI 400 in response to a user selecting the virtual keyboard 202. For example, the virtual keyboard 410 may be presented simultaneously or nearly simultaneously with the graphical element 402. For example, the virtual keyboard 310 may be presented in response to the user device 102a receiving an input specified by the prompt 208. The virtual keyboard 410 may include a typical virtual keyboard layout for selecting characters.

[0046] In some implementations, the virtual keyboard 410 may include graphical elements 412 and / or 414 for navigating between text controls (e.g., text entry boxes, text entry fields, etc.) presented on the display of the media device 106. For example, the operating system 108 or an application 110 on the media device 106 may request text entry into multiple text entry controls. The operating system 108 or application 110 may request, for example, a user's username and password. The user can select the graphical element 412 to select the next text entry control. The user can select the graphical element 414 to select the previous text entry control.

[0047] In some implementations, the virtual keyboard 410 may include a graphical element 416 that indicates that the user is finished providing text input. For example, the user may select the graphical element 416, indicating that the user is finished providing input to the current text control. When the user selects the graphical element 416, the user device 102a may send a message to the media device 106 causing the media device 106 to select another (e.g., next) text input control. When the user selects the graphical element 416, the user device 102a may send a message to the media device 106 to terminate the remote virtual keyboard service. Alternatively, the user may select the graphical element 418 to terminate the remote virtual keyboard service and dismiss the graphical element 402 and the virtual keyboard 410. However, in some implementations, the termination of the service may be delayed to allow for the selection of additional text input controls and the provision of additional text input on the media device 106, as described above. Example Process

[0048] 5 is a flow diagram of an example process 500 for providing remote keyboard services on a user device. More specifically, process 500 described herein is an approach for providing remote keyboard services from a user device to a media device (or any other type of computing device). A user device implementing process 500 can provide remote keyboard services and present a remote keyboard without launching a user application (e.g., remote control application 104a) on the user device. Process 500 can be efficiently handled by an operating system and makes the remote keyboard service available regardless of the applications installed on the user device and regardless of whether the user device is locked or unlocked.

[0049] In step 502, the user device 102a may receive a status advertisement message broadcast from the media device 106. For example, the status advertisement message may identify the media device and may include data describing the current state of the media device. According to implementations described herein, the status advertisement message may include data indicating that a text input control presented by the media device 106 is currently selected. Alternatively, the status advertisement message may include data indicating that the media device 106 requests connection to a remote keyboard.

[0050] In step 504, the user device 102a may determine that a text input control on the media device is currently selected. For example, based on the status advertisement message received in step 502, the user device 102a may determine that a text input control on the media device 106 is currently selected.

[0051] At step 506, user device 102a may present a remote keyboard notification. For example, in response to determining that a text input control on media device 106 is currently selected, user device 102a may present a graphical keyboard notification prompting the user to activate a virtual keyboard on user device 102a. For example, when a user has user device 102a (e.g., a smartphone), user device 102b (e.g., a laptop), and user device 102n (e.g., a tablet) all within broadcast range of media device 106, each of these devices may determine that a text input control on the media device is currently selected based on the status advertisement message and present a graphical notification prompting the user to activate a virtual keyboard on each of these user devices.

[0052] In step 508, the user device 102a may receive the user's notification selection. For example, the user device 102a may receive user input (e.g., a tap, swipe, long press, pressure touch, etc.) on the notification to select the notification. In some implementations, the first device selected by the user (e.g., the user may touch the graphical notification to activate a virtual keyboard on that device) presents a remote keyboard on the selected device for the user to enter text into a text field on the media device 106. The remaining devices dismiss the graphical notification from their screens but leave the notifications in their devices.

[0053] At step 510, the user device 102a may establish a connection with the media device 106. As discussed above, after a user selects a notification on the user device 102a by tapping or clicking on the notification, the user device 102a may establish a two-way communication session with the media device 106 over the communication network 112. For example, the user device 102a may establish a communication session with the media device 106 over the network 112 because a communication session over the network 112 may allow for the transmission of more data than a status advertisement message.

[0054] At step 512, the user device 102a may receive application data from the media device 106. For example, after establishing a communication session with the media device 106, the media device 106 may send the application data to the user device 102a. For example, the application data may include media device data such as a description 204 and / or an image 206 representing the media device 106. The application data may include a graphical representation 304 of the application 110 and / or a prompt 306 from the application 110 (e.g., "please enter your password"), as shown in FIG.

[0055] In step 514, the user device 102a may present the device data, application data, and / or keyboard on the display of the user device 102a. For example, after receiving the device data and / or application data, the user device 102a may present the device data and / or application data as shown in FIG.

[0056] In step 516, the user device 102a may receive user input to a virtual keyboard presented on the user device. For example, the user input may be the selection of a single character or other object (graphical element 312 and / or 314 to navigate between text controls) presented on the virtual keyboard.

[0057] At step 518, the user device 102a may send the user input to the media device 106. For example, the user device 102a may send the user input to the media device 106 via the communications network 112. For example, after step 518, the process 500 may return to step 516. Steps 516 and 518 may be repeated until the user has finished providing input to the virtual keyboard. This allows the media device 106 to respond to each keyboard input as if the keyboard were directly connected to or part of the media device 106.

[0058] 6 is a flow diagram of an example process 600 for requesting remote keyboard service from a user device. More specifically, the process 600 described herein is an approach for advertising the need for a remote keyboard, thereby allowing the user to select the most convenient device for activating the remote keyboard, as described above.

[0059] At step 602, the media device 106 may present an application 110 on a display associated with the media device 106. In some implementations, the media device 106 may be configured with a wide variety of media applications for presenting media items to a user. A user may launch the application 110 on the media device 106 using a remote control device. The application 110 may present a graphical user interface that allows the user to control the application 110. For example, the application 110 may present a graphical user interface that allows the user to browse, search, and view media items available through the application 110.

[0060] At step 604, media device 106 may receive an indication from application 110 that a text input control has been selected. For example, while interacting with application 110, application 110 may request the user to provide text input. Application 110 may request a username and password. Application 110 may request search parameters for performing a media search. In either case, application 110 may present the text input control, and the user may select the text input control presented by application 110 (e.g., using remote control device 120). Once the text input control is selected, application 110 may send a message (e.g., using the APIs described above) to operating system 108 on media device 106 indicating that the text input control has been selected and / or providing a description of the information requested by the text input control. In some implementations, the message may include an identifier for application 110 and / or other application metadata, such as graphical representation 304.

[0061] At step 606, the media device 106 may generate an advertisement message indicating that the text input control has been selected. In some implementations, the media device 106 may generate a state advertisement message that includes state information indicating that the text input control presented by the application 110 has been selected. For example, in response to receiving a message from the application 110 at step 604, the operating system 108 may generate a state advertisement message that includes data indicating that the text input control is currently selected on the media device 106.

[0062] In step 608, the media device 106 may broadcast a status advertisement message. In some implementations, after the media device 106 generates a status advertisement message indicating that the text input control is selected, the media device 106 may broadcast the status advertisement message. For example, the advertisement message may be broadcast without identifying a specific recipient device. This allows all user devices within the broadcast range of the media device 106 to receive the status advertisement message.

[0063] At step 610, the media device 106 may receive a message indicating that a user device will provide a remote keyboard to the media device. For example, after all user devices 102 within broadcast range of the media device 106 receive a media device state advertisement broadcast from the media device 106, the media device 106 may receive a message indicating that a user device 102a will provide a remote keyboard to the media device 106. For example, the message may be sent to the media device 106 via the network 112. The message may initiate a two-way communication session between the media device 106 and the user device 102a such that the media device 106 and the user device 102a can exchange data related to text input control.

[0064] In step 612, the media device 106 may send application data to the user device 102a. For example, the application data may include device data describing the media device (e.g., the media device 106), such as a description 204 and / or an image 206. The application data may include a graphical representation 304 of the application 110 and / or a prompt 306 from the application 110 (e.g., "please enter your password"), as shown in FIG. 3. The application data may be received from the application 110 by the operating system 108, as described above in step 604.

[0065] In step 614, the media device 106 may receive user keyboard input from the user device 102a. For example, when the user device 102a receives user input selecting characters and / or other objects on a remote keyboard (e.g., keyboard 310, keyboard 410) presented on the user device 102a, the user device 102a may transmit the user input (e.g., the selected characters or other objects) to the media device 106.

[0066] In step 616, the media device 106 may send the user's keyboard input to the application 110. For example, after receiving the user's keyboard input from the user device 102a, the operating system 108 may send the keyboard input to the application 110. The application 110 may then perform an action depending on the purpose of the selected text control. For example, if the currently selected text control is configured to receive username and / or password input, the application 110 may process the keyboard input to determine whether the entered username and / or password is correct for that user. If the currently selected text control is configured to receive search parameters, the application 110 may process the keyboard input by performing a search after each character entered by the user.

[0067] As media device 106 receives and processes each keyboard input (e.g., character selection, object selection, etc.) as it receives it, process 600 may return to step 614 to receive the next keyboard input and to step 616 to process the next keyboard input. Thus, process 600 may repeat steps 614 and 616 until the user has provided input to the selected text control.

[0068] Also, although process 600 is described for receiving text input for application 110, process 600 can similarly be implemented to receive text input for operating system 108. For example, process 600 can cause application 110 to intercept operating system 108 in order to obtain remote keyboard services for operating system 108 and receive text input for text input controls presented by operating system 108. Graphical User Interface

[0069] The above disclosure describes various graphical user interfaces (GUIs) for implementing various functions, processes, or workflows. These GUIs may be presented on a variety of electronic devices, including, but not limited to, laptop computers, desktop computers, computer terminals, television systems, tablet computers, e-readers, and smartphones. One or more of these electronic devices may include a touch-sensitive surface. The touch-sensitive surface can process multiple simultaneous input points, including processing data related to the pressure, degree, or location of each input point. Such processing can facilitate multi-finger gestures, such as pinching and swiping.

[0070] When this disclosure refers to "selecting" or "selecting" a user interface element in a GUI, these terms are understood to include clicking or "hovering" a mouse or other input device over the user interface element, or touching, tapping, or gesturing with one or more fingers or a stylus over the user interface element. A user interface element may be a virtual button, menu, selector, switch, slider, scrubber, knob, thumbnail, link, icon, radio button, checkbox, and any other mechanism for receiving input from or providing feedback to a user. privacy

[0071] This disclosure recognizes that the use of such personal information data in the present technology can be used to the benefit of the user. For example, personal information data may be used to deliver targeted content that is more likely to interest the user. Thus, use of such personal information data allows for intentional control of the content that is delivered. Furthermore, other uses of personal information data that benefit the user are also contemplated by this disclosure.

[0072] This disclosure further contemplates that entities responsible for the collection, analysis, disclosure, transmission, storage, or other uses of such personal information data will comply with well-defined privacy policies and / or privacy practices. Specifically, such entities should implement and consistently use privacy policies and practices that are generally recognized as meeting or exceeding industry or government requirements for maintaining the strict confidentiality of personal information data. For example, personal information from users should be collected for the entity's lawful and legitimate uses and should not be shared or sold except for those lawful uses. Furthermore, such collection should occur only after the user's informed consent is obtained. Furthermore, such entities will take all necessary measures to protect and secure access to such personal information data and to ensure that others with access to that personal information data comply with their privacy policies and procedures. Furthermore, such entities may submit themselves to third-party assessment to attest to their compliance with widely accepted privacy policies and practices.

[0073] Notwithstanding the foregoing, the present disclosure also contemplates embodiments in which a user selectively blocks use of or access to personal information data. That is, the present disclosure contemplates that hardware and / or software elements may be provided to prevent or block access to such personal information data. For example, in the case of an advertising distribution service, the present technology may be configured to allow a user to "opt in" or "opt out" of participating in the collection of personal information data during registration for the service. In another example, a user may choose not to provide location information to a targeted content distribution service. In yet another example, a user may choose not to provide precise location information but allow the transfer of location zone information. Exemplary System Architecture

[0074] Figure 7 is a block diagram of an exemplary computing device 700 capable of implementing the functions and processes shown in Figures 1-6. The computing device 700 may include a memory interface 702, one or more data processors, image processors, and / or central processing units 704, and a peripheral interface 706. The memory interface 702, the one or more processors 704, and / or the peripheral interface 706 may be separate components or may be integrated within one or more integrated circuits. The various components of the computing device 700 may be coupled to one or more communication buses or signal lines.

[0075] Sensors, devices, and subsystems can be coupled to the peripherals interface 706 to facilitate multiple functionality. For example, a motion sensor 710, a light sensor 712, and a proximity sensor 714 can be coupled to the peripherals interface 706 to facilitate orientation, lighting, and proximity functions. Other sensors 716, such as a global navigation satellite system (GNSS) (e.g., a GPS receiver), a temperature sensor, a biometric sensor, a magnetometer, or other sensing devices, can also be coupled to the peripherals interface 706 to facilitate related functionality.

[0076] A camera subsystem 720 and optical sensor 722 (e.g., a charge-coupled device (CCD) or complementary metal-oxide semiconductor (CMOS) optical sensor) may be utilized to facilitate camera functions such as recording photographs and video clips. The camera subsystem 720 and optical sensor 722 may be used to collect an image of a user, which may be used to authenticate the user, for example, through facial recognition analysis.

[0077] Communication functions may be facilitated through one or more wireless communication subsystems 724, which may include radio receivers and transmitters and / or optical (e.g., infrared) receivers and transmitters. The specific design and implementation of the communication subsystems 724 may depend on the communication network(s) over which the computing device 700 is intended to operate. For example, the computing device 700 may include a communication subsystem 724 designed to operate over a GSM network, a GPRS network, an EDGE network, a Wi-Fi or WiMax network, and a Bluetooth network. In particular, the wireless communication subsystem 724 may include a hosting protocol that enables the device 100 to be configured as a base station for other wireless devices.

[0078] The audio subsystem 726 can be coupled to a speaker 728 and a microphone 730 to facilitate voice-enabled functions such as speaker recognition, voice replication, digital recording, and telephony functions. The audio subsystem 726 can be configured to facilitate processing of voice commands, voiceprint recognition, and voice authentication, for example.

[0079] The I / O subsystem 740 can include a touch surface controller 742 and / or other input controller(s) 744. The touch surface controller 742 can be coupled to a touch surface 746. The touch surface 746 and touch surface controller 742 can detect contact and movement, or the cessation of contact and movement, using, for example, any of a number of touch sensitivity technologies, including, but not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch surface 746.

[0080] Other input controller(s) 744 may be coupled to other input / control devices 748, such as one or more buttons, rocker switches, thumbwheels, infrared ports, USB ports, and / or pointer devices such as styluses. The one or more buttons (not shown) may include up / down buttons for volume control of the speaker 728 and / or microphone 730.

[0081] In one implementation, pressing the button for a first duration can unlock the touch surface 746, and pressing the button for a second duration, longer than the first duration, can turn power on / off to the computing device 700. Pressing the button for a third duration can activate voice control, i.e., voice commands, a module that causes the device to execute commands spoken by a user into the microphone 730. A user can customize the functionality of one or more buttons. The touch surface 746 can also be used to implement, for example, virtual or soft buttons and / or a keyboard.

[0082] In some implementations, the computing device 700 can present recorded audio and / or video files, such as MP3 files, AAC files, and MPEG files. In some implementations, the computing device 700 can include the functionality of an MP3 player, such as an iPod®. Thus, the computing device 700 can include a 36-pin connector compatible with an iPod. Other input / output and control devices can also be used.

[0083] The memory interface 702 can be coupled to memory 750. The memory 750 can include high-speed random access memory and / or non-volatile memory, such as one or more magnetic disk storage devices, one or more optical storage devices, and / or flash memory (e.g., NAND, NOR). The memory 750 can store an operating system 752, such as Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks.

[0084] Operating system 752 may include instructions for handling basic system services and for performing hardware-dependent tasks. In some implementations, operating system 752 may be a kernel (e.g., a UNIX kernel). In some implementations, operating system 752 may include instructions for performing voice authentication. For example, operating system 752 may implement the remote keyboard functionality described with reference to FIGS. 1-6.

[0085] Memory 750 may also store communications instructions 754 for facilitating communication with one or more additional devices, one or more computers, and / or one or more servers. Memory 750 may include graphical user interface instructions 756 for facilitating graphical user interface operations, sensor processing instructions 758 for facilitating sensor-related operations and functions, telephony instructions 760 for facilitating telephony-related operations and functions, electronic messaging instructions 762 for facilitating electronic messaging-related operations and functions, web browsing instructions 764 for facilitating web browsing-related operations and functions, media processing instructions 766 for facilitating media processing-related operations and functions, GNSS / navigation instructions 768 for facilitating GNSS- and navigation-related processes and instructions, and / or camera instructions 770 for facilitating camera-related operations and functions.

[0086] The memory 750 may store other software instructions 772 for facilitating other processes and functions, such as the remote keyboard processes and functions described with reference to FIGS.

[0087] The memory 750 may also store other software instructions 774, such as web video instructions for facilitating web video related processes and functions, and / or web shopping instructions for facilitating web shopping related processes and functions. In some implementations, the media processing instructions 766 may be divided into audio processing instructions for facilitating audio processing related processes and functions, and video processing instructions for facilitating video processing related processes and functions, respectively.

[0088] Each of the above-identified instructions and applications may correspond to a set of instructions for performing one or more of the functions described above. These instructions need not be implemented as separate software programs, procedures, or modules. Memory 750 may include additional or fewer instructions. Furthermore, various functions of computing device 700 may be implemented in hardware and / or software, including one or more signal processing and / or application specific integrated circuits.

Claims

1. 1. A method comprising: receiving, by the user device, a broadcast of a status advertisement message from the media device; determining, based on the status advertisement message, that a text input control on the media device is currently selected by the user device; In response to determining that the text input control on the media device is currently selected, presenting, by the user device, a first graphical notification prompting a user to activate a virtual keyboard on the user device; receiving, by the user device, a first user input that activates the virtual keyboard on the user device; presenting the virtual keyboard on a display of the user device to provide input to the text input control selected on the media device; The method of claim 1, wherein the virtual keyboard is provided by a software system of the first user device that is separate from any user applications on the first user device.

2. establishing, by the user device, a connection with the media device in response to a first selection of the first graphical notification on the user device; receiving, by the user device, a second user input selecting one or more characters into the virtual keyboard; transmitting, by the first user device, the second user input to the media device; 10. The method of claim 1, further comprising:

3. The method of claim 1 , wherein the user device is locked when the virtual keyboard is presented on the display of the user device.

4. receiving, by the user device, information describing a purpose of the currently selected control; presenting the control information simultaneously with the presentation of the virtual keyboard; 10. The method of claim 1, further comprising:

5. 2. The method of claim 1, wherein the first graphical notification includes application data from the media device and instructions to activate the virtual keyboard.

6. receiving a user input to dismiss the virtual keyboard; delaying the release of the virtual keyboard for a period of time; 10. The method of claim 1, further comprising:

7. 1. A system comprising: one or more processors; and a non-transitory computer-readable medium containing one or more sequences of instructions, which when executed by one or more processors: receiving, by the user device, a broadcast of a status advertisement message from the media device; determining, by the user device based on the status advertisement message, that a text input control on the media device is currently selected; In response to determining that the text input control on the media device is currently selected, causing the user device to present a first graphical notification prompting a user to activate a virtual keyboard on the user device; receiving, by the user device, a first user input that activates the virtual keyboard on the user device; presenting the virtual keyboard on a display of the user device to provide input to the text input control selected on the media device; The virtual keyboard is provided by a software system of the first user device that is separate from any user applications on the first user device. A system characterized by:

8. The instruction: establishing, by the user device, a connection with the media device in response to a first selection of the first graphical notification on the user device; receiving, by the user device, a second user input selecting one or more characters into the virtual keyboard; causing the first user device to transmit the second user input to the media device; 8. The system of claim 7.

9. The system of claim 7, wherein the user device is locked when the virtual keyboard is presented on the display of the user device.

10. The instruction: receiving, by the user device, information describing a purpose of the currently selected control; presenting the control information simultaneously with the presentation of the virtual keyboard; 8. The system of claim 7.

11. The first graphical notification includes application data from the media device and instructions to activate the virtual keyboard.

8. The system of claim 7.

12. The instruction: receiving a user input to dismiss the virtual keyboard; delaying the release of the virtual keyboard for a period of time; 8. The system of claim 7.

13. 1. A method comprising: presenting, by the media device, on a display on the media device, a currently selected text input control for the application; receiving, by the media device, an indication from the application that the text input control is currently selected; generating, by the media device, a first status advertisement message indicating that the text input control is currently selected; broadcasting, by the media device, the first status advertisement message; receiving, by the media device, a response to the first status advertisement message indicating that the user device provides a virtual keyboard; A method comprising:

14. broadcasting a second status advertisement message indicating that the media device is connected to a remote virtual keyboard in response to receiving a response to the first status advertisement message indicating that the user device provides a virtual keyboard; 14. The method of claim 13 further comprising:

15. responsive to receiving a response to the first status advertisement message indicating that the user device provides a virtual keyboard, establishing a two-way communication session with the user device; transmitting metadata corresponding to the application and the text input control to the user device; 14. The method of claim 13 further comprising:

16. receiving, by the media device, text input from the user device; wherein the text input corresponds to user input into a virtual keyboard presented on the user device. performing an action based on the text input; and 14. The method of claim 13 further comprising:

17. 1. A system comprising: one or more processors; and a non-transitory computer-readable medium containing one or more sequences of instructions, which when executed by the one or more processors: causing the media device to present on a display of said media device a currently selected text input control for the application; receiving, by the media device, an indication from the application that the text input control is currently selected; generating, by the media device, a first status advertising message indicating that the text input control is currently selected; causing the media device to broadcast the first status advertisement message; receiving, by the media device, a response to the first status advertisement message indicating that the user device provides a virtual keyboard; A system characterized by:

18. The instruction: In response to receiving a response to the first status advertisement message indicating that the user device provides a virtual keyboard, broadcasting a second status advertisement message indicating that the media device is connected to a remote virtual keyboard.

20. The system of claim 17.

19. The instruction: establishing a two-way communication session with the user device in response to receiving a response to the first status advertisement message indicating that the user device provides a virtual keyboard; transmitting metadata corresponding to the application and the text input control to the user device; 20. The system of claim 17.

20. The instruction: receiving, by the media device, text input from the user device; wherein the text input corresponds to user input into a virtual keyboard presented on the user device. performing an action based on the text input; 20. The system of claim 17.

Images