Remote keyboard service
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- APPLE INC
- Filing Date
- 2025-07-11
- Publication Date
- 2026-08-05
Smart Images
Figure 0007901220000001 
Figure 0007901220000002 
Figure 0007901220000003
Abstract
Description
Technical Field
[0001] (Cross - Reference to Related Applications) This application claims priority to U.S. Provisional Patent Application No. 62 / 348,771, filed Jun. 10, 2016, and U.S. Patent Application 15 / 272,010, filed Sep. 21, 2016, the entire contents of which are incorporated herein by reference.
[0002] This disclosure generally relates to coordinating services across different devices.
Background Art
[0003] Streaming media devices are becoming increasingly popular. Streaming media devices enable users to browse, search for, and view media content on demand. However, the input mechanisms for these streaming media devices are often simple remote controls without a keyboard that require the user to search for and select individual text characters when entering a search query into a text input control (e.g., a text input box). As a result, the process of providing text input to these streaming media devices can be cumbersome for users.
[0004] Some streaming media devices have corresponding applications that allow users to perform text input using a virtual keyboard on the user's handheld device (e.g., a smartphone, a tablet computer, etc.). However, the user must install and launch the application before the virtual keyboard can be used to provide text input to the streaming media device.
Summary of the Invention
[0005] The system described herein provides a remote keyboard service for a media device on a separate computing device without requiring the launch of an application or the unlocking of the computing device. The computing device can receive a message indicating that a text input field on the other device has been selected, and can receive a notification prompting the user to launch a virtual keyboard on the computing device to provide text input to the selected text input field. The computing device can receive user input to select the notification and present a virtual keyboard to provide text input to the other device. The computing device can receive keyboard input from the user to select characters and send those characters to the other device. The other device can then use those characters as text input to the selected text input field.
[0006] A particular implementation offers at least the following advantages: It allows a remote virtual keyboard to be provided by the user device without requiring the launch of a specific application. It allows a remote virtual keyboard to be provided even when the user's device is locked. Since all user devices are made aware of the need for a keyboard, users can choose the user device that is most convenient for them.
[0007] Details of one or more implementations are described in the accompanying drawings and the description below. Other features, embodiments, and potential advantages will become apparent from the description and drawings, as well as from the claims. [Brief explanation of the drawing]
[0008] [Figure 1] This is a block diagram of an exemplary system 100 for providing remote keyboard services on a user device.
[0009] [Figure 2]This figure shows an exemplary graphical user interface for presenting remote keyboard notifications on a user device.
[0010] [Figure 3] This figure shows an exemplary graphical user interface for presenting a virtual keyboard on a user device.
[0011] [Figure 4] This figure shows an exemplary graphical user interface for presenting a virtual keyboard on a user device.
[0012] [Figure 5] This is a flowchart illustrating an exemplary process for providing a remote keyboard service on a user device.
[0013] [Figure 6] This is a flowchart illustrating an exemplary process for requesting remote keyboard services from a user device.
[0014] [Figure 7] These are block diagrams of exemplary computing devices capable of implementing the functions and processes shown in Figures 1 to 6.
[0015] Similar reference symbols in each drawing indicate similar elements. [Modes for carrying out the invention]
[0016] Overview Examples of methods, apparatus, and computer programs for providing remote keyboard services on a user device are disclosed below. In the following description, many specific details are described for the purpose of providing an explanation and a complete understanding of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention can be practiced without these specific details or with equivalent configurations. In other cases, well-known structures and devices are shown in block diagram form to avoid unnecessarily obscuring embodiments of the invention.
[0017] Figure 1 is a block diagram of an exemplary system 100 for providing a remote keyboard service on a user device. In some implementations, system 100 may include user devices 102 (e.g., 102a, 102b, ..., 102n). For example, user device 102a may be a computing device such as a smartphone, tablet computer, television, desktop computer, laptop computer, home device, e-reader, game device, smartwatch, smart glasses and / or other mobile or wearable device, accessories and peripherals for these devices, or any combination thereof.
[0018] In some implementations, user devices 102 (e.g., user devices 102a, 102b, ..., 102n) can be configured with various operating systems 104 that provide different services and functions. For example, the operating system 104 can be configured to receive and decode state advertisement messages (e.g., Bluetooth® packets, Bluetooth LE packets, etc.) from different devices in the vicinity of user device 102. State advertisement messages may include information describing the current state of other devices, including the availability of different services, data necessary for processing coordination between different devices, and / or requests for information and / or services to other devices. For example, user device 102a can receive state advertisement messages from user devices 102b and / or 102n that describe the current state of these devices.
[0019] In some implementations, user device 102 can receive status advertisement messages from media device 106. For example, media device 106 may be a computing device such as a streaming media player, set-top box, smart TV, laptop computer, or other computing device. The user of user device 102 can interact with media device 106 to browse and search for media items to play and / or other content to watch. For example, media device 106 may present a graphical user interface on a display connected to media device 106 (e.g., a television, computer monitor, etc.) that allows the user to browse, search, and select media items (e.g., movies, websites, images, photographs, etc.) that media device 106 presents on the display of media device 106.
[0020] In some implementations, the media device 106 may prompt the user for text input. For example, the media device 106 may prompt the user for text input when searching for media items. The media device 106 may prompt the user for text input when logging into a subscription service that requires the user's login credentials. The operating system 108 of the media device 106 may occasionally prompt for some text input. For example, the media device 106 may display prompts and text input controls on its display for receiving network configuration input, login credentials, etc. An application 110 (e.g., a media content provider software application, a video application, a music application, a game application, a weather application, etc.) may occasionally display prompts and text input controls for receiving text input from the user. For example, application 110 may provide a media item search function that allows the user to enter text defining a search query.
[0021] In some implementations, a user may provide text input using a remote control device 120. For example, media device 106 may be sold with or operate with a remote control device 120 that includes a few directional buttons and a select button for moving a cursor on the media device 106 and selecting items on a display. Sometimes, the remote control device 120 may have a touch interface, and the remote control device 120 may receive touch input gestures (e.g., swipe up, swipe down, swipe left, swipe right, tap, etc.) for moving a cursor on the media device 106 and selecting items on a display. When providing text input to the media device 106 using the remote control device 120, typically, the user must use the remote control device 120 to move a cursor over an array of characters presented on a graphical user interface and select each character one by one until an appropriate string is constructed. This process is slow and can be cumbersome for the user, and generally results in frustrating the user when the user must repeatedly perform this hunt-and-peck process to find a media item that the user desires to play on the media device 106.
[0022] To solve the above problems related to text input using the remote control device 120, the user device 102 can be configured with a remote control application 104 (e.g., 104a, 104b) for remote interaction with the media device 106. For example, the remote control application can control the media device 106 via the network 112. The network 112 can be, for example, a local area network, a wide area network, a local Wi-Fi (registered trademark) network, an ad hoc peer-to-peer network, etc. The network 112 can be implemented using the Internet protocol, Bluetooth, Bluetooth Low Energy, or any other networking protocol.
[0023] In some implementations, the remote control application 104 can be configured to provide functionality similar to that of the remote control device 120. For example, the remote control application 104 can present on the display of the user device 102, a direction control for moving the cursor and a selection control for selecting an item presented by the media device 106. However, the remote control application 104 can also provide a virtual keyboard that the user can use to provide text input to the media device 106. For example, the remote control application 104a can present a virtual keyboard on the touch-sensitive display of the user device 102a, and the user can type on the virtual keyboard, select characters, and generate a string corresponding to the required text input. Then, the user device 102a can transmit the text input to the media device 106 via the network 112. Then, the media device 105 can use the text input for performing a search, configuring the operating system 108, configuring the application 110, or any other purpose.
[0024] Providing text input to the virtual keyboard provided by the remote control application 104 is an improvement over using the remote control device 120 to input text, but the user is still burdened with unlocking the user device 102a and starting the remote control application 104a before they can provide text input to the media device 106 by accessing and using the virtual keyboard. As a result, in some implementations, the operating system 104a of the user device 102a can provide a virtual keyboard service to provide text input to the media device 106. For example, the user device 102a may provide the virtual keyboard service without unlocking the user device 102a. The user device 102a can provide the virtual keyboard service without starting the remote control application 104a.
[0025] In some implementations, the media device 106 can broadcast a remote keyboard service request to connect to a remote keyboard service provided by the user device 102. For example, the remote keyboard service request may be a device status advertisement message (e.g., part thereof) indicating that a text input control has been selected on the media device 106. For example, when a user is interacting with the media device 106 (e.g., using a remote control device 120) and selects a text input field, the media device 106 can generate a status advertisement message indicating that the text input field has been selected. For example, if application 110 presents a media item search graphical user interface, the user may select a text input control on the graphical user interface so that the user can provide a search query. In response to the selection of the text input control, application 110 can invoke the application programming interface (API) of the operating system 108 to request keyboard support to provide input to the selected text input control.
[0026] If the operating system 108 presents a configuration graphical user interface that requires text input, the user may select a text input control on the graphical user interface so that the user can provide appropriate configuration strings (e.g., network identifier, account credentials, etc.). Depending on the selection of a text input control, the operating system 108 may invoke its APIs to request keyboard support to provide input to the selected text input control. When the keyboard support API is invoked, the operating system 108 may generate a status advertisement message containing data indicating that a text input control has been selected on the media device 106. For example, the status advertisement message may include an identifier for the media device 106, an identifier for the application 110, data indicating that a text input control has been selected (e.g., true / false), data describing a text input control or text input provided by the user, and / or text input already provided by the user to the media device 106. The status advertisement message data can be encoded into and decoded from a status advertisement message using a status advertisement message schema such as that described in U.S. Patent Application No. 15 / 179,676, which is incorporated entirely herein by reference.
[0027] In some implementations, the media device 106 can broadcast the generated status advertisement message. For example, the media device 106 can broadcast the generated status advertisement message using a wireless (e.g., peer-to-peer) technology such as Bluetooth, Bluetooth Low Energy, peer-to-peer Wi-Fi, or some other wireless technology. For example, the status advertisement message can be broadcast without selecting a specific recipient device. For example, any user device (e.g., user device 102) within range of the wireless signal transmitted by the media device 106 can receive and / or process the advertisement message generated by the media device 106. Alternatively, only user devices associated with the same user account (e.g., subscriber account, cloud service account, etc.) as the transmitting device can receive and decode messages from that transmitting device. Alternatively, the transmitting device and / or receiving devices do not have to be associated with the same user and / or the same user account. For example, user device 102 and media device 106 may all be associated with different user accounts, or may not be associated with any user account at all. The user device 102 and the media device 106 may be different types of computing devices from different manufacturers, with different configurations and / or different operating systems.
[0028] In some implementations, user devices 102 can receive broadcasted state advertisement messages. For example, user devices 102 within the broadcast range of media device 106 (e.g., user devices 102a, 102b, ..., user device 102n) can receive state advertisement messages. Upon receiving a state advertisement message, user device 102 can determine, based on the state data in the message, that text input control on media device 106 is currently selected. When user device 102 determines that text input control is selected on media device 106, the operating system 104 on user device 102 presents a graphical notification on each device prompting the user to activate the virtual keyboard. For example, if a user has user device 102a (e.g., a smartphone), user device 102b (e.g., a laptop), and user device 102n (e.g., a tablet) all within the broadcast range of media device 106, all of these devices can determine, based on a status advertisement message, that text input control on media device 106 is currently selected, and all of these user devices can display a graphical notification prompting the user to activate a virtual keyboard on each of these user devices.
[0029] In some implementations, user devices 102 (e.g., user devices 102a, 102b, ..., 102n) can display keyboard notifications on their screens even when the user device screen is locked. For example, user device 102 can display a remote keyboard notification when it receives a status advertisement message broadcast from media device 106.
[0030] Continuing with the above example, assuming that the user has user device 102a within the broadcast range of media device 106, user device 102a can present a remote keyboard notification on its display. After the user selects the notification on user device 102a (e.g., by touching or swiping the notification), user device 102a can establish a bidirectional communication session with media device 106 via communication network 112. For example, communication network 112 may include various networks such as a data network, a wireless network, a telephone network, or any combination thereof. After user device 102a responds to a remote keyboard service request (e.g., embodied in a status advertisement message), media device 106 can broadcast another status advertisement message indicating that media device 106 is connected to the remote keyboard. This allows other user devices (e.g., user device 102b, user device 102n, etc.) to dismiss their corresponding keyboard notifications (e.g., hide them, remove them from the display) upon receiving a status advertisement message indicating that media device 106 is connected to a remote keyboard.
[0031] In some implementations, the media device 106 can send information describing the requested text input to the user device 102a. For example, after establishing a bidirectional communication session over the network 112, the operating system 108 of the media device 106 can send information describing the text input provided by the user via a remote keyboard to the operating system 104a of the user device 102a. For example, the information may include prompts for text input fields (e.g., "Enter user name", "Enter password", "Enter search query", etc.). The information may include identification information or representation of the application 110 or the operating system 108. For example, the information may include an image, icon, or graphic representing the manufacturer of the application 110 or the operating system 108. The operating system 104a can then use this information to present text input controls and / or a virtual keyboard on the display of the user device 102a, as further described below. Alternatively, this information can be obtained from a status advertisement message received by the user device 102a, as described above.
[0032] In some implementations, the operating system 104a can send virtual keyboard input to the media device 106. For example, the operating system 104a can detect when the user provides input to select a character or other graphical element presented on the virtual keyboard. After detecting each input, the operating system 104a can send a message to the operating system 108 identifying the selected character or graphical element. For example, the operating system 104a can send each character or graphical element as selected so that the operating system 108 and / or the application 110 can respond appropriately to each keyboard input (e.g., by presenting search results, navigating between text input controls, etc.). For example, the operating system 108 can send keyboard input to the application 110 so that the application 110 can respond to keyboard input according to the programming of the application 110. In some implementations, the keyboard service provided by the user device 102a can be terminated after the user provides input indicating that they have finished providing text input to the selected control. For example, when a user selects a graphical element on the virtual keyboard (such as the "Enter" key or the "Done" element), the keyboard service can be terminated, and the virtual keyboard can be dismissed or hidden.
[0033] In some implementations, the operating system 104a may wait for a certain period of time before terminating the remote virtual keyboard service. For example, even after the user has finished providing input to the text controls presented by the media device 106, the operating system 104a may continue to present the virtual keyboard for a certain period of time (e.g., 5 seconds, 7 seconds, etc.) after the last input, allowing the user to provide additional input to the text controls presented by the media device 106. This eliminates the burden on the user of having to restart the virtual keyboard if the media device 106 requests additional text input.
[0034] Figure 2 shows an exemplary graphical user interface 200 for presenting a remote keyboard notification 202 on a user device 102a. For example, the notification 202 may be a graphical element presented by the operating system 104a on the display of the user device 102a in response to receiving a status advertisement message indicating that text input control is selected on the media device 106, as described above. The notification 202 may be presented on each user device 102 that receives the status advertisement message. The notification 202 may include, for example, a description 204 and / or an image 206 representing the media device 106 and / or application 110. The notification 202 may include a prompt 208 describing how to activate a virtual keyboard on the user device 102a. For example, the user may activate the virtual keyboard by selecting (e.g., long press, pressure touch, tap, swipe, etc.) the notification 202.
[0035] In some implementations, user devices 102 (e.g., user device 102a, user device 102b, ... user device 102n) can display a keyboard notification 202 on the GUI 200 of user device 102a when user device 102a is locked. For example, GUI 200 can be the lock screen of user device 102a. GUI 200 can also be the home screen of user device 102a when user device 102a is unlocked (e.g., the user has been authenticated as the user of user device 102a).
[0036] Figure 3 shows an exemplary graphical user interface 300 for presenting a virtual keyboard on the user device 102a. For example, the GUI 300 may be presented by the operating system 104a on the display of the user device 102a in response to the user selecting a notification 202, as described above with reference to the GUI 200 in Figure 2. In some implementations, the user device 102a may present graphical elements 302 representing an application 110 and / or a media device 106. For example, the graphical elements 302 may include a description 204 and / or an image 206 representing the media device 106. When the selected text input control is associated with the application 110, the graphical elements 302 may include a graphical representation 304 of the application 110 (e.g., an image, icon, graphic, etc.).
[0037] In some implementations, the graphical element 302 may include a prompt 306 that describes text input that the user must provide. For example, the prompt 306 may include instructions such as "enter your password" or "enter search term". In some implementations, the graphical element 302 may include a prompt instruction 308 that describes how to activate a virtual keyboard on the user device 102a. For example, the prompt 308 may include instructions to provide a tap gesture, swipe gesture, long press gesture, or pressure touch gesture to the graphical element 302 in order to activate the virtual keyboard.
[0038] In some implementations, the GUI 300 may include a virtual keyboard 310. For example, the virtual keyboard 310 may be presented on the GUI 300 in response to the user selecting a graphical element 302 to activate the virtual keyboard 310. For example, the virtual keyboard 310 may be presented in response to the user device 102a receiving input specified by prompt 208. The virtual keyboard may include, for example, a typical virtual keyboard layout for selecting characters. In some implementations, the GUI 300 may be presented without the graphical element 302. For example, the virtual keyboard 310 may be presented similarly to the GUI 300 in Figure 3, except that the graphical element 302 is absent.
[0039] In some implementations, the user device 102a can present a virtual keyboard 310 in response to motion detection. For example, the user device 102a can present a notification 202 on the GUI 200 shown in Figure 2. When the user picks up the user device 102a to view the notification 202, the user device 102a can detect its movement (e.g., using a built-in motion sensor, accelerometer, etc.), and the operating system 104a automatically presents the virtual keyboard 310 on the user device 102a's display. For example, in response to detecting the movement of the user device 102a, the operating system 104a can present the GUI 300 shown in Figure 3. This allows the user of the user device 102a to activate the virtual keyboard 310 simply by picking up the user device 102a.
[0040] In some implementations, the virtual keyboard 310 may include a text control 308. For example, if the user selects a character displayed on the virtual keyboard 310, the selected character may be displayed on the text control 308. As described above, characters are also sent to the media device 106 when selected on the virtual keyboard 310, so both the user device 102a and the media device 106 can display the same string based on the character selected by the user. This allows the user to avoid diverting their attention from the display of the user device 102a while typing text, but to compare the text entered on the user device 102a with the text received by the media device 106 if necessary.
[0041] In some implementations, the virtual keyboard 310 may include graphical elements 314 and / or 312 for navigating between text controls (e.g., text input boxes, text input fields, etc.) presented on the display of the media device 106. For example, the operating system 108 or application 110 on the media device 106 may request text input to multiple text input controls. The operating system 108 or application 110 may, for example, request the user's username and password. The user can select graphical element 312 to select the next text input control. The user can select graphical element 314 to select the previous text input control.
[0042] In some implementations, the virtual keyboard 310 may include a graphical element 316 that indicates when the user has finished providing text input. For example, the user may select a graphical element 316 that indicates when the user has finished inputting to the current text control. When the user selects a graphical element 316, the user device 102a may send a message to the media device 106 prompting the media device 106 to select another (e.g., the next) text input control. When the user selects a graphical element 316, the user device 102a may send a message to the media device 106 to terminate the remote virtual keyboard service. Alternatively, the user may select a graphical element 318 to terminate the remote virtual keyboard service and release the graphical element 302 and the virtual keyboard 310. However, in some implementations, as described above, the termination of the service may be delayed to allow for the selection of additional text input controls and the provision of additional text input on the media device 106.
[0043] Figure 4 shows an exemplary graphical user interface 400 for presenting a virtual keyboard on a user device. For example, the GUI 400 may be presented by the operating system 104a on the display of the user device 102a in response to the user selecting notification 202 in Figure 2. For example, the GUI 400 may be presented as an alternative to the GUI 300, based on the functions (or lack thereof) provided by various user devices 102. For example, if the user device 102a is configured with a pressure-sensitive touch input function, the user device 102a may present the GUI 300. If the user device 102a is not configured with a pressure-sensitive touch input function, the user device 102a may present the GUI 400.
[0044] In some implementations, the GUI 400 may include a graphical element 402 for presenting text input. For example, the graphical element 402 may be presented in response to the selection of a notification 202. The graphical element 402 may be presented adjacent to or touching the notification 202. The graphical element 402 may include a prompt 406. For example, the prompt 406 may provide the user with instructions describing what type of information the user should provide to the selected text input control. The graphical element 402 may include a text input control 408. For example, the text input control 408 may present text input provided by the user using a virtual keyboard, as described above.
[0045] In some implementations, the GUI 400 may include a virtual keyboard 410. For example, a graphical element 410 may be presented on the GUI 400 in response to a user selecting a virtual keyboard 202. For example, the virtual keyboard 410 may be presented simultaneously or nearly simultaneously with the graphical element 402. For example, the virtual keyboard 310 may be presented in response to the user device 102a receiving input specified by prompt 208. The virtual keyboard 410 may include a typical virtual keyboard layout for selecting characters.
[0046] In some implementations, the virtual keyboard 410 may include graphical elements 412 and / or 414 for navigating between text controls (e.g., text input boxes, text input fields, etc.) presented on the display of the media device 106. For example, the operating system 108 or application 110 on the media device 106 may request text input to multiple text input controls. The operating system 108 or application 110 may, for example, request the user's username and password. The user can select graphical element 412 to select the next text input control. The user can select graphical element 414 to select the previous text input control.
[0047] In some implementations, the virtual keyboard 410 may include a graphical element 416 that indicates when the user has finished providing text input. For example, the user may select a graphical element 416 that indicates when the user has finished providing input to the current text control. When the user selects a graphical element 416, the user device 102a may send a message to the media device 106 prompting the media device 106 to select another (e.g., the next) text input control. When the user selects a graphical element 416, the user device 102a may send a message to the media device 106 to terminate the remote virtual keyboard service. Alternatively, the user may select a graphical element 418 to terminate the remote virtual keyboard service and release the graphical element 402 and the virtual keyboard 410. However, in some implementations, as described above, the termination of the service may be delayed to allow for the selection of additional text input controls and the provision of additional text input on the media device 106. Exemplary process
[0048] Figure 5 is a flowchart of an exemplary process 500 for providing remote keyboard services on a user device. More specifically, the process 500 described herein is a method for providing remote keyboard services from a user device to a media device (or any other type of computing device). A user device implementing process 500 can provide remote keyboard services and present a remote keyboard without launching a user application on the user device (e.g., remote control application 104a). Process 500 can be efficiently handled by an operating system and makes the remote keyboard service available regardless of the applications installed on the user device and regardless of whether the user device is locked or unlocked.
[0049] In step 502, the user device 102a can receive a broadcast of a status advertisement message from the media device 106. For example, the status advertisement message may include data that can identify the media device and describe the current state of the media device. According to the implementation described herein, the status advertisement message may include data indicating that a text input control presented by the media device 106 is currently selected. Alternatively, the status advertisement message may include data indicating that the media device 106 is requesting a connection to a remote keyboard.
[0050] In step 504, user device 102a can determine that text input control on media device 106 is currently selected. For example, based on the status advertisement message received in step 502, user device 102a can determine that text input control on media device 106 is currently selected.
[0051] In step 506, user device 102a can present a remote keyboard notification. For example, if user device 102a determines that text input control is currently selected on media device 106, user device 102a can present a graphical keyboard notification prompting the user to activate a virtual keyboard on user device 102a. For example, if a user has user device 102a (e.g., smartphone), user device 102b (e.g., laptop), and user device 102n (e.g., tablet) all within the broadcast range of media device 106, each of these devices can determine, based on a status advertisement message, that text input control is currently selected on the media device and present a graphical notification prompting the user to activate a virtual keyboard on each of these user devices.
[0052] In step 508, user device 102a can receive the user's notification selection. For example, user device 102a can receive user input (e.g., tap, swipe, long press, pressure touch, etc.) to select a notification. In some implementations, the first device selected by the user (for example, the user might touch a graphical notification to activate a virtual keyboard on that device) presents a remote keyboard on the selected device for the user to enter text into a text field on media device 106. The remaining devices dismiss the graphical notification from their screens but leave the notification on them.
[0053] In step 510, the user device 102a can establish a connection with the media device 106. As discussed above, after the user selects a notification on the user device 102a by tapping or clicking the notification, the user device 102a can establish a bidirectional communication session with the media device 106 via the communication network 112. For example, since the communication session via the network 112 may allow for the transmission of more data than a status advertisement message, the user device 102a can establish a communication session with the media device 106 via the network 112.
[0054] In step 512, the user device 102a can receive application data from the media device 106. For example, after establishing a communication session with the media device 106, the media device 106 can send application data to the user device 102a. For example, the application data may include media device data such as a description 204 and / or an image 206 representing the media device 106. The application data may also include a graphical representation 304 of the application 110 and / or a prompt 306 from the application 110 (for example, "Please enter your password"), as shown in Figure 3.
[0055] In step 514, the user device 102a can display device data, application data, and / or a keyboard on its display. For example, after receiving device data and / or application data, the user device 102a can display the device data and / or application data as shown in Figure 3.
[0056] In step 516, the user device 102a can receive user input to a virtual keyboard presented on the user device. For example, the user input may be a selection of a single character or other object (graphical elements 312 and / or 314 for navigating between text controls) presented on the virtual keyboard.
[0057] In step 518, the user device 102a can send user input to the media device 106. For example, the user device 102a can send user input to the media device 106 via the communication network 112. For example, after step 518, process 500 can return to step 516. Steps 516 and 518 can be repeated until the user has finished providing input to the virtual keyboard. This allows the media device 106 to respond to each keyboard input as if the keyboard were directly connected to or part of the media device 106.
[0058] Figure 6 is a flowchart of an exemplary process 600 for requesting remote keyboard services from a user device. More specifically, the process 600 described herein is a method for advertising the need for a remote keyboard so that the user can select the most convenient device to activate the remote keyboard, as described above.
[0059] In step 602, the media device 106 can present the application 110 on the display associated with the media device 106. In some implementations, the media device 106 can be configured with a variety of media applications for presenting media items to the user. The user can launch the application 110 on the media device 106 using a remote control device. The application 110 can present a graphical user interface that allows the user to control the application 110. For example, the application 110 may present a graphical user interface that allows the user to browse, search, and view media items available through the application 110.
[0060] In step 604, the media device 106 may receive an indication from application 110 that a text input control has been selected. For example, while interacting with application 110, application 110 may request the user to provide text input. Application 110 may request a username and password. Application 110 may request search parameters to perform a media search. In any case, application 110 may present a text input control, and the user may select the text input control presented by application 110 (for example, using a remote control device 120). Once a text input control is selected, application 110 may send a message (for example, using the API described above) to the operating system 108 on the media device 106 indicating that a text input control has been selected and / or providing a description of the information requested by the text input control. In some implementations, the message may include an identifier for application 110 and / or metadata for other applications, such as a graphical representation 304.
[0061] In step 606, the media device 106 can generate an advertisement message indicating that a text input control is selected. In some implementations, the media device 106 can generate a status advertisement message that includes status information indicating that a text input control presented by the application 110 is selected. For example, in response to receiving a message from the application 110 in step 604, the operating system 108 can generate a status advertisement message that includes data indicating that a text input control is currently selected on the media device 106.
[0062] In step 608, the media device 106 can broadcast a status advertisement message. In some implementations, after the media device 106 generates a status advertisement message indicating that text input control is selected, the media device 106 can broadcast the status advertisement message. For example, the advertisement message can be broadcast without identifying a specific recipient device. This allows all user devices within the broadcast range of the media device 106 to receive the status advertisement message.
[0063] In step 610, media device 106 may receive a message indicating that a user device is providing a remote keyboard to the media device. For example, after all user devices 102 within the broadcast range of media device 106 have received a media device status advertisement broadcast from media device 106, media device 106 may receive a message indicating that user device 102a is providing a remote keyboard to media device 106. For example, a message can be sent to media device 106 via network 112. The message can initiate a bidirectional communication session between media device 106 and user device 102a so that media device 106 and user device 102a can exchange data related to text input control.
[0064] In step 612, the media device 106 can send application data to the user device 102a. For example, the application data may include device data representing the media device (e.g., media device 106), such as a description 204 and / or an image 206. The application data may include a graphical representation 304 of the application 110 and / or a prompt 306 from the application 110 (e.g., "Please enter your password"), as shown in Figure 3. As described above in step 604, the operating system 108 can receive the application data from the application 110.
[0065] In step 614, the media device 106 can receive keyboard input from the user device 102a. For example, when the user device 102a receives user input in which it selects characters and / or other objects on a remote keyboard (e.g., keyboard 310, keyboard 410) presented on the user device 102a, the user device 102a can send the user input (e.g., the selected character or other object) to the media device 106.
[0066] In step 616, the media device 106 can send the user's keyboard input to the application 110. For example, after receiving the user's keyboard input from the user device 102a, the operating system 108 can send the keyboard input to the application 110. The application 110 can then perform actions according to the purpose of the selected text control. For example, if the currently selected text control is configured to receive username and / or password input, the application 110 can process the keyboard input to determine whether the entered username and / or password is correct for that user. If the currently selected text control is configured to receive search parameters, the application 110 can process the keyboard input by performing a search after the user enters each character.
[0067] The media device 106 receives and processes each keyboard input (e.g., character selection, object selection, etc.), so process 600 can return to step 614 to receive the next keyboard input and return to step 616 to process the next keyboard input. Thus, process 600 can repeat steps 614 and 616 until the user has finished providing input to the selected text control.
[0068] Furthermore, while process 600 is described as receiving text input to application 110, process 600 can be similarly implemented to receive text input to operating system 108. For example, in process 600, application 110 can be replaced with operating system 108 to acquire remote keyboard services for operating system 108 and receive text input for text input controls presented by operating system 108. Graphical User Interface
[0069] The above disclosure describes various graphical user interfaces (GUIs) for implementing various functions, processes, or workflows. These GUIs may be presented on a variety of electronic devices, including, but not limited to, laptop computers, desktop computers, computer terminals, television systems, tablet computers, e-readers, and smartphones. One or more of these electronic devices may have a touch-sensitive surface. The touch-sensitive surface can process multiple simultaneous input points, including processing data related to the pressure, degree, or position of each input point. Such processing can facilitate multi-finger gestures, such as pinch and swipe operations.
[0070] Where this disclosure refers to “selecting” or “having selected” a user interface element within a GUI, these terms are understood to include clicking or “hovering” over the user interface element with a mouse or other input device, or touching, tapping, or gesturing over the user interface element with one or more fingers or a stylus. User interface elements may be virtual buttons, menus, selectors, switches, sliders, scrubbers, knobs, thumbnails, links, icons, radio buttons, checkboxes, and any other mechanisms for receiving user input or providing user feedback. privacy
[0071] This disclosure acknowledges that in this technology, such personal data can be used for the benefit of the user. For example, personal data may be used to deliver targeted content that is more interesting to the user. Thus, such use of personal data enables intentional control over the content delivered. Furthermore, other uses of personal data that benefit the user are also conceived in this disclosure.
[0072] This disclosure further assumes that any entity responsible for collecting, analyzing, disclosing, transferring, storing, or otherwise using such personal data will comply with appropriately defined privacy policies and / or privacy practices. Specifically, such entities should implement and consistently use privacy policies and practices that are generally recognized as meeting or exceeding industry or government requirements for keeping personal data strictly confidential. For example, personal information from users should be collected for the entity's lawful and legitimate use and should not be shared or sold for any other purpose. Furthermore, such collection should only be carried out after informing and obtaining the user's consent. Furthermore, such entities will take all necessary measures to protect and secure access to such personal data and to ensure that others with access to such personal data comply with those privacy policies and procedures. Furthermore, such entities may undergo third-party evaluations to demonstrate their compliance with widely accepted privacy policies and practices.
[0073] Notwithstanding the foregoing, the Disclosure also conceives embodiments that allow a user to selectively prevent the use of or access to personal data. That is, the Disclosure conceives that hardware and / or software elements can be provided to prevent or prevent access to such personal data. For example, in the case of an advertising service, the Technology can be configured to allow a user to choose to "opt in" or "opt out" of participating in the collection of personal data during registration for the Service. In another embodiment, the user may choose not to provide location information to the target content delivery service. In yet another embodiment, the user may choose not to provide precise location information but to allow the transfer of location zone information. Exemplary system architecture
[0074] Figure 7 is a block diagram of an exemplary computing device 700 capable of implementing the functions and processes shown in Figures 1 to 6. The computing device 700 may include a memory interface 702, one or more data processors, an image processor and / or a central processing unit 704, and a peripheral device interface 706. The memory interface 702, one or more processors 704, and / or peripheral device interface 706 may be separate components or may be integrated within one or more integrated circuits. The various components of the computing device 700 may be coupled to one or more communication buses or signal lines.
[0075] Sensors, devices, and subsystems can be easily made multifunctional by coupling them with the peripheral device interface 706. For example, by coupling a motion sensor 710, a light sensor 712, and a proximity sensor 714 to the peripheral device interface 706, direction, illumination, and proximity functions can be easily enabled. Other sensors 716, such as global navigation satellite systems (GNSS) (e.g., GPS receivers), temperature sensors, biosensors, magnetometers, or other sensing devices, can also be easily made related functionalities by coupling them with the peripheral device interface 706.
[0076] The camera subsystem 720 and optical sensor 722 (for example, a charge-coupled device (CCD) or a capture-type metal-oxide-semiconductor (CMOS) optical sensor) can be used to facilitate camera functions such as recording photographs and video clips. The camera subsystem 720 and optical sensor 722 can also be used to collect images of the user, for example, for user authentication by facial recognition analysis.
[0077] Communication functions can be facilitated through one or more wireless communication subsystems 724, such subsystems as wireless receivers and transmitters, and / or optical (e.g., infrared) receivers and transmitters. The specific design and implementation of the communication subsystem 724 may depend on the communication network(s) on which the computing device 700 is intended to operate. For example, the computing device 700 may include communication subsystems 724 designed to operate on GSM networks, GPRS networks, EDGE networks, Wi-Fi or WiMAX networks, and Bluetooth networks. In particular, the wireless communication subsystem 724 may include a hosting protocol that allows the device 100 to be configured as a base station for other wireless devices.
[0078] By coupling the audio subsystem 726 with the speaker 728 and microphone 730, voice-enabled functions such as speaker recognition, voice duplication, digital recording, and telephony functions can be easily implemented. The audio subsystem 726 can be configured to facilitate, for example, the processing of voice commands, voiceprint recognition, and voice authentication.
[0079] The I / O subsystem 740 may include a touch surface controller 742 and / or other input controllers (one or more) 744. The touch surface controller 742 may be coupled to the touch surface 746. The touch surface 746 and the touch surface controller 742 may, for example, use one of several touch sensitivity techniques to detect contact and movement, or to detect interruptions in contact and movement. Touch sensitivity techniques include, but are not limited to, capacitive techniques, resistive techniques, infrared techniques, and surface acoustic wave techniques, as well as other proximity sensor arrays or other elements for determining one or more contact points with the touch surface 746.
[0080] Other input controllers (one or more) 744 can be coupled to other input / control devices 748, such as one or more buttons, rocker switches, thumbwheels, infrared ports, USB ports, and / or pointer devices such as styluses. One or more buttons (not shown) may include up / down buttons for volume control of the speaker 728 and / or microphone 730.
[0081] In one implementation, pressing the button for a first duration unlocks the touch surface 746, and pressing the button for a second duration longer than the first duration turns the computing device 700 on / off. Pressing the button for a third duration activates a voice control module, i.e., a voice command module that causes the device to execute commands spoken by the user into the microphone 730. The user can customize the functionality of one or more buttons. The touch surface 746 can also be used to implement, for example, virtual buttons or soft buttons, and / or a keyboard.
[0082] In some implementations, the computing device 700 can present recorded audio and / or video files, such as MP3, AAC, and MPEG files. In some implementations, the computing device 700 may include the functionality of an MP3 player, such as an iPod®. Therefore, the computing device 700 may include a 36-pin connector compatible with an iPod. Other input / output and control devices may also be used.
[0083] The memory interface 702 can be coupled to memory 750. Memory 750 may include one or more magnetic disk storage devices, one or more optical storage devices, and / or high-speed random-access memory such as flash memory (e.g., NAND, NOR) and / or non-volatile memory. Memory 750 can store operating systems 752 such as Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or embedded operating systems such as VxWorks.
[0084] The operating system 752 may include instructions for handling basic system services and instructions for performing hardware-dependent tasks. In some implementations, the operating system 752 can function as a kernel (e.g., the UNIX kernel). In some implementations, the operating system 752 may include instructions for performing voice authentication. For example, the operating system 752 can implement the remote keyboard functionality described with reference to Figures 1 to 6.
[0085] Memory 750 may also store communication instructions 754 to facilitate communication with one or more additional devices, one or more computers, and / or one or more servers. Memory 750 may include graphical user interface instructions 756 to facilitate graphical user interface processing, sensor processing instructions 758 to facilitate sensor-related processing and functions, telephone instructions 760 to facilitate telephone-related processing and functions, electronic messaging instructions 762 to facilitate electronic messaging-related processing and functions, web browsing instructions 764 to facilitate web browsing-related processing and functions, media processing instructions 766 to facilitate media processing-related processing and functions, GNSS / navigation instructions 768 to facilitate GNSS and navigation-related processes and instructions, and / or camera instructions 770 to facilitate camera-related processing and functions.
[0086] Memory 750 can store other software instructions 772 to facilitate other processing and functions, such as the remote keyboard processing and functions described with reference to Figures 1 to 6.
[0087] Memory 750 can also store other software instructions 774, such as web video instructions to facilitate web video-related processing and functions, and / or web shopping instructions to facilitate web shopping-related processing and functions. In some implementations, media processing instructions 766 can be divided into audio processing instructions to facilitate audio processing-related processing and functions, and video processing instructions to facilitate video processing-related processing and functions.
[0088] Each of the identified instructions and applications described above may correspond to an instruction set for performing one or more of the functions described above. These instructions do not need to be implemented as separate software programs, procedures, or modules. Memory 750 may include additional instructions, or it may contain fewer instructions. Furthermore, various functions of the computing device 700 can be implemented in hardware and / or software, including one or more signal processing and / or application-specific integrated circuits.
Claims
1. It is a method, The media device determines whether text input control is currently selected. The media device broadcasts a status advertisement message containing status information indicating that the text input control has been selected. After the broadcast of the aforementioned status advertisement message, the media device establishes a bidirectional communication channel with the user device. The media device receives user input from the user device to the text input control using the bidirectional communication channel, The user input to the text input control is provided by the media device. A method for providing this.
2. Determining that the text input control is currently selected includes receiving an indication from an application running on the media device in which the text input control is currently selected. The method according to claim 1.
3. The status advertisement message includes status information indicating the purpose of the currently selected text input control. The method according to claim 1.
4. Before receiving the user input, the media device receives a message indicating that the user device provides a remote keyboard for entering text for the text input control. The method according to claim 1, further comprising:
5. In response to receiving the message indicating that the user device provides the remote keyboard, the media device transmits to the user device, via the media device, information describing either the text input control, the media device, or an application requesting the currently selected text input control. The method according to claim 4, further comprising:
6. The user input is received by the media device via the bidirectional communication channel established by the user device. The method according to claim 4.
7. It is a system, One or more processors, A non-temporary computer-readable medium containing one or more sequences of instructions, wherein when the instructions are executed by the one or more processors, The media device determines whether text input control is currently selected. The media device broadcasts a status advertisement message containing status information indicating that the text input control has been selected. After the broadcast of the aforementioned status advertisement message, the media device establishes a bidirectional communication channel with the user device. The media device receives user input from the user device to the text input control using the bidirectional communication channel, The user input to the text input control is provided by the media device. A system that enables this to happen.
8. Determining that the text input control is currently selected further includes receiving an indication from an application running on the media device on which the text input control is currently selected. The system according to claim 7.
9. The status advertisement message includes status information indicating the purpose of the currently selected text input control. The system according to claim 7.
10. The instruction means that, before receiving the user input, the media device receives a message indicating that the user device provides a remote keyboard for entering text for the text input control. The system according to claim 7, further comprising the following steps.
11. The instruction, upon receiving the message indicating that the user device is providing the remote keyboard, involves the media device sending to the user device information describing either the text input control, the media device, or an application requesting the currently selected text input control. The system according to claim 10, further comprising the following steps.
12. The user input is received by the media device via the bidirectional communication channel established by the user device. The system according to claim 10.
13. A non-temporary computer-readable medium containing one or more sequences of instructions, wherein when the instructions are executed by one or more processors, The media device determines whether text input control is currently selected. The media device broadcasts a status advertisement message containing status information indicating that the text input control has been selected. After the broadcast of the aforementioned status advertisement message, the media device establishes a bidirectional communication channel with the user device. The media device receives user input from the user device to the text input control using the bidirectional communication channel, The user input to the text input control is provided by the media device. A non-temporary computer-readable medium that enables the operation of [the computer].
14. Determining that the text input control is currently selected further includes receiving an indication from an application running on the media device on which the text input control is currently selected. The non-temporary computer-readable medium according to claim 13.
15. The status advertisement message includes status information indicating the purpose of the currently selected text input control. The non-temporary computer-readable medium according to claim 13.
16. The instruction means that, before receiving the user input, the media device receives a message indicating that the user device provides a remote keyboard for entering text for the text input control. A non-temporary computer-readable medium according to claim 13, further comprising the following steps.
17. The instruction, upon receiving the message indicating that the user device is providing the remote keyboard, involves the media device sending to the user device information describing either the text input control, the media device, or an application requesting the currently selected text input control. A non-temporary computer-readable medium according to claim 16, further comprising the following steps.
18. The user input is received by the media device via the bidirectional communication channel established by the user device. The non-temporary computer-readable medium according to claim 16.