Remotely controllable smart device and method

Abstract

The controllable device provides data location information to the controller device via the first communications interface indicating where user interface data can be obtained, the data location information may include an address of the controllable device, and the at least one hardware processor is configured to provide user interface data upon request from the controller device via the second communications interface, enabling the controller device to render a user interface, receive user input via the user interface, and send messages to the controllable device corresponding to commands received through the user interface, and implement the commands based on the messages.

Description

[Technical Field]

[0001] FIELD OF THE DISCLOSURE This disclosure relates generally to connected devices, and more particularly to remote control of such devices. [Background technology]

[0002] This section is intended to introduce the reader to various aspects of art that may be related to various aspects of the present disclosure, which are described and / or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. Accordingly, it should be understood that these statements are to be read in this light, and not as admissions of prior art.

[0003] Smart devices, i.e., devices that are connected to a network or that can connect to a network, offer certain benefits to their users. Users or other devices can connect to smart devices, for example, through a network connection. Most smart devices can also acquire new functionality. Examples of smart devices include automobiles, thermostats, refrigerators, ovens, and televisions.

[0004] Taking television as an example, a smart television can typically offer different applications such as television channel viewing, video on demand (VOD), playback, web browsing and games.

[0005] However, interacting with smart devices is not always easy. A prominent example is televisions, where traditional remote controls tend not to be well-suited for users to interact with one or more of the television's functions. In this case, the user may need to navigate through menu trees or use complex combinations of buttons to perform a given action.

[0006] One existing solution to this problem is to download and use an app on a smartphone, which typically requires the smart device and smartphone to be connected to the same Wi-Fi network. Downloading such apps can be facilitated using a Near-Field Communication (NFC) tag on the smart device or by scanning a QR code displayed on the smart device's screen.

[0007] However, even if a user is automatically "directed" to an app site, the app still must be manually installed. Additionally, the app remains on the smartphone once the interaction ends. Furthermore, traditional solutions may require proprietary technology compatibility between the smartphone and the smart device.

[0008] It can therefore be appreciated that a solution that addresses at least some of the shortcomings of interacting with smart devices is desirable. The present principles provide such a solution. Summary of the Invention

[0009] In a first aspect, the present principles are directed to a controllable device comprising: a first communications interface configured to provide information to a controller device indicating a data location from which user interface data can be obtained; and at least one hardware processor configured to provide user interface data to the controller device via a second communications interface upon receipt of a request at the data location, the user interface data enabling the controller device to render a user interface, and to receive messages from the controller device via a third communications interface corresponding to commands received through the user interface.

[0010] In a second aspect, the present principles are directed to a method in a controllable device, comprising: providing information to a controller device via a first communications interface indicating a data location from which user interface data can be obtained; providing user interface data to the controller device via a second communications interface upon receipt of a request at the data location, the user interface data enabling the controller device to render a user interface; and receiving from the controller device via a third communications interface a message corresponding to a command received through the user interface.

[0011] In a third aspect, the present principles are directed to a computer program product stored on a non-transitory computer-readable medium and comprising program code instructions executable by a processor to implement the steps of the method according to any embodiment of the second aspect. [Brief explanation of the drawings]

[0012] Features of the present principles will now be described, by way of non-limiting example, with reference to the accompanying drawings, in which: [Figure 1] 1 illustrates a system in accordance with an embodiment of the present principles; [Figure 2] 1 illustrates a flowchart of an embodiment of a method according to the present principles. [Figure 3] 1 illustrates a flowchart of a method for limiting interaction in time, in accordance with an embodiment of the present principles; DETAILED DESCRIPTION OF THE INVENTION

[0013] 1 illustrates a system 100 according to an embodiment of the present principles. System 100 includes a smart device 110 and a controller 120. As a non-limiting example, the smart device is described as a television. It should be understood that the smart device may in fact be of any type, such as another type of content renderer, an oven, a refrigerator, or a light bulb, and that all of the features described for a television may not be included in another type of smart device.

[0014] The smart device 110 may typically include a user interface (UI) 111, at least one hardware processor ("processor") 112, memory 113, a first communication interface 114, a second communication interface 115, and a display 116.

[0015] The user interface 111 is configured to receive input (e.g., commands) from a user either directly (e.g., through buttons or a touch screen) or indirectly from a user interface unit such as a conventional remote control (not shown).

[0016] The processor 112 is configured to execute program code instructions for implementing methods according to the present principles.

[0017] The memory 113, which may be at least partially non-transitory, is configured to store program code instructions, parameters, image data, intermediate results, etc. to be executed by the processor 112.

[0018] The first communication interface 114 is configured to transmit UI data position information to the controller 120, as will be further described. The first communication interface 114 can implement any suitable technology, wired or wireless, or a combination of the two. Examples include an NFC interface and a QR code (e.g., shown on the display 116 or printed in a location readable by the controller).

[0019] The second communication interface 115 is configured to communicate with the device over a network, for example, a WiFi network or the Internet.

[0020] The display 116 is configured to display information directed to the user. In a television, the display 116 may also be configured to display traditional content, menus, and the like.

[0021] Controller 120 may be, for example, a conventional smartphone, tablet, or "universal" remote control with the functionality referenced in FIG.

[0022] The non-transitory storage medium 140 stores computer readable instructions that, when executed by a processor, perform a method according to an embodiment of the present principles.

[0023] FIG. 2 illustrates a method 200 in accordance with an embodiment of the present principles.

[0024] In step S202, the controller 120 interacts with the first communication interface 114 in a manner that depends on the technology used. For example, if the first communication interface 114 is a near field communication (NFC) tag, the controller 120 interacts with the NFC tag via the NFC reader when the user places the NFC reader close enough to the tag. If the first communication interface uses a QR code, the controller interacts via a camera and QR code detector and interpreter when the user points the camera at the QR code and triggers interpretation. Other possibilities include text recognition and over-the-air broadcasting.

[0025] During the interaction, data location information is transferred to the controller 120. The UI data location information, which may be, for example, a Uniform Resource Locator (URL) or an Internet Resource Locator (IRL), indicates where (e.g., at what address) the UI data can be obtained. Note that the UI data typically does not refer to the user interface 111 of the smart device 110, which may have at least partially similar functionality but are typically different. Alternatively, the smart device displays the data location information so that the user can manually enter it on the controller 120.

[0026] The data location information may be dynamically provided by the processor 112 to the first communication interface 114, thus allowing different data location information to be provided to the controller 120 at different times.

[0027] In other words, in step S202, the controller obtains the data location information provided by the smart device 110.

[0028] In step S204, resident software on the controller 120 uses the UI data location information to retrieve UI data that can be provided by the smart device 110 upon request. In one embodiment, the resident software is a web browser that can use a URL to download information on that web page, as is well known. Other methods of using addressing information to retrieve data from networked devices can also be used.

[0029] In one embodiment, the UI data is located in memory 113 of the smart device 110. Thus, the smart device 110 can control the UI data, for example, update it, replace, add, and / or delete information. In one embodiment, the UI data can be provided and / or modified by downloaded programs running on the smart device 110.

[0030] The UI data enables the controller to generate a user interface, which in one embodiment is a graphical user interface (GUI).

[0031] In step S206, resident software on controller 120 displays the formatted UI data to the user in the form of a user interface, which may include one or more interactive objects, such as buttons, widgets, menus, and a virtual keyboard. Formatting data for display, for example, formatting HTML data (by a web browser), Java bytecode (by a Java virtual machine (VM)), or Python script (in a Python environment), is well known in the art. It will be understood that other methods of rendering a user interface, such as via audio, are possible, and display is only one example.

[0032] The user interface may, for example, mimic a traditional TV remote control with buttons (where appropriate), although those skilled in the art will appreciate that more advanced user interfaces with widgets and submenus are possible. The TV user interface may include, for example, arrow keys for navigating within a media gallery, channel buttons (e.g., up and down), volume buttons, and a keyboard for entering text within a web browser.

[0033] Interactive objects are associated with actions. Different interactive objects are typically associated with different actions, although it is possible for an interactive object to be associated with a single action. An action can be, for example, sending a command, one or more steps when navigating a menu, and entering text.

[0034] In step S208, the controller 120 receives user input, e.g., commands, via the displayed user interface. Resident software interprets the user input and obtains its associated action, as is well known in, e.g., web browsers.

[0035] In step S210, the resident software of the controller 120 sends a message regarding the action to the smart device 110.

[0036] For example, a button in a displayed web page may include an address (e.g., a URL) that links to the smart device 110. Activating the button causes a message, such as an HTTP request or a Web Socket (WS) message, to be sent to the smart device 110. Different interactive objects can be associated with different actions, so that different interactive objects can cause different messages to be sent to the smart device 110.

[0037] In one embodiment, the address is that of a server and includes an action-specific suffix. The server may be executed by the smart device 110. The address may also indicate an intermediate device, such as a WiFi router or server, with which the smart device is registered and which, upon interpreting a message for a particular device (e.g., smart device 110), translates the address into the address of a server running on the smart device 110, e.g., using an identifier in the action. The server may also run on a device, e.g., a decoder, that interprets the message and sends the corresponding interpreted message to the smart device, e.g., using High-Definition Multimedia Interface (HDMI) Consumer Electronics Control (CEC) technology.

[0038] In step S212, the server receives the message from the controller 120 and interprets it as a command. If the server is not hosted by the smart device 110, the server determines the corresponding command to send to the smart device 110.

[0039] As mentioned above, the message may be, for example, an HTTP request or a WebSocket message that may be interpreted as a command resulting from options provided by a user interface on the controller.

[0040] In step S214, the smart device 110 implements the command. The implementation may be directly by the smart device 110 or indirectly by software such as an application running on the smart device 110. In the latter case, the smart device 110 provides the command to the software.

[0041] As can be seen, the present principles can be implemented on a controller using conventional technology, so there is no need for dedicated software running on the controller 120. Thus, the memory requirements of the controller can be less than, for example, a device that installs dedicated software. Indeed, after interacting with a smart device, a controller according to the present principles can typically store little, or even no, data in cache and / or browser history.

[0042] In one embodiment, the user interface can depend on the state of the smart device, which can depend, for example, on the type of application currently being executed by the smart device. Thus, in the case of a television, a first user interface can be used when watching live television, a second user interface can be used when using the television to surf the Internet, and so on. UI data for the second user interface can be transferred to the controller in response to a command (e.g., a selection in a menu) entered using the first user interface. UI data for the second user interface can also be provided to the controller upon interaction with the first communication interface, as described with reference to step S202 of FIG. 2. In the case of a television or other content renderer, UI data can also be provided by currently rendered content, such as a broadcast program, which can, for example, enable interaction with the content.

[0043] For example, in the case of different applications downloaded to the smart device, each application may include or otherwise provide its own UI data to be transferred by the smart device to the controller.

[0044] In one embodiment, the server hosts a WebSocket (WS) server to connect smart devices or smart device applications on the one hand and resident software that uses the UI data as connected clients on the other hand.

[0045] In one embodiment, multiple users can have controllers interacting with the smart device, even simultaneously. Each user can obtain UI data as previously described. To distinguish between users, the smart device can provide different UI data to each user, such that each controller provides a different identifier, although it is also possible to use the same UI data if the controller or user is identified differently, for example, using a controller identifier such as an IP address.

[0046] In one embodiment, the interaction between the controller and the smart device may be time-limited. Figure 3 illustrates such a method 300, achieved, for example, by implementing a temporary chat room mechanism inside WebSockets, as follows: 1) The smart device 30 sends a "room request" to the "room" server 32 (S302). 2) Upon receiving the request, the server generates a random number (rnd) and creates a new room with a name that is a function of rnd: Rname = f(rnd). 3) The server 32 sends the room name (S304) to software in the smart device 30, which initiates a connection to the room (S306). 4) The software provides data location information, including the room name as a parameter, to the first communication interface 34 (S308). 5) The controller 36 interacts with the first communication interface 34 (S310), leading to a UI being rendered. The UI allows messages to be sent to the software via the room and room server. 6) Upon receiving a first message "command" in a room (S312), the server 32 starts a timer to limit the opening of the room (S314), and the server 32 transfers the corresponding command to the smart device 30 (S316). After the timer expires (S318), the room is closed (S320). No further messages can be communicated through the room. 7) A new room can be created upon any request.

[0047] In one embodiment, the interaction between the controller and the smart device may be distance-limited. This may be achieved, for example, by implementing communication between the smart device and the controller via Bluetooth. Due to the low range of Bluetooth, the controller will lose connection with the smart device when it is out of range. Proximity detection may also be achieved using RSSI (Received Signal Strength Indication) during the Bluetooth scanning phase. When the controller renders a UI, it opens a connection to the room and provides its Bluetooth ID as a parameter. The smart device can then attempt a Bluetooth connection and measure the RSSI to infer the distance of the controller. The smart device can then ignore messages from a controller that is determined to be too far away.

[0048] As another example, a smart device may be an oven that has a very limited display or no display at all. Providing a UI on the controller may provide an easier way and / or at some distance to enter commands that could also be entered using the UI on the oven itself. It may also be possible to provide a more advanced user interface that can suggest predefined cooking options for a set of meals, for example.

[0049] It will therefore be appreciated that the present principles can be used to provide a dynamic user interface that can have a small footprint on the controller.

[0050] It should be understood that the elements shown in the figures may be implemented in various forms of hardware, software, or a combination thereof. Preferably, these elements are implemented in a combination of hardware and software on one or more appropriately programmed general-purpose devices, which may include a processor, memory, and input / output interfaces.

[0051] This specification illustrates the principles of the present disclosure, and it will thus be appreciated that those skilled in the art will be able to devise various arrangements which, although not explicitly described or shown herein, embody the principles of the present disclosure and are included within its scope.

[0052] All examples and conditional language recited herein are intended for educational purposes to aid the reader in understanding the principles of the present disclosure and concepts contributed by the inventors to further the art, and should not be construed as being limited to such specifically recited examples and conditions.

[0053] Moreover, all statements herein reciting principles, aspects, and embodiments of the present disclosure, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. In addition, such equivalents are intended to include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.

[0054] Thus, for example, it will be appreciated by those skilled in the art that the block diagrams presented herein represent conceptual views of illustrative circuitry embodying the principles of the present disclosure. Similarly, it will be appreciated that any flowcharts, flow diagrams, or the like may be substantially represented on a computer-readable medium and represent various processes that may be performed by a computer or processor, whether or not such a computer or processor is explicitly shown.

[0055] The functions of the various elements shown in the figures may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. If provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by multiple individual processors, some of which may be shared. Furthermore, explicit use of the terms "processor" or "controller" should not be construed as referring exclusively to hardware capable of executing software, but may implicitly include, without limitation, digital signal processor (DSP) hardware, read only memory (ROM) for storing software, random access memory (RAM), and non-volatile storage.

[0056] Other hardware, conventional and / or custom, may also be included. Similarly, any switches shown in the figures are conceptual only. Their functions may be performed through the operation of program logic, through the operation of dedicated logic, through program control and interaction with dedicated logic, or even manually, with the particular technique being selectable by the implementer as more particularly understood from the context.

[0057] In the claims herein, any element expressed as a means for performing a certain function is intended to encompass any way for performing that function, including, for example, a) a combination of circuit elements that perform that function, or b) software in any form, including firmware, microcode, etc., in combination with appropriate circuitry for executing the software to perform the function. The disclosure defined by such claims resides in the fact that the functionality provided by the various recited means are combined and brought together in the manner the claims call for. Thus, any means that can provide those functionality are regarded as equivalent as those shown herein.

Claims

1. A controllable device, comprising: a first communication interface configured to provide information to a controller device indicating a data location from which user interface data can be obtained, the user interface data enabling the controller device to render a user interface; and at least one hardware processor, providing the user interface data to the controller device via a second communication interface upon receiving a request from the controller device at the data location; receiving, by a server at an address provided with the user interface data, from the controller device via a third communication interface, messages corresponding to commands received by the controller device through the user interface; at least one hardware processor configured to:

1. A controllable device comprising:

2. The controllable device of claim 1 , wherein the information indicating the data location includes an address of the controllable device.

3. The controllable device of claim 2 , wherein the address is one of a uniform resource locator (URL) and an internet resource locator (IRL).

4. The controllable device of claim 1 , wherein the first communication interface is a near field communication tag.

5. The controllable device of claim 1 , wherein the user interface data comprises at least one of an HTML web page, a Java program, and a Python script.

6. The controllable device of claim 1 , wherein the message is one of an HTTP request and a WebSocket message.

7. The controllable device of claim 1 , wherein the user interface data depends on at least one of an application executed by the controllable device and content rendered by the controllable device.

8. The controllable device of claim 1 , wherein the at least one hardware processor is further configured to distinguish between messages received from multiple controller devices using a controller device identifier.

9. The controllable device of claim 1 , wherein the at least one hardware processor is further configured to implement a time limit for interactions with the controller device.

10. The controllable device of claim 1 , wherein the at least one hardware processor is further configured to implement distance restrictions on interactions with the controller device.

11. The controllable device of claim 1 , wherein the at least one hardware processor is further configured to execute the command based on the message.

12. The controllable device of claim 1 , wherein the second communication interface and the third communication interface are implemented in one communication interface.

13. 1. A method in a controllable device, comprising: providing information to the controller device via the first communication interface indicating a data location from which user interface data can be obtained; upon receiving a request at the data location, providing the user interface data to the controller device via a second communications interface, the user interface data enabling the controller device to render a user interface; receiving, by a server at an address provided with the user interface data, from the controller device via a third communications interface, a message corresponding to a command received through the user interface; A method comprising:

14. The method of claim 13 , wherein the user interface data depends on at least one of an application executed by the controllable device and content rendered by the controllable device.

15. The method of claim 13 , further comprising using a controller device identifier to distinguish between messages received from multiple controller devices.

16. The method of claim 13 , further comprising implementing a time limit for interactions with the controller device.

17. The method of claim 13 , further comprising implementing distance restrictions on interactions with the controller device.

18. The method of claim 13 , further comprising: executing the command based on the message.

19. 20. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one hardware processor, cause the at least one hardware processor to perform the method of any one of claims 13 to 18.

Images