System, method, computer program, mobile device, and kit for operating a device with low computational power

Abstract

A system, method, computer program, and device are provided in which a virtual operating system application (22) that essentially operates a less computationally powerful device runs on a mobile device (16). The virtual operating system may allow other applications running on the mobile device to control the execution of applications (26A, 26B) for, or act as a proxy for, another device.

Description

[Technical Field]

[0001] The present invention generally relates to the operation of devices having relatively low computing power. [Background technology]

[0002] Many devices include a display for displaying content and / or an input device that allows a user to enter input. Such an input device may be combined with a display using a so-called touchscreen. Some of these devices have relatively low computing power available due to space or cost constraints. For example, a display may be integrated into so-called smart glasses, but the space constraints and weight considerations of such smart glasses make it difficult to provide such smart glasses with high computing power. However, it may be desirable to provide various information to a user, for example, via such a device with low computing power. Such a device with low computing power may also be referred to herein as a "thin device."

[0003] There are several ways to enhance the functionality of relatively small devices by linking them to mobile devices such as smartphones and tablets. For example, in the case of a smartwatch, application programs (also referred to herein as apps) are installed on the smartwatch and corresponding application programs are installed on the mobile device, and these apps communicate with each other to provide the smartwatch with functionality such as access to multimedia files such as music and videos on the mobile device, forwarding of incoming calls from the mobile device, and other features.

[0004] However, this solution requires that a corresponding app be installed on the smart device, which may not be possible on devices with very low computing power or that are otherwise unable to install apps.

[0005] Another conventional approach is to use remote display technology. In this case, content displayed on a mobile device, such as a smartphone or tablet, is essentially mirrored and displayed on a less powerful device. For example, Bluetooth or other wireless communication may be used for this screen mirroring. However, this approach also has several drawbacks. For example, less powerful devices and mobile devices typically have different screen sizes. Content shown on a mobile device is typically designed for the mobile device's screen, and displaying the same content on another device may lead to unsatisfactory results. Furthermore, this approach makes it difficult to operate the mobile device essentially independently of the less powerful device; in other words, this solution may make it difficult to use different applications on the devices. Summary of the Invention [Problem to be solved by the invention]

[0006] The object is therefore to provide an improved solution for operating such thin devices. [Means for solving the problem]

[0007] In this regard, there is provided a system as defined in claim 1, a method as defined in claim 12, a computer program as defined in claim 18, a mobile device as defined in claim 20, and a kit as defined in claim 21. The dependent claims define further embodiments.

[0008] According to one embodiment, a system is provided, comprising: a mobile device including an operating system and a first interface; and another device including a display and a second interface; The mobile device further includes a virtual operating system application that runs under the operating system of the mobile device, the virtual operating system application comprising: managing a connection between the mobile device and another device via a first interface and a second interface; - providing another device with a screen frame that will be displayed on the display of the other device; -Receive and process input from another device, - Allows the execution of applications for other devices on a mobile device It is configured as follows.

[0009] By running a virtual operating system for a low-computational device on a mobile device, functionality can be provided for the low-computational device while still allowing the mobile device to operate with other applications. In this manner, applications tailored to the low-computational device may be provided. Furthermore, such applications may use both input (e.g., sensor input) from the low-computational device and input from the mobile device's sensors, e.g., a GPS sensor.

[0010] The other device may have less computational power than the mobile device, ie, may be a "thin device" as described above.

[0011] In some embodiments, the other device does not allow for the installation and execution of additional applications. For such other devices, applications may run on the mobile device using a virtual operating system application, either within or alongside the virtual operating system, but as independent applications that communicate using inter-process communication mechanisms that may be provided by the operating system of the mobile device. Thus, the virtual operating system may communicate with other applications on the mobile device.

[0012] The virtual operating system may also be configured to send commands and / or configuration information to another device, so that the other device may be operated essentially entirely from the mobile device.

[0013] In some embodiments, the virtual operating system application allows the application to communicate with devices (components) of the mobile device so that these devices may be used by the application.

[0014] The virtual operating system application may be configured to receive screen content from an application and provide a screen frame based on that screen to another device.

[0015] The application may include a navigation application.

[0016] The mobile device may be a smartphone or a tablet computer.

[0017] Another device may be smart glasses.

[0018] According to another embodiment, a method is provided, comprising: running, on the mobile device, a virtual operating system application for another device, the virtual operating system application comprising: managing communication between the mobile device and another device via a first interface of the mobile device and a second interface of the other device; - providing another device with a screen frame that will be displayed on the display of the other device; -Receive and process input from another device, - Allows the execution of applications for other devices on a mobile device.

[0019] The virtual operating system application may enable the application to communicate with the devices (components) of the mobile device.

[0020] The virtual operating system may send commands and / or configuration information to another device.

[0021] The virtual operating system may communicate with other applications on the mobile device, so that these other applications may, for example, send screens to or receive input from other devices via the virtual operating system.

[0022] The virtual operating system application may receive screen content from an application and provide a screen frame to another device based on the screen content.

[0023] The application may include a navigation application.

[0024] According to another embodiment, a computer program is provided, comprising program code which, when executed on a processor, causes the above-mentioned method to be carried out.

[0025] A storage medium, particularly a tangible storage medium, containing the computer program is also provided.

[0026] Additionally, a mobile device having a computer program stored thereon is provided.

[0027] According to another embodiment, a kit is provided comprising: The computer program or storage medium described above; another device operated using a virtual operating system application of the computer program; Includes.

[0028] With such a kit, a user may enable their mobile device to operate another device by installing a computer program on the mobile device.

[0029] Another device may be smart glasses.

[0030] For a better understanding, embodiments will now be described with reference to the accompanying drawings, in which: [Brief explanation of the drawings]

[0031] [Figure 1] FIG. 1 is a block diagram of a system according to an embodiment. [Figure 2] FIG. 1 is a block diagram illustrating the structure of an operating system of some embodiments, including a virtual operating system. [Figure 3] 1 is a schematic diagram of a system according to another embodiment. [Figure 4] 1 is a flowchart illustrating a method according to an embodiment. [Figure 5] 1 is a diagram illustrating an exemplary operation of a system according to an embodiment for a particular application. DETAILED DESCRIPTION OF THE INVENTION

[0032] Various embodiments are described in detail below. It should be noted that these embodiments are merely examples and should not be construed as limiting. Features of different embodiments may be combined to form another embodiment. Changes, improvements, or details described with respect to one of the embodiments may also apply to other embodiments and therefore will not be described again and again.

[0033] Before describing the embodiments in detail, some definitions of terms are provided below.

[0034] A mobile device, as used herein, is a computing device designed to be carried by a person. As a result, such mobile devices weigh less than 2 kg, typically less than 1 kg, and have dimensions of less than 50 cm x 50 cm x 2 cm, typically even smaller. A typical mobile device is a smartphone or a tablet. Many mobile devices have touchscreens for displaying content and receiving user input, as well as other sensors, such as GPS sensors. Mobile devices typically have sufficient computing power to run various custom applications (apps). In embodiments, mobile devices typically run an operating system that allows for the installation of additional such applications.

[0035] A device with low computing power, referred to herein as a thin device, is a device with less computing power than a mobile device. Often, a thin device only has firmware and does not allow the installation of custom applications, i.e., additional applications that are not part of the program absolutely necessary for the operation of the device, or in other words, programs that are not related to the core functionality of the device. The processors used in such thin devices are less powerful than processors used in mobile devices and may, for example, be relatively simple microcontrollers or even simple logic circuits that handle basic inputs or outputs.

[0036] In an embodiment, the operating system for the thin device runs as an application on the mobile device. This operating system is also referred to as a "virtual operating system application" because it runs on another device (the mobile device) rather than on the device (the thin device) for which it provides the operating system capabilities. As explained in more detail below, this allows for the installation of additional applications for the thin device and / or flexible use of the mobile device that is at least partially independent in its use from the thin device.

[0037] FIG. 1 illustrates a system according to one embodiment, and FIG. 2 is a logic diagram further illustrating the operating system structure of the system of FIG.

[0038] The system of Figure 1 includes a thin device 10 and a mobile device 16. Thin device 10 includes a processor 13 coupled to a display 11, one or more input devices 12, and memory 14. Processor, as used herein, is not limited to a single processor and may include the possibility that multiple processors may be provided.

[0039] A display, as used herein, is any device that may be used to display content. For example, in the case of smart glasses mentioned earlier, the display may include some type of projection system that projects content for viewing by a user of the glasses, or may include a more traditional display such as a TFT display or LCD display. An input device may be any device that allows a user to provide input. In some embodiments, an input device may be implemented by providing the display 11 as a so-called touchscreen. Other examples of input devices include a camera sensor that detects gestures by a person operating the thin device 10 or a microphone for receiving voice commands from a person. The memory 14 stores instructions for the processor 13 necessary to operate the thin device 10. In some embodiments, the thin device 10 does not allow custom applications to be installed in the memory 14 or elsewhere on the thin device 10.

[0040] Thin device 10 further includes an interface 15 for communicating with mobile device 16. For this communication, mobile device 16 includes a corresponding interface 17. In the embodiment of FIG. 1, interfaces 15, 17 are wireless interfaces. One example of a suitable wireless interface is a Bluetooth interface. However, other types of wireless interfaces may also be used. Such interfaces are typically present in commercially available mobile devices such as smartphones or tablets. In addition to the illustrated devices, device 10 may also include other components, such as sensors.

[0041] The mobile device 16 includes a processor 18 , one or more other devices 19 , and a memory 110 .

[0042] Other devices 19 may include any devices conventionally used in mobile devices such as smartphones and tablets, such as sensors, for example, a GPS sensor, a loudspeaker, a microphone, a display, a touchscreen, buttons, etc. Memory 110 includes writeable memory, such as flash memory, in which custom applications may be installed. In the system of FIG. 1, one of these custom applications is a virtual operating system application, which essentially provides an operating system for thin device 10. This concept will now be explained with further reference to FIG. 2.

[0043] Thin device 10 is provided with "thin firmware" 20. Thin firmware is firmware on thin device 10 that is essentially limited to providing the various capabilities of thin device 10 necessary for the operation of thin device 10. Additionally, a virtual operating system application 22 is installed within mobile device 16 and runs under the mobile device's normal operating system (OS) 21. Thin firmware 20 interacts with this virtual operating system application 22 to provide the various capabilities to thin device 10.

[0044] The thin firmware 20 has the functionality to pair and unpair the thin device with a mobile device, i.e., to establish or terminate a wireless connection using interfaces 15, 17 in Figure 1. Pairing, particularly for Bluetooth communication or other wireless applications, is a process that occurs when two devices connect for the first time.

[0045] Additionally, thin firmware 20 has the ability to connect to and disconnect from a virtual operating system application 22 running on mobile device 16. Thin firmware 20 then receives screen frames from virtual operating system application 22 for display on display 11 and sends other data, such as user input or sensor data from input device 12, to virtual operating system application 22. In some embodiments, thin firmware 20 may also send device state of thin device 10, such as battery state, to virtual operating system application 22 upon request.

[0046] Thus, the thin firmware 20 sends inputs (user inputs and / or other sensor inputs) to the virtual operating system application 22 and receives screen frames to be displayed from the virtual operating system application 22 .

[0047] The virtual operating system application 22 provides a screen frame to the thin firmware 20 and includes a remote display and input / output handler 23 for receiving input from the thin firmware 20. Additionally, commands and / or configuration information may be sent to the thin firmware 20 via the remote display and input / output handler 23. Exemplary commands may include a screen on / off command, an audio capture on / off command, a disconnect command, etc. Exemplary configurations may include a display off time threshold, a display brightness, etc.

[0048] The input and output are further processed by a content provider interface unit 24, which in one direction (to thin firmware 20) converts screen content into a screen frame format that is sent to thin firmware 20 via remote display and input / output handler 23, and in the other direction (from thin firmware 20) receives input from remote display and input / output handler 23 and provides these to other components such as applications or parts of an operating system. Extension manager 25 is responsible for managing the installation / configuration / removal of mini-apps running "on" the virtual operating system, of which only two applications 26A, 26B (referred to here as mini-apps) are shown, and may communicate with other devices on mobile device 16 (e.g., other devices 19) to provide, for example, sensor input from mobile device 16 to mini-apps 26A, 26B. It should be noted that although the remote display and input / output handler 23, the input / output abstraction component 24, and the extension manager 25 are depicted as separate components of the virtual operating system application 22, this is merely for convenience and to provide a better understanding of some of the functionality provided by the virtual operating system application 22, and they do not have to be implemented as separate components.

[0049] Mini-apps 26A, 26B are applications designed for thin device 10 to provide additional functionality for thin device 10. Examples of such mini-apps are a navigation app that enables navigation using display 11 of thin device 10, or any other application for providing content on the thin device, such as a video player application. Mini-apps 26A, 26B are generally applications that are installed in addition to the basic virtual operating system application 22 and may be provided by a different vendor than virtual operating system application 22, for example. In other words, extension manager 25 enables installation / configuration / removal of mini-apps within the virtual operating system, and content provider interface unit 24 provides an interface with mini-apps 26A, 26B and other applications on mobile operating system 21, allowing them to access input from thin firmware 20 or from other devices 19 on mobile device 16 and modify the content displayed on display 11 of thin device 10.

[0050] Additionally, the virtual mobile operating system application 22 may communicate with other applications 28 (e.g., third-party applications) installed on the mobile device under the mobile device operating system 21. The other applications 28 are independently developed mobile applications running on the mobile device that may use a communication interface 27 provided by the mobile device operating system 21. The communication interface 27 generally functions for communication between applications under the mobile operating system 21. Such a communication interface is typically provided by the operating system of the mobile device. Using the communication interface 27, the virtual mobile operating system application 22 may communicate with the other applications 28. In this manner, the other applications 28 may receive sensor / user data from the thin device and send screen content (if desired) to the thin device, without needing to connect directly to the thin device using interface 17 of FIG. 1 , but only by communicating directly with the virtual mobile operating system application 22.

[0051] Other applications 28 typically have their own independent user interface and business logic, but may extend the user interaction model by sending content to the thin device and receiving thin device input data as described above. Examples of such other applications 28 include sports activity apps (which may, for example, present sports-related information on the thin device and receive motion data from the thin device), navigation apps developed separately from the virtual operating system (although navigation also provides a similar scenario to mini-apps), social networking and messaging apps, etc.

[0052] In some embodiments, the thin device 10 may be provided in a kit with a virtual operating system application 22 as software (on a storage medium or for download) so that a user can install the virtual operating system application on their mobile device (e.g., smartphone, tablet) to operate the thin device 10.

[0053] Figure 3 shows an exemplary system illustrating examples of a thin device and a mobile device. In the exemplary system of Figure 3, the thin device is implemented as smart glasses 30, and display units 31A, 31B are embedded within the glasses to show content to a person wearing the smart glasses 30. The mobile device is implemented as a smartphone or tablet 32.

[0054] FIG. 4 is a flow chart illustrating a method according to one embodiment, illustrating the operation of a virtual operating system application.

[0055] 4 is shown as a series of steps, the order in which these steps are presented should not be construed as limiting, and in particular it should be noted that some steps may be performed sequentially.

[0056] At 40, the method includes receiving input from a thin device at the mobile device, for example, input from a sensor on the thin device that detects user input. At 41, the input is processed via a virtual operating system (e.g., 22 of FIG. 2), optionally using mini-apps installed in the virtual operating system application and / or devices on the mobile device, such as sensors on the mobile device. At 42, the method includes sending to the thin device a screen frame to be displayed on the thin device. Additionally, commands or configuration information may be sent to the thin device, as described with respect to FIG. 2.

[0057] Exemplary operation of the virtual operating system application and mini-apps will now be described with reference to Figure 5. Figure 5 assumes that a user has already installed the virtual operating system on their mobile device and has a thin device, such as the smart glasses 30 of Figure 3, paired with the mobile device.

[0058] It is further assumed that the user has installed a navigation mini-app in the virtual operating system application on the mobile device.

[0059] FIG. 5 shows, from top to bottom, an exemplary timeline of how a thin device operates with a virtual operating system application and a navigation mini-app installed on top of it.

[0060] First, the user opens (launches) the virtual operating system application on the mobile device at 50. The virtual operating system application then launches at 51 and locates the thin device at 52. Communication then begins, for example, using a handshake procedure at 53, and a start screen frame, also called a home screen, is sent from the virtual operating system application to the thin device for display at 54. The user then opens a navigation mini-app running under the virtual operating system on the mobile device at 55. After the navigation mini-app launch procedure and initialization at 56, the start screen of the mini-app is rendered at 57 and sent to the thin device as a screen frame to be displayed at 58.

[0061] The user then inputs the desired destination at 59. In some embodiments, this may be done using an input device such as a touchscreen on the mobile device. In other embodiments, this may be done using an input device on the thin device, such as a microphone, through which the destination is entered by voice. The navigation mini-app then requests the current location from the mobile device at 510, for example using the mobile device's GPS sensor, and receives a location update at 511. Numeral 512 indicates that this update may occur repeatedly. As shown at 530 and 531, sensor data updates from sensors (e.g., orientation sensors) on the thin device may also be requested by the navigation mini-app.

[0062] At 514, the navigation mini-app then renders a confirmation screen, which at 515 is sent to the thin device as a screen frame to be displayed on the thin device.

[0063] The user then initiates navigation at 516. In the embodiment of FIG. 5, this is done by pressing a button on the thin device. In other embodiments, this may be done by other actions on the thin device, such as a voice command, or by using an input device on the mobile device. At 517, the thin device reports the button press to the virtual operating system, which provides this to the navigation mini-app at 518. The navigation mini-app may also receive sensor updates (e.g., a heading sensor) from the thin device, as shown at 532 and 533. Next, at 519, the navigation mini-app calculates the first step (first instruction on how far to go or drive) and renders a corresponding screen at 520, which is provided to the thin device for display at 521. As the user moves, navigation continues in a loop at 522, with the location updated at 523, the next step calculated by the navigation mini-app at 524, and a corresponding screen rendered at 525, which is sent to the thin device for display at 526. Before the next step is calculated at 524, in some embodiments the navigation mini-app may also receive sensor updates (eg, orientation sensor) from the thin device, as shown at 534, 535.

[0064] Once navigation is complete, at some point at 527 the user exits the navigation mini-app, which then closes at 528. Thereafter, at 529, the homescreen corresponding to the homescreen sent for display at 54 is sent to the thin device for display.

[0065] It should be noted that the flow of Figure 5 is merely an example to illustrate possible communications between a mobile device and a thin device on which a virtual operating system application and mini-apps are installed, and that corresponding communications may also occur for other types of mini-apps.

Claims

1. 1. A system comprising: a mobile device (16) including an operating system (21) and a first communication interface (17); and another device (10) including a display (11), a second communication interface (15) and an input device (12); the other device (10) is configured to manage Bluetooth connections via the first communication interface (17) and the second communication interface (15); The mobile device further includes a virtual operating system application (22) that runs under the operating system (21) of the mobile device; The virtual operating system application (22) providing said other device (10) with a screen frame to be displayed on said display (11) of said other device; receiving and processing input from said other device (10), said input representing user input received by said input device (12); enabling execution on said mobile device of an application (26A, 26B) for said other device (10), said application (26A, 26B) being separate from the application currently running on said mobile device; a system configured to perform operations including:

2. The system of claim 1 , wherein the other device (10) has less computing power than the mobile device (16).

3. 3. The system of claim 1 or 2, wherein the other device (10) is not capable of installing and running additional applications.

4. The system of any one of claims 1 to 3, wherein the virtual operating system application (22) is configured to enable communication of the applications (26A, 26B) with the mobile device (19).

5. 5. The system of claim 1, wherein the virtual operating system application is configured to receive screen content from the application and provide the screen frame to the other device based on the screen content.

6. The system of any one of claims 1 to 5, wherein the applications (26A, 26B) include a navigation application for providing navigation to a destination.

7. The system of any one of claims 1 to 6, wherein the mobile device (16) is a smartphone or a tablet computer.

8. The system according to any one of claims 1 to 7, wherein the other device (10) is a smart glass (30).

9. The system of any one of claims 1 to 8, wherein the virtual operating system application (22) is configured to send commands to the other device (10).

10. The system of any one of claims 1 to 9, wherein the virtual operating system application (22) is configured to send configuration information to the other device (10).

11. The system of any one of claims 1 to 10, wherein the virtual operating system application (22) is configured to communicate with another application (28) on the mobile device (16).

12. 12. The system of claim 11, wherein the separate application (28) runs under the operating system of the mobile device (16) independently of the virtual operating system application.

13. The system of any one of claims 1 to 12, wherein managing the connection comprises establishing a connection between the mobile device (16) and the other device (10).

14. 1. A method comprising: Executing, on the mobile device, a virtual operating system application (22) for another device; The virtual operating system application (22) the other device (10) establishing a Bluetooth connection via a first communication interface (17) of the mobile device (16) and a second communication interface (15) of the other device (10) by pairing the other device (10) with the mobile device (16); providing said other device (10) with a screen frame to be displayed on a display (11) of said other device; receiving and processing input from said other device (10), said input representing user input received by an input device (12) of said other device (10); enabling execution on the mobile device (16) of an application (26A, 26B) for the other device (10), the application (26A, 26B) being separate from an application currently running on the mobile device; The other device (10) cancels the pairing between the other device (10) and the mobile device (16) to terminate the Bluetooth connection via the first communication interface (17) and the second communication interface (15); How it is configured to run.

15. 15. The method of claim 14, wherein the virtual operating system application (22) is configured to enable the application (26A, 26B) to communicate with the mobile device (19).

16. 16. The method of claim 14 or 15, wherein the virtual operating system application (22) is configured to receive screen content from the application (26A, 26B) and provide a screen frame to the other device (10) based on the screen content.

17. 17. The method of any one of claims 14 to 16, wherein the application (26A, 26B) comprises a navigation application for providing navigation to a destination.

18. 18. The method of any one of claims 14 to 17, wherein the virtual operating system application (22) is configured to send commands to the other device (10).

19. 19. The method of any one of claims 14 to 18, wherein the virtual operating system application (22) is configured to send configuration information to the other device (10).

20. 20. The method of any one of claims 14 to 19, wherein the virtual operating system application (22) is configured to communicate with another application (28) on the mobile device (16).

21. 21. The method of any one of claims 14 to 20, wherein the step of managing a connection comprises the step of establishing a connection between the mobile device (16) and the other device (10).

22. A computer program comprising program code which, when executed on a processor, causes the processor to carry out the method of any one of claims 14 to 21.

23. A computer-readable storage medium storing the computer program according to claim 22.

24. A mobile device (16) having stored thereon a computer program according to claim 22.

25. a computer program according to claim 22 or a storage medium according to claim 23; another device (10) operated using the virtual operating system application (22) of the computer program; Kit including:

26. 26. The kit of claim 25, wherein the other device (10) is a smart glass.

Images