Split Compute Architecture including Wearable Device and Companion Device

Abstract

The method includes starting a computing process on a wearable device, the computing process including a plurality of tasks, the method including identifying a companion device and determining that the companion device is available to execute at least one of the plurality of tasks, causing the companion device to execute at least one task including communicating data generated by the wearable device to the companion device, receiving, by the wearable device, a result associated with completion of the at least one task by the companion device, and completing, by the wearable device, the computing process based on the result associated with completion of the at least one task.

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims the benefit and priority of U.S. Provisional Patent Application No. 63 / 363,592, filed on April 26, 2022, entitled "SPLIT - COMPUTE ARCHITECTURE", the disclosure of which is hereby incorporated by reference in its entirety.

[0002] This application also incorporates by reference herein the disclosures of related co - pending applications: PCT Application No. PCT / US2023 / 019563, filed on April 24, 2023; PCT Application No. PCT / US2023 / 019832, filed on April 25, 2023; "SPLIT - COMPUTE ARCHITECTURE" (Attorney Docket No. 0120 - 497WO1), filed on April 26, 2023; "PERIPHERAL DEVICES IN A SPLIT - COMPUTE ARCHITECTURE" (Attorney Docket No. 0120 - 498WO1), filed on April 26, 2023; "MULTIPLE APPLICATION RUNTIMES IN A SPLIT - COMPUTE ARCHITECTURE" (Attorney Docket No. 0120 - 505WO1), filed on April 26, 2023; and "MACHINE LEARNING PROCESSING OFFLOAD IN A SPLIT - COMPUTE ARCHITECTURE" (Attorney Docket No. 0120 - 509WO1), filed on April 26, 2023.

[0003] Embodiments relate to wearable device processing architectures.

Background Art

[0004] Some devices (e.g., wearable devices) may have advanced display capabilities. In these devices, it may be a problem that there is not enough room to house the electronic device in a small form factor. These problems become even more difficult in applications where the device is expected to be worn throughout the day, such as accessibility.

[0005] Existing commercially available systems cannot support continuous usage scenarios. For example, wearable devices (e.g., smart glasses, smartwatches, head-mounted displays, etc.) are intended for intermittent use and are built around phone-class system-on-chips (SoCs). These devices may only provide a few hours of battery life with the display on. Additionally, due to the small size of head-mounted displays, heat comfort may be an issue. SUMMARY OF THE INVENTION

[0006] Exemplary embodiments include wearable devices and companion devices that use a split-compute architecture. To conserve the resources of the wearable device, the companion device can handle computing tasks that would otherwise be performed by the wearable device. The split-compute architecture facilitates the offloading of computing tasks to the companion device.

[0007] In a general aspect, a device, a system, a non-transitory computer-readable medium (which stores computer-executable program code that can be executed on a computer system), and / or a method can execute a process by a method that includes starting a computing process on a wearable device, the computing process including a plurality of tasks, the method including identifying a companion device and determining that the companion device is available to execute at least one of the plurality of tasks, causing the companion device to execute at least one task including communicating data generated by the wearable device to the companion device, receiving, by the wearable device, a result associated with completion of at least one task by the companion device, and completing, by the wearable device, the computing process based on the result associated with completion of at least one task.

[0008] Exemplary embodiments will become more fully understood from the detailed description given hereinbelow and the accompanying drawings, wherein like elements are represented by like reference numerals, which are given by way of example only and thus are not limiting of the exemplary embodiments.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

[0010] Note that these figures are intended to show the general characteristics of the methods and / or structures utilized in specific exemplary embodiments and to supplement the written description provided below. However, these drawings are not to scale and may not accurately reflect the exact structural or performance characteristics of any given embodiment, and should not be construed as defining or limiting the range of values or characteristics encompassed by the exemplary embodiments. For example, the positions of modules and / or structural elements may be reduced or exaggerated for clarity. The use of similar or identical reference numbers in the various drawings is intended to indicate the presence of similar or identical elements or features.

[0011] A wearable device may only provide a battery life of a few hours with the display on. For example, computationally expensive operations, such as image rendering for the optical system of a wearable display, distortion correction, location services, etc., cannot always be efficiently executed on a low-power embedded system implemented in a wearable device. Therefore, the display architecture of a sink client wearable device (e.g., smart glasses) can have the opportunity to reduce the device's on-board power and thermal footprint by offloading computationally intensive operations (e.g., graphics operations) to a companion device (e.g., a mobile device, smartphone, server, etc.).

[0012] Some wearable devices may have implementation constraints. For example, the implementation constraints of smart glasses shall include (1) that smart glasses need to amplify important services through wearable computing, which may include assistive technologies such as AR and vision. For example, the implementation constraints of smart glasses shall include (2) that smart glasses need to be able to be used throughout the day with a single charge. For example, the implementation constraints of smart glasses shall include (3) that smart glasses need to look and feel like actual glasses. Wearable devices can include extended reality (AR) and virtual reality (VR) devices. Wearable devices can include smart glasses, head-mounted devices, and / or head-mounted displays. Wearable devices, head-mounted devices, and / or head-mounted displays can be AR / VR devices. A fully stand-alone wearable device (e.g., smart glasses) solution with a mobile SoC having the ability to support desired features may not meet the power and industrial design constraints listed above. An on-device computing solution that meets constraints (1), (2), and (3) may be difficult to achieve with current technology. Current technology is limited in that a stand-alone solution with mobile system-on-chip (SoC) technology that meets the functional requirements does not meet the power and industrial design constraints listed above.

[0013] A split-compute architecture can be used to solve problems associated with the implementation of existing wearable device technologies that satisfy the aforementioned constraints. The split-compute architecture can be an architecture that moves the application runtime environment to a remote compute endpoint such as a smartphone, server, cloud, desktop computer, etc., which will often be referred to hereinafter in this specification as a companion device. In some embodiments, data sources such as inertial measurement units (IMUs) and camera sensors can be streamed from the wearable device to the companion device. In some embodiments, display content can be streamed from the companion device and returned to the wearable device. Continuing with the example of smart glasses, since most of the computing and rendering is not done on the smart glasses, the split-compute architecture enables the use of a low-power processor and / or a low-power microcontroller MCU-based system. In some embodiments, a split-compute architecture combined with a wearable device including an MCU can minimize power usage and satisfy constraints (1), (2), and (3). New technological innovations in codecs and networks make it possible to maintain the required network bandwidth in a low-power manner. In some embodiments, the wearable device can communicate with the companion device via a clearly defined protocol. This architecture can be platform-independent.

[0014] Figure 1 shows a block diagram of a high-level split compute architecture according to an exemplary embodiment. As shown in Figure 1, the split compute architecture can include a wearable device 105 and a companion device 110. In an exemplary embodiment, the wearable device 105 can be a smart glass, an extended reality / virtual reality headset, a head-mounted display (HMD), a smartwatch, a smart ring, etc. As an example, Figure 1 shows the wearable device 105 as a smart glass 135. In an exemplary embodiment, the companion device 110 can be another wearable device, a mobile device, a smartphone, a tablet, a server, and a device including a processor and an operating system. As an example, Figure 1 shows the companion device 110 as a smartphone 140.

[0015] The wearable device 105 and the companion device 110 can be communicatively coupled. For example, the wearable device 105 and the companion device 110 can be communicatively coupled either wired or wirelessly. In other words, the wearable device 105 and the companion device 110 can be endpoints of a bidirectional wired and / or wireless communication link. As an example, the wearable device 105 and the companion device 110 can communicate an audio stream and / or a video stream via a communication line 115. As an example, the wearable device 105 and the companion device 110 can communicate data (e.g., IMU, camera, input, etc.) via a communication line 120. As an example, the wearable device 105 and the companion device 110 can communicate a control stream via a communication line 125. The communication line 115, the communication line 120, and / or the communication line 125 can be collectively referred to as a communication line 130. In some embodiments, the communication line 130 can be bidirectional.

[0016] In some embodiments, the companion device 110 can include a runtime environment to which the wearable device 105 can connect. In some embodiments, the wearable device 105 can stream data, such as IMU and camera images, to the runtime environment. In some embodiments, the runtime environment of the companion device 110 can be configured to perform tracking, perception, and / or application rendering and / or deliver an output, such as an encoded video or a rendering command, back to the wearable device 105 via a graphics application programming interface (API). In some embodiments, the runtime environment can be code or a software application and / or container running on the companion device 110. In some embodiments, the runtime environment can be software operating on the companion device 110 while the wearable device 105 is simultaneously and / or while the wearable device 105 and the companion device 110 are communicatively coupled. In some embodiments, the input is captured from the wearable device 105 and introduced (e.g., communicated) into the runtime environment of the companion device 110.

[0017] As an example, in the case where there is no companion device 110, the computing process is executed entirely on the wearable device 105. The computing process can be any computing process associated with the functions of the wearable device. For example, the computing process can be a computing process configured to display content (e.g., as an image or video) on the display of the wearable device 105. The computing process can be any computing process that includes at least one task (or multiple tasks). The at least one task can be computer instructions (e.g., code) stored in the memory of the wearable device 105 and executed by the processor of the wearable device 105. The at least one task can be computer instructions (e.g., code) stored in the memory of the companion device 110 and executed by the processor of the companion device 110.

[0018] The at least one task can be computer instructions (e.g., code) configured to generate a result. The result can be an intermediate result or output of the computing process. The result can be the completion or final result or output of the computing process. The at least one task can be instructions executed by the processor of the wearable device 105. The at least one task can be instructions executed by the processor of the companion device 110. In some embodiments, the task(s) can be implemented as a service. The service can be, for example, a machine-to-machine interaction over a network. The service can be implemented as a background operation. For example, the service can execute network transactions, play audio, perform I / O, interact with content providers, etc. from the background.

[0019] By including the companion device 110 (e.g., the runtime environment of the companion device 110), the device onboard power usage and thermal footprint can be reduced by offloading one or more of the plurality of tasks to the companion device 110. In an exemplary embodiment, the plurality of tasks can be fragmented. Fragmenting the tasks can include systematically and / or randomly allocating the plurality of tasks between the wearable device 105 and the companion device 110. For example, systematically allocating the plurality of tasks between the wearable device 105 and the companion device 110 can be based on the resource usage. For example, if the amount of resources used to execute one task (or a plurality of tasks) on the companion device 110 (where executing a task includes performing a computer operation) is more than executing that one task (or a plurality of tasks) on the wearable device 105, then that one task (or a plurality of tasks) can be executed on the wearable device 105.

[0020] For example, one task (or a plurality of tasks) can include and / or use computer data (e.g., a high-resolution image captured by the camera of the wearable device 105). If communicating the computer data uses more resources (e.g., the battery resources of the wearable device 105) than a task that includes processing the data by the wearable device 105, then the wearable device 105 needs to be allocated to complete the task. Otherwise, the companion device 110 needs to be allocated to complete the task. In this exemplary embodiment, both the wearable device 105 and the companion device 110 can execute that one task (or a plurality of tasks). In some embodiments, two or more companion devices can be used to complete a process that includes a plurality of tasks.

[0021] FIG. 2 shows a block diagram of a high-level split compute architecture with a shared runtime environment, according to an exemplary embodiment. As shown in FIG. 2, the wearable device 105 is communicatively coupled to two or more companion devices 110-1, 110-2,... 110-n via communication lines 130-1, 130-2,... 130-n, respectively. In some embodiments, the wearable device 105 may roam among the various companion devices 110-1, 110-2,... 110-n and select the companion device 110-1, 110-2,... 110-n that provides the best experience at that time. The wearable device 105 can be simultaneously connected to the runtime of a plurality of companion devices 110-1, 110-2,... 110-n. Accordingly, the wearable device 105 can be simultaneously connected to a plurality of runtime environments.

[0022] For example, the runtime environment associated with the companion device 110-1 can be configured to project content onto the display of the wearable device 105, and the runtime environment associated with the companion device 110-2 can be configured to access a data source (e.g., a sensor) from the wearable device 105, a database server, the Internet, etc. for processing. Additionally or alternatively, the companion devices 110-1, 110-2,... 110-n can be communicatively coupled (e.g., wired or wirelessly) to share resources and / or data. For example, by communicatively coupling the companion device 110-1 and the companion device 110-2, the runtime environment associated with the companion device 110-1 can receive data from the runtime environment associated with the companion device 110-2 and be used when the runtime environment associated with the companion device 110-2 generates an image (or frame) to project content onto the display of the wearable device 105.

[0023] For example, by being able to communicatively couple the companion device 110-1 and the companion device 110-2 (e.g., via the communication line 205), the runtime environment associated with the companion device 110-2 can share processing resources with the runtime environment associated with the companion device 110-1. For example, the runtime environment associated with the companion device 110-1 may be instructed to generate content (e.g., an image, a frame, a map, etc.), and the content may be more efficiently generated by the runtime environment associated with the companion device 110-2. For example, the companion device 110-2 may include a map database and a map (e.g., an image) generator. The runtime environment associated with the companion device 110-1 may be instructed to generate content including a map. In this example, the runtime environment associated with the companion device 110-1 can request a map from the runtime environment associated with the companion device 110-2.

[0024] Continuing with the above example, if the companion device 110 is determined to execute a task (s), the companion device 110 (or the wearable device 105) can determine that two or more companion devices 110 can execute the task (s). For example, the task (s) can include generating content (e.g., an image) for display on the wearable device 105. In this example, the companion device 110-1 can be configured to generate the content. However, the data used to generate the content may be generated by the companion device 110-2 and then communicated from the companion device 110-2 to the companion device 110-1 (e.g., via the communication line 205). For example, the companion device 110-2 can be a wearable smartwatch configured to sense the user's heart rate. The data representing the heart rate can be communicated from the companion device 110-2 to the companion device 110-1. Next, the companion device 110-1 can use the data representing the heart rate to generate content, and this content can be communicated to the wearable device 105 for display on the display of the wearable device 105.

[0025] Figure 3 shows a block diagram of a split-compute architecture of a wearable device according to an exemplary embodiment. As shown in Figure 3, the wearable device split-compute architecture can include a hardware abstraction layer (HAL) 310 block. HAL 310 can be a layer of software configured to interface between an operating system (e.g., RTOS 340) and a hardware device at a general or abstract level rather than at the hardware level. In some embodiments, the abstraction layer can be used to isolate the split-compute architecture platform. HAL 310 can be called from an operating system kernel. In some embodiments, HAL 310 can be a virtual HAL. The virtual HAL can minimize the latency of interpretation based on the similarity in the architectures of the guest and host platforms. Virtualization techniques help map virtual resources to physical resources and use native hardware for computations in the virtual HAL.

[0026] Accordingly, by way of example, HAL 310 can include a connectivity 315 block, a codec 320 block, a graphics processing unit (GPU) 325 block, and a display 330 block, each of which can be configured to interface with a corresponding hardware device. For example, connectivity 315 can be configured to interface between an operating system (e.g., RTOS 340) and Bluetooth hardware, WIFI hardware, ultra-wideband (UWB) hardware, 5G hardware, etc. Accordingly, the wearable device split-compute architecture can be configured to utilize any connectivity hardware designed within the wearable device 105.

[0027] For example, codec 320 can be configured to interface between an operating system (e.g., RTOS 340) and an encoder and / or decoder (e.g., a hardware-based encoder and / or decoder). The codec 320 standard can be, for example, H.265, H.264, MPEG, VP9, machine learned, etc. The codec 320 can be an image, video, and / or audio codec. Thus, the split compute architecture of the wearable device can be configured to utilize any codec software and / or hardware designed within the wearable device 105. For example, GPU 325 can be configured to interface between an operating system (e.g., RTOS 340) and GPU hardware (e.g., a GPU ASIC). Thus, the wearable device split compute architecture can be configured to utilize any GPU designed within the wearable device 105 (e.g., to render an image on a display system). For example, display 330 can be configured to interface between an operating system (e.g., RTOS 340) and display hardware (e.g., a wearable device display). Thus, the wearable device split compute architecture can be configured to utilize any display (e.g., a display driver system) designed within the wearable device 105.

[0028] As shown in FIG. 3, a wearable device split compute architecture can include an operating system (OS) abstraction layer (OSAL) 335. In some embodiments, the abstraction layer can be used to isolate the split compute architecture platform. OSAL 335 can be configured to provide an interface to common system functions presented by the OS of wearable device 105. These OSALs 335 can simplify the development and porting of software (e.g., applications) to multiple OSs and hardware platforms. In some embodiments, OSAL 335 can operate as (or similar to) an application programming interface (API). In some embodiments, OSAL 335 can be platform dependent.

[0029] OSAL335 can include a real-time operating system (RTOS) 340 block. The RTOS 340 can be configured to process multi-threaded applications, for example, to meet real-time deadlines. For example, the RTOS 340 can be configured to process a plurality of tasks (or groupings of tasks) each having a maximum completion time. Although an RTOS is shown, any OS can be used. For example, a high-level operating system (HLOS) can be used. In a split compute system, by using tasks, these tasks (or groups of tasks) can be distributed among computing devices. For example, the content display operation of the wearable device 105 can be split between the wearable device 105 and the companion device 110 (e.g., the runtime environment associated with the companion device 110). In other words, the wearable device 105 (and / or the companion device 110) can be configured to cause the companion device 110 to execute a portion (one task or a grouping of tasks) of a process (e.g., content display), for example, using the split compute architecture of the wearable device.

[0030] As shown in FIG. 3, the wearable device split compute architecture can include a peripheral driver 345 block. The peripheral driver 345 can be configured to interface between the OS and the peripheral device. For example, the wearable device 105 can include a plurality of peripheral devices, such as, for example, one or more cameras, one or more microphones, one or more speakers, one or more inputs, one or more inertial measurement units (IMUs), etc. Accordingly, the peripheral driver 345 can be configured to interface between the RTOS 340 and the peripheral device(s) of the wearable device 105.

[0031] As shown in FIG. 3, the wearable device split compute architecture can include a device client 305. The device client 305 can be configured to control communication between the wearable device 105 and the companion device 110. For example, the device client 305 can be configured to generate, initialize, and control communication over the communication line 130. The communication line 130 can operate as, for example, a socket (e.g., a network socket, a TCP / IP network socket, etc.). A socket can be one endpoint of a bidirectional communication link between computer code (e.g., an application, a program, a software system, etc.) running on two computing devices. The socket mechanism can be configured to provide inter-process communication (IPC) by establishing named communication contact points between two endpoints and / or between two endpoints and an intermediate device (e.g., an access point (AP)). A socket can be configured to provide a bidirectional first-in first-out (FIFO) communication channel. Sockets that connect to a network are created at each end of the communication. As an example, each socket can have an address (or memory location). The address (or memory location) can be, for example, an IP address and a port number. Thus, the device client 305 can be configured to write to and read from a socket associated with the companion device 110 (or a runtime environment associated with the companion device 110). In some embodiments, the device client 305 can be platform independent.

[0032] In some embodiments, the software development kit (SDK) can be associated with the wearable device 105 and the companion device 110. The SDK can be used when developing applications for the wearable device 105 and / or the companion device 110. The SDK enables the implementation of a split compute architecture. Accordingly, any wearable device 105 and / or companion device 110 (regardless of the hardware and / or software platform) that includes a split compute architecture can use an application developed using the SDK. Accordingly, there is no need to develop an application and port it to each hardware and / or software platform that can be used as the wearable device 105 and / or the companion device 110. The SDK can be included with (or can have elements that are included with) the application when the application is installed on the wearable device 105 and / or the companion device 110.

[0033] Figure 4 shows a high-level split compute architecture according to an exemplary embodiment. As shown in Figure 4, the system can include a wearable device 105 and a companion device 110. The split compute architecture of the system can include a device client 305 block, an application 410 block, an SDK 415 block associated with the wearable device 105, and a core 420 block associated with the companion device 110. As described above, the device client 305 can be configured to control communication between the wearable device 105 and the companion device 110. The core 420 can be configured to control communication between the companion device 110 and the wearable device 105. Accordingly, the core 420 can be configured to at least generate, initialize, and control the communication line 130 and the communication via the communication line 130. The communication line 130 can operate as a socket, for example (as described above).

[0034] In some embodiments, the application 410 can include a file format for an application (e.g., an Android Package Kit (APK)) that is used on an operating system that holds application logic. In some embodiments, the wearable device application can link with a stub (e.g., an incomplete) version of the SDK to support compilation and test services. In some embodiments, at runtime, the application 410 can load the wearable device SDK 415 directly from the wearable device runtime environment. In some embodiments, the SDK 415 can provide developers with application programming interfaces (APIs) that are used to build wearable device applications. In some embodiments, an API versioning scheme can allow for the introduction of new APIs while maintaining backward compatibility. In some embodiments, the wearable device runtime environment can be a collection of core services that are responsible for maintaining the wearable device execution environment. In some embodiments, the wearable device application may not interact directly with the core services. In some embodiments, one or more interactions may be via the SDK. In some embodiments, the device client 305 can be a sink client that runs on the wearable device 105 hardware.

[0035] As an example, application 410 can be configured to generate (or assist in generating) content for display on wearable device 105. For example, the application can be configured to process task(s). For example, the application can generate content (e.g., as an image) and communicate the content to core 420 via SDK 415. Communicating the content to core 420 via SDK 415 can be one of the features that enables application 410 to be developed for any hardware and / or software platform. For example, SDK 415 can be configured to communicate with application 410 when application 410 is developed. Also, SDK 415 can be configured to communicate with core 420 associated with multiple hardware and / or software platforms. After receiving the content, core 420 can communicate the content to wearable device 105 via device client 305, for example, using a pre-established socket.

[0036] As described above, in some embodiments, wearable device 105 can be configured to connect to more than one companion device 110 at a given time. In some embodiments, different companion devices 110 can be configured to provide different services (e.g., using application 410). In some embodiments, a low latency, high bandwidth 5G connection is predominant, but companion device 110 can be configured to operate in the cloud (e.g., connected by 5G standards).

[0037] FIG. 5 shows a block diagram of an activity element within a wearable device application according to an exemplary embodiment. As shown in FIG. 5, the application 410 can include a wearable activity 505 block, a wearable activity service 510 block, and a wearable activity host 515 block, and the core 420 can include a core service 520 block.

[0038] In some exemplary embodiments, the design of the wearable device application can be similar to an activity model. Referring to FIG. 3, the RTOS 340 can be configured to process one task, a plurality of tasks (or groupings of tasks) each having a maximum completion time. Thus, each activity can be a task that is executed in a parallel process and has a time (or amount of time) to be completed.

[0039] In an exemplary embodiment, an activity can be executed in a service context. In some embodiments, by executing in a service context, the application 410 can be enabled to execute in parallel with the application of the companion device 110. In some embodiments, when the display of the companion device 110 is off, the application can continue execution and rendering.

[0040] In some embodiments, one or more applications 410 can include a wearable activity service 510. Since the wearable activity service 510 can be configured to expose a wearable activity 505, the wearable activity 505 can be instantiated by the SDK 415. Thus, the wearable activity 505 can be instantiated at a later point in time. From the developer's perspective, the wearable activity service 510 can be boilerplate code that is not directly related to the logic of the application 410. The wearable activity 505 can be code related to the logic of the application 410. In some embodiments, the wearable activity 505 can be managed by an activity manager and can behave similarly to the activity counter part of a standard OS.

[0041] In some embodiments, service binding can be used by the wearable device runtime environment to start and manage the lifecycle of the application 410. When an activity manager binds to a service as part of the launch flow, the SDK 415 can instantiate and attach a wearable activity host 515 as a class that can, for example, take on the functions of general activity state control. For example, during initialization, the wearable activity host 515 can request a surface from the window manager. This surface is then used as the backing store for a virtual display that is used to render the content of the application 410.

[0042] FIG. 6 shows a block diagram of a wearable device application within the runtime environment of a wearable device, according to an exemplary embodiment. As shown in FIG. 6, the companion device OS 605 can process two or more application environments simultaneously. For example, the companion device OS 605 can include an application module 610. The application module 610 can be associated with standard OS application activities. Further, the companion device OS 605 can include an application module 630 that operates in relation to the wearable runtime environment 625. The wearable runtime environment 625 can be associated with the wearable device 105.

[0043] Activities 620, 640 can be a single focused task that an application can execute. For example, some applications can include a user interface (UI). Accordingly, activities 620, 640 can be configured to create a window for placing the UI. The window can be a full-screen window, a floating window, one embedded in another window, a hidden window, etc. Different types of windows can be associated with different activities 620, 640. Activities 620, 640 can be configured for any task, and the window is just an example.

[0044] Activity managers 615, 635 can be configured to communicate information about and interact with activities 620, 640. Further, activity managers 615, 635 can be configured to communicate information about and interact with tasks, threads, services, and other processes.

[0045] In an exemplary embodiment, the wearable runtime environment 625 can be a virtual runtime environment. The virtual runtime environment can be configured to operate in the background of a computing device. Accordingly, the wearable runtime environment 625 can be a virtual runtime environment that is associated with the wearable device 105 and configured to operate as a background process on the companion device 110. In other words, the wearable runtime environment 625 can operate without presenting a user interface on the display of the companion device 110. For example, the wearable runtime environment 625 can operate within a hidden window of the companion device 110. Accordingly, a user of the companion device 110 may not have visual or I / O control of an application using the wearable runtime environment 625 if the wearable runtime environment 625 is a virtual runtime environment.

[0046] In an alternative or additional embodiment, the application 410 and / or the application module 630 can be a virtual process. The virtual process can be configured to operate in the background of a computing device. Accordingly, the application 410 and / or the application module 630 can be a virtual process that is associated with the wearable device 105 and configured to operate as a background process on the companion device 110. In other words, the application 410 and / or the application module 630 can operate without presenting a user interface on the display of the companion device 110. For example, the application 410 and / or the application module 630 can operate within a hidden window of the companion device 110. Accordingly, a user of the companion device 110 may not have visual or I / O control of the application 410 and / or the application module 630 if the process is a virtual runtime process.

[0047] FIG. 7 shows a block diagram of a system using a split compute architecture, according to an exemplary embodiment. As shown in FIG. 7, the system includes a wearable device 105 and a companion device 110. The wearable device 105 can include a device client 305, a codec 320, a GPU 325, a display 330, and a peripheral driver 345. The companion device 110 can include a core 420, an application module 610 having an encoder 705, and an application module 630 having an application 410. This exemplary embodiment can be used to describe a signal flow associated with generating and displaying content on the wearable device 105 using the split compute architecture described herein. This is only one of many possible system configurations and uses, and other configurations can be used and are within the scope of the present disclosure.

[0048] In this exemplary embodiment, application 410 can be configured to generate content for display on wearable device 105. Application 410 can be configured to generate content based on data received from wearable device 105. For example, peripheral driver 345 can sense and communicate data as peripheral data (e.g., IMU data) to application module 630 via device client 305 and core 420 using communication line 120. In other words, peripheral data can be collected by a peripheral device via peripheral driver 345 and / or processed thereby (e.g., compressed, packaged, parsed, filtered, noise removed, etc.) and can be data that is packaged for communication with and use by wearable device 105 and / or companion device 110. Application 410 can generate content, for example, as a rendered image and / or frame. Encoder 705 can compress the image and / or frame (e.g., using the H.264 codec). The compressed image and / or frame can be communicated to wearable device 105 via core 420 and device client 305 using communication line 115. Codec 320 can decompress the image and / or frame (e.g., using the H.264 codec). GPU 325 can cause the image and / or frame to be displayed on display 330.

[0049] Example 1. FIG. 8 is a block diagram of a method of operating a split computing system including a wearable device and a companion device communicatively coupled to the wearable device according to an exemplary embodiment. As shown in FIG. 8, in step S805, a computing process is initiated on the wearable device, and the computing process includes a plurality of tasks. For example, the computing process can be initiated by a user of the wearable device providing a corresponding user input (e.g., a gesture) to the wearable device. In some embodiments, the wearable device can initiate the computing process based on other computing processes, based on the spatial location of the wearable device, etc. In step S810, the companion device is determined to be available for executing at least one of the plurality of tasks. For example, the companion device can be identified by the wearable device as an available companion device in a list of potentially available companion devices stored in the memory of the companion device. Alternatively, the available companion device can be any companion device in the vicinity of the wearable device that can establish a connection with the wearable device via a two-way or three-way handshake (e.g., the handshake can be bidirectional in that both the client and the host determine which functions each supports). The available companion device can send a signal of a list of tasks that the companion device can execute to the wearable device, or the wearable device can already recognize the tasks that a particular type of companion device can execute. In step S815, the wearable device communicates data generated by the wearable device to the companion device.For example, the data can include images of an IMU and a camera, and / or metadata related to the images. In step S820, the companion device executes the at least one task. For example, when the companion device receives the data, it can automatically execute the at least one task. In step S825, the companion device communicates a result associated with the completion of the at least one task. Here, the term "result" can be used to refer to the processed data. In step S830, the wearable device receives the result associated with the completion of the at least one task. In step S835, the wearable device completes the computing process based on the result associated with the completion of the at least one task. Here, the wearable device can combine the processed data received from the companion device with further processed data associated with the completion of the remaining tasks of the plurality of tasks by the wearable device.

[0050] Example 2. The companion device can be a first companion device, the at least one task can be at least one first task, the system can further include a second companion device, and the system can be configured to cause the second companion device to execute at least one second task of the plurality of tasks, the method according to Example 1. Causing a device to execute a task can include sending instructions configured to initiate and / or trigger the processing of the task by the device and / or other device(s).

[0051] Example 3. The first companion device and the second companion device can be communicatively coupled, the method according to Example 2.

[0052] Example 4. The method according to Example 1, wherein the wearable device can be smart glasses.

[0053] Example 5. The method according to Example 1, wherein the companion device can be at least one of other wearable devices, mobile devices, smartphones, tablets, servers, and devices including a processor and an operating system.

[0054] Example 6. The method according to Example 1, wherein the runtime environment can be a virtual runtime environment that operates as a background process on the companion device.

[0055] Example 7. The method according to Example 6, wherein the at least one task can include an application, and the at least one task can further include processing an output of the application.

[0056] Example 8. The method according to Example 7, wherein processing the output of the application can include rendering an image on a display of the wearable device.

[0057] Example 9. The method according to Example 7, wherein the device client can be configured to control communications associated with the application.

[0058] Example 10. The method according to Example 1, wherein the computing process can be initiated by the wearable device.

[0059] Example 11. The method according to Example 1, wherein the computing process can be initiated by the companion device.

[0060] Example 12. FIG. 9 is a block diagram of a method for operating a wearable device according to an exemplary embodiment. As shown in FIG. 9, in step S905, a computing process is started on the wearable device, and the computing process includes a plurality of tasks. In step S910, a companion device is identified and it is determined that the companion device is available for performing at least one of the plurality of tasks. In step S915, the companion device is to perform the at least one task including communicating data generated by the wearable device. In step S920, the wearable device receives a result associated with completion of the at least one task by the companion device. In step S925, the wearable device completes the computing process based on the result associated with the completion of the at least one task.

[0061] Example 13. The result is a first result, and the method may further include receiving, by the wearable device, a second result associated with processing of the at least one task by the companion device; processing, by the wearable device, the second result as a processed second result; and communicating, by the wearable device to the companion device, the processed second result, wherein the result associated with the completion of the at least one task by the companion device can be based on the processed second result, the method according to Example 12. Here, the term second result can be used to refer to intermediate processed data, and the term first result can be used to refer to finally processed data.

[0062] Example 14. The computing process is the method according to Example 12, which can be initiated by the wearable device.

[0063] Example 15. The computing process can be initiated by the companion device, and starting the computing process on the wearable device can include receiving a trigger from the companion device by the wearable device. The method is the method according to Example 12.

[0064] Example 16. The data can be associated with a peripheral device of the wearable device. The method is the method according to Example 12.

[0065] Example 17. The wearable device can include a first socket, and the companion device can include a second socket communicatively coupled to the first socket. The method includes instructing the companion device to execute the at least one task and writing the data to the first socket, and further can include reading the result associated with the completion of the at least one task from the second socket. The method is the method according to Example 12.

[0066] Example 18. The result can include an image, and the method further includes displaying the image on a display of the wearable device. The method is the method according to Example 12.

[0067] Example 19. The wearable device can be smart glasses. The method is the method according to Example 12.

[0068] Example 20. The method according to Example 12, wherein the companion device can be at least one of other wearable devices, mobile devices, smartphones, tablets, servers, and devices including a processor and an operating system.

[0069] Example 21. The companion device can include a virtual runtime environment, and causing the companion device to execute the at least one task can include causing the virtual runtime environment to execute the computing process, the result can be the completion of the computing process, and causing the wearable device to complete the computing process that can be based on the result can include rendering an image on a display of the wearable device. The method according to Example 12.

[0070] Example 22. FIG. 10 is a block diagram of a method of operating a companion according to an exemplary embodiment. As shown in FIG. 10, in step S1005, the companion device determines that it is available to execute at least one task of a plurality of tasks associated with a computing process. In step S1010, the companion device receives data associated with the computing process generated by the wearable device. In step S1015, the companion device executes the at least one task. In step S1020, the companion device communicates a result associated with the completion of the at least one task.

[0071] Example 23. The above result is the first result, and the method can further include: the companion device communicating to the wearable device a second result associated with the processing of the at least one task by the companion device; the companion device receiving from the wearable device the processed second result; and the companion device completing the at least one task based on the processed second result, where the first result is based on the processed second result, the method according to Example 22.

[0072] Example 24. The computing process can be started by the wearable device, the method according to Example 22.

[0073] Example 25. The computing process can be started by the companion device, and the method can further include: the companion device communicating to the wearable device a trigger associated with starting the computing process, the method according to Example 22.

[0074] Example 26. The data can be associated with a peripheral device of the wearable device, the method according to Example 22.

[0075] Example 27. The wearable device can include a first socket, the companion device can include a second socket communicatively coupled to the first socket, and the method can further include: causing the companion device to execute the at least one task, including reading the instructions and the data from the second socket, and communicating the result associated with the completion of the at least one task, including writing the result to the second socket, the method according to Example 22.

[0076] Example 28. The method according to Example 22, which can include an image in the result. Example 29. The method according to Example 22, wherein the wearable device can be smart glasses.

[0077] Example 30. The method according to Example 22, wherein the companion device can be at least one of other wearable devices, mobile devices, smartphones, tablets, servers, and devices including a processor and an operating system.

[0078] Example 31. The method according to Example 22, wherein the companion device can include a virtual runtime environment, causing the at least one task to be executed on the companion device can include causing the plurality of tasks to be executed on the virtual runtime environment, and the result can be the completion of the plurality of tasks.

[0079] Example 32. The method can include any one or a combination of one or more of Examples 1 to 31.

[0080] Example 33. A non-transitory computer-readable storage medium including instructions, wherein the instructions are stored in the non-transitory computer-readable storage medium and, when executed by at least one processor, are configured to cause a computing system to execute the method according to any one of Examples 1 to 32.

[0081] Example 34. An apparatus including means for executing the method according to any one of Examples 1 to 32.

[0082] Example 35. An apparatus comprising at least one processor and at least one memory having computer program code, wherein the at least one memory and the computer program code are configured to cause the at least one processor to at least execute the method according to any one of Examples 1 to 32 in the apparatus.

[0083] Exemplary embodiments can include a non-transitory computer-readable storage medium having instructions, which are stored on the non-transitory computer-readable storage medium and configured to cause a computing system to execute any of the above methods when executed by at least one processor. Exemplary embodiments can include an apparatus having means for executing any of the above methods. Exemplary embodiments can include an apparatus comprising at least one processor and at least one memory having computer program code, wherein the at least one memory and the computer program code are configured to cause the at least one processor to at least execute any of the above methods in the apparatus.

[0084] The various embodiments of the systems and techniques described herein can be implemented in digital electronic circuits, integrated circuits, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementations in one or more computer programs executable and / or interpretable in a programmable system including at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0085] These computer programs (also known as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented in high-level procedural and / or object-oriented programming languages and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., magnetic disks, optical disks, memory, programmable logic circuits (PLDs)) used to provide machine instructions and / or data to a programmable processor that includes a machine-readable medium that receives the machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0086] To provide for interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., an LED (light emitting diode), or OLED (organic LED), or LCD (liquid crystal display) monitor / screen) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and input received from the user can be in any form including acoustic, speech, or tactile input.

[0087] The systems and techniques described in this application can be implemented in a computing system that includes backend components (e.g., as a data server), or includes middleware components (e.g., an application server), or includes frontend components (e.g., a client computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described in this application), or can be implemented in any combination of such backend, middleware, or frontend components. The components of the system can be interconnected by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), and the Internet.

[0088] A computing system can include clients and servers. Clients and servers are generally located far apart from each other and typically interact through a communication network. The relationship between clients and servers results from computer programs that are executed on respective computers and have a client-server relationship with each other.

[0089] Some embodiments have been described. It will be understood that various modifications can be made without departing from the spirit and scope of this specification.

[0090] Furthermore, the logic flows shown in the figures do not require the particular order shown, i.e., a sequential order, to achieve the desired results. Further, other steps may be provided in the described flows, steps may be eliminated from the described flows, other components may be added to the described systems, or other components may be removed from the described systems. Accordingly, other embodiments are within the scope of the following claims.

[0091] As described herein, while specific features of the described embodiments have been illustrated, numerous modifications, substitutions, changes, and equivalents will occur to those skilled in the art. Accordingly, it is to be understood that the appended claims are intended to cover all such modifications and changes that fall within the scope of the embodiments. It is to be understood that these are not limitations but are presented by way of example only, and that various changes in form and detail may be made. Any part of the apparatus and / or method described herein may be combined in any combination, except mutually exclusive combinations. The embodiments described herein may include various combinations and / or sub-combinations of the functions, components, and / or features of the different embodiments described.

[0092] Exemplary embodiments may include various modified forms and alternative forms, but the embodiments are shown by way of example in the drawings and described in detail herein. However, while there is no intention to limit the exemplary embodiments to the specific forms disclosed, it is to be understood that the exemplary embodiments are intended to cover all modified forms, equivalents, and alternatives that fall within the scope of the claims. Like numbers refer to like elements throughout the description of the drawings.

[0093] Some of the above exemplary embodiments are described as a process or method shown as a flowchart. In the flowchart, the operations are described as a sequential process, but many of the operations may be performed in parallel, concurrently, or simultaneously. Also, the order of the operations may be rearranged. The process may end when those operations are completed, but may have additional steps not included in the figure. The process may correspond to a method, function, procedure, subroutine, subprogram, etc.

[0094] The above methods, some of which are illustrated by flowcharts, may be implemented by hardware, software, firmware, middleware, microcode, a hardware description language, or any combination thereof. When implemented in software, firmware, middleware or microcode, the program code or code segments for performing the necessary tasks may be stored in a machine or computer-readable medium such as a storage medium. The processor(s) may execute the necessary tasks.

[0095] The specific structural and functional details disclosed herein are merely representative for the purpose of describing exemplary embodiments. However, the exemplary embodiments may be embodied in many alternative forms and should not be construed as limited to only the embodiments described herein.

[0096] The terms, first, second, etc. may be used herein to describe various elements, but it is understood that these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the scope of the exemplary embodiments, the first element may be referred to as the second element, and similarly, the second element may be referred to as the first element. As used herein, the term and / or includes any and all combinations of one or more of the associated listed items.

[0097] It will be understood that when an element is referred to as being connected or coupled to another element, it can be directly connected or coupled to the other element, or intervening elements may be present. In contrast, when an element is referred to as being directly connected to or directly coupled to another element, no intervening elements are present. Other words used to describe the relationship between elements should be interpreted in a similar manner (e.g., between vs. directly between, adjacent vs. directly adjacent, etc.).

[0098] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments. As used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. The terms "comprises", "comprising", "includes" and / or "including", when used herein, specify the presence of stated features, elements, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, elements, steps, operations, elements, components, and / or groups thereof.

[0099] Also, in some alternative embodiments, note that the functions / acts shown may occur in a different order than shown in the figures. For example, two figures shown in succession may, in fact, be executed simultaneously or, depending on the functions / acts involved, may be executed in the reverse order.

[0100] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which exemplary embodiments belong. Further, terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and should not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0101] The foregoing exemplary embodiments and corresponding detailed description portions have been presented with respect to software or algorithms and symbolic representations of operations on data bits within a computer memory. These descriptions and representations are what enable those skilled in the art to effectively convey the substance of their work to other skilled artisans. An algorithm, as the term is used herein and as commonly used, is considered to be a self-consistent sequence of steps leading to a desired result. These steps require physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical, magnetic, or optical signals capable of being stored, transferred, combined, compared, or otherwise manipulated. For primarily reasons of common usage, it has proven convenient at times to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, and the like.

[0102] In the foregoing exemplary embodiments, references to acts and symbolic representations of operations (e.g., in the form of flowcharts) that may be implemented as program modules or functional processes include routines, programs, objects, components, data structures, etc., which may be described and / or implemented using existing hardware on existing structural elements to perform a particular task or implement a particular abstract data type. Such existing hardware may include, for example, one or more central processing units (CPUs), digital signal processors (DSPs), application specific integrated circuits, field programmable gate arrays (FPGAs), computers, and the like.

[0103] However, it should be noted that all of these and similar terms are associated with appropriate physical quantities and are merely convenient labels applied to these quantities. Unless otherwise specifically stated, or as apparent from the discussion, terms such as processing or computing or calculating or determining or displaying refer to actions and processes of a computer system or similar electronic computing device that manipulate data represented as physical electronic quantities within the registers and memories of the computer system and convert them into other data similarly represented as physical quantities within the memories or registers of the computer system or other such information storage, transmission, or display devices.

[0104] Also, it should be noted that software implementations of exemplary embodiments are typically encoded on some form of non-transitory program storage medium or implemented via some type of transmission medium. The program storage medium may be magnetic (e.g., a floppy disk or hard drive) or optical (e.g., a compact disc read-only memory, or CD ROM), and may be read-only or random access. Similarly, the transmission medium may be a twisted pair, coaxial cable, optical fiber, or any other suitable transmission medium known in the art. Exemplary embodiments are not limited by these aspects of any given embodiment.

[0105] Finally, in the appended claims, specific combinations of features described herein are recited, but the scope of the disclosure is not limited to the specific combinations hereinafter claimed in the claims. Instead, it should be noted that the scope is extended to encompass any combination of features or embodiments disclosed herein, whether or not that specific combination is specifically recited in the appended claims at this time.

Claims

1. A method comprising: starting a computing process on a wearable device, the computing process including a plurality of tasks, the method comprising: identifying a companion device and determining that the companion device is available for performing at least one of the plurality of tasks; causing the companion device to perform the at least one task including communicating data generated by the wearable device to the companion device; receiving, by the wearable device, a result associated with completion of the at least one task by the companion device; completing, by the wearable device, the computing process based on the result associated with completion of the at least one task; A method comprising the above.

2. The result is a first result, and the method further comprises: receiving, by the wearable device, a second result associated with processing of the at least one task by the companion device; processing, by the wearable device, the second result as a processed second result; communicating, by the wearable device, the processed second result to the companion device, wherein the first result is based on the processed second result. The method according to claim 1.

3. The computing process is started by the wearable device. The method according to claim 1 or 2.

4. The computing process is started by the companion device, and starting the computing process on the wearable device includes receiving a trigger from the companion device by the wearable device. The method according to claim 1 or 2.

5. The data is associated with a peripheral device of the wearable device. The method according to any one of claims 1 to 4.

6. The wearable device includes a first socket, and the companion device includes a second socket communicatively coupled to the first socket.

7. The wearable device includes a first socket, and the companion device includes a second socket communicatively coupled to the first socket. The wearable device includes a first socket, and the companion device includes a second socket communicatively coupled to the first socket. Causing the companion device to execute the at least one task includes writing the instructions and the data to the first socket. The method according to any one of claims 1 to 5, wherein receiving the result associated with the completion of the at least one task includes reading the result from the second socket. **Claim 7** The method according to any one of claims 1 to 6, wherein the result includes an image, and the method further includes displaying the image on a display of the wearable device. **Claim 8** The method according to any one of claims 1 to 7, wherein the wearable device is smart glasses. **Claim 9** The method according to any one of claims 1 to 8, wherein the companion device is at least one of another wearable device, a mobile device, a smartphone, a tablet, a server, and a device including a processor and an operating system. **Claim 10** The companion device includes a virtual runtime environment. Causing the companion device to execute the at least one task includes causing the virtual runtime environment to execute the computing process. The method according to any one of claims 1 to 9, wherein the result is the completion of the computing process, and the method further includes causing the wearable device to complete the computing process based on the result, and the computing process includes rendering an image on a display of the wearable device. **Claim 11** A system comprising: a wearable device, and a companion device, wherein the wearable device includes: a device client, a hardware abstraction layer, an operating system abstraction layer, and at least one peripheral device driver, and the companion device includes a runtime environment associated with the wearable device, and the system includes: starting a computing process on the wearable device, the computing process including a plurality of tasks, and the system includes: determining that the companion device is available for executing at least one of the plurality of tasks. The wearable device communicates data generated by the wearable device to the companion device, the companion device executes the at least one task, the companion device communicates a result associated with completion of the at least one task, the wearable device receives the result associated with the completion of the at least one task, the wearable device completes the computing process based on the result associated with the completion of the at least one task, A system configured to perform.

12. The companion device is a first companion device, and the at least one task is at least one first task, the system further includes a second companion device, The system according to claim 11, wherein the system is configured to cause the second companion device to execute at least one second task of the plurality of tasks.

13. The system according to claim 12, wherein the first companion device and the second companion device are communicatively coupled.

14. The system according to any one of claims 11 to 13, wherein the wearable device is smart glasses.

15. The companion device is at least one of another wearable device, a mobile device, a smartphone, a tablet, a server, and a device including a processor and an operating system. The system according to any one of claims 11 to 14.

16. The system according to any one of claims 11 to 15, wherein the runtime environment is a virtual runtime environment that operates as a background process on the companion device.

17. The at least one task includes an application, The system according to claim 16, wherein the at least one task further includes processing an output of the application.

18. The system according to claim 17, wherein processing the output of the application includes rendering an image on a display of the wearable device.

19. The system according to claim 17, wherein the device client is configured to control communications associated with the application.

20. The system according to any one of claims 11 to 19, wherein the computing process is initiated by the wearable device.

21. The system according to any one of claims 11 to 19, wherein the computing process is initiated by the companion device.

22. Determining, by the companion device, that the companion device is available to execute at least one task of a plurality of tasks associated with a computing process; Receiving, by the companion device from the wearable device, data generated by the wearable device and associated with the computing process; Executing, by the companion device, the at least one task; Communicating, by the companion device to the wearable device, a result associated with completion of the at least one task; A method comprising:

23. The result is a first result, and the method comprises: Communicating, by the companion device to the wearable device, a second result associated with processing of the at least one task by the companion device; Receiving, by the companion device from the wearable device, the processed second result; Completing, by the companion device, the at least one task based on the processed second result, wherein the first result is based on the processed second result. The method according to claim 22.

24. The method according to claim 22 or 23, wherein the computing process is initiated by the wearable device.

25. The computing process is initiated by the companion device, and the method further comprises communicating, by the companion device to the wearable device, a trigger associated with initiating the computing process. The method according to claim 22 or 23.

26.

22. ​ The data is the method according to any one of claims 22 to 25, associated with a peripheral device of the wearable device. **Claim 27** The wearable device includes a first socket, The companion device includes a second socket communicatively coupled to the first socket, The method includes: Causing the companion device to execute the at least one task includes reading the instructions and the data from the second socket; Communicating the result associated with completion of the at least one task includes writing the result to the second socket; The method according to any one of claims 22 to 26, further comprising. **Claim 28** The method according to any one of claims 22 to 27, wherein the result includes an image. **Claim 29** The method according to any one of claims 22 to 28, wherein the wearable device is smart glasses. **Claim 30** The method according to any one of claims 22 to 29, wherein the companion device is at least one of another wearable device, a mobile device, a smartphone, a tablet, a server, and a device including a processor and an operating system. **Claim 31** The companion device includes a virtual runtime environment that operates as a background process on the companion device, Causing the companion device to execute the at least one task includes causing the virtual runtime environment to execute the plurality of tasks, The method according to any one of claims 22 to 30, wherein the result is the completion of the plurality of tasks. **Claim 32** A non-transitory computer-readable storage medium including instructions, The instructions are stored in the non-transitory computer-readable storage medium and configured to cause a computing system to execute the method according to any one of claims 1 to 10 when executed by at least one processor. **Claim 33** A non-transitory computer-readable storage medium including instructions, The command is stored in the non-transitory computer-readable storage medium and, when executed by at least one processor, is configured to cause a computing system to execute the method according to any one of claims 22 to 31, which is a non-transitory computer-readable storage medium.

34. An apparatus comprising means for executing the method according to any one of claims 1 to 10.

35. An apparatus comprising means for executing the method according to any one of claims 22 to 31.

36. At least one processor, and At least one memory having computer program code, An apparatus comprising: The at least one memory and the computer program code are configured to cause the at least one processor to cause the apparatus to execute at least the method according to any one of claims 1 to 10.

37. An apparatus, At least one processor, and At least one memory having computer program code, and The at least one memory and the computer program code are configured to cause the at least one processor to cause the apparatus to execute at least the method according to any one of claims 22 to 31.

Images