System and method for remote interaction with cloud-based client applications
By emulating client applications on the cloud platform and streaming media in real time, the limitations of client applications running across platforms are solved, seamless interaction on different devices and operating systems is achieved, and application distribution and interaction are simplified. Experience.
Patent Information
- Application Number
- CN202280092899.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2021-12-30
- Filing Date
- 2022-12-28
- Publication Date
- 2024-12-03
AI Technical Summary
Existing client applications are limited by operating systems and hardware platforms and cannot run seamlessly between different devices and operating systems, limiting user access and use.
Streaming video and audio in real-time to the user's browser by emulating client applications on a server or cloud platform, allowing users to interact with cloud-based client applications through the browser, supporting running on any operating system and hardware .
Achieving cross-platform client application interaction, reducing hardware and software requirements for user devices, simplifying application distribution, allowing users to try and switch different applications on any device, and enhancing third-party applications and websites Interactive function.
Smart Images

Figure CN119072680A_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 266,189, filed on December 30, 2021, the entire contents of which are incorporated herein by reference. Technical Field
[0003] The development and distribution of client applications designed to run on mobile devices has increased significantly to support the surge in mobile devices. Client applications can provide users with a wide variety of functions and features and access to a large amount of information to meet almost any needs or interests. Although client applications provide users with flexibility and portability, client applications may be limited by both the operating system running on the mobile device and the mobile device itself. For example, such client applications can only run in the mobile device operating system designed for it. Therefore, users with mobile devices running a specific operating system may not be able to access or use client applications designed to run in another mobile device operating system and / or on different mobile device hardware. These mobile device users also cannot access or use client applications designed to run on completely different (non-mobile) hardware platforms and operating systems. In addition, users of mobile devices with hardware and / or software requirements that do not support locally installed client applications may not be able to access those client applications without the need to upgrade to the latest mobile devices expensively. Client applications are also inherently bound to mobile devices and cannot be run or accessed via other non-mobile devices. Summary of the invention
[0004] The present invention relates to a system and method for remotely interacting with a cloud-based client application. According to the present invention, a client application can be remotely simulated on a server or other cloud-based platform. Video and audio from a cloud-based client application can be streamed in real time to a browser application running on a user's client device. A user can interact with a cloud-based client application through a local browser application, just as the cloud-based client application runs natively on the client device. Input from the user can be transmitted back to the cloud-based client application executed in an application engine to allow the user to control and interact with the cloud-based client application in real time. The present invention can support interacting with any type of cloud-based client application on any type of client device (e.g., smart phone, tablet computer, laptop, desktop computer, game console, etc.) running any type of operating system and independent of the underlying hardware and device characteristics of the client device. Embodiments of the present invention can also support integrating cloud-based client applications into third-party applications and websites. This integration can allow these third-party applications and websites to more easily provide different features and functions through integrated cloud-based client applications, thereby supplementing and enhancing interaction and participation with third-party applications and websites.
[0005] Systems and methods are provided for enabling various devices to remotely interact with cloud-based client applications. Related devices, techniques, and articles are also described.
[0006] In one aspect, a first request may be received from a first client device of a user to initiate an interactive session with a cloud-based client application. The cloud-based client application may be configured to execute on a second client device, and the second client device may include a computing platform different from the first client device. In response to the first request, an application engine may be reserved from a pre-instantiated application engine pool in a plurality of pre-instantiated application engine pools for remotely executing the cloud-based client application from the first client device. In addition, when the user engages with first media data associated with the cloud-based client application, interactive data may be received from the first client device. The cloud-based client application executed within the reserved application engine may be modified based on the interactive data received from the first client device. A second request may be received from the first client device to end the interactive session with the modified cloud-based client application. The application engine that can be reserved may be released in response to the second request. The application engine that can be reserved may be unlinked from the first client device.
[0007] In addition, the first media data can be streamed from the cloud-based client application to the first client device. In addition, the second media data from the modified cloud-based client application can be streamed to the browser application executed on the first client device. In addition, the application engine that can be scheduled can be linked to the first client device for the duration of the interactive session, and the pre-instantiated application engine can be added to the pre-instantiated application engine pool to replace the application engine that can be scheduled. Each pre-instantiated application engine pool in the plurality of pre-instantiated application engine pools can be associated with a different cloud-based client application. The first request can include at least one characteristic associated with the first client device. The at least one characteristic can characterize a geographic region associated with the first client device, and the modification of the cloud-based client application can include presenting one or more visual features specific to the geographic region associated with the first client device on the first client device. In addition, input from a user can be received before the second request from the first client device to end the interactive session, and in response to the input, the interactive session with the cloud-based client application at a specific location can be paused. The pause can be different from the end of the interactive session. Additional input can be received for resuming the interactive session on a third client device. Note that in response to receiving the additional input, the interactive session at the particular location may be resumed on the third client device.
[0008] In another aspect, a system may include at least one data processor and a memory storing instructions that, when executed by the at least one data processor, cause the at least one data processor to perform operations. The operations may include receiving a first request from a first client device of a user to initiate an interactive session with a cloud-based client application, wherein the cloud-based client application may be configured to execute on a second client device, and wherein the second client device may include a computing platform different from the first client device. In addition, the operations may include reserving an application engine from a pre-instantiated application engine pool of a plurality of pre-instantiated application engine pools in response to the first request for remote execution of the cloud-based client application from the first client device. The operations may also include receiving interaction data from the first client device when the user engages with first media data associated with the cloud-based client application. In addition, the operations may include modifying the cloud-based client application executed within the reservable application engine based on the interaction data, receiving a second request from the first client device to end the interactive session with the modified cloud-based client application; and releasing the allocated reservable application engine in response to the second request. The reservable application engine may be unlinked from the first client device.
[0009] In yet another aspect, a non-volatile computer program product stores executable instructions that implement various operations when executed by at least one data processor forming part of at least one computing system. The operations may include receiving a first request from a first client device of a user to initiate an interactive session with a cloud-based client application, wherein the cloud-based client application may be configured to execute on a second client device, and wherein the second client device may include a computing platform different from the first client device. In addition, the operations may include reserving an application engine from a pre-instantiated application engine pool of a plurality of pre-instantiated application engine pools in response to the first request for remote execution of the cloud-based client application from the first client device. The operations may also include receiving interaction data from the first client device when the user engages with first media data associated with the cloud-based client application. In addition, the operations may include modifying the cloud-based client application executed within the reserved application engine based on the interaction data, receiving a second request from the first client device to end the interactive session with the modified cloud-based client application, and releasing the allocated reservable application engine in response to the second request. The reservable application engine may be unlinked from the first client device.
[0010] A computer system is also described, which may include one or more data processors and a memory connected to one or more data processors. The memory may temporarily or permanently store instructions that cause at least one processor to perform one or more operations described herein. In addition, these methods may be implemented by one or more data processors distributed in a single computing system or between two or more computing systems. Such computing systems may be connected, and data and / or commands or other instructions may be exchanged by one or more connections, by direct connections between one or more of the computing systems, etc., including connections by a network (e.g., the Internet, wireless wide area network, local area network, wide area network, wired network, etc.). BRIEF DESCRIPTION OF THE DRAWINGS
[0011] The above embodiments will be more fully understood from the following detailed description in conjunction with the accompanying drawings. The accompanying drawings are not intended to be drawn to scale. For clarity, not every component will be labeled in every drawing. In the drawings:
[0012] Figure 1 is a block diagram illustrating an exemplary cloud-based client application platform for remotely interacting with a cloud-based client application;
[0013] Figure 2 is a block diagram illustrating an exemplary cloud-based client application platform having multiple application engine module pools for remotely interacting with cloud-based client applications;
[0014] Figure 3 is a block diagram illustrating an exemplary cloud-based client application platform having multiple application engine module pools and multiple application instance module pools for remotely interacting with cloud-based client applications;
[0015] Figure 4 is a flow chart illustrating an exemplary method for remotely interacting with a cloud-based client application;
[0016] Figure 5 is a flow chart illustrating an exemplary method for remotely interacting with a cloud-based client application;
[0017] Figure 6 is another flow chart illustrating an exemplary method for remotely interacting with a cloud-based client application; and
[0018] Figure 7 is a block diagram of an exemplary computing device that may perform one or more of the operations described herein, according to present embodiments. DETAILED DESCRIPTION
[0019] Certain exemplary embodiments will now be described to provide an overall understanding of the structure, function, manufacture and use of the device disclosed herein and the principle of the method. One or more examples of these embodiments are shown in the accompanying drawings. It will be appreciated by those skilled in the art that the devices and methods specifically described herein and shown in the accompanying drawings are non-restrictive exemplary embodiments, and the scope of the present invention is limited only by the claims. The features shown or described in conjunction with an exemplary embodiment can be combined with the features of other embodiments. Such modifications and variations are intended to be included within the scope of the present invention. In addition, in the present invention, the eponymous components of the embodiments generally have similar features, so in a specific embodiment, each feature of each eponymous component may not be fully described.
[0020] The present invention relates to a system and method for remotely interacting with a cloud-based client application. According to the present invention, a client application can be remotely presented on a simulated client device on a cloud-based client application platform. Video and audio from a cloud-based client application can be streamed in real time to a browser or other application running on a user's client device. A user can interact with a cloud-based client application through a local browser or application, just as the client application runs locally on the client device. Input from the user can be transmitted back to a remote simulation device to allow the user to control and interact with the cloud-based client application in real time. The present invention can support remote interaction with any appropriate type of cloud-based client application on any type of client device (e.g., smart phone, tablet computer, laptop, desktop computer, game console, etc.) running any type of operating system and independent of the underlying hardware and device characteristics of the client device. According to an embodiment of the present invention, a cloud-based client application can include a mobile game or any other appropriate type of client application designed to operate on a client device but capable of remotely presenting and remotely controlling on the client device. The present invention can simplify the process of client application distribution, because the user will not need to install each cloud-based client application, so it is easy to try many different cloud-based client applications, and it is easy to switch and move between cloud-based client applications on any client device or combination of client devices. Some implementations of the present invention can also support the integration of cloud-based client applications into third-party applications and websites. This integration can allow these third-party applications and websites to more easily provide different features and functions by integrating cloud-based client applications that complement and enhance the interaction and participation with third-party applications and websites. For the purpose of discussion and not limitation only, the present invention may refer to cloud-based mobile games as exemplary cloud-based client applications to illustrate various aspects of the present invention. However, the present invention can be used in and used with any appropriate type of client application that can be remotely presented and remotely controlled on a client device.
[0021] Figure 11 is a block diagram illustrating an exemplary cloud-based client application platform 100 for remote interaction with a cloud-based client application according to an embodiment of the present invention. The cloud-based client application platform 100 may include a server system 102. The server may be cloud-based (e.g., Amazon Web Services, Google Cloud, Microsoft Azure or other appropriate cloud-based infrastructure providers) or composed of dedicated server hardware. The server system 102 may include an application engine module controller 104. The application engine module controller 104 may include multiple application engine modules 106 to support any appropriate type of cloud-based client application. In an embodiment, each of the multiple application engine modules 106 may be configured to simulate a specific type and configuration of a client device for executing a cloud-based client application. For example, in some implementations of the present invention, each or any one of the application engine modules may include an appropriate client device simulator. Each cloud-based client application may be executed within a corresponding application engine module in the multiple application engine modules 106, just as a cloud-based client application is executed locally on a given client device. The application engine module controller 104 may allow registration / deregistration of application engine modules, atomic reservations of application engine modules, queries of application engine modules, etc. The plurality of application engine modules 106 may include a first application engine module 108, a second application engine module 114, a third application engine module 120, ..., and an Nth application engine module 126, where N may be any suitable natural number and may vary over time, as described below. Each application engine module 108, 114, 120, ... 126 may include a video / audio module for managing video and / or audio media data generated by a cloud-based client application executed in the corresponding application engine module, and for sending or otherwise streaming such video and / or audio media data to a client device for display to a user. Each application engine module 108, 114, 120, ... 126 may also include a state module for receiving and processing inputs from a client device generated when a user (remotely) interacts with a cloud-based client application (e.g., mapping inputs to video / audio streams, etc.). Each state module may also maintain the state of the cloud-based client application executing in the corresponding application engine module 108, 114, 120, ... 126. For example, the first application engine module 108 may include a first video / audio module 110 in communication with a first state module 112.The second application engine module 114 may include a second video / audio module 116 that communicates with the second state module 118. The third application engine module 120 may include a third video / audio module 122 that communicates with the third state module 124. The Nth application engine module 126 may include an Nth video / audio module 128 that communicates with the Nth state module 120. The application engine module controller 104 may maintain any appropriate number of application engine modules and register (i.e., instantiate, allocate, or otherwise create) or unregister (i.e., delete or release allocation) application engine modules as needed. In one embodiment, the server system 102 may include multiple application engine module controllers 104, each of which manages a corresponding plurality of application engine modules 106 to form one or more application engine module clusters.
[0022] In some embodiments of the present invention, each application engine module 108, 114, 120, ... 126 can be run as a basic container in a replica set in Kubernetes, etc., for example, using a webRTC bridge (a media channel for video / audio media data), OpenGL or other appropriate technologies to support real-time media communication capabilities with other appropriate applications running on a browser or client device. A replica set is a process that can run multiple instances of an application engine module and keep a specified number of application engine modules unchanged. A replica set can maintain a specified number of application engine module instances running in or with the application engine module controller 104 at any given time to prevent users from losing access to their cloud-based client applications when the application engine module fails or is inaccessible. In an embodiment, each application engine module 108, 114, 120, ... 126 can also include an Envoy agent (as either or both of an egress agent or an ingress agent) that can handle health checks, etc. and, for example, cross-origin resource sharing (CORS) requests or other similar requests. Additionally or alternatively, each application engine module 108, 114, 120, ... 126 may include a debug bridge for receiving and processing debug and other application engine commands (eg, from an application engine reservation controller 134 discussed below).
[0023] In an embodiment, the application engine module controller 104 may include a video / audio communication module 132 to support media communications between application engine modules in the plurality of application engine modules 106 and client devices. In an alternative embodiment, the video / audio communication module 132 may reside in the server system 102, but outside the application engine module controller 104, and communicate with the application engine module controller 104. Additionally or alternatively, each or any one of the plurality of application engine modules 106 may include or communicate with a corresponding video / audio communication module 132. In other words, each or any one of the plurality of video / audio communication modules 132 may reside in or be associated with each or any one of the plurality of application engine modules 106. In some implementations of the present invention, the video / audio communication module 132 may include a Traversal Using Relay NAT (TURN) server, etc. (e.g., a Session Traversal Utilities for NAT (STUN) server, etc.) to support communication of video / audio media data between the application engine module and the client device. Additionally or alternatively, the video / audio communication module 132 may include an appropriate encoder to encode the video into, for example, h.264 or VP8 for WebRTC, etc. However, other appropriate server communication protocols, encoding techniques, and / or transmission protocols for video / audio media data are feasible.
[0024] The server system 102 may include an application engine reservation controller 134 in communication with the application engine module controller 104. In an embodiment, the application engine reservation controller 134 may control various aspects of the plurality of application engine modules 106, including, for example, coordinating application engine module calibration, controlling application engine module access, managing application engine module initialization for users, etc. The application engine reservation controller 134 may include an application engine inventory and reservation module 136. The application engine inventory and reservation module 136 may track and manage an inventory of all application engine modules in the plurality of application engine modules 106. The application engine inventory and reservation module 136 may also grant application engine module reservations for use in active sessions. In some implementations of the invention, when a user requests to initiate an active session with a cloud-based client application, application engine inventory and reservation module 136 may reserve an appropriate application engine module, load or cause the reserved application engine module to be loaded with the requested cloud-based client application (if not already loaded or preloaded), populate or cause the loaded cloud-based client application to be populated with appropriate user and / or application data (e.g., user profile information, user account information, application state information, etc. to support the user's interaction and engagement with the cloud-based client application), and bind or otherwise link the reserved application engine module to the active session. If no application engine module is available for the active session, application engine inventory and reservation module 136 may instruct application engine module controller 104 (e.g., via application engine designator module 138 discussed below) to add and register a new application engine module for the requested active session.
[0025] In some implementations of the present invention, the registration process can be performed asynchronously. For example, the application engine inventory and reservation module 136 can wait for new application engine modules to be added and reserved by periodically polling the application engine module controller 104, or receiving appropriate messages from the application engine module controller 104 once the task is completed. Once the application engine module is registered, it can be reserved for an active session and bound to the active session. In some implementations of the present invention, the application engine module that has been reserved for an active session can only be accessed by the user who requested it, and only during the validity period of the active session. In other words, the application engine module that has been reserved for a specific active session may not be used or accessed by another user during the active session. In some implementations of the present invention, the data stored in the reservation application engine module or otherwise used by the reservation application engine module during the active session can be short-lived. For example, the application engine module can be deregistered at the end of the active session. Therefore, during the active session, any and all data stored or otherwise used by the application engine module can be deleted, for example, no later than the deregistration of the application engine module. In an alternative embodiment, data stored in or used by a subscribed application engine module during an active session can be stored by the user (e.g., in database 146), retrieved, and used in and with subsequent active sessions. Once an application engine module is deregistered, it can be reused for another active session, or its computing resources can be reallocated to a different application engine module.
[0026] In an embodiment, the application engine reservation controller 134 may include an application engine calibrator module 138 that communicates with the application engine inventory and reservation module 136. The application engine calibrator module 138 may respond to changes in inventory and reservations by adding and instantiating new application engine modules to the plurality of application engine modules 106. The application engine calibrator module 138 may proactively calibrate the number of application engine modules in the plurality of application engine modules 106 based on, for example, usage trends or other appropriate metrics to maintain a pool of reservable application engine modules of sufficient size. For example, the application engine calibrator module 138 may store a predetermined threshold value (e.g., provided or generated based on usage trends or other appropriate metrics) for the size of the pool of the plurality of application engine modules 106. If the number of reservable application engine modules decreases below a predetermined threshold value, the application engine calibrator module 138 may add a new application engine module to the plurality of application engine modules 106 to increase the number of reservable application engine modules above the predetermined threshold value. If the number of reservable application engine modules increases too much (e.g., above a second predetermined threshold), the application engine calibrator module 138 may either do nothing or may delete or otherwise release the additional application engine modules that are not needed for allocation from the plurality of application engine modules 106. Additionally or alternatively, appropriate machine learning / artificial intelligence techniques may be utilized by or in conjunction with the application engine calibrator module 138 to appropriately and dynamically calibrate the number of reservable application engine modules in the plurality of application engine modules 106. For example, a machine learning model may be trained based on usage data from a user (e.g., usage trends, peak usage, non-working hours usage, etc.). The machine learning model may then be used to dynamically add and delete / release allocations of application engine modules in the plurality of application engine modules 106. As usage data evolves over time, the machine learning model may be updated or otherwise adapted.
[0027] In some implementations of the present invention, multiple application engine modules 106 may be associated with or otherwise correspond to and communicate with one or more application engine module pools, which may be managed by, for example, an application engine inventory and reservation module 136 and / or an application engine calibrator module 138. For example, the application engine modules for each or any one of the simulated client devices supported by the cloud-based client application platform 100 may be maintained in a single large pool of application engine modules. Application engine modules requested by users for active sessions may be allocated, reserved, and instantiated from such a single pool of application engine modules by, for example, the application engine inventory and reservation module 136 and / or the application engine calibrator module 138. However, accessing and instantiating each application engine module from such a single large pool of application engine modules may create a potential bottleneck. For example, launching each application engine module individually may take up a significant amount of time and computing resources, especially when there are tens of thousands, hundreds of thousands, or even millions of computing users instantiating corresponding application engine modules for the cloud-based client application platform 100 at any given time. The resulting distribution delays may adversely affect the performance and user interaction of the cloud-based client application platform 100, which may lead to user dissatisfaction and even user churn.
[0028] Figure 2 2 is a block diagram illustrating an exemplary cloud-based client application platform 200 having multiple pools of application engine modules for remotely interacting with cloud-based client applications according to an embodiment of the present invention. In some embodiments of the present invention, each or any of the multiple application engine modules 206 can be associated with and communicate with a separate pool of application engine modules, which can be maintained by, for example, the application engine inventory and reservation module 136 and / or the application engine calibrator module 138, wherein each application engine module pool supports a separate type of application engine module or a group of related types of application engine modules. The multiple application engine modules 206 can be in a manner similar to that described with respect to Figure 1 The multiple application engine modules 106 operate in the manner discussed and described in the. Figure 2In the illustrated embodiment, each cloud-based client application can have its own separate application engine module pool or be associated with it. For example, each pool in a plurality of application engine module pools can be configured to support a single cloud-based client application, so that each pool in a plurality of application engine module pools can be allocated on the basis of each cloud-based client application. For the purpose of illustration and not limitation, a klondike solitaire mobile game designed to run on an Android smartphone can be supported by a first application engine module 108, which can be associated with a first application engine module pool 205 of a first application engine module. A klondike solitaire PC game designed to run on a PC laptop can be supported by a second application engine module 114, which can be associated with a second application engine module pool 210 of a second application engine module. A klondike solitaire mobile game designed to run on an iOS smartphone can be supported by a third application engine module 120, which can be associated with a third application engine module pool 215 of a third application engine module. A klondike solitaire game designed to run on a game console may be supported by the Nth application engine module 126, which may be associated with the Nth application engine module pool 220 of the Nth application engine modules, and so on. Alternatively, each of the plurality of application engine module pools may be configured to support a set of related application engine modules (e.g., application engine modules that emulate an Android smartphone, such as having a particular operating system version), such that each of the plurality of application engine module pools may be allocated per application engine module type. For example, any appropriate number of application engine module pools may be maintained and supported for any appropriate number of application engine modules by, for example, the application engine inventory and reservation module 136 and / or the application engine calibrator module 138, depending on the number of cloud-based client applications (or alternatively, application engine module types) supported by the system or any appropriate subset thereof.
[0029] In some implementations of the present invention, each or any one of the application engine modules in each of the plurality of application engine module pools may be "pre-warmed". For example, the application engine modules in each of the plurality of application engine module pools may be pre-allocated and pre-instantiated within the application engine module pool and then placed in a standby or sleep mode by, for example, the application engine inventory and reservation module 136 and / or the application engine calibrator module 138. By starting the application engine modules in each application engine module pool in the background and maintaining those started application engine modules in standby mode, the application engine modules may be immediately (or almost immediately) reserved and assigned to a user upon an active session request with little or no attendant delay, because waking up an application engine module in standby mode generally takes much less time than instantiating an application engine module from scratch. Once an application engine module in an application engine module pool is reserved and assigned, the reserved and assigned application engine module may be immediately (or almost immediately) replaced by another "pre-warmed" application engine module from the corresponding application engine module pool so that it may be ready for immediate (or almost immediate) reservation and assignment to the next user requesting an active session. This "warm-up" replacement process can continue as each application engine module in the application engine module pool is reserved and assigned. In this way, application engine modules from each of the multiple application engine module pools can be assigned to users in real time. In the present disclosure, "real time" can refer to a process that can occur instantly or in a short period of time (e.g., a few seconds) so that there is minimal delay between receiving a request and processing the request and providing a response. Once an active session ends and the reserved application engine module is no longer needed for the active session, the computing resources allocated to the reserved application engine module assigned to the active session can be deleted or otherwise released immediately (or almost immediately). In some implementations of the present invention, the released computing resources can be reallocated by, for example, the application engine inventory and reservation module 136 and / or the application engine calibrator module 138 to add and warm up another application engine module in an associated application engine module pool or a different application engine module pool, rather than maintaining computing resources for reserved application engine modules that are no longer needed for inactive sessions.
[0030] The size of each or any one of the multiple application engine module pools may be fixed or dynamic. For example, the size of the application engine module pool may be fixed so that a predetermined application engine module pool size number of preheated application engine modules can be maintained in the pool. When preheated application engine modules in the application engine module pool are allocated, the application engine inventory and reservation module 136 and / or the application engine calibrator module 138 may replace the allocated application engine modules with additional preheated application engine modules to maintain the size of the application engine module pool at or near a predetermined application engine module pool size number. The predetermined application engine module pool size number may be based on, for example, usage trends of a particular application engine module, historical usage information, or other appropriate metrics. Thus, application engine modules with high usage rates may have a larger predetermined application engine module pool size number, while application engine modules with low usage rates may have a smaller predetermined application engine module pool size number.
[0031] Additionally or alternatively, the size of the application engine module pool can be dynamic. In some embodiments of the present invention, the application engine inventory and reservation module 136 and / or the application engine calibrator module 138 can monitor the usage trend of each or any one of the multiple application engine modules 206 in real time. For example, as the usage trend of a particular application engine module increases, the application engine calibrator module 138 can increase the size of the associated application engine module pool (and therefore, increase the number of preheated application engine modules maintained in the application engine module pool) to accommodate the larger usage, so that the application engine module allocation can continue in real time without increasing delays. Conversely, as the usage trend of a particular application engine module decreases, the application engine calibrator module 138 can reduce the size of the associated application engine module pool (and therefore, reduce the number of preheated application engine modules maintained in the application engine module pool) to accommodate the smaller usage, so that the computing resources are not unnecessarily overused and can be allocated elsewhere. In this embodiment, the size of each or any one of the application engine module pools can be dynamically changed to accommodate periods of increased usage (e.g., peak usage) and intervals of usage to maintain real-time allocation of application engine modules.
[0032] In some embodiments of the present invention, the application engine inventory and reservation module 136 and / or the application engine calibrator module 138 may include a suitable prediction model or utilize appropriate machine learning / artificial intelligence techniques to determine the size of the application engine module pool of multiple application engine module pools. For example, a machine learning model may be trained based on usage trend data for each supported application engine module to modify the size of the application engine module pool of the application engine module, thereby minimizing the time required to allocate the application engine module and maintaining reduced allocation time for real-time application engine module allocation. The machine learning model may then be used to dynamically and appropriately modify the size of the application engine module pool associated with the application engine module based on the usage of the application engine module at a particular time. The machine learning model may be monitored by, for example, the application engine inventory and reservation module 136 and / or the application engine calibrator module 138, and updated or otherwise adjusted as the usage trend of the application engine module changes and evolves over time. In some implementations of the present invention, the size of each application engine module pool in a subset of multiple application engine module pools may be fixed, while the size of other application engine module pools in the multiple application engine module pools is dynamic. For example, application engine modules with high usage rates can be associated with an application engine module pool whose size is dynamic, while application engine modules with low usage rates can be associated with an application engine module pool whose size is fixed. Additionally or alternatively, the size of a particular application engine module pool can be fixed at certain times and dynamic at other times. For example, for application engine modules that are generally less utilized, the size of the corresponding application engine module pool can be fixed. However, during (e.g., intermittent) periods of higher usage (e.g., peak usage), the size of the corresponding application engine module pool can be dynamic. Once the period of higher usage ends, the application engine inventory and reservation module 136 and / or the application engine calibrator module 138 can switch the size of the application engine module pool back to fixed. In this way, the application engine inventory and reservation module 136 and / or the application engine calibrator module 138 can adapt to changing usage conditions as needed to maintain reduced allocation time for application engine modules in real time.
[0033] In some implementations of the present invention, Figure 2As shown, one or more application engine module pools may be maintained in and with the plurality of application engine modules 206. However, in some implementations of the invention, each or any of the one or more application engine module pools may be maintained in or with the application engine module controller 104, which is external to the plurality of application engine modules 206 but within the server system 102. Additionally or alternatively, each or any of the one or more application engine module pools may be maintained in or with the server system 102 but external to the application engine module controller 104 (e.g., in the database 146). Additionally or alternatively, each or any of the one or more application engine module pools may be maintained in or with a suitable cloud or server storage system that is remote from and in communication with the server system 102.
[0034] like Figure 1 and Figure 2 As shown, the server system 102 may include a session management controller 140 for managing active sessions. The session management controller 140 may communicate with the application engine reservation controller 134. In some embodiments of the present invention, the session management controller 140 may also be coupled to the application engine module controller 104 or otherwise communicate with the application engine module controller 104. The session management controller 140 may include a session manager module 142 for managing the allocation of application engine modules, including registration, deregistration, and calibration (e.g., communicating and coordinating with the application engine inventory and reservation module 136 and the application engine calibrator module 138). The session manager module 142 may also act as a coordinator for the server system 102 to support, for example, load balancing and latency. The session manager module 142 may receive a request to initiate an active session from a client device. In addition, the session manager module 142 may receive session information from the client device (e.g., in addition to the request or as part of the request), which may be used to select an appropriate application engine module. In an embodiment, the session information received from the client device may include, for example, one or more characteristics of the cloud-based client application requesting the active session, one or more characteristics of the client device (e.g., client device type, operating system, etc.), and other similar characteristics or data.
[0035] The session management controller 140 may include an application engine selector module 144 in communication with the session manager module 142. The session manager module 142 may pass the received session information to the application engine selector module 144, which may utilize the information to select an appropriate application engine module for the requested active session. For purposes of illustration and not limitation, a user with a non-Android client device (e.g., a PC laptop) may request an active session with a (cloud-based) klondike solitaire mobile game designed to run on an Android smartphone. Based on the session information passed to the application engine selector module 144, the application engine selector module 144 may select an Android application engine module for the (cloud-based) klondike solitaire mobile game for the active session. The application engine selector module 144 may communicate with the application engine inventory and reservation module 136. Therefore, the application engine selector module 144 can instruct the application engine inventory and reservation module 136 to reserve an Android application engine module for the (cloud-based) klondike solitaire mobile phone game from a plurality of application engine modules 106 or 206. Once reserved, the Android klondike solitaire mobile phone game application engine module can be bound to the requested active session so that no other user can use the reserved application engine module during the active session. The application engine selector module 144 can issue a command (e.g., delivered to the reserved Android mobile phone game application engine module via the application engine inventory and reservation module 136) to start the (cloud-based) klondike solitaire mobile phone game in the Android application engine module. Then, the user can start to interact remotely with the (cloud-based) klondike solitaire mobile phone game through their PC laptop computer, just as the klondike solitaire mobile phone game is executed locally on the PC laptop computer itself. If the cloud-based Android Klondike solitaire mobile game application engine module is not available, the application engine inventory and reservation module 136 can instruct the application engine calibrator module 138 to add and reserve the cloud-based Android Klondike solitaire client game application engine module to multiple application engine modules 106 or 206 in the above manner.
[0036] In some embodiments of the present invention, session management controller 140 (e.g., session manager module 142 and application engine selector module 144) can coordinate with application engine reservation controller 134 (e.g., application engine inventory and reservation module 136 and application engine identifier module 138) and control application engine reservation controller 134 to manage such Figure 2One or more application engine module pools shown and discussed. Additionally or alternatively, the session management controller 140 can directly coordinate with and control the application engine module controller 104 to manage one or more application engine module pools.
[0037] In an embodiment, because application engine reservation controller 134 can identify and maintain usage trends of, for example, application engine modules (e.g., per user, for all users, or any subset thereof), some implementations of the present invention can also recommend cloud-based client applications to each or any one user. In an embodiment, session manager module 142 can include a recommendation engine, etc. The recommendation engine can receive usage trend information from application engine reservation controller 134 and use the information to recommend different cloud-based client applications to users. For example, the recommendation engine can recommend the most popular cloud-based client applications (e.g., overall, in a particular type, for users using a particular type of client device, etc.), a list of the most popular cloud-based client applications at a particular time or within a time range (e.g., the past hour, the past day, the past week, etc.), etc. The recommendation engine can also recommend cloud-based client applications that are similar or complementary to one or more cloud-based client applications that the user has previously interacted with. For example, if the user has played the cloud-based Klondike solitaire mobile game, the recommendation engine can recommend other cloud-based games of the same or similar type (e.g., spider solitaire, tri peaks solitaire, etc.), regardless of the device platform on which the game is based (because the device platform will be simulated by one of the multiple application engine modules 106 or 206). Other recommendations are also feasible. Additionally or alternatively, the recommendation engine can use appropriate machine learning / artificial intelligence techniques to dynamically provide cloud-based client application recommendations to users. For example, a machine learning model can be trained based on usage data from users (e.g., usage trends, peak usage, non-working hours usage, etc. for all users or any subset thereof). The machine learning model can then be used to dynamically recommend cloud-based client applications to users based on usage trends. As usage data evolves over time, the machine learning model can be updated or otherwise adapted. The recommendation can be transmitted to the client device (e.g., as a URL or other link or hyperlink in a modal window or other appropriate notification or message). If the user accepts the recommendation (e.g., by selecting a URL or other link or hyperlink to a cloud-based client application), session manager module 142 may select and schedule the indicated application engine module in the manner described herein, thereby initiating an active session for the user.
[0038] Server system 102 may include one or more databases 146 that may be in communication with one or more of application engine module controller 104, application engine reservation controller 134, and session management controller 140. Database 146 may be cloud-based or reside in one or more physical storage systems. Database 146 may include, for example, any suitable information related to application engine modules, management of application engine modules, active session information, historical session information, usage trends or other metric data, cloud-based client application state data, copies of each or any one of the cloud-based client applications, input from client devices, session logs, and other similar data.
[0039] In some implementations of the invention, the cloud-based client application platform 100 or 200 may include one or more client devices 148 that communicate with the server system 102 (e.g., via the Internet or other appropriate network). According to an embodiment, the client device 148 may be any suitable type of device capable of running an appropriate browser or other appropriate application through which a user may interact (e.g., via a touch screen or pointing device, such as a mouse, stylus, etc.), such as a smartphone, tablet computer, laptop computer, desktop computer, game console, smart TV, smart watch, or any other suitable device capable of supporting such user input and interaction through a browser or other appropriate application. In an embodiment, the present invention may support interaction with any suitable type of cloud-based client application on any suitable number of client devices 148, each client device 148 running any suitable type of operating system, and independent of the underlying hardware and device characteristics of the client device 148.
[0040] According to an embodiment of the present invention, a cloud-based client application may be any suitable client application that is configured to execute locally on any suitable type of client device 148, but may be remotely executed and presented in the cloud in an application engine module of a plurality of application engine modules 106 or 206 of a server system 102. A cloud-based client application (e.g., a cloud-based mobile game or other web-based or suitable cloud-based client application) may be remotely provided as an end-user client application that a user may interact with via a server system 102. A cloud-based client application may relate to and / or provide a wide variety of functions and information, including, for example, entertainment (e.g., games, music, videos, etc.), business (e.g., word processing, accounting, spreadsheets, etc.), news, weather, finance, sports, etc. In a specific embodiment, a cloud-based client application may provide a digital game, such as a mobile digital game, a PC game, or a console game. The digital game may be or include, for example, a sports game, an adventure game, a virtual poker game, a virtual table game, a puzzle game, a racing game, or any other suitable type of digital game. In an embodiment, the digital game can be an asynchronous competitive skill-based game, wherein players can compete with each other in the digital game, but do not have to play the digital game at the same time. In an alternative embodiment, the digital game can be a synchronous competitive skill-based game, wherein players can play the digital game at the same time and can compete with each other in real time in the digital game. Other suitable cloud-based digital games are feasible, whether mobile-based, PC-based, console-based or other. For a cloud-based client application in which two or more users can compete with each other or otherwise participate, each user can interact with the cloud-based client application in a corresponding reservation and assigned application engine module, and compete, fight or otherwise participate or interact with each other using the corresponding application engine module and the back-end architecture supported by the cloud-based client application (e.g., a game server in the context of a digital game, etc.). Additionally or alternatively, one or more users interacting with the cloud-based client application in a corresponding reservation and assigned application engine module can compete, confront or otherwise participate or interact with a user interacting with a client application that is directly installed and executed locally on its client device (i.e., without using an application engine module). Thus, embodiments of the present invention may allow users to engage in interactions across a variety of different computing platforms, independent of the underlying client device 148 and the operating platform used by each or any one user for their client device 148 .
[0041] The client device 148 may include a browser module 150. The browser module 150 may support any suitable type of browser application, such as Google Chrome, Mozilla Firefox, Microsoft Edge, AppleSafari, DuckDuckGo, Opera, Brave, Vivaldi, Firefox Focus, proprietary browsers, etc. Additionally or alternatively, the browser module 150 may be a suitable application capable of supporting local rendering of video / audio information streamed (e.g., via OpenGL, etc.) from a cloud-based client application to the client device 148. In an embodiment, the browser module 150 may include a display module 152, an interaction capture module 154, and a session information module 156 that communicate with each other. The display module 152 may support the display of video and / or audio information streamed from a cloud-based client application running in a corresponding application engine module executing on the server system 102. In an embodiment, the video / audio output of the cloud-based client application running in the application engine module may be streamed to the browser module 150 on the client device 148 in real time via the Internet (or other suitable network) and displayed to the user via the display module 152. The display module 152 may receive the streamed video / audio output from the application engine module via the video / audio communication module 132 and present it appropriately on the display screen of the client device 148. The user may interact with the video / audio stream of the cloud-based client application in real time. In some implementations of the present invention, the user's input may be collected by the interaction capture module 154 and transmitted (via the Internet or other appropriate network) to the application engine module of the active session of the user in the plurality of application engine modules 106 or 206 through one or more signaling communication modules 158 (e.g., appropriate data and signaling channels, such as gRPC gateways, etc.). The user's input may then be mapped to the video / audio stream by the state module of the assigned application engine module to allow the user to (remotely) control the cloud-based client application and interact with it in real time, as if the cloud-based client application were running locally on the client device 148.
[0042] In an embodiment, the display module 152 may also provide a display of any or all of the cloud-based client applications available to the user (e.g., such as a URL or other link or hyperlink in a browser application, such as a link icon on a desktop in a user interface of the client device 148, etc.). The interaction capture module 154 may capture the user's selection of the desired cloud-based client application (e.g., by the user clicking or otherwise selecting one of the displayed URLs, links, hyperlinks, or icons). Once the user selects the desired cloud-based client application, the session information module 156 may initiate an active session by transmitting a request to the session management controller 140 (e.g., via a REST call, etc.). The session management controller 140 may then begin the process of reserving an appropriate application engine module for the active session, as described above. The session information module 156 may periodically (e.g., via REST, etc.) poll the session management controller 140 until an application engine module is reserved and assigned for the active session. Alternatively, the session management controller 140 may send an appropriate message or notification to the session information module 156 when an application engine module is reserved and assigned for the active session. Once the application engine module is reserved and assigned to the user's active session, the display module 152 on the client device 148 can connect to the reserved application engine module via a media (video / audio) channel (e.g., via the video / audio communication module 132) and begin receiving streamed video / audio media data from the cloud-based client application. As the user interacts with the video / audio media data streamed from the cloud-based client application executing within the reserved application engine module, the interaction capture module 154 can provide captured user input to the reserved application engine module via the signaling communication module 158 (e.g., webRTC signaling, via OpenGL, or other appropriate technology) to allow the user to (remotely) control and interact with the cloud-based client application.
[0043] Figure 3is a block diagram illustrating an exemplary cloud-based client application platform 300 having multiple application engine module pools and multiple application instance module pools for remotely interacting with cloud-based client applications according to an embodiment of the present disclosure. As previously discussed, each of the multiple application engine modules 206 can be configured to emulate a specific type and configuration of a client device for executing a cloud-based client application. Additionally or alternatively, the server system 102 can support cloud-based client application instances (also referred to as "application instances") that can be designed and configured to run locally in the cloud on the server system 102. Thus, the application instance can be an instance of a cloud-based client application that can be configured to run locally on the server system 302 without utilizing an application engine module from the multiple application engine modules 206 (or 106). Thus, the server system 102 can include a module controller 304 that can be configured to execute a module instance in a manner similar to that described with respect to the embodiment of the present disclosure. Figure 1 and Figure 2 The module controller 304 may include multiple application engine modules 206 and multiple application instance modules 310, and the multiple application engine modules 206 support and emulate any appropriate type of client device to execute any appropriate type of cloud-based client application (such as reference Figure 1 and Figure 2), the plurality of application instance modules 310 support and execute any suitable type of application instance that can be run locally on the server system 102. The module controller 304 can allow registration / deregistration of application instance modules, atomic retention of application instance modules, querying of application instance modules, etc. in a manner similar to that described for the plurality of application engine modules 206 (and 106). The plurality of application instance modules 310 can include a first application instance module 312, ..., and an Mth application instance module 320, where M can be any suitable natural number and can change over time. Each application instance module 312, ... 320 can include a video / audio module for managing video and / or audio media data generated by the application instance and sending the video and / or audio media data to a client device for display to a user. Each application instance module 312, ... 320 can also include a state module for receiving and processing input from the client device 148 generated when a user interacts with the application instance (remotely) (e.g., mapping the input to a video / audio stream, etc.). Each state module can also maintain the state of the application instance. For example, a first application instance module 312 may include a first video / audio module 314 that communicates with a first state module 316. An Mth application instance module 320 may include an Mth video / audio module 322 that communicates with an Mth state module 324. The module controller 304 may maintain any appropriate number of application instance modules, and register (i.e., instantiate, allocate, or otherwise create) or unregister (i.e., delete or release allocation) application instance modules as needed. In an embodiment, the server system 102 may include a plurality of module controllers 304, wherein each module controller 304 manages a corresponding plurality of application engine modules 206 to form an application engine module cluster, and manages a corresponding plurality of application instance modules 310 to form an application instance module cluster. The server system 102 and the module controller 304 may support any appropriate number and combination of application engine modules and application instance modules.
[0044] like Figure 3 As shown, the server system 102 may include a reservation controller 328 in communication with the module controller 304, which may be similar to the reservation controller 328 in the embodiment of the present invention. Figure 1 and Figure 2In an embodiment, the reservation controller 134 may also control various aspects of the plurality of application instance modules 310, including, for example, orchestrating application instance module calibration, controlling application instance module access, managing application instance module initialization for users, etc. The reservation controller 328 may include an inventory and reservation module 330, which may be similar to the one described with respect to Figure 1 and Figure 2The inventory and reservation module 330 may operate in the manner discussed and described above with respect to the application engine inventory and reservation module 136 of the present invention. The inventory and reservation module 330 may also track and manage the inventory of all application instance modules in the plurality of application instance modules 310. The inventory and reservation module 330 may grant reservations to application instance modules for use in an active session. In some embodiments of the present invention, when a user requests to initiate an active session with an application instance module, the inventory and reservation module 330 may reserve the appropriate application instance module, populate or cause the application instance module to be populated with appropriate user and / or application data (e.g., user profile information, user account information, application instance state data, etc., to support the user's interaction and participation with the application instance), and bind the reserved application instance module to the active session. If there are no application instance modules available for the active session, the inventory and reservation module 330 may instruct the module controller 304 (e.g., via the calibrator module 332 discussed below) to add and register a new application instance module for use in the requested active session. In some implementations of the present invention, the registration process may be performed asynchronously. For example, the inventory and reservation module 330 may wait for new application instance modules to be added and reserved by periodically polling the module controller 304, or receiving appropriate messages from the module controller 304 once a task has been completed. Once an application instance module is registered, it may be reserved for an active session and bound to the active session. In some implementations of the present invention, an application instance module that has been reserved for an active session may only be accessed by the user who requested it, and only during the validity period of the active session. In other words, an application instance module that has been reserved for a particular active session may not be used or accessed by another user during the active session. In some implementations of the present invention, data stored in or otherwise used by a reserved application instance module during an active session may be transient. For example, an application instance module may be deregistered at the end of an active session. Therefore, during the active session, any and all data stored or otherwise used by the application instance module may be deleted, for example, no later than the deregistration of the application instance module. In an alternative embodiment, data stored in or otherwise used by a reserved application instance module during an active session may be stored (e.g., in a database 146), retrieved, and used in a subsequent active session by a user. Once an application instance module is deregistered, it can be reused for another active session, or its computing resources can be reallocated to a different application instance module.
[0045] In an embodiment, the reservation controller 328 may include a calibrator module 332 that communicates with the inventory and reservation module 330, and the calibrator module 332 may communicate with the inventory and reservation module 330 in a manner similar to that described with respect to Figure 1 and Figure 2 The calibrator module 332 can operate in the manner discussed and described with respect to the application engine calibrator module 138 of . The calibrator module 332 can also respond to inventory and reservation changes by adding and instantiating new application instance modules to the plurality of application instance modules 310. The calibrator module 332 can actively calibrate the number of application instance modules in the plurality of application instance modules 310 based on, for example, usage trends or other appropriate metrics to maintain a pool of application instance modules of sufficient size that can be reserved. For example, the calibrator module 332 can store a predetermined threshold value (e.g., provided or generated based on usage trends or other appropriate metrics) for the size of the pool of the plurality of application instance modules 310. If the number of reservable application instance modules decreases below a predetermined threshold value, the calibrator module 332 can add new application instance modules to the plurality of application instance modules 310 to increase the number of reservable application instance modules to above a predetermined threshold value. If the number of reservable application instance modules increases too much (e.g., above a second predetermined threshold value), the calibrator module 332 can either do nothing, or can delete or otherwise release the allocation of unnecessary additional application instance modules from the plurality of application instance modules 310. Additionally or alternatively, appropriate machine learning / artificial intelligence techniques may be utilized by or in conjunction with calibrator module 332 to appropriately and dynamically calibrate the number of reservable application instance modules in plurality of application instance modules 310. For example, a machine learning model may be trained based on usage data from users (e.g., usage trends, peak usage, off-hours usage, etc.). The machine learning model may then be used to dynamically add and remove / release allocations of application instance modules in plurality of application instance modules 310. As usage data evolves over time, the machine learning model may be updated or otherwise adapted.
[0046] In some implementations of the present invention, each or any one of the multiple application instance modules 310 can be associated with and communicate with a separate pool of application instance modules, which can be maintained by, for example, an inventory and reservation module 330 and / or a calibrator module 332, wherein each application instance module pool supports a separate type of application instance module or a group of related types of application instance modules. In this embodiment, each application instance can have its own separate pool of application instance modules or be associated with it. For example, each pool of multiple application instance module pools can be configured to support a single application instance, so that each pool of multiple application instance module pools can be allocated on a per application instance basis. For purposes of illustration and not limitation, a klondike solitaire game designed to run locally on a server system 102 can be supported by a first application instance module 312, which can be associated with a first application instance module pool 318 of first application instance modules. A bingo game designed to run locally on server system 102 may be supported by Nth application instance module 320, which may be associated with Nth application instance module pool 326 of Nth application instance modules, etc. For example, inventory and reservation module 330 and / or calibrator module 332 may maintain and support any suitable number of application instance module pools for any suitable number of application instance modules, depending on, for example, the number of application instances supported by the system or any suitable subset thereof.
[0047] In some implementations of the present invention, each or any one of the application instance modules in each of the plurality of application instance module pools may be "pre-warmed". For example, the application instance modules in each of the plurality of application instance module pools may be pre-allocated and pre-instantiated within the application instance module pool and then placed in a standby or hibernation mode by, for example, the inventory and reservation module 330 and / or the calibrator module 332. By starting the application instance modules in each of the application instance module pools in the background and maintaining these started application instance modules in standby mode, the application instance modules may be immediately (or almost immediately) reserved and assigned to a user upon an active session request with little or no attendant delay, because the time required to wake up an application instance module in standby mode is much less than that of an application instance module instantiated from scratch. Once an application instance module in an application instance module pool is reserved and assigned, the reserved and assigned application instance module may be immediately (or almost immediately) replaced by another "pre-warmed" application instance module in the corresponding application instance module pool so that it may be ready for immediate (or almost immediate) reservation and assignment to the next user requesting an active session. This "warmed up" replacement process can continue as each application engine module in the pool of application instance modules is reserved and assigned. In this manner, application instance modules from each of the multiple application instance module pools can be assigned to users in real time. Once an active session ends and the reserved application instance modules are no longer needed for the active session, the computing resources allocated to the reserved application instance modules assigned to the active session can be deleted or otherwise released immediately (or nearly immediately). In some implementations of the invention, for example, the inventory and reservation module 330 and / or the calibrator module 332 can reallocate the released computing resources to add and warm up another application instance module in an associated application instance module pool or a different application instance module pool, rather than maintaining computing resources for reserved application instance modules that are no longer needed for an inactive session.
[0048] The size of each or any one of the multiple application instance module pools may be fixed or dynamic. For example, the size of the application instance module pool may be fixed so that a predetermined application instance module pool size number of preheated application instance modules can be maintained in the pool. When preheated application instance modules in the application instance module pool are allocated, the inventory and reservation module 330 and / or the calibrator module 332 may replace the allocated application instance modules with additional preheated application instance modules to maintain the size of the application instance module pool at or near a predetermined application instance module pool size number. The predetermined application instance module pool size number may be based on, for example, usage trends or other historical usage information of a particular application instance module. Therefore, a highly utilized application instance module may have a larger predetermined application instance module pool size number, while a less utilized application instance module may have a smaller predetermined application instance module pool size number.
[0049] Additionally or alternatively, the size of the application instance module pool can be dynamic. In some implementations of the present invention, the inventory and reservation module 330 and / or the calibrator module 332 can monitor the usage trend of each or any one of the multiple application instance modules 310 in real time. For example, as the usage trend of a particular application instance module increases, the calibrator module 332 can increase the size of the associated application instance module pool (therefore, increase the number of preheated application instance modules maintained in the application instance module pool) to accommodate the higher usage, so that the application instance module allocation can continue in real time without more delays. Conversely, as the usage trend of a particular application instance module decreases, the calibrator module 332 can reduce the size of the associated application instance module pool (therefore, reduce the number of preheated application instance modules maintained in the application instance module pool) to accommodate the lower usage, so that computing resources are not unnecessarily overused and can be allocated elsewhere. In this embodiment, the size of each or any one of the application instance module pools can be dynamically changed to accommodate periods of increased usage (e.g., peaks of usage) and troughs of usage, thereby maintaining application instance module allocation in real time.
[0050] In some embodiments of the present invention, the inventory and reservation module 330 and / or the calibrator module 332 may include an appropriate prediction model, or utilize appropriate machine learning / artificial intelligence techniques to determine the size of the application instance module pool of the multiple application instance module pools. For example, a machine learning model may be trained based on usage trend data for each supported application instance module to modify the size of the application instance module pool of the application instance module, thereby minimizing the amount of time used to allocate the application instance module and maintaining a reduced allocation time for real-time application instance module allocation. The machine learning model may then be used to dynamically and appropriately modify the size of the application instance module pool associated with the application instance module based on the usage of the application instance module at a particular time. The machine learning model may be updated or otherwise adjusted as the usage trend of the application instance module changes and evolves over time, as monitored by, for example, the inventory and reservation module 330 and / or the calibrator module 332. In some implementations of the present invention, the size of each application instance module pool in a subset of the multiple application instance module pools may be fixed, while the sizes of other application instance module pools in the multiple application instance module pools are dynamic. For example, application instance modules with high usage rates can be associated with an application instance module pool whose size is dynamic, while application instance modules with low usage rates can be associated with an application instance module pool whose size is fixed. Additionally or alternatively, the size of a particular application instance module pool can be fixed at certain times and dynamic at other times. For example, for application instance modules that typically experience lower usage rates, the size of the corresponding application instance module pool can be fixed. However, during periods of (e.g., intermittent) higher usage rates (e.g., peak usage), the size of the corresponding application instance module pool can be dynamic. Once the period of higher usage ends, the inventory and reservation module 330 and / or the calibrator module 332 can switch the size of the application instance module pool back to fixed. In this way, the inventory and reservation module 330 and / or the calibrator module 332 can adapt to changing usage conditions as needed to maintain a reduced allocation time for real-time allocation of application instance modules.
[0051] In some implementations of the present invention, Figure 3As shown, one or more application instance module pools may be maintained in and with multiple application instance modules 310. However, in some implementations of the invention, each or any of the one or more application instance module pools may be maintained in or with module controller 304, outside of multiple application instance modules 310 but within server system 102. Additionally or alternatively, each or any of the one or more application instance module pools may be maintained in or with server system 102, but outside of module controller 304 (e.g., in database 146). Additionally or alternatively, each or any of the one or more application instance module pools may be maintained in or with a suitable cloud or server storage system remote from and in communication with server system 102.
[0052] In some implementations of the present invention, the reserved and assigned application engine modules and / or the reserved and assigned application instance modules may each be associated with a different unique module identification (ID) number or other suitable identifier (e.g., including numbers or alphanumeric characters, etc.) that uniquely identifies the reserved and assigned application engine modules and / or the reserved and assigned application instance modules during an active session. For example, the session management controller 140 may utilize the module ID number to track and manage the reserved and assigned application engine modules and / or the reserved and assigned application instance modules during an active session to ensure that communications from the client device 148 in the active session are addressed and forwarded to the correct application engine module or application instance module, and vice versa, etc. In an embodiment, when the active session is completed, the session information module 156 may send an appropriate message to the session management controller 140 so that the session management controller 140 may disconnect the active session and deregister the application engine modules or application instance modules used during the active session. In some embodiments of the present invention, when the application engine module or application instance module is deregistered, any and all data stored or otherwise used by the registered application engine module or registered application instance module during the active session may be deleted. In alternative embodiments, data stored in or otherwise used by a subscribed application engine module or subscribed application instance module during an active session can be stored (e.g., in database 146), retrieved, and used by the user in subsequent active sessions. Once an application engine module or application instance module has been deregistered, it can be reused for another active session, or its computing resources reallocated to a different application engine module or application instance module.
[0053] In some implementations of the invention, a cloud-based client application (which is executed in an application engine module or an application instance module) may be paused during an active session. For example, a user may pause the cloud-based client application at any particular or desired location, point, or time during an active session with the cloud-based client application (e.g., by pressing an appropriate "pause" button displayed to the user in or with the browser application). Additionally or alternatively, the cloud-based client application may be paused if the connection between the browser module 150 and the server system 102 is interrupted or lost during an active session. Additionally or alternatively, the cloud-based client application may be paused if the user leaves the active session (e.g., by pressing a browser "back" button or closing the browser application). In some implementations of the invention, one or more appropriate cache files (cookies) may be stored on the client device 148 (e.g., in or associated with the browser module 150), which the browser module 150 may check to determine if there are any active sessions for the user. Such information may be transmitted to the session management controller 140 via the session information module 156. Once the connection is reestablished or the user returns to the active session (e.g., by pressing an appropriate "play" button displayed to the user in or using the browser, pressing a "forward" button, or reopening the browser application), the user's interaction with the cloud-based client application at the point of suspension can be resumed. In an embodiment, when the active session is suspended or if the user logs out or disconnects from the active session, the active session can remain open or otherwise maintained for a predetermined period of time (e.g., 1 hour, 2 hours, 1 day, etc.). After the predetermined period of time expires, the active session can be ended and the application engine module or application instance module can be logged off. If the user wishes to continue the active session before the predetermined period of time expires, the user can, for example, press a "play" button, refresh a web page in a browser application, or select the same URL, link, or hyperlink used to launch the original active session to rejoin the previously assigned and reserved application engine module or application instance module. In some implementations of the present invention, the active session can remain open or otherwise maintained indefinitely until the user logs out or otherwise ends the active session.
[0054] By maintaining the state of the reserved application engine module or reserved application instance module and the cloud-based client application during the pause, some implementations of the present invention may allow a user to start an active session on one client device 148, pause the active session at a specific or desired location, point, or time, and then resume the active session from the paused location, point, or time on one or more other (different) client devices 148. For purposes of illustration and not limitation, a user may request to initiate an active session on a first client device 148 (e.g., a smartphone). The user may then pause the active session at a specific or desired location, point, or time (e.g., by closing the browser application on the smartphone). The state of the cloud-based client application may be saved in the state module of the application engine module or application instance module assigned and registered for the active session. The user may then switch to a second client device 148 (e.g., a desktop computer) and select the same URL, link, or hyperlink used to initiate the original active session on the first client device 148 (via the display module 152 in the browser module 150 running on the second client device 148). This action may resume the active session in the browser module 150 of the second client device 148. Thus, the user may be presented with the same digital page of an active session of a cloud-based client application that was paused at the same location, point, or time as the first client device 148. The user may seamlessly continue to interact with the same cloud-based client application via the second client device 148 from the location, point, or time where it was paused in the same or similar manner as the user continued to interact with the cloud-based client application on the first client device 148 (e.g., by pressing an appropriate "play" button in the browser application). To support such platform switching, the session management controller 140 may maintain a user ID or other appropriate identifier that uniquely identifies the user in addition to the module ID. Utilizing both the user ID and the module ID in conjunction with the cloud-based client application selection from the user (received from the second client device 148) and the saved state of the cloud-based client application, the session management controller 140 may identify the correct (paused) application engine module or application instance module to reallocate to the second client device 148, thereby allowing the user to continue the active session on the second client device 148 from the same location, point, or time where it was paused on the first client device 148. In an alternative embodiment, if the user selects the same cloud-based client application from the same URL, link, or hyperlink on a second client device 148, the user can be presented with the option to either continue the same active session with the cloud-based client application from the same paused location, point, or time, or to start a new active session with the cloud-based client application.
[0055] In the event of an interruption or crash of an application engine module or an application instance module during an active session, session management controller 140 can reserve and assign a new application engine module or a new application instance module (depending on which was being used for the active session before the interruption or crash) to the user. The cloud-based client application running in the application engine module or application instance module can be populated with information from the previous active session (e.g., using the cloud-based client application state and other session information retrieved from database 146) to reflect the state of the cloud-based client application at the time just before the interruption or crash, so that the active session appears to the user to be merely paused. The user can then return to and re-engage in the active session in the manner previously described.
[0056] In some implementations of the invention, the session management controller 140 (e.g., via the session manager module 142) can operate in conjunction with the browser module 150 (e.g., via one or both of the display module 152 or the interaction capture module 154) to measure and monitor network delay information or other appropriate communication characteristics between the client device 148 and the server system 102 during an active session, and communicate the network delay information (e.g., via the session information module 156) to the session management controller 140. If the network delay is above a predetermined or dynamic threshold (e.g., based on network delay trends, usage trends, etc.), the session management controller 140 can use the network delay information to temporarily suspend the active session and then resume the active session once the network delay drops below the threshold. Additionally or alternatively, the network delay information can be used by the server system 102 (e.g., via the video / audio communication module 132) for network connection quality management. For example, the video / audio communication module 132 can use the network delay information to determine the quality of the network connection to the client device 148. If the network connection quality is below a predetermined or dynamic threshold (e.g., based on network latency trends, usage trends, etc.), the video / audio communication module 132 may modify the quality and / or characteristics of the video / audio media data being streamed to the client device 148. For example, if the network connection quality is below the threshold, the video / audio communication module 132 may reduce the frame rate, resolution, etc. of the streamed video / audio media data in order to maintain smooth display and interaction of a cloud-based client application on the client device 148 during the active session (e.g., to maintain smooth game play on the client device 148 in the context of a cloud-based mobile game) so that the active session does not need to be paused or otherwise interrupted. The video / audio communication module 132 may periodically or continuously update the quality and / or characteristics of the video / audio media data stream throughout the active session (e.g., reduce and / or increase the frame rate, resolution, etc. of the video / audio media data stream as the network connection quality fluctuates during the active session). Thus, server system 102 may utilize network latency information to determine, change, and control the streaming quality of video / audio to client device 148 to maintain smooth display and interaction with cloud-based client applications for the user.
[0057] Additionally or alternatively, the session management controller 140 may utilize network delay information to determine and select an appropriate application engine module or application instance module for the user. For example, in some implementations of the present invention, the server system 102 may include multiple application engine module controllers 104, wherein each application engine module controller 104 manages a corresponding plurality of application engine modules 106 to form one or more application engine module clusters, as shown in FIG. Figure 1 and Figure 2 In some implementations of the present invention, the server system 102 may include a plurality of module controllers 304, wherein each module controller 304 manages a corresponding plurality of application engine modules 206 to form one or more application engine module clusters, and / or manages a corresponding plurality of application instance modules 310 to form one or more application instance module clusters, as described with reference to FIG. Figure 3 As illustrated and discussed. The application engine module clusters and / or application instance module clusters can be geographically distributed such that each of the corresponding plurality of application engine modules 106 or 206 and / or the corresponding plurality of application instance modules 310 can be located in a different geographic location. When a user requests an active session with a cloud-based client application, the session management controller 140 can utilize the network latency information to enable reservation and assignment of an application engine module or application instance module that is geographically closest or otherwise has the least network latency to the user's client device 148. By measuring and monitoring the network latency information of all client devices 148, the server system 102 can create a heat map of network latency throughout the cloud-based client application platform 100, 200, or 300, etc., which can be used to reserve and assign the best application engine module or best application instance module (e.g., best from a network latency or network connection quality perspective) for each user, thereby maintaining smooth display and interaction of the cloud-based client application on the client device 148 during the corresponding active session.
[0058] According to an embodiment, since cloud-based client applications can be remotely run in the server system 102 and can be displayed on any appropriate client device 148, the present invention can also integrate cloud-based client applications into third-party platforms by embedding or otherwise providing URLs or links or hyperlinks to one or more cloud-based client applications into third-party platforms (including third-party websites and applications). This integration can allow these third parties to more easily provide different features and functions through integrated cloud-based client applications, thereby supplementing and enhancing interaction and participation with third-party platforms. For purposes of illustration and not limitation, retailers can incorporate any appropriate cloud-based client applications into their consumer contact points by embedding or otherwise providing URLs or other links or hyperlinks to cloud-based client applications. For example, according to an embodiment of the present invention, retailers can incorporate cloud-based mobile games into retailers' websites or applications. Users can interact with cloud-based mobile games on retailers' websites or applications in the manner described herein, just as cloud-based mobile games are being executed on users' client devices. If the user wins the integrated cloud-based mobile game, the user may receive a reward (e.g., loyalty rewards, coupons, product / service discounts, free merchandise, etc.) in the retailer's own currency. Reward information may be transmitted from the integrated cloud-based mobile game to a third-party platform (e.g., via server system 102) and maintained on the retailer's website or application for use by the user. For example, according to an embodiment of the present invention, a fast food restaurant may integrate a cloud-based mobile game into its website or application. A user may select and participate in an integrated cloud-based mobile game through a website or application, for example, by selecting a URL or other link or hyperlink appropriately embedded or otherwise provided for the cloud-based mobile game, and then interacting (remotely) with the cloud-based mobile game. If the user wins the integrated cloud-based mobile game, the user may be awarded a free side order of food (or other appropriate rewards or gifts suitable for the retailer) that can be redeemed at a fast food restaurant. If the user loses, the user may be allowed to compete again, a lower reward may be provided, another chance to play may be provided, etc. The present invention may support integration of any appropriate type of cloud-based client application into any appropriate third-party platform. According to the present invention, by allowing users to interact with and via a cloud-based client application that is integrated into a third-party website or application, a third party can increase user engagement metrics with its platform without requiring users to download and install the cloud-based client application to their client device 148.
[0059] By integrating one or more cloud-based client applications into a third party's platform, different experiences can be customized for users on the third-party platform that may not be available to the general population accessing a non-integrated version of the cloud-based client application or accessing it from a different website or application. For example, an integrated cloud-based client application on a third-party platform can provide features, functions, interface modifications, etc. that are available to users of the integrated cloud-based client application. For example, if the integrated cloud-based client application is a cloud-based mobile game, users interacting with the integrated cloud-based mobile game can access special or promotional tournaments that are not available to general users of the cloud-based mobile game. Additionally or alternatively, users interacting with the integrated cloud-based client application can be presented with special limited time offers (limited time offers, LTOs), advertisements, or other promotions that are not available to general users of the cloud-based client application. Additionally or alternatively, users interacting with the integrated cloud-based client application can be presented with a modified or enhanced user interface (e.g., different screens, customizations, third-party brand information, etc.) that is not available to general users of the cloud-based client application. Other modifications to the user experience of the integrated cloud-based client application, including modifications to the graphical user interface within and / or outside the integrated cloud-based client application, to support third-party platforms are possible.
[0060] In some implementations of the invention, data collected from users interacting with the integrated cloud-based client application may also be used to additionally or alternatively customize the integrated cloud-based and client application user experience. In an embodiment, the features, functionality, and performance of the integrated cloud-based client application on the third-party platform may be targeted to individual users based on the third-party platform's user data, data from the user's client device 148 or data associated with the user's client device 148, and the like. For example, the server system 102 may identify each user's geographic location, such as via the GPS of the client device 148 (e.g., communicated to the session management controller 140 via the session information module 156), from the user's account profile information (e.g., retrieved from the database 146), and the like. A user accessing the integrated cloud-based client application from one geographic location (e.g., a first state or region) may be presented with a different user experience than a user accessing the integrated cloud-based client application from a different geographic location (e.g., a different state or region). The different user experiences may include, for example, different user interfaces, different tournaments (e.g., in the context of a cloud-based mobile game), different features and functionality, and the like. For example, if the integrated cloud-based client application is a cloud-based mobile game, the user data may be used to match players of the integrated cloud-based mobile game. For example, for a competitive, skill-based integrated cloud-based mobile game, the user information may be used to match the user with other users who are interacting with the integrated cloud-based mobile game, rather than with users from the general population of (non-integrated) cloud-based mobile games. Such player matching may help ensure fairness in play and / or implement anti-fraud / anti-cheating measures, as users of the integrated cloud-based mobile game may have access to features and functionality that are not available to the general user population of the (non-integrated) cloud-based mobile game.
[0061] Some implementations of the present invention can be used to integrate cloud-based client applications into online or digital advertising. For purposes of illustration and not limitation, cloud-based mobile games can be integrated into digital advertising to create interactive advertising units, such as playable ads. In some implementations of the present invention, integrated cloud-based mobile games can be accessed from playable ads by causing users to click on URLs, links, or hyperlinks embedded in or otherwise set in playable ads. Alternatively, when playable ads are exposed, presented, or displayed to users, cloud-based mobile games can be automatically started within playable ads. Users can interact with integrated cloud-based mobile games in and through playable ads, for example, to win rewards, promotions, and other similar rewards as part of digital advertising campaigns. In an embodiment, the reward can be used for cloud-based mobile games themselves (e.g., virtual coins or rewards) to encourage users to interact with playable ads. For example, users can use the reward to make in-app purchases or other types of redemption or purchases within cloud-based mobile games themselves when interacting with playable ads. In an alternative embodiment, the reward can be used for another application or website associated with the digital advertising campaign, or used with another application or website to drive user participation. However, any suitable cloud-based client application can be integrated into the digital advertisement to create an interactive advertising unit. For example, an interactive advertising unit or other digital advertisement for a cloud-based client application product can include a trial version (even a fully usable but time-limited version) of the cloud-based client application product integrated into the interactive advertising unit or other digital advertisement. Thus, users can participate in the cloud-based client application product (e.g., within and through the interactive advertising unit or other digital advertisement) and evaluate it without having to wait for download and installation or otherwise purchase it before evaluation.
[0062] Figure 4 is a diagram showing a method for utilizing, for example, Figure 1Flowchart of an exemplary method 400 for remotely interacting with a cloud-based client application by a server system 102 of the embodiment of the present invention. At block 405, for example, the server system 102 may receive a first request from a client device of a user (e.g., client device 148) to initiate an interactive session with a cloud-based client application. The first request may include at least one characteristic of the client device. For purposes of illustration and not limitation, the client device characteristic may be, for example, a geographic location of the client device, although other characteristics are possible. The cloud-based client application may be configured to execute on a computing platform that is different from and remote from the client device. For purposes of illustration and not limitation, the client device may be, for example, an Android-based smartphone, and the computing platform may be, for example, a desktop PC. At block 410, the server system 102 (e.g., via the application engine inventory and reservation module 136) may reserve an application engine module from a plurality of application engine modules (e.g., a plurality of application engine modules 106) in response to the first request for remote execution of the cloud-based client application from the client device. The application engine module may be bound to the user for the duration of the interactive session. In an embodiment, each application engine module can be reserved for one user, so that during an interactive session, each application engine module is used by a single (bound) user at most. In block 415, the server system 102 can modify the cloud-based client application executed in the reserved application engine module based on at least one characteristic of the client device. In an embodiment, the characteristics of the client device can be used to change the cloud-based client application executed in the reserved application engine module. For the purpose of illustration and not limitation, if the characteristic of the client device is a geographic location, the server system 102 can modify the cloud-based client application to present visual features (e.g., interface changes, limited-time offers, advertisements, etc.) specific to the geographic region where the client device is located, so that users in different geographic locations can experience a cloud-based client application that can be customized according to their specific location or other appropriate client device characteristics. In block 420, the server system 102 can execute the modified cloud-based client application in the reserved application engine module. At block 425, the server system 102 may stream media (e.g., video and / or audio) data from the modified cloud-based client application to a browser application or other appropriate application executing on the user's client device (e.g., via display module 152 in browser module 150).At block 430, while the user engages with the streamed media data from the modified cloud-based client application via the browser application on the client device 148, the server system 102 may receive interaction data (e.g., input, response, feedback, etc.) from the client device (e.g., via the interaction capture module 154). At block 435, the server system 102 may receive a second request from the client device (e.g., sent to the session management controller 140 via the session information module 156) to end the interactive session with the modified cloud-based client application. In an embodiment, the second request may be a termination of the media stream to which the client device is connected, rather than a specific request from the client device to terminate the interactive session. At block 440, the server system 102 may deregister or release the application engine module that assigned the reservation in response to the second request (e.g., via the application engine inventory and reservation module 136). In an embodiment, the application engine module that assigned the reservation may be deregistered or released from the inventory, and the processors and processes of the application engine module that support the reservation may be spun down or otherwise terminated so that the application engine module is not reused by multiple users. In an alternative embodiment, the subscribed application engine module may be unbound from the user and made available for use by a second (or subsequent) user.
[0063] Figure 5 is a diagram showing the use of, for example, Figure 2 or Figure 3Flowchart of an exemplary method 500 for remotely interacting with a cloud-based client application by a server system 102 of the embodiment of the present invention. At block 505, the server system 102 may receive a first request from a client device of a user (e.g., client device 148) to initiate an interactive session with a cloud-based client application. The cloud-based client application may be configured to execute on a second client device. The second client device may include a computing platform that is different from and remote from the client device. For purposes of illustration and not limitation, the client device may be, for example, an Android-based smartphone, and the second client device may be, for example, a desktop PC. At block 510, the server system 102 may reserve an application engine module from a pre-instantiated application engine module pool (e.g., through the application engine inventory and reservation module 136) in response to the first request for remote execution of the cloud-based client application from the client device. The reserved application engine module may be bound to the client device for the duration of the interactive session. The pre-instantiated application engine module may be added to the pre-instantiated application engine module pool to replace the reserved application engine module. In some implementations of the present invention, each of the plurality of pre-instantiated application engine module pools may be associated with a different cloud-based client application. At block 515, a cloud-based client application may be executed within the reserved application engine module. At block 520, first media data may be streamed from the cloud-based client application executed within the reserved application engine module to a browser application executed on a client device (e.g., via display module 152 of browser module 150). At block 525, when a user engages with the streamed first media data from the cloud-based client application via the browser application, interaction data may be received from the client device (e.g., via interaction capture module 154). At block 530, the cloud-based client application may be modified within the reserved application engine module based on the interaction data received from the client device. At block 535, second media data may be streamed from the modified cloud-based client application to the browser application. At block 540, a second request may be received from the client device (e.g., sent to session management controller 140 via session information module 156) to end the interactive session with the modified cloud-based client application. At block 545, the application engine module that allocated the reservation may be released in response to the second request. The reserved application engine module may be unbound from the client device. A similar process may be performed for the application instance module, as described with reference to FIG. Figure 3 As illustrated and discussed.
[0064] Figure 66 is another flow chart illustrating an exemplary method 600 for remote interaction with a cloud-based client application. At block 602, a first request from a first client device of a user may be received to initiate an interactive session with a cloud-based client application. The cloud-based client application may be configured to execute on a second client device. It should be noted that the first client device may be different from the second client device because the second client device may include a computing platform different from the first client device. In aspects of the present invention, the first client device may be a smartphone and the second client device may be a desktop computer. In aspects of the present invention, a cloud-based client application as described herein may operate on a second client device, wherein media (i.e., video and / or audio) data output from the cloud-based client application is streamed to the first client device and displayed on the first client device. As described above, the first request may include characteristics of the client device, which may correspond to, for example, a geographic location specific to the client device. For example, the geographic location may correspond to a city, a neighborhood, a specific address, etc.
[0065] At block 604, an application engine may be reserved from a pool of pre-instantiated application engines in a plurality of pools of pre-instantiated application engines for the purpose of remotely executing a cloud-based client application from a first client device. As described above, such a reservation operation enables a cloud-based client application executed within the application engine to be specifically linked to the first client device, such that the cloud-based client application operates only in conjunction with the first client device for a predefined duration.
[0066] At block 606, interaction data may be received from the first client device as the user engages with first media data associated with the cloud-based client application. For example, the interaction data may correspond to input, responses, feedback, etc. from the user as the user (remotely) engages with the cloud-based client application. The first media data may include video and / or audio data streamed from the cloud-based client application to the user on the first client device.
[0067] At block 608, the cloud-based client application executed within the reserved application engine may be modified based on the interaction data received from the first client device. As described above, in various aspects, the modification may involve presenting, via the display module 152 of the first client device, one or more visual features that are updated as the user engages with the cloud-based client application. Additionally or alternatively, the visual features of the cloud-based client application may be modified based on or specific to the geographic region in which the first client device is located or other appropriate characteristics of the first client device. For example, such visual features may correspond to advertisements or offers that are related to the location of the first client device or are in some way specific to the location of the first client.
[0068] At block 610, a second request may be received from the first client device to end the interactive session with the modified cloud-based client application. For example, the user may select an icon on the first client device or interact with a button on the first client device to end the interactive session with the cloud-based client application. Various ways of terminating the interactive session are contemplated.
[0069] At block 612, the allocated reserved application engine may be released. The reserved application engine may be unlinked or otherwise unbound from the first client device. For example, releasing the allocation may involve unregistering the application engine so that the application engine is no longer linked, connected, or otherwise bound for operation exclusively with the first client device. In other words, after releasing the allocation, the application engine may be linked, bound, or allocated to a client device other than the first client device.
[0070] Embodiments of the present invention can improve network efficiency by not requiring users to download and install many (e.g., dozens, hundreds, or more) different client applications that they want to interact with to their client devices. Instead, the present invention can improve network traffic and network bandwidth utilization because what is transmitted between the server system 102 and the client device 148 is video / audio data and user input, rather than the entire client application itself. In addition, the present invention can reduce computer storage requirements on both the client and server sides. For example, users do not need to download and install any client application on their client devices. In some implementations of the present invention, the server system 102 can store a single version of a cloud-based client application that can be accessed from and distributed to any type of client device, without having to store many different versions of the client application to support too many client devices.
[0071] Embodiments of the present invention may also improve processing efficiency of client devices 148, because client devices 148 may process video / audio data from cloud-based client applications faster and more efficiently than executing the entire client application itself locally on the client device. Embodiments of the present invention may also improve processing efficiency of server system 102. Server system 102 may allocate computing resources when application engine modules and / or application instance modules are registered for an active session, and server system 102 may release these resources when the active session is completed, rather than maintaining continuous execution and computing resources to support cloud-based client applications on server system 102. In other words, server system 102 may register and unregister application engine modules and / or application instances as needed, respectively, without the need to continuously maintain and process each cloud-based client application for each client device. Therefore, server system 102 may more efficiently allocate computer processing resources to those application engine modules (and cloud-based client applications running in those application engine modules) and / or application instances that are being used and away from those application engine modules and / or application instances that are not being used. In some implementations of the present invention, pre-warming of application engine modules in an application engine module pool and / or application instances in an application instance pool can also significantly reduce the response time to establish an active session. Thus, embodiments of the present invention can manage users in active sessions on a large scale to maintain instant (or near instant) launch and interaction with cloud-based client applications (e.g., "instant play" in the context of cloud-based mobile games), particularly when, for example, there are hundreds or thousands of cloud-based client applications and millions, tens of millions, or more users interacting with those cloud-based client applications simultaneously.
[0072] Figure 7 It is a block diagram of an exemplary computing device 700 that can perform one or more operations described in this article according to the present embodiment. Computing device 700 can be connected to other computing devices in a local area network, an intranet, an extranet and / or the Internet. Computing device 700 can be operated according to the capacity of a server machine in a client-server network environment, or can be operated according to the capacity of a client in a peer-to-peer network environment. Computing device 700 can be provided by a machine that can (sequentially or otherwise) perform a group of instructions (it specifies the action that this machine will take). In addition, although only a single computing device 700 is shown, the term "computing device" should also be considered to include the set of any computing devices that perform a group (or multiple groups) of instructions to perform method discussed herein, either individually or in combination.
[0073] The exemplary computing device 700 may include a computer processing device 702 (e.g., a general purpose processor, ASIC, etc.), a main memory 704, a static memory 706 (e.g., flash memory, etc.), and a data storage device 708 that can communicate with each other via a bus 730. The computer processing device 702 may be provided by one or more general purpose processing devices (e.g., a microprocessor, a central processing unit, etc.). In an illustrative example, the computer processing device 702 may include a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor that implements other instruction sets or a processor that implements a combination of instruction sets. The computer processing device 702 may also include one or more special processing devices, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, etc. According to one or more aspects of the present invention, the computer processing device 702 may be configured to perform the operations described herein to perform the operations and steps discussed herein.
[0074] The computing device 700 may further include a network interface device 712 that can communicate with a network 714. According to one or more aspects of the present invention, the data storage device 708 may include a machine-readable storage medium 728 on which one or more sets of instructions may be stored, for example, instructions for performing the operations described herein. The instructions 718 implementing the core logic instructions 726 may also reside in the main memory 704 and / or the computer processing device 602 during execution by the computing device 700, all or at least in part, the main memory 704 and the computer processing device 602 also constituting computer-readable media. The instructions may further be sent or received via the network 714 via the network interface device 712.
[0075] Although the machine-readable storage medium 728 is shown as a single medium in the illustrative example, the term "computer-readable storage medium" should be considered to include a single medium or multiple media (e.g., a centralized or distributed database and / or associated caches and servers) that store one or more sets of instructions. The term "computer-readable storage medium" should also be considered to include any medium that can store, encode, or carry a set of instructions for execution by a machine and causes the machine to perform the methods described herein. Therefore, the term "computer-readable storage medium" should be considered to include, but is not limited to, solid-state memories, optical media, magnetic media, and the like.
[0076] The subject matter and the embodiment of operation described in the present invention can be implemented in a digital electronic circuit including the structure disclosed in the present invention and its structural equivalent form, or computer software, firmware or hardware, or a combination of one or more thereof. The embodiment of the subject matter described in the present invention can be implemented as one or more computer programs, that is, a module of one or more computer program instructions encoded on a computer storage medium to be executed by a data processing device or to control the operation of a data processing device. Alternatively, or in addition, the program instructions can be encoded on an artificially generated propagation signal (e.g., a machine-generated electrical, optical or electromagnetic signal), which is generated as coded information to be sent to a suitable receiver device to be executed by a data processing device. The computer storage medium can be or be included in a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more thereof. In addition, although the computer storage medium is not a propagation signal, the computer storage medium can be the source or target of the computer program instructions encoded in the artificially generated propagation signal. The computer storage medium can also be or be included in one or more separate physical components or media (e.g., multiple CDs, disks or other storage devices).
[0077] The operations described in the present invention may be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.
[0078] The term "data processing device" includes various devices, means and machines for processing data, for example, including a programmable processor, a computer processing device, a computer, a system on a chip, or a plurality or combination of the foregoing. The computer processing device may include one or more processors, and the one or more processors may include a dedicated logic circuit, for example, a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC), a central processing unit (CPU), a multi-core processor, etc. In addition to hardware, the device may also include code that creates an execution environment for the relevant computer program, for example, code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more thereof. The device and the execution environment can implement a variety of different computing model infrastructures, for example, network services, distributed computing, and grid computing infrastructures.
[0079] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language, including compiled or interpreted languages, declarative languages, procedural languages, or functional languages, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may (but does not necessarily) correspond to a file in a file system. A program may be stored as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language resource), in a single file dedicated to a related program, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or portions of code). A computer program may be deployed to execute on a single computer, or on multiple computers located at one site or distributed across multiple sites and interconnected by a communications network.
[0080] The processes and logic flows described in the present invention may be performed by one or more programmable processors executing one or more computer programs to perform actions by running input data and generating output. The processes and logic flows may also be performed by a dedicated logic circuit (e.g., a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC)), and the device may also be implemented as a dedicated logic circuit.
[0081] For example, a processor suitable for executing a computer program includes any one or more processors of both general and special microprocessors and any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic element of a computer is a processor for performing actions according to instructions and one or more storage devices for storing instructions and data. Typically, a computer will also include one or more large-capacity storage devices (e.g., disks, magneto-optical disks, optical disks, solid-state drives, etc.) for storing data, or a computer may be operably connected to receive data from one or more large-capacity storage devices or send data to one or more large-capacity storage devices, or two situations. However, a computer may not necessarily have such a device. In addition, a computer may be embedded in another device (e.g., a smart phone, a mobile audio or media player, a game console, a global positioning system (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), etc.). Suitable devices for storing computer program instructions and data include all forms of non-volatile memory, media and storage devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM and flash memory devices); magnetic disks (e.g., internal hard disks or removable disks); magneto-optical disks; and CDROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.
[0082] To provide interaction with a user, embodiments of the subject matter described in this specification may be implemented on a computer having a display device (e.g., a cathode ray tube (CRT), a liquid crystal display (LCD) monitor, a light emitting diode (LED) monitor, etc.) for displaying information to the user and a keyboard and pointing device (e.g., a mouse, a trackball, a touchpad, a stylus, etc.) that enables the user to provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and the input from the user may be received in any form, including acoustic, voice, or tactile input. Other possible input devices include touch screens or other touch-sensitive devices (e.g., single-point or multi-point resistive or capacitive tracking pads), voice recognition hardware and software, optical scanners, optical pointers, digital image capture devices, and associated interpretation software, etc. In addition, the computer may interact with the user by sending resources to and receiving resources from devices used by the user; for example, by sending a web page to a web browser on a user's client device in response to a request received from the web browser.
[0083] Embodiments of the subject matter described in this invention may be implemented in a computing system that includes a back-end component (e.g., as a data server), or includes a middleware component (e.g., an application server), or includes a front-end component (e.g., a client computer with a graphical user interface or a web browser that enables a user to interact with an implementation of the subject matter described in this invention), or any combination of one or more of the back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LANs") and wide area networks ("WANs"), internetworks (e.g., the Internet), peer-to-peer networks (e.g., ad hoc peer-to-peer networks), and the like.
[0084] The computing system may include a client and a server. The client and the server are usually far away from each other and generally interact through a communication network. The relationship between the client and the server is generated by means of a computer program that runs on a corresponding computer and has a client-server relationship with each other. In some embodiments, the server sends data (e.g., an HTML page) to a client device (e.g., for displaying data to a user interacting with the client device and receiving user input from the user). The data generated at the client device (e.g., the result of the user interaction) can be received from the client device at the server.
[0085] A system of one or more computers may be configured to perform specific operations or actions by having software, firmware, hardware, or a combination thereof installed on the system, which, when executed, causes the system to perform actions. One or more computer programs may be configured to perform specific operations or actions by including instructions, which, when executed by a data processing device, causes the device to perform actions.
[0086] Reference throughout this disclosure to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this disclosure are not necessarily all referring to the same embodiment. Furthermore, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or".
[0087] Although the present invention includes many specific implementation details, these details should not be interpreted as limitations on any invention scope or content that may be claimed, but should be interpreted as descriptions of features specific to specific embodiments of specific inventions. Certain features described in the context of separate embodiments in the present invention may also be implemented in combination in a single embodiment. On the contrary, the various features described in the context of a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. In addition, although features may be described above as working in certain combinations, even initially claimed in this way, in some cases, one or more features in the claimed combination may be removed from the combination, and the claimed combination may point to a variant of a sub-combination or a sub-combination.
[0088] Similarly, although operations and / or logic flows are depicted in the accompanying drawings and / or described herein in a particular order, this should not be understood as requiring that these operations and / or logic flows be performed in the particular order or sequence shown, or requiring that all operations shown be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system components in the above-described embodiments should not be understood as requiring such separation in all embodiments, but it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged as multiple software products.
[0089] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. In addition, the processes described in the accompanying drawings do not necessarily require the specific order or sequence shown to achieve the desired results. In some implementations, multitasking and parallel processing may be advantageous.
[0090] The words "example" or "exemplary" are used herein to indicate use as an example, instance or illustration. Any aspect or design described herein as "example" or "exemplary" is not necessarily to be interpreted as preferred or superior to other aspects or designs. On the contrary, the use of the words "example" or "exemplary" is intended to present the concept in a specific way. As used in this application, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or clear from the context, "X includes A or B" is intended to mean any natural inclusive arrangement. That is, if X includes A; X includes B; or X includes both A and B, "X includes A or B" is satisfied in any of the above cases. In addition, the articles "one" and "a" used in this application and the appended claims should generally be interpreted as meaning "one or more", unless otherwise specified or clearly pointed to the singular form in the context. In addition, the terms "one embodiment" or "an embodiment" or "an implementation" or "an implementation" used throughout the text are not intended to mean the same embodiment or implementation, unless so described. In addition, the terms “first,” “second,” “third,” “fourth,” etc. as used herein are indicated as labels to distinguish different elements, and do not necessarily have ordinal meanings according to their numerical names.
[0091] In the above description and in the claims, phrases such as "at least one of" or "one or more of" may appear before a list of combinations of elements or features. The term "and / or" may also appear in a list of two or more elements or features. Unless otherwise implicitly or explicitly contradictory to the context in which it is used, this phrase is intended to mean any individually listed element or feature, or any combination of a recorded element or feature with any other recorded element or feature. For example, the phrases "at least one of A and B"; "one or more of A and B"; and "A and / or B" are intended to mean "A alone, B alone, or A and B together", respectively. A similar interpretation is also intended for a list including three or more items. For example, the phrases "at least one of A, B, and C"; "one or more of A, B, and C"; and "A, B, and / or C" are intended to mean "A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B and C together", respectively. Furthermore, the term “based on” as used above and in the claims is intended to mean “based, at least in part, on” such that non-recited features or elements are also permitted.
[0092] The above description of the implementation of the illustrated invention is not intended to be exhaustive or to limit the invention to the precise form disclosed. Although the specific implementation and examples of the invention are described herein for the purpose of illustration, various equivalent modifications are possible within the scope of the invention, as recognized by those skilled in the relevant art. Depending on the desired configuration, the subject matter described herein can be implemented in systems, devices, methods and / or articles. The implementations set forth in the above specification do not represent all implementations consistent with the subject matter described herein. On the contrary, they are only some examples consistent with aspects related to the described subject matter. Although some variations have been described in detail above, other modifications or additions may also be made. Specifically, in addition to those features and / or variations set forth herein, further features and / or variations may also be provided. For example, the above implementations may be directed to various combinations and sub-combinations of the disclosed features, and / or combinations and sub-combinations of several further features disclosed above. Other implementations may be within the scope of the appended claims.
Claims
1. A method, include: receiving, by at least one data processor, a first request from a first client device of a user to initiate an interactive session with a cloud-based client application, wherein the cloud-based client application is configured to execute on a second client device, and wherein the second client device comprises a computing platform different from the first client device; In response to the first request, reserving, by at least one data processor, an application engine from a pool of pre-instantiated application engines in a plurality of pools of pre-instantiated application engines for remotely executing a cloud-based client application from a first client device; receiving, by at least one data processor, interaction data from a first client device when a user engages with first media data associated with a cloud-based client application; modifying, by at least one data processor, a cloud-based client application executing within a subscribed application engine based on the interaction data received from the first client device; receiving, by at least one data processor, from the first client device a second request to end the interactive session with the modified cloud-based client application; and In response to the second request, the allocated reserved application engine is released by at least one data processor, wherein the reserved application engine is unlinked from the first client device.
2. The method according to claim 1, further comprising: include: First media data is streamed from a cloud-based client application to a first client device by at least one data processor.
3. The method according to claim 1, further comprising: include: Second media data from the modified cloud-based client application is streamed, by at least one data processor, to a browser application executing on the first client device.
4. The method according to claim 1, in, The reserved application engine is linked to the first client device for the duration of the interactive session, wherein the pre-instantiated application engine is added to a pool of pre-instantiated application engines to replace the reserved application engine, and wherein each pre-instantiated application engine pool in the plurality of pre-instantiated application engine pools is associated with a different cloud-based client application.
5. The method according to claim 1, in, The first request includes at least one characteristic associated with a first client device.
6. The method according to claim 5, in, The at least one characteristic characterizes a geographic region associated with the first client device, and wherein the modification of the cloud-based client application further comprises: One or more visual features specific to a geographic region associated with the first client device are presented on the first client device.
7. The method according to claim 6, in, The one or more visual features specific to the geographic region associated with the first client device include one or more of an interface change or an advertisement.
8. The method according to claim 1, further comprising: include: receiving input from the user prior to the second request from the first client device to end the interactive session; as well as In response to the input, an interactive session with the cloud-based client application at the particular location is paused, the pause being different than an end of the interactive session.
9. The method according to claim 8, further comprising: include: receiving additional input for resuming the interactive session on the third client device; as well as In response to receiving the additional input, the interactive session at the particular location is resumed on the third client device.
10. A system, include: at least one data processor; as well as A memory storing instructions, which, when executed by at least one data processor, causes at least one data processor to perform operations including: receiving a first request from a first client device of a user to initiate an interactive session with a cloud-based client application, wherein the cloud-based client application is configured to execute on a second client device, and wherein the second client device comprises a computing platform different from the first client device; In response to the first request, reserving an application engine from a pool of pre-instantiated application engines in a plurality of pools of pre-instantiated application engines for remotely executing a cloud-based client application from a first client device; receiving interaction data from a first client device when a user engages with first media data associated with a cloud-based client application; modifying a cloud-based client application executing within the subscribed application engine based on the interaction data received from the first client device; receiving a second request from the first client device to end the interactive session with the modified cloud-based client application; and In response to the second request, the allocated reserved application engine is released, wherein the reserved application engine is unlinked from the first client device.
11. The system according to claim 10, in, The operations further include: First media data is streamed from a cloud-based client application to a first client device.
12. The system according to claim 10, in, The operations further include: The second media data from the modified cloud-based client application is streamed to the browser application executing on the first client device.
13. The system according to claim 10, in, The reserved application engine is linked to the first client device for the duration of the interactive session, wherein the pre-instantiated application engine is added to a pool of pre-instantiated application engines to replace the reserved application engine, and wherein each pre-instantiated application engine pool in the plurality of pre-instantiated application engine pools is associated with a different cloud-based client application.
14. The system according to claim 10, in, The first request includes at least one characteristic associated with a first client device.
15. The system according to claim 14, in, The at least one characteristic characterizes a geographic region associated with the first client device, and wherein the modified operation of the cloud-based client application further comprises: One or more visual features specific to a geographic region associated with the first client device are presented on the first client device.
16. The system according to claim 15, in, The one or more visual features specific to the geographic region associated with the first client device include one or more of an interface change or an advertisement.
17. The system according to claim 10, in, The operations further include: receiving input from the user prior to the second request from the first client device to end the interactive session; and In response to the input, an interactive session with the cloud-based client application at the particular location is paused, the pause being different than an end of the interactive session.
18. The system according to claim 17, in, The operations further include: receiving additional input for resuming the interactive session on the third client device; and In response to receiving the additional input, the interactive session at the particular location is resumed on the third client device.
19. A non-volatile computer program product having executable instructions stored thereon which, when executed by at least one data processor forming part of at least one computing system, perform operations comprising: receiving a first request from a first client device of a user to initiate an interactive session with a cloud-based client application, in, The cloud-based client application is configured to execute on a second client device, and wherein the second client device comprises a different computing platform than the first client device; In response to the first request, reserving an application engine from a pool of pre-instantiated application engines in a plurality of pools of pre-instantiated application engines for remotely executing a cloud-based client application from a first client device; receiving interaction data from a first client device when a user engages with first media data associated with a cloud-based client application; modifying a cloud-based client application executing within the subscribed application engine based on the interaction data received from the first client device; receiving a second request from the first client device to end the interactive session with the modified cloud-based client application; and In response to the second request, the allocated reserved application engine is released, wherein the reserved application engine is unlinked from the first client device.
20. The non-volatile computer program product according to claim 19, in, The operations further include: First media data is streamed from a cloud-based client application to a first client device.
