Application running method and electronic equipment

Abstract

The embodiment of the invention discloses an application running method and electronic equipment. The method comprises the following steps: acquiring connection request information sent by a plurality of clients to the same cloud mobile phone instance; creating a plurality of mutually independent virtual display environments according to the connection request information, the plurality of virtual display environments being in one-to-one correspondence with the plurality of clients; creating an independent user environment for each target application in the one or more target applications in response to a starting operation performed on the one or more target applications by the user through the plurality of clients; and running the one or more target applications in the user environment, so that the plurality of virtual display environments respectively present running interfaces of the one or more target applications on the plurality of clients. Through the scheme, an independent application operation environment is provided for a plurality of clients in a single cloud mobile phone instance, and the resource utilization rate of a server and the concurrent processing capability of the system are remarkably improved.

Description

Technical Field

[0001] This application relates to the field of cloud services, and more specifically, to an application running method and an electronic device. Background Technology

[0002] Cloud phone technology is based on a cloud computing platform, which can virtualize and run an operating system on a remote server. Users can remotely operate the virtual phone in the cloud from their local device via a network connection. Currently, the mainstream cloud phone implementation solutions include lightweight virtualization based on container technology and complete system virtualization based on virtual machines. Traditional cloud phone single-interface screen mirroring technology typically employs a scheme where a single virtual instance mirrors to a single client. Specifically, a complete operating system runs on the server side, and the system interface is transmitted to the client via video encoding. The client's input events are then transmitted back to the server over the network. In this scheme, a single cloud phone instance can only serve one user at a time, and the user sees the complete operating system desktop interface.

[0003] Since a cloud phone instance can only serve one user at a time, traditional cloud phone technology suffers from drawbacks such as low resource utilization, poor concurrent processing capabilities, limited interactive experience, and wasted network bandwidth. Summary of the Invention

[0004] This application addresses some of the shortcomings mentioned in the background technology by providing an application running method and an electronic device. This solution enables multiple clients to run applications independently within a single cloud phone instance, significantly improving server resource utilization and system concurrency capabilities, while reducing unnecessary network bandwidth consumption and enhancing user experience.

[0005] Firstly, an application running method is provided, applied to a cloud phone instance on a cloud server, comprising: obtaining connection request information sent by multiple clients to the same cloud phone instance; creating multiple independent virtual display environments based on the connection request information, wherein each virtual display environment corresponds one-to-one with the multiple clients; in response to users respectively performing a startup operation on one or more target applications through the multiple clients, creating an independent user environment for each of the one or more target applications; running the one or more target applications in the user environment, such that the multiple virtual display environments respectively present the running interface of the one or more target applications on the multiple clients. This solution enables multiple clients to be served simultaneously on a single cloud phone instance. Each client has an independent virtual display environment and application runtime space, which significantly improves the utilization efficiency of server resources such as central processing unit (CPU), memory, graphics processing unit (GPU) and the overall concurrent processing capability of the system.

[0006] In conjunction with the first aspect, in a possible implementation of the first aspect, the connection request information includes one or more of the following for each of the plurality of clients: the target application package name, the expected image parameters, and the device type. This solution enables the cloud server to create and configure virtual display environments based on the client's specific needs, such as the applications to be launched, desired resolution, and device type. Ultimately, this ensures that the provided services are highly matched to the client's device and usage scenario, enhancing the customization and adaptability of the user experience.

[0007] In conjunction with the first aspect, in a possible implementation of the first aspect, the multiple virtual display environments correspond one-to-one with multiple display identifiers, and the multiple display identifiers are different from each other. This scheme assigns a unique identifier to each virtual display environment, which facilitates the accurate differentiation and management of different display sessions within the system. It ensures that input events, video streams, and other data can be accurately routed to the corresponding virtual display environment and application instance, avoiding confusion and errors.

[0008] In conjunction with the first aspect, in a possible implementation of the first aspect, each of the plurality of virtual display environments includes a rendering interface, an image reading interface, and a display density parameter setting module. This solution provides each virtual display environment with complete rendering, image capture, and display configuration capabilities, ensuring that the application interface can be correctly rendered in the virtual environment, efficiently captured for encoding and transmission, and adapted to the display based on the characteristics of the client device, such as screen density, thus guaranteeing display quality and compatibility.

[0009] In conjunction with the first aspect, in a possible implementation of the first aspect, the display parameters of each virtual display environment are determined based on the display performance of the corresponding client, which includes the adapted resolution, color space, and dots per inch (DPI). This solution dynamically configures the parameters of the virtual display environment based on the actual display capabilities of the client device, such as screen resolution, color support, and pixel density. This allows the cloud-rendered images to perfectly adapt to the screens of different clients, avoiding problems such as stretching, distortion, or color distortion, and optimizing the visual experience.

[0010] In conjunction with the first aspect, in a possible implementation of the first aspect, after running the one or more target applications, the method further includes: obtaining the running status information of the one or more target applications, the running status information including one or more of startup time, application activity, and resource usage; and determining one or more of the pause, resume, destroy, and migration operations of the one or more target applications based on the running status information. This solution enables the system to monitor the real-time running status of each application instance, such as startup time, user interaction frequency, and resource consumption. Based on this, it intelligently performs application lifecycle management operations, such as pausing background applications to release resources, resuming user-interactive applications, destroying abnormal applications, or migrating workloads. This significantly improves the dynamic scheduling efficiency of system resources and optimizes overall performance.

[0011] In conjunction with the first aspect, in a possible implementation of the first aspect, the method further includes: obtaining resource usage information of the plurality of virtual display environments; determining the resource usage priority of the plurality of virtual display environments based on the resource usage information; and allocating resources to the plurality of virtual display environments based on the resource usage priority of each virtual display environment among the plurality of virtual display environments. This solution allows the system to calculate the priorities of each virtual display environment based on their actual resource consumption, such as CPU, memory, and GPU usage, and dynamically adjust resource allocation strategies accordingly. This ensures that high-priority, high-demand application sessions receive more resources, optimizes overall resource utilization, and guarantees the quality of service for critical business or highly interactive sessions.

[0012] In conjunction with the first aspect, in a possible implementation of the first aspect, the plurality of virtual display environments includes a first virtual display environment. Determining the resource usage priority of the plurality of virtual display environments based on the resource usage information includes: determining the user interaction activity level of the first virtual display environment, the network quality information of the first client corresponding to the first virtual display environment, and the application type of the first application presented by the first virtual display environment based on the resource usage information; and determining the resource usage priority of the first virtual display environment based on the user interaction activity level, the application type, and the network quality information. This approach, when determining resource priorities, comprehensively considers the real-time activity level of user interaction, the characteristics of the application type itself (e.g., games typically require higher resource guarantees), and the client's network conditions (e.g., poor network conditions may require higher bitrates or more stable transmission). This multi-dimensional evaluation model makes resource allocation decisions more refined and intelligent, enabling more rational allocation of limited resources in complex scenarios, maximizing user experience and system efficiency.

[0013] In conjunction with the first aspect, in a possible implementation of the first aspect, the user interaction activity corresponds to the first weight, the application type corresponds to the second weight, the network quality information corresponds to the third weight, and the resource usage priority is obtained by summing the first product, the second product, and the third product. The first product is the product of the user interaction activity and the first weight, the second product is the product of the application type and the second weight, and the third product is the product of the network quality information and the third weight. This scheme provides a concrete and quantifiable priority calculation model. By assigning weights to different influencing factors (i.e., interaction activity, application type, and network quality) and performing weighted summation, the proportion of each factor in the final priority determination can be flexibly adjusted to adapt to different business scenarios and optimization goals, making resource scheduling strategies more adaptable and configurable.

[0014] In conjunction with the first aspect, in possible implementations of the first aspect, the resource usage information includes one or more of the following: CPU utilization, memory usage, GPU rendering load, and network input / output I / O. In conjunction with the first aspect, in a possible implementation of the first aspect, the method further includes: determining, based on the resource usage information of the plurality of virtual display environments, that the amount of resources provided by the cloud phone instance is less than or equal to the amount of resources required by the plurality of virtual display environments; reducing the activity level of at least one of the plurality of virtual display environments, wherein the at least one virtual display environment is in a non-interactive or inactive state. This solution intelligently identifies inactive virtual sessions (such as those without user interaction or running in the background) when system resources are strained, and reduces their resource consumption by lowering their frame rate, compressing their image quality, or even pausing their application. This effectively alleviates resource pressure, prioritizes the service quality of active user sessions, and improves the system's stability and resource utilization under high load.

[0015] In conjunction with the first aspect, in a possible implementation of the first aspect, the method further includes: obtaining user input event information sent by the multiple clients, the user input event information including the identification information of the multiple clients and user events; determining the target application launched by the multiple clients based on the identification information of the multiple clients; and sending the user events to the target application launched by the multiple clients. This solution enables the system to accurately receive user operation events such as touch and click operations from different clients, and precisely route them to the target application instance running in a specific virtual display environment corresponding to that client based on the client identifier. This ensures accurate response to user operations and maintains the consistency and correctness of the interaction.

[0016] In conjunction with the first aspect, in possible implementations of the first aspect, the user event includes one or more of the following: timestamp, event type, and coordinate information. This scheme provides detailed structural information about user input events, including the precise time of the event, the specific operation type, and the location coordinates of the event, providing the necessary data foundation for the server to accurately simulate user operations and achieve low-latency interaction.

[0017] In conjunction with the first aspect, in the possible implementations of the first aspect, the event type includes one or more of pressing, moving, lifting, and canceling. This solution covers the most basic and common event types in mobile device interaction, and can fully describe the user's operation sequence on the touchscreen, ensuring that cloud applications can accurately understand and respond to the user's intent.

[0018] In conjunction with the first aspect, in a possible implementation of the first aspect, the method further includes: converting the coordinate information of the multiple clients into virtual display local coordinate information based on the identification information of the multiple clients. This approach allows the system to dynamically transform and map the coordinates of input events reported from different client devices, based on their own screen coordinate systems, to the coordinate system of their corresponding virtual display environment. This enables applications to correctly understand the location of events, regardless of the user's device resolution, ensuring the accuracy of interaction and cross-device compatibility.

[0019] In conjunction with the first aspect, in a possible implementation of the first aspect, the method further includes: obtaining the operation habit information of the users corresponding to the multiple clients; and determining the predicted input events of the users of the multiple clients based on the operation habit information. This approach allows the system to learn and analyze users' specific operating habits, such as common gestures, click areas, and operation sequences, and predict the user's likely next action, preloading resources or pre-rendering the interface in advance. This helps reduce the actual response latency of user operations, improving the smoothness of interaction and system responsiveness.

[0020] In conjunction with the first aspect, in a possible implementation of the first aspect, the method further includes: allocating hardware encoder instances to the plurality of virtual display environments; encoding the image data of the plurality of virtual display environments through the hardware encoder instances; and sending the encoding results of the image data of the plurality of virtual display environments to the corresponding plurality of clients respectively. This solution allocates independent hardware encoding resources to each virtual display session or client, enabling parallel encoding processing of video streams. This significantly improves encoding efficiency and system throughput, reduces video transmission latency, and ensures the smooth transmission of multiple video streams simultaneously.

[0021] In conjunction with the first aspect, in a possible implementation of the first aspect, the encoding parameters of the hardware encoder instance are determined based on the network status information of the multiple clients, and the encoding parameters include one or more of bitrate, frame rate, and I-frame interval. This solution dynamically adjusts the encoding parameters of each client's video stream based on real-time network bandwidth, latency, and packet loss rate. For example, it reduces the bitrate to adapt to low bandwidth, adjusts the frame rate for smooth playback, and shortens the I-frame interval to improve packet loss resilience. This achieves adaptive optimization of video transmission, maximizing its ability to adapt to network fluctuations while ensuring basic image quality, thus enhancing the user experience.

[0022] In conjunction with the first aspect, in a possible implementation of the first aspect, the encoding processing of image data of the multiple virtual display environments through the hardware encoder instance includes: determining the user interaction area in the image data of the multiple virtual display environments; and encoding the image data of the multiple virtual display environments through the hardware encoder instance, wherein the encoding quality of the user interaction area is greater than the encoding quality of other areas in the image data excluding the user interaction area. This scheme identifies the screen areas the user is currently interacting with or frequently focusing on during video encoding. Higher encoding quality, such as lower compression rates and higher resolution, is applied to these areas, while lower encoding quality is used for less important areas. This partitioned encoding strategy prioritizes the clarity and detail of the areas most relevant to the user within limited network bandwidth, optimizing bandwidth utilization and enhancing the user's subjective visual experience.

[0023] In conjunction with the first aspect, in a possible implementation of the first aspect, sending the encoding processing results of the image data of the multiple virtual display environments to the corresponding multiple clients respectively includes: sending the encoding processing results of the image data of the multiple virtual display environments to the corresponding multiple clients respectively through the real-time transport protocol (RTP), wherein the encoding processing results include RTP data packets, and the RTP header of the RTP data packets includes the display identifiers of the multiple virtual display environments and the identification information of the multiple clients. This scheme utilizes the RTP protocol to efficiently transmit video stream data packets, embedding virtual display identifiers and client identifiers in the RTP header. This allows the receiving end to accurately associate received video data packets with a specific virtual display session and client, resolving the data packet attribution issue during concurrent transmission of multiple video streams and ensuring that the client receives the correct content from its own session.

[0024] In conjunction with the first aspect, in a possible implementation of the first aspect, the plurality of virtual display environments includes a first virtual display environment and a second virtual display environment. The method further includes: sending data and / or information of the target application in the first virtual display environment to the second virtual display environment through a unified inter-process communication (IPC) interface. This solution provides a secure and standardized inter-application communication mechanism across virtual display environments (i.e., across client sessions). This enables applications running in different virtual environments to exchange data and share information, such as text copying and pasting, and file transfer, breaking down barriers between sessions and enhancing user convenience and system flexibility.

[0025] In conjunction with the first aspect, in a possible implementation of the first aspect, the data and / or information of the target application in the first virtual display environment is sent to the second virtual display environment by: obtaining communication request information from the target application in the first virtual display environment, the communication request information including the display identifier of the first virtual display environment, the display identifier of the second virtual display environment, the data type, and the security token; verifying whether the communication permission between the first virtual display environment and the second virtual display environment is approved based on the communication request information; and if the communication permission is approved, sending the data and / or information of the target application in the first virtual display environment to the second virtual display environment. This scheme enforces strict authorization verification before allowing cross-environment communication. Communication requests must explicitly specify source and destination identifiers, data types, and carry a valid security token (potentially based on user identity or application signature). This ensures the security, controllability, and traceability of data exchange, preventing unauthorized information leaks or malicious operations, and safeguarding system and user data security.

[0026] In a second aspect, an electronic device is provided, including one or more processors; one or more memories; the one or more memories storing one or more computer programs, the one or more computer programs including instructions that, when executed by the one or more processors, cause a method as described in the first aspect and any possible implementation thereof to be performed.

[0027] Thirdly, a computer-readable storage medium is provided that stores computer instructions that, when executed on a computer, cause the method of the first aspect and any possible implementation thereof to be performed.

[0028] Fourthly, a chip is provided, the chip including a processor and a communication interface for receiving a signal and transmitting the signal to the processor, the processor processing the signal such that a method as described in the first aspect and any possible implementation thereof is executed.

[0029] Fifthly, a computer program product is provided that, when run on a computer, causes the computer to perform a method as described in the first aspect and any possible implementation thereof. Attached Figure Description

[0030] Figure 1 A schematic flowchart of an application interaction method provided in an embodiment of this application is shown; Figure 2 A schematic diagram of the cloud phone multi-application parallel interaction system architecture provided in an embodiment of this application is shown; Figure 3 A schematic diagram showing part of the flow timing of the application interaction method provided in the embodiments of this application is shown; Figure 4 A schematic diagram illustrating the remaining process timing of the application interaction method provided in the embodiments of this application is shown; Figure 5 A schematic diagram of the application interaction method implementation architecture provided in an embodiment of this application is shown; Figure 6 A schematic diagram of the application instance isolation mechanism provided in an embodiment of this application is shown; Figure 7 A schematic diagram of the resource allocation strategy provided in an embodiment of this application is shown; Figure 8 A schematic diagram illustrating the priority calculation method provided in an embodiment of this application is shown. Figure 9 A flowchart of the resource scheduling algorithm provided in an embodiment of this application is shown; Figure 10 A flowchart of the input event routing process provided in an embodiment of this application is shown; Figure 11 A schematic diagram of the video encoding and transmission architecture provided in an embodiment of this application is shown; Figure 12 A schematic diagram of the data flow process provided in an embodiment of this application is shown; Figure 13A schematic diagram of the architecture of the cross-virtual display application collaboration mechanism provided in an embodiment of this application is shown; Figure 14 This is a structural schematic diagram of a device provided in an embodiment of this application; Figure 15 This is a structural schematic diagram of a system on a chip (SoC) provided in an embodiment of this application. Detailed Implementation

[0031] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0032] The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one, two, or more than two. The term “and / or” is used to describe the relationship between related objects, indicating that three relationships may exist; for example, A and / or B can indicate: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character “ / ” generally indicates that the preceding and following related objects are in an “or” relationship.

[0033] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0034] As mentioned in the background technology section, since a cloud phone instance can only serve one user at a time, traditional cloud phone technology suffers from drawbacks such as low resource utilization, poor concurrent processing capabilities, limited interactive experience, and wasted network bandwidth.

[0035] For example, a single cloud phone instance can only serve one user, resulting in underutilization of server resources such as CPU, memory, and GPU, leaving a large amount of computing resources idle. Furthermore, its concurrency processing capabilities are poor, unable to handle the different application needs of multiple users simultaneously, limiting scalability. Another example is the limited user experience; users only see the complete desktop, unable to provide customized application interfaces based on different client needs. Finally, network bandwidth is wasted, as transmitting the complete desktop interface includes a large amount of visual information that users don't need, wasting network bandwidth.

[0036] To address the aforementioned issues, this application proposes an application operation method 100. Figure 1 A schematic flowchart of the application running method 100 provided in an embodiment of this application is shown. Figure 1 As shown, method 100 includes steps S110 to S140. In method 100, an independent application runtime environment is provided for multiple clients within a single cloud phone instance, significantly improving server resource utilization and system concurrency processing capabilities, while reducing unnecessary network bandwidth consumption and enhancing user experience.

[0037] Step S110: Obtain connection request information sent by multiple clients to the same cloud phone instance.

[0038] Specifically, this connection request information is used by the client to request a connection to the cloud phone instance. The cloud phone instance is a virtualized mobile device environment running on a cloud server, providing remote application execution services to multiple end users.

[0039] Optionally, to address the issue of the server's inability to dynamically configure the environment based on client characteristics, in embodiments of this application, the connection request information includes one or more of the following for each of the multiple clients: the target application package name, desired image parameters, and device type. Specifically, the target application package name may identify the specific application the user wishes to launch, the desired image parameters may include resolution, frame rate, and color space configuration, and the device type may include terminal device categories such as mobile phones, tablets, or televisions. For example, when a user initiates a connection from a mobile client, the request may carry information such as the target application package name, desired 1080P resolution, and the device type being a mobile phone.

[0040] Step S120: Based on the connection request information, create multiple independent virtual display environments.

[0041] Specifically, each of the multiple virtual display environments corresponds one-to-one with a specific client, and each virtual display environment is used to present the corresponding client. It should be understood that each virtual display environment is an independent display channel allocated to each client session, ensuring that the operations of each client do not interfere with each other.

[0042] Optionally, to address the issue of multi-session identifier confusion, in the embodiments of this application, each of the multiple virtual display environments corresponds to a multiple display identifier, and these multiple display identifiers are distinct from each other. Specifically, the display identifier can be a unique identifier such as a digital identity (ID). For example, display identifiers 0x01, 0x02, and 0x03 can be assigned to the three virtual display environments corresponding to three clients, respectively.

[0043] Optionally, to ensure the integrity of the virtual display function, in the embodiments of this application, each of the multiple virtual display environments includes a rendering interface, an image reading interface, and a display density parameter setting module. Specifically, the rendering interface is responsible for drawing the application interface, the image reading interface captures the rendered frame data for encoding and transmission, and the display density parameter setting module adjusts the size of the interface elements according to the client's DPI.

[0044] Optionally, to address the cross-device display adaptation issue, in embodiments of this application, the display parameters of each virtual display environment are determined based on the display performance of the corresponding client. This display performance includes the adapted resolution, color space, and dots per inch (DPI). For example, if a 4K TV is detected as the client, the virtual environment resolution can be automatically set to 3840×2160 and the HDR color space can be enabled.

[0045] Step S130: In response to the user performing a launch operation on one or more target applications through the multiple clients respectively, create an independent user environment for each of the one or more target applications.

[0046] Specifically, this user environment is used to run the corresponding target application. Each target application within its corresponding user environment includes one or more of the following: an independent process space, a data directory structure, a network session connection, a surface buffer, and system permissions. It should be understood that the user environment is implemented through user mechanisms or containerization technology, providing a sandboxed, isolated runtime environment for each application.

[0047] Optionally, to address the efficiency issue of application state management, in embodiments of this application, after running one or more target applications, the method further includes: acquiring the running state information of the one or more target applications, the running state information including one or more of startup time, application activity, and resource usage; and determining one or more of the pause, resume, destroy, and migration operations for the one or more target applications based on the running state information. For example, if the system monitors that an application has been inactive for more than 5 minutes and is consuming a large amount of memory, it automatically triggers a pause operation to release resources.

[0048] Step S140: Run one or more target applications in the user environment, so that the multiple virtual display environments respectively present the running interface of one or more target applications on the multiple clients.

[0049] Optionally, to address the resource contention problem, in embodiments of this application, the method further includes: obtaining resource usage information of the plurality of virtual display environments; determining the resource usage priority of the plurality of virtual display environments based on the resource usage information; and allocating resources to the plurality of virtual display environments according to the resource usage priority of each virtual display environment. Specifically, the resource usage information may include, for example, CPU utilization, memory usage, GPU rendering load, and network I / O throughput.

[0050] Optionally, to optimize the priority calculation dimension, in the embodiments of this application, the plurality of virtual display environments includes a first virtual display environment. Determining the resource usage priority of the plurality of virtual display environments based on the resource usage information includes: determining the user interaction activity of the first virtual display environment, the network quality information of the first client corresponding to the first virtual display environment, and the application type of the first application presented in the first virtual display environment based on the resource usage information; and determining the resource usage priority of the first virtual display environment based on the user interaction activity, the application type, and the network quality information. For example, user interaction activity is calculated using touch event frequency, application types are categorized as games / videos / tools, etc., and network quality is evaluated using packet loss rate, etc.

[0051] Optionally, to quantify priority calculation, in the embodiments of this application, the user interaction activity corresponds to a first weight, the application type corresponds to a second weight, and the network quality information corresponds to a third weight. The resource usage priority is obtained by summing the first product, the second product, and the third product. The first product is the product of the user interaction activity and the first weight, the second product is the product of the application type and the second weight, and the third product is the product of the network quality information and the third weight. For example, if the interaction activity is 0.8 (weight 0.5), the application type is a game (weight 0.3), and the network quality is good (weight 0.2), then the priority score = 0.8 × 0.5 + 0.9 × 0.3 + 0.7 × 0.2 = 0.83.

[0052] Optionally, to address the resource overload issue, in embodiments of this application, the method further includes: determining, based on resource usage information of the plurality of virtual display environments, that the amount of resources provided by the cloud phone instance is less than or equal to the amount of resources required by the plurality of virtual display environments; and reducing the activity level of at least one of the plurality of virtual display environments, wherein the at least one virtual display environment is in a non-interactive or inactive state. For example, when the total CPU usage exceeds 95%, the system reduces the frame rate of the non-interactive session from 60fps to 10fps.

[0053] Optionally, to address the multi-input routing problem, in embodiments of this application, the method further includes: acquiring user input event information sent by the multiple clients, the user input event information including identification information of the multiple clients and user events; determining the target application launched by the multiple clients based on the identification information of the multiple clients; and sending the user events to the target application launched by the multiple clients.

[0054] Optionally, in order to structure input events, in embodiments of this application, the user event includes one or more of timestamps, event types, and coordinate information.

[0055] Optionally, to cover all interaction types, in embodiments of this application, the event type includes one or more of pressing, moving, releasing, and canceling. For example, a touch screen operation sequence may include pressing (ACTION_DOWN), moving (ACTION_MOVE), and releasing (ACTION_UP).

[0056] Optionally, in order to solve the coordinate mapping problem, in the embodiments of this application, the method further includes: converting the coordinate information of the multiple clients into virtual display local coordinate information based on the identification information of the multiple clients.

[0057] Optionally, to optimize response speed, in embodiments of this application, the method further includes: obtaining user operation habit information corresponding to the multiple clients; and determining the predicted input events of the users of the multiple clients based on the operation habit information. For example, if the system learns that a user frequently swipes in the lower right corner of the game interface, it preloads resources for that area.

[0058] Optionally, to address the video encoding bottleneck, in embodiments of this application, the method further includes: allocating hardware encoder instances to the plurality of virtual display environments; encoding the image data of the plurality of virtual display environments using the hardware encoder instances; and sending the encoding results of the image data of the plurality of virtual display environments to the corresponding plurality of clients. For example, an independent GPU encoder instance is allocated to each session to support H.265 parallel encoding.

[0059] Optionally, to dynamically adapt to network conditions, in embodiments of this application, the encoding parameters of the hardware encoder instance are determined based on the network condition information of the multiple clients. These encoding parameters include one or more of the following: bitrate, frame rate, and I-frame interval. For example, if a network bandwidth drop to 2 Mbps is detected, the bitrate can be dynamically adjusted from 5 Mbps to 1.5 Mbps.

[0060] Optionally, to optimize bitrate allocation, in embodiments of this application, encoding the image data of the multiple virtual display environments using the hardware encoder instance includes: determining the user interaction area in the image data of the multiple virtual display environments; and encoding the image data of the multiple virtual display environments using the hardware encoder instance, wherein the encoding quality of the user interaction area is greater than the encoding quality of other areas in the image data besides the user interaction area. For example, the character control area of ​​the game interface uses QP20 high-quality encoding, and the background area uses QP35 low-quality encoding.

[0061] Optionally, to address the multistream identification issue, in embodiments of this application, sending the encoded processing results of the image data from the multiple virtual display environments to the corresponding multiple clients includes: sending the encoded processing results of the image data from the multiple virtual display environments to the corresponding multiple clients via Real-time Transport Protocol (RTP), wherein the encoded processing results include RTP data packets, and the RTP header of the RTP data packets includes display identifiers for the multiple virtual display environments and identification information for the multiple clients. The RTP header extension field may carry the display identifier 0x01 and the client ID.

[0062] Optionally, to enable cross-session interaction, in embodiments of this application, the multiple virtual display environments include a first virtual display environment and a second virtual display environment. The method further includes: sending data and / or information of the target application in the first virtual display environment to the second virtual display environment through a unified inter-process communication (IPC) interface. Specifically, a cross-user IPC channel based on the Binder mechanism can be established.

[0063] Optionally, to ensure communication security, in embodiments of this application, sending the data and / or information of the target application in the first virtual display environment to the second virtual display environment includes: obtaining communication request information from the target application in the first virtual display environment, the communication request information including the display identifier of the first virtual display environment, the display identifier of the second virtual display environment, data type, and security token; verifying whether the communication permission between the first virtual display environment and the second virtual display environment is approved based on the communication request information; and, if the communication permission verification is successful, sending the data and / or information of the target application in the first virtual display environment to the second virtual display environment. The security token may contain a user digital signature and validity verification information to ensure communication security across virtual display environments.

[0064] In summary, this application achieves efficient reuse of server resources and enhances concurrent processing capabilities by creating independent virtual display environments and user environments for multiple clients on the same cloud phone instance. Furthermore, through dynamic resource allocation strategies, intelligent input event routing, adaptive video encoding technology, and secure cross-environment communication mechanisms, it significantly optimizes system response speed, network bandwidth utilization, and multi-user interaction experience, providing technical support for the large-scale deployment of cloud phone services.

[0065] The following describes a detailed embodiment of method 100 with reference to the accompanying drawings.

[0066] Figure 2 A schematic diagram of the cloud phone multi-application parallel interaction system architecture provided in an embodiment of this application is shown. Figure 2 As shown, the system consists of two parts: a cloud server and client devices.

[0067] The cloud server includes cloud phone instances, a virtual display manager, an input event router, a resource scheduling engine, and a network transmission module. The cloud phone instances provide a virtualized environment for running an operating system; the virtual display manager creates and manages multiple virtual display instances; the input event router handles and accurately routes input events from different clients; the resource scheduling engine dynamically allocates and optimizes system resources; and the network transmission module handles video encoding and network transmission.

[0068] Client devices can be different types of terminal devices such as mobile phones, tablets, and PCs. Each client connects to the cloud server via the network, receives video streams from specific applications, and sends input events.

[0069] First, combine Figure 3 and Figure 4 Introduce the process sequence corresponding to Method 100. Figure 3 and Figure 4 A schematic diagram illustrating the flow and timing of the application interaction method provided in the embodiments of this application is shown.

[0070] like Figure 3 As shown, in step 1, during virtual display creation and application startup, corresponding to step S110, client 1 and client 2 send connection request information to the cloud server. This connection request information may include target application information and display parameters. Then, corresponding to step S120, the virtual display manager creates corresponding virtual display instances -1 and -2 for client 1 and client 2 respectively, based on the connection request information. For example, client 1 corresponds to virtual display instance -1, and client 2 corresponds to virtual display instance -2. Then, corresponding to step S130, applications A and B are started in virtual display instances -1 and -2 respectively, creating independent user environments for applications A and B. For example, virtual display instance -1 corresponds to application A, and virtual display instance -2 corresponds to application B. In step 2, during resource allocation and optimization, the resource scheduling engine can allocate resource quotas to the newly created virtual display instances -1 and -2 based on the application load characteristics of applications A and B.

[0071] like Figure 4 As shown, in step 3, interface rendering and transmission, applications A and B can run normally in virtual display instance-1 and virtual display instance-2 respectively. Consequently, the video stream of the interface content is captured, encoded, and transmitted in real time to the corresponding clients 1 and 2. In step 4, input event processing, user operations from clients 1 and 2 are converted into input events and sent to the server via the network, where they are routed to the correct application instance by the input event router. In step 5, the dynamic optimization phase, the system continuously monitors the operating status and dynamically adjusts resource allocation and transmission parameters to ensure optimal performance.

[0072] The following is in conjunction with the appendix Figures 5 to 13 Introduction to implementation Figure 3 and Figure 4 The principle and algorithm flow of the process shown.

[0073] Figure 5 A schematic diagram of the application interaction method implementation architecture provided in the embodiments of this application is shown. Figure 5 This section details six core technology modules from a technical architecture perspective. Unlike the chronological flow described above, this section analyzes the specific implementation mechanisms, algorithmic principles, and innovative features of each key technology in depth, categorized by functional modules. These six technology modules collaborate to form a complete cloud phone multi-application parallel interaction technology solution. For example... Figure 5As shown, the core technology modules include multi-virtual display creation and configuration management, application instance isolation startup and binding, adaptive resource scheduling and load balancing, input event routing and processing, video encoding and adaptive transmission, and cross-virtual display application collaboration and data synchronization.

[0074] For example, the multi-virtual display creation and configuration management module is responsible for creating an independent virtual display environment for each connected client. For instance, first, the virtual display manager receives a connection request from the client, which includes parameters such as the target application package name, desired resolution, and device type. Based on these parameters, the virtual display manager calls the operating system's DisplayManager service to create a VirtualDisplay instance with a unique display ID. Each virtual display is configured with an independent Surface rendering surface, an ImageReader image reader, and dedicated display density parameters.

[0075] Furthermore, virtual display instances can be created asynchronously to avoid blocking other client connections; each virtual display is assigned a unique display identifier, ranging from 1000. Additionally, display parameters can be dynamically configured based on client device characteristics, including resolution adaptation, DPI adjustment, and color space settings. A fixed-size frame buffer is pre-allocated to each virtual display, with a default size of resolution × 4 bytes (RGBA format); a maximum display refresh rate can be set, dynamically adjusted based on application type (60fps for games, 30fps for general applications).

[0076] For example, the application instance isolation startup and binding module is used to ensure that each application instance in the virtual display runs completely independently and does not interfere with each other. Figure 6 A schematic diagram of the application instance isolation mechanism provided in an embodiment of this application is shown. Specifically, as Figure 6 As shown, after receiving the launch command, the application instance manager first creates an independent user environment for the target application, assigning a unique user ID, for example, starting from 10000 and incrementing. Then, it constructs an Intent launch object with a display ID parameter and binds the application to the specified virtual display using the ActivityOptions.setLaunchDisplayId() method. After the application starts, all its Activities, Services, and BroadcastReceivers run in isolated user space. Furthermore, as... Figure 6As shown, each application instance can have an independent data directory structure ( / data / user / {userId} / ), including application-private data, SharedPreferences, database files, etc. Network connections can use independent socket handles to avoid session conflicts. System permissions can be isolated by user ID to ensure that applications cannot access each other.

[0077] In addition, the application instance isolation startup and binding module can monitor the running status of each application instance, including startup time, activity level, resource usage, etc.; support the pause, resume, and destruction of application instances; and realize hot migration of application instances, which can be migrated between virtual displays without interrupting services.

[0078] For example, the adaptive resource scheduling and load balancing module can be used to dynamically allocate and optimize system resources through intelligent algorithms. For instance, the resource scheduling engine continuously monitors the resource usage of each virtual display, including metrics such as CPU utilization, memory usage, GPU rendering load, and network I / O. Based on this data, a weighted round-robin algorithm can be used to calculate the resource priority of each virtual display. High-priority virtual displays receive more CPU time slices and memory quotas, while low-priority ones have their resource usage restricted.

[0079] Figure 7 A schematic diagram illustrating the resource allocation strategy provided in an embodiment of this application is shown. Figure 7 As shown, the resource allocation strategy categorizes resource allocation into three priority levels, clearly defining the allocation rules for resources (CPU, memory, GPU) corresponding to each priority level. For example, high-priority CPU allocation is 40-60%, memory allocation is 2-4GB, and GPU allocation is prioritized; medium-priority CPU allocation is 20-40%, memory allocation is 1-2GB, and GPU allocation is on demand; low-priority CPU allocation is 10-20%, memory allocation is 0.5-1GB, and GPU allocation is minimal.

[0080] Figure 8 A schematic diagram illustrating the priority calculation method provided in an embodiment of this application is shown. For example... Figure 8 As shown, the priority calculation formula is P = α×Activity + β×AppType + γ×NetworkQuality, where Activity represents user interaction activity, AppType represents the application type weight (e.g., game 0.8 > video 0.6 > social 0.4 > utility 0.2), and NetworkQuality represents the client network quality, such as excellent (1), good (0.8), average (0.6), and poor (0.4). Figure 8As shown, the system dynamically adjusts the resource quota for each virtual display according to priority, such as α = 0.5, β = 0.3, and γ = 0.2, to ensure that important applications receive sufficient resources.

[0081] Furthermore, when system resources are strained, the adaptive resource scheduling and load balancing module automatically reduces the refresh rate and rendering quality of inactive virtual displays. For example, it supports a sleep mechanism for virtual displays, automatically putting displays that have been inactive for extended periods into a low-power state. Additionally, it implements on-demand allocation of GPU resources, allocating GPU resources only to applications that require graphics rendering.

[0082] Figure 9 A flowchart of the resource scheduling algorithm provided in an embodiment of this application is shown. Figure 9 As shown, the process can be broken down into five stages: initialization monitoring, status assessment, strategy execution, resource adjustment, and effect closure. During the initialization and real-time monitoring stages, after system startup, the resource scheduling engine is automatically activated. This engine, as the core control unit, immediately enters continuous monitoring mode. Subsequently, full-dimensional status collection is performed, monitoring all created virtual display instances.

[0083] During the status assessment phase, the first step is to determine whether any resource changes have been detected. If resource changes are detected (yes), the system automatically categorizes and quantifies the collected information, such as labeling the application type as a game, indicating high user interaction frequency, or excellent network quality, and calculates a priority score based on the formula P = α × Activity + β × AppType + γ × NetworkQuality. If no resource changes are detected (no), the system skips the data re-analysis and priority calculation, reducing system overhead.

[0084] Next, a second assessment is made to determine if current resources are sufficient. If resources are sufficient (yes), optimal resources are allocated according to the virtual display's priority. For example, high priority allocates 40-60% CPU, 2-4GB memory, and prioritizes GPU; medium priority allocates 20-40% CPU, 1-2GB memory, and allocates GPU on demand; and low priority allocates 10-20% CPU, 0.5-1GB memory, and minimum GPU allocation. If resources are scarce (no), a load balancing strategy is activated, releasing redundant resources through measures such as degrading inactive displays. For example, it checks for inactive virtual displays. If inactive displays exist (yes), their resource quotas are reduced (e.g., high priority is reduced to medium priority); if no inactive displays exist (yes), a display hibernation mechanism is enabled, such as reducing the refresh rate from 60fps to 15fps or pausing GPU rendering.

[0085] Next, resource adjustments and configuration updates are performed. First, resource quotas are recalculated, and the final resource allocation value for each virtual display is determined based on priority or load balancing strategies. Then, hardware resources are reallocated, such as CPU reallocation via time slices, updating quota limits in the memory manager, dynamically binding computing power through the GPU allocator, and synchronizing application configurations.

[0086] Finally, monitor resource usage effectiveness: After resource adjustments, continuously collect key metrics such as application frame rate, input latency, and resource utilization to determine if resource allocation is reasonable. If reasonable (yes), record the current optimization strategy as a reference template for similar scenarios in the future; if unreasonable (no), adjust algorithm parameters, recalculate priority branches (e.g., temporarily increase the weight of α to 0.6, increase the weight of user interaction activity on priority), and re-execute resource allocation.

[0087] For example, the input event routing and processing module ensures that input events from different clients are accurately delivered to the corresponding application instances. Specifically, the input event router acts as a central scheduler, receiving input data packets from all clients. Each input event can carry a unique identifier for the client, a timestamp, event type, and coordinate information. The input event router queries a mapping table based on the client identifier to determine the target virtual display ID. Then, a coordinate system transformation is performed, converting the client device coordinates to the local coordinates of the virtual display. Furthermore, the input event routing and processing module supports coordinate mapping for different resolutions and screen ratios, such as using bilinear interpolation to ensure coordinate accuracy, and can handle display transformations such as rotation and scaling to guarantee the accuracy of touch positions. In addition, the input event routing and processing module can also support concurrent processing of multi-touch events, with each touch point having an independent tracking ID.

[0088] Furthermore, the input event routing and processing module can implement an input event prediction mechanism, anticipating the next operation based on user habits, and supporting batch processing of events to improve processing efficiency. Additionally, the module can provide input latency compensation, adjusting event timestamps based on network latency.

[0089] Figure 10 A flowchart of the input event routing process provided in an embodiment of this application is shown. Figure 10 As shown, the process can be broken down into four stages: event reception and parsing, coordinate transformation and positioning, event scheduling and conflict handling, and instruction execution and feedback.

[0090] First, the event triggering process begins with client input events (such as user touches the screen or clicks a button), parses the event data packet, extracts the client identifier ID, event type, and coordinates, and adds a server-side timestamp to the event. Then, by querying the client mapping table, coordinate transformation and target positioning are achieved across multiple application scenarios, resulting in a virtual display coordinate system.

[0091] Next, the event type is determined based on the virtual display coordinate system, such as whether it is a touch event, key event, or mouse event. Then, event preprocessing is performed, and input events are initially classified (e.g., multi-touch, single-point input, key trigger, etc.). Furthermore, events are added to a queue according to the server-side timestamp for conflict detection and resolution. If a conflict exists, an application conflict resolution strategy is executed, such as time-slice rotation for time conflicts, priority handling for region conflicts, and blocking other inputs for exclusive conflicts. If there is no conflict, the event is directly sent to the target application.

[0092] After the processed event command is sent to the target application, the event processing log is recorded, the statistics are updated, and the final processing result is returned. Then the process returns to the initial state of waiting for the next input event and continues to loop.

[0093] For example, the video encoding and adaptive transmission module is responsible for encoding the content of the virtual display in real time and transmitting it to the client. Specifically, the video encoding and adaptive transmission module can allocate an independent hardware encoder instance to each virtual display, supporting both H.264 and H.265 encoding formats. The encoder reads image data from the virtual display Surface and performs real-time encoding processing. Encoding parameters can be dynamically adjusted according to the client's network conditions, including bitrate, frame rate, and I-frame interval. The encoded video stream is then transmitted to the corresponding client via the RTP protocol.

[0094] Furthermore, the video encoding and adaptive transmission module can employ region coding technology to use higher encoding quality for user interaction areas and achieve inter-frame prediction optimization, reducing the amount of encoded data. It can also support hardware encoding acceleration, utilizing the GPU's video encoding unit to improve encoding efficiency.

[0095] Optionally, the video encoding and adaptive transmission module can use a custom RTP extension protocol, adding virtual display ID and client ID information to the standard RTP header to achieve adaptive bitrate adjustment and dynamic adjustment of transmission parameters based on network congestion. Furthermore, it supports packet loss retransmission and error concealment to ensure video quality.

[0096] Figure 11 A schematic diagram of the video encoding and transmission architecture provided in an embodiment of this application is shown. Figure 11As shown, the virtual display layer's interface reads and performs color space conversion through the image acquisition layer and preprocessing layer, and then encodes the image through an encoder cluster. Encoding optimization strategies can include adaptive quantization complexity adjustment, RO1 region coding for high-quality key regions, rate-distortion optimization, optimal parameter selection, inter-frame prediction optimization, and motion vector algorithm optimization. Encoding parameters can be controlled through quality controllers, bitrate controllers, and frame rate controllers. Finally, the encoded result is sent to the adapted client through the network adaptation layer and transport protocol layer.

[0097] For example, the cross-virtual display application collaboration and data synchronization module is used to implement secure communication and collaboration functions between applications in different virtual displays. Specifically, the cross-display communication manager provides a unified IPC interface, supporting message passing and data sharing between applications in different virtual displays. Communication can be implemented using an extension based on the Binder mechanism, with each communication request containing the source display ID, target display ID, data type, and security token. After verifying communication permissions, the manager securely transmits the data to the target application.

[0098] In the security mechanism design, all cross-display communication requires authorization verification to prevent malicious applications from stealing data from other applications; communication data is encrypted using AES to ensure secure transmission; and a communication audit function is implemented to record all cross-display communication activities. Furthermore, it supports file sharing between applications, such as images and documents; implements clipboard synchronization, allowing users to copy and paste between different applications; and provides an event notification mechanism, enabling applications to send notification messages to applications in other displays; it also supports application state synchronization, such as sharing login status and configuration information.

[0099] Figure 12 A schematic diagram illustrating the data flow process for cross-display communication provided in an embodiment of this application is shown. For example... Figure 12As shown, the source application first initiates a request, for example, when application A needs to call a function of application B or obtain data from application B, it initiates a request. Next, permission verification is performed, that is, the source application's request is validated for legality, such as verifying whether the source application has the permission to make a request to the target application. Then, data encryption is performed, such as encrypting data carried in the request, such as parameters and commands. Next, message routing and forwarding are performed, using a message routing mechanism to accurately deliver the encrypted request to the target application. The routing module selects the corresponding transmission path based on the target application's identifier (such as application ID, address), ensuring the request is correctly distributed. Then, the target application receives the routed encrypted request and decrypts and verifies the validity of the received encrypted data. For example, it first decrypts the data using the corresponding key, and then verifies the integrity of the data, such as verifying the hash value and the legality of the source. After successful verification, the target application executes the collaborative task requested by the source application and returns the execution result.

[0100] Figure 13 A schematic diagram illustrating the architecture of the cross-virtual display application collaboration mechanism provided in an embodiment of this application is shown. Figure 13 As shown, the cross-display communication manager acts as a global coordination hub, providing a unified IPC interface and handling the reception, routing, and distribution of collaboration requests between different virtual displays. The security controller includes encryption and decryption modules such as those employing AES-256 encryption, Access Control List (ACL) modules, digital signature verification modules, and abnormal behavior detection modules to ensure data security and access compliance in cross-display communication. Shared data pools, file sharing areas, and event notification queues provide underlying storage and transmission support for data exchange, state synchronization, and file transfer between applications.

[0101] Figure 13 The exhibition also showcased three typical virtual display instances and their corresponding running applications. For example, the first virtual display ran a social networking application, including a chat module, a file management module, and independent application data storage. The second virtual display ran a game application, including a screenshot module, a game engine, and independent application data storage. The third virtual display ran a video application, including a live streaming module, a video editing module, and independent application data storage.

[0102] Furthermore, it can be connected via a data transmission channel. Figure 13 The applications in each virtual display shown are uniformly scheduled by the cross-display communication manager to achieve... Figure 12 The complete process is shown below. Furthermore, all cross-display data transmissions are encrypted using AES-256, and communication requires ACL permission verification and digital signature validation. Simultaneously, the communication audit module records operation logs, and the abnormal behavior detection module monitors risks in real time.

[0103] In summary, the user interaction method proposed in this application can be achieved through, as follows: Figure 5 The collaborative operation of the six core technology modules shown enables the simultaneous provision of independent application services to multiple clients within a single cloud phone instance. Specifically, the core innovation of this solution lies in the deep integration of virtual display functionality with multi-client interaction technology, constructing a complete multi-tenant cloud phone service architecture. This technical solution not only solves the technical problems of low resource utilization and poor concurrency capabilities in traditional cloud phone services, but also provides new technical pathways for multiple industries such as cloud computing, mobile internet, and gaming.

[0104] Compared to the background technology described above, the solution in this application has the following advantages: 1. Significantly improve resource utilization: By running multiple applications in parallel on a single cloud phone instance, the server resource utilization rate is increased from 20-30% in a single-user scenario to over 80%, and resource reuse is achieved by using multi-virtual display parallel management technology.

[0105] 2. Significantly improve concurrent processing capabilities: A single cloud phone instance can serve 3-8 clients simultaneously, increasing user capacity by 3-8 times compared to traditional solutions, and achieving high concurrency support through intelligent resource scheduling algorithms.

[0106] 3. Optimize user interaction experience: Each client gets its own application interface, avoiding the redundant information of the complete Android desktop that users see in traditional solutions. Application instance isolation technology ensures the independence of user experience.

[0107] 4. Reduce network bandwidth consumption: Only the application interface content needed by the user is transmitted, which can save 40-60% of network bandwidth compared to transmitting the entire desktop. Adaptive video encoding technology is used to optimize bandwidth.

[0108] 5. Enhanced system scalability: Supports dynamic addition and removal of virtual display instances, allowing for flexible expansion of service capabilities based on actual needs, and supports complex application scenarios through cross-virtual display collaboration mechanisms.

[0109] This application provides a computer program product that, when run on an electronic device, causes the electronic device to execute the technical solutions described in the above embodiments. Its implementation principle and technical effects are similar to those of the related embodiments described above, and will not be repeated here.

[0110] This application provides a readable storage medium containing instructions that, when executed by an electronic device, cause the electronic device to perform the technical solution described in the above embodiments. The implementation principle and technical effects are similar and will not be repeated here.

[0111] This application provides a chip for executing instructions. When the chip is running, it executes the technical solutions described in the above embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0112] The hardware module of this application is described below, which can be used to implement the aforementioned method 100, and can also implement... Figure 3 and Figure 4 as well as Figures 5 to 13 The process is shown below.

[0113] Now for reference Figure 14 The diagram shows a block diagram of a device 1400 according to one embodiment of this application. Device 1400 may include one or more processors 1401 coupled to a controller hub 1403. In at least one embodiment, the controller hub 1403 communicates with the processor 1401 via a multi-branch bus such as a front side bus (FSB), a point-to-point interface such as a quick path interconnect (QPI), or a similar connection 1410. The processor 1401 executes instructions controlling general types of data processing operations. In one embodiment, the controller hub 1403 includes, but is not limited to, a graphics memory controller hub (GMCH) (not shown) and an input / output hub (IOH) (which may be on a separate chip) (not shown), wherein the GMCH includes memory and a graphics controller and is coupled to the IOH.

[0114] Device 1400 may also include a coprocessor 1402 and a memory 1404 coupled to a controller hub 1403. Alternatively, one or both of the memory and the GMCH may be integrated within the processor, with memory 1404 and coprocessor 1402 directly coupled to processor 1401 and controller hub 1403, which resides on a single chip with the IOH. Memory 1404 may be, for example, dynamic random access memory (DRAM), phase change memory (PCM), or a combination of both. In one embodiment, coprocessor 1402 is a dedicated processor, such as, for example, a high-throughput many integrated core (MIC) processor, a network or communication processor, a compression engine, a graphics processor, a general-purpose computing on GPU (GPGPU), or an embedded processor, etc. Optional properties of coprocessor 1402 are indicated by dashed lines. Figure 14 middle.

[0115] As a computer-readable storage medium, memory 1404 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. For example, memory 1404 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device such as one or more hard-disk drives (HDD(s)), one or more compact disc (CD) drives, and / or one or more digital versatile disc (DVD) drives.

[0116] In one embodiment, device 1400 may further include a network interface controller (NIC) 1406. NIC 1406 may include a transceiver for providing a radio interface to device 1400, thereby enabling communication with any other suitable device (such as a front-end module, antenna, etc.). In various embodiments, NIC 1406 may be integrated with other components of device 1400. NIC 1406 can implement the functionality of the communication unit in the above embodiments.

[0117] Device 1400 may further include input / output (I / O) device 1405. I / O 1405 may include: a user interface designed to enable a user to interact with device 1400; a peripheral component interface designed to enable peripheral components to also interact with device 1400; and / or sensors designed to determine environmental conditions and / or location information related to device 1400.

[0118] It is worth noting that, Figure 14 This is merely an example. That is, although... Figure 14 The diagram shows that device 1400 includes multiple devices such as processor 1401, controller hub 1403, and memory 1404. However, in actual applications, devices using the methods of this application may include only a portion of the devices in device 1400. For example, it may include only processor 1401 and NIC 1406. Figure 14 The properties of the optional devices are shown in dashed lines. According to some embodiments of this application, the memory 1404, which is a computer-readable storage medium, stores instructions that, when executed on a computer, cause the device 1400 to perform the methods according to the above embodiments. Specific details can be found in the methods of the above embodiments, and will not be repeated here.

[0119] Now for reference Figure 15The diagram shown is a block diagram of a system-on-chip (SoC) 1500 according to an embodiment of this application. Figure 15 In the diagram, similar components share the same reference numerals. Additionally, dashed boxes are an optional feature for more advanced SoCs. Figure 15 In this SoC 1500, the following are included: an interconnect unit 1550 coupled to an application processor 1510; a system proxy unit 1580; a bus controller unit 1590; an integrated memory controller unit 1540; a group or one or more coprocessors 1520, which may include integrated graphics logic, an image processor, an audio processor, and a video processor; a static random access memory (SRAM) unit 1530; and a direct memory access (DMA) unit 1560. In one embodiment, the coprocessor 1520 includes a dedicated processor, such as, for example, a network or communication processor, a compression engine, a GPGPU, a high-throughput MIC processor, or an embedded processor.

[0120] The static random-access memory (SRAM) cell 1530 may include one or more computer-readable media for storing data and / or instructions. The computer-readable storage medium may store instructions, specifically, temporary and permanent copies of those instructions. These instructions may include, when executed by at least one unit in the processor, causing the SoC 1500 to perform the attention training method according to the above embodiments, as detailed in the methods described above, which will not be repeated here.

[0121] Various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or combinations of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0122] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0123] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0124] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, CD-ROMs, compact disc read-only memory (CD-ROMs), magneto-optical disks, read-only memory (ROM), random-access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other forms of propagated signals. Therefore, machine-readable media includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0125] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the accompanying drawings. Furthermore, including structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0126] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0127] It should be noted that in the examples and description of this patent, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0128] Although this application has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art should understand that various changes in form and detail may be made thereto without departing from the spirit and scope of this application.

Claims

1. An application running method, characterized in that, Cloud phone instances used on cloud servers include: Obtain connection request information sent by multiple clients to the same cloud phone instance, wherein the connection request information is used by the client to request a connection to the cloud phone instance; Based on the connection request information, multiple independent virtual display environments are created, wherein each virtual display environment corresponds one-to-one with a client, and each virtual display environment is used to present the corresponding client; In response to a user performing a startup operation on one or more target applications through the multiple clients, an independent user environment is created for each of the one or more target applications, wherein the user environment is used to run the corresponding target application, and each target application in the corresponding user environment includes one or more of the following: independent process space, data directory structure, network session connection, surface buffer, and system permissions; The one or more target applications are run in the user environment, so that the multiple virtual display environments respectively present the running interface of the one or more target applications on the multiple clients.

2. The method according to claim 1, characterized in that, The connection request information includes one or more of the following for each of the multiple clients: the target application package name, the expected image parameters, and the device type.

3. The method according to claim 1 or 2, characterized in that, Each of the multiple virtual display environments corresponds to a multiple display identifier, and each pair of display identifiers is different.

4. The method according to claim 1 or 2, characterized in that, Each of the multiple virtual display environments includes a rendering interface, an image reading interface, and a display density parameter setting module.

5. The method according to claim 1 or 2, characterized in that, The display parameters of each virtual display environment are determined based on the display performance of the corresponding client, including the adapted resolution, color space, and dots per inch (DPI).

6. The method according to claim 1 or 2, characterized in that, After running the one or more target applications, the method further includes: Obtain the running status information of one or more target applications, wherein the running status information includes one or more of the following: startup time, application activity, and resource usage; Based on the running status information, determine one or more of the pause, resume, destroy, and migrate operations for the one or more target applications.

7. The method according to claim 1 or 2, characterized in that, The method further includes: Obtain resource usage information for the multiple virtual display environments; Based on the resource usage information, the resource usage priority of the multiple virtual display environments is determined; Resources are allocated to the multiple virtual display environments based on the resource usage priority of each virtual display environment.

8. The method according to claim 7, characterized in that, The plurality of virtual display environments includes a first virtual display environment. Determining the resource usage priority of the plurality of virtual display environments based on the resource usage information includes: Based on the resource usage information, determine the user interaction activity level of the first virtual display environment, the network quality information of the first client corresponding to the first virtual display environment, and the application type of the first application presented in the first virtual display environment. Based on the user interaction activity, the application type, and the network quality information, the resource usage priority of the first virtual display environment is determined.

9. The method according to claim 8, characterized in that, The user interaction activity corresponds to the first weight, the application type corresponds to the second weight, the network quality information corresponds to the third weight, and the resource usage priority is obtained by summing the first product, the second product, and the third product. The first product is the product of the user interaction activity and the first weight, the second product is the product of the application type and the second weight, and the third product is the product of the network quality information and the third weight.

10. The method according to claim 7, characterized in that, The resource usage information includes one or more of the following: CPU utilization, memory usage, GPU rendering load, and network input / output I / O.

11. The method according to claim 7, characterized in that, The method further includes: Based on the resource usage information of the multiple virtual display environments, it is determined that the amount of resources provided by the cloud phone instance is less than or equal to the amount of resources required by the multiple virtual display environments; Reduce the activity level of at least one of the plurality of virtual display environments, wherein the at least one virtual display environment is in a non-interactive or inactive state.

12. The method according to claim 1 or 2, characterized in that, The method further includes: Obtain user input event information sent by the multiple clients, wherein the user input event information includes the identification information of the multiple clients and user events; Based on the identification information of the multiple clients, the target application launched by the multiple clients is determined; The user event is sent to the target application launched by the multiple clients.

13. The method according to claim 12, characterized in that, The user event includes one or more of the following: timestamp, event type, and coordinate information.

14. The method according to claim 13, characterized in that, The event types include one or more of the following: press, move, lift, and cancel.

15. The method according to claim 12, characterized in that, The method further includes: Based on the identification information of the multiple clients, the coordinate information of the multiple clients is converted into virtual display local coordinate information.

16. The method according to claim 12, characterized in that, The method further includes: Obtain the user operation habit information corresponding to the multiple clients; Based on the aforementioned user behavior information, predictable input events for users of the multiple clients are determined.

17. The method according to claim 1 or 2, characterized in that, The method further includes: Assign hardware encoder instances to the multiple virtual display environments; The image data of the multiple virtual display environments are encoded using the hardware encoder instance. The encoding processing results of the image data of the multiple virtual display environments are sent to the corresponding multiple clients respectively.

18. The method according to claim 17, characterized in that, The encoding parameters of the hardware encoder instance are determined based on the network status information of the multiple clients, and the encoding parameters include one or more of the following: bit rate, frame rate, and I-frame interval.

19. The method according to claim 17, characterized in that, The process of encoding the image data of the multiple virtual display environments using the hardware encoder instance includes: Determine the user interaction area in the image data of the multiple virtual display environments; The hardware encoder instance is used to encode the image data of the multiple virtual display environments, wherein the encoding quality of the user interaction area is greater than the encoding quality of other areas of the image data besides the user interaction area.

20. The method according to claim 17, characterized in that, The step of sending the encoding processing results of the image data of the multiple virtual display environments to the corresponding multiple clients includes: The encoding processing results of the image data of the multiple virtual display environments are sent to the corresponding multiple clients via the Real-time Transport Protocol (RTP). The encoding processing results include RTP data packets, and the RTP header of the RTP data packets includes the display identifiers of the multiple virtual display environments and the identification information of the multiple clients.

21. The method according to claim 1 or 2, characterized in that, The plurality of virtual display environments includes a first virtual display environment and a second virtual display environment, and the method further includes: Data and / or information of the target application in the first virtual display environment are sent to the second virtual display environment through a unified inter-process communication (IPC) interface.

22. The method according to claim 21, characterized in that, The step of sending the data and / or information of the target application in the first virtual display environment to the second virtual display environment: Obtain communication request information from the target application in the first virtual display environment. The communication request information includes the display identifier of the first virtual display environment, the display identifier of the second virtual display environment, the data type, and the security token. Based on the communication request information, verify whether the communication permissions between the first virtual display environment and the second virtual display environment are granted; If the communication permission verification is successful, the data and / or information of the target application in the first virtual display environment will be sent to the second virtual display environment.

23. An electronic device, characterized in that, It includes one or more processors; one or more memories; said one or more memories storing one or more computer programs, said one or more computer programs including instructions that, when executed by said one or more processors, cause the method of any one of claims 1 to 22 to be performed.

24. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed on a computer, cause the method as described in any one of claims 1 to 22 to be performed.

25. A chip, characterized in that, The chip includes a processor and a communication interface, the communication interface being used to receive signals and transmit the signals to the processor, the processor processing the signals such that the method as described in any one of claims 1 to 22 is executed.

26. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1 to 22.

Images