A cross-container application fusion switching method of a honkong system

Abstract

The application discloses a cross-container application fusion switching method of a Hongmeng system, a system service layer deploys a container application management service, and an application framework layer implements a proxy application manager; when a container application starts, a container side allocates shared memory and configures permissions, a container manager collects metadata and initiates registration, a container application management service converts a memory handle into a texture handle, allocates a unique identifier, and triggers the proxy application manager to generate a proxy application, and the proxy application initializes a Vulkan rendering environment; the container side renders an application interface to the shared memory, the proxy application imports a texture and constructs a lightweight Vulkan rendering pipeline, scales sampling to generate a thumbnail of the container application; when the container application exits, the shared memory is released, the container application management service cleans up shared memory references, triggers the destruction of the proxy application, and recycles texture resources through a reference counter, so that the container application is seamlessly fused in the form of a native application thumbnail, low-delay switching is realized, and resource efficient utilization is realized.

Description

Technical Field

[0001] This invention belongs to the field of computer software development technology, specifically relating to a method for cross-container application integration and switching in the HarmonyOS system. Background Technology

[0002] When running Android or Linux applications using container technology within HarmonyOS, the application's running state and interface information are relatively isolated within the container. Existing technologies for displaying thumbnails across container applications suffer from three main problems: first, the high latency due to bitmap copying results in low data transmission efficiency; second, container applications cannot participate in HarmonyOS's native task management and switching processes, leading to poor system integration; and third, frequent image data processing consumes excessive CPU or GPU resources, resulting in high resource usage. Summary of the Invention

[0003] In view of this, the present invention provides a method for cross-container application fusion and switching in HarmonyOS, which realizes seamless integration and display of container applications and native HarmonyOS applications, thereby improving system performance and user experience.

[0004] This invention provides a method for cross-container application integration and switching in the HarmonyOS system, which specifically includes the following steps:

[0005] Step 1: Deploy the container application management service in the system service layer of HarmonyOS, implement the proxy application manager in the application framework layer, and establish a persistent connection between the container application management service and the container manager.

[0006] Step 2: When starting the container application, the container allocates a graphics memory region as the first shared memory. The proxy application manager has read permissions, and the container has read and write permissions, generating a first memory handle for the first shared memory. The container manager collects the container application's metadata and sends a first registration request, including metadata and the first memory handle, to the container application management service. After receiving the first registration request, the container application management service converts the first memory handle into a texture handle, assigns a unique identifier to the container application, and sends a proxy application creation request, including the unique identifier, metadata, and texture handle, to the proxy application manager. After receiving the proxy application creation request, the proxy application manager instantiates the proxy application template to complete the installation and startup of the first proxy application. During the initialization of the first proxy application, a Vulkan rendering environment is built.

[0007] Step 3: The container side creates a first texture for the container application and binds it to the first shared memory, then renders the interface onto the first texture; the container application management service creates a reference counter for the texture; the first proxy application imports the texture handle as an external texture, scales and samples to obtain a thumbnail, and renders the thumbnail;

[0008] Step 4: When the container application exits, it releases the first shared memory. After the container application management service detects the memory release, it cleans up the reference to the memory and notifies the proxy application manager to destroy the proxy application. When the proxy application is destroyed, it decrements the reference counter of the corresponding texture. When the count is 0, the texture resource is released.

[0009] Furthermore, the method for instantiating the proxy application template to complete the installation and startup of the first proxy application is as follows: the name placeholder and icon placeholder in the proxy application template are replaced with the name and icon of the container application to obtain the first proxy application. The application configuration of the first proxy application is generated according to the application type, and the UUID constant code is dynamically generated and injected into the first proxy application.

[0010] Furthermore, the container allocates two contiguous graphics memory regions as the first shared memory, including a first texture buffer and a second texture buffer of the same size; the container renders to the background buffer, and the container application management service reads texture data from the foreground buffer; after the container completes the rendering of a frame, it swaps the foreground and background buffer pointers through atomic operations, and at the same time ensures that the switching operation is only performed after rendering is completed through synchronization primitives; the container application management service creates semaphores for the first texture buffer and the second texture buffer respectively, and activates the corresponding buffer semaphore after the container completes the buffer switching, and the container application management service reads the data of the new foreground buffer after receiving the signal.

[0011] Furthermore, set a timeout for waiting for semaphores for the container application management service.

[0012] Furthermore, the method for obtaining the thumbnail through scaling sampling is as follows: the first proxy application creates a descriptor set containing an adapted image view and texture sampler, uses native thumbnail rendering parameter configuration, sets vertex shaders and fragment shaders, and constructs a lightweight rendering pipeline optimized for thumbnails; after the container side completes the update of the first texture, it binds the lightweight rendering pipeline and descriptor set to obtain the thumbnail through scaling sampling from the external texture and completes the rendering of the thumbnail.

[0013] Furthermore, the method for constructing a lightweight rendering pipeline optimized for thumbnails is as follows: the vertex shader and fragment shader are compiled into bytecode, the vertex shader is responsible for handling texture coordinate transformation, and the fragment shader is responsible for texture sampling and color adjustment; the pipeline layout is defined, and the descriptor set layout and push constants are associated; the rendering pass is configured to be single-draw, without depth testing, and with color blending enabled to achieve blending with the window background; and pipeline caching is used to accelerate pipeline creation.

[0014] Furthermore, before the first proxy application imports the texture handle as an external texture, the metadata in the texture handle is parsed to extract the original format of the container texture. If the original format is compatible with the target format of the proxy application's rendering pipeline, it is directly imported; otherwise, real-time format conversion is performed through the GPU shader, and the texture sampling method is automatically adjusted according to the size of the proxy application window.

[0015] Furthermore, the proxy application template includes an application configuration file, capability components, and resource files. The application configuration file is used to define the basic information, permission declarations, and supported device types of the proxy application. The capability components include the primary capability responsible for background communication and the secondary capability responsible for interface rendering. The resource files include icon placeholders and name placeholders.

[0016] Furthermore, the proxy application includes: a communication layer that uses the HarmonyOS IPC mechanism to achieve long-connection communication between the first capability and the container application management service; a data layer that maintains locally cached application state and textures to achieve local data persistence; a rendering layer that integrates the Vulkan rendering engine within the second capability to render shared textures to the application window; and an interaction layer that implements user interaction logic.

[0017] Furthermore, the lifecycle of the proxy application is synchronized with that of the container application. Specifically, the proxy application is installed and started within a set time after the container application starts. When the container application enters the foreground or background, the proxy application updates the application status synchronously. When the container application exits, the proxy application actively exits after receiving a notification from the container application management service. When the proxy application crashes unexpectedly, the container application management service detects the anomaly and triggers a reconstruction process to recover from the anomaly. Beneficial effects

[0018] This invention deploys a container application management service at the system service layer and implements a proxy application manager at the application framework layer. When a container application starts, the container allocates shared memory and configures permissions. The container manager collects metadata and initiates registration. The container application management service converts memory handles to texture handles, assigns unique identifiers, and triggers the proxy application manager to generate a proxy application. The proxy application initializes the Vulkan rendering environment. The container renders the application interface to shared memory. The proxy application imports textures and builds a lightweight Vulkan rendering pipeline, scaling and sampling to generate a thumbnail of the container application. When the container application exits, the shared memory is released. The container application management service cleans up shared memory references, destroys the proxy application, and reclaims texture resources through a reference counter. This achieves seamless integration of the container application with native application thumbnails, low-latency switching, and efficient resource utilization. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a cross-container application fusion and switching method for the HarmonyOS system provided by the present invention. Detailed Implementation

[0020] The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0021] This invention provides a method for cross-container application fusion and switching in HarmonyOS. The core idea is as follows: A container application management service is deployed at the system service layer, and a proxy application manager is implemented at the application framework layer. When a container application starts, the container allocates shared memory and configures permissions. The container manager collects metadata and initiates registration. The container application management service converts memory handles to texture handles, assigns unique identifiers, and triggers the proxy application manager to generate a proxy application. The proxy application initializes the Vulkan rendering environment. The container renders the application interface to shared memory. The proxy application imports textures and builds a lightweight Vulkan rendering pipeline, scaling and sampling to generate a thumbnail of the container application. When the container application exits, the shared memory is released. The container application management service cleans up shared memory references, triggers the destruction of the proxy application, and reclaims texture resources through a reference counter. This achieves seamless fusion of container applications in the form of native application thumbnails, low-latency switching, and efficient resource utilization.

[0022] This invention provides a method for cross-container application integration and switching in the HarmonyOS system, the process of which is as follows: Figure 1 As shown, the specific steps include:

[0023] Step 1: In the HarmonyOS system, establish and deploy a container application management service in the system service layer. The container application management service is used to maintain the full lifecycle state machine of container applications, manage cross-process shared graphics resources and proxy applications, and establish a persistent connection between the container application management service and the HarmonyOS system container manager. In the application framework layer, implement a proxy application manager, which is used to generate proxy applications according to the proxy application template, install proxy applications, and complete the lifecycle binding with container applications.

[0024] Furthermore, the container application management service runs as an independent system process, has access to shared memory and graphics hardware, and is registered as a system-level service through the HarmonyOS service registration mechanism to ensure high priority and stability.

[0025] The proxy application template includes an application configuration file, capability components, and resource files. The application configuration file is used to define the basic information, permission declarations, and supported device types of the proxy application. The capability components include the primary capability responsible for background communication and the secondary capability responsible for interface rendering. The resource files include basic resources such as icon placeholders and name placeholders.

[0026] Furthermore, the internal architecture of the proxy application includes: a communication layer, which uses the HarmonyOS IPC mechanism to achieve long-connection communication between the first capability and the container application management service; a data layer, which maintains locally cached application state and texture information to achieve local data persistence; a rendering layer, which integrates the Vulkan rendering engine to render shared textures to the application window; and an interaction layer, which implements basic user interaction logic, such as switching to the corresponding container application when clicking the proxy application thumbnail. The application state includes running, paused, crashed, foreground process, and background process.

[0027] Furthermore, the lifecycle synchronization between the proxy application and the container application is as follows: after the container application starts, the proxy application is installed and started within a set time to achieve startup synchronization; when the container application enters the foreground or background, the proxy application updates the application status synchronously to achieve status synchronization; when the container application exits, the proxy application actively exits after receiving a notification from the container application management service to achieve exit synchronization; when the proxy application crashes unexpectedly, the container application management service detects the anomaly and triggers the reconstruction process to recover from the anomaly.

[0028] Furthermore, the proxy application template can adopt a modular design, separating variable parts such as application name and icon from fixed parts such as communication logic and rendering framework, which facilitates dynamic replacement later.

[0029] Step 2: When the container application is started in the HarmonyOS system, the container side allocates a contiguous graphics memory region as the first shared memory through the ION memory management mechanism. The ION permission interface is used to allocate read permissions for the first shared memory to the agent application manager. The container side has read and write permissions for the first shared memory. The first memory handle of the first shared memory is generated, including information such as physical address, memory size, and format.

[0030] The container manager collects the metadata of the container application, generates the container application's signature based on the container application's package name and container ID, and sends a first registration request to the container application management service. The first registration request includes the container application's metadata and a first memory handle.

[0031] After receiving the first registration request, the container application management service converts the first memory handle into a texture handle on the HarmonyOS system side, assigns a unique identifier to the container application, registers the process death notification of the container application, and sends a proxy application creation request to the proxy application manager, which includes the unique identifier, metadata and texture handle of the container application.

[0032] After receiving the proxy application creation request, the proxy application manager instantiates the proxy application template based on the container application's unique identifier and metadata to generate the first proxy application, generates the application configuration of the first proxy application based on the application type, dynamically generates UUID constant code and injects it into the first proxy application, installs and starts the first proxy application, and establishes a lifecycle binding between the container application and the first proxy application.

[0033] When the first agent application initializes, it builds a Vulkan rendering environment, converts the native window surface of the HarmonyOS system into a Vulkan-compatible surface according to the Vulkan standard, creates a swap chain, and creates a frame buffer and rendering channel for each image buffer in the swap chain.

[0034] The metadata of a container application includes its name, package name, process ID, main window handle, window size, icon, first launch time, and application type. The application type describes the source platform of the container application, including Android applications and Linux applications.

[0035] The first proxy application is obtained by instantiating the proxy application template based on the unique identifier and metadata of the container application. Specifically, the name placeholder and icon placeholder in the proxy application template are replaced with the name and icon of the container application, respectively, to obtain the first proxy application.

[0036] Furthermore, to eliminate concurrent conflicts between container-side texture rendering and container application management service texture reading, and to ensure data consistency, this invention employs double buffering to achieve shared graphics memory regions, specifically:

[0037] The container allocates two contiguous graphics memory regions as the first shared memory through the ION memory management mechanism, including a first texture buffer and a second texture buffer of the same size; the container buffers the rendering to the background, and the container application management service reads the texture data from the foreground buffer.

[0038] After the container side completes the rendering of a frame, it swaps the foreground and background buffer pointers through atomic operations, and at the same time, it ensures through synchronization primitives that the switching operation is only performed after the rendering is complete.

[0039] The container application management service creates semaphores for the first texture buffer and the second texture buffer respectively. After the container side completes the buffer switch, it activates the semaphore of the corresponding buffer. After receiving the signal, the container application management service immediately reads the data of the new foreground buffer.

[0040] Furthermore, to prevent container application management services from waiting indefinitely due to container-side anomalies, a timeout period for waiting for semaphores is set.

[0041] The method for creating the swap chain is as follows: query the supported formats, color spaces, and rendering modes of the surface, select the optimal configuration, call vkCreateSwapchainKHR to create the swap chain, and set the minimum number of images to 3 to balance smoothness and memory usage.

[0042] Step 3: The container side creates a first texture for the container application and binds the first texture to the first shared memory. The compositor periodically renders the container application interface to the first texture and writes the first texture data to the first shared memory.

[0043] The container application management service creates a reference counter for each texture; the first proxy application imports the texture handle as an external texture recognizable by Vulkan, and creates a descriptor set containing an adapted image view and texture sampler, configures native thumbnail rendering parameters, sets vertex shaders and fragment shaders, and builds a lightweight rendering pipeline optimized for thumbnails; after the container side completes the update of the first texture data, it binds the lightweight rendering pipeline and descriptor set to obtain thumbnails by scaling and sampling from the external texture and completes the rendering of the thumbnails, so that the container application is displayed in the thumbnail mode of the native application, and the user switches the container application through the thumbnails.

[0044] Furthermore, in order to be compatible with the rendering output of different types of container applications, this invention parses the metadata in the texture handle to extract the original format of the container texture before the first proxy application imports the texture handle as an external texture that Vulkan can recognize. If the original format is compatible with the target format of the proxy application's rendering pipeline, it is directly imported; otherwise, it performs real-time format conversion through the GPU shader and automatically adjusts the texture sampling method according to the window size of the proxy application.

[0045] The texture sampler configuration includes setting the filtering method, addressing mode, and anisotropic filtering. The filtering method can be linear filtering when zooming in and mipmap filtering when zooming out.

[0046] To reduce rendering overhead, this invention constructs a lightweight rendering pipeline optimized for thumbnails, specifically:

[0047] The SPIR-V compiler is used to compile the vertex shader and fragment shader into bytecode. The vertex shader is responsible for handling texture coordinate transformations, and the fragment shader is responsible for texture sampling and color adjustment. The pipeline layout is defined, and the descriptor set layout and push constants are associated to pass dynamic parameters such as texture size. The rendering pass is configured to be single-draw, without depth testing, and with color blending enabled to achieve blending with the window background. Pipeline caching is used to accelerate pipeline creation and reduce application startup time.

[0048] Step 4: When the container application exits, it actively releases the first shared memory through the ION interface. After the container application management service detects the release of ION memory, it cleans up the local reference to that memory and notifies the proxy application manager to destroy the proxy application. When the proxy application is destroyed, the reference counter of the corresponding texture is decremented. When the count is 0, the texture resource is released. Example

[0049] Taking the OpenHarmony system as an example in this embodiment, a cross-container application fusion and switching method for the HarmonyOS system provided by this invention is adopted to achieve seamless fusion and display of container applications and OpenHarmony native applications, thereby improving user experience and system performance. The specific process includes:

[0050] S1. Registration and management of application metadata within the container.

[0051] S1.1 System Service Deployment: Deploy the Container Application Management Service (CAMS) in the OpenHarmony System Service Layer. The CAMS service has the following core capabilities:

[0052] Establish a persistent Binder IPC connection with the Container Manager, maintain the full lifecycle state machine of container applications, manage cross-process shared graphics resources, and coordinate the creation and destruction of proxy applications.

[0053] CAMS runs as an independent system process, i.e., uid=system, and has special permissions to access shared memory and graphics hardware. It is registered as a system-level service through OpenHarmony's service registration mechanism SystemAbilityManager to ensure high priority and stability.

[0054] S1.2 Application startup event listener: The container manager monitors the state changes of applications within the container in real time, such as startup, switching, pausing, and exiting. When a new application is detected to be starting, the following steps are executed:

[0055] Collect application metadata, including but not limited to application name, package name, process ID, main window handle, icon resources, first launch time, application type, etc.; generate application feature code, which is a unique identifier based on the combination of package name and container ID, used to distinguish applications with the same name in different containers; establish a communication channel, and send a registration request to CAMS through the Binder mechanism, carrying application metadata and shared texture descriptor.

[0056] S1.3 Metadata Management Mechanism: After receiving a registration request, CAMS performs the following processing steps:

[0057] The UUID is generated by combining the OpenHarmony distributed device ID and timestamp algorithm to generate a globally unique application identifier. The global UUID generation strategy based on the OpenHarmony distributed capabilities supports cross-device application identification.

[0058] A mapping relationship is established, and a four-tuple mapping table of UUID, application metadata, texture handle and state is created and stored in a secure storage area. This four-tuple mapping relationship design realizes the unified management of application metadata, graphics resources and state.

[0059] State monitoring registers the Death Recipient for container processes, automatically triggering the cleanup process when the container application exits, achieving end-to-end state management that combines Binder death notification with distributed synchronization;

[0060] S1.4 Exception handling mechanism: CAMS implements a comprehensive exception handling strategy.

[0061] Timeout retransmission: When a registration request times out and does not respond, it will automatically retry 3 times at intervals of 100ms, 200ms, and 400ms respectively; Data verification: The integrity of the received metadata will be verified, and incomplete or incorrectly formatted registration requests will be rejected; Resource reclamation: When the container process exits abnormally, the UUID mapping table will be cleaned up and the proxy application will be destroyed.

[0062] S2. Dynamic generation of proxy applications based on the OpenHarmony Ability framework.

[0063] S2.1 Design a proxy application template that conforms to the OpenHarmony application model. This template includes the following core components, and a dynamic proxy application generation mechanism based on the template enables real-time system integration of container applications:

[0064] The application configuration file app.json5 defines basic application information, permission declarations, supported device types, etc. Permission declarations include graphical access permissions and system interaction permissions; capability components include ServiceAbility responsible for background communication and UIAbility responsible for interface rendering; resource files contain basic resources such as default icons and name placeholders; and lifecycle management scripts define the processing logic for application creation, destruction, and switching states.

[0065] S2.2 Implementation of the Proxy Application Manager: The Proxy Application Manager (PAM) is implemented at the application framework layer. This manager performs the following tasks:

[0066] Template parsing: Reads the proxy application template and identifies replaceable variables and resources; Application generation: Dynamically replaces variables in the template based on container application metadata to generate a complete application package structure; Application signing: Signs the dynamically generated application using a system-level signing certificate to ensure it obtains the necessary system permissions; Application installation management: Implements silent installation of the proxy application through OpenHarmony's AppInstallerService; Lifecycle binding: Establishes a lifecycle binding relationship between the proxy application and the container application to achieve synchronized startup and shutdown.

[0067] S2.3 Dynamic Generation Process: After PAM receives the creation request for CAMS, it executes the following steps:

[0068] Metadata parsing: Extracting key information such as the container application's name, icon, and window size; resource replacement: Replacing icon placeholders in the template with the actual icons of the container application, and name placeholders with the actual names; configuration generation: Generating adapted application configurations based on the container application type, such as window mode and display density; code injection: Dynamically generating UUID-related constant code to ensure the proxy application can correctly associate with the corresponding container application; package building: Packaging the replaced resources and code into HAP (Harmony Ability Package) format; silent installation: Calling the AppInstallerService interface for silent installation, skipping the user confirmation process; startup triggering: After installation, starting the proxy application's UIAbility and ServiceAbility through the application launcher Launcher interface.

[0069] S2.4 Internal Architecture of the Proxy Application: The dynamically generated proxy application adopts a layered architecture design. This layered architecture design achieves decoupling and independent optimization of communication, data, rendering, and interaction.

[0070] Communication Layer: ServiceAbility implements long-term connection communication with CAMS, using OpenHarmony's IPC mechanism, including Binder and message queue; Data Layer: Maintains locally cached application state and texture information, achieving local data persistence; Rendering Layer: UIAbility integrates the Vulkan rendering engine, responsible for rendering shared textures to the application window; Interaction Layer: Implements basic user interaction logic, such as switching to the corresponding container application when clicking on the proxy application thumbnail.

[0071] S2.5 Lifecycle Management: Mechanism for synchronizing the lifecycle of proxy applications with that of container applications.

[0072] Startup Synchronization: The proxy application is installed and started within 500ms after the container application starts; State Synchronization: When the container application enters the foreground or background, the proxy application updates its state synchronously; Exit Synchronization: When the container application exits, the proxy application actively calls the terminateSelf() method to exit after receiving a notification from CAMS; Exception Recovery: When the proxy application crashes unexpectedly, CAMS detects the exception and triggers the reconstruction process.

[0073] S3, a cross-container texture synchronization mechanism based on shared video memory.

[0074] S3.1 Shared memory management: Utilizing OpenHarmony's ION (Input / Output Memory) memory management mechanism to achieve cross-container memory sharing. Based on this ION cross-container memory sharing mechanism, zero-copy texture data transfer is achieved.

[0075] Memory allocation: At application startup, the container allocates two contiguous graphics memory regions (double buffering) via ION, specifying the memory type as GRAPHICS_MEMORY to support direct GPU access. Permission configuration: Through ION's permission management interface, read permissions for the memory region are set for the CAMS process on the OpenHarmony side, while preserving read and write permissions on the container side. Handle passing: The container passes the ION memory handle to CAMS via Binder IPC; the handle contains information such as the memory's physical address, size, and format. Memory reclamation: When the container application exits, the container releases shared memory through the ION interface; CAMS detects the memory release and cleans up local references.

[0076] S3.2 Texture creation and sharing: A unified texture creation strategy is adopted on both the container side and the OpenHarmony side.

[0077] On the container side, 2D textures are created using OpenGL ES and bound to the ION memory region. The texture format is set to RGBA8888 to ensure compatibility, and mipmap generation is enabled to support scaled display. For texture updates, the container-side compositor renders the application interface to the texture at a default 30fps. After completion, it calls relevant APIs, such as OpenGL ES's glFlush() and Vulkan's vkQueueSubmit, to ensure that data is written to video memory. For handle conversion, CAMS converts the received ION handles into local texture IDs on the OpenHarmony side and creates texture objects that can be shared across contexts through EGL's eglCreateImageKHR interface. For reference management, CAMS creates a reference counter for each texture. The counter is decremented when the proxy application is destroyed, and the texture resource is released when the counter reaches zero. This reference-counted texture resource management achieves efficient utilization of system resources.

[0078] S3.3, Double Buffering Mechanism: To avoid conflicts between texture rendering and reading, a double buffering strategy is adopted. Double buffering, combined with atomic operations, eliminates conflicts between rendering and display.

[0079] Buffer initialization: Create two texture buffers of the same size (Buffer A and Buffer B);

[0080] Render buffer: The container side always renders to the back buffer (initially Buffer B);

[0081] Display buffer: The OpenHarmony side always reads data from the foreground buffer (initially Buffer A);

[0082] Buffer switching: After the container side completes the rendering of a frame, it swaps the pointers of the foreground and background buffers through atomic operations and notifies the OpenHarmony side to use the new foreground buffer.

[0083] Synchronization Guarantee: Use the GPU's fences synchronization primitives to ensure that buffer switching operations are performed after rendering is complete.

[0084] S3.4 Semaphore Synchronization Mechanism: Implementing cross-process synchronization based on semaphores.

[0085] Semaphore creation: CAMS creates two anonymous semaphores (semA and semB) during texture initialization, each corresponding to one of the two buffers;

[0086] Waiting mechanism: On the OpenHarmony side, sem_wait() is called before reading the texture to wait for the corresponding buffer semaphore;

[0087] Notification mechanism: After the container side completes buffer rendering and switches, it calls sem_post() to activate the corresponding buffer's semaphore;

[0088] Timeout handling: Set a timeout of 100ms to avoid permanent waiting caused by container-side anomalies.

[0089] S3.5 Texture format adaptation: Handles texture format differences across different application types, with an adaptive texture format conversion mechanism that ensures compatibility with rendering output from various container applications.

[0090] Format detection: CAMS detects the texture format of the container application during the registration phase, such as RGB565, RGBA8888, BGRA8888, etc.

[0091] Format conversion: If the formats are incompatible, real-time format conversion is performed by the GPU shader to avoid CPU involvement;

[0092] Size adaptation: Automatically adjusts the texture sampling method based on the size of the proxy application window, such as stretching, cropping, and maintaining proportions;

[0093] Color correction: Apply the color management configuration of the OpenHarmony system to ensure that the color performance of container applications is consistent with that of native applications.

[0094] / / Core implementation of texture receiver and synchronization mechanism

[0095] class TextureReceiver : public ITextureReceiver {

[0096] public:

[0097] TextureReceiver(const std::string& uuid)

[0098] : uuid_(uuid), currentBufferIndex_(0), bufferCount_(2) {

[0099] / / Initialize semaphores

[0100] sem_init(&semaphores_[0], 1, 0); / / For cross-process sharing, pshared=1 needs to be set.

[0101] sem_init(&semaphores_[1], 1, 0);

[0102] / / Initialize fences

[0103] for (int i = 0; i < bufferCount_; i++) {

[0104] fences_[i] = EGL_NO_SYNC_KHR;

[0105] }

[0106] }

[0107] bool Initialize(int format, int width, int height) {

[0108] / / Store texture properties

[0109] format_ = format;

[0110] width_ = width;

[0111] height_ = height;

[0112] / / Create an EGL context

[0113] eglDisplay_ = eglGetDisplay(EGL_DEFAULT_DISPLAY);

[0114] eglInitialize(eglDisplay_, nullptr, nullptr);

[0115] / / Configure EGL attributes

[0116] const EGLint configAttrs[] = {

[0117] EGL_SURFACE_TYPE, EGL_PBUFFER_BIT,

[0118] EGL_RED_SIZE, 8,

[0119] EGL_GREEN_SIZE, 8,

[0120] EGL_BLUE_SIZE, 8,

[0121] EGL_ALPHA_SIZE, ۸,

[0122] EGL_NONE

[0123] };

[0124] EGLConfig config;

[0125] EGLint numConfigs;

[0126] eglChooseConfig(eglDisplay_, configAttrs, &config, 1, &numConfigs);

[0127] / / Create an EGL context

[0128] const EGLint contextAttrs[] = {

[0129] EGL_CONTEXT_CLIENT_VERSION, 3,

[0130] EGL_NONE

[0131] };

[0132] eglContext_ = eglCreateContext(eglDisplay_, config, EGL_NO_CONTEXT,contextAttrs);

[0133] return eglContext_ != EGL_NO_CONTEXT;

[0134] }

[0135] void OnTextureUpdated(int bufferIndex, EGLSyncKHR fence) override {

[0136] std::lock_guard <std::mutex>lock(mutex_);

[0137] / / Waiting for the fence signal from the previous frame

[0138] if (fences_[bufferIndex] != EGL_NO_SYNC_KHR) {

[0139] eglClientWaitSyncKHR(eglDisplay_, fences_[bufferIndex],

[0140] EGL_SYNC_FLUSH_COMMANDS_BIT_KHR, EGL_FOREVER_KHR);

[0141] eglDestroySyncKHR(eglDisplay_, fences_[bufferIndex]);

[0142] }

[0143] / / Save the new fence

[0144] fences_[bufferIndex] = fence;

[0145] / / Switch the current buffer index

[0146] currentBufferIndex_ = bufferIndex;

[0147] / / Notify the waiting read threads

[0148] sem_post(&semaphores_[bufferIndex]);

[0149] }

[0150] int AcquireTexture(GLuint& textureId, int& width, int& height) {

[0151] / / Waiting for semaphore

[0152] int bufferIndex = currentBufferIndex_;

[0153] int ret = sem_wait(&semaphores_[bufferIndex]);

[0154] if (ret != 0) {

[0155] return -errno;

[0156] }

[0157] std::lock_guard <std::mutex>lock(mutex_);

[0158] / / Waiting for the GPU to finish rendering

[0159] if (fences_[bufferIndex] != EGL_NO_SYNC_KHR) {

[0160] eglClientWaitSyncKHR(eglDisplay_, fences_[bufferIndex],

[0161] EGL_SYNC_FLUSH_COMMANDS_BIT_KHR, EGL_FOREVER_KHR);

[0162] }

[0163] / / Return texture information

[0164] textureId = textures_[bufferIndex];

[0165] width = width_;

[0166] height = height_;

[0167] return 0;

[0168] }

[0169] ~TextureReceiver() {

[0170] / / Clean up resources

[0171] for (int i = 0; i < bufferCount_; i++) {

[0172] sem_destroy(&semaphores_[i]);

[0173] if (fences_[i] != EGL_NO_SYNC_KHR) {

[0174] eglDestroySyncKHR(eglDisplay_, fences_[i]);

[0175] }

[0176] }

[0177] eglDestroyContext(eglDisplay_, eglContext_);

[0178] eglTerminate(eglDisplay_);

[0179] }

[0180] private:

[0181] std::string uuid_;

[0182] int format_;

[0183] int width_;

[0184] int height_;

[0185] GLuint textures_[2];

[0186] int currentBufferIndex_;

[0187] int bufferCount_;

[0188] sem_t semaphores_[2];

[0189] EGLSyncKHR fences_[2];

[0190] EGLDisplay eglDisplay_;

[0191] EGLContext eglContext_;

[0192] std::mutex mutex_;

[0193] };

[0194] S4. High-efficiency thumbnail rendering implementation based on Vulkan.

[0195] S4.1 Vulkan rendering environment initialization: The UIAbility of the proxy application builds a complete Vulkan rendering environment during the initialization phase.

[0196] Instance Creation: Call `vkCreateInstance` to create a Vulkan instance, enabling necessary extensions specific to OpenHarmony, such as `VK_KHR_surface`, `VK_HARMONY_surface`, and `VK_KHR_swapchain`; Physical Device Selection: Enumerate the GPU devices in the system, prioritizing integrated graphics cards that support texture sharing and hardware acceleration; Logical Device Creation: Create logical devices based on the selected physical devices, specifying the required queue families, including graphics queues and rendering queues; Command Pool Initialization: Create an independent command pool for each rendering thread to avoid inter-thread synchronization overhead; Debug Layer Integration: Enable the Vulkan debug layer in development mode to capture rendering errors and performance bottlenecks.

[0197] S4.2 Integration with OpenHarmony Window System: Achieve deep integration between Vulkan and the OpenHarmony window system, deeply integrating the Vulkan rendering solution of the OpenHarmony window system to achieve efficient hardware-accelerated rendering.

[0198] Surface creation: Obtain the window surface through OpenHarmony's window manager WindowManager, and call vkCreateHarmonySurfaceKHR to convert it into a Vulkan-compatible surface VkSurfaceKHR;

[0199] Switch chain configuration: Query the supported formats, color spaces and rendering modes of the surface, select the optimal configuration, and prioritize VK_PRESENT_MODE_FIFO_KHR to ensure low latency. Dynamic switch chain reconstruction mechanism adapts to changes in window size and display mode switching.

[0200] Swap chain creation: vkCreateSwapchainKHR is called to create a swap chain, and the minimum number of images is set to 3 (triple buffering) to balance smoothness and memory usage. Triple buffering and multi-semaphore synchronization strategy are used to balance rendering smoothness and resource usage.

[0201] Render target generation: Creates a framebuffer and a renderpass for each image in the swap chain, and configures color attachments and clear values.

[0202] S4.3 Texture resource binding: Efficiently binds shared textures to the Vulkan rendering pipeline.

[0203] External Texture Import: Import textures from shared video memory as Vulkan external textures (VkImage) using the vkImportAndroidHardwareBufferEXT interface (compatible with Android containers) or the vkImportMemoryFdKHR interface; Image View Creation: Create an image view (VkImageView) for the imported textures, specifying the correct format and dimensions; Sampler Configuration: Create a texture sampler (VkSampler), setting the filtering method, addressing mode (Clamp to edge), and anisotropic filtering; Descriptor Set Management: Create a descriptor pool and descriptor set layout, binding the texture view and sampler to the descriptor set, thus establishing the association with the fragment shader.

[0204] S4.4 Rendering Pipeline Construction: Optimize the rendering pipeline for efficient thumbnail rendering. A lightweight rendering pipeline optimized for thumbnail rendering reduces drawing overhead.

[0205] Shader compilation: The SPIR-V compiler is used to compile the vertex shader and fragment shader into bytecode. The vertex shader is responsible for handling texture coordinate transformations, and the fragment shader is responsible for texture sampling and color adjustment.

[0206] Pipeline Layout: Defines the pipeline layout VkPipelineLayout, associates the descriptor set layout with push constants, and is used to pass dynamic parameters such as texture size; Rendering Pass Configuration: Configures the rendering pass to single draw, without depth testing, and enables color blending to achieve blending with the window background; Pipeline Cache: Uses the pipeline cache VkPipelineCache to accelerate pipeline creation and reduce application startup time.

[0207] S4.5 On-demand rendering mechanism: Implements an intelligent rendering strategy based on content changes, and an intelligent on-demand rendering mechanism based on content changes and time control, significantly reducing system resource consumption.

[0208] Change detection: Determines whether a redraw is needed by comparing texture metadata, such as modification timestamps and content hashes, between two consecutive frames; Rendering trigger: Rendering is only performed when a change in texture content is detected, the window size is changed, or a system-mandated redraw notification is received; Frequency limit: Sets the maximum rendering frequency (default 60fps) to avoid excessive rendering and resource consumption; Background pause: Pauses the rendering process to release GPU resources when the proxy application enters the background.

[0209] S4.6 Rendering Synchronization and Presentation: Ensure that the rendering results are displayed on the screen in a timely and accurate manner.

[0210] Semaphore Synchronization: Two semaphores, imageAvailableSemaphore and renderFinishedSemaphore, are used to synchronize the image availability and rendering completion events of the swap chain, respectively; Fence Mechanism: A fence is set for each submitted rendering command to ensure that the rendering of the previous frame is completed before the rendering of the new frame begins; Rendering Queue: The rendered image is submitted to the rendering queue and processed by the OpenHarmony compositor for final display; Frame Time Control: The rendering time of each frame is monitored, and the rendering resolution is dynamically adjusted when it exceeds 16ms (60fps standard).

[0211] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.< / std::mutex> < / std::mutex>

Claims

1. A method for cross-container application fusion switching of a HongMeng system, characterized in that, Specifically comprising the following steps: Step 1, the system service layer of the Hongmeng system deploys a container application management service, and the application framework layer implements a proxy application manager, and the container application management service establishes a persistent connection with a container manager; Step 2, when starting the container application, a graphics memory area is allocated on the container side as a first shared memory, the proxy application manager has read permission thereof, the container side has read and write permissions, a first memory handle of the first shared memory is generated; The container manager collects metadata of the container application, and sends a first registration request including the metadata and the first memory handle to the container application management service; After receiving the first registration request, the container application management service converts the first memory handle into a texture handle, allocates a unique identifier for the container application, and sends a proxy application creation request including the unique identifier, the metadata and the texture handle to the proxy application manager; after receiving the proxy application creation request, the proxy application manager instantiates a proxy application template to complete installation and start of a first proxy application; the first proxy application initializes to build a Vulkan rendering environment; Step 3, the container side creates a first texture for the container application and binds it to the first shared memory, and renders an interface to the first texture; the container application management service creates a reference counter for the texture; the first proxy application imports the texture handle as an external texture, scales and samples to obtain a thumbnail, and renders the thumbnail; Step 4, when the container application exits, the first shared memory is released, the container application management service detects the memory release, cleans up the reference of the memory, notifies the proxy application manager to destroy the proxy application, and when the proxy application is destroyed, the reference counter of the corresponding texture is decremented, and when the counter is 0, the texture resource is released; The container side allocates two continuous graphics memory areas as the first shared memory, including a first texture buffer and a second texture buffer with the same size; the container side renders to a background buffer, and the container application management service reads texture data from a foreground buffer; after the container side completes a frame of rendering, the foreground and background buffer pointers are exchanged through atomic operation, and the switching operation is ensured to be executed only after the rendering is completed through a synchronization primitive; the container application management service creates a semaphore for the first texture buffer and the second texture buffer respectively, the container side activates the semaphore of the corresponding buffer after completing buffer switching, and the container application management service reads the data of the new foreground buffer after receiving the signal.

2. The cross-container application fusion handover method of claim 1, wherein, The way of instantiating the proxy application template to complete installation and start of the first proxy application is: replacing the name placeholder and the icon placeholder in the proxy application template with the name and the icon of the container application to obtain the first proxy application, generating an application configuration of the first proxy application according to the application type, and dynamically generating a UUID constant code to inject into the first proxy application.

3. The cross-container application fusion handover method of claim 2, wherein, The container application management service is set with a timeout time of a waiting semaphore.

4. The cross-container application fusion handover method of claim 1, wherein, The way of scaling and sampling to obtain a thumbnail is: the first proxy application creates a descriptor set including an adaptive image view and a texture sampler, adopts native thumbnail rendering parameter configuration, sets a vertex shader and a fragment shader, and constructs a lightweight rendering pipeline optimized for the thumbnail; After the container side completes the update of the first texture, the lightweight rendering pipeline and the descriptor set are bound to obtain a thumbnail by external texture scaling sampling and complete the rendering of the thumbnail.

5. The cross-container application fusion handover method of claim 4, wherein, The lightweight rendering pipeline optimized for the thumbnail is constructed in the following manner: a vertex shader and a fragment shader are compiled into bytecode, the vertex shader is responsible for processing texture coordinate transformation, and the fragment shader is responsible for texture sampling and color adjustment; a pipeline layout is defined, a descriptor set layout is associated and a constant is pushed; a rendering channel is configured as single draw, no depth test and color blending enabled to realize fusion with the window background; and pipeline caching is used to accelerate pipeline creation.

6. The cross-container application fusion handover method of claim 1, wherein, Before the first proxy application imports the texture handle as an external texture, metadata in the texture handle is parsed to extract the original format of the container texture, if the original format is compatible with the target format of the rendering pipeline of the proxy application, the texture handle is directly imported, otherwise real-time format conversion is performed through a GPU shader, and the texture sampling mode is automatically adjusted according to the window size of the proxy application.

7. The cross-container application fusion handover method of claim 1, wherein, The proxy application template includes an application configuration file, a capability component and a resource file, the application configuration file is used to define basic information, permission declaration and supported device types of the proxy application, the capability component includes a first capability responsible for background communication and a second capability responsible for interface rendering, and the resource file includes an icon placeholder and a name placeholder.

8. The cross-container application fusion handover method of claim 7, wherein, The proxy application includes: a communication layer that realizes long connection communication between the first capability and the container application management service by using an IPC mechanism of a Hongmeng system; a data layer that maintains locally cached application states and textures and realizes local persistence of data; a rendering layer that internally integrates a Vulkan rendering engine in the second capability and is responsible for rendering shared textures to an application window; and an interaction layer that realizes user interaction logic.

9. The cross-container application fusion handover method of claim 1, wherein, The life cycle of the proxy application is synchronized with that of the container application, specifically: after the container application is started, installation and startup of the proxy application are completed within a set time, the application state is updated synchronously when the container application enters the foreground or the background, the proxy application actively exits after receiving a notification from the container application management service when the container application exits, and the container application management service detects an exception and triggers a reconstruction process to recover the exception when the proxy application crashes unexpectedly.

Images