Cross-container application fusion switching method of swan gap system

Abstract

The invention discloses a cross-container application fusion switching method of a swan monk system, which comprises the following steps: a system service layer deploys a container application management service, an application framework layer realizes a proxy application manager, when a container application is started, a container side allocates a shared memory and configures authority, the container manager collects metadata to initiate registration, and the application framework layer realizes a proxy application manager; the container application management service converts a memory handle into a texture handle, allocates a unique identifier and triggers a proxy application manager to generate a proxy application, and the proxy application initializes a Vulkan rendering environment; the container side renders an application interface to a shared memory, the proxy application imports texture and constructs a lightweight Vulkan rendering pipeline, and scaling sampling is carried out to generate a thumbnail of the container application; and when the container application exits, the shared memory is released, the container application management service cleans the shared memory reference, triggers and destroys the proxy application, and recycles the texture resources through the reference counter, so that seamless fusion, low-delay switching and efficient resource utilization of the container application in a native application thumbnail form are realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer software development, and particularly relates to a cross-container application fusion switching method of a Hongmeng system. BACKGROUND

[0002] When an Android or Linux application is run in a Hongmeng system through a container technology, the running state and interface information of the application in the container are in a relatively isolated environment. In the prior art, there are three core problems in the display of a cross-container application thumbnail: first, the bitmap copying method is used, so that the delay is too high, resulting in low data transmission efficiency; second, the container application cannot participate in the task management and switching process of the Hongmeng system, resulting in poor system integration; and third, frequent image data processing consumes too much CPU or GPU resources, resulting in a large resource occupancy rate. SUMMARY

[0003] Therefore, the application provides a cross-container application fusion switching method of a Hongmeng system, which realizes seamless fusion display of a container application and a Hongmeng system native application, and improves system performance and user experience.

[0004] The application provides a cross-container application fusion switching method of a Hongmeng system, which specifically comprises the following steps: Step 1, a system service layer of the Hongmeng system deploys a container application management service, and an application framework layer implements a proxy application manager, the container application management service and a container manager establish a persistent connection; Step 2, when starting the container application, a graphics memory area on the container side is allocated as a first shared memory, the proxy application manager has read permission thereof, the container side has read and write permissions, and a first memory handle of the first shared memory is generated; the container manager collects metadata of the container application, sends a first registration request including the metadata and the first memory handle to the container application management service; after receiving the first registration request, the container application management service converts the first memory handle into a texture handle, allocates a unique identifier for the container application, and sends a proxy application creation request containing the unique identifier, the metadata and the texture handle to the proxy application manager; after receiving the proxy application creation request, the proxy application manager instantiates a proxy application template to complete installation and startup of a first proxy application; when the first proxy application is initialized, a Vulkan rendering environment is constructed; Step 3, the container side creates a first texture for the container application and binds it to the first shared memory, and renders an interface to the first texture; the container application management service creates a reference counter for the texture; the first proxy application imports the texture handle as an external texture, scales and samples to obtain a thumbnail, and renders the thumbnail; Step 4, the first shared memory is released when the container application exits, the container application management service detects the memory release, cleans up the reference of the memory, notifies the agent application manager to destroy the agent application, the agent application is destroyed, the reference counter of the corresponding texture is decremented, and the texture resource is released when the counter is 0.

[0005] Further, the installation and starting of the first agent application by instantiating the agent application template are as follows: the name placeholder and the icon placeholder in the agent application template are replaced by the name and the icon of the container application to obtain the first agent application, the application configuration of the first agent application is generated according to the application type, and the UUID constant code is dynamically generated and injected into the first agent application.

[0006] Further, two continuous graphics memory areas are allocated on the container side as the first shared memory, including a first texture buffer and a second texture buffer with the same size; the container side renders in the background buffer, the container application management service reads the texture data from the front buffer; after the container side completes a frame of rendering, the front and background buffer pointers are exchanged through atomic operation, and the switching operation is ensured to be executed only after the rendering is completed through a synchronization primitive; the container application management service creates a semaphore for the first texture buffer and the second texture buffer, the semaphore of the corresponding buffer is activated after the container side completes the buffer switching, and the container application management service reads the data of the new front buffer after receiving the signal.

[0007] Further, a timeout time for the container application management service to wait for the semaphore is set.

[0008] Further, the way of scaling and sampling to obtain the thumbnail is as follows: the first agent application creates a descriptor set containing an adaptive image view and a texture sampler, adopts native thumbnail rendering parameters configuration, sets a vertex shader and a fragment shader, and constructs a lightweight rendering pipeline optimized for a thumbnail; after the container side completes the update of the first texture, the lightweight rendering pipeline and the descriptor set are bound to sample the external texture to obtain the thumbnail and complete the rendering of the thumbnail.

[0009] Further, the way of constructing the lightweight rendering pipeline optimized for the thumbnail is as follows: the vertex shader and the fragment shader are compiled into bytecode, the vertex shader is responsible for processing texture coordinate transformation, and the fragment shader is responsible for texture sampling and color adjustment; a pipeline layout is defined, the descriptor set layout is associated and constant is pushed; the rendering channel is configured as single draw, no depth test and color blending enabled, and fusion with the window background is realized; the pipeline cache is used to accelerate pipeline creation.

[0010] Further, before the first agent application imports the texture handle as an external texture, the metadata in the texture handle is parsed to extract the original format of the container texture, if the original format is compatible with the target format of the rendering pipeline of the agent application, it is directly imported, otherwise real-time format conversion is performed through a GPU shader, and the texture sampling mode is automatically adjusted according to the window size of the agent application.

[0011] Further, the agent application template includes an application configuration file, a capability component and a resource file, the application configuration file is used to define basic information, permission declaration and supported device types of the agent application, the capability component includes a first capability responsible for background communication and a second capability responsible for interface rendering, and the resource file includes an icon placeholder and a name placeholder.

[0012] Further, the agent application comprises: a communication layer, which realizes long connection communication of the first capability and the container application management service by adopting an IPC mechanism of the hypercongnitive system; a data layer, which maintains locally cached application state and texture, and realizes local persistence of data; a rendering layer, which internally integrates a Vulkan rendering engine in the second capability, and is responsible for rendering the shared texture to an application window; and an interaction layer, which realizes user interaction logic.

[0013] Further, the life cycle of the agent application is synchronized with that of the container application, specifically: after the container application is started, installation and starting of the agent application are completed within a set time, the application state is synchronously updated when the container application enters the foreground or the background, the agent application actively exits after receiving the notification of the container application management service when the container application exits, and the container application management service detects the exception and triggers the reconstruction process to recover the exception when the agent application crashes unexpectedly. Beneficial effects

[0014] The application realizes seamless fusion, low-delay switching and efficient resource utilization of the container application in the form of a native application thumbnail by deploying a container application management service in a system service layer, implementing an agent application manager in an application framework layer, when the container application is started, allocating shared memory and configuring permissions on the container side, collecting metadata to initiate registration by the container manager, converting the memory handle into a texture handle by the container application management service, allocating a unique identifier, and triggering the agent application manager to generate an agent application, and initializing the Vulkan rendering environment by the agent application; rendering the application interface to the shared memory on the container side, importing the texture by the agent application and constructing a lightweight Vulkan rendering pipeline, and generating a thumbnail of the container application by scaling sampling; when the container application exits, releasing the shared memory, cleaning up the shared memory reference by the container application management service, destroying the agent application, and recycling the texture resource through the reference counter. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 A flowchart of a cross-container application fusion and switching method of a hypercongnitive system is provided. DETAILED DESCRIPTION

[0016] The application will be described in detail below with reference to the accompanying drawings.

[0017] The application provides a cross-container application fusion switching method of a Hongmeng system, and the core idea is that a container application management service is deployed in a system service layer, a proxy application manager is implemented in an application framework layer, when a container application starts, shared memory is allocated on the container side and permissions are configured, a container manager collects metadata and initiates registration, the container application management service converts a memory handle into a texture handle, allocates a unique identifier, and triggers the proxy application manager to generate a proxy application, and the proxy application initializes a Vulkan rendering environment; the container side renders an application interface to the shared memory, the proxy application imports a texture and constructs a lightweight Vulkan rendering pipeline, scales sampling to generate a thumbnail of the container application; when the container application exits, the shared memory is released, the container application management service cleans up shared memory references, triggers the proxy application to be destroyed, and recycles texture resources through a reference counter, and seamless fusion, low-delay switching and efficient resource utilization of the container application in the form of a native application thumbnail are realized.

[0018] The application provides a cross-container application fusion switching method of a Hongmeng system, and the core idea is that a container application management service is deployed in a system service layer, a proxy application manager is implemented in an application framework layer, when a container application starts, shared memory is allocated on the container side and permissions are configured, a container manager collects metadata and initiates registration, the container application management service converts a memory handle into a texture handle, allocates a unique identifier, and triggers the proxy application manager to generate a proxy application, and the proxy application initializes a Vulkan rendering environment; the container side renders an application interface to the shared memory, the proxy application imports a texture and constructs a lightweight Vulkan rendering pipeline, scales sampling to generate a thumbnail of the container application; when the container application exits, the shared memory is released, the container application management service cleans up shared memory references, triggers the proxy application to be destroyed, and recycles texture resources through a reference counter, and seamless fusion, low-delay switching and efficient resource utilization of the container application in the form of a native application thumbnail are realized. Figure 1 Step 1, a container application management service is established and deployed in a system service layer in the Hongmeng system, the container application management service is used for maintaining a full life cycle state machine of a container application, managing cross-process shared graphic resources and a proxy application, and establishing a persistent connection between the container application management service and a container manager of the Hongmeng system; a proxy application manager is implemented in an application framework layer, and is used for generating a proxy application according to a proxy application template, installing the proxy application and completing life cycle binding with the container application.

[0019] Further, the container application management service runs in an independent system process, has the permission to access shared memory and graphic hardware, is registered as a system-level service through a service registration mechanism of the Hongmeng system, and ensures high priority and stability.

[0020] The proxy application template includes an application configuration file, a capability component and a resource file, the application configuration file is used for defining basic information, permission declaration and supported device types of the proxy application, the capability component includes a first capability responsible for background communication and a second capability responsible for interface rendering, and the resource file includes an icon placeholder, a name placeholder and other basic resources.

[0021] ​Further, the internal architecture of the agent application comprises: a communication layer, which realizes long connection communication of the first capability and the container application management service by using the IPC mechanism of the Hongmeng system; a data layer, which maintains locally cached application state and texture information, and realizes local persistence of data; a rendering layer, which internally integrates a Vulkan rendering engine for the second capability, and is responsible for rendering shared textures to an application window; and an interaction layer, which realizes basic user interaction logic, such as switching to the corresponding container application when the agent application thumbnail is clicked. The application state includes running, pausing, crashing, foreground process, background process, and the like.

[0022] Further, the life cycle of the agent application is synchronized with the container application, specifically: the container application completes installation and startup of the agent application within a set time after startup, to realize startup synchronization; the agent application synchronously updates the application state when the container application enters the foreground or the background, to realize state synchronization; the agent application actively exits after receiving a notification from the container application management service when the container application exits, to realize exit synchronization; and the container application management service detects an exception and triggers a reconstruction process to recover the exception when the agent application crashes unexpectedly.

[0023] Further, the agent application template can be designed modularly, with variable parts such as application name and icon separated from fixed parts such as communication logic and rendering framework, to facilitate subsequent dynamic replacement.

[0024] In step 2, when the container application is started in the Hongmeng system, the container side allocates a continuous graphics memory area as the first shared memory through the ION memory management mechanism, allocates read permission of the first shared memory for the agent application manager through the ION permission interface, the container side has read and write permissions of the first shared memory, and a first memory handle of the first shared memory is generated, including physical address, memory size, format, and the like information; The container manager collects metadata of the container application, generates a feature code of the container application based on the package name and the container ID of the container application, sends a first registration request to the container application management service, and the first registration request includes the metadata and the first memory handle of the container application; After the container application management service receives the first registration request, the first memory handle is converted into a texture handle on the Hongmeng system side, a unique identifier is allocated for the container application, a process death notification of the container application is registered, a proxy application creation request is sent to the agent application manager, and the proxy application creation request includes the unique identifier, the metadata, and the texture handle of the container application; After the agent application manager receives the proxy application creation request, the agent application template is instantiated to generate a first agent application according to the unique identifier and the metadata of the container application, the application configuration of the first agent application is generated according to the application type, the UUID constant code is dynamically generated and injected into the first agent application, the first agent application is installed and started, and the container application and the first agent application are bound in life cycle; The first proxy application initializes to build a Vulkan rendering environment, converts a native window surface of the Harmony system into a Vulkan standard compatible surface of Vulkan, creates a swap chain, and creates a frame buffer and a rendering channel for each image buffer in the swap chain.

[0025] The metadata of the container application includes a name, a package name, a process ID, a main window handle, a window size, an icon, a first start time, an application type, and the like, wherein the application type is used to indicate that the source platform of the container application includes an Android application and a Linux application.

[0026] The first proxy application is instantiated according to the unique identifier and the metadata of the container application, and the specific mode is as follows: the name placeholder and the icon placeholder in the proxy application template are replaced by the name and the icon of the container application to obtain the first proxy application.

[0027] Further, in order to eliminate the concurrent conflict between the container-side rendering texture and the texture reading of the container application management service and guarantee data consistency, the application adopts double buffering to realize sharing of a graphics memory area, and the specific mode is as follows: The container side allocates two continuous graphics memory areas as the first shared memory through an ION memory management mechanism, including a first texture buffer and a second texture buffer with the same size; the container side renders in the background buffer, and the container application management service reads texture data from the foreground buffer; After the container side completes one frame of rendering, the foreground and background buffer pointers are exchanged through atomic operation, and at the same time, a synchronization primitive is used to ensure that the switching operation is only executed after the rendering is completed. The container application management service creates a semaphore for the first texture buffer and the second texture buffer, the container side activates the semaphore of the corresponding buffer after completing the buffer switching, and the container application management service immediately reads the data of the new foreground buffer after receiving the signal.

[0028] Further, in order to avoid that the container-side exception causes the container application management service to wait permanently, a timeout time of the waiting semaphore is set.

[0029] The method for creating the swap chain is as follows: the supported format, color space and rendering mode of the surface are queried, the optimal configuration is selected, vkCreateSwapchainKHR is called to create the swap chain, and the minimum number of images is set to 3 to balance the smoothness and memory occupation.

[0030] Step 3: The container side creates a first texture for the container application and binds the first texture to the first shared memory, and the compositor periodically renders the container application interface to the first texture and writes the first texture data into the first shared memory. The container application management service creates a reference counter for each texture; the first proxy application imports the texture handle as an external texture recognizable by Vulkan, creates a descriptor set containing an adapted image view and a texture sampler, configures the native thumbnail rendering parameters, sets the vertex shader and the fragment shader, and builds a lightweight rendering pipeline optimized for thumbnails; after the container side completes the update of the first texture data, the lightweight rendering pipeline and the descriptor set are bound to scale the external texture to obtain a thumbnail and complete the rendering of the thumbnail, so that the container application is displayed in the thumbnail mode of the native application, and the user completes the switching of the container application through the thumbnail.

[0031] Further, in order to be compatible with the rendering output of different types of container applications, the application parses the metadata in the texture handle to extract the original format of the container texture before the first proxy application imports the texture handle as an external texture recognizable by Vulkan; if the original format is compatible with the target format of the rendering pipeline of the proxy application, the texture handle is directly imported; otherwise, real-time format conversion is performed through a GPU shader, and the texture sampling mode is automatically adjusted according to the window size of the proxy application.

[0032] The configuration of the texture sampler includes setting the filtering mode, the addressing mode, and the anisotropic filtering, and the filtering mode can be linear filtering when zooming in and mipmap filtering when zooming out.

[0033] In order to reduce the rendering overhead, the application builds a lightweight rendering pipeline optimized for thumbnails, specifically: The vertex shader and the fragment shader are compiled into bytecode using the SPIR-V compiler, the vertex shader is responsible for processing texture coordinate transformation, and the fragment shader is responsible for texture sampling and color adjustment; the pipeline layout is defined, the descriptor set layout and the push constant are associated, and dynamic parameters such as texture size are used for transmission; the rendering channel is configured as single draw, no depth test, and color blending enabled, to realize fusion with the window background; pipeline caching is used to accelerate pipeline creation, to reduce application startup time.

[0034] Step 4, when the container application exits, the first shared memory is actively released through the ION interface; after the container application management service detects the release of the ION memory, the local reference to the memory is cleaned up, and the proxy application manager is notified to destroy the proxy application; when the proxy application is destroyed, the reference counter of the corresponding texture is decremented, and when the counter is 0, the texture resource is released. Embodiment

[0035] Taking the OpenHarmony system as an example, the seamless fusion display of the container application and the OpenHarmony native application is realized by using the cross-container application fusion switching method of the Hongmeng system provided by the application, the user experience and the system performance are improved, and the specific process includes: S1, Container internal application metadata registration and management.

[0036] S1.1, System service deployment, deploy the container application management service (ContainerAppManagerService, CAMS for short) in the system service layer System Service Layer of OpenHarmony, and the CAMS service has the following core capabilities: Establish a persistent Binder IPC connection with the container manager Container Manager, maintain the full life cycle state machine of the container application, manage the cross-process shared graphics resources, and coordinate the creation and destruction of the proxy application.

[0037] CAMS runs in an independent system process, that is, uid=system, has special permissions to access shared memory and graphics hardware, is registered as a system-level service through the service registration mechanism SystemAbilityManager of OpenHarmony, and ensures high priority and stability.

[0038] S1.2, Application startup event listening, the container manager Container Manager monitors the startup, switching, suspension, and exit of the container internal application in real time, and when a new application is detected to start, the following steps are performed: Collect application metadata, including but not limited to application name, package name, process ID, main window handle, icon resource, first startup time, application type, etc.; generate an application feature code, generate a unique identifier based on the combination of the package name and the container ID, which is used to distinguish the same named applications in different containers; establish a communication channel, and send a registration request to the CAMS through the Binder mechanism, carrying the application metadata and the shared texture descriptor.

[0039] S1.3, Metadata management mechanism, after the CAMS receives the registration request, the following processing steps are performed: Generate a UUID, use the OpenHarmony distributed device ID and timestamp combination algorithm to generate a globally unique application identifier, support cross-device application identification based on the global UUID generation strategy of the OpenHarmony distributed capability; Establish a mapping relationship, create a four-tuple mapping table of UUID, application metadata, texture handle, and state, and store it in a secure storage area. This four-tuple mapping relationship design realizes unified management of application metadata, graphics resources, and state; State listening, register the Death Recipient of the container process, and when the container application exits, automatically trigger the cleaning process to realize full-link state management combined with Binder death notification and distributed synchronization; S1.4, Abnormal processing mechanism, CAMS implements perfect abnormal processing strategy.

[0040] Timeout retransmission, when the registration request is not responded after timeout, it is automatically retried for 3 times with intervals of 100ms, 200ms and 400ms respectively; data check, integrity check is performed on the received metadata, and the registration request with incomplete or incorrect format is rejected; resource recycling, when the container process abnormally exits, the UUID mapping table cleaning and proxy application destruction process are triggered.

[0041] S2, Proxy application dynamic generation based on OpenHarmony Ability framework.

[0042] S2.1, Design the proxy application template conforming to the OpenHarmony application model, which contains the following core components, and the dynamic generation mechanism based on the template realizes the instant system integration of the container application: Application configuration file app.json5, defines the basic information of the application, permission declaration, supported device type, etc., the permission declaration such as graphic access permission, system interaction permission; capability component, contains ServiceAbility responsible for background communication and UIAbility responsible for interface rendering; resource file, contains default icon, name placeholder and other basic resources; life cycle management script, defines the processing logic of application creation, destruction, switching and other states.

[0043] S2.2, Proxy application manager implementation, implement the proxy application manager PAM (ProxyAppManager, PAM) in the application framework layer, which performs the following work: Template parsing, read the proxy application template, identify replaceable variables and resources; application generation, dynamically replace the variables in the template according to the container application metadata, generate a complete application package structure; application signature, sign the dynamically generated application using system-level signature certificate, ensure necessary system permissions; manage application installation, implement silent installation of proxy application through OpenHarmony application installation service AppInstallerService; life cycle binding, establish the life cycle binding relationship between the proxy application and the container application, realize the linkage start and exit.

[0044] S2.3, Dynamic generation process, when PAM receives the creation request of CAMS, the following steps are performed: Metadata parsing: Extracting key information such as the container application's name, icon, and window size; resource replacement: Replacing icon placeholders in the template with the actual icons of the container application, and name placeholders with the actual names; configuration generation: Generating adapted application configurations based on the container application type, such as window mode and display density; code injection: Dynamically generating UUID-related constant code to ensure the proxy application can correctly associate with the corresponding container application; package building: Packaging the replaced resources and code into HAP (Harmony Ability Package) format; silent installation: Calling the AppInstallerService interface for silent installation, skipping the user confirmation process; startup triggering: After installation, starting the proxy application's UIAbility and ServiceAbility through the application launcher Launcher interface.

[0045] S2.4 Internal Architecture of the Proxy Application: The dynamically generated proxy application adopts a layered architecture design. This layered architecture design achieves decoupling and independent optimization of communication, data, rendering, and interaction. Communication Layer: ServiceAbility implements long-term connection communication with CAMS, using OpenHarmony's IPC mechanism, including Binder and message queue; Data Layer: Maintains locally cached application state and texture information, achieving local data persistence; Rendering Layer: UIAbility integrates the Vulkan rendering engine, responsible for rendering shared textures to the application window; Interaction Layer: Implements basic user interaction logic, such as switching to the corresponding container application when clicking on the proxy application thumbnail.

[0046] S2.5 Lifecycle Management: Mechanism for synchronizing the lifecycle of proxy applications with that of container applications. Startup Synchronization: The proxy application is installed and started within 500ms after the container application starts; State Synchronization: When the container application enters the foreground or background, the proxy application updates its state synchronously; Exit Synchronization: When the container application exits, the proxy application actively calls the terminateSelf() method to exit after receiving a notification from CAMS; Exception Recovery: When the proxy application crashes unexpectedly, CAMS detects the exception and triggers the reconstruction process.

[0047] S3, a cross-container texture synchronization mechanism based on shared video memory.

[0048] S3.1 Shared memory management: Utilizing OpenHarmony's ION (Input / Output Memory) memory management mechanism to achieve cross-container memory sharing. Based on this ION cross-container memory sharing mechanism, zero-copy texture data transfer is achieved. Memory allocation, the container side allocates two contiguous graphics memory regions through ION at application startup, namely double buffering, and specifies the memory type as GRAPHICS_MEMORY to support GPU direct access; permission configuration, through the permission management interface of ION, set the read permission of the memory region for the CAMS process on the OpenHarmony side, and reserve the read and write permissions of the container side; handle transfer, the container side transfers the ION memory handle to CAMS through Binder IPC, which contains information such as memory physical address, size, and format; memory recycling, when the container application exits, the container side releases the shared memory through the ION interface, and CAMS cleans up the local reference after detecting the memory release S3.2, texture creation and sharing, the container side and the OpenHarmony side use a unified texture creation strategy: The container side uses OpenGL ES to create a 2D texture, binds it to an ION memory region, sets the texture format to RGBA8888 to ensure compatibility, and enables mipmap generation to support scaling display; texture update, the container side's Compositor synthesizer renders the application interface to the texture at a default rate of 30fps, and after completion, calls related APIs such as OpenGL ES's glFlush(), Vulkan's vkQueueSubmit to ensure data writing to the video memory; handle conversion, CAMS converts the received ION handle into a local texture ID on the OpenHarmony side, and creates a cross-context sharable texture object through the eglCreateImageKHR interface of EGL; reference management, CAMS creates a reference counter for each texture, decrements the counter when the proxy application is destroyed, and releases the texture resource when the counter is zero, achieving efficient use of system resources.

[0049] S3.3, double buffering mechanism: to avoid conflicts between texture rendering and reading, a double buffering strategy is adopted, and double buffering combined with atomic operations eliminates conflicts between rendering and display: Buffer initialization: create two texture buffers of the same size (Buffer A and Buffer B); Rendering buffer: the container side always renders to the background buffer (initially Buffer B); Display buffer: the OpenHarmony side always reads data from the foreground buffer (initially Buffer A); Buffer switching: after the container side completes a frame of rendering, it exchanges the pointers of the foreground and background buffers through atomic operations, and notifies the OpenHarmony side to use the new foreground buffer; Synchronization guarantee: use the fences synchronization primitive of GPU to ensure that the buffer switching operation is executed after rendering is completed.

[0050] S3.4, semaphore synchronization mechanism: implement cross-process synchronization based on semaphores: Semaphore creation: CAMS creates two anonymous semaphores (semA and semB) during texture initialization, corresponding to two buffers respectively; Waiting mechanism: OpenHarmony side calls sem_wait() to wait for the semaphore of the corresponding buffer before reading the texture; Notification mechanism: after the container side completes buffer rendering and switching, it calls sem_post() to activate the semaphore of the corresponding buffer; Timeout processing: set a 100ms timeout to avoid permanent waiting due to container side exceptions.

[0051] S3.5, texture format adaptation, handle the differences in texture formats of different application types, adaptive texture format conversion mechanism, compatible with the rendering output of different types of container applications: Format detection: CAMS detects the texture format of the container application during the registration phase, such as RGB565, RGBA8888, BGRA8888, etc. Format conversion: if the format is incompatible, real-time format conversion is performed through GPU shader Shader to avoid CPU participation; Size adaptation: automatically adjust the texture sampling method according to the proxy application window size, such as stretching, cropping, and maintaining the proportion; Color correction: use the color management configuration of OpenHarmony system to ensure that the color performance of the container application is consistent with that of the native application.

[0052] / / Texture receiver and synchronization mechanism core implementation class TextureReceiver : public ITextureReceiver { public: TextureReceiver(const std::string& uuid) : uuid_(uuid), currentBufferIndex_(0), bufferCount_(2) { / / Initialize semaphores sem_init(&semaphores_[0], 1, 0); / / Set pshared=1 for cross-process sharing sem_init(&semaphores_[1], 1, 0); / / Initialize fences for (int i = 0; i < bufferCount_; i++) { fences_[i] = EGL_NO_SYNC_KHR; } } bool Initialize(int format, int width, int height) { / / Store texture properties format_ = format; width_ = width; height_ = height; / / Create an EGL context eglDisplay_ = eglGetDisplay(EGL_DEFAULT_DISPLAY); eglInitialize(eglDisplay_, nullptr, nullptr); / / Configure EGL attributes const EGLint configAttrs[] = { EGL_SURFACE_TYPE, EGL_PBUFFER_BIT, EGL_RED_SIZE, 8, EGL_GREEN_SIZE, 8, EGL_BLUE_SIZE, 8, EGL_ALPHA_SIZE, 8, EGL_NONE }; EGLConfig config; EGLint numConfigs; eglChooseConfig(eglDisplay_, configAttrs, &config, 1, &numConfigs); / / Create an EGL context const EGLint contextAttrs[] = { EGL_CONTEXT_CLIENT_VERSION, 3, EGL_NONE }; eglContext_ = eglCreateContext(eglDisplay_, config, EGL_NO_CONTEXT, contextAttrs); return eglContext_!= EGL_NO_CONTEXT; } void OnTextureUpdated(int bufferIndex, EGLSyncKHR fence)override { std::lock_guard <std::mutex>lock(mutex_); / / Wait for the fence signal of the previous frame if (fences_[bufferIndex]!= EGL_NO_SYNC_KHR) { eglClientWaitSyncKHR(eglDisplay_, fences_[bufferIndex], EGL_SYNC_FLUSH_COMMANDS_BIT_KHR, EGL_FOREVER_KHR); eglDestroySyncKHR(eglDisplay_, fences_[bufferIndex]); } / / Save the new fence fences_[bufferIndex] = fence; / / Switch the current buffer index currentBufferIndex_ = bufferIndex; / / Notify the waiting read thread sem_post(&semaphores_[bufferIndex]); } int AcquireTexture(GLuint& textureId, int& width, int& height) { / / Wait for the semaphore int bufferIndex = currentBufferIndex_; int ret = sem_wait(&semaphores_[bufferIndex]); if (ret!= 0) { return -errno; } std::lock_guard <std::mutex>lock(mutex_); / / Wait for the GPU to finish rendering if (fences_[bufferIndex]!= EGL_NO_SYNC_KHR) { eglClientWaitSyncKHR(eglDisplay_, fences_[bufferIndex], EGL_SYNC_FLUSH_COMMANDS_BIT_KHR, EGL_FOREVER_KHR); } / / Return the texture information textureId = textures_[bufferIndex]; width = width_; height = height_; return 0; } ~TextureReceiver() { / / Clean up resources for (int i = 0; i < bufferCount_; i++) { sem_destroy(&semaphores_[i]); if (fences_[i]!= EGL_NO_SYNC_KHR) { eglDestroySyncKHR(eglDisplay_, fences_[i]); } } eglDestroyContext(eglDisplay_, eglContext_); eglTerminate(eglDisplay_); } private: std::string uuid_; int format_; int width_; int height_; GLuint textures_[2]; int currentBufferIndex_; int bufferCount_; sem_t semaphores_[2]; EGLSyncKHR fences_[2]; EGLDisplay eglDisplay_; EGLContext eglContext_; std::mutex mutex_; }; S4. High-efficiency thumbnail rendering implementation based on Vulkan.

[0053] S4.1 Vulkan rendering environment initialization: The proxy application's UIAbility builds a complete Vulkan rendering environment during the initialization phase. Instance Creation: Call `vkCreateInstance` to create a Vulkan instance, enabling necessary extensions specific to OpenHarmony, such as `VK_KHR_surface`, `VK_HARMONY_surface`, and `VK_KHR_swapchain`; Physical Device Selection: Enumerate the GPU devices in the system, prioritizing integrated graphics cards that support texture sharing and hardware acceleration; Logical Device Creation: Create logical devices based on the selected physical devices, specifying the required queue families, including graphics queues and rendering queues; Command Pool Initialization: Create an independent command pool for each rendering thread to avoid inter-thread synchronization overhead; Debug Layer Integration: Enable the Vulkan debug layer in development mode to capture rendering errors and performance bottlenecks.

[0054] S4.2 Integration with OpenHarmony Window System: Achieve deep integration between Vulkan and the OpenHarmony window system, deeply integrating the Vulkan rendering solution of the OpenHarmony window system to achieve efficient hardware-accelerated rendering. Surface creation: Obtain the window surface through OpenHarmony's window manager WindowManager, and call vkCreateHarmonySurfaceKHR to convert it into a Vulkan-compatible surface VkSurfaceKHR; Switch chain configuration: Query the supported formats, color spaces and rendering modes of the surface, select the optimal configuration, and prioritize VK_PRESENT_MODE_FIFO_KHR to ensure low latency. Dynamic switch chain reconstruction mechanism adapts to changes in window size and display mode switching. Swapchain creation: Call vkCreateSwapchainKHR to create a swapchain, set the minimum image count to 3 (triple buffering) to balance smoothness and memory usage, triple buffering with multiple semaphore synchronization strategy to balance rendering smoothness and resource usage. Render target generation: Create a frame buffer Framebuffer and a render pass RenderPass for each image in the swapchain, configure the color attachment and clear value.

[0055] S4.3, Texture resource binding, efficiently bind shared texture to Vulkan rendering pipeline: External texture import: Import the texture in shared memory as an external texture VkImage in Vulkan through the vkImportAndroidHardwareBufferEXT compatible Android container or vkImportMemoryFdKHR interface; Image view creation: Create an image view VkImageView for the imported texture, specify the correct format and dimension; Sampler configuration: Create a texture sampler VkSampler, set the filtering method, addressing mode (Clamp to edge), and anisotropic filtering; Descriptor set management: Create a descriptor pool and descriptor set layout, bind the texture view and sampler to the descriptor set to associate with the fragment shader.

[0056] S4.4, Render pipeline construction: Optimize the rendering pipeline to achieve efficient thumbnail rendering, lightweight rendering pipeline optimized for thumbnail rendering, reduce rendering overhead: Shader compilation: Use SPIR-V compiler to compile vertex shader and fragment shader to bytecode, vertex shader is responsible for processing texture coordinate transformation, fragment shader is responsible for texture sampling and color adjustment Pipeline layout: Define pipeline layout VkPipelineLayout, associate descriptor set layout and push constant, used to pass texture size and other dynamic parameters; Render pass configuration: Configure the render pass for single draw, no depth test, enable color blending, realize fusion with window background; Pipeline cache: Use pipeline cache VkPipelineCache to speed up pipeline creation, reduce application startup time.

[0057] S4.5, On-demand rendering mechanism: Implement intelligent rendering strategy based on content change, intelligent on-demand rendering mechanism based on content change and time control, significantly reduce system resource consumption: Change detection: compare texture metadata of consecutive frames, such as modification timestamps, content hashes, to determine if a redraw is needed; Render trigger: only perform rendering when texture content changes are detected, window size changes, or system forced redraw notifications are received; Frequency limit: set a maximum rendering frequency (default 60fps) to avoid excessive rendering resource consumption; Background pause: pause rendering process when the agent application enters the background to release GPU resources.

[0058] S4.6, Render synchronization and presentation: ensure that the rendering result is displayed to the screen in a timely and accurate manner: Semaphore synchronization: use two semaphores, imageAvailableSemaphore and renderFinishedSemaphore, to synchronize the exchange of image available and rendering complete events; Fence mechanism: set a fence for each submitted rendering command to ensure that a new frame of rendering starts only after the previous frame of rendering is completed; Presentation queue: submit the rendered image to the presentation queue for processing by the compositor of OpenHarmony for final display; Frame time control: monitor the rendering time of each frame, and dynamically adjust the rendering resolution when it exceeds 16ms (60fps standard).

[0059] In summary, the above is only a preferred embodiment of the present application, and is not intended to limit the scope of protection of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included within the scope of protection of the present application.< / std::mutex> < / std::mutex>

Claims

1. A method for cross-container application fusion switching of a HongMeng system, characterized in that, Specifically comprising the following steps: Step 1, the system service layer of the Hongmeng system deploys a container application management service, and the application framework layer implements a proxy application manager, and the container application management service establishes a persistent connection with a container manager; Step 2, when starting the container application, a graphics memory area is allocated on the container side as a first shared memory, the proxy application manager has read permission thereof, the container side has read and write permissions, a first memory handle of the first shared memory is generated; The container manager collects metadata of the container application, and sends a first registration request including the metadata and the first memory handle to the container application management service; After receiving the first registration request, the container application management service converts the first memory handle into a texture handle, allocates a unique identifier for the container application, and sends a proxy application creation request including the unique identifier, the metadata and the texture handle to the proxy application manager; after receiving the proxy application creation request, the proxy application manager instantiates a proxy application template to complete installation and start of a first proxy application; the first proxy application initializes to build a Vulkan rendering environment; Step 3, the container side creates a first texture for the container application and binds it to the first shared memory, and renders an interface to the first texture; the container application management service creates a reference counter for the texture; the first proxy application imports the texture handle as an external texture, scales and samples to obtain a thumbnail, and renders the thumbnail; Step 4, when the container application exits, the first shared memory is released, the container application management service detects the memory release, cleans up the reference of the memory, notifies the proxy application manager to destroy the proxy application, and when the proxy application is destroyed, the reference counter of the corresponding texture is decremented, and when the counter is 0, the texture resource is released.

2. The cross-container application fusion handover method of claim 1, wherein, The way of instantiating the proxy application template to complete installation and start of the first proxy application is: replacing the name placeholder and the icon placeholder in the proxy application template with the name and the icon of the container application to obtain the first proxy application, generating an application configuration of the first proxy application according to the application type, and dynamically generating a UUID constant code to inject the first proxy application.

3. The cross-container application fusion handover method of claim 1, wherein, The container side allocates two continuous graphics memory areas as the first shared memory, including a first texture buffer and a second texture buffer with the same size; the container side renders to the background buffer, and the container application management service reads texture data from the foreground buffer; after the container side completes a frame of rendering, the foreground and background buffer pointers are exchanged through atomic operation, and the switching operation is ensured to be executed only after the rendering is completed through a synchronization primitive; the container application management service creates a semaphore for the first texture buffer and the second texture buffer respectively, the container side activates the semaphore of the corresponding buffer after completing buffer switching, and the container application management service reads the data of the new foreground buffer after receiving the signal.

4. The cross-container application fusion handover method of claim 3, wherein, The timeout time of waiting for the semaphore for the container application management service is set.

5. The cross-container application fusion handover method of claim 1, wherein, The way of scaling and sampling to obtain the thumbnail is: the first proxy application creates a descriptor set including an adaptive image view and a texture sampler, adopts native thumbnail rendering parameters configuration, sets a vertex shader and a fragment shader, and constructs a lightweight rendering pipeline optimized for the thumbnail; After the container side completes the update of the first texture, the lightweight rendering pipeline and the descriptor set are bound to obtain a thumbnail by external texture scaling sampling and complete the rendering of the thumbnail.

6. The cross-container application fusion handover method of claim 5, wherein, The lightweight rendering pipeline optimized for the thumbnail is constructed in the following manner: a vertex shader and a fragment shader are compiled into bytecode, the vertex shader is responsible for processing texture coordinate transformation, and the fragment shader is responsible for texture sampling and color adjustment; a pipeline layout is defined, a descriptor set layout is associated and a constant is pushed; a rendering channel is configured as single draw, no depth test and color blending enabled to realize fusion with the window background; and pipeline caching is used to accelerate pipeline creation.

7. The cross-container application fusion handover method of claim 1, wherein, Before the first proxy application imports the texture handle as an external texture, metadata in the texture handle is parsed to extract the original format of the container texture, if the original format is compatible with the target format of the rendering pipeline of the proxy application, the texture handle is directly imported, otherwise real-time format conversion is performed through a GPU shader, and the texture sampling mode is automatically adjusted according to the window size of the proxy application.

8. The cross-container application fusion handover method of claim 1, wherein, The proxy application template includes an application configuration file, a capability component and a resource file, the application configuration file is used to define basic information, permission declaration and supported device types of the proxy application, the capability component includes a first capability responsible for background communication and a second capability responsible for interface rendering, and the resource file includes an icon placeholder and a name placeholder.

9. The cross-container application fusion handover method of claim 8, wherein, The proxy application includes: a communication layer that realizes long connection communication between the first capability and the container application management service by using an IPC mechanism of a Hongmeng system; a data layer that maintains locally cached application states and textures and realizes local persistence of data; a rendering layer that internally integrates a Vulkan rendering engine in the second capability and is responsible for rendering shared textures to an application window; and an interaction layer that realizes user interaction logic.

10. The cross-container application fusion handover method of claim 1, wherein, The life cycle of the proxy application is synchronized with that of the container application, specifically: after the container application is started, installation and startup of the proxy application are completed within a set time, the application state is updated synchronously when the container application enters the foreground or the background, the proxy application actively exits after receiving a notification from the container application management service when the container application exits, and the container application management service detects an exception and triggers a reconstruction process to recover the exception when the proxy application crashes unexpectedly.

Images