A multi-window display optimization method for a harmony platform Android application

Abstract

The application discloses a multi-window display optimization method for an Android application of a Hongmeng platform, and the method comprises the following steps: after an Android container is started, a first shared memory containing multiple partitions is created on the Hongmeng side; when an Android application component is started, the component ID is allocated and bound, the GLES resource is configured, the double-buffer queue and the GLES context are created, the shared image resource handle is generated, and the mapping table of the component ID, the window ID and the texture identification is established; when the component is rendered, the context and the component ID are checked; after the pixel data is rendered, the differential area is calculated and the first data and the checksum are generated and processed; the data header and the first data are written into the shared memory partition, and the Hongmeng side is informed; after the Hongmeng side is verified, the information is extracted to generate the texture identification and the associated Surface buffer; finally, the Android and Hongmeng window data are synthesized and submitted to the display hardware to complete the on-screen display.

Description

Technical Field

[0001] This invention belongs to the field of computer software development technology, specifically relating to a method for optimizing multi-window display in Android applications on the HarmonyOS platform. Background Technology

[0002] With the expansion of the OpenHarmony ecosystem, compatibility with Android applications through container technology has become a key requirement. Existing Android container solutions generally adopt a globally shared rendering context and a single texture aggregation model, where the rendering results of all Android applications are merged into the same frame buffer and finally displayed through a single OpenHarmony window.

[0003] However, this approach has the following problems: First, rendering resources are coupled, with multiple applications sharing the OpenGL ES (GLES) context and frame buffer. Rendering errors in one application (such as texture conflicts or state pollution) can spread to other applications, and it's impossible to configure rendering parameters differently for different components (such as full-screen applications and pop-ups). Second, window mapping is rigid, relying on static configuration files to associate applications and windows. Adding or updating applications requires manual configuration changes, and it cannot dynamically respond to lifecycle changes of temporary components (such as dialog boxes and floating windows). Third, transmission and display are inefficient, using full-frame pixel copy to transmit rendering results. Even for partial updates (such as button state changes), complete frame data still needs to be transmitted, and cross-system shared memory is not clearly defined. The creation of negotiation mechanisms and the principle of cross-process texture sharing often lead to problems in practice, such as the inability to map shared memory across sides or invalid texture handles; fourth, the lack of behavioral coordination, the window behavior of Android applications (such as resizing and focus switching) is disconnected from the native window management logic of OpenHarmony, and the problem of vertical synchronization signal coordination between rendering and display is not resolved, resulting in display stuttering or tearing; fifth, the screen display link is unclear, the specific screen display process of Android rendering results in the OpenHarmony window is not clearly defined, including the binding of textures and window Surface, and screen composition logic, resulting in the rendering results not being displayed accurately. Summary of the Invention

[0004] In view of this, the present invention provides a multi-window display optimization method for Android applications on the HarmonyOS platform. Based on rendering end-to-end isolation, cross-system resource collaboration and display process closed loop, it enables multiple Android application components to be displayed in corresponding independent windows on the HarmonyOS platform.

[0005] This invention provides a method for optimizing multi-window display of Android applications on the HarmonyOS platform, which specifically includes the following steps:

[0006] After the Android container starts, the HarmonyOS side creates a first shared memory containing multiple partitions;

[0007] When an Android application component starts, it assigns a first component ID, binds the first component ID to the current rendering thread, configures GLES resources, creates a double-buffered queue including a first buffer and a second buffer, and creates a first GLES context; it creates a base texture and converts it into a shared image resource handle, associates the first component ID with the shared image resource handle; it creates a frame buffer with the base texture as a color attachment and binds it to the component, binds the rendering buffer to the component, sends a component startup event to trigger the creation of a window on the HarmonyOS side, sets its display hierarchy in multiple windows, and establishes a first mapping table between the first component ID, window ID, and texture identifier.

[0008] When a component performs rendering, it obtains the current GLES context and the second component ID bound to the current rendering thread. If the current GLES context is different from the first GLES context, it switches to the first GLES context to perform rendering. If the second component ID is different from the first component ID, it returns an error code. The rendered pixel data is written to the second buffer. The difference region between the pixel data in the first buffer and the second buffer is calculated. If the ratio of the area of ​​the difference region to the area of ​​the whole frame is less than a threshold, then all the current pixel data is the first data. Otherwise, the difference region is compressed into the first data, and the checksum of the first data is calculated.

[0009] The data header consists of a checksum of the component ID, a ready flag, and a shared image resource handle. The data header and the first data are written to the partition, and the ready flag is set to TRUE to notify the HarmonyOS side. The HarmonyOS side verifies the checksum of the ready flag TRUE. If it passes, it extracts the shared image resource handle, generates a texture identifier, and adds it to the first mapping table. It obtains the window ID from the first mapping table, requests a Surface buffer, associates the texture identifier with the Surface buffer, and releases the first data into the Surface buffer. When the callback is triggered, the HarmonyOS side draws the texture pixel data onto the Surface buffer, synthesizes the window Surface data of Android and HarmonyOS, and submits it to the display hardware to complete the on-screen display.

[0010] Furthermore, a three-tiered resource pool is constructed, including: an active pool, which stores the first GLES context, shared image resource handles, and frame buffers of currently visible components, retaining a set number of active resources using a least recently used strategy; a cache pool, which stores resources of background components, retains texture data, releases the first GLES context binding of background components, and starts an independent thread to periodically traverse the cache pool, moving resources to the recycling pool when they are idle for more than a set time; and a recycling pool, which stores resources of destroyed components, clears texture data, retains shared image resource handles, prioritizes the reuse of the same recycled resources when a new component is created, reinitializes the context before reuse, and releases resources that have not been reused for more than a set time.

[0011] Furthermore, the method for creating windows on the HarmonyOS side is as follows: active components correspond to APP_MAIN_WINDOW to support split-screen and full-screen switching, dialog components correspond to APP_DIALOG_WINDOW, and floating window components correspond to APP_FLOAT_WINDOW; the window size is determined according to the clarity of the component, and the default position of the window is set according to the component type; the transparency parameter of the component is mapped to the transparency of the HarmonyOS window, and the Z-axis layer of the component is converted into the HarmonyOS global layer.

[0012] Furthermore, the bandwidth allocation method for the first shared memory is as follows:

[0013] The scoring formula is defined as: score = (Z-axis level * 0.5) + (recent interaction time * 0.3) + (visibility * 0.2). The transmission queues are sorted according to the scores. The transmission queues with scores greater than the threshold occupy the bandwidth of the first shared memory, while the transmission queues with scores less than the threshold are delayed until the bandwidth is available.

[0014] Furthermore, HarmonyOS monitors window visibility in real time. If the window is completely obscured, it sends an event to the Android container to pause the rendering and transmission of the corresponding component. If the window is partially visible, only the pixel data of the visible area is transmitted in the next incremental transmission.

[0015] Furthermore, user operations on the HarmonyOS side window are intercepted, converted into instructions recognizable by the Android container side, and sent to the Android container side via cross-process communication. When the window size is adjusted, the Android container side updates the component layout parameters and triggers re-rendering; when the window is moved, the component position parameters are updated to modify the position of the pop-up window within the container; when the window is closed, the component process is terminated. When the window gains focus, the frame rate is increased; when it loses focus, the frame rate is decreased; and when it is completely invisible, rendering is paused. The animation parameters of the HarmonyOS side window system are sent to the Android container side, and the Android container side sets the rendering parameters to synchronize the component animation with the window animation.

[0016] Furthermore, when an Android container-side component is destroyed, it sends a destruction event to the HarmonyOS side. Upon receiving the event, the HarmonyOS side queries and destroys the corresponding window based on the component ID, clears the first mapping table, releases textures and Surface resources, and sends a confirmation message. After receiving the confirmation message, the Android container-side reclaims the shared image resource handle, textures, and the first GLES context. If the first shared memory is not used by other components, it releases its corresponding memory address. When the HarmonyOS-side window is closed, the HarmonyOS side sends a close event to request the Android container-side to terminate the corresponding component process. After terminating the process and releasing all component resources, the Android container-side sends a confirmation message. After receiving the confirmation message, the HarmonyOS side clears the window, the first mapping table, and local texture resources; otherwise, it forcibly clears local resources.

[0017] Furthermore, the method for configuring GLES resources is as follows:

[0018] For components of type "Active", configure GLES version 3.2, a texture memory limit of 256MB, a 24-bit depth buffer, and support for 4x anti-aliasing; for components of type "Dialog", configure GLES version 2.0, a texture memory limit of 32MB, a 16-bit depth buffer, and retain only the basic color format; for components of type "Floating Window", reuse the GLES version and color format of the same application component, with a texture memory limit of 16MB and no depth buffer.

[0019] Furthermore, when a component enters the foreground, it activates the corresponding resource from the cache pool and switches to the first GLES context corresponding to the component; when a component enters the background, it pauses the execution of the rendering thread bound to the component, moves the corresponding resource to the cache pool, and retains the texture data corresponding to the current frame; when the component stops, it releases the binding relationship between the frame buffer and the corresponding texture.

[0020] Furthermore, the method for compressing the differential region into the first data is as follows: if the differential region is a UI control, then ETC2 is used to compress the differential region; if it is dynamic content, then ASTC 6x6 is used to compress the differential region.

[0021] Beneficial effects:

[0022] This invention, after the Android container starts, creates a first shared memory and passes a file descriptor on the HarmonyOS side for cross-system data transfer. When a component starts, it generates a unique component ID and binds it to the rendering thread, configures GLES resources, creates a double-buffered queue and an isolated context, generates a shared image handle, completes frame buffer binding, and then sends a component startup event. Upon receiving the component startup event, the HarmonyOS side establishes a mapping table of component ID, window ID, and texture identifier, creates a window, and sets the display level of the new window in multiple windows. During component rendering, it verifies and switches the GLES context, and manages resources and threads according to the window state. After frame rendering is completed, pixel data is written to the buffer, a differential mask is generated by comparison, and the differential region is extracted. The transmission method is selected according to the proportion, and a data header containing verification information is assembled and written to the first shared memory. After the HarmonyOS side polls for a valid marker, it imports the handle to generate a local texture identifier and completes the mapping table, obtains the window Surface and associates it with the buffer pointing to the shared memory data, and configures scaling and blending modes to complete the binding. The HarmonyOS side registers a VSync callback, draws data into the buffer when the signal is triggered, merges the layers, and submits them to the hardware to complete the on-screen display. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating a method for optimizing multi-window display of Android applications on the HarmonyOS platform, provided by the present invention. Detailed Implementation

[0024] The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0025] This invention provides a method for optimizing multi-window display in Android applications on the HarmonyOS platform. The core idea is as follows: After the Android container starts, the HarmonyOS side creates a first shared memory and passes file descriptors for cross-system data transfer. When a component starts, a unique component ID is generated and bound to a rendering thread. GLES resources are configured, a double-buffered queue and isolation context are created, a shared image handle is generated, frame buffer binding is completed, and then a component startup event is sent. Upon receiving the component startup event, the HarmonyOS side establishes a mapping table of component ID, window ID, and texture identifier, creates a window, and sets the display hierarchy of the new window within the multi-window framework. Component rendering... At that time, the GLES context is verified and switched, and resources and threads are managed according to the window state. After the frame rendering is completed, the pixel data is written to the buffer, the differential mask is generated by comparison and the differential region is extracted, the transmission method is selected according to the proportion, and the data header containing the verification information is assembled and written to the first shared memory. After the HarmonyOS side polls for a valid marker, the handle is imported to generate a local texture identifier and the mapping table is completed. The window Surface is obtained and the buffer points to the shared memory data. The scaling and blending modes are configured to complete the binding. The HarmonyOS side registers the VSync callback, draws the data into the buffer when the signal is triggered, merges the layers and submits them to the hardware to complete the screen display.

[0026] This invention provides a method for optimizing multi-window display in Android applications on the HarmonyOS platform. The processing flow is as follows: Figure 1 As shown, the specific steps include:

[0027] Step 1: After the Android container starts, the HarmonyOS side creates the first shared memory and passes the file descriptor of the first shared memory to the Android container side. The first shared memory is divided into multiple equal partitions. The partitions are used to store the data generated by the rendering of Android application components, realizing the data transfer between the Android container side and the HarmonyOS side.

[0028] When a launch event for a displayable component in an Android application is detected, the following steps are taken: Component information is obtained, including the application ID and component type. An instance ID is assigned to the component, and the unique identifier of the component, composed of the application ID, component type, and instance ID, is denoted as the component ID. The component ID is then bound to the current rendering thread. GLES resources are configured for the component based on its type, and corresponding shared memory blocks are reserved. A double-buffered queue is created for the component, including a first buffer storing pixel data from the previous frame's texture and a second buffer storing pixel data from the current frame's texture. The default display device is initialized, and a GLES context is created based on the configured GLES resources. Context state isolation is set to ensure that the rendering states of components do not contaminate each other. A base texture is created and its texture filtering mode and size are configured. The base texture is converted into a shared image resource handle, and the component ID is associated with the shared image resource handle. A frame buffer is created for the component, and the base texture is bound to it as a color attachment. Finally, a rendering buffer is bound to the component based on its type.

[0029] Send a component launch event containing basic attributes such as component ID, component type, initial size, clarity, transparency, and Android Z-axis layer. After receiving the component launch event, the HarmonyOS side establishes a first mapping table between component ID, window ID, and texture identifier. It creates a window by configuring window attributes according to the basic attributes, parses the Android Z-axis layer to obtain the Android Z-axis layer of all current Android windows, sets the display layer of the newly created window in the multiple windows according to the Android Z-axis layer, and adds the component ID and window ID to the first mapping table.

[0030] The method for configuring GLES resources for components based on their component type is as follows: For components of type "Activity", configure GLES version 3.2, a texture memory limit of 256MB, a 24-bit depth buffer, support for 4x anti-aliasing, and adapt to high-quality full-screen display; for components of type "Dialog", configure GLES version 2.0, a texture memory limit of 32MB, a 16-bit depth buffer, retain only the basic color format, and reduce resource consumption; for components of type "Floating Window", reuse the GLES version and color format of the main component of the same application, with a texture memory limit of 16MB, no depth buffer, and adapt to lightweight floating display.

[0031] Furthermore, to achieve efficient resource reuse and recycling, this invention constructs a three-level resource pool, specifically including: an active pool, used to store resources such as the GLES context, shared image resource handles, and frame buffers of currently visible components, retaining a predetermined number of active resources using a least recently used strategy to prioritize the rendering response speed of foreground components; a cache pool, used to store resources of background components, retaining texture data, releasing the GLES context binding of background components, and starting an independent thread to periodically traverse the cache pool, moving resources to the recycling pool if they are idle for more than a predetermined time; and a recycling pool, used to store resources of destroyed components, clearing texture data, retaining shared image resource handles, prioritizing the reuse of recycled resources of the same type and specification when a new component is created, reinitializing the context before reuse, and completely releasing resources that have not been reused for more than a predetermined time.

[0032] Furthermore, windows are created based on basic attribute configuration. Specifically, for window type matching, active components correspond to APP_MAIN_WINDOW to support split-screen and full-screen switching, dialog components correspond to APP_DIALOG_WINDOW, and floating window components correspond to APP_FLOAT_WINDOW. Size and position calculations are performed by converting the window size based on the component's clarity and setting the default position based on the component type. Transparency and layer configurations are also implemented by mapping the transparency parameters of Android components to the transparency of HarmonyOS windows, extracting the Z-axis layer of Android components and converting it to the HarmonyOS global layer to ensure layer consistency between multiple windows on the HarmonyOS side and within the Android container.

[0033] Furthermore, using the sum of the starting value of the corresponding Android interval on the HarmonyOS side and the scaled Android Z-axis layer as the base layer, and the sum of the base layer and the top offset as the HarmonyOS window layer for the new window, the modified Android Z-axis layer is written into the HarmonyOS global layer management table to complete the setting of the new window's display layer, ensuring that the new window is displayed at the forefront of all current windows. Specifically, scaling the Android Z-axis layer involves multiplying the Android Z-axis layer by a scaling factor.

[0034] Step 2: When a component executes a rendering instruction, it obtains the current GLES context and the component ID bound to the current rendering thread. It then queries the GLES context corresponding to the component ID as the target GLES context. If the current GLES context and the target GLES context are inconsistent, it switches to the target GLES context before executing the rendering instruction. If the obtained bound component ID is different from the component ID corresponding to the texture processed by the rendering instruction, it returns an error code and logs it. When a component enters the foreground, it activates the corresponding resource from the cache pool and switches to the GLES context corresponding to the component. When a component enters the background, it pauses the execution of the rendering thread bound to the component, moves the corresponding resource to the cache pool, and retains the texture data corresponding to the current frame. When the component stops, it releases the binding relationship between the frame buffer and the corresponding texture.

[0035] Step 3: When the component finishes rendering the current frame, write the pixel data of the current frame texture into the second buffer, compare the pixel data of the two frame textures in the first buffer and the second buffer, generate a differential mask texture, parse the mask texture to obtain the minimum bounding rectangle, which is the differential region. If the ratio of the area of ​​the differential region to the area of ​​the whole frame is less than the threshold, then all the pixel data of the current frame texture is used as the first data; otherwise, select a compression algorithm according to the content of the differential region to compress the differential region into the first data, and calculate the checksum of the first data.

[0036] Obtain the shared image resource handle of the component. The data header consists of the length of the component ID, the length of the transmitted data, the checksum, the ready flag, the shared image resource handle, and the reserved fields. Write the data header into the data header field of the partition in the first shared memory. Write the first data into the subsequent space of the partition. Then set the ready flag to TRUE and notify the HarmonyOS side to read the data.

[0037] Furthermore, the transmission bandwidth of the first shared memory is allocated according to the importance of the components:

[0038] The priority scoring formula is defined as: score = (Z-axis level * 0.5) + (recent interaction time * 0.3) + (visibility * 0.2), that is, the higher the Z-axis level, the closer the recent interaction time, and the higher the visibility, the higher the score. The transmission queue is sorted according to the score. High-priority components with scores greater than the threshold occupy the first shared memory bandwidth first, while low-priority components with scores not greater than the threshold are delayed until the bandwidth is available, thereby reducing system resource contention.

[0039] Step 4: After receiving the message to read data, the HarmonyOS side polls the ready flag of the partition in the first shared memory. When the ready flag is TRUE, the checksum is checked first. After the checksum passes, the shared image resource handle is extracted from the data header field. The shared image resource from the Android container side is imported into the local GLES context to generate a HarmonyOS texture identifier that can be used by the HarmonyOS side. The HarmonyOS texture identifier is added to the first mapping table according to the component ID. The corresponding window ID is obtained from the first mapping table. The Surface of the window is obtained and a Surface buffer is requested. The HarmonyOS texture identifier is associated with the Surface buffer, so that the Surface buffer points to the first data in the corresponding partition in the first shared memory. After setting the texture scaling mode and Alpha blending mode, the first data is released to the Surface buffer, and the texture pixel data is bound to the Surface buffer.

[0040] Step 5: Register the VSync callback on the HarmonyOS side to obtain the screen refresh rate. When the VSync callback is triggered, pass in the Surface buffer and the HarmonyOS texture identifier. Draw the texture pixel data onto the Surface buffer according to the set parameters. The HarmonyOS compositor merges the Surface data of the Android component window and the HarmonyOS native window according to the Z-axis level and submits it to the display hardware to complete the screen display operation.

[0041] Furthermore, HarmonyOS monitors window visibility in real time. If the window is completely obscured, it sends an event to the Android container to pause the rendering and transmission of the corresponding component. If the window is partially visible, it only transmits the pixel data of the visible area in the next incremental transmission, reducing bandwidth usage.

[0042] Furthermore, to ensure that the interactive experience of Android applications is consistent with that of native HarmonyOS applications, this invention achieves real-time linkage between the rendering behavior of HarmonyOS windows and Android components, specifically as follows:

[0043] Intercept user operations on the HarmonyOS side window, convert them into instructions recognizable by the Android container side, and send them via cross-process communication. When the window size is adjusted, the Android container side updates component layout parameters to trigger re-rendering. When the window is moved, the component position parameters are updated to adapt the pop-up position within the container. When the window is closed, the corresponding component process is terminated to avoid resource leaks. When the HarmonyOS side window gains focus, the frame rate is increased to optimize response; when it loses focus, the frame rate is reduced; and when it is completely invisible, rendering is paused. The animation parameters of the HarmonyOS side window system are sent to the Android container side, and the Android container side adjusts the rendering parameters through animation-driven tools to synchronize component animations with HarmonyOS window animations.

[0044] Furthermore, to avoid memory leaks and invalid resource residues, this invention implements cross-system resource reclamation and mapping cleanup during component and window destruction, specifically as follows:

[0045] When an Android container-side component is destroyed, it sends a destruction event to the HarmonyOS side. Upon receiving the event, the HarmonyOS side queries and destroys the corresponding window based on the component ID, clears the first mapping table, releases textures and Surface resources, and sends a confirmation message. After receiving the confirmation message, the Android container-side reclaims local rendering resources such as shared image resource handles, textures, and GLES context. If the first shared memory is not used by other components, it releases its corresponding memory address. When the HarmonyOS-side window is closed, the HarmonyOS side sends a close event to request the Android container-side to terminate the corresponding component process. After the Android container-side terminates the process and releases all component resources, it sends a confirmation message. After receiving the confirmation message, the HarmonyOS side clears the window, the first mapping table, and local texture resources. If no confirmation message is received, it forcibly clears local resources to prevent leakage.

[0046] Furthermore, to ensure that resource reclamation is not overlooked in abnormal scenarios, this invention establishes a protection mechanism for anomalies such as container crashes and shared memory failures. Specifically, the HarmonyOS side uses heartbeat detection to periodically check the validity of the first shared memory. If it fails, the transmission channel is rebuilt and restored. If texture transmission fails continuously, the component process is terminated and resources are reclaimed after retrying.

[0047] Example:

[0048] This embodiment employs a multi-window display optimization method for Android applications on the Harmony platform provided by this invention. Based on the complete lifecycle of Android application components, and following the rendering process sequence of rendering resource preparation, isolated rendering execution, incremental transmission, window binding display, behavior coordination, and source recycling, it enables each Android application component to be displayed in a corresponding independent window on the OpenHarmony side. The specific process includes:

[0049] S1. Application component-level rendering resource pre-allocation and dynamic initialization: Allocate independent rendering resources for each Android displayable component to ensure resource isolation and reusability.

[0050] S1.1 Component identification and UCID globally unique identifier generation.

[0051] By hooking or modifying the Android system's ActivityThread.handleLaunchActivity (Activity launch) and WindowManagerGlobal.addView (Dialog or floating window addition) methods, the component launch event can be intercepted and core information extracted: application UID (ApplicationInfo.uid), component type (enumerated as Activity, Dialog, floating window), and instance ID. The instance ID is generated by a global atomic auto-incrementing variable maintained within the container, with an initial value of 1, to avoid multi-threaded concurrency conflicts through atomic operations.

[0052] Based on the above information, a unique identifier (UCID) is generated for the component, in the format UID_\({uid}_TYPE_\){type}ID\({instance ID}; if a duplicate UCID is detected (such as ID reuse caused by an abnormal restart), _RETRY_\){number of retries} is appended to the instance ID to ensure global uniqueness and provide a unique index for subsequent resource association and mapping.

[0053] S1.2, Differentiated pre-allocation of rendering resources.

[0054] Establish rules for mapping component types to resource specifications, and configure GLES resources differently based on component display requirements:

[0055] Activity component: Configured with GLES 3.2, 256MB texture memory limit, 24-bit depth buffer, supports 4x anti-aliasing, and adapts to full-screen high-quality display;

[0056] Dialog component: Configured with GLES 2.0 version, 32MB texture memory limit, 16-bit depth buffer, retaining only the basic color format (RGB565) to reduce resource consumption;

[0057] Floating window component: reuses the GLES version and color format of the same application's main component, with a texture memory limit of 16MB, no depth buffer, and is adapted for lightweight floating display.

[0058] Based on the above rules, matching EGL configurations (EGLConfig) are filtered using eglChooseConfig, and a shared memory block of the corresponding specification is reserved for each UCID to avoid delays caused by subsequent dynamic allocation.

[0059] S1.3 Independent GLES Context Initialization and EGLImage Creation.

[0060] Before the component is first drawn (intercepting the ViewRootImpl.performTraversals or SurfaceView.onDraw trigger points), an independent GLES context is initialized for each UCID. The specific steps are as follows:

[0061] First, obtain the default display device through eglGetDisplay, call eglInitialize to complete the initialization, and then create an independent EGLContext based on the pre-allocated EGLConfig. Force the isolation of the context state by setting the EGL_CONTEXT_CLIENT_VERSION and EGL_CONFIG_CAVEAT attributes to ensure that the rendering states (such as matrix stacks and texture bindings) between components do not pollute each other.

[0062] Create a base texture and associate it with a cross-process shared carrier: First, call glGenTextures to generate a base texture ID, configure the texture filtering mode and size, then use eglCreateImageKHR to convert the base texture into an EGLImage handle. This handle supports cross-process sharing and can avoid the cross-system invalidity problem caused by directly passing the texture ID. Finally, associate the EGLImage handle with the UCID and store it to prepare for subsequent cross-system transmission.

[0063] Bind dedicated frame buffer (FBO) and render buffer (RBO): Create an independent FBO for each UCID, bind the base texture as a color attachment to the FBO, and bind the corresponding RBO according to the component type (such as binding a 24-bit depth RBO for Activity components) to ensure that the rendering result is only written to its own dedicated texture.

[0064] S1.4 Dynamic scheduling and scheduled cleanup of the three-level resource pool.

[0065] Build a three-tiered resource pool consisting of active, cached, and recycled pools to achieve efficient resource reuse and recycling:

[0066] Active pool: Stores GLES context, EGLImage, FBO and other resources of currently visible components. It adopts the LRU (Least Recently Used) strategy to retain the top 10 active resources and prioritize the rendering response speed of foreground components.

[0067] Cache pool: Stores resources of background components (such as the onPause state), releases GLES context binding (calls eglMakeCurrent to bind an empty context) but retains texture data, starts an independent thread to traverse the cache pool every 5 minutes, and moves resources to the recycling pool if they are idle for more than 30 minutes.

[0068] Recycling pool: Stores resources of destroyed components (such as the onDestroy state), clears texture data but retains EGLConfig, prioritizes the reuse of recycled resources of the same type and specification when a new component is created, and reinitializes the context through eglInitialize before reuse to avoid old state residue, and completely releases unused resources after 10 minutes (by calling eglDestroyContext and glDeleteTextures).

[0069] S2. Real-time rendering isolation execution based on lifecycle and instruction interception ensures that the rendering instructions of each component are executed only in its own GLES context, avoiding resource conflicts and state pollution, while realizing the linkage between rendering state and component lifecycle.

[0070] S2.1, GLES command hook and directed routing.

[0071] GLES instruction interception is achieved through dynamic injection technology based on ptrace:

[0072] When the container starts, it attaches to the Android application process via ptrace (PTRACE_ATTACH), calls ptrace (PTRACE_GETREGS) to obtain the process register status, and parses the load address of libGLESv2.so by combining the / proc / [pid] / maps file;

[0073] Write the custom Hook library path into the process memory, trigger dlopen to load the Hook library by modifying the register value, hook the core GLES functions (such as glDrawArrays, glBindTexture, eglSwapBuffers), and call ptrace(PTRACE_DETACH) to unattach after loading is complete;

[0074] After intercepting the instruction, retrieve the UCID bound to the current thread from the thread-local storage (TLS) (the UCID is associated with the rendering thread when the component is created), query the independent GLES context corresponding to the UCID, and if the current context is inconsistent with the target context, call eglMakeCurrent to switch to the target context and then execute the original rendering instruction; if a cross-UCID instruction operation is detected (such as an instruction from component A attempting to modify the texture of component B), return an error code and log it to avoid state pollution.

[0075] S2.2 Rendering state is linked to component lifecycle.

[0076] Register Android component lifecycle callbacks (via Application.registerActivityLifecycleCallbacks) to dynamically adjust rendering parameters:

[0077] When a component enters the foreground (onResume): the corresponding resource is activated from the cache pool, the component-specific GLES context is switched, and the rendering thread priority is increased (set to THREAD_PRIORITY_URGENT_DISPLAY) to ensure smooth foreground rendering.

[0078] When a component goes into the background (onPause): the rendering thread is paused (via pthread_suspend), resources are moved to the cache pool, and the texture data of the current frame is retained for quick resumption;

[0079] Component stop (onStop): Releases the binding between the FBO and the texture (calls glBindFramebuffer to bind an empty FBO), but does not destroy resources, reducing the latency of subsequent recovery.

[0080] S2.3, Rendering conflict detection and quick snapshot recovery.

[0081] Introducing a rendering state validator to ensure rendering stability:

[0082] Every 100ms, glGetError checks the GLES error code. If GL_INVALID_OPERATION (cross-context operation flag) appears 3 times consecutively, depth conflict detection is triggered.

[0083] The GPU ComputeShader is used to compare the current context state with the expected state in parallel: the current texture, FBO configuration and historical snapshots (periodically saved to memory via eglSaveState) are compared at the pixel level, and the difference areas are marked.

[0084] If state contamination is detected (such as illegal modification of texture data), the context state is restored from the most recent correct snapshot. The restoration process is controlled within 10ms to prevent rendering errors from spreading to the user's visible plane.

[0085] S3, region difference texture increment capture and cross-system transfer, only transmits the changed areas of the rendering result, and achieves efficient transfer through cross-system shared memory and EGLImage, ensuring data integrity and validity.

[0086] S3.1 Incremental Area Intelligent Detection and Optimization.

[0087] Capture incrementally changing regions when the eglSwapBuffers component is triggered (after each frame is rendered):

[0088] A double-buffered queue is maintained for each UCID (prevBuffer stores the texture of the previous frame, and currentBuffer stores the texture of the current frame). The texture data of the current frame is copied to the previous frame buffer by glCopyTexSubImage2D to avoid memory race caused by direct reference.

[0089] By comparing the pixel data of two frames of texture using GPU ComputeShader, a differential mask texture is generated (the pixel value of the changed area is 1, and the rest is 0). The mask texture is then parsed to obtain the minimum bounding rectangle (deltaRect, which contains the coordinate range of all changed pixels).

[0090] If the differential region area accounts for more than 80% of the total frame area (such as in a page transition scenario), then differential calculation is abandoned and the full frame data is transmitted directly to avoid the additional overhead of differential logic.

[0091] S3.2 Texture compression and data integrity verification.

[0092] Process incremental region data to ensure transmission efficiency and reliability:

[0093] Based on the adaptive selection compression algorithm for region content: If the region is a UI control (the edge continuity is calculated by the Sobel operator, and if the edge is continuous, it is determined to be a UI), ETC2 compression is used (compression ratio 4:1, ensuring edge clarity); if it is dynamic content (such as video, game screen), ASTC 6x6 compression is used (balancing compression ratio and color reproduction).

[0094] Convert the RGB565 format on the Android side to the RGBA8888 format supported by OpenHarmony (the format conversion is achieved through GPU texture sampling, avoiding latency caused by CPU involvement).

[0095] Calculate the CRC32 checksum of the compressed data (using the crc32 function in the zlib library), transmit it along with the data, and OpenHarmony will verify it upon receipt. If they do not match, it will request retransmission to avoid data corruption.

[0096] S3.3, Cross-system shared memory creation and negotiation.

[0097] Led by OpenHarmony, a cross-system shared memory channel was built:

[0098] On the OpenHarmony side, a 10MB anonymous shared memory (named AndroidTextureShm) is created via OH_Ashmem_Create, and a file descriptor ohAshmemFd is returned.

[0099] The ohAshmemFd is passed to the Android container via inter-process communication (IPC): the OpenHarmony side writes the file descriptor through Parcel.writeFileDescriptor, and the Android side receives it through Parcel.readFileDescriptor, ensuring consistent permissions.

[0100] Both parties will divide the shared memory into 10 equal partitions (1MB each), corresponding to 10 concurrent components. The first 64 bytes of each partition are defined as DataHeader (containing UCID length, data length, CRC32 checksum, atomic ready flag, EGLImage handle, and reserved fields). Subsequent space will store compressed data, and a fixed structure will be used to ensure data parsing consistency.

[0101] S3.4, EGLImage cross-process transfer and import.

[0102] Efficient transmission of texture handles based on EGLImage:

[0103] When transmitting on the Android side, the EGLImage handle is obtained from the UCID associated storage, written to the DataHeader field of the shared memory partition, the compressed incremental data is written to the subsequent space of the partition, and finally the ready flag is set to True through atomic operation to notify the OpenHarmony side to read.

[0104] The OpenHarmony side polls the ready flag of the shared memory partition. If True is detected, the CRC32 value is checked first. If the check passes, the EGLImage handle is extracted from the DataHeader. The EGLImage from the Android side is imported into the local GLES context through eglImportImageKHR to generate a texture identifier that can be used by the OpenHarmony side, thus avoiding invalidity issues caused by directly using the handle.

[0105] S3.5 Dynamic scheduling of transmission priorities.

[0106] Allocate transmission bandwidth based on component importance:

[0107] Define the priority scoring formula: score = (Z-axis level * 0.5) + (recent interaction time * 0.3) + (visibility * 0.2) (the higher the Z-axis level, the closer the recent interaction time, and the higher the visibility, the higher the score).

[0108] The transmission queue is sorted by score, and high-priority components (such as foreground interactive windows) occupy shared memory bandwidth first, while low-priority components (such as background windows) are delayed until bandwidth is available, thereby reducing system resource contention.

[0109] S4 features dynamic creation and texture binding display of OpenHarmony side windows, creating matching OpenHarmony windows for each Android component to achieve full-link texture loading and ensure accurate display of rendering results.

[0110] S4.1 Intelligent adaptation and initialization of window properties.

[0111] OpenHarmony launches a cross-system window manager (an independent service) that receives ComponentStartEvents (containing UCID, component type, initial size, dpi, and transparency) from the Android side and dynamically configures window properties.

[0112] Window type matching: Activity components correspond to APP_MAIN_WINDOW (supports split-screen and full-screen switching), Dialog components correspond to APP_DIALOG_WINDOW (with 8dp shadow and rounded corners by default), and floating window components correspond to APP_FLOAT_WINDOW (set FLAG_NOT_FOCUSABLE to avoid stealing focus).

[0113] Size and position calculation: Convert the window size according to the component dpi, the formula is: OpenHarmony size = Android size * (OpenHarmonydpi / Android dpi)), and set the default position according to the component type (Activity is centered, Dialog is above its own Activity, and floating window is in the upper right corner of the screen).

[0114] Transparency and hierarchy configuration: Map the transparency parameter (WindowManager.LayoutParams.alpha) of the Android component to the transparency of the OpenHarmony window, extract the Z-axis hierarchy (zOrder) of the Android component and convert it to the global hierarchy of OpenHarmony (multiply by 10 to avoid conflict with the native window) to ensure hierarchy consistency.

[0115] S4.2 Dynamic binding of windows and textures.

[0116] Establish a ternary mapping table of UCID, window ID, and texture identifier to achieve precise association between textures and windows:

[0117] After the window is created, the UCID and window ID are stored in the mapping table;

[0118] After importing the Android-side EGLImage to generate local texture identifiers, the corresponding window ID is queried from the mapping table to obtain the window's Surface (rendering carrier).

[0119] Request a buffer from the Surface and associate the local texture identifier with the buffer: write the texture data to the buffer using glEGLImageTargetTexture2DOES, set the texture scaling mode (such as SCALE_FIT to adapt to the window size) and alpha blending mode (BLEND_ALPHA to ensure proper transparency), and finally release the buffer to the Surface (flushBuffer) to complete the texture binding.

[0120] S4.3, VSync synchronization and full-link texture display.

[0121] Rendering and display are synchronized using the VSync signal, ensuring smooth on-screen display.

[0122] VSync signal registration: On the OpenHarmony side, VSync callbacks are registered through OH_VSync_RegisterCallback to obtain the screen refresh rate (e.g., 60Hz corresponds to triggering a callback every 16.67ms) to ensure that texture rendering is synchronized with screen refresh and to avoid screen tearing;

[0123] Texture drawing to Surface: In the VSync callback, initialize the OpenHarmony2D rendering engine (e.g., based on libnative_media_render.so), pass in the window Surface and texture identifier, set the rendering parameters (scaling mode, alpha value, blending mode), and draw the texture data to the Surface buffer;

[0124] Screen Compositing and Display: OpenHarmony's compositor combines the Surface data of Android component windows with native window (such as desktop and status bar) layers according to the global Z-axis hierarchy. The composited frame data is then submitted to the display hardware (GPU or Display Controller) through OH_Display_SubmitBuffer and finally displayed on the screen.

[0125] S4.4 Display status feedback and resource optimization.

[0126] Dynamically adjust transmission strategies to reduce unnecessary resource consumption:

[0127] The window visibility is obtained through Window.getVisibility. If the window is completely obscured (visibility is 0), a WindowInvisibleEvent is sent to the Android container to pause the rendering and transmission of the corresponding component.

[0128] If the window is partially visible (visibility greater than 0), only incremental data of the visible area is transmitted (the differential area is clipped using the window's visible coordinates), further reducing transmission bandwidth usage.

[0129] S5 enables cross-system window behavior coordination and rendering parameter adjustment, achieving real-time linkage between OpenHarmony window operations and Android component rendering behavior, ensuring a consistent interactive experience with native applications.

[0130] S5.1 Window operation reverse mapping and component response.

[0131] Intercept user interaction events from the OpenHarmony window and convert them into instructions that Android components can recognize:

[0132] Window resizing: When the user drags the window edge to resize it, the cross-system window manager calculates the resized OpenHarmony size, converts it to Android component size by dpi, and sends a ResizeEvent (containing UCID and target size) via IPC; after receiving it on the Android side, it calls View.setLayoutParams to update the component size, triggering component re-layout and rendering to ensure that the displayed content adapts to the new window;

[0133] Window position movement: When the user drags the window to change its position, a MoveEvent is sent to the Android container. The Android side updates the window position parameters of the component (WindowManager.LayoutParams.x / y) to ensure that the position of the pop-up (such as the menu) inside the component remains consistent with the main window.

[0134] Window closing: When the user manually closes the OpenHarmony window, a CloseEvent is sent. On the Android side, ActivityManager.killBackgroundProcesses is called to terminate the corresponding component process, thus avoiding resource leaks when the application continues to run even after the window is closed.

[0135] S5.2, intelligent linkage between focus and rendering frame rate.

[0136] Dynamically adjust the rendering frame rate based on the window focus state to balance smoothness and resource consumption:

[0137] When a window gains focus: a FocusGainEvent is sent, and the Android side increases the component's rendering frame rate to the screen refresh rate (e.g., 60fps). If it is a game component and a touch event is detected, it is temporarily increased to 90fps (for 1 second) to optimize operation response.

[0138] Window loses focus: Send a FocusLossEvent, and the Android side will reduce the frame rate to 15fps to reduce GPU and battery consumption; if the window is completely invisible, rendering will be paused (only texture data will be retained).

[0139] S5.3, Window Animation Collaboration and Visual Consistency.

[0140] Achieve seamless integration of animations across systems:

[0141] When an OpenHarmony window performs system animations (such as fade-in / fade-out, scaling, and panning), the cross-system window manager extracts animation parameters (such as transparency range, scaling factor, and animation duration) and sends an AnimationEvent via IPC.

[0142] After receiving the data on the Android side, the rendering parameters are driven to change via ValueAnimator: for example, during fade-in animation, the texture transparency is dynamically adjusted via glBlendFuncSeparate, and during scaling animation, the rendering matrix is ​​adjusted via glScalef to ensure that the Android component animation is synchronized with the OpenHarmony window animation, so that there is no visual disconnect.

[0143] S5.4, Cross-system resource constraint collaboration.

[0144] Based on the OpenHarmony system resource status, dynamically adjust the rendering resources of Android components:

[0145] On the OpenHarmony side, the system resources are monitored through ResourceManager (memory usage exceeds 80% or GPU load exceeds 90%), and ResourceLimitEvent (including UCID and limit type, such as texture memory, GLES version) is sent.

[0146] After receiving the data on the Android side, resource limitation strategies are executed: when limiting texture memory, glDeleteTextures is called to release unnecessary textures (such as cached historical frames) to ensure that memory usage does not exceed the limit; when downgrading the GLES version, the context is switched to a lower version (such as downgrading from 3.2 to 2.0) to prioritize the rendering quality of foreground components.

[0147] S6. Based on lifecycle, bidirectional resource reclamation and mapping cleanup synchronously reclaim cross-system resources and clean up mapping relationships when components are destroyed or windows are closed, avoiding memory leaks and invalid resource residues.

[0148] S6.1, Android-led resource recycling process.

[0149] When an Android component is destroyed (onDestroy), a full-chain recycling process is triggered:

[0150] The Android side sends a ComponentDestroyEvent (including UCID) to the OpenHarmony side and waits for the OpenHarmony side's recycling confirmation;

[0151] After receiving the message, the OpenHarmony side queries the corresponding window ID from the ternary mapping table, calls WindowManager.destroyWindow to destroy the window, deletes the mapping table record, releases the texture identifier and Surface resources, and sends WindowDestroyedAck for confirmation.

[0152] After receiving confirmation on the Android side, resource reclamation is performed: the EGLImage handle is destroyed (eglDestroyImageKHR), textures are deleted (glDeleteTextures), the GLES context (eglDestroyContext) and FBO are destroyed, and the EGLConfig is moved to the recycling pool; if no other components are using the shared memory, munmap is called to unmap and release the memory address.

[0153] S6.2, OpenHarmony-led resource recycling process.

[0154] When the user manually closes the OpenHarmony window, ensure that components and resources are cleaned up synchronously:

[0155] The cross-system window manager sends a WindowCloseEvent to the Android container, requesting the termination of the corresponding component process;

[0156] After receiving the message on the Android side, ActivityManager.killBackgroundProcesses is called to terminate the component process, release all resources associated with the component, and send ComponentKilledAck confirmation.

[0157] Upon receiving confirmation, OpenHarmony destroys the window and cleans up the mapping table and local texture resources; if no confirmation is received (e.g., the Android container is unresponsive), it forcibly cleans up local resources to avoid memory leaks.

[0158] S6.3 Resource protection and recovery in abnormal scenarios.

[0159] To address anomalies such as container crashes and shared memory failures, establish a fault tolerance mechanism:

[0160] Heartbeat detection: OpenHarmony sends a HeartbeatProbe (including UCID) to the Android container every 2 seconds, and the Android side must return a HeartbeatAck within 1 second; if no Ack is received for 3 consecutive times, the component is judged to have crashed, and OpenHarmony will forcibly destroy the corresponding window, clean up the mapping table, and release local resources.

[0161] Shared memory failure handling: The OpenHarmony side periodically checks the validity of shared memory through OH_Ashmem_GetSize. If memory failure is detected (such as the container crash causing memory to be released), the shared memory is recreated, and the file descriptor is retransmitted to the Android container via IPC to restore the transmission channel.

[0162] Texture transfer failure handling: If the Android side fails to receive read confirmation from the OpenHarmony side after three consecutive writes to shared memory, the transfer channel is deemed abnormal. The transfer is paused and an attempt is made to re-establish shared memory. If the retry fails after three attempts, the corresponding component process is terminated and resources are reclaimed.

[0163] Experimental verification shows that this invention offers the following key advantages over existing technologies: Fine-grained independence: Component-level rendering resource isolation allows Android applications' Activities, Dialogs, and floating windows to obtain independent windows on the OpenHarmony side, preventing rendering errors from affecting each other. It also supports all native window operations (switching, scaling, floating, etc.), providing a user experience close to native applications; High-efficiency transmission and display: Differential transmission reduces transmission speed by 70% to 90%. EGLImage's cross-process sharing avoids texture data copying, while VSync synchronization ensures seamless rendering and display, keeping the display latency of 1080P components within 15ms. The entire process is automated, from resource allocation and window creation at component startup to interactive behavior coordination and resource reclamation upon destruction, requiring no manual configuration and reducing the adaptation cost of new Android applications by 90%. Cross-system collaboration is achieved through reverse mapping of window operations, frame rate linkage, and animation coordination, ensuring consistent interactive experiences between Android components and OpenHarmony's native windows. Resource constraint coordination ensures overall system resource balance. High stability and maintainability are enhanced by rendering conflict detection and abnormal scene fault tolerance mechanisms, improving system stability by 90%. A three-level resource pool and bidirectional reclamation prevent memory leaks and reduce the difficulty of later maintenance. Broad adaptability is ensured through differentiated resource configurations supporting adaptation across all scenarios, from low-power devices (such as smartwatches) to high-performance devices (such as tablets), meeting the resource constraints of different hardware.

[0164] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for optimizing multi-window display in Android applications on the HarmonyOS platform, characterized in that, Specifically, the following steps are included: After the Android container starts, the HarmonyOS side creates a first shared memory containing multiple partitions; When an Android application component starts, it assigns a first component ID, binds the first component ID to the current rendering thread, configures GLES resources, creates a double-buffered queue including a first buffer and a second buffer. The first buffer stores the pixel data of the texture from the previous frame, and the second buffer stores the pixel data of the texture from the current frame. It also creates a first GLES context, creates a base texture and converts it into a shared image resource handle, and associates the first component ID with the shared image resource handle. Create a frame buffer with the base texture as the color attachment and bind it to the component. Bind the render buffer to the component. Send the component start event to trigger the creation of the window on the HarmonyOS side. Set its display level in multiple windows. Establish the first mapping table between the first component ID, window ID and texture identifier. When a component performs rendering, it obtains the current GLES context and the ID of the second component bound to the current rendering thread. If the current GLES context is different from the first GLES context, it switches to the first GLES context to perform rendering. If the second component ID is different from the first component ID, it returns an error code. Write the rendered pixel data into the second buffer, calculate the difference region between the pixel data in the first buffer and the second buffer, if the ratio of the area of ​​the difference region to the area of ​​the whole frame is greater than the threshold, then all the current pixel data is the first data; otherwise, compress the difference region into the first data, and calculate the checksum of the first data. The data header consists of the component ID length, transmission data length, checksum, ready flag, shared image resource handle, and reserved fields. The data header and the first data are written to the partition, and the ready flag is set to TRUE to notify the HarmonyOS side. The HarmonyOS side verifies the checksum of the ready flag TRUE. If it passes, it extracts the shared image resource handle, generates a texture identifier, and adds it to the first mapping table. It obtains the window ID from the first mapping table, requests a Surface buffer, associates the texture identifier with the Surface buffer, and releases the first data into the Surface buffer. When the callback is triggered, the HarmonyOS side draws the texture pixel data onto the Surface buffer, synthesizes the window Surface data of Android and HarmonyOS, and submits it to the display hardware to complete the on-screen display.

2. The multi-window display optimization method according to claim 1, characterized in that, A three-tiered resource pool is constructed, including: an active pool, which stores the first GLES context, shared image resource handles, and frame buffers of currently visible components, retaining a set number of active resources using a least recently used strategy; a cache pool, which stores resources of background components, retains texture data, releases the first GLES context binding of background components, and starts an independent thread to periodically traverse the cache pool, moving resources to the recycling pool when they are idle for more than a set time; and a recycling pool, which stores resources of destroyed components, clears texture data, retains shared image resource handles, prioritizes the reuse of the same recycled resources when a new component is created, reinitializes the context before reuse, and releases resources that have not been reused for more than a set time.

3. The multi-window display optimization method according to claim 1, characterized in that, The method for creating windows on the HarmonyOS side is as follows: active components correspond to APP_MAIN_WINDOW to support split-screen and full-screen switching, dialog components correspond to APP_DIALOG_WINDOW, and floating window components correspond to APP_FLOAT_WINDOW; the window size is determined according to the clarity of the component, and the default position of the window is set according to the component type; the transparency parameter of the component is mapped to the transparency of the HarmonyOS window, and the Z-axis layer of the component is converted into the HarmonyOS global layer.

4. The multi-window display optimization method according to claim 1, characterized in that, The bandwidth allocation method for the first shared memory is as follows: The scoring formula is defined as: score = (Z-axis level * 0.5) + (recent interaction time * 0.3) + (visibility * 0.2). The transmission queues are sorted according to the scores. The transmission queues with scores greater than the threshold occupy the bandwidth of the first shared memory, while the transmission queues with scores less than the threshold are delayed until the bandwidth is available.

5. The multi-window display optimization method according to claim 1, characterized in that, HarmonyOS monitors window visibility in real time. If the window is completely obscured, it sends an event to the Android container to pause the rendering and transmission of the corresponding component. If the window is partially visible, it only transmits the pixel data of the visible area in the next incremental transmission.

6. The multi-window display optimization method according to claim 1, characterized in that, Intercept user operations on the HarmonyOS side window, convert them into instructions recognizable by the Android container side, and send them to the Android container side via cross-process communication; when the window size is adjusted, the Android container side updates the component layout parameters and triggers re-rendering; when the window is moved, the component position parameters are updated to modify the position of the pop-up window within the container; when the window is closed, the component process is terminated; when the window gains focus, the frame rate is increased; when it loses focus, the frame rate is decreased; when it is completely invisible, rendering is paused; send the animation parameters of the HarmonyOS side window system to the Android container side, and the Android container side sets the rendering parameters to synchronize the component animation with the window animation.

7. The multi-window display optimization method according to claim 1, characterized in that, When an Android container-side component is destroyed, it sends a destruction event to the HarmonyOS side. Upon receiving the event, the HarmonyOS side queries and destroys the corresponding window based on the component ID, clears the first mapping table, releases textures and Surface resources, and sends a confirmation message. After receiving the confirmation message, the Android container-side reclaims the shared image resource handle, textures, and the first GLES context. If the first shared memory is not used by other components, it releases its corresponding memory address. When the HarmonyOS-side window is closed, the HarmonyOS side sends a close event to request the Android container-side to terminate the corresponding component process. After the Android container-side terminates the process and releases all component resources, it sends a confirmation message. After receiving the confirmation message, the HarmonyOS side clears the window, the first mapping table, and local texture resources; otherwise, it forcibly clears local resources.

8. The multi-window display optimization method according to claim 1, characterized in that, The method for configuring GLES resources is as follows: For components of type "Active", configure GLES version 3.2, a texture memory limit of 256MB, a 24-bit depth buffer, and support for 4x anti-aliasing; for components of type "Dialog", configure GLES version 2.0, a texture memory limit of 32MB, a 16-bit depth buffer, and retain only the basic color format; for components of type "Floating Window", reuse the GLES version and color format of the same application component, with a texture memory limit of 16MB and no depth buffer.

9. The multi-window display optimization method according to claim 1, characterized in that, When a component enters the foreground, the corresponding resource is activated from the cache pool, and the component's first GLES context is switched. When a component enters the background, the rendering thread bound to the component is paused, the corresponding resource is moved to the cache pool, and the texture data corresponding to the current frame is retained. When the component stops, the binding relationship between the frame buffer and the corresponding texture is released.

10. The multi-window display optimization method according to claim 1, characterized in that, The method for compressing the differential region into the first data is as follows: if the differential region is a UI control, then ETC2 is used to compress the differential region; if it is dynamic content, then ASTC 6x6 is used to compress the differential region.

Images