An Android container rendering optimization method based on cross-domain hard real-time fence synchronization

Abstract

This invention discloses an Android container rendering optimization method based on cross-domain hard real-time fence synchronization. Using HarmonyOS as the host and Android as the container, the host creates a hash table corresponding to the GPU core, a synchronization thread, and shared memory upon startup. It obtains a VSync periodic registration callback, passes the shared memory file descriptor (FD) to the container, and the container completes shared memory mapping and alignment, registering a GPU queue completion callback. When the Android application starts, the container obtains the queue and the physical address of the rendering buffer, creates a fence and binds it to the queue, submits the queue, writes metadata to shared memory to notify the host, and upon receiving this, the host creates a node, stores it in the hash table, and registers a timeout timer. After the GPU completes the command queue, the container sends a callback to update the status and checksum, notifying the host. After the host verifies the validity, it updates the corresponding hash table, resets the timer, and triggers asynchronous on-screen display. The host executes buffer compositing and the display device submission to complete the on-screen display, improving the stability of the rendering frame rate and ensuring the reliability of cross-domain synchronization.

Description

Technical Field

[0001] This invention belongs to the field of computer software development technology, specifically relating to an Android container rendering optimization method based on cross-domain hard real-time Fence synchronization. Background Technology

[0002] When HarmonyOS runs Android applications using containerization technology, there are still underlying technical bottlenecks in graphics rendering synchronization that are difficult to overcome with existing solutions.

[0003] Specifically, the core function `glFinish` of the OpenGL graphics API is used to force the drawing instructions in the buffer to be submitted to the graphics hardware and block the calling thread until all operations are completed. This function ensures the integrity of operations such as frame buffer modification and state change by synchronizing the execution flow of the CPU and GPU. It is often used in scenarios that require strict timing control, such as screen capture and resource release. However, the blocking of `glFinish` may lead to a serial deadlock between the CPU and GPU. For example, in existing methods, the `eglSwapBuffers` function of the Android system implicitly calls `glFinish`, forcing the CPU to wait for the GPU to complete all current rendering tasks before executing subsequent logic. This creates a serial dependency between the CPU and GPU. In a high frame rate scenario of 120fps, the maximum allowed time for a single frame is only 8.3ms. If the GPU rendering takes 8ms, the CPU must wait 8ms before starting the next frame task. This directly limits the frame rate to 62.5fps. During the waiting period, the CPU resource idle rate exceeds 40%, and it cannot handle other core tasks such as UI layout and touch event response, resulting in a serious waste of system resources.

[0004] The Android container system and the HarmonyOS host system belong to independent address spaces. Existing cross-domain synchronization solutions have significant defects that lead to high latency and unreliability in cross-domain synchronization. These defects mainly include: First, high IPC communication overhead. Cross-domain communication based on Socket and Binder requires three levels of data forwarding from Android user space to Linux kernel space and then to HarmonyOS user space. The time for a single synchronization can reach 20μs to 50μs. In a 120fps scenario, the synchronization overhead accounts for more than 12%, directly causing a decrease in the overall frame rate. Second, the lack of a signal mechanism for shared memory. Traditional circular shared memory relies on polling to detect data updates. When the polling interval is 1ms, the CPU utilization rate reaches 8% to 10%. If the interval is increased, the synchronization latency can exceed 1ms, resulting in frame loss. Third, the lack of data verification. Critical information such as FenceID and physical address of rendering buffer transmitted across domains is prone to failure to be displayed due to memory tampering or address mapping errors, with an error rate of about 0.5%, and there is a lack of effective degradation mechanism.

[0005] Because HarmonyOS lacks a native Fence, it cannot meet the requirements of hard real-time operation, resulting in limitations in Android application adaptation. First, kernel-level Fence adaptation is costly. Linux dma_fence requires modifications to the HarmonyOS kernel and GPU driver, involving more than 20 files in the kernel / drivers / gpu directory. This adaptation process is lengthy and does not support user-level Fence calls from Android containers. Second, user-level polling is inefficient. Fixed-interval polling cannot match the dynamic time consumption of GPU tasks. In high-frame-rate scenarios, polling latency may exceed 1ms, while in low-frame-rate scenarios, it excessively consumes CPU resources, making it difficult to balance synchronization latency and resource consumption.

[0006] Furthermore, because the existing solution is not bound to the VSync signal of the HarmonyOS display device, it can lead to a disconnect between the screen display timing and hardware signals. Specifically, if the screen is displayed immediately after rendering, and this happens to coincide with the display device's scanning cycle, such as when the CRT scan line has not reached the bottom of the screen, it can result in a screen tearing rate of up to 5%. The lack of priority scheduling among multiple types of buffers can cause high frame rate game frames to be easily blocked by low priority UI frames, resulting in stuttering.

[0007] In summary, existing methods cannot simultaneously solve technical problems such as hard real-time synchronization, low CPU usage, cross-domain reliability, and tear-free display. Summary of the Invention

[0008] In view of this, the present invention provides an Android container rendering optimization method based on cross-domain hard real-time Fence synchronization, which uses HarmonyOS as the host system and Android as the container system to achieve cross-domain hard real-time Fence synchronization and on-screen display.

[0009] This invention provides an Android container rendering optimization method based on cross-domain hard real-time fence synchronization, which specifically includes the following steps:

[0010] Using HarmonyOS as the host and Android as the container;

[0011] When the host starts, it creates a hash table corresponding to each GPU core, containing metadata, timeout, and timer ID, with the index bound to a singly linked list. It then starts a low-priority synchronization thread. It creates a physically contiguous first shared memory, divides it into a metadata area and a synchronization signal bit, maps it to user space to obtain a virtual address, and records the file descriptor (FD). It obtains the VSync cycle and registers a callback, passing the first shared memory FD to the container. After receiving the FD, the container maps the memory and initializes and aligns the second shared memory with it, registering the GPU command queue to complete the callback.

[0012] After the Android application starts, the container obtains the physical addresses of the active GPU command queue and rendering buffer, creates a Fence and binds it to the end of the queue, generates metadata containing FenceID, physical address, command queue ID, status and check value, and the status is pending trigger. After storing it in the second shared memory, it notifies the host through Futex to submit the GPU command queue to the hardware. The host synchronization thread receives the signal, creates a metadata node and stores it in the corresponding hash table, and registers a one-time timer.

[0013] After the GPU completes the command queue, the container callback updates the Fence status to complete and the verification value, and notifies the host through Futex; after the host synchronous thread verifies the validity, it cancels the original timer, updates the hash table node and re-registers the timer, backs up the physical address to trigger asynchronous screen display, and sets the Fence status to idle.

[0014] The host obtains the current time and VSYNC timestamp, calculates the display window, performs a sleep based on the display window, and then performs buffer compositing and display device submission to complete the display.

[0015] Furthermore, the hash table index is bound to a singly linked list. When there is no conflict, the metadata node is saved to the position corresponding to the index. When there is a conflict, the original metadata node is not replaced, and the new metadata node is added to the head of the linked list corresponding to the index.

[0016] Furthermore, the method by which the host synchronization thread receives signals, creates metadata nodes, and stores them in the corresponding hash table is as follows:

[0017] If the semaphore count is 0, the thread blocks; otherwise, acquire the semaphore from the synchronization signal and decrement the semaphore count by 1; create a metadata node, read the metadata and fill the corresponding fields of the metadata node with the FenceID, physical address, command queue ID, status, and checksum; calculate the GPU group index based on the command queue ID in the metadata and determine the corresponding first hash table; calculate the current timeout of the host system, register a one-time timer that will be triggered after a set time, associate the timer callback function with the FenceID and command queue ID, write the metadata node, timeout, and timer ID to the head of the singly linked list of the first hash table, and update the head pointer of the first hash table.

[0018] Furthermore, the method for calculating the GPU group index based on the command queue ID in the metadata is as follows: the lower N bits to be taken are calculated using log2 (number of GPU cores), and 2^N-1 is used as the mask value. The command queue ID and the mask value are then subjected to a bitwise AND operation to obtain the GPU group index.

[0019] Furthermore, the verification value is a combined CRC32 value of FenceID, physical address, command queue ID, and status.

[0020] Furthermore, after the timer is triggered, the host parses the parameters to extract the FenceID and command queue ID, and looks up the corresponding metadata node; if the status of the metadata node is pending triggering, the status is updated to timeout, and the physical address corresponding to the previous frame of data is displayed on the screen; if the status is triggered, it returns directly without performing any additional operations.

[0021] When the Fence trigger completes or times out, the corresponding metadata node is removed from the first hash table, the timer associated with the node is destroyed, and the memory of the cache line aligned node is released.

[0022] Furthermore, the method for calculating the on-screen window is as follows: the host obtains the current system time and VSync timestamp, and calculates the start time and end time of the on-screen window based on the VSync timestamp and the set preceding and following times. That is, the start time of the on-screen window is the difference between the VSync timestamp and the preceding time, and the end time of the on-screen window is the sum of the VSync timestamp and the following time.

[0023] Furthermore, the method of executing sleep based on the on-screen window is as follows: if the current system time is less than the start time of the on-screen window, the difference between the start time of the on-screen window and the current time is used as the waiting time, and sleep is executed; if the current system time is greater than the end time of the on-screen window, the difference between the sum of the start time of the on-screen window and the VSync cycle and the current time is used as the waiting time, and sleep is executed.

[0024] Furthermore, when the container writes to the second shared memory, the buffer type is marked by the high 4 bits of the physical address, including three categories: highest priority, medium priority, and lowest priority.

[0025] The host maintains composition queues with corresponding priorities, including the highest priority queue, the medium priority queue, and the lowest priority queue. When the frame is displayed, the queues are checked in descending order of priority, and high-priority frames are processed first. If there is a queue whose length exceeds the threshold, the earliest frame is discarded.

[0026] Furthermore, in the GPU driver source code, a Fence trigger callback is added to the command queue completion interrupt handling function of the GPU driver. First, the interrupt register is parsed to extract the command queue ID, then the FenceID at the end of the queue is read through the driver's internal interface, and finally the ohFenceTrigger function is called to trigger the status update and screen upload callback. The interrupt priority is set to high priority and one-time trigger, and the priority parameter is set in the driver interrupt registration function. Beneficial effects

[0027] This invention uses HarmonyOS as the host system and Android as the container system. Upon startup, the host system creates a grouped hash table corresponding to each GPU core, a low-priority synchronization thread, and a physically contiguous first shared memory. It acquires the VSync cycle, registers callbacks, and passes the shared memory file descriptor (FD) to the container side. The container side receives the FD, completes shared memory mapping and alignment, and registers a GPU command queue to complete the callback. Upon startup, the container side acquires the GPU command queue and the physical address of the rendering buffer, creates a Fence and binds it to the queue. After submitting to the queue, it writes metadata containing FenceID, physical address, and other information to the shared memory. The host side is notified via Futex. Upon receiving the signal, the host-side main synchronization thread creates a node and stores it in the corresponding hash table according to the GPU command queue ID, and registers a timeout timer. After the GPU completes the command queue, the container side calls back to update the Fence status and CRC32 checksum and notifies the host side. After the host-side synchronization thread verifies the validity, it updates the corresponding hash table node, resets the timer, backs up the buffer address, and triggers asynchronous display. The host side, based on the current time and VSync display window, precisely sleeps and then performs buffer compositing and display device submission to complete the display, effectively improving the stability of the rendering frame rate and ensuring the reliability of cross-domain synchronization. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating an Android container rendering optimization method based on cross-domain hard real-time Fence synchronization provided by the present invention. Detailed Implementation

[0029] The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0030] This invention provides an Android container rendering optimization method based on cross-domain hard real-time Fence synchronization. The core idea is as follows: using HarmonyOS as the host system and Android as the container system, when the host system starts, it creates a grouped hash table corresponding one-to-one with the GPU cores, a low-priority synchronization thread, and a physically contiguous first shared memory. It obtains the VSync cycle, registers callbacks, and passes the shared memory FD to the container side. The container side receives the FD, completes shared memory mapping and alignment, and registers the GPU command queue to complete the callback. When the Android application starts, the container side obtains the GPU command queue and the physical address of the rendering buffer, creates a Fence and binds it to the queue, and... After the queue is handed over, metadata containing information such as FenceID and physical address is written to shared memory. The host side is notified through Futex. After receiving the signal, the host-side synchronization thread creates a node and stores it in the corresponding hash table according to the GPU command queue ID, and registers a timeout timer. After the GPU completes the command queue, the container side calls back to update the Fence status and CRC32 check value and notifies the host side. After the host-side synchronization thread verifies the validity, it updates the corresponding hash table node, resets the timer, backs up the buffer address, and triggers asynchronous screen display. The host side performs buffer synthesis and display device submission according to the current time and VSync screen display window, and then performs precise sleep to complete the screen display.

[0031] This invention provides an Android container rendering optimization method based on cross-domain hard real-time fence synchronization, the process of which is as follows: Figure 1 As shown, the specific steps include:

[0032] Step 1: Use HarmonyOS as the host system and Android as the container system;

[0033] When the host system starts, it obtains the number of GPU cores, creates a hash table array containing the same number of first hash tables as the number of cores, with each first hash table corresponding to a GPU core. Each first hash table contains Fence metadata, timeout duration, and timer ID, and each index in the first hash table is bound to a singly linked list. It then starts a low-priority first synchronization thread; creates a physical contiguous memory with read and write permissions, designated as the first shared memory; divides the first shared memory into a metadata area and synchronization signal bits. The metadata area stores Fence metadata nodes, and the synchronization signal bits store synchronization signals. It maps the first shared memory to the user-space address space to obtain its corresponding virtual address, designated as the first virtual address; records the file descriptor (FD), first virtual address, frame count, metadata size, and synchronization signal bit address of the first shared memory; obtains the current VSync cycle of the display device and registers the VSync signal callback function; and passes the FD of the first shared memory to the container system.

[0034] After receiving the file descriptor (FD) of the first shared memory, the container system maps the first shared memory to the user-space address space of the container system, initializes the second shared memory in the container system according to the parsed metadata area and synchronization signal bit address, aligns the second shared memory with the first shared memory on the host system side, and registers a first callback function, which is called after the GPU completes the command queue.

[0035] Specifically, each index of the first hash table is bound to a singly linked list. When there is no conflict between the metadata nodes, the Fence metadata node is directly saved to the position corresponding to the index. When there is a conflict between the nodes, the original node is not replaced. Instead, the new Fence metadata node is added to the head of the linked list corresponding to the index. All conflicting nodes are linked together through the linked list.

[0036] Step 2: When the Android application process starts, the container system obtains the active GPU command queue and records it as the current GPU command queue. It records the physical address of the rendering buffer as the first physical address, creates a GPU synchronization type Fence object and records it as the first Fence, writes the ID of the first Fence (first FenceID) to the end of the current GPU command queue, and saves the association between the first Fence and the first physical address to the first mapping table. The Fence metadata consists of the first FenceID, the first physical address, the current GPU command queue ID, the Fence status, and the CRC32 check value. The Fence status is pending triggering. The current GPU command queue is submitted to the hardware for execution. The Fence metadata is written to the free metadata area in the second shared memory. The generated synchronization signal is written to the corresponding synchronization signal bit. The Futex mechanism is used to notify the host system that new data has been written.

[0037] After receiving the synchronization signal, the first synchronization thread in the host system blocks if the semaphore count is 0; otherwise, it retrieves the semaphore from the synchronization signal, decrements the semaphore count by 1, creates a Fence metadata node, reads the Fence metadata and fills the corresponding fields of the Fence metadata node with the first FenceID, first physical address, current GPU command queue ID, Fence status, and CRC32 checksum. Based on the current GPU command queue ID in the Fence metadata, it calculates the GPU group index and determines the first hash table corresponding to that GPU. It calculates the current timeout of the host system, registers a one-time timer triggered after a set time, associates the timer callback function with the first FenceID and the current GPU command queue ID, writes the Fence metadata node, timeout, and timer ID to the head of the singly linked list of the first hash table, and updates the head pointer of the first hash table.

[0038] This invention modifies the eglSwapBuffers function, removing the glFinish call and adding operations such as obtaining GPU command queue and rendering buffer information, creating Fence objects, binding Fence to the GPU command queue, submitting the queue and recording Fence information to cross-domain shared memory, thereby realizing the creation and binding of Fences related to rendering and display operations.

[0039] Furthermore, to adapt to the Fence interfaces of different GPU manufacturers, this invention encapsulates a unified FenceID acquisition interface in the Android HAL layer to ensure compatibility.

[0040] The calculation and verification process of the CRC32 checksum is as follows: When the container system builds the Fence metadata, it calculates the combined CRC32 value of the fields other than the CRC32 checksum, and uses this combined CRC32 value as the CRC32 checksum of the Fence metadata; After the host system reads the Fence metadata, it calculates the combined CRC32 value of the fields other than the CRC32 checksum. If the calculation result is consistent with the CRC32 checksum, the current Fence metadata is parsed; otherwise, the current Fence metadata is marked as an invalid node and subsequent processing is skipped.

[0041] This invention employs the Linux Futex mechanism to achieve CPU-free signal synchronization between the container and the host, efficiently transmitting Fence metadata and avoiding resource waste caused by polling. Specifically:

[0042] After the container system writes Fence metadata, it actively triggers a signal, locates the free metadata area in the second shared memory through the writeIdx pointer of the circular queue, fills in core fields such as the first FenceID, the first physical address, the current GPU command queue ID, and the Fence status, calculates and writes the CRC32 check value, increments the writeIdx pointer in a circular manner to avoid out-of-bounds errors, and calls the futex_wake function with the signal bit address and the number of wake-up threads to notify the host system to process the new data.

[0043] The host system blocks and waits to process Fence metadata, starts the first synchronization thread with low priority (SCHED_IDLE), and repeatedly executes the operations from waiting for the signal to reading and processing. The futex_wait function is called, and the thread blocks when the signal bit is 0 and returns immediately when it is not 0. The node to be processed is located through the readIdx pointer, and CRC32 verification is performed. If the verification is successful, the Fence metadata is saved and the readIdx pointer is updated. If the verification fails, it is skipped and the readIdx pointer is updated.

[0044] Step 3: After the GPU finishes executing the current GPU command queue, the first callback function in the container system updates the Fence status in the Fence metadata corresponding to the first FenceID and the current GPU command queue to "completed", calculates and updates the CRC32 check value; updates the Fence status and CRC32 check value of the corresponding metadata area in the second shared memory, writes the generated synchronization signal to the corresponding synchronization signal bit, and uses the Futex mechanism to notify the host system that new data has been written.

[0045] After receiving the synchronization signal, the first synchronization thread in the host system blocks if the semaphore count is 0; otherwise, it retrieves the semaphore from the synchronization signal and decrements the semaphore count by 1. It then traverses the Fence metadata nodes in the first shared memory. For Fence metadata nodes with a completed Fence state and a valid CRC32 checksum, it calculates the GPU group index based on the corresponding current GPU command queue ID to obtain the corresponding first hash table. It cancels the original timer, updates the Fence metadata node using the first FenceID, first physical address, current GPU command queue ID, Fence state, and CRC32 checksum from the Fence metadata, calculates the current timeout of the host system, re-registers a one-time timer triggered after a set time, associates the timer callback function with the first FenceID and current GPU command queue ID, writes the Fence metadata node, timeout, and timer ID to the head of the singly linked list of the first hash table, and updates the head pointer of the first hash table. Finally, it updates the first physical address of the first Fence to the global backup variable, triggers asynchronous screen display using the first physical address, and sets the Fence state of the first Fence to idle.

[0046] The method for calculating the GPU group index based on the current GPU command queue ID in the Fence metadata is as follows: based on the number of GPU cores, log2 (number of GPU cores) is used as the N value, and 2^N-1 is used as the mask value. The N value represents the lower N bits to be extracted, and the mask value is N bits of all 1s in binary, which are used for bitwise AND operation to extract the lower N bits; the current GPU command queue ID and the mask value are subjected to a bitwise AND operation, and the result is the GPU group index.

[0047] This invention avoids memory leaks and display anomalies by setting a fixed timeout period and controlling the fence lifecycle, specifically including:

[0048] After the timer is triggered, the host system parses the parameters to extract the first FenceID and the current GPU command queue ID, and looks up the corresponding Fence metadata node. If the Fence status of the Fence metadata node is pending triggering, the Fence status is updated to timeout, and the first physical address corresponding to the previous frame of data is displayed on the screen. If the Fence status is triggered or recycled, it returns directly without performing any additional operations.

[0049] When the Fence trigger completes or times out, the corresponding Fence metadata node is removed from the first hash table, the timer associated with the node is destroyed, and the cache line aligned node memory is released.

[0050] Furthermore, for cases where the GPU driver source code can be obtained, this invention modifies the driver's interrupt handling function and injects a Fence trigger callback to achieve real-time Fence triggering based on GPU hardware interrupts, specifically as follows:

[0051] The interrupt callback injection logic adds a Fence trigger callback to the interrupt handling function of the GPU driver's command queue. First, the interrupt register is parsed to extract the current GPU command queue ID. Then, the first Fence ID at the end of the queue is read through the driver's internal interface. Finally, the host system's ohFenceTrigger function is called to trigger the status update and on-screen callback.

[0052] Interrupt priority configuration: Set the interrupt priority to high priority and one-time trigger to avoid preemption by low-priority interrupts and reduce response latency; set the priority parameter in the driver interrupt registration function to ensure the configuration takes effect.

[0053] Step 4: The host system obtains the current system time and VSync timestamp. Based on the VSync timestamp, it calculates the start time and end time of the current on-screen window. If the current system time is less than the start time of the on-screen window, the difference between the start time and the current time is used as the waiting time, and a precise sleep is executed. After the sleep ends, the buffer compositing and display device submission operations are executed sequentially to complete the on-screen display. If the current system time is greater than the end time of the on-screen window, the difference between the sum of the start time and the VSync period and the current time is used as the waiting time, and a precise sleep is executed. After the sleep ends, the buffer compositing and display device submission operations are executed sequentially to complete the on-screen display.

[0054] Furthermore, the calculation method for the on-screen window is as follows: the host obtains the current system time and VSync timestamp, and calculates the start time and end time of the on-screen window based on the VSync timestamp and the set preceding and following times. That is, the start time of the on-screen window is the difference between the VSync timestamp and the preceding time, and the end time of the on-screen window is the sum of the VSync timestamp and the following time.

[0055] Furthermore, this invention implements static priority scheduling based on buffer type, ensuring that high frame rate frames are displayed on the screen first, specifically as follows:

[0056] When the container system writes to the second shared memory, the type is marked by the high 4 bits of the 64-bit buffer physical address. For example, 0x1 is for game frames with the highest priority, 0x2 is for video frames with medium priority, and 0x3 is for UI frames with the lowest priority.

[0057] The host system maintains composition queues with corresponding priorities, including the highest priority queue, the medium priority queue, and the lowest priority queue, all of which use FIFO order to avoid frame corruption; when uploading to the screen, the queues are checked in descending order of priority, and high-priority frames are processed first; if there is a queue whose length exceeds the threshold, the earliest frame is discarded to prevent stuttering;

[0058] After extracting the target buffer, multi-layer compositing is completed. After compositing, the buffer is submitted to the display device to complete the on-screen display. Example

[0059] This embodiment uses OpenHarmony as the host machine as an example, and adopts an Android container rendering optimization method based on cross-domain hard real-time Fence synchronization provided by this invention. By modifying the underlying Android source code to achieve deep binding between the Fence and GPU commands, and optimizing cross-domain synchronization based on physical contiguous memory and hardware signals, it adapts to the OpenHarmony environment without a native Fence, achieving a stable Android container graphics rendering frame rate exceeding 120fps, CPU utilization controlled below 1.2%, and cross-domain synchronization latency ≤5μs, meeting the requirements of hard real-time rendering scenarios. The specific process includes:

[0060] S1. Modify the Android source code to deeply bind OpenGL ES rendering with the fence.

[0061] By modifying the source code of the Android underlying EGL library, GPU command queue management module, and Hardware Abstraction Layer (HAL), a one-to-one binding between the Fence and the GPU command queue is achieved, removing the glFinish blocking operation and ensuring the underlying controllability of Fence creation, binding, and state synchronization. Simultaneously, it adapts to the interface differences of mainstream GPU manufacturers, such as Qualcomm Adreno, ARM Mali, and Imagination PowerVR. For mainstream container adaptation versions such as Android V10 to V14, the core modules and file paths requiring modification are determined to ensure adaptability and compatibility, including the EGL core library, GPU command queue management, and graphics HAL layer.

[0062] S1.1 Modify the EGL core interface to implement Fence creation and binding.

[0063] The eglSwapBuffers function was modified, including removing the glFinish call, obtaining GPU command queue and rendering buffer information, creating a Fence object, binding the Fence to the GPU command queue, submitting the queue, and recording the Fence information to cross-domain shared memory. The specific process is as follows:

[0064] Remove blocking calls: Locate the original glFinish call code in the eglSwapBuffers function, which is usually located in the middle of the function and is used to force the GPU to finish rendering. Comment out or delete it to eliminate the root cause of CPU blocking.

[0065] Obtaining GPU and buffer resources: The current EGL context is obtained using eglGetCurrentContext. The currently active GPU command queue is obtained through the GPU queue management interface associated with the current EGL context, using the getGpuQueue method added in egl_context.cpp. At the same time, the eglQuerySurface interface is called, relying on the EGL_EXT_physical_address extension, to read the physical address of the post-rendering buffer EGL_BACK_BUFFER, ensuring that it can be directly accessed by the OpenHarmony side.

[0066] Create a Fence object: Call the OpenGL ES GL_ARB_sync extension interface glCreateSyncARB, specify the synchronization type as GL_SYNC_GPU_COMMANDS_COMPLETE, and create a Fence object strongly associated with the current GPU command queue; if the device does not support GL_ARB_sync, then use the vendor-specific interface, such as Adreno's adreno_fence_create or Mali's mali_fence_get_id.

[0067] Bind Fence to GPU Queue: Add the bindFence method in gpu_queue.cpp to write the FenceID to the end of the GPU command queue as a queue completion marker, ensuring that the GPU automatically marks the Fence as complete after executing all commands in the queue; at the same time, record the association between the Fence and the physical address of the buffer and store it in the _fenceMap hash table for easy subsequent status query;

[0068] Submit GPU command queue: Call the submit method in gpu_queue.cpp to submit the GPU command queue bound to the Fence to the hardware for execution. This operation is non-blocking, and the CPU can return without waiting to continue processing the next frame task or other logic.

[0069] Record Fence to shared memory: Call the interface of the cross-domain communication module to write metadata such as FenceID, buffer physical address, and GPU command queue ID to shared memory for OpenHarmony to read.

[0070] Here is an example of a core code snippet:

[0071] / / Modified Android source code

[0072] EGLBoolean eglSwapBuffers(EGLDisplay dpy, EGLSurface surf) {

[0073] / / 1. Remove the original glFinish blocking call

[0074] / / glFinish();

[0075] / / 2. Obtain the EGL context, GPU command queue, and physical address of the buffer.

[0076] EGLContext ctx = eglGetCurrentContext();

[0077] GLContext* glCtx = reinterpret_cast<GLContext*> (ctx);

[0078] GpuQueue* gpuQueue = glCtx->getGpuQueue(); / / Add a new interface for obtaining the GPU queue

[0079] EGLint bufPhysAddr = 0;

[0080] eglQuerySurface(dpy, surf, EGL_PHYSICAL_ADDRESS_EXT, &bufPhysAddr);

[0081] / / 3. Create a Fence object (adapting to GL_ARB_sync and vendor interfaces)

[0082] GLsync fence = glCreateSyncARB(GL_SYNC_GPU_COMMANDS_COMPLETE, 0);

[0083] if (fence == 0) return EGL_FALSE;

[0084] / / 4. Bind the fence to the GPU command queue and submit.

[0085] gpuQueue->bindFence(fence, bufPhysAddr);

[0086] gpuQueue->submit();

[0087] / / 5. Record Fence metadata to cross-domain shared memory

[0088] ohFenceWriteToSharedMem(fence, bufPhysAddr, gpuQueue->getId());

[0089] return EGL_TRUE;

[0090] }

[0091] S1.2 Adapting to Fence interfaces from different GPU manufacturers. To address the differences in underlying interfaces among mainstream GPU manufacturers, a unified FenceID acquisition interface is encapsulated at the Android HAL layer to ensure compatibility: the unified function ohFenceGetVendorId is defined in hardware / libhardware / include / hardware / oh_fence_adapt.h, which internally calls the corresponding interface based on the GPU manufacturer type.

[0092] Manufacturer adaptation logic:

[0093] Qualcomm Adreno GPU: Calling adreno_fence_create requires including adreno_fence.h, passing in the GPU command queue handle, and obtaining the FenceID;

[0094] ARM Mali GPU: mali_fence_get_id requires including mali_fence.h to extract the ID from the created Fence object;

[0095] Imagination PowerVR GPU: To use pvrsrv_fence_get_handle, you need to include pvrsrv_fence.h and convert it to the standard FenceID format;

[0096] Compatibility check: When the Android container starts, eglGetString(EGL_EXTENSIONS) is used to detect the extension types supported by the device and the corresponding Fence creation interface is automatically selected to ensure coverage of more than 99% of Android devices.

[0097] S2, Hard real-time cross-domain communication based on physically contiguous shared memory and Futex

[0098] A cross-domain communication scheme using physically contiguous shared memory and Futex signals replaces traditional IPC or polling mechanisms, achieving low-latency and highly reliable transmission of Fence metadata. Contiguous memory is created through OpenHarmony's physical memory allocation interface, ensuring direct access by both the GPU and CPU. This is combined with the Linux Futex mechanism to achieve polling-free signal synchronization, while structured metadata and CRC32 checksums ensure data reliability.

[0099] S2.1 Creation and Cross-Domain Mapping of Physically Contiguous Shared Memory. The creation and mapping of physically contiguous shared memory requires initialization on the OpenHarmony side and mapping on the Android side to ensure that both sides access the same physical memory region. The specific process is as follows:

[0100] S2.1.1 OpenHarmony-side memory creation. Call the OpenHarmony's ohos_shm_create interface, specifying the memory name such as oh_fence_shm. The size can be calculated as follows: Size = Maximum concurrent frames × Single frame metadata size + futex signal bits. For example, when there are 8 concurrent frames and the single frame metadata is 32 bytes, the size is 8 × 32 + 4 = 260 bytes. Set the memory type to OHOS_SHM_TYPE_PHYS_CONTIG and the read / write permission to OHOS_SHM_PERM_RW.

[0101] Call the mmap interface to map the created shared memory to the OpenHarmony user-space address space and obtain the mapped virtual address;

[0102] Initialize the shared memory structure: Divide the memory into a metadata area to store 8 Fence metadata nodes and Futex signal bits, including 4 bytes for signal synchronization, and initialize the Futex signal bits to 0, indicating that there is no data to be processed;

[0103] Record shared memory information, including file descriptors, mapping addresses, number of frames, metadata size, and futex signal bit addresses, into a global management structure for subsequent read and write operations.

[0104] S2.1.2 Android container-side memory mapping: On the OpenHarmony side, the file descriptor (FD) of shared memory is passed to the process ID (PID) of the Android container through the ohos_shm_share_to_pid interface. After receiving the FD, the Android side calls the mmap interface to map the shared memory to its own user-space address space, usually specifying a fixed virtual address such as 0x7f000000 for easy management. The shared memory structure is parsed, the address range of the metadata area and Futex signal bits is identified, and the local shared memory management structure is initialized and aligned with the OpenHarmony side structure.

[0105] S2.2 Structured Design and CRC32 Verification of Fence Metadata. To ensure the reliability of cross-domain transmission, a fixed-format Fence metadata structure is defined, including ID, buffer address, status, and verification fields. The specific design and verification process are as follows:

[0106] Meta information structure definition: The OhFenceMeta structure is defined uniformly and contains 5 fields:

[0107] fenceId (32-bit): A unique identifier for the fence, assigned by the GPU manufacturer; bufPhysAddr (64-bit): The physical address of the rendering buffer, directly accessible from the OpenHarmony side; gpuQueueId (32-bit): The GPU command queue ID, associated with a specific GPU core; status (8-bit): Fence status, including 0 indicating pending trigger, 1 indicating completion, and 2 indicating timeout; crc32 (32-bit): CRC32 checksum, used to prevent data tampering.

[0108] CRC32 check implementation: When writing metadata on the Android side, the combined CRC32 value of all fields except CRC32 is calculated using the standard polynomial 0xEDB88320, and the check value is written to the CRC32 field. After reading the metadata on the OpenHarmony side, the same CRC32 calculation is repeated, and the result is compared with the CRC32 field value: if they match, the metadata is parsed; if they do not match, they are marked as invalid nodes and skipped to avoid incorrect data causing screen display failure.

[0109] S2.3 Lightweight signal synchronization based on Futex: The Linux Futex mechanism is used to proactively notify the OpenHarmony side after Android sidewriting is completed, avoiding CPU idle time. The specific process is as follows:

[0110] S2.3.1 Signaling after Android side write is complete: The Android side uses the ohFenceWriteToSharedMem function, employing a circular queue mechanism, to locate the currently free metadata node in shared memory using the writeIdx pointer; it populates the fenceId, bufPhysAddr, gpuQueueId, and status fields of the metadata structure, calculates and writes the CRC32 checksum value; it updates the writeIdx pointer, incrementing it cyclically and taking the modulo of the maximum number of frames to ensure it does not go out of bounds; it calls the futex_wake function, passing in the futex signal bit address and the number of wake-up threads, and notifies the OpenHarmony side that there is new metadata to be processed.

[0111] On the OpenHarmony side, a low-priority thread is started with a priority set to SCHED_IDLE to avoid consuming core computing power. This thread loops through the waiting and processing logic for signals. The `futex_wait` function is called, passing in the address of the futex signal bit and the waiting value (0). If the signal bit is 0, the thread enters a blocked state and does not occupy the CPU; if the signal bit is not 0, it immediately returns and processes the metadata. The `readIdx` pointer is used to locate the currently pending metadata node in shared memory and performs a CRC32 check. If the check passes, the metadata is passed, and the `readIdx` pointer is updated; if the check fails, the `readIdx` pointer is updated directly, skipping invalid nodes. After processing, there is no need to reset the futex signal bit; `futex_wait` automatically detects signal bit changes, and `futex_wake` will be triggered again on the next write operation.

[0112] The core code snippet retains only the key logical framework as follows:

[0113] / / OpenHarmony-side shared memory waiting and processing threads

[0114] void* ohFenceShmThread(void* arg) {

[0115] while (1) {

[0116] / / 1. Waiting for the futex signal (blocking if there is no signal)

[0117] futex_wait(g_ohFenceShm.futex, 0);

[0118] / / 2. Read and verify Fence metadata

[0119] OhFenceMeta* meta = &((OhFenceMeta*)g_ohFenceShm.addr)[g_ohFenceShm.readIdx];

[0120] if (ohFenceCalculateCrc(meta) != meta->crc32) {

[0121] g_ohFenceShm.readIdx = (g_ohFenceShm.readIdx + 1) % g_ohFenceShm.frameCount;

[0122] continue?

[0123] }

[0124] / / 3. Transmit metadata

[0125] ohFenceAddToManager(meta);

[0126] g_ohFenceShm.readIdx = (g_ohFenceShm.readIdx + 1) % g_ohFenceShm.frameCount;

[0127] }

[0128] return NULL;

[0129] }

[0130] S3, Fence hard real-time management based on GPU core groups.

[0131] Fence objects are managed using a hash table grouped by GPU cores, and timeout handling is implemented using OpenHarmony high-precision timers to ensure fast fence lookup, status updates, and timely resource reclamation. The core is a grouping strategy tailored to the GPU's multi-core architecture, reducing lookup conflicts and improving management efficiency, while fixed timeouts and degradation strategies ensure display continuity.

[0132] S3.1, Hash table based on GPU core groups. In a multi-core GPU architecture, each core's command queue is executed independently. Managing fences based on core groups can significantly improve lookup efficiency.

[0133] S3.1.1 Grouping criteria and hash table initialization.

[0134] At startup, the number of GPU cores is obtained through OpenHarmony's GPUHAL interface, such as ohos_gpu_get_core_count, for example, Adreno 650 has 8 cores and Mali G52 has 2 cores. A hash table array equal to the number of cores is created. For example, an 8-core GPU creates 8 hash tables, with each hash table corresponding to one GPU core. The hash tables use chaining to resolve collisions, and the size of the linked list nodes is fixed at 64 bytes to ensure cache line alignment. Each linked list node OhFenceNode in the hash table contains Fence core information: Fence identifier fenceId, buffer physical address bufPhysAddr, GPU command queue ID gpuQueueId, Fence status status, timeout Ms, timer ID timerId(), and next node pointer next.

[0135] S3.1.2 Group Index Calculation: The group index is determined based on the lower N bits of the GPU command queue ID (gpuQueueId). For example, for an 8-core GPU, the lower 3 bits are used, i.e., gpuQueueId & 0x07, with an index range of 07; for a 2-core GPU, the lower 1 bit is used, i.e., gpuQueueId & 0x01, with an index range of 01. This ensures that the fences corresponding to the command queues of the same GPU core all fall into the same hash table, reducing cross-core lookups and improving efficiency.

[0136] S3.2, Fence addition and search process.

[0137] S3.2.1 Fence Addition Process: After receiving the Fence metadata from S2, calculate the group index based on gpuQueueId to locate the corresponding hash table; call the ohos_malloc_aligned interface to create an OhFenceNode node aligned to 64 bytes, filling in the node's fenceId, bufPhysAddr, gpuQueueId, and status fields, where the status is initially 0 to indicate pending triggering; calculate the timeout: current system time (milliseconds) + 100ms (fixed timeout to avoid uncertainty in intelligent prediction), and write it to the timeoutMs field; insert the node at the head of the hash table's linked list and update the hash table's head pointer; call the OpenHarmony high-precision timer interface ohos_timer_create to register a one-time timer that triggers after 100ms, the timer callback function is associated with fenceId and gpuQueueId, passed through parameter packaging, and the timer ID is written to the node's timerId field for subsequent timeout cancellation.

[0138] S3.2.2 Fence Lookup Process: After the GPU triggers the Fence completion signal, it calculates the group index based on the fenceId and gpuQueueId to locate the corresponding hash table; it traverses the linked lists of the hash table, comparing the fenceId and gpuQueueId of each node: if a complete match is found, the node is returned; if no match is found after traversing to the end of the linked list, NULL is returned, indicating that the fence has timed out or has been reclaimed. Because the nodes are grouped according to GPU cores, the average length of each linked list is no more than 2, the collision rate is less than 0.1%, and the time taken for a single lookup is no more than 20ns, which is much faster than a traditional single hash table.

[0139] S3.3, Hard real-time timeout handling and resource reclamation.

[0140] A fixed 100ms timeout and a clear resource reclamation process are used to ensure a controllable fence lifecycle and avoid memory leaks and display anomalies. The specific process is as follows:

[0141] S3.3.1 Timeout Callback Handling: After the timer is triggered, the callback function parses the parameters to extract fenceId and gpuQueueId, and finds the corresponding Fence node. If the node status is still pending triggering (status is 0), the status is updated to timeout (status is set to 2), and a degradation strategy is executed: the address of the previous frame buffer g_ohFenceBackupBuf in the global backup is read, and the on-screen callback function is called to ensure that the display does not go black. If the node has been triggered (status is 1) or has been recycled, it returns directly without performing any additional operations.

[0142] S3.3.2 Resource reclamation process: After the Fence triggers and times out, the node is removed from the hash table and resources are released: The node to be reclaimed is located, the linked list is traversed to find the predecessor node, the predecessor node's next pointer is updated, the current node is skipped, and the linked list is removed; if the node is the head node of the linked list, the head pointer of the hash table is updated to the node's next pointer; the ohos_timer_destroy interface is called to destroy the timer associated with the node to prevent the timer from triggering repeatedly; the ohos_free_aligned interface is called to release the node's cache line aligned memory; resource reclamation is strongly bound to the Fence lifecycle to ensure no memory leaks and that memory usage is never greater than 10KB.

[0143] The core code snippet retains only the grouped hash table structure and core interfaces:

[0144] / / OpenHarmony side Fence group hash table structure and addition logic

[0145] typedef struct {

[0146] uint32_t fenceId; / / FenceID

[0147] uint64_t bufPhysAddr; / / Physical address of the buffer

[0148] uint32_t gpuQueueId; / / GPU command queue ID

[0149] uint8_t status; / / Status: 0 = Pending, 1 = Completed, 2 = Timeout

[0150] uint64_t timeoutMs; / / Timeout duration (ms)

[0151] uint32_t timerId; / / Timer ID

[0152] struct OhFenceNode* next; / / Pointer to the next node

[0153] OhFenceNode;

[0154] / / Grouped hash table array (initialized according to the number of GPU cores)

[0155] OhFenceNode* g_fenceHashTable[GPU_CORE_COUNT] = {NULL};

[0156] / / Add Fence to the group hash table

[0157] int ohFenceAddToManager(const OhFenceMeta* meta) {

[0158] / / 1. Calculate the grouping index (using the lower 3 bits for an 8-core GPU)

[0159] uint32_t coreIdx = meta->gpuQueueId & 0x07;

[0160] / / 2. Create cache line alignment node

[0161] OhFenceNode* node = (OhFenceNode*)ohos_malloc_aligned(sizeof(OhFenceNode), 64);

[0162] if (node ​​== NULL) return -1;

[0163] / / 3. Populate node information

[0164] node->fenceId = meta->fenceId;

[0165] node->bufPhysAddr = meta->bufPhysAddr;

[0166] node->gpuQueueId = meta->gpuQueueId;

[0167] node->status = meta->status;

[0168] node->timeoutMs = ohos_get_current_ms() + 100;

[0169] / / 4. Insert at the head of the hash table

[0170] node->next = g_fenceHashTable[coreIdx];

[0171] g_fenceHashTable[coreIdx] = node;

[0172] / / 5. Register timeout timer

[0173] ohos_timer_attr_t attr = {.type=OHOS_TIMER_TYPE_ONE_SHOT, .cb=ohFenceTimeoutCb};

[0174] attr.arg = (void*)((uint64_t)node->fenceId | ((uint64_t)node->gpuQueueId << 32));

[0175] node->timerId = ohos_timer_create(&attr, 100);

[0176] return 0;

[0177] }

[0178] S4, Fence hard real-time triggering based on GPU hardware interrupt.

[0179] By modifying the GPU driver or hardware abstraction layer, the Fence can be triggered directly after the GPU completes the command queue, replacing the traditional polling mechanism. The core is to use the GPU's hardware interrupt signal in scenarios with driver source code, or to use shared memory flags and semaphores in scenarios without driver source code, to ensure that the Fence triggering delay is no more than 5μs, while adapting to different driver accessibility scenarios.

[0180] S4.1. For scenarios with GPU driver source code, interrupt callback injection is implemented. For GPUs officially supported by OpenHarmony, such as Mali G52 and Adreno 618, the interrupt handler function of the driver is modified to inject a fence to trigger the callback. The specific process is as follows:

[0181] S4.1.1 Interrupt Callback Injection Logic. In the GPU driver's command queue completion interrupt handling function, a Fence trigger callback interface is added. After the interrupt is triggered, the interrupt register value is parsed, and the currently completed GPU command queue ID (gpuQueueId) is extracted. Internal driver interfaces, such as Mali's `mali_queue_read_fence` and Adreno's `adreno_queue_get_fence`, are called to read the fenceId bound to the end of the command queue. The user-space visible interface `ohFenceTrigger` on the OpenHarmony side is called, passing in the fenceId and gpuQueueId, to trigger the fence state update and on-screen callback.

[0182] The process is completed entirely within a hardware interrupt context, without CPU polling, and the trigger latency is no more than 2μs.

[0183] S4.1.2 Interrupt Priority Configuration. Set the priority of the GPU command queue completion interrupt to IRQF_ONESHOT|IRQF_HIGH to ensure that the interrupt is not preempted by other low-priority interrupts, further reducing response latency; In the interrupt registration functions of the driver, such as mali_irq_init and adreno_irq_init, set the priority parameter through the request_irq interface to ensure that the configuration takes effect.

[0184] The core code snippet only retains the driver interrupt callback framework:

[0185] / / OpenHarmony kernel Mali GPU driver interrupt handling (modified)

[0186] static irqreturn_t mali_queue_complete_irq(int irq, void* dev_id) {

[0187] struct mali_queue* queue = (struct mali_queue*)dev_id;

[0188] / / 1. Extract the completed GPU command queue ID and Fence ID

[0189] uint32_t gpuQueueId = queue->id;

[0190] uint32_t fenceId = mali_queue_read_fence(queue); / / Read the fence ID at the end of the queue

[0191] / / 2. Trigger OpenHarmony-side Fence processing

[0192] if (fenceId != 0) {

[0193] ohFenceTrigger(fenceId, gpuQueueId);

[0194] }

[0195] return IRQ_HANDLED;

[0196] }

[0197] S4.2, in scenarios without GPU driver source code, adopts a shared memory flag and semaphore approach.

[0198] If the GPU driver cannot be modified (e.g., a third-party custom SoC), a compatible solution using shared memory flags and semaphores is adopted to avoid polling. The specific process is as follows:

[0199] S4.2.1 Android-side flag update logic. In the Android GPUHAL layer, such as hardware / libhardware / modules / gpu / gpu_module.cpp, a new command queue completion callback function is added: when the GPU completes a command queue, the HAL layer automatically calls this callback; in the callback function, the corresponding Fence metadata node in shared memory is found based on fenceId and gpuQueueId, and the node's status field is updated to 1 to indicate completion; since the status field has changed, the CRC32 checksum of the node is recalculated, and the crc32 field is updated; the sem_post function is called to increment the count of the global semaphore g_ohFenceSem by 1, notifying the OpenHarmony side that a fence has been completed.

[0200] S4.2.2, OpenHarmony-side semaphore wait logic. Start a low-priority thread with priority set to SCHED_IDLE. Within the thread, call the sem_wait function in a loop: if the semaphore count is 0, the thread blocks and does not occupy CPU; if the count is > 0, acquire the semaphore, decrement the count by 1, and continue execution.

[0201] After the semaphore is triggered, all Fence metadata nodes in the shared memory are traversed, and the status field is checked: if it is 1, it means the process is complete and the CRC32 check passes. The corresponding Fence node is found, and the node status is updated to 1, indicating that the process is complete. The on-screen callback function is then called. After processing is complete, the status field of the shared memory node is reset to 0, indicating that it is idle and can be reused later.

[0202] The trigger latency of this solution is no more than 3μs, and the CPU utilization rate is reduced from 8% to 1.2%. Although it is slightly higher than the interrupt solution, it does not require driver modification and has stronger compatibility.

[0203] S4.3 Unified callback execution after Fence triggering. For both triggering methods described above, a unified callback logic is executed after the Fence is triggered, connecting the Fence management system with the on-screen display system. The specific process is as follows:

[0204] State Update: Locate the corresponding Fence node and update its state from pending to completed; Timer Cancellation: Call the ohos_timer_destroy interface to destroy the timeout timer associated with the node, preventing repeated triggering of timeout logic; Buffer Backup: Update the physical address of the buffer associated with the current Fence, bufPhysAddr, to the global backup variable g_ohFenceBackupBuf for subsequent timeout degradation; Screen Submission Callback: Call the ohGraphicSubmitBuf interface of the OpenHarmony graphics screen submission module, passing in the physical address of the buffer, to trigger asynchronous screen submission; Resource Reclamation: Call the resource reclamation function to remove the node from the hash table and release memory.

[0205] S5. Hard real-time on-screen display based on VSync signal locking. This achieves precise synchronization between the Fence trigger and the VSync signal of the OpenHarmony display device, ensuring that the on-screen timing perfectly matches the display sequence and avoiding screen tearing. Simultaneously, static priority scheduling based on buffer type ensures smoothness in high frame rate scenes.

[0206] S5.1, VSync signal capture and timestamp acquisition. The VSync signal is the vertical synchronization signal of the display device, and its period is equal to the reciprocal of the screen refresh rate, such as 8.333ms for 120fps and 16.666ms for 60fps. Capturing the signal and timestamp is the basis for timing alignment. The specific process is as follows:

[0207] VSync Period Acquisition: When OpenHarmony starts, it calls the ohos_display_get_VSync_period interface in milliseconds to obtain the current display device's VSync period, converts it to microseconds (e.g., 8.333ms = 8333μs), and stores it in the global variable g_VSyncPeriod. If the device supports multiple refresh rate switching, such as 60fps / 120fps, it listens for refresh rate change events and dynamically updates g_VSyncPeriod.

[0208] VSync timestamp capture. The ohos_display_register_VSync_callback interface is called to register a VSync signal callback function: each time a VSync signal arrives, the callback function receives the timestamp of the current signal in microseconds, starting from system startup, and updates it to the global variable g_VSyncTimestamp; the execution time of the callback function is controlled within 1μs to avoid blocking the signal processing flow of the display device.

[0209] S5.2 Hard real-time calculation and waiting for screen display timing. The screen display window is calculated based on the VSync timestamp, such as 2ms before the VSync signal arrives to 1ms after its arrival. This ensures the buffer is committed within the window to avoid screen tearing. The specific process is as follows:

[0210] Calculation of the top screen window:

[0211] The start time of the on-screen window = g_VSyncTimestamp-2000, which is 2ms before the VSync signal arrives, and a buffer is reserved for synthesis time.

[0212] The end time of the on-screen window = g_VSyncTimestamp + 1000, that is, 1ms after the VSync signal arrives, to ensure that the on-screen display is completed within the current scan cycle;

[0213] The window length is 3ms, which is sufficient to complete the entire process from buffer synthesis to submission to the display device.

[0214] Display timing and waiting: After the Fence is triggered, the ohos_get_current_time_us interface is called to obtain the current system time, in microseconds. If the current time is less than the start time of the display window: the waiting time is calculated as the difference between the start time and the current time, and the usleep function is called to perform precise sleep. After the sleep ends, the current time will just enter the display window. If the current time is greater than the end time of the display window: it means that the current VSync cycle has been missed. The waiting time is calculated as the sum of the VSync timestamp of the next cycle and the cycle, minus the current time. The system sleeps until the start time of the display window of the next cycle. If the current time is within the display window, no waiting is required, and subsequent synthesis and submission operations are executed directly.

[0215] S5.3, Static scheduling with multiple buffer priorities.

[0216] Buffer Composition and Submission: After extracting the target buffer, the ohGraphicCompose interface is called to perform multi-layer composition, such as overlaying the Android rendering layer with the OpenHarmony status bar and navigation bar layers; after composition is completed, the ohos_display_submit_buffer interface is called to submit the composed buffer to the display device to complete the on-screen display.

[0217] The core code snippet includes the VSync callback and on-screen function framework:

[0218] / / OpenHarmony-side VSync callback and on-screen timing control

[0219] uint64_t g_VSyncTimestamp = 0; / / VSync timestamp (μs)

[0220] size_t g_VSyncPeriod = 0; / / VSync period (μs)

[0221] void ohGraphicVSyncCallback(uint64_t timestamp) {

[0222] g_VSyncTimestamp = timestamp; / / Update VSync timestamp

[0223] }

[0224] void ohGraphicSubmitBuf(uint64_t bufPhysAddr) {

[0225] / / 1. Get the current time and calculate the display window

[0226] uint64_t currTime = ohos_get_current_time_us();

[0227] uint64_t windowStart = g_VSyncTimestamp - 2000;

[0228] uint64_t windowEnd = g_VSyncTimestamp + 1000;

[0229] / / 2. Waiting to enter the top screen window

[0230] if (currTime < windowStart) {

[0231] usleep(windowStart-currTime);

[0232] } else if (currTime > windowEnd) {

[0233] uint64_t nextVSync = g_VSyncTimestamp + g_VSyncPeriod;

[0234] usleep(nextVSync - currTime);

[0235] windowStart = nextVSync - 2000;

[0236] }

[0237] / / 3. Composition and Commit Buffer

[0238] uint64_t composedBuf = ohGraphicCompose(bufPhysAddr);

[0239] ohos_display_submit_buffer(composedBuf);

[0240] }

[0241] Experiments have verified that this invention replaces glFinish blocking by deeply binding the Fence to the GPU command queue, enabling parallel execution of the CPU and GPU. Android container graphics rendering frame rates consistently exceed 120fps, with game scene frame rate fluctuations not exceeding 1fps, thus improving real-time frame rate performance. Employing a communication scheme using physically continuous shared memory and Futex signals, single synchronization time is no more than 5μs, CPU utilization is no more than 1.2%, and CRC32 verification reduces the cross-domain data error rate from 0.5% to 0.01%, ensuring efficient and reliable cross-domain synchronization. It supports both scenarios with and without GPU driver source code, adapting to all OpenHarmony devices, including ARMv7-A and ARMv8-A, and 99% of Android GPUs such as Adreno, Mali, and PowerVR, without requiring modification to the OpenHarmony kernel, demonstrating strong hardware-level compatibility. VSync signal locking keeps screen tearing no more than 0.1%, and static priority scheduling ensures smoothness in high frame rate scenes, achieving display quality comparable to native OpenHarmony applications, resulting in significant display quality optimization.

[0242] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An Android container rendering optimization method based on cross-domain hard real-time fence synchronization, characterized in that, Specifically, the following steps are included: Using HarmonyOS as the host and Android as the container; When the host starts, it creates a hash table corresponding to each GPU core, containing metadata, timeout time and timer ID, and the index is bound to a singly linked list. It starts a low-priority synchronization thread; it creates a physically contiguous first shared memory, divides it into a metadata area and a synchronization signal bit, maps it to user space to obtain virtual addresses, and records the file descriptors (FDs). Obtain the VSync cycle and register the callback, pass the first shared memory FD to the container; after receiving the FD, the container maps the memory and initializes and aligns the second shared memory with it, and registers the GPU command queue to complete the callback. After the Android application starts, the container obtains the physical address of the active GPU command queue and rendering buffer, creates a Fence and binds it to the end of the queue, generates metadata containing FenceID, physical address, command queue ID, status and check value, with the status being pending trigger, stores it in the second shared memory and then notifies the host through Futex to submit the GPU command queue to the hardware. The host synchronization thread receives the signal, creates a metadata node and stores it in the corresponding hash table, and registers a one-time timer; After the GPU completes the command queue, the container callback updates the Fence status to complete and the verification value, and notifies the host through Futex; after the host synchronous thread verifies the validity, it cancels the original timer, updates the hash table node and re-registers the timer, backs up the physical address to trigger asynchronous screen display, and sets the Fence status to idle. The host obtains the current system time and VSYNC timestamp, calculates the display window, performs a sleep based on the display window, and then performs buffer compositing and display device submission to complete the display.

2. The Android container rendering optimization method according to claim 1, characterized in that, The hash table index is bound to a singly linked list. When there is no conflict, the metadata node is saved to the position corresponding to the index. When there is a conflict, the original metadata node is not replaced, and the new metadata node is added to the head of the linked list corresponding to the index.

3. The Android container rendering optimization method according to claim 1, characterized in that, The method by which the host synchronization thread receives signals, creates metadata nodes, and stores them in the corresponding hash table is as follows: If the semaphore count is 0, the thread is blocked; otherwise, the semaphore in the synchronization signal is acquired, and the semaphore count is decremented by 1. Create a metadata node, read the metadata and fill the corresponding fields of the metadata node with the FenceID, physical address, command queue ID, status and checksum; calculate the GPU group index based on the command queue ID in the metadata and determine the corresponding first hash table; calculate the current timeout of the host system, register a one-time timer to be triggered after the set time, associate the timer callback function with the FenceID and command queue ID, write the metadata node, timeout and timer ID to the head of the singly linked list of the first hash table, and update the head pointer of the first hash table.

4. The Android container rendering optimization method according to claim 3, characterized in that, The method for calculating the GPU group index based on the command queue ID in the metadata is as follows: the lower N bits to be taken are calculated using log2 (number of GPU cores), and 2^N-1 is used as the mask value. The command queue ID and the mask value are then bitwise ANDed to obtain the GPU group index.

5. The Android container rendering optimization method according to claim 1, characterized in that, The verification value is a combined CRC32 value of FenceID, physical address, command queue ID, and status.

6. The Android container rendering optimization method according to claim 1, characterized in that, After the timer is triggered, the host parses the parameters to extract the FenceID and command queue ID, and looks up the corresponding metadata node. If the status of the metadata node is pending triggering, the status is updated to timeout, and the physical address corresponding to the previous frame of data is displayed on the screen. If the status is triggered, it returns directly without performing any additional operations. When the Fence trigger completes or times out, the corresponding metadata node is removed from the first hash table, the timer associated with the node is destroyed, and the memory of the cache line aligned node is released.

7. The Android container rendering optimization method according to claim 1, characterized in that, The method for calculating the on-screen window is as follows: the host obtains the current system time and VSync timestamp, and calculates the start time and end time of the on-screen window based on the VSync timestamp and the set preceding and following times. That is, the start time of the on-screen window is the difference between the VSync timestamp and the preceding time, and the end time of the on-screen window is the sum of the VSync timestamp and the following time.

8. The Android container rendering optimization method according to claim 7, characterized in that, The method for executing sleep based on the on-screen window is as follows: if the current system time is less than the start time of the on-screen window, the difference between the start time of the on-screen window and the current time is used as the waiting time, and sleep is executed; if the current system time is greater than the end time of the on-screen window, the difference between the sum of the start time of the on-screen window and the VSync cycle and the current time is used as the waiting time, and sleep is executed.

9. The Android container rendering optimization method according to claim 1, characterized in that, When a container writes to the second shared memory, the buffer type is marked by the high 4 bits of the physical address, including three categories: highest priority, medium priority, and lowest priority. The host maintains composition queues with corresponding priorities, including the highest priority queue, the medium priority queue, and the lowest priority queue. When the frame is displayed, the queues are checked in descending order of priority, and high-priority frames are processed first. If there is a queue whose length exceeds the threshold, the earliest frame is discarded.

10. The Android container rendering optimization method according to claim 1, characterized in that, In the GPU driver source code, a Fence trigger callback is added to the command queue completion interrupt handling function of the GPU driver. First, the interrupt register is parsed to extract the command queue ID, then the FenceID at the end of the queue is read through the driver's internal interface, and finally the ohFenceTrigger function is called to trigger the status update and screen upload callback. The interrupt priority is set to high priority and one-time trigger, and the priority parameter is set in the driver interrupt registration function.

Images