Gpu synchronization accelerated image optimization method and system for graphics display engine

Abstract

The application discloses a GPU synchronization acceleration image optimization method and system for a graphics display engine, relates to the technical field of image processing, and comprises the following steps: identifying the time synchronization relationship between a graphics display engine and a GPU, and determining an idle waiting interval; embedding a hardware semaphore in a command queue of the GPU; when a micro event in a frame occurs, performing a signal operation of the hardware semaphore by GPU hardware to release an occupied buffer or send a state change notification to a CPU; and based on the state change notification, dynamically adjusting the submission time of a rendering task according to the length of the idle waiting interval and the historical rendering time consumption of the GPU. The application solves the technical problems of coarse synchronization granularity between a CPU and a GPU, low utilization rate of an idle waiting interval, and rendering pipeline stagnation or underload in the prior art, and achieves the technical effects of improving the time overlap degree of CPU and GPU task execution and the overall graphics rendering throughput.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a GPU synchronous acceleration image optimization method and system for graphics display engines. Background Technology

[0002] Graphics display engines such as desktop window managers, game engines, or real-time rendering frameworks typically operate in an asynchronous pipeline mode with the graphics processing unit (GPU). The CPU is responsible for preparing rendering commands and resources, while the GPU is responsible for executing the actual graphics rendering operations. Traditional synchronization mechanisms based on binary semaphores employ a "global wait" strategy. When the CPU needs to access resources that the GPU has not yet finished rendering, the driver layer forces the CPU into a blocked state until the GPU completes all pending tasks, resulting in a large number of idle waiting intervals—at which point the CPU is idling while the GPU is still working, or the GPU is underloaded because it is waiting for the CPU to submit the next frame of data. In scenarios with stringent requirements for latency and frame rate stability, such as high refresh rate displays, real-time interaction, or virtual reality, traditional synchronization methods frequently trigger intra-frame micro-events such as memory page table updates, rendering channel switching, and buffer releases. Each event may trigger coarse-grained synchronization operations, disrupting pipeline parallelism. There is an inherent discrepancy between the CPU and GPU's perception of the completion time of the "current frame." The CPU usually submits tasks based on historical statistics or fixed offsets, which is difficult to adapt to the dynamic fluctuations in rendering load, leading to redundant idle waiting or buffer contention or page faults.

[0003] Therefore, current technologies suffer from coarse synchronization granularity between the CPU and GPU, low utilization of idle waiting intervals, and technical problems that lead to rendering pipeline stalls or underload. Summary of the Invention

[0004] This application provides a GPU synchronous acceleration image optimization method and system for graphics display engines, which solves the technical problems in the prior art that the synchronization granularity between CPU and GPU is coarse and the utilization rate of idle waiting intervals is low, resulting in rendering pipeline stagnation or underload. It achieves the technical effect of improving the time overlap of CPU and GPU task execution and the overall graphics rendering throughput.

[0005] This application provides a GPU synchronous acceleration image optimization method for a graphics display engine. The method includes: identifying the time synchronization relationship between the graphics display engine and the GPU, and determining an idle waiting interval; based on the idle waiting interval, embedding a hardware semaphore in the GPU's command queue, wherein the trigger condition of the hardware semaphore is an intra-frame micro-event of the graphics display engine, the intra-frame micro-event including completion of video memory page table update, end of rendering channel, or buffer release; when the intra-frame micro-event occurs, the GPU hardware executes the signal operation of the hardware semaphore to release the occupied buffer or send a state change notification to the CPU; based on the state change notification, the CPU dynamically adjusts the submission time of the rendering task according to the length of the idle waiting interval and the historical rendering time of the GPU, so that the operation of the CPU for the next frame of image data overlaps at least partially with the operation of the GPU for rendering the current frame of image data.

[0006] In a possible implementation, identifying the time synchronization relationship between the graphics display engine and the GPU and determining the idle waiting interval includes: obtaining the vertical synchronization period and phase of the graphics display engine and establishing a first time axis; obtaining the event timestamp of the GPU rendering completion and establishing a second time axis; mapping the rendering completion time on the second time axis onto the first time axis to obtain the offset relative to the start time of the next vertical synchronization signal; and determining the positive time period corresponding to the offset as the idle waiting interval.

[0007] In a possible implementation, the hardware semaphore is a counting semaphore used to manage the idle state of at least three buffers, wherein each buffer has three states: idle, rendering, and displaying, and each buffer is associated with a state machine, and the initial value of the counting semaphore is the total number of buffers.

[0008] In a possible implementation, the signal operation includes: when a frame of image is displayed and the corresponding buffer is released, the display engine hardware in the GPU automatically performs an increment operation on the counting semaphore to increment the free buffer count by 1; when the CPU needs to acquire a free buffer, it performs a decrement operation on the counting semaphore; if the decrement is successful, the buffer is acquired; if the decrement fails, the CPU enters a light sleep or performs other non-rendering tasks until the next semaphore increment occurs.

[0009] In a possible implementation, the submission time of the rendering task is dynamically adjusted based on the length of the idle waiting interval and the historical rendering time of the GPU, including: obtaining the GPU rendering time and the length of the idle waiting interval for the most recent N frames based on recorded data of a preset time period; predicting the rendering time of the current frame using exponential smoothing based on the GPU rendering time of the most recent N frames; comparing the rendering time of the current frame with the length of the idle waiting interval, and adjusting the submission time of the rendering task according to the time comparison relationship.

[0010] In a possible implementation, the submission time of the rendering task is adjusted according to the time comparison relationship, including: when the length of the idle waiting interval is greater than or equal to the predicted rendering time of the current frame, it is determined that the current frame can be rendered before the next vertical synchronization signal arrives, and the rendering instruction of the next frame is inserted into the GPU command queue in advance before the current frame is rendered, so that the operation of the CPU preparing the image data of the next frame and the operation of the GPU rendering the image data of the current frame overlap at least partially in time; when the length of the idle waiting interval is less than the predicted rendering time of the current frame, the rendering instruction of the next frame is not inserted in advance, and the rendering command of the next frame is submitted after the next vertical synchronization signal arrives, wherein the operations of the CPU and the GPU are executed serially.

[0011] In a possible implementation, the state machine transition rules for each buffer include: transitioning from idle to rendering, where the CPU, after successfully acquiring the semaphore, marks the buffer as rendering and submits a rendering command to the GPU; transitioning from rendering to display, where the GPU completes rendering and waits for the next vertical synchronization signal, automatically switching the buffer to display when the vertical synchronization signal arrives, and starting to scan for output; and transitioning from display to idle, where, after the display engine completes the output of all pixels in the corresponding buffer, the display engine hardware automatically sets the buffer state to idle and performs an increment operation on the counting semaphore; wherein the state transition from display to idle does not depend on any CPU instructions and is entirely completed by hardware timing logic.

[0012] In a possible implementation, the GPU synchronous acceleration image optimization method for the graphics display engine further includes: acquiring frame display anomaly events of the graphics display engine; tracing and locating the source of image processing parameters based on the frame display anomaly events, generating anomaly feedback information and sending it to the GPU; and dynamically adjusting response parameters based on the anomaly feedback information.

[0013] In possible implementations, the frame display anomaly events include: event rhythm anomalies and image generation anomalies. The event rhythm anomalies include screen tearing during buffer swapping, frames not being ready before the vertical synchronization signal arrives, multiple consecutive frames displaying the same content, and frame intervals exceeding a preset fluctuation range. The image generation anomalies include blockiness, color deviation, overexposure / underexposure, motion artifacts, or flickering in the displayed image. The response parameters for the event rhythm anomalies include the trigger threshold of the hardware semaphore, the initial value of the counting semaphore, the smoothing factor of the exponential smoothing prediction, or the vertical synchronization mode. The response parameters for the image generation anomalies include rendering resolution, anti-aliasing level, texture filtering quality, shadow detail level, or color correction parameters.

[0014] This application also provides a GPU synchronous acceleration image optimization system for a graphics display engine. The system includes: an idle waiting interval determination module for identifying the time synchronization relationship between the graphics display engine and the GPU and determining an idle waiting interval; a hardware semaphore embedding module for embedding a hardware semaphore in the GPU's command queue based on the idle waiting interval, wherein the trigger condition for the hardware semaphore is an intra-frame micro-event of the graphics display engine, including completion of memory page table update, end of rendering channel, or buffer release; a signal operation execution module for executing the signal operation of the hardware semaphore by the GPU hardware when the intra-frame micro-event occurs, to release the occupied buffer or send a state change notification to the CPU; and a task dynamic adjustment module for dynamically adjusting the submission time of the rendering task based on the state change notification, according to the length of the idle waiting interval and the historical rendering time of the GPU, so that the operation of the CPU for the next frame of image data overlaps at least partially with the operation of the GPU for rendering the current frame of image data.

[0015] This application proposes a GPU-synchronized accelerated image optimization method and system for graphics display engines. The method identifies the time synchronization relationship between the graphics display engine and the GPU, and determines the idle waiting interval. Based on the idle waiting interval, hardware semaphores are embedded in the GPU's command queue. When an intra-frame micro-event occurs, the GPU hardware executes the signal operation of the hardware semaphores to release occupied buffers or send a state change notification to the CPU. Based on the state change notification, the CPU dynamically adjusts the submission time of the rendering task according to the length of the idle waiting interval and the GPU's historical rendering time. This solves the technical problems in existing technologies, such as coarse synchronization granularity between the CPU and GPU and low utilization of the idle waiting interval, leading to rendering pipeline stagnation or underload. It achieves the technical effect of improving the time overlap of CPU and GPU task execution and the overall graphics rendering throughput. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments of this disclosure will be briefly described below. Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0017] Figure 1 This is a schematic diagram of a GPU synchronous acceleration image optimization method for a graphics display engine provided in an embodiment of this application.

[0018] Figure 2 This is a schematic diagram of the GPU synchronous acceleration image optimization system structure for a graphics display engine provided in an embodiment of this application.

[0019] Explanation of reference numerals in the attached diagram: Idle waiting interval determination module 10, hardware semaphore embedding module 20, signal operation execution module 30, and task dynamic adjustment module 40. Detailed Implementation

[0020] To further illustrate the technical means and effects adopted by the present invention in order to achieve the intended purpose, the following detailed description is provided in conjunction with the accompanying drawings and preferred embodiments, based on the specific implementation methods, structures, features and effects of the present invention.

[0021] This application provides a GPU-synchronized accelerated image optimization method for graphics display engines, such as... Figure 1 As shown, the method includes: Step S100: Identify the time synchronization relationship between the graphics display engine and the GPU, and determine the idle waiting interval.

[0022] Step S100 further includes: obtaining the vertical synchronization period and phase of the graphics display engine and establishing a first time axis; obtaining the event timestamp of the GPU rendering completion and establishing a second time axis; mapping the rendering completion time on the second time axis to the first time axis to obtain the offset relative to the start time of the next vertical synchronization signal; and determining the positive time period corresponding to the offset as the idle waiting interval.

[0023] Preferably, the vertical synchronization period and phase of the graphics display engine are obtained. The vertical synchronization period refers to the fixed time interval between two vertical synchronization signals, for example, 16.67ms at a 60Hz refresh rate and 8.33ms at 120Hz. The phase refers to the starting position of the vertical synchronization signal within one period. The rising or falling edge of the vertical synchronization signal is used as the phase zero point. An absolute time coordinate system is established based on the period and phase of the vertical synchronization signal to establish a first time axis. The origin of this time axis is usually aligned with the start time of the vertical synchronization signal and covers one or more vertical synchronization periods. The GPU completes all rendering operations in the current frame, including geometry processing, rasterization, pixel shading, etc. The GPU hardware records event timestamps. An independent time coordinate system is established based on the GPU internal clock or system clock. This time axis records the absolute time point of each rendering event of the GPU. Its zero point is aligned with the system startup time or GPU initialization time. Then, the absolute time of GPU rendering completion recorded on the second time axis is transformed into the coordinate system of the first time axis through time coordinate transformation. That is, the absolute time value of GPU rendering completion is taken, and the absolute time value of the start time of the current vertical synchronization cycle is subtracted. Then, the modulo operation is performed on the vertical synchronization cycle to obtain the offset relative to the start time of the next vertical synchronization signal. This offset represents the time interval between the start point of the current vertical synchronization cycle and the actual completion time of GPU rendering. The value range is 0 to the length of the vertical synchronization cycle. The positive time period corresponding to the offset, that is, the time period from the start time of GPU rendering completion to the end time of the start time of the next vertical synchronization signal, is used as the time synchronization relationship between the graphics display engine and the GPU to determine the idle waiting interval. The length of the positive time period is equal to the length of the vertical synchronization cycle minus the offset. During this period, the GPU has completed the rendering of the current frame but has not yet reached the vertical synchronization signal time, and cannot immediately scan and output to the display device.

[0024] Step S200: Based on the idle waiting interval, a hardware semaphore is embedded in the GPU's command queue. The triggering condition of the hardware semaphore is an intra-frame micro-event of the graphics display engine. The intra-frame micro-event includes completion of video memory page table update, end of rendering channel, or buffer release.

[0025] Step S200 further includes that the hardware semaphore is a counting semaphore used to manage the idle state of at least three buffers, wherein each buffer has three states: idle, rendering, and displaying, and each buffer is associated with a state machine, and the initial value of the counting semaphore is the total number of buffers.

[0026] Preferably, the GPU's command queue is a first-in-first-out queue maintained internally by the GPU, used to store rendering commands submitted by the CPU, such as drawing calls, state transitions, and resource bindings. When constructing the command buffer, the CPU embeds a hardware semaphore at a specific position in the command queue. This semaphore is a synchronization primitive directly supported by the GPU hardware and is managed by the GPU's internal synchronization logic unit. Based on the start point and length of the idle waiting interval, the hardware semaphore is arranged in the command queue after the occurrence of the corresponding intra-frame micro-event, so that the triggering of the semaphore is aligned with the moment when the GPU executes the command. The hardware semaphore is a counting semaphore, supporting increment and decrement operations. The incrementing and decrementing of this counter are directly executed by the GPU hardware logic, with an operation latency of a fixed number of hardware cycles. The hardware semaphore manages the idle state of at least three buffers, each large enough to store a complete frame of image data. The current value of the counting semaphore indicates the number of buffers currently in an idle state, i.e., not occupied by the GPU or display engine. Each buffer has three states: idle, rendering, and displaying. Idle indicates the buffer is not used by any unit, and its content is invalid or expired. Rendering indicates the GPU is writing rendering data to the buffer. Displaying indicates the display engine is reading data from the buffer and outputting it to the screen. Each buffer has an independent state register to record its current state. The state machine defines the state transition paths and the triggering conditions for each transition. Assuming N buffers are allocated (N≥3), the initial value of the counting semaphore is N. The initial value N indicates that all N buffers are in an idle state when the system starts or resets, and the CPU can acquire any buffer for rendering tasks.

[0027] Preferably, the triggering condition for the hardware semaphore is an intra-frame micro-event of the graphics display engine. That is, during the rendering and display of a frame of image, a more granular hardware event occurring below the frame level, and an intra-frame micro-event may occur multiple times within a frame. This may include: completion of the video memory page table update; the GPU's memory management unit completing the virtual address to physical address mapping table modification operation; the end of a rendering channel; the GPU completing all drawing operations for the current rendering channel, such as the color rendering channel and the depth rendering channel; and buffer release; the GPU completing read and write operations on a certain buffer, which can then be reused. When the GPU executes the embedded semaphore command, it does not trigger immediately, but waits for the intra-frame micro-event to actually occur. The GPU hardware monitors the event status, and only when the event occurs does the semaphore change from a waiting state to a ready state.

[0028] Furthermore, the state machine transition rules for each buffer include: transitioning from idle to rendering, where the CPU, after successfully acquiring the semaphore, marks the buffer as rendering and submits a rendering command to the GPU; transitioning from rendering to display, where the GPU completes rendering and waits for the next vertical synchronization signal, automatically switching the buffer to display when the vertical synchronization signal arrives, and starting to scan for output; and transitioning from display to idle, where, after the display engine completes the output of all pixels in the corresponding buffer, the display engine hardware automatically sets the buffer state to idle and performs an increment operation on the counting semaphore; wherein the state transition from display to idle does not depend on any CPU instructions and is entirely completed by hardware timing logic.

[0029] Preferably, if the CPU successfully decrements the counter semaphore, the idle state transitions to rendering mode. After successfully acquiring the semaphore, it obtains the identifier of the free buffer, such as a memory address or index number, and writes it to the GPU's status register via memory-mapped I / O. This changes the buffer's status register from "idle" to "rendering mode," and a rendering command is submitted to the GPU, writing the drawing command to the GPU command queue and binding the buffer as a rendering target. If the GPU completes all rendering operations on the buffer, including geometry processing, rasterization, pixel shading, and memory page table updates, and transitions from rendering mode to display mode, the buffer remains in the "rendering mode" state. The GPU completes all write operations, and the display controller switches on the rising or falling edge of the vertical sync signal, changing the buffer's status register from "rendering mode" to "display mode." The display engine immediately reads pixel data line by line from the starting address of the buffer and outputs it to the display interface. If the display engine completes the output of all pixels in the buffer, it changes the status register of the buffer from "displaying" to "idle" and performs an increment operation on the counting semaphore, that is, increments the count of the free buffer by 1. The transition from displaying to idle does not depend on any instructions from the CPU and is completed entirely by hardware timing logic. The CPU does not need to actively wake up to detect the release of the buffer.

[0030] In step S300, when the intra-frame micro-event occurs, the GPU hardware performs a signal operation on the hardware semaphore to release the occupied buffer or send a state change notification to the CPU.

[0031] Preferably, when the GPU detects specific hardware events such as the completion of memory page table updates, the end of the rendering channel, or the release of buffers, the synchronization control logic unit inside the GPU executes signal operations of hardware semaphores, including releasing the occupied buffers. For example, modifying the buffer's status register from "rendering" or "displaying" to "free", clearing the binding relationship between the buffer and the rendering engine or display engine, putting the buffer's identifier back into the free buffer pool, and sending a status change notification to the CPU through an interrupt request or memory-mapped I / O write. The notification message includes at least an event type identifier and a buffer identifier, so that the CPU can know about the resource status change without polling or waiting for the entire frame to complete, thereby starting to prepare the next frame of data earlier.

[0032] Furthermore, step S300 also includes the following steps: when a frame of image is displayed and the corresponding buffer is released, the display engine hardware in the GPU automatically performs an increment operation on the counting semaphore to increment the count of the free buffer by 1; when the CPU needs to acquire a free buffer, it performs a decrement operation on the counting semaphore. If the decrement is successful, the buffer is acquired; if the decrement fails, the CPU enters a light sleep or performs other non-rendering tasks until the next semaphore increment occurs.

[0033] Preferably, the display engine outputs a complete frame of image data pixel by pixel from the buffer to the display interface, and this buffer is no longer occupied by the display engine. The display engine hardware in the GPU is used to periodically read pixel data from the buffer and output it according to the vertical synchronization timing. An incremental operation is performed on the counting semaphore, including reading the current value of the counting semaphore, incrementing it by 1, and writing the result back. The process from reading to writing back is not interrupted by other hardware or software operations. The free buffer count is incremented by 1, indicating that a buffer has changed from the "displaying" or "released but not updated" state to a clear "free and available" state. If the CPU needs to prepare the next frame of image data, it needs to obtain a free buffer for writing. A decrement operation is performed on the counting semaphore, including reading the current value of the counting semaphore and determining whether it is greater than 0. If it is greater than 0, it is decremented by 1, the new value is written back, and "success" is returned to the caller. If it is equal to 0, the count value is not modified, and "failure" is returned. After the CPU successfully completes the decrement operation, it obtains the buffer, where the count value of the counting semaphore is reduced by 1, indicating that the number of free buffers has decreased by one. If the current value of the counting semaphore is 0, it means that all buffers are currently in a "rendering" or "displaying" state, and no free buffers are available. This indicates that the decrement has failed, and the CPU enters a light sleep state. In other words, the CPU core enters a low-power idle state, but maintains cache consistency and interrupt response capability, or performs other non-rendering tasks, such as processing input events, performing audio processing, updating non-rendering related game logic, processing network packets, performing file I / O operations, etc., until the next semaphore increment occurs. The hardware event generated by the increment operation wakes up the CPU, thereby achieving a buffer synchronization mechanism with zero CPU polling overhead.

[0034] In step S400, based on the state change notification, the CPU dynamically adjusts the submission time of the rendering task according to the length of the idle waiting interval and the historical rendering time of the GPU, so that the operation of the CPU to render the next frame of image data overlaps with the operation of the GPU to render the current frame of image data at least partially.

[0035] Step S400 further includes obtaining the GPU rendering time and idle waiting interval length of the most recent N frames based on the recorded data of a preset time period; predicting the rendering time of the current frame based on the GPU rendering time of the most recent N frames using an exponential smoothing method; comparing the rendering time of the current frame with the idle waiting interval length, and adjusting the submission time of the rendering task according to the time comparison relationship.

[0036] Preferably, the rendering time of the current frame is predicted based on historical data and compared with the idle waiting interval to determine whether to submit the next frame rendering task in advance. Here, the preset time period refers to a fixed time window length, such as the time span corresponding to the past 100 milliseconds, the past 1 second, or the past 60 frames. The recorded data of the preset time period is obtained, and the GPU rendering time of the most recent N frames is determined by subtracting the timestamp of the rendering start event from the timestamp of the GPU rendering completion event. That is, the time length from when the GPU starts executing the current frame rendering command to when the rendering is completed. The length of the idle waiting interval of the most recent N frames is determined by the mapping calculation of the first time axis and the second time axis. This refers to the length of the positive time period corresponding to the offset, that is, the time interval between when the GPU completes rendering and when the next vertical synchronization signal starts. Here, N is a positive integer, with a typical value of 3 to 16 frames. The larger the value of N, the smoother the prediction result but the higher the response latency. The smaller the value of N, the more sensitive the prediction but may be affected by abnormal fluctuations in a single frame.

[0037] Preferably, the prediction formula for exponential smoothing is as follows: ,in, This represents the prediction rendering time for frame t. This represents the actual rendering time of frame t. This represents the prediction rendering time for frame t-1. This represents the smoothing factor, with a value range of (0, 1). It is calculated recursively using exponential smoothing, requiring only the predicted value from the previous frame and the actual value from the current frame each time. Furthermore, the weight of earlier historical data decays exponentially. The larger the value, the more the prediction depends on the most recent frame. α The smaller the value, the more the prediction relies on historical trends; specifically, the arithmetic mean of the actual rendering times of the previous few frames is used as the initial prediction value, and the actual rendering time is obtained after the current frame is rendered. The prediction formula is updated to obtain the prediction value for the next frame, and then the rendering time of the current frame is predicted and output.

[0038] Furthermore, step S400 also includes: when the length of the idle waiting interval is greater than or equal to the predicted rendering time of the current frame, it is determined that the current frame can be rendered before the next vertical synchronization signal arrives, and before the current frame is rendered, the rendering instruction for the next frame is inserted into the GPU command queue in advance, so that the operation of the CPU preparing the image data for the next frame overlaps with the operation of the GPU rendering the image data for the current frame at least in part; when the length of the idle waiting interval is less than the predicted rendering time of the current frame, the rendering instruction for the next frame is not inserted in advance, and the rendering instruction for the next frame is submitted after the next vertical synchronization signal arrives, wherein the operations of the CPU and the GPU are executed serially.

[0039] The rendering time of the current frame is compared with the length of the idle waiting interval. Based on the time comparison, the submission time of the rendering task is adjusted. If the rendering time of the current frame is less than or equal to the length of the idle waiting interval, it means that the current frame can be rendered before the next vertical synchronization signal arrives, and there is still idle time remaining after completion. In this case, before the current frame is rendered, the CPU inserts the rendering instruction for the next frame into the GPU command queue in advance, so that the operation of the CPU preparing the image data for the next frame and the operation of the GPU rendering the image data for the current frame overlap in time at least partially. If the rendering time of the current frame is greater than the length of the idle waiting interval, it means that the current frame cannot be rendered before the next vertical synchronization signal arrives. In this case, the rendering instruction for the next frame is not inserted in advance, and the rendering instruction for the next frame is submitted after the next vertical synchronization signal arrives, avoiding buffer contention or display tearing. In this case, the operations of the CPU and GPU are executed serially without time overlap.

[0040] Furthermore, the method also includes: acquiring frame display anomaly events of the graphics display engine; tracing and locating the source of image processing parameters based on the frame display anomaly events, generating anomaly feedback information and sending it to the GPU; and dynamically adjusting response parameters based on the anomaly feedback information.

[0041] Preferably, the graphics driver running on the CPU or the display monitoring hardware unit inside the GPU acquires frame display anomaly events of the graphics display engine, i.e., abnormal conditions that do not conform to expectations during the display output process, including abnormal event rhythm and image generation anomalies. Image processing parameters refer to configurable parameters that control the behavior of the rendering pipeline and the quality of the output image. Then, based on the frame display anomaly events, the source of the image processing parameters is traced and located to determine the image frame, vertical synchronization period, rendering channel, screen area, and specific image processing parameter configuration in which the anomaly event occurred. For example, screen tearing may cause the vertical synchronization mode to be turned off or the number of buffers to be insufficient, and frame drops may cause hardware signal issues. Setting the trigger threshold too high can cause CPU submission delays, block artifacts that may result in excessively low anti-aliasing levels or insufficient texture filtering quality, and thus generate anomaly feedback information. This feedback includes the anomaly type, occurrence time, location, and current anomaly parameters, and is sent to the GPU. The GPU responds to the anomaly feedback by modifying the corresponding configuration registers or state machine parameters. For example, for screen tearing, the vertical synchronization mode is changed from "off" to "strict"; for frequent frame drops, the smoothing factor of exponential smoothing prediction is reduced to decrease premature submission of errors; for color deviation, the color correction matrix is ​​reset to an identity matrix or a calibration value is loaded; for motion artifacts, the rendering resolution is increased or temporal anti-aliasing is enabled. Through dynamic parameter adjustments, the system reconverges to a stable state without manual intervention or a system restart.

[0042] Furthermore, the method also includes the following: the frame display anomaly events include: event rhythm anomaly and image generation anomaly. The event rhythm anomaly includes screen tearing during buffer swapping, frames not being ready before the vertical synchronization signal arrives, multiple consecutive frames displaying the same content, and frame intervals exceeding a preset fluctuation range. The image generation anomaly includes blockiness, color deviation, overexposure / underexposure, motion artifacts, or flickering in the displayed image. The response parameters for the event rhythm anomaly include the trigger threshold of the hardware semaphore, the initial value of the counting semaphore, the smoothing factor of the exponential smoothing prediction, or the vertical synchronization mode. The response parameters for the image generation anomaly include rendering resolution, anti-aliasing level, texture filtering quality, shadow detail level, or color correction parameters.

[0043] Preferably, frame display anomalies include event rhythm anomalies and image generation anomalies. Event rhythm anomalies refer to timing-related anomalies related to frame timing, buffer swapping, and vertical synchronization signals. These anomalies involve the timing and order of frame display, including screen tearing during buffer swapping (where new content is written to the display engine while it is reading the buffer, causing a single frame to contain image data from two consecutive frames, resulting in horizontal misalignment lines in the image); frames not being ready before the arrival of the vertical synchronization signal (where the GPU fails to complete rendering of the current frame before the next vertical synchronization signal arrives, the display engine cannot acquire new frames, and is forced to repeat the display of the previous frame); multiple consecutive frames displaying the same content (where the display engine outputs identical content in two or more consecutive vertical synchronization cycles, resulting in a static image); and frame intervals exceeding a preset fluctuation range (where the time interval between two adjacent frames deviates from the vertical synchronization cycle by a set threshold). Image generation anomalies refer to anomalies related to the visual quality of the rendered output, involving the correctness and clarity of the image content. These include block artifacts in the displayed image, i.e., rectangular block-shaped areas appearing in the image, with consistent pixels within each block but discontinuities between blocks; color deviation, i.e., the colors of the displayed image do not match the actual rendering intent; overexposure / underexposure, i.e., brightness exceeding the normal dynamic range; motion artifacts, i.e., abnormal trailing or ghosting left by fast-moving objects on the image; and flicker, i.e., high-frequency alternating changes in image brightness or color.

[0044] Preferably, response parameters for event rhythm anomalies are used to adjust frame timing and synchronization behavior, including the trigger threshold of hardware semaphores (increasing the threshold for frequent tearing and decreasing it for frequent frame drops); the initial value of the counter semaphore (increasing the initial value for frequent frame drops and decreasing it for memory constraints); the smoothing factor for exponential smoothing prediction (increasing the smoothing factor for large load fluctuations and decreasing it for stable loads); and the vertical synchronization mode (strict vertical synchronization for tearing, adaptive vertical synchronization for frame drops, and disabling vertical synchronization for low latency). Response parameters for image generation anomalies are used to adjust rendering quality and image processing behavior, including rendering resolution (increasing rendering resolution for blockiness and decreasing it for insufficient performance); anti-aliasing level (increasing the aliasing level for blockiness / aliasing); texture filtering quality (trilinear or anisotropic filtering for blockiness); shadow detail level (increasing shadow resolution for jagged shadow edges); and color correction parameters (resetting the color matrix for color deviation and decreasing brightness gain for overexposure).

[0045] In the above text, refer to Figure 1 A GPU synchronous acceleration image optimization method for a graphics display engine according to embodiments of the present invention is described in detail. Next, reference will be made to... Figure 2 A GPU synchronous acceleration image optimization system for a graphics display engine is described according to an embodiment of the present invention.

[0046] The GPU synchronous acceleration image optimization system for graphics display engines according to embodiments of the present invention addresses the technical problems in the prior art, such as coarse synchronization granularity between the CPU and GPU, low utilization of idle waiting intervals, leading to rendering pipeline stagnation or underload. It achieves the technical effect of improving the time overlap of CPU and GPU task execution and the overall graphics rendering throughput. Figure 2 As shown, the GPU synchronous acceleration image optimization system for the graphics display engine includes: an idle waiting interval determination module 10, a hardware semaphore embedding module 20, a signal operation execution module 30, and a task dynamic adjustment module 40.

[0047] The idle waiting interval determination module 10 is used to identify the time synchronization relationship between the graphics display engine and the GPU and determine the idle waiting interval; the hardware semaphore embedding module 20 is used to embed a hardware semaphore in the GPU's command queue based on the idle waiting interval. The trigger condition for the hardware semaphore is an intra-frame micro-event of the graphics display engine, including completion of memory page table update, end of rendering channel, or buffer release; the signal operation execution module 30 is used to execute the signal operation of the hardware semaphore by the GPU hardware when the intra-frame micro-event occurs, so as to release the occupied buffer or send a state change notification to the CPU; the task dynamic adjustment module 40 is used to dynamically adjust the submission time of the rendering task based on the state change notification, the length of the idle waiting interval and the historical rendering time of the GPU, so that the operation of the CPU for the next frame of image data overlaps with the operation of the GPU for rendering the current frame of image data at least partially in time.

[0048] The specific configuration of the idle waiting interval determination module 10 will be described in detail below. The idle waiting interval determination module 10 further includes: acquiring the vertical synchronization period and phase of the graphics display engine to establish a first time axis; acquiring the event timestamp of GPU rendering completion to establish a second time axis; mapping the rendering completion time on the second time axis to the first time axis to obtain an offset relative to the start time of the next vertical synchronization signal; and determining the positive time period corresponding to the offset as the idle waiting interval.

[0049] The specific configuration of the hardware semaphore embedding module 20 will be described in detail below. The hardware semaphore embedding module 20 further includes: the hardware semaphore is a counting semaphore used to manage the idle state of at least three buffers, wherein each buffer has three states: idle, rendering, and displaying, and each buffer is associated with a state machine, and the initial value of the counting semaphore is the total number of buffers.

[0050] The specific configuration of the signal operation execution module 30 will be described in detail below. The signal operation execution module 30 further includes: when a frame of image is displayed and the corresponding buffer is released, the display engine hardware in the GPU automatically performs an increment operation on the counting semaphore to increment the free buffer count by 1; when the CPU needs to acquire a free buffer, it performs a decrement operation on the counting semaphore. If the decrement is successful, the buffer is acquired; if the decrement fails, the CPU enters a light sleep or executes other non-rendering tasks until the next semaphore increment occurs.

[0051] The specific configuration of the task dynamic adjustment module 40 will be described in detail below. The task dynamic adjustment module 40 further includes: obtaining the GPU rendering time and idle waiting interval length of the most recent N frames based on recorded data of a preset time period; predicting the rendering time of the current frame based on the GPU rendering time of the most recent N frames using an exponential smoothing method; comparing the rendering time of the current frame with the idle waiting interval length, and adjusting the submission time of the rendering task according to the time comparison relationship.

[0052] The specific configuration of the task dynamic adjustment module 40 will be described in detail below. The task dynamic adjustment module 40 further includes: when the length of the idle waiting interval is greater than or equal to the predicted rendering time of the current frame, it is determined that the current frame can be rendered before the next vertical synchronization signal arrives, and before the current frame is rendered, the rendering instruction of the next frame is inserted into the GPU command queue in advance, so that the operation of the CPU preparing the image data of the next frame and the operation of the GPU rendering the image data of the current frame overlap at least partially; when the length of the idle waiting interval is less than the predicted rendering time of the current frame, the rendering instruction of the next frame is not inserted in advance, and the rendering instruction of the next frame is submitted after the next vertical synchronization signal arrives, wherein the operations of the CPU and the GPU are executed serially.

[0053] The following describes in detail the specific configuration of the GPU synchronous acceleration image optimization system used in the graphics display engine. It further includes the state machine transition rules for each buffer, including: transitioning from idle to rendering, where after the CPU successfully acquires the semaphore, it marks the buffer as rendering and submits a rendering command to the GPU; transitioning from rendering to display, where the GPU completes rendering and waits for the next vertical synchronization signal, automatically switching the buffer to display when the vertical synchronization signal arrives, and starting to scan for output; and transitioning from display to idle, where after the display engine completes the output of all pixels in the corresponding buffer, the display engine hardware automatically sets the buffer state to idle and performs an increment operation on the counting semaphore; wherein the state transition from display to idle does not depend on any CPU instructions and is entirely completed by hardware timing logic.

[0054] The following describes in detail the specific configuration of the GPU synchronous acceleration image optimization system used for the graphics display engine. It further includes: acquiring frame display anomaly events from the graphics display engine; tracing and locating the source of image processing parameters based on the frame display anomaly events, generating anomaly feedback information and sending it to the GPU; and dynamically adjusting response parameters based on the anomaly feedback information.

[0055] The following describes in detail the specific configuration of the GPU Synchronous Accelerated Image Optimization System used in the graphics display engine. It further includes: frame display anomalies including: event rhythm anomalies and image generation anomalies; the event rhythm anomalies include screen tearing during buffer swapping, frames not being ready before the vertical sync signal arrives, multiple consecutive frames displaying the same content, and frame intervals exceeding a preset fluctuation range; the image generation anomalies include blockiness, color deviation, overexposure / underexposure, motion artifacts, or flickering in the displayed image; wherein, the response parameters for the event rhythm anomalies include the trigger threshold of the hardware semaphore, the initial value of the counting semaphore, the smoothing factor of the exponential smoothing prediction, or the vertical sync mode; the response parameters for the image generation anomalies include rendering resolution, anti-aliasing level, texture filtering quality, shadow detail level, or color correction parameters.

[0056] The GPU synchronous acceleration image optimization system for graphics display engines provided in the embodiments of the present invention can execute the GPU synchronous acceleration image optimization method for graphics display engines provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0057] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A GPU-synchronized accelerated image optimization method for graphics display engines, characterized in that, include: Identify the time synchronization relationship between the graphics display engine and the GPU to determine the idle waiting interval; Based on the idle waiting interval, a hardware semaphore is embedded in the GPU's command queue. The triggering condition of the hardware semaphore is an intra-frame micro-event of the graphics display engine. The intra-frame micro-event includes completion of memory page table update, end of rendering channel, or buffer release. When the intra-frame micro-event occurs, the GPU hardware performs a signal operation on the hardware semaphore to release the occupied buffer or send a state change notification to the CPU. Based on the state change notification, the CPU dynamically adjusts the submission time of the rendering task according to the length of the idle waiting interval and the historical rendering time of the GPU, so that the operation of the CPU to render the next frame of image data overlaps with the operation of the GPU to render the current frame of image data at least partially.

2. The GPU-synchronized accelerated image optimization method for graphics display engines according to claim 1, characterized in that, Identify the time synchronization relationship between the graphics display engine and the GPU, and determine the idle waiting interval, including: Obtain the vertical synchronization cycle and phase of the graphics display engine to establish the first time axis; Obtain the event timestamp of GPU rendering completion and establish a second timeline; Map the rendering completion time on the second time axis onto the first time axis to obtain the offset relative to the start time of the next vertical synchronization signal; The positive time period corresponding to the offset is determined as the idle waiting interval.

3. The GPU-synchronized accelerated image optimization method for graphics display engines according to claim 1, characterized in that, The hardware semaphore is a counting semaphore used to manage the idle state of at least three buffers. Each buffer has three states: idle, rendering, and displaying. Each buffer is associated with a state machine. The initial value of the counting semaphore is the total number of buffers.

4. The GPU-synchronized accelerated image optimization method for graphics display engines according to claim 3, characterized in that, The signal operation includes: when a frame of image is displayed and the corresponding buffer is released, the display engine hardware in the GPU automatically performs an increment operation on the counting semaphore to increment the count of the free buffer by 1. When the CPU needs to acquire a free buffer, it performs a decrement operation on the counting semaphore. If the decrement is successful, the buffer is acquired. If the semaphore increment fails, the CPU enters a light sleep or performs other non-rendering tasks until the next semaphore increment occurs.

5. The GPU-synchronized accelerated image optimization method for graphics display engines according to claim 1, characterized in that, Based on the length of the idle waiting interval and the historical rendering time of the GPU, the submission time of the rendering task is dynamically adjusted, including: Based on the recorded data within a preset time period, obtain the GPU rendering time and idle waiting interval length of the most recent N frames; Based on the GPU rendering time of the most recent N frames, the rendering time of the current frame is predicted using the exponential smoothing method. The rendering time of the current frame is compared with the length of the idle waiting interval, and the submission time of the rendering task is adjusted according to the time comparison relationship.

6. The GPU-synchronized accelerated image optimization method for a graphics display engine according to claim 5, characterized in that, Based on time comparisons, adjust the submission time of the rendering task, including: When the length of the idle waiting interval is greater than or equal to the predicted rendering time of the current frame, it is determined that the current frame can be rendered before the next vertical synchronization signal arrives. Before the current frame is rendered, the rendering instruction of the next frame is inserted into the GPU command queue in advance, so that the operation of the CPU preparing the image data of the next frame overlaps with the operation of the GPU rendering the image data of the current frame at least in part. When the length of the idle waiting interval is less than the predicted rendering time of the current frame, the rendering command for the next frame is not inserted in advance. Instead, the rendering command for the next frame is submitted after the next vertical synchronization signal is received. The operations of the CPU and GPU are executed serially.

7. The GPU-synchronized accelerated image optimization method for graphics display engines according to claim 3, characterized in that, The state machine transition rules for each buffer include: The idle state is transitioned to rendering. After the CPU successfully acquires the semaphore, it marks the buffer as being in the rendering state and submits a rendering command to the GPU. The process transitions from rendering to display mode. During this transition, the GPU completes rendering and waits for the next vertical sync signal. When the vertical sync signal arrives, the buffer is automatically switched to display mode, and the scanning output begins. The display engine transitions from being in the display state to being idle. This occurs when the display engine finishes outputting all pixels in the corresponding buffer. The display engine hardware automatically sets the buffer state to idle and performs an increment operation on the counting semaphore. The transition from the displayed state to the idle state does not depend on any CPU instructions; it is entirely handled by hardware timing logic.

8. The GPU-synchronized accelerated image optimization method for a graphics display engine according to claim 1, characterized in that, Also includes: Get frame display exception events from the graphics engine; Based on the abnormal event displayed in the frame, the image processing parameters are traced and located, and abnormal feedback information is generated and sent to the GPU. The GPU dynamically adjusts its response parameters based on the anomaly feedback information.

9. The GPU-synchronized accelerated image optimization method for a graphics display engine according to claim 8, characterized in that, The frame display anomalies include: event rhythm anomalies and image generation anomalies. Event rhythm anomalies include screen tearing during buffer swapping, frames not being ready before the vertical synchronization signal arrives, multiple consecutive frames displaying the same content, and frame intervals exceeding a preset fluctuation range. Image generation anomalies include blockiness, color deviation, overexposure / underexposure, motion artifacts, or flickering in the displayed image. The response parameters for the event rhythm anomaly include the trigger threshold of the hardware semaphore, the initial value of the counting semaphore, the smoothing factor of the exponential smoothing prediction, or the vertical synchronization mode; the response parameters for the image generation anomaly include the rendering resolution, anti-aliasing level, texture filtering quality, shadow detail level, or color correction parameters.

10. A GPU-synchronized accelerated image optimization system for a graphics display engine, characterized in that, The system is used to implement the GPU synchronous acceleration image optimization method for a graphics display engine as described in any one of claims 1 to 9, the system comprising: The idle waiting interval determination module is used to identify the time synchronization relationship between the graphics display engine and the GPU, and determine the idle waiting interval; A hardware semaphore embedding module is used to embed hardware semaphores in the GPU's command queue based on the idle waiting interval. The triggering condition of the hardware semaphore is an intra-frame micro-event of the graphics display engine, which includes completion of video memory page table update, end of rendering channel, or buffer release. The signal operation execution module is used to execute the signal operation of the hardware semaphore by the GPU hardware when the intra-frame micro-event occurs, so as to release the occupied buffer or send a state change notification to the CPU. The task dynamic adjustment module is used to dynamically adjust the submission time of the rendering task based on the state change notification, the CPU according to the length of the idle waiting interval and the historical rendering time of the GPU, so that the operation of the CPU to render the next frame of image data overlaps with the operation of the GPU to render the current frame of image data at least partially.

Images