Method, apparatus, terminal and storage medium for displaying optimized latency

Abstract

This invention discloses a method, apparatus, terminal, and storage medium for optimizing display latency. The method includes: calculating the total rendering time of the previous frame; obtaining the remaining rendering time of the current frame, and determining a delay time based on the remaining time, the total rendering time, and the current frame's safety margin; and using the delay time to determine the submission timing of the current frame. This delay safety margin solves the problem of frame dropping in extreme cases and resolves display stuttering in normal cases.

Description

Technical Field

[0001] This invention relates to the field of Wayland technology, and more particularly to a method, apparatus, terminal, and storage medium for optimizing display latency. Background Technology

[0002] Wayland is a display server protocol for Linux and other Unix-like operating systems. It defines the communication mechanism between Wayland clients and Wayland synthesizers. Compared to the traditional X11 protocol, Wayland has a simpler and more efficient architecture, especially in providing better performance support for high-resolution, high-refresh-rate displays. Therefore, it has become the mainstream development direction of modern open-source graphics stacks and has been adopted as the default display protocol by major Linux distributions.

[0003] In a typical Wayland compositor implementation, the update of displayed content follows a periodic rendering mechanism based on the refresh rate. Taking a 60Hz refresh rate as an example, the display cycle is approximately 16.67ms, and the compositor typically starts the rendering process for the next frame immediately after the previous frame is rendered. However, due to the periodic constraints of display output, the rendered frame data often needs to wait until the next vertical sync time point before it can be submitted to the display device. During this period, if new screen changes occur, such as screen content updates caused by input device events, these changes must be delayed until a subsequent display cycle before entering the rendering queue, thus introducing additional display latency. Summary of the Invention

[0004] This invention provides a method, apparatus, terminal, and storage medium for optimizing display latency, in order to solve the technical problem of additional display latency caused by the Wayland synthesizer implementation in the prior art.

[0005] In a first aspect, embodiments of the present invention provide a method for optimizing display latency, comprising: In the Wayland environment, the total rendering time of the previous frame is calculated; Obtain the remaining rendering time of the current frame, and determine the delay time based on the remaining rendering time, the total rendering time, and the insurance time of the current frame; The timing of submitting the current frame is determined using the aforementioned delay duration.

[0006] Secondly, embodiments of the present invention also provide a display delay optimization device, comprising: The calculation module is used to calculate the total rendering time of the previous frame in the Wayland environment; The acquisition module is used to acquire the remaining duration of the current frame rendering and determine the delay duration based on the remaining duration, the total rendering duration, and the insurance duration of the current frame. The submission timing determination module is used to determine the submission timing of the current frame using the delay duration.

[0007] Thirdly, embodiments of the present invention also provide a terminal, including: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the display optimization latency method as provided in any of the above embodiments.

[0008] Fourthly, embodiments of the present invention also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the display optimization latency method provided in the above embodiments.

[0009] The display latency optimization method, apparatus, terminal, and storage medium provided in this invention calculate the overall rendering time of the previous frame in a Wayland environment; obtain the remaining rendering time of the current frame; and determine the delay time based on the remaining time, the overall rendering time, and the current frame's safety margin; then, use the delay time to determine the submission timing of the current frame. By accurately calculating the overall rendering time of the previous frame, dynamically obtaining the remaining time of the current frame, and combining it with the safety margin, the required delay time is accurately calculated. Even in the event of stuttering, it ensures that the current frame is successfully submitted to the display controller, preventing the page from freezing or crashing. The delay safety margin solves the problem of frame dropping in extreme cases and resolves display stuttering in normal cases. Attached Figure Description

[0010] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating the display delay optimization method provided in Embodiment 1 of the present invention; Figure 2 This is a flowchart illustrating the display delay optimization method provided in Embodiment 2 of the present invention; Figure 3 This is a flowchart illustrating the display delay optimization method provided in Embodiment 3 of the present invention; Figure 4 This is a schematic diagram of the display delay optimization device provided in Embodiment 4 of the present invention; Figure 5 This is a schematic diagram of the terminal provided in Embodiment 5 of the present invention. Detailed Implementation

[0011] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0012] Example 1 Figure 1 This is a flowchart of a display delay optimization method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where the rendering submission timing is optimized during Wayland rendering. This method can be executed by a Wayland rendering display delay optimization device, and specifically includes the following steps: Step 110: In the Wayland environment, calculate the total rendering time of the previous frame.

[0013] In this embodiment, the CPU and GPU rendering times of the previous frame are continuously monitored and used as a reference for later rendering. For example, calculating the overall rendering time of the previous frame may include: obtaining the CPU start time corresponding to the submission of the rendering request; obtaining the CPU end time and GPU start time corresponding to the issuance of the rendering command to the GPU in the previous frame; obtaining the GPU end time when the rendering of the previous frame ends; and calculating the overall rendering time based on the CPU start time, CPU end time, GPU start time, and GPU end time. For example, when the rendering request is submitted in the previous frame, the start and end times of the CPU and GPU are recorded. CpuStart is the CPU start recording time. CpuEnd / GpuStart is the time when the rendering command is issued to the GPU. GpuEnd is the time when the GPU completes rendering. Accordingly, the overall rendering time = (CpuEnd - CpuStart) + (GpuEnd - GpuStart).

[0014] For example, the CPU start time can be the point in time when the compositor receives the signal that the previous frame is complete, such as the Wayland frame event or the time when it receives user input / window update requests. The CPU end time and GPU start time are usually the same or very close. It can also be the moment when the compositor submits rendering instructions, such as OpenGL / Vulkan commands, to the GPU. At this point, the CPU has completed the instruction assembly, and the GPU begins execution. The GPU end time is the moment when the GPU has completely executed all rendering instructions. For CPU time, a high-precision timer can be obtained through system calls, such as `clock_gettime(CLOCK_MONOTONIC)`. For GPU time: this depends on the characteristics of the graphics API. For Vulkan statements, `vkGetQueryPoolResults` can be used to retrieve the GPU's timestamp. For OpenGL statements, `GL_TIMESTAMP` can be used to query the object.

[0015] Step 120: Obtain the remaining rendering time of the current frame, and determine the delay time based on the remaining rendering time, the overall rendering time, and the insurance time of the current frame.

[0016] In this embodiment, the safety margin can be a safety reserve time. This prevents delayed frame submission and ensures the GPU has sufficient time to complete rendering. The safety margin is not a fixed duration. In this embodiment, the overall processing time of the current frame can be determined based on the refresh rate, thereby determining the remaining rendering time of the current frame. For example, it can be calculated by subtracting the currently consumed time from the system clock.

[0017] Optionally, the delay duration can be calculated using the following method: Remaining time - Total rendering time - Insurance time.

[0018] Step 130: Determine the submission time of the current frame using the delay duration.

[0019] When the delay duration is greater than 0, it indicates that the system has sufficient margin, and the delay time can be set to 0 or a very small value. Conversely, the delay time will be a positive number, indicating that additional delay is needed to ensure that the system does not force a submission before rendering is complete, causing a freeze. By using a variable safety margin, the gap between rendering submission and sending to the display can be covered as much as possible, and multiple image changes can be composited at once, improving efficiency.

[0020] This embodiment calculates the total rendering time of the previous frame in the Wayland environment; obtains the remaining rendering time of the current frame; and determines the delay time based on the remaining time, the total rendering time, and the current frame's safety margin. The timing of the current frame's submission is then determined using this delay time. By accurately calculating the total rendering time of the previous frame, dynamically obtaining the remaining time of the current frame, and combining this with the safety margin, the required delay time is precisely calculated. Even in the event of stuttering, this ensures that the current frame is successfully submitted to the display controller, preventing the page from freezing or crashing. The delay safety margin solves the problem of frame drops in extreme cases and resolves display stuttering in normal situations.

[0021] In a preferred embodiment of this example, the method may further include the following steps: determining whether a timeout and frame drop have occurred based on the effect of the delayed rendering of the previous frame; adjusting the safety margin of the current frame based on the timeout and frame drop phenomenon. The safety margin is not a fixed value; in this embodiment, a dynamic adjustment mechanism is adopted. For example, if frame drop is detected, the safety margin of the current frame is increased to provide a larger safety margin in case the next frame freezes again. When the system is busy or experiences sudden lag, the safety margin automatically increases to prevent the screen from freezing and improve the smoothness of the display.

[0022] Example 2 Figure 2 This is a flowchart illustrating the display latency optimization method provided in Embodiment 2 of the present invention. This embodiment is based on the above embodiment and is optimized. The method may also include the following steps: obtaining the current GPU temperature, power consumption and rendering queue length; adjusting the insurance duration of the current frame according to the current GPU temperature, power consumption and rendering queue length.

[0023] See Figure 2 The method for optimizing display latency includes: Step 210: In the Wayland environment, calculate the total rendering time of the previous frame.

[0024] Step 220: Obtain the remaining duration of the current frame rendering, and determine the delay duration based on the remaining duration, the overall rendering duration, and the insurance duration of the current frame.

[0025] Step 230: Determine the submission timing of the current frame using the delay duration.

[0026] Step 240: Obtain the current GPU temperature, power consumption, and rendering queue length.

[0027] Between compression latency and fault tolerance in high frame rate graphics rendering, the safety margin is a key factor. Therefore, in this embodiment, the safety margin needs to be flexibly adjusted according to the actual rendering situation. When the GPU temperature rises, the chip may trigger thermal throttling, causing a sharp drop in rendering performance. Optionally, the current GPU temperature can be obtained using the interface provided by the graphics card. The GPU's instantaneous power consumption directly reflects its workload. Power spikes are usually accompanied by rendering-intensive tasks; similarly, the current GPU power consumption can be obtained using the interface provided by the graphics card. The rendering queue length represents the number of tasks waiting to be executed in the GPU scheduler. The longer the queue, the more likely the current frame is to be delayed.

[0028] Step 250: Adjust the insurance duration of the current frame based on the current GPU temperature, power consumption, and rendering queue length.

[0029] For example, this could include: shortening the buffer duration to prevent rendering timeouts when the detected increase in GPU power consumption exceeds an increase threshold or the queue depth exceeds a depth threshold; and extending the buffer duration to prevent frequent frame drops when the GPU is under high load or the temperature exceeds a normal temperature threshold.

[0030] This embodiment adds the following steps: obtaining the current GPU temperature, power consumption, and rendering queue length; and adjusting the safety duration of the current frame based on these parameters. This introduces a hardware feedback loop, dynamically adjusting the rendering strategy by monitoring the GPU's physical state in real time. It is particularly suitable for real-time rendering scenarios that require a balance between extreme performance and hardware safety.

[0031] Example 3 Figure 3 This is a flowchart illustrating the display latency optimization method provided in Embodiment 3 of the present invention. This embodiment is based on the above embodiment and is optimized. The method may further include the following steps: obtaining the Wayland protocol object of the display output and adding a delayed frame event to the protocol; obtaining user configuration, GPU load, input event frequency and frame rate deviation, calculating a latency evaluation value based on the user configuration, GPU load, input event frequency and frame rate deviation; when receiving the vertical synchronization signal, comparing the latency evaluation value with a preset evaluation threshold, and when the latency evaluation value is less than the preset evaluation threshold, sending a delayed frame event to the Wayland compositor; after receiving the delayed frame event, the Wayland compositor continuously processes input events, generates damaged regions, and merges the damaged region queue; performing a single composite rendering and submitting the composite rendering frame.

[0032] See Figure 3 The method for optimizing display latency includes: Step 310: In the Wayland environment, calculate the total rendering time of the previous frame.

[0033] Step 320: Obtain the remaining rendering time of the current frame, and determine the delay time based on the remaining rendering time, the total rendering time, and the insurance time of the current frame; Step 330: Determine the submission timing of the current frame using the delay duration.

[0034] Step 340: Obtain the Wayland protocol object for display output and add a delayed frame event to the protocol.

[0035] While dynamically adjusted buffer durations can ensure rendering tasks continue normally without frame drops during periods of abnormal system load, this approach may negatively impact display quality on high-performance devices with high refresh rates. Therefore, this embodiment utilizes a compressed latency method to achieve faster rendering response. For example, by adding delayed frame events, rendering can be optimized under certain conditions.

[0036] Optionally, the Wayland protocol XML file can be edited to add a delayed frame event declaration in the wl_surface interface.

[0037] Step 350: Obtain user configuration, GPU load, input event frequency, and frame rate deviation; calculate latency evaluation value based on the user configuration, GPU load, input event frequency, and frame rate deviation.

[0038] The user configuration can include performance mode and power-saving mode. GPU load can be obtained by monitoring GPU utilization in real time and by monitoring peripherals such as mouse / keyboard to obtain input event frequency. It can also be based on the difference between the current FPS and the target FPS.

[0039] For example, the data obtained can be normalized separately, and corresponding weight values ​​can be set according to the current situation. The delay evaluation value can be calculated by weighted summation based on the weight values ​​and normalized values.

[0040] Step 360: When receiving the vertical synchronization signal, the delay evaluation value is compared with a preset evaluation threshold. If the delay evaluation value is less than the preset evaluation threshold, a delay frame event is sent to the Wayland synthesizer.

[0041] The vertical synchronization signal is a hardware interrupt signal actively sent to the GPU by the display hardware every frame cycle. After the GPU captures the VSync signal, it will begin drawing the image of the next frame to the frame buffer. In this embodiment, the delay evaluation value of the aforementioned steps can be compared with a preset evaluation threshold. If it is greater than the preset evaluation threshold, it can be proven that the current system load is relatively light, and the current system's surplus computing power can be used to execute delayed frame events.

[0042] Step 370: After receiving a delayed frame event, the Wayland synthesizer continues to process the input events, generate damaged regions, and merge the damaged region queue.

[0043] Upon receiving a `later_frame` event from the client, the Wayland compositor does not immediately enter an idle state but rather an intermediate state. At this point, there is typically a period before the VSync signal triggers, determined by the buffer duration. While continuously processing input, the Wayland compositor performs a series of pre-rendering preparations. When an input event is detected, the compositor captures changes in the current window's state. Based on the impact of the input event, the compositor generates one or more Damage regions. Furthermore, to improve rendering efficiency, the compositor does not immediately render each small Damage region but merges them, thus enhancing rendering efficiency. When the VSync signal triggers or the buffer duration expires, the compositor performs a one-time rendering using the merged Damage queue, drawing only the regions that need updating, while reusing the cache for other unchanged regions. After rendering is complete, the entire frame is submitted to the display hardware.

[0044] Step 380: Perform a single composite rendering and submit the composite rendering frame.

[0045] When the later_frame event is triggered, the Wayland compositor checks the Damage Queue, merges all updated regions of the current window frame, generates one or more minimized rectangular regions, and only draws these regions.

[0046] This embodiment adds the following steps: Obtaining the Wayland protocol object for display output and adding a delayed frame event to the protocol; obtaining user configuration, GPU load, input event frequency, and frame rate deviation; calculating a latency evaluation value based on the user configuration, GPU load, input event frequency, and frame rate deviation; comparing the latency evaluation value with a preset evaluation threshold when receiving the vertical synchronization signal; and sending a delayed frame event to the Wayland compositor when the latency evaluation value is less than the preset evaluation threshold; after receiving the delayed frame event, the Wayland compositor continuously processes input events, generates corrupted regions, and merges the corrupted region queue; performing a single composite rendering and submitting the composite rendering frame. This allows for compression using a safety margin during system idle times, ensuring input events can enter the next frame rendering faster and improving the interactive experience. Under high load or GPU throttling, the overall rendering time of the previous frame is used as a safety benchmark to ensure sufficient time for the rendering task to complete, avoiding screen stuttering or loss. By evaluating the remaining time of the current frame and the safety margin, the submission timing is dynamically adjusted to maximize the use of each refresh cycle, improving smoothness and stability.

[0047] Example 4 Figure 4 This is a schematic diagram of the display delay optimization device provided in Embodiment 4 of the present invention. See also... Figure 4 The display latency optimization device includes: The calculation module 410 is used to calculate the total rendering time of the previous frame in the Wayland environment; The acquisition module 420 is used to acquire the remaining duration of the current frame rendering and determine the delay duration based on the remaining duration, the overall rendering duration, and the insurance duration of the current frame. The submission timing determination module 430 is used to determine the submission timing of the current frame using the delay duration.

[0048] The display delay optimization device provided in this embodiment calculates the total rendering time of the previous frame in the Wayland environment; obtains the remaining rendering time of the current frame; and determines the delay time based on the remaining time, the total rendering time, and the current frame's safety margin. The timing of submitting the current frame is then determined using the delay time. By accurately calculating the total rendering time of the previous frame, dynamically obtaining the remaining time of the current frame, and combining it with the safety margin, the required delay time is precisely calculated. Even in the event of stuttering, it ensures that the current frame is successfully submitted to the display controller, preventing the page from freezing or crashing. The delay safety margin solves the problem of frame drops in extreme cases and resolves display stuttering in normal situations.

[0049] Based on the above embodiments, the computing module includes: The first acquisition unit is used to acquire the CPU start time corresponding to the submission of the rendering request; The second acquisition unit is used to acquire the CPU end time and GPU start time corresponding to the time when the rendering command was sent to the GPU in the previous frame. The third acquisition unit is used to acquire the GPU end time when the rendering of the previous frame ended; The calculation unit is used to calculate the overall rendering time based on the CPU start time, CPU end time, GPU start time, and GPU end time.

[0050] Based on the above embodiments, the device further includes: The timeout and frame dropping judgment module is used to determine whether a timeout and frame dropping has occurred based on the effect of the previous frame's delayed rendering submission. The adjustment module is used to adjust the insurance duration of the current frame based on timeout and frame drop phenomena.

[0051] Based on the above embodiments, the device further includes: The parameter acquisition module is used to obtain the current GPU temperature, power consumption, and rendering queue length; The insurance duration adjustment module is used to adjust the insurance duration of the current frame based on the current GPU temperature, power consumption, and rendering queue length.

[0052] Based on the above embodiments, the insurance duration adjustment module includes: The shortening unit is used to shorten the protection time when the GPU power consumption increase exceeds the increase threshold or the queue depth exceeds the depth threshold, so as to prevent rendering timeout.

[0053] Based on the above embodiments, the insurance duration adjustment module further includes: The extension unit is used to extend the protection period when the GPU is under high load or the temperature exceeds the normal temperature threshold, in order to prevent frequent frame drops.

[0054] Based on the above embodiments, the device further includes: The event addition module is used to obtain the Wayland protocol object of the display output and add a delayed frame event to the protocol; The latency evaluation value calculation module is used to obtain user configuration, GPU load, input event frequency and frame rate deviation, and calculate latency evaluation value based on the user configuration, GPU load, input event frequency and frame rate deviation; The transmitting module is used to compare the delay evaluation value with a preset evaluation threshold when receiving the vertical synchronization signal, and to send a delay frame event to the Wayland synthesizer when the delay evaluation value is less than the preset evaluation threshold. The merging module is used to continuously process input events after receiving delayed frame events, generate damaged regions, and merge the damaged region queues. The rendering module is used to perform a single composite rendering and submit the composite rendering frame.

[0055] The display delay optimization device provided in the embodiments of the present invention can execute the display delay optimization method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0056] Example 5 Figure 5 This is a schematic diagram of the structure of a terminal provided in Embodiment 5 of the present invention. Figure 5 A block diagram is shown of an exemplary terminal 12 suitable for implementing embodiments of the present invention. Figure 5 The terminal 12 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0057] like Figure 5 As shown, terminal 12 is presented in the form of a general-purpose computing terminal. The components of terminal 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0058] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0059] Terminal 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by terminal 12, including volatile and non-volatile media, removable and non-removable media.

[0060] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache 32. Terminal 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 5 Not shown; usually referred to as a "hard drive"). Although Figure 5Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0061] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0062] Terminal 12 can also communicate with one or more external terminals 14 (e.g., keyboard, pointing terminal, display 24, etc.), one or more terminals that enable a user to interact with terminal 12, and / or any terminal (e.g., network card, modem, etc.) that enables terminal 12 to communicate with one or more other computing terminals. This communication can be performed via input / output (I / O) interface 22. Furthermore, terminal 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of terminal 12 via bus 18. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with terminal 12, including but not limited to: microcode, terminal drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0063] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the display delay optimization method provided in the embodiments of the present invention.

[0064] Example 6 Embodiment 6 of the present invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the display optimization latency method as described in any of the above embodiments.

[0065] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0066] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0067] The program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0068] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or terminal. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0069] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for optimizing display latency, characterized in that, include: In the Wayland environment, the total rendering time of the previous frame is calculated; Obtain the remaining rendering time of the current frame, and determine the delay time based on the remaining rendering time, the total rendering time, and the insurance time of the current frame; The timing of submitting the current frame is determined using the aforementioned delay duration.

2. The method according to claim 1, characterized in that, The calculation yields the total rendering time of the previous frame, including: Get the CPU start time corresponding to when the rendering request was submitted; Get the CPU end time and GPU start time corresponding to when the rendering command was sent to the GPU in the previous frame; Get the GPU end time when the previous frame finished rendering; The overall rendering time is calculated based on the CPU start time, CPU end time, GPU start time, and GPU end time.

3. The method according to claim 1, characterized in that, The method further includes: Determine whether a timeout or frame drop occurred based on the effect of the previous frame's delayed rendering submission. Adjust the insurance duration of the current frame based on the timeout and frame drop phenomenon.

4. The method according to claim 1, characterized in that, The method further includes: Get the current GPU temperature, power consumption, and rendering queue length; The insurance duration for the current frame is adjusted based on the current GPU temperature, power consumption, and rendering queue length.

5. The method according to claim 1, characterized in that, The step of adjusting the insurance duration of the current frame based on the current GPU temperature, power consumption, and rendering queue length includes: When the increase in GPU power consumption exceeds the increase threshold or the queue depth exceeds the depth threshold, the insurance duration is shortened to prevent rendering timeout.

6. The method according to claim 5, characterized in that, The step of adjusting the insurance duration of the current frame based on the current GPU temperature, power consumption, and rendering queue length also includes: Extend the protection duration when the GPU is under high load or the temperature exceeds the normal temperature threshold to prevent frequent frame drops.

7. The method according to claim 1, characterized in that, The method further includes: Obtain the Wayland protocol object for display output, and add a delayed frame event to the protocol; Obtain user configuration, GPU load, input event frequency, and frame rate deviation; calculate latency evaluation value based on the user configuration, GPU load, input event frequency, and frame rate deviation. When receiving a vertical synchronization signal, the delay evaluation value is compared with a preset evaluation threshold. If the delay evaluation value is less than the preset evaluation threshold, a delay frame event is sent to the Wayland synthesizer. Upon receiving a delayed frame event, the Wayland synthesizer continues to process the input events, generate damaged regions, and merge the damaged region queue. Perform a single composite rendering and submit the composite rendering frame.

8. A device for optimizing display latency, characterized in that, include: The calculation module is used to calculate the total rendering time of the previous frame in the Wayland environment; The acquisition module is used to acquire the remaining duration of the current frame rendering and determine the delay duration based on the remaining duration, the total rendering duration, and the insurance duration of the current frame. The submission timing determination module is used to determine the submission timing of the current frame using the delay duration.

9. A terminal, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the display optimization latency method as described in any one of claims 1-7.

10. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the display optimization delay method as described in any one of claims 1-7.

Images