Display image buffer processing method, system and device in high-load scenario and medium

Abstract

The application provides a high-load scene display image buffer processing method, system, device and medium, the method comprises the following steps: determining the physical register configuration information of the display engine according to the interrupt response speed and the backup register configuration information of the last frame; configuring the display engine according to the physical register configuration information and generating a display buffer identifier; controlling whether to release the display buffer of the old frame of the display engine according to the display buffer identifier, obtaining new frame display content, and updating the backup register configuration information with the new frame display content; copying the updated backup register configuration information to the physical register to update the physical register configuration information, configuring the display engine according to the updated physical register configuration information, and displaying the content of the new frame through the updated display engine, so that smooth display can be realized in the high-load and interrupt response not timely scene, and the scheme can be widely applied to the related fields of graphic processing technology.

Description

Technical Field

[0001] This invention relates to the field of graphics processing technology, and particularly to a method, system, device, and medium for processing display image buffers under high load scenarios. Background Technology

[0002] Due to the online nature of the display engine (display composition controller), it must complete the processing and output of display data at a specific time (e.g., outputting one frame every 16.6ms for an FPS of 60Hz). In some hardware architectures, this requires the display driver to configure the hardware registers for the new frame within the vblank region. This operation is typically performed within the vblank interrupt. This makes the display driver highly dependent on the fast response of the vblank interrupt. When the system load is too high, leading to slow interrupt response, the probability of the display driver failing to update registers increases significantly. Furthermore, the register configuration also includes the image buffer data to be displayed; improper management can cause screen tearing or display artifacts.

[0003] In the relevant technical solutions, the method of sending the image to the display once per frame simplifies buffer management. Since each frame can only be sent to the display once, in a specific display call, the buffer from the previous display call is currently in use, and the buffers from the previous display call and earlier have been used up. Therefore, if the register update is correct each time the image is sent, the image buffer from the previous display call can be released to achieve the desired result.

[0004] However, the above methods cannot handle scenarios where the caller needs to execute the display call multiple times using multiple threads or even multiple processes, or even a single thread. Furthermore, since it's necessary to limit each frame to be displayed only once, other mechanisms are often required to prevent multiple display calls for a single frame, such as Android's fence synchronization mechanism or simply performing a delay operation based on the display's frame rate. These additional mechanisms all need to be implemented and ensured to be effective by the caller. This results in existing technical solutions having low buffer management flexibility and being difficult to use. Summary of the Invention

[0005] In view of this, and to address the aforementioned problems, this application proposes a display image buffering method that allows multiple displays within a single frame without requiring an additional synchronization mechanism. This method is compatible with one-frame-at-a-time usage scenarios, and can achieve smooth display under high load and untimely interruption responses, avoiding display anomalies such as screen tearing while providing greater flexibility. Furthermore, this application also provides corresponding systems, devices, and media.

[0006] On the one hand, the technical solution of this application provides a method for processing display image buffering under high load scenarios, including the following steps: determining the interrupt response speed of the software response to the first frame blanking interrupt, and determining the physical register configuration information of the display engine according to the interrupt response speed and the backup register configuration information of the previous frame;

[0007] In the software blanking interrupt response, the display engine is configured according to the physical register configuration information and a display buffer identifier is generated;

[0008] During the display call, the display buffer flag is used to control whether to release the display buffer of the old frame in the display engine, and several new frame display contents are obtained. The new frame display contents are used to update the backup register configuration information. In response to the second frame blanking interrupt, the updated backup register configuration information is copied to the physical register to update the physical register configuration information. The display engine is configured according to the updated physical register configuration information, and the new frame content is displayed by configuring the updated display engine.

[0009] Preferably, determining the interrupt response speed of the software response to the first frame blanking interrupt includes:

[0010] Obtain the current number of scan lines and the number of frame blanking lines of the display device, and determine a first difference between the number of scan lines and the number of frame blanking lines;

[0011] Obtain the register count information of the display engine and the copy time of the display engine, and determine the interrupt response speed based on the register count information, the copy time, and the first difference.

[0012] Preferably, the step of acquiring several new frame display contents and updating the backup register configuration information with the new frame display contents includes:

[0013] The display call information is determined based on the display content of the target frame, and the display call information is updated and overwritten according to the changes in the display content.

[0014] The new frame display content is obtained by integrating the updated display call information.

[0015] Preferably, the step of acquiring several new frame display contents and updating the backup register configuration information with the new frame display contents further includes:

[0016] In the second frame blanking interrupt, the physical register configuration information is confirmed based on the backup register configuration information and the interrupt response speed, and the physical register configuration information is written to the display engine physical register.

[0017] Preferably, the response to the second frame blanking interrupt includes:

[0018] The first line of visible content in the new frame is scanned until the last line of visible content in the new frame is scanned, triggering the second frame blanking interrupt.

[0019] Preferably, the method further includes:

[0020] It is determined that the display buffer identifier has not been generated or the display engine has not released the display buffer of the old frame;

[0021] The content of the new frame is displayed based on the display buffer of the old frame in the display engine.

[0022] Preferably, the frame blanking interrupt process and the process of acquiring and displaying several new frames are mutually exclusive through a spin lock; the frame blanking interrupt process includes a first frame blanking interrupt and a second frame blanking interrupt.

[0023] On the other hand, the technical solution of this application also provides a display image buffering processing system for high-load scenarios, the system comprising:

[0024] The frame blanking interrupt handling unit is used to determine the interrupt response speed of the software response to the first frame blanking interrupt, and to determine the physical register configuration information of the display engine based on the interrupt response speed and the backup register configuration information of the previous frame; and in the software blanking interrupt response, the display engine is configured according to the physical register configuration information and a display buffer identifier is generated; and in the display call, the display buffer identifier is used to control whether to release the display buffer of the old frame of the display engine.

[0025] The display call processing unit is used to obtain several new frame display contents and update the backup register configuration information with the new frame display contents;

[0026] The content display unit is used to respond to the second frame blanking interruption by copying the updated backup register configuration information to the physical register to update the physical register configuration information, configuring the display engine according to the updated physical register configuration information, and displaying the content of the new frame by configuring the updated display engine.

[0027] On the other hand, the technical solution of this application also provides a display image buffer processing device for high-load scenarios, the device comprising:

[0028] At least one processor;

[0029] At least one memory for storing at least one program;

[0030] When the at least one program is executed by the at least one processor, the at least one processor performs the image buffering method for high-load scenarios as described in any one of the first aspects.

[0031] On the other hand, the present application also provides a storage medium storing a processor-executable program, which, when executed by a processor, is used to perform the image buffering method for displaying images in a high-load scenario as described in any one of the first aspects.

[0032] The advantages and beneficial effects of the present invention will be set forth in part in the following description, and the rest will become apparent from the specific embodiments thereof:

[0033] This application proposes a display and buffer management method that allows multiple displays within a single frame without requiring an additional synchronization mechanism. The key mechanism is to respond to each display call from the upper layer as non-blockingly as possible, and to detect and release the used display buffer only in the first display call of each frame. The final display result is based on the cumulative effect of the displays. The method is compatible with the use case of one display per frame, and can achieve smooth display under high load and untimely interrupt response, avoiding display abnormalities such as screen tearing while providing greater flexibility. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart illustrating the steps of a method for processing display image buffers under high load scenarios provided in the technical solution of this application.

[0036] Figure 2 This is a timing diagram of the display and vblank interrupt in the technical solution of this application;

[0037] Figure 3 This is a flowchart of the vblank interrupt handling process in the technical solution of this application;

[0038] Figure 4 This is a flowchart of the data transmission and display call process in the technical solution of this application;

[0039] Figure 5 This is a system structure diagram of the image buffer processing system for high-load scenarios in the technical solution of this application. Detailed Implementation

[0040] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0041] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0043] Because improper management of the display image buffer in related technical solutions can cause screen tearing, display distortion, and other abnormal display phenomena, this application provides a display buffer management solution that does not rely on a synchronization mechanism, can meet the needs of single / multi-threaded multi-display calls, and can cope with untimely interrupt responses. Figure 1 As shown, in the first aspect, the solution provides a method for displaying image buffering in high-load scenarios, which may include steps S01-S04:

[0044] S01. Determine the interrupt response speed of the software response to the first frame blanking interrupt, and determine the physical register configuration information of the display engine based on the interrupt response speed and the backup register configuration information of the previous frame.

[0045] Specifically, in the embodiments, such as Figure 2 As shown, the timing flow of the strategy implemented in this invention focuses on the interactive synchronization processing between the vblank interrupt and the display call, updating the hardware by assigning values ​​to hardware registers. In this embodiment, whether the first frame blanking (vblank) interrupt successfully updates the register serves as a flag to determine whether the display buffer of the old frame is still in use, and determines whether the display buffer of the old frame should be released in the next display call. Furthermore, as... Figure 3The diagram shows the processing flow of the vblank interrupt in the embodiment. The vblank interrupt needs to determine whether to copy the new frame backup register configuration of the display configuration to the actual display engine register based on the interrupt response speed, forming the display engine configuration information, i.e., the second configuration information. The software display call needs to convert the display configuration information of the new frame into register configuration through the driver. The display engine hardware completes the processing and output of display data according to the corresponding register configuration.

[0046] S02. In the software blanking interrupt response, the display engine is configured according to the physical register configuration information and a display buffer identifier is generated. In the display send call, the display buffer identifier is used to control whether to release the display buffer of the old frame of the display engine.

[0047] Specifically, in the embodiment, after the new frame backup register configuration of the display sending configuration is copied to the actual display engine register, the flag allowing the display sending call to release the previous display buffer is set only when the register update is successful. This is because if any part of the register is not updated, it means that the display engine will continue to use the previous display buffer in the next frame. Therefore, the embodiment will only perform the buffer release operation if it detects that the flag allowing the display sending call to release the previous display buffer is valid.

[0048] S03. Obtain several new frame display contents and update the backup register configuration information with the new frame display contents.

[0049] Specifically, in the embodiments, such as Figure 4 As shown, during a display send call, the flag allowing the display send call to release the previous display buffer is detected. After the buffer release operation is completed, each subsequent display send call will directly convert the user's display request into the value of the display engine register. More specifically, in this embodiment, each display send call will ultimately be converted into the configuration of the display engine hardware registers, but it first needs to be stored in a backup register. The display engine register value is copied to the hardware and made effective by the next vblank interrupt.

[0050] S04. In response to the second frame blanking interrupt, copy the updated backup register configuration information to the physical register to update the physical register configuration information, configure the display engine according to the updated physical register configuration information, and display the content of the new frame by configuring the updated display engine.

[0051] Specifically, in this embodiment, during the next vblank interrupt, the value stored in the backup register is copied to the hardware and takes effect during the next vblank interrupt. Through the coordination of the aforementioned vblank interrupt and display call, it is ensured that the previous frame can be displayed by skipping frames when the system load is high, and the buffer that needs to be displayed is prevented from being released, thereby ensuring the effectiveness of the display buffer and the smooth and correct display.

[0052] In some feasible implementations, the step S01 of determining the interrupt response speed of the first frame blanking interrupt and determining the second configuration information of the display engine based on the interrupt response speed and the first configuration information of the backup register can further include steps S011 and S012:

[0053] S011. Obtain the current number of scan lines and the number of frame blanking lines of the display device, and determine the first difference between the number of scan lines and the number of frame blanking lines;

[0054] S012. Obtain the register count information of the display engine and the copy time of the display engine, and determine the interrupt response speed based on the register count information, the copy time and the first difference.

[0055] Specifically, in this embodiment, the SOC has internal registers that can determine the current number of scan lines on the display screen. When the number of lines exceeds the vblank line count, the display engine must begin outputting the image. Since the number of registers configured in this embodiment is known, the register copying time is also known. Therefore, by obtaining the difference between the current number of scan lines on the display screen and the vblank line count in the vblank interrupt handler, it can be determined whether there is sufficient time to update the registers; if the time is insufficient, the interrupt response speed is considered slow.

[0056] In some feasible implementations, the step S03 of the method of the embodiment, which involves obtaining several new frame display contents and updating the backup register configuration information with the new frame display contents, may include steps S031-S032:

[0057] S031. Determine the display call information based on the display content of the target frame, and update and overwrite the display call information according to the changes in the display content;

[0058] S032. The new frame display content is obtained by integrating the updated display call information.

[0059] Specifically, in this embodiment, a single display call (information) may only configure a portion of the content to be displayed in a single frame. The application can decide whether a particular frame's data is processed through multiple display calls. For example, in a video playback scenario, application layer process A is responsible for updating user interface layers such as the progress bar, while process B is responsible for updating the video layer. From the driver's perspective, this is actually a two-time display call, and the driver needs to merge the results of the two calls to display the next frame. Alternatively, in this embodiment, multiple modifications can be made by the application layer calling a new display call to overwrite the previous one. The driver will then use the final result of all display calls before the new frame arrives as the final display effect for the next frame.

[0060] In some feasible implementations, the step S03 of obtaining several new frame display contents and updating the backup register configuration information with the new frame display contents may also include step S033: in the second frame blanking interrupt, confirming the physical register configuration information according to the backup register configuration information and the interrupt response speed, and writing the physical register configuration information to the display engine physical register.

[0061] Specifically, in this embodiment, each display send call is ultimately converted into a configuration of the display engine hardware registers and stored in a backup register. The register value is copied to the hardware and takes effect by the next vblank interrupt. More specifically, the registers for the display send call are not directly assigned to the display engine hardware, but are temporarily stored in a "backup register" by the CPU. This "backup register" is actually a buffer in memory. The CPU copies the "backup register" to the actual display engine hardware registers only during the vblank interrupt handler. This operation is determined by the display engine's hardware architecture, because the vblank area does not require display and therefore does not affect normal display; thus, register configuration must be completed at this time.

[0062] In some feasible implementations, the method of the embodiment, in response to the second frame blanking interruption, copies the updated backup register configuration information to the physical register to update the physical register configuration information, configures the display engine according to the updated physical register configuration information, and displays the content of the new frame by configuring the updated display engine, can further include step S041:

[0063] S041. Scan the first line of visible content in the new frame display content until the last line of visible content in the new frame display content is scanned, triggering the second frame blanking interrupt.

[0064] Specifically, in this embodiment, a complete image scan signal in the new frame's display content is composed of a sequence of line signals separated by horizontal blanking intervals, referred to as a frame. After scanning the first line of visible content and finishing processing the last line, a vblank interrupt is generated. For a period afterward (occupying the scanning time of several lines), no visible content is updated on the display. After scanning a frame, the scan point needs to return from the lower right corner of the image to the upper left corner to begin scanning a new frame and handling the vblank interrupt.

[0065] In some feasible implementations, the image buffer processing method for high-load scenarios may further include steps S05 and S06:

[0066] S05. Determine that the display buffer identifier has not been generated or that the display engine has not released the display buffer of the old frame;

[0067] S06. Display the content of the new frame according to the display buffer of the old frame in the display engine.

[0068] Specifically, in the embodiment, the failure to generate the display buffer flag may also mean that the flag allowing the display call to release the previous display buffer is not set. The flag allowing the display call to release the previous display buffer is only set when the register update is successful, because if any part of the register is not updated, it means that the display engine in the next frame may continue to use the previous display buffer (partially or entirely).

[0069] In some feasible implementations, the frame blanking interrupt handling process and the process of acquiring and displaying several new frames are mutually exclusive through a spinlock; wherein the frame blanking interrupt handling includes the processing flow of each vblank interrupt in the embodiments.

[0070] Reference Figure 5 This invention provides a system structure diagram of a display image buffering processing system for high-load scenarios. Furthermore, this application also provides a display image buffering processing system for high-load scenarios, comprising:

[0071] The frame blanking interrupt handling unit is used to determine the interrupt response speed of the software response to the first frame blanking interrupt, and to determine the physical register configuration information of the display engine based on the interrupt response speed and the backup register configuration information of the previous frame; and in the software blanking interrupt response, the display engine is configured according to the physical register configuration information and a display buffer identifier is generated; and in the display call, the display buffer identifier is used to control whether to release the display buffer of the old frame of the display engine.

[0072] The display call processing unit is used to obtain several new frame display contents and update the backup register configuration information with the new frame display contents;

[0073] The content display unit is used to respond to the second frame blanking interruption by copying the updated backup register configuration information to the physical register to update the physical register configuration information, configuring the display engine according to the updated physical register configuration information, and displaying the content of the new frame by configuring the updated display engine.

[0074] On the other hand, the present application also provides a display image buffering processing device for high-load scenarios, the device comprising: at least one processor; at least one memory for storing at least one program; when the at least one program is executed by the at least one processor, the at least one processor runs the display image buffering processing method for high-load scenarios as described in the first aspect.

[0075] This invention also provides a storage medium storing a corresponding executable program, which is executed by a processor to implement the image buffering method for high-load scenarios in the first aspect.

[0076] From the above specific implementation process, it can be concluded that the technical solution provided by the present invention has the following advantages or strengths compared with the prior art:

[0077] 1. The key point of this application is to propose and implement a strategy for managing the lifecycle of a display image buffer in complex scenarios; and to achieve a flexible and stable display effect with single / multi-threaded multi-display in scenarios with high system load or insufficient CPU performance.

[0078] 2. The technical solution of this application, while being compatible with the aforementioned prior art, implements a driving strategy that can satisfy multiple displays of a single frame, reducing the difficulty of using the display driver, providing greater ease of use, and further expanding the processing scenarios of the display HAL layer.

[0079] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0080] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0081] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0082] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0083] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A method for display image buffer processing in a high load scenario, characterized in that, Includes the following steps: Determine the interrupt response speed of the software in response to the first frame blanking interrupt, and determine the physical register configuration information of the display engine based on the interrupt response speed and the backup register configuration information of the previous frame. In the software blanking interrupt response, the display engine is configured according to the physical register configuration information and a display buffer identifier is generated; During the display call, the display buffer flag is used to control whether to release the display buffer of the old frame in the display engine, and several new frame display contents are obtained. The new frame display contents are used to update the backup register configuration information. In response to the second frame blanking interrupt, the updated backup register configuration information is copied to the physical register to update the physical register configuration information. The display engine is configured according to the updated physical register configuration information, and the new frame content is displayed by configuring the updated display engine.

2. The method of claim 1, wherein, The determination of the interrupt response speed for the software to respond to the first frame blanking interrupt includes: Obtain the current number of scan lines and the number of frame blanking lines of the display device, and determine a first difference between the number of scan lines and the number of frame blanking lines; Obtain the register count information of the display engine and the copy time of the display engine, and determine the interrupt response speed based on the register count information, the copy time, and the first difference.

3. The method of claim 1, wherein, The step of acquiring several new frame display contents and updating the backup register configuration information with the new frame display contents includes: The display call information is determined based on the display content of the target frame, and the display call information is updated and overwritten according to the changes in the display content. The new frame display content is obtained by integrating the updated display call information.

4. The method of claim 3, wherein, The step of acquiring several new frame display contents and updating the backup register configuration information with the new frame display contents further includes: In the second frame blanking interrupt, the physical register configuration information is confirmed based on the backup register configuration information and the interrupt response speed, and the physical register configuration information is written to the display engine physical register.

5. The method of claim 1, wherein, The response to the blanking interrupt in the second frame includes: The first line of visible content in the new frame is scanned until the last line of visible content in the new frame is scanned, triggering the second frame blanking interrupt.

6. The method of claim 1, wherein, The method further includes: It is determined that the display buffer identifier has not been generated or the display engine has not released the display buffer of the old frame; The content of the new frame is displayed based on the display buffer of the old frame in the display engine.

7. The method of claim 1-6, wherein, The frame blanking interrupt handling process and the process of acquiring and displaying several new frames are mutually exclusive through a spin lock; the frame blanking interrupt handling includes a first frame blanking interrupt and a second frame blanking interrupt.

8. A display image buffer processing system in a high load scenario, characterized by, include: The frame blanking interrupt handling unit is used to determine the interrupt response speed of the software response to the first frame blanking interrupt, and to determine the physical register configuration information of the display engine based on the interrupt response speed and the backup register configuration information of the previous frame; and in the software blanking interrupt response, the display engine is configured according to the physical register configuration information and a display buffer identifier is generated; and in the display call, the display buffer identifier is used to control whether to release the display buffer of the old frame of the display engine. The display call processing unit is used to obtain several new frame display contents and update the backup register configuration information with the new frame display contents; The content display unit is used to respond to the second frame blanking interruption by copying the updated backup register configuration information to the physical register to update the physical register configuration information, configuring the display engine according to the updated physical register configuration information, and displaying the content of the new frame by configuring the updated display engine.

9. An apparatus for display image buffer processing in a high load scenario, characterized by, The device includes: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor performs the image buffering method for high-load scenarios as described in any one of claims 1-7.

10. A storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to run the image buffer processing method for displaying images in a high-load scenario as described in any one of claims 1-7.

Images