Screen display method and device, terminal, storage medium and product

Abstract

The application provides a screen display method and device, a terminal, a storage medium and a product, and belongs to the technical field of terminals. The method comprises the following steps: acquiring a first sending time and a reading duration of a first vertical synchronization signal, the first vertical synchronization signal being used for indicating rendering data of a first image frame which has been cached in a first image buffer; determining an end time of reading the rendering data of the first image frame based on the first sending time and the reading duration; in the case that a second vertical synchronization signal is received and when the current time reaches the end time, rendering image data of a second image frame in the first image buffer to obtain rendering data of the second image frame and cache the rendering data of the second image frame in the first image buffer, and performing image display based on the rendering data of the second image frame in the first image buffer; the second vertical synchronization signal is used for indicating rendering of the image data of the second image frame in the first image buffer, and the screen display effect is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of terminals, in particular to a screen display method and device, a terminal, a storage medium and a product. BACKGROUND

[0002] With the development of terminal technology, the handwriting function of the terminal is becoming more and more powerful. Among them, the display system of the terminal generally includes three parts of a central processing unit (CPU), a graphics processing unit (GPU) and a display (Display), the CPU is responsible for determining the image data of the image frame in the handwriting process, and the image data of the image frame is given to the GPU, the GPU is responsible for rendering the image data in the image buffer to obtain rendering data, and then the Display is responsible for reading the rendering data in the buffer and presenting the rendering data to the screen.

[0003] However, in the handwriting process, the content displayed on the screen needs to be constantly refreshed, if the buffer is simultaneously read and rendered, the data in the buffer that has not been completed rendering will be read, and then the screen is refreshed based on the data, so that part of the screen still displays the old content, and another part of the screen has displayed the new content, that is, the screen is torn, thereby reducing the display effect of the screen. SUMMARY

[0004] The embodiments of the present application provide a screen display method, device, terminal, storage medium and product, which can improve the display effect of the screen. The technical solution is as follows:

[0005] On the one hand, a screen display method is provided, the method comprising:

[0006] obtaining a first sending time of a first vertical synchronization signal and a reading duration, the first vertical synchronization signal being used to indicate reading rendering data of a first image frame which has been cached in the first image buffer, and the reading duration being a duration required for reading the rendering data of the first image frame;

[0007] determining an end time of reading the rendering data of the first image frame based on the first sending time and the reading duration;

[0008] in the case of receiving a second vertical synchronization signal, and when the current time reaches the end time, rendering image data of a second image frame in the first image buffer to obtain rendering data of the second image frame and cache it in the first image buffer, and performing image display based on the rendering data of the second image frame in the first image buffer;

[0009] The second vertical synchronization signal is used to instruct rendering of image data of the second image frame in the first image buffer, the first image frame being a previous image frame of the second image frame.

[0010] In another aspect, a screen display device is provided, the device comprising:

[0011] an obtaining module, configured to obtain a first sending time of a first vertical synchronization signal and a reading duration, the first vertical synchronization signal being used to instruct reading of rendering data of a first image frame that has been buffered in a first image buffer, the reading duration being a duration required for reading the rendering data of the first image frame;

[0012] a determining module, configured to determine an ending time of reading the rendering data of the first image frame based on the first sending time and the reading duration;

[0013] a first rendering module, configured to, in a case where a second vertical synchronization signal is received and when a current time reaches the ending time, render image data of a second image frame in the first image buffer to obtain rendering data of the second image frame and buffer the rendering data of the second image frame in the first image buffer, and perform image display based on the rendering data of the second image frame in the first image buffer.

[0014] The second vertical synchronization signal is used to instruct rendering of image data of the second image frame in the first image buffer, the first image frame being a previous image frame of the second image frame.

[0015] In another aspect, a terminal is provided, the computer device comprising one or more processors and one or more memories, the one or more memories storing at least one piece of program code, the at least one piece of program code being loaded and executed by the one or more processors to implement the screen display method according to any of the above implementation manners.

[0016] In another aspect, a computer readable storage medium is provided, the computer readable storage medium storing at least one piece of program code, the at least one piece of program code being loaded and executed by a processor to implement the screen display method according to any of the above implementation manners.

[0017] In another aspect, a computer program product is provided, the computer program product storing at least one piece of program code, the at least one piece of program code being used to be executed by a processor to implement the screen display method according to any of the above implementation manners.

[0018] The embodiment of the present application provides a screen display method, which determines the end time of reading the rendering data of a first image frame in a first buffer, and then renders the image data of a second image frame in the first image buffer when the current time reaches the end time, so that the image data of the current image frame is rendered after the rendering data of the previous image frame is read, the situation of simultaneously reading and rendering in the first image buffer is avoided, the screen tearing caused by reading the data which has not been rendered is avoided, and the display effect of the screen is improved. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 A schematic diagram of an implementation environment shown by an example embodiment of the present application is shown;

[0020] Figure 2 A flow chart of a screen display method shown by an example embodiment of the present application is shown;

[0021] Figure 3 A flow chart of a screen display method shown by an example embodiment of the present application is shown;

[0022] Figure 4 A schematic diagram of screen display shown by an example embodiment of the present application is shown;

[0023] Figure 5 A schematic diagram of screen display shown by an example embodiment of the present application is shown;

[0024] Figure 6 A flow chart of a screen display method shown by an example embodiment of the present application is shown;

[0025] Figure 7 A schematic diagram of screen display shown by an example embodiment of the present application is shown;

[0026] Figure 8 A block diagram of a screen display device shown by an example embodiment of the present application is shown;

[0027] Figure 9 A block diagram of a terminal shown by an example embodiment of the present application is shown. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical scheme and advantages of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0029] The "multiple" mentioned in the present document refers to two or more. The "and / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A existing alone, A and B existing together, and B existing alone. The character " / " generally represents an "or" relationship between the associated objects. In addition, the related data involved in the present application can be data authorized by the user or fully authorized by all parties.

[0030] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the present application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions. For example, the rendering data involved in the present application is obtained under full authorization.

[0031] Please refer to Figure 1 , which shows a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application. The implementation environment involves a terminal 100, which includes a hardware layer 110 and an application layer 120, and the hardware layer 110 and the application layer 120 can transmit data.

[0032] In some embodiments, the hardware layer 110 sends a vertical synchronization signal to the application layer 120, and the vertical synchronization signal is used to indicate that the image data of the image frame is rendered in the image buffer; after receiving the vertical synchronization signal, the application layer 120 renders the image data of the image frame in the image buffer to obtain rendering data, and then displays the image by reading the rendering data in the image buffer.

[0033] In some embodiments, the application layer 120 runs a target application, and receives the vertical synchronization signal and renders the image data through the target application. The target application runs in the form of APP or embedded program, and the target application can be a drawing application or an application for inputting information by handwriting, so that the user realizes the handwriting function of the terminal 100 by drawing or handwriting to display the image; the target application can also be an application that needs to realize screen refresh through touch operation, such as a game application; the target application can also be an application that automatically performs screen refresh, such as a video playing application, etc., which is not limited here.

[0034] Among them, the terminal 100 is a smart phone, a tablet computer, a notebook computer, a desktop computer, etc., but is not limited thereto.

[0035] Please refer to Figure 2Fig. 1 is a flowchart showing a screen display method according to an example embodiment of the present application. The execution subject in the example embodiment of the present application is a terminal, and the method comprises the following steps.

[0036] 201. The terminal acquires a first sending time of a first vertical synchronization signal and a reading duration.

[0037] In the example embodiment of the present application, the first vertical synchronization signal (HW_VSYNC) is used to instruct reading of rendering data of a first image frame that has been buffered in a first image buffer, and the reading duration is the duration required for reading the rendering data of the first image frame. The terminal reads the rendering data of the first image frame to perform synthesis of the rendering data, and then performs image display based on the synthesized data.

[0038] The rendering data of the first image frame in the first image buffer is obtained by rendering image data of the first image frame in the first image buffer by the terminal in response to a vertical synchronization signal that is used to instruct rendering of the image data of the first image frame in the first image buffer, and the sending time of the vertical synchronization signal is earlier than the first sending time. The first vertical synchronization signal is sent at the first sending time after the rendering of the image data of the first image frame is completed.

[0039] It should be noted that the on-screen time of the terminal, i.e., the screen refresh period, is fixed, i.e., the terminal needs to refresh the screen content once every fixed screen refresh period. Since the terminal reads the rendering data in the image buffer to perform image display in response to the first vertical synchronization signal, the screen content is refreshed. Obviously, the on-screen time is associated with the first vertical synchronization signal, and accordingly, the first sending time of the first vertical synchronization signal is fixed, so that the terminal refreshes the screen content at the fixed on-screen time.

[0040] 202. The terminal determines an end time of reading the rendering data of the first image frame based on the first sending time and the reading duration.

[0041] In the example embodiment of the present application, the terminal takes the sum of the first sending time and the reading duration as the end time. Since the terminal reads the rendering data of the first image frame in the first image buffer immediately after receiving the first vertical synchronization signal, accordingly, the start time of reading the rendering data of the first image frame is the same as the first sending time, and the end time of reading the rendering data of the first image frame determined based on the first sending time and the reading duration is highly accurate.

[0042] 203. When the terminal receives the second vertical synchronization signal and when the current time reaches the end time, it renders the image data of the second image frame in the first image buffer to obtain the rendering data of the second image frame and caches it in the first image buffer, and displays the image based on the rendering data of the second image frame in the first image buffer.

[0043] In this embodiment, after the terminal renders the image data of the second image frame in the first image buffer, the resulting rendered data is cached in the first image buffer, and then the cached rendered data is read from the first image buffer for image display. The second vertical synchronization signal (VSYNC_APP) is used to indicate that the image data of the second image frame is rendered in the first image buffer, where the first image frame is the previous image frame of the second image frame.

[0044] This application provides a screen display method. The method determines the end time of reading the rendering data of the first image frame in the first buffer, and then renders the image data of the second image frame in the first image buffer only when the current time reaches the end time. Since the first image frame is the previous image frame of the second image frame, this realizes that the rendering data of the current image frame is rendered only after the rendering data of the previous image frame is finished. This avoids the situation where data reading and rendering are performed simultaneously in the first image buffer, thereby avoiding screen tearing caused by reading data that has not yet been rendered, and thus improving the display effect of the screen.

[0045] Please refer to Figure 3 The document illustrates a flowchart of a screen display method according to an exemplary embodiment of this application. This embodiment uses screen display via an image buffer as an example, and the method includes:

[0046] 301. The application layer obtains the time phase offset between the first vertical synchronization signal and the second vertical synchronization signal, and obtains the reading duration.

[0047] It should be noted that, since the data amount of the rendering data of different image frames can be different, that is, the memory size occupied by the first image buffer is different, thereby causing the actual reading duration of the terminal to the rendering data of different image frames to be different; optionally, the terminal determines the reading duration based on the maximum memory of the first image buffer or the maximum memory of the rendering data that can be occupied by the first image buffer determined based on historical experience; for example, the terminal takes the duration corresponding to the rendering data of the maximum memory of the first image buffer as the reading duration; or the terminal takes the reading duration corresponding to the maximum memory of the rendering data occupied by the first image buffer as the reading duration, thereby ensuring that the rendering data of most image frames can be completely read within the reading duration. Optionally, the reading duration is also related to the performance of the terminal; wherein the better the performance of the terminal in reading the rendering data in the first image buffer, that is, the shorter the time for reading the rendering data, the shorter the reading duration.

[0048] Wherein, the time phase offset can be directly obtained from the configuration data of the terminal.

[0049] 302、The application layer determines the first sending time based on the time phase offset and the second sending time of the second vertical synchronization signal.

[0050] Optionally, if the first sending time is a time before the second sending time, the terminal takes the difference between the second sending time and the time phase offset as the first sending time. If the first sending time is a time after the second sending time, the terminal takes the sum of the second sending time and the time phase offset as the first sending time.

[0051] 303、The application layer determines the end time of reading the rendering data of the first image frame based on the first sending time and the reading duration.

[0052] In the embodiment of the present application, the application layer takes the sum of the first sending time and the reading duration as the end time.

[0053] In the embodiment of the present application, the first sending time of the first vertical synchronization signal is determined based on the second sending time of the second vertical synchronization signal and the time phase offset, thereby enabling the end time of reading the rendering data of the first image frame to be determined based on the first sending time and the reading duration, and thereby based on the end time, it is convenient to subsequently render the image data of the second image frame after the rendering data of the first image frame is read.

[0054] 304、The hardware layer sends the second vertical synchronization signal to the application layer.

[0055] Wherein, the application layer runs a target application, and optionally, the hardware layer sends the second vertical synchronization signal to the target application. Correspondingly, the application layer executes step 305 through the target application.

[0056] 305、application layer, in case of receiving the second vertical synchronization signal, and when the current time reaches the end time, renders the image data of the second image frame in the first image buffer, and displays the image based on the rendering data of the rendered second image frame.

[0057] The second vertical synchronization signal is used to indicate that the image data of the second image frame is rendered in the first image buffer, and the first image frame is the previous image frame of the second image frame.

[0058] Optionally, the application layer calls the image processor to render the image data of the second image frame in the first image buffer. The terminal reads the rendering data of the second image frame in the first image buffer through a hardware compositor (hwcomposer) to display the image; optionally, the hardware compositor is a compositor of the hardware layer of the terminal.

[0059] In some embodiments, the application layer, in response to the second vertical synchronization signal, generates rendering instructions based on the instruction data of the second image frame corresponding to the second vertical synchronization signal, the rendering instructions being used to indicate the step of rendering the image data of the second image frame in the first image buffer.

[0060] Please refer to Figure 4 which shows a schematic diagram of screen display according to an example embodiment of the present application, Figure 4 The left side is a schematic diagram of ideal screen display; wherein the target application of the application layer, in response to the second vertical synchronization signal sent by the terminal, calls the CPU to generate rendering instructions, the rendering instructions being used to indicate that the image data of the second image frame is rendered in the first image buffer. Then the target application, in response to the rendering instructions, calls the GPU to render the image data of the second image frame in the first buffer; then the hardware compositor, in response to the fourth vertical synchronization signal sent by the terminal, reads the rendering data of the second image frame in the first image buffer; wherein the fourth vertical synchronization signal is used to indicate that the rendering data of the second image frame in the first image buffer is read. It should be noted that the terminal can perform hardware composition on the read rendering data through the hardware compositor, or perform software composition on the read rendering data through a display composition system (SurfaceFlinger), and in the embodiment of the present application, the hardware composition on the read rendering data through the hardware compositor is taken as an example for description. However, before the screen is refreshed, i.e. when the hardware compositor starts to read the rendering data in the first image buffer, the rendering process is not completed, so the rendering data read by the hardware compositor is incomplete, as shown in Figure 4The right side shows a schematic diagram of screen display that can cause screen tearing. Therefore, in the embodiments of the present application, a read-after-write screen display method is provided, that is, after the rendering data of the previous image frame in the first image buffer is read, the image data of the current image frame is rendered in the first image buffer, thereby avoiding the screen tearing caused by the simultaneous reading and rendering operations in a single image buffer.

[0061] It should be noted that in some cases, the terminal cannot complete the rendering of the image data of the image frame within the rendering time window allocated for it, and thus there can be a case where the terminal has started reading the rendering data of the image frame before the rendering of the image data of the image frame is completed. Since the total time length for the terminal to generate the rendering instruction and render the image data is relatively fixed, in some embodiments, the terminal generates the rendering instruction by the following steps: in a case where the terminal receives the second vertical synchronization signal within the target time period, the terminal generates the rendering instruction based on the instruction data of the second image frame corresponding to the second vertical synchronization signal.

[0062] The starting time of the target time period is the first sending time, and the time length of the target time period is the reading time length. Please refer to Figure 5 The right side shows a schematic diagram of screen display that can cause screen tearing. Therefore, in the embodiments of the present application, a read-after-write screen display method is provided, that is, after the rendering data of the previous image frame in the first image buffer is read, the image data of the current image frame is rendered in the first image buffer, thereby avoiding the screen tearing caused by the simultaneous reading and rendering operations in a single image buffer.

[0063] In this embodiment, since the total time length for the terminal to generate the rendering instruction and render the rendering data is relatively fixed, and the time length for generating the rendering instruction is always less than the time length for reading the rendering data of the first image frame, by generating the rendering instruction within the time period for reading the rendering data of the first image frame, the process of generating the rendering instruction can be completed during the process of reading the rendering data of the first image frame, so that the total time length for generating the rendering instruction and rendering the data can be used for rendering the data, thereby increasing the time length for rendering the image data of the image frame, and further avoiding the case where the terminal has started reading the rendering data of the image frame before the rendering of the image data of the image frame is completed, i.e., further avoiding the screen tearing, and improving the screen display effect.

[0064] It should be noted that since the rendering data of the previous image frame has been stored in the first image buffer, the image content of the current image frame and the previous image frame is partially the same, that is, the rendering data for image display of the two is partially the same, and then only the image data different from the image data of the previous image frame in the image frame needs to be rendered; therefore, in some embodiments, the application layer renders the image data of the second image frame in the first image buffer, and performs image display based on the rendering data of the second image frame obtained by rendering, including the following steps: the application layer determines first image data and second image data in the second image frame, the first image data being image data different from the first image frame in the second image frame, and the second image data being image data same as the first image frame in the second image frame; the application layer renders the first image data to obtain first rendering data; the application layer obtains second rendering data corresponding to the second image data from the rendering data of the first image frame; the application layer obtains the rendering data of the second image frame based on the first rendering data and the second rendering data; and the application layer performs image display based on the rendering data of the second image frame.

[0065] In this embodiment, by rendering only the image data different from the previous image frame, only the newly added image data needs to be rendered each time, unnecessary rendering is reduced, partial update of the rendering data in the first image buffer is realized, full rendering of the image data is avoided, the rendering time length of the image data of the image frame is reduced, it is ensured that the terminal can complete the rendering of the image data of the image frame within the rendering time window allocated thereto, the screen tearing situation is avoided, and the display effect of the screen is improved.

[0066] In some embodiments, the second image frame is multiple, the application layer renders the image data of the second image frame in the first image buffer, and performs image display based on the rendering data of the second image frame obtained by rendering, including the following steps: the application layer obtains priority levels of multiple second image frames; the application layer determines a processing order of the multiple second image frames based on the priority levels of the multiple second image frames; and the application layer sequentially renders the multiple second image frames in the first image buffer based on the processing order of the multiple second image frames.

[0067] In this embodiment, by determining the priority levels of the multiple image frames, the multiple image frames are processed based on the processing order corresponding to the priority levels, so that the image frames with high priority can be processed preferentially, and the waiting delay of data rendering is reduced.

[0068] In an implementation manner, the application layer renders the plurality of second image frames in the first image buffer in sequence based on a processing order of the plurality of second image frames, including the following steps: when the processing time of a second image frame is reached, the application layer associates the second image frame with a rendering interface, and the rendering interface is used to call a GPU; the application layer calls the GPU through the rendering interface, and renders the second image frame in the first image buffer through the GPU.

[0069] Optionally, the rendering interface is an Enterprise Generation Language (EGL) interface; and the rendering interface provides a service of setting a priority through a context thereof, so as to implement processing of the plurality of second image frames in sequence based on priorities of the plurality of second image frames.

[0070] In some other embodiments, only part of the plurality of second image frames needs to be set with a priority, that is, the priority of the part of the second image frames is higher than a preset level, and the part of the second image frames needs to be processed preferentially; the rendering interface sets the part of the second image frames as high priority through a context thereof, and then submits a rendering instruction of the second image frame through the context of the rendering interface, so that the rendering instruction submitted based on the context can be processed preferentially, that is, image data corresponding to the rendering instruction can be rendered preferentially, thereby reducing a processing waiting time delay of the rendering instruction of the high priority.

[0071] It should be noted that, in the embodiments of the present application, by associating the second image frame with the rendering interface, the GPU can be called in sequence based on the priority of the plurality of second image frames to render the second image frame through the rendering interface, so that the image frame of the high priority can be processed preferentially, thereby improving the flexibility of rendering the image data.

[0072] The embodiments of the present application provide a screen display method, which determines an end time of reading rendering data of a first image frame in a first buffer, and then renders image data of a second image frame in the first image buffer when a current time reaches the end time, because the first image frame is a previous image frame of the second image frame, so that the image data of the current image frame is rendered only after the rendering data of the previous image frame is read, thereby avoiding the case of reading and rendering at the same time in the first image buffer, and further avoiding the case of screen tearing caused by reading the data which has not been completed rendering, thereby improving the display effect of the screen.

[0073] Please refer to Figure 6 which shows a flowchart of a screen display method according to an example embodiment of the present application. The embodiment takes the screen display through a plurality of image buffers as an example for description, and the method includes the following steps:

[0074] 601, the application layer acquires a time phase offset between the first vertical synchronization signal and the second vertical synchronization signal, and acquires a reading duration.

[0075] 602, the application layer determines a first sending time based on the time phase offset and a second sending time of the second vertical synchronization signal.

[0076] 603, the application layer determines an end time of reading the rendering data of the first image frame based on the first sending time and the reading duration.

[0077] 604, the hardware layer sends the second vertical synchronization signal to the application layer.

[0078] 605, the application layer, in a case where the second vertical synchronization signal is received, and when a current time reaches the end time, renders image data of a second image frame in the first image buffer, and performs image display based on rendering data of the second image frame obtained by the rendering.

[0079] Steps 601-605 are the same as steps 301-305, and will not be described here.

[0080] 606, the hardware layer sends a fifth vertical synchronization signal to the application layer.

[0081] The fifth vertical synchronization signal is used to instruct to render image data of a third image frame in the second image buffer.

[0082] 607, the application layer, in a case where the fifth vertical synchronization signal is received, renders image data of the third image frame in the second image buffer while reading the rendering data of the second image frame in the first image buffer, to obtain rendering data of the third image frame.

[0083] The third image frame is a next image frame of the second image frame, and the second image buffer is a different image buffer from the first image buffer. In the embodiment of the application, the image data of the next image frame is rendered in the second image buffer while reading the rendering data of the second image frame in the first image buffer, so that parallel processing of multiple image frames is realized through multiple image buffers, and the processing efficiency of the image frames is improved.

[0084] 608, the hardware layer sends a third vertical synchronization signal to the hardware synthesizer.

[0085] The third vertical synchronization signal is used to instruct to read the rendering data of the third image frame in the second image buffer.

[0086] 609、The hardware compositor reads the rendering data of the third image frame in the second image buffer in the case that the third vertical synchronization signal is received, and performs image display based on the rendering data of the third image frame.

[0087] It should be noted that, for the convenience of description, in the embodiments of the present application, the mode of rendering only through the first image buffer is referred to as the single image buffer rendering mode, and the mode of rendering through the first image buffer and the second image buffer is referred to as the multi image buffer rendering mode.

[0088] Optionally, the next image frame of the third image frame is rendered in the first image buffer, i.e., the multi image buffer rendering mode is maintained; or the next image frame of the third image frame is rendered in the second image buffer after the rendering data of the third image frame is read, i.e., the single image buffer rendering mode is switched to, which is not limited here.

[0089] Please refer to Figure 7 which shows a schematic diagram of screen display according to the rendering mode of the image buffer, as shown in an example embodiment of the present application. Wherein, the image data of the Nth image frame is rendered in the case that the target application receives a vertical synchronization signal, which is used to indicate that the image data of the Nth image frame is rendered in the second image buffer; after the rendering is completed, the second image buffer is functionally converted, and the rendering of the image data is no longer performed, but the result of the rendering is submitted to the display composition system; then in the case that the display composition system receives a vertical synchronization signal, the rendering data of the third image frame in the second image buffer is read by the hardware compositor to perform the composition of the rendering data, which is used to indicate that the rendering data of the third image frame in the second image buffer is read; after the composition is completed, the result of the composition is submitted to the kernel (CRTC) by the hardware compositor, and in the case that the kernel receives a vertical synchronization signal, the composed data of the Nth frame is displayed on the screen. And Figure 7It can be known that, by using multiple image buffers to display images, there is a delay of at least two frames for displaying each image frame, that is, the display delay of the image frame is high. In the embodiment of the present application, the screen is displayed by using a single image buffer. The image data is rendered to the single image buffer, and then the rendering data is read from the image buffer. When the hardware compositor receives each vertical synchronization signal, the rendering data in the image buffer can be read for composition, regardless of whether the window changes. Thus, the image on-screen speed can be accelerated, and the image display delay can be reduced. It should be noted that, after the target application renders the image data in the second image buffer and submits the rendering result to the display composition system, the rendering data can be read, thereby avoiding screen tearing.

[0090] It should be noted that the rendering interface in the embodiment of the present application supports both the single image buffer rendering mode and the multiple image buffer rendering mode. Thus, the rendering mode can be controlled by controlling the properties of the surface hardware (Surface), thereby facilitating flexible switching between the two rendering modes.

[0091] In the embodiment of the present application, the single image buffer rendering mode and the multiple image buffer rendering mode are combined to balance the display delay and the screen tearing problem, thereby effectively avoiding the screen tearing problem while reducing the screen display delay.

[0092] It should be noted that the single image buffer rendering mode is suitable for a scenario with high display delay requirement, and the multiple image buffer rendering mode is suitable for a scenario with low display delay requirement. Therefore, in some embodiments, when the display delay requirement of the second image frame is higher than that of the third image frame, the step of rendering the image data of the third image frame in the second image buffer to obtain the rendering data of the third image frame is performed.

[0093] Optionally, the display delay requirement is a preset display delay length, such as 0.1 ms, 1 ms, etc. The display delay length can be set and changed as needed, and is not limited herein.

[0094] It should be noted that the image frame with high display delay requirement generally has short image data rendering time, small rendering pressure, and thus the fast and accurate rendering of image data can be realized based on a single image buffer. For example, in a handwriting scenario, the rendering content of each frame during writing is very short, i.e., the rendering data amount is small, the rendering pressure is small, and the display delay requirement is high, and thus the single image buffer rendering mode is suitable. For the image frame with low display delay requirement, the rendering data needs to be updated, the rendering pressure is large, and thus the rendering of image data of multiple image frames cannot be completed in the rendering window through the single image buffer, resulting in screen tearing. Therefore, the multi-image buffer rendering mode is suitable to avoid screen tearing. For example, editing of the strokes of writing or the up and down sliding of the canvas in the writing scenario is suitable for the multi-image buffer rendering mode.

[0095] In the embodiments of the present application, the rendering mode is adjusted based on the display delay requirement of the image frame, so that the image frame with high display delay requirement is displayed on the screen in the single image buffer rendering mode, the display delay is reduced, and the image frame with low display delay requirement is displayed on the screen in the multi-image buffer rendering mode, the image data is fully rendered, the screen tearing is effectively avoided, and the display effect of the screen is improved.

[0096] Please refer to Figure 8 which shows a block diagram of a screen display device according to an example embodiment of the present application, and the device includes:

[0097] The obtaining module 801 is configured to obtain a first sending time of a first vertical synchronization signal and a reading time length, the first vertical synchronization signal being used to indicate reading of rendering data of a first image frame that has been buffered in a first image buffer, and the reading time length being a time length required for reading the rendering data of the first image frame;

[0098] The determining module 802 is configured to determine an end time of reading the rendering data of the first image frame based on the first sending time and the reading time length.

[0099] The first rendering module 803 is configured to, in the case of receiving a second vertical synchronization signal and when the current time reaches the end time, render image data of a second image frame in the first image buffer to obtain rendering data of the second image frame and buffer the rendering data of the second image frame in the first image buffer, and perform image display based on the rendering data of the second image frame in the first image buffer.

[0100] The second vertical synchronization signal is used to indicate rendering of the image data of the second image frame in the first image buffer, and the first image frame is a previous image frame of the second image frame.

[0101] In some embodiments, the device further includes:

[0102] read the image data of the third image frame in the second image buffer to obtain the rendering data of the third image frame, the third image frame being a next image frame of the second image frame, and the second image buffer being a different image buffer from the first image buffer;

[0103] read the rendering data of the third image frame in the second image buffer based on the third image frame, the third vertical synchronization signal being used to instruct reading the rendering data of the third image frame in the second image buffer.

[0104] In some embodiments, the second rendering module is configured to perform the step of rendering the image data of the third image frame in the second image buffer to obtain the rendering data of the third image frame, in a case where a display latency requirement of the second image frame is higher than a display latency requirement of the third image frame.

[0105] In some embodiments, the apparatus further comprises:

[0106] The first generating module is configured to, in response to the second vertical synchronization signal, generate the rendering instruction based on instruction data of the second image frame corresponding to the second vertical synchronization signal, the rendering instruction being used to instruct the first rendering module to render the image data of the second image frame in the first image buffer; or

[0107] The first generating module is configured to, in a case where the second vertical synchronization signal is received within a target time period, generate the rendering instruction based on instruction data of the second image frame corresponding to the second vertical synchronization signal, a start time of the target time period being the first sending time, and a time length of the target time period being the reading time length.

[0108] In some embodiments, the obtaining module 801 is configured to:

[0109] obtain a time phase offset between the first vertical synchronization signal and the second vertical synchronization signal;

[0110] determine the first sending time based on the time phase offset and a second sending time of the second vertical synchronization signal.

[0111] In some embodiments, the first rendering module 803 is configured to:

[0112] determine first image data and second image data in the second image frame, the first image data being image data in the second image frame that is different from the first image frame, and the second image data being image data in the second image frame that is the same as the first image frame;

[0113] render the first image data to obtain first rendering data;

[0114] obtain second rendering data corresponding to the second image data from the rendering data of the first image frame;

[0115] obtain rendering data of the second image frame based on the first rendering data and the second rendering data;

[0116] perform image display based on the rendering data of the second image frame.

[0117] In some embodiments, the second image frame is multiple, and rendering image data of the second image frame in the first image buffer and performing image display based on rendering data of the second image frame obtained by rendering include:

[0118] obtain priority levels of the multiple second image frames;

[0119] determine processing orders of the multiple second image frames based on the priority levels of the multiple second image frames;

[0120] render the multiple second image frames in the first image buffer in sequence based on the processing orders of the multiple second image frames.

[0121] In some embodiments, the rendering module is configured to:

[0122] when the processing time of the second image frame is reached, associate the second image frame with a rendering interface, and the rendering interface is configured to call a graphics processing unit (GPU);

[0123] call the GPU through the rendering interface, and render the second image frame in the first image buffer through the GPU.

[0124] Embodiments of the present application provide a screen display device. By determining an end time of reading rendering data of a first image frame in a first buffer, and then rendering image data of a second image frame in the first image buffer when a current time reaches the end time, since the first image frame is a previous image frame of the second image frame, it is achieved that the image data of the current image frame is rendered only after the rendering data of the previous image frame is read, the situation of simultaneously reading data and rendering in the first image buffer is avoided, and the situation of screen tearing caused by reading data that has not been completed rendering is avoided, thereby improving the display effect of the screen.

[0125] It should be noted that the screen display device provided in the above embodiments is only used as an example to illustrate the division of the above functional modules. In actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the terminal is divided into different functional modules to complete all or part of the functions described above. In addition, the screen display device and the screen display method provided in the above embodiments belong to the same concept, and the specific implementation process is described in the method embodiments, which will not be repeated here.

[0126] Please refer to Figure 9 which shows a structural block diagram of a terminal 900 provided by an example embodiment of the present application. The terminal 900 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc. The terminal 900 in the present application can include one or more of the following components: a processor 910, a memory 920, and a display screen 930.

[0127] The processor 910 can include one or more processing cores. The processor 910 connects various parts in the entire terminal 900 through various interfaces and lines, and performs various functions of the terminal 900 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 920, and calling data stored in the memory 920. Optionally, the processor 910 can be implemented in at least one of the following hardware forms: digital signal processing (DSP), field programmable gate array (FPGA), programmable logic array (PLA). The processor 910 can integrate a combination of one or several of the following: central processing unit, image processor, neural network processor (NPU), and modem. Among them, the CPU mainly processes operating systems, user interfaces, and application programs, etc.; the GPU is responsible for rendering and drawing the content to be displayed by the display screen 930; the NPU is used to implement artificial intelligence (AI) functions; and the modem is used to process wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 910, but can be realized by a separate chip.

[0128] The memory 920 can include a random access memory (RAM) and can also include a read-only memory (ROM). Optionally, the memory 920 includes a non-transitory computer-readable storage medium. The memory 920 can be used to store instructions, programs, codes, code sets, or instruction sets. The memory 920 can include a program storage area and a data storage area, where the program storage area can store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing each of the methods described below, etc., and the data storage area can store data created according to the use of the terminal 900 (such as audio data, a phone book, etc.).

[0129] The display screen 930 is a display component for displaying a user interface. Optionally, the display screen 930 is a display screen with a touch function, through which a user can perform a touch operation on the display screen 930 using a finger, a stylus, or any suitable object.

[0130] The display screen 930 is usually arranged on the front panel of the terminal 900. The display screen 930 can be designed to be a full screen, a curved screen, a special-shaped screen, a double-sided screen, or a folding screen. The display screen 930 can also be designed to be a combination of a full screen and a curved screen, a combination of a special-shaped screen and a curved screen, etc., and the present embodiment does not limit this.

[0131] In addition, those skilled in the art can understand that the structure of the terminal 900 shown in the above-described figures does not constitute a limitation on the terminal 900, and the terminal 900 can include more or fewer components than those shown in the figures, or combine certain components, or different component arrangements. For example, the terminal 900 also includes a microphone, a speaker, a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (Wi-Fi) module, a power supply, a Bluetooth module, etc., which are not described here in detail.

[0132] The embodiments of the present application also provide a computer readable medium, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the screen display method shown in each of the above embodiments.

[0133] The embodiments of the present application also provide a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the screen display method shown in each of the above embodiments.

[0134] In some embodiments, the computer program product related to the embodiments of the present application can be deployed on one terminal for execution, or on multiple terminals located in one place for execution, or on multiple terminals distributed in multiple places and interconnected through a communication network for execution, and the multiple terminals distributed in multiple places and interconnected through a communication network can constitute a blockchain system.

[0135] Those skilled in the art can realize that, in one or more examples described above, the functions described by the embodiments of the present application can be implemented by hardware, software, firmware or any combination thereof. When implemented by software, the functions can be stored in a computer readable medium or transmitted as one or more instructions or codes on a computer readable medium. The computer readable medium includes a computer storage medium and a communication medium, wherein the communication medium includes any medium that facilitates the transfer of computer programs from one place to another. The storage medium can be any available medium accessible by a general or special purpose computer.

[0136] The above description is only optional embodiments of the present application and does not limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A screen display method, characterized in that, The method includes: Receive a first vertical synchronization signal, the first vertical synchronization signal being used to indicate reading the rendering data of the first image frame that has been cached in the first image buffer; Based on the first vertical synchronization signal, the rendering data of the first image frame is read from the first image buffer to synthesize the rendering data, and then the image is displayed based on the synthesized data; Obtain the first transmission time and reading duration of the first vertical synchronization signal, wherein the reading duration is the time required to read the rendering data of the first image frame; Based on the first sending time and the reading duration, determine the end time for reading the rendering data of the first image frame; Upon receiving the second vertical synchronization signal, and when the current time reaches the end time, it is determined that the reading of rendering data for the first image frame in the first image buffer has ended; The image data of the second image frame is rendered in the first image buffer to obtain the rendering data of the second image frame and cache it in the first image buffer, and the image is displayed based on the rendering data of the second image frame in the first image buffer; the second vertical synchronization signal is used to indicate that the image data of the second image frame is rendered in the first image buffer, and the first image frame is the previous image frame of the second image frame; When reading the rendering data of the second image frame in the first image buffer, if the display latency requirement of the second image frame is higher than that of the third image frame, the image data of the third image frame is rendered in the second image buffer to obtain the rendering data of the third image frame. The third image frame is the next image frame after the second image frame, and the second image buffer is an image buffer different from the first image buffer. Upon receiving a third vertical synchronization signal, the rendering data of the third image frame in the second image buffer is read, and the image is displayed based on the rendering data of the third image frame. The third vertical synchronization signal is used to indicate the reading of the rendering data of the third image frame in the second image buffer.

2. The method according to claim 1, characterized in that, The method further includes: In response to the second vertical synchronization signal, a rendering instruction is generated based on the instruction data of the second image frame corresponding to the second vertical synchronization signal. This rendering instruction instructs the execution of the step of rendering the image data of the second image frame in the first image buffer; or... If the second vertical synchronization signal is received within the target time period, the rendering instruction is generated based on the instruction data of the second image frame corresponding to the second vertical synchronization signal. The start time of the target time period is the first sending time, and the duration of the target time period is the reading duration.

3. The method according to claim 1, characterized in that, The step of obtaining the first transmission time of the first vertical synchronization signal includes: Obtain the time phase offset between the first vertical synchronization signal and the second vertical synchronization signal; The first transmission time is determined based on the time phase offset and the second transmission time of the second vertical synchronization signal.

4. The method according to claim 1, characterized in that, The step of rendering the image data of the second image frame in the first image buffer to obtain the rendered data of the second image frame and caching it in the first image buffer, and displaying the image based on the rendered data of the second image frame in the first image buffer, includes: Determine the first image data and the second image data in the second image frame, wherein the first image data is image data in the second image frame that is different from the first image frame, and the second image data is image data in the second image frame that is the same as the first image frame; The first image data is rendered to obtain the first rendered data; Obtain the second rendering data corresponding to the second image data from the rendering data of the first image frame; Based on the first rendering data and the second rendering data, the rendering data of the second image frame is obtained; The image is displayed based on the rendering data of the second image frame.

5. The method according to claim 1, characterized in that, The second image frame is multiple, and the rendering of the image data of the second image frame in the first image buffer includes: Obtain the priority levels of multiple second image frames; The processing order of the multiple second image frames is determined based on their priority levels. Based on the processing order of the plurality of second image frames, the plurality of second image frames are rendered sequentially in the first image buffer.

6. The method according to claim 5, characterized in that, The step of rendering the plurality of second image frames sequentially in the first image buffer based on the processing order of the plurality of second image frames includes: When the processing time for the second image frame is reached, the second image frame is associated with the rendering interface, which is used to call the image processor GPU; The GPU is invoked through the rendering interface, and the GPU renders the second image frame in the first image buffer.

7. A screen display device, characterized in that, The device includes: The acquisition module is configured to receive a first vertical synchronization signal, which indicates the reading of rendering data of a first image frame cached in a first image buffer; based on the first vertical synchronization signal, read the rendering data of the first image frame from the first image buffer to synthesize the rendering data, and then display the image based on the synthesized data; and acquire the first transmission time and reading duration of the first vertical synchronization signal, wherein the reading duration is the time required to read the rendering data of the first image frame. The determining module is used to determine the end time of reading the rendering data of the first image frame based on the first sending time and the reading duration; A first rendering module is configured to, upon receiving a second vertical synchronization signal and when the current time reaches the end time, determine that the reading of rendering data of the first image frame in the first image buffer has ended; render the image data of the second image frame in the first image buffer to obtain the rendering data of the second image frame and cache it in the first image buffer; and display the image based on the rendering data of the second image frame in the first image buffer; the second vertical synchronization signal is used to indicate that the image data of the second image frame in the first image buffer is rendered, and the first image frame is the previous image frame of the second image frame; The second rendering module is used to render the image data of the third image frame in the second image buffer when reading the rendering data of the second image frame in the first image buffer, if the display latency requirement of the second image frame is higher than that of the third image frame, to obtain the rendering data of the third image frame. The third image frame is the next image frame after the second image frame, and the second image buffer is an image buffer different from the first image buffer. The reading module is used to read the rendering data of the third image frame in the second image buffer when a third vertical synchronization signal is received, and to display the image based on the rendering data of the third image frame. The third vertical synchronization signal is used to indicate the reading of the rendering data of the third image frame in the second image buffer.

8. The apparatus according to claim 7, characterized in that, The device further includes: A first generation module is configured to, in response to a second vertical synchronization signal, generate a rendering instruction based on instruction data of the second image frame corresponding to the second vertical synchronization signal. The rendering instruction instructs the execution of the step of rendering the image data of the second image frame in the first image buffer; or... The first generation module is used to generate the rendering instruction based on the instruction data of the second image frame corresponding to the second vertical synchronization signal when the second vertical synchronization signal is received within the target time period. The start time of the target time period is the first sending time, and the duration of the target time period is the reading duration.

9. The apparatus according to claim 7, characterized in that, The acquisition module is used to acquire the time phase offset between the first vertical synchronization signal and the second vertical synchronization signal; and to determine the first transmission time based on the time phase offset and the second transmission time of the second vertical synchronization signal.

10. The apparatus according to claim 7, characterized in that, The first rendering module is used to determine first image data and second image data in the second image frame, wherein the first image data is image data in the second image frame that is different from the first image frame, and the second image data is image data in the second image frame that is the same as the first image frame; The first image data is rendered to obtain the first rendered data; Obtain the second rendering data corresponding to the second image data from the rendering data of the first image frame; Based on the first rendering data and the second rendering data, the rendering data of the second image frame is obtained; The image is displayed based on the rendering data of the second image frame.

11. The apparatus according to claim 7, characterized in that, The second image frame is multiple, and the first rendering module is used to obtain the priority level of the multiple second image frames; and determine the processing order of the multiple second image frames based on the priority level of the multiple second image frames; Based on the processing order of the plurality of second image frames, the plurality of second image frames are rendered sequentially in the first image buffer.

12. The apparatus according to claim 11, characterized in that, The first rendering module is used to associate the second image frame with the rendering interface when the processing time of the second image frame is reached. The rendering interface is used to call the image processor GPU. The GPU is called through the rendering interface to render the second image frame in the first image buffer.

13. A terminal, characterized in that, The terminal includes one or more processors and one or more memories, wherein at least one piece of program code is stored in the one or more memories, and the at least one piece of program code is loaded and executed by the one or more processors to implement the screen display method as described in any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that, The storage medium stores at least one piece of program code, which is loaded and executed by a processor to implement the screen display method as described in any one of claims 1 to 6.

15. A computer program product, characterized in that, The computer program product stores at least one piece of program code, which is executed by a processor to implement the screen display method as described in any one of claims 1 to 6.

Images