Image processing method based on vertical synchronization signal, electronic equipment and storage medium

Abstract

The invention relates to the technical field of image processing and display, and provides an image processing method based on a vertical synchronization signal, electronic equipment and a storage medium, and the method comprises the steps that the electronic equipment responds to a first vertical synchronization signal, draws and renders, and buffers a first image layer in a first buffer queue at a first moment; and under the condition that the first moment is after the second vertical synchronization signal and the third vertical synchronization signal does not have the first image frame to be refreshed and displayed, the electronic equipment immediately triggers the first image layer at the first moment to carry out image layer synthesis to obtain a second image frame. Wherein the second vertical synchronization signal is after the first vertical synchronization signal and is spaced by a first preset period, and the third vertical synchronization signal is after the second vertical synchronization signal and is spaced by a second preset period. Therefore, when the drawing of the first image layer is delayed and the second image frame can be refreshed and displayed by the corresponding third vertical synchronizing signal, the immediate synthesis is triggered, so that frame loss can be avoided, and the smoothness of image display is ensured.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical fields of image processing and display, and particularly to an image processing method, an electronic device, and a storage medium based on a vertical synchronization signal. Background Art

[0002] With the development of electronic technology, the performance of various electronic devices (such as mobile phones) is getting better and better. Consumers' requirements for the human-computer interaction performance of electronic products are also getting higher and higher. Among them, the visual coherence of the display content of an electronic device is an important human-computer interaction performance.

[0003] Currently, ensuring that the display screen of an electronic device does not drop frames is one of the prerequisites for ensuring the coherence of the displayed images of the electronic device. However, existing application threads may cause frame drops due to excessive drawing and rendering time, thereby affecting the smoothness of the images displayed on the display screen and resulting in a poor user visual experience. Summary of the Invention

[0004] The embodiments of the present application provide an image processing method, an electronic device, and a storage medium based on a vertical synchronization signal, which are used to solve the problem of frame drops in the display screen of an electronic device and ensure the smoothness of image display.

[0005] To achieve the above object, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, an image processing method based on a vertical synchronization signal is provided, which is applied to an electronic device. The vertical synchronization signal includes a first vertical synchronization signal for triggering drawing, a second vertical synchronization signal for triggering layer composition, and a third vertical synchronization signal for triggering the user to refresh the display. The second vertical synchronization signal is after the first vertical synchronization signal and is separated from the first vertical synchronization signal by a first preset period; the third vertical synchronization signal is after the second vertical synchronization signal and is separated from the second vertical synchronization signal by a second preset period; that is, the third vertical synchronization signal is after the first vertical synchronization signal, and the period separated from the first vertical synchronization signal is the sum of the first preset period and the second preset period. The method includes:

[0007] When the first vertical synchronization signal arrives, the electronic device responds to the first vertical synchronization signal, draws and renders the first layer of the first application, and at the same time caches the drawn and rendered first layer in the first cache queue at the first moment. Then, if the first moment of caching the first layer is after the second vertical synchronization signal and there is no first image frame to be refreshed and displayed in the third vertical synchronization signal, the electronic device immediately triggers layer composition of the first layer in the first cache queue at the first moment to obtain a second image frame, and caches the second image frame in the second cache queue.

[0008] It can be seen that, on the one hand, when the electronic device renders and draws the first layer in response to the timeout of the first vertical synchronization signal, and there is no first image frame to be refreshed and displayed corresponding to the third vertical synchronization signal, that is, at the first moment after the second vertical synchronization signal, resulting in the first layer missing the second vertical synchronization signal for layer composition but the corresponding third vertical synchronization signal not being missed yet, the electronic device of the present application immediately triggers the layer composition of the first image, so as to avoid frame loss in the case of timeout rendering and drawing and ensure the smoothness of image display.

[0009] In a possible implementation manner of the first aspect, since there may still be a situation where the third vertical synchronization signal is missed due to the timeout rendering and drawing of the first layer, in this case, even if the first layer is immediately composed, it is impossible to refresh and display the corresponding second image frame at this third vertical synchronization signal, resulting in frame loss. Therefore, in order to minimize the impact of frame loss as much as possible, in this case, the second image frame can be refreshed and displayed in advance.

[0010] That is, the image processing method based on the vertical synchronization signal may further include: at the first moment after the third vertical synchronization signal, refreshing and displaying the second image frame in the second buffer queue at the screen refresh moment; wherein, the screen refresh moment is after the third vertical synchronization signal and before the fourth vertical synchronization signal; wherein, the fourth vertical synchronization signal is used to trigger the display of the image frame, the fourth vertical synchronization signal is after the third vertical synchronization signal, and is separated from the third vertical synchronization signal by a synchronization period. Thus, after missing the third vertical synchronization signal, the second image frame is refreshed and displayed in advance as much as possible between the fourth vertical synchronization signals, and the display of the second image frame is delayed, so as to minimize the impact of frame loss as much as possible.

[0011] In another possible implementation manner of the first aspect, that is, when the first moment is before the third vertical synchronization signal, in this case, the electronic device can generally normally refresh and display the second image frame when the third vertical synchronization signal arrives. Therefore, the image processing method based on the vertical synchronization signal may further include: at the first moment before the third vertical synchronization signal, the electronic device responds to the third vertical synchronization signal and refreshes and displays the second image frame in the second buffer queue.

[0012] In a possible implementation manner of the first aspect, the screen refresh moment can be determined according to a predetermined frame rate. Wherein, the predetermined frame rate is greater than the frame rate of the display screen of the electronic device. Furthermore, the number of screen refresh moments determined based on the predetermined frame rate is larger and more compact than the number of screen refresh moments determined based on the frame rate of the display screen, so as to ensure that the second image frame can be refreshed and displayed in advance.

[0013] In a possible implementation of the first aspect, the second image frame is cached in the second cache queue at the second moment. Generally speaking, in addition to the rendering timeout of the first layer drawing that causes the display to not be refreshed in the third vertical synchronization signal, it may also be due to the timeout of layer composition that causes the display to not be refreshed in the third vertical synchronization signal. That is, when the second moment is after the third vertical synchronization signal, it will also cause the display to not be refreshed in the third vertical synchronization signal, resulting in frame loss.

[0014] Therefore, to avoid frame loss in this case, the image processing method based on the vertical synchronization signal may further include: when the second moment is after the third vertical synchronization signal, refreshing and displaying the second image frame in the second cache queue at the screen refresh moment. The screen refresh moment in this aspect is also after the third vertical synchronization signal and before the fourth vertical synchronization signal. And, it can also be determined according to a predetermined frame rate greater than the frame rate of the display screen, which will not be elaborated here.

[0015] In a possible implementation of the first aspect, since there are usually many screen refresh moments, in order to ensure that the second image frame can be successfully refreshed and displayed in advance, the second image frame can be refreshed and displayed in advance by traversing the screen refresh moments. Therefore, the screen refresh moment can include a first refresh moment and at least one second refresh moment; the second refresh moment is after the first refresh moment. Based on this, refreshing and displaying the second image frame in the second cache queue at the screen refresh moment can include: at the first screen refresh moment, refreshing and displaying the second image frame in the second cache queue; if the refreshing and displaying fails at the first screen refresh moment, then traverse at least one second screen refresh moment in chronological order, and refresh and display the second image frame at each second screen refresh moment until the refreshing and displaying is successful or the traversal is completed.

[0016] In a possible implementation of the first aspect, there are two cases regarding whether there is a first image frame to be refreshed and displayed in the third vertical synchronization signal. One is that there is a first image frame being synthesized in response to the second vertical synchronization signal. The other is that there is a first image frame to be refreshed and displayed in the second cache queue.

[0017] Both of the above two cases will cause the first image frame to be refreshed and displayed in the third vertical synchronization signal before the second image frame, resulting in the second image frame being delayed until the fourth vertical synchronization signal for refreshing and displaying, causing frame loss.

[0018] Therefore, when the electronic device does not perform layer composition in response to the second vertical synchronization signal, and at the first moment, there is no first image frame to be refreshed and displayed in the second cache queue, the electronic device can determine that there is no first image frame to be refreshed and displayed in the third vertical synchronization signal.

[0019] In another possible implementation of the first aspect, if the first moment is before the second vertical synchronization signal, it means that the third vertical synchronization signal will not be missed. Therefore, the electronic device can wait conventionally for the second vertical synchronization signal to arrive, and in response to the second vertical synchronization signal, perform layer composition on the first layer in the first buffer queue to obtain the second image frame, and cache the second image frame in the second buffer queue. Similarly, the electronic device can wait conventionally for the third vertical synchronization signal to arrive, and in response to the third vertical synchronization signal, refresh and display the second image frame in the second buffer queue.

[0020] In a possible implementation of the first aspect, since layer composition requires a certain amount of processing time, even if the first moment coincides with the second vertical synchronization signal, it may miss the third vertical synchronization signal due to the processing time required for layer composition. Furthermore, in order to avoid this situation as much as possible, in this aspect, even if the first moment is before the second vertical synchronization signal, as long as the time difference between the first moment and the second vertical synchronization signal is less than or equal to a preset time threshold, the first moment can be regarded as being after the second vertical synchronization signal.

[0021] In a second aspect, the present application provides an electronic device, including: one or more processors and a memory, the memory being coupled to the processor; one or more computer program codes are stored in the memory, and the computer program codes include computer instructions; when the processor executes the computer instructions, the electronic device is caused to perform the following steps:

[0022] In response to the first vertical synchronization signal, draw and render the first layer of the first application, and cache the first layer in the first buffer queue at the first moment; the first moment is after the second vertical synchronization signal, and there is no first image frame to be refreshed and displayed in the third vertical synchronization signal. At the first moment, perform layer composition on the first layer in the first buffer queue to obtain the second image frame, and cache the second image frame in the second buffer queue.

[0023] Wherein, the second vertical synchronization signal is used to trigger layer composition, the second vertical synchronization signal is after the first vertical synchronization signal and is separated from the first vertical synchronization signal by a first preset period; the third vertical synchronization signal is used to trigger image frame refresh and display, the third vertical synchronization signal is after the second vertical synchronization signal and is separated from the second vertical synchronization signal by a second preset period.

[0024] In a possible implementation of the second aspect, when the above computer instructions are executed by the processor, the electronic device is further caused to perform the following steps: the first moment is before the third vertical synchronization signal, and in response to the third vertical synchronization signal, refresh and display the second image frame in the second buffer queue.

[0025] In a possible implementation of the second aspect, when the above computer instructions are executed by a processor, the electronic device is further caused to perform the following steps: At a first moment after a third vertical synchronization signal, refresh and display a second image frame in a second buffer queue at a frame refresh moment; wherein, the frame refresh moment is after the third vertical synchronization signal and before a fourth vertical synchronization signal; wherein, the fourth vertical synchronization signal is used to trigger the display of an image frame, and the fourth vertical synchronization signal is after the third vertical synchronization signal and is separated from the third vertical synchronization signal by one synchronization period.

[0026] In a possible implementation of the second aspect, when the above computer instructions are executed by a processor, the electronic device is further caused to perform the following steps: Determine a frame refresh moment according to a predetermined frame rate; wherein, the predetermined frame rate is greater than the frame rate of the display screen of the electronic device.

[0027] In a possible implementation of the second aspect, when the above computer instructions are executed by a processor, the electronic device is further caused to perform the following steps: At a second moment after a third vertical synchronization signal, refresh and display a second image frame in a second buffer queue at a frame refresh moment; wherein, the frame refresh moment is after the third vertical synchronization signal and before a fourth vertical synchronization signal; the frame refresh moment is determined according to a predetermined frame rate.

[0028] In a possible implementation of the second aspect, the frame refresh moment includes a first refresh moment and at least one second refresh moment; the second refresh moment is after the first refresh moment. When the above computer instructions are executed by a processor, the electronic device is further caused to perform the following steps: At the first frame refresh moment, refresh and display a second image frame in a second buffer queue; if the refresh and display fails at the first frame refresh moment, traverse at least one second frame refresh moment in chronological order, and refresh and display the second image frame at each second frame refresh moment until the refresh and display is successful or the traversal is completed.

[0029] In a possible implementation of the second aspect, when the above computer instructions are executed by a processor, the electronic device is further caused to perform the following steps: In response to the second vertical synchronization signal, no layer composition is performed; and, at a first moment, when there is no first image frame to be refreshed and displayed in the second buffer queue, determine that there is no first image frame to be refreshed and displayed in the third vertical synchronization signal.

[0030] In a possible implementation of the second aspect, when the above computer instructions are executed by a processor, the electronic device is further caused to perform the following steps: At a first moment before a second vertical synchronization signal, in response to the second vertical synchronization signal, perform layer composition on a first layer in a first buffer queue to obtain a second image frame; in response to a third vertical synchronization signal, refresh and display the second image frame in the second buffer queue.

[0031] In a possible implementation of the second aspect, when the above computer instructions are executed by a processor, the electronic device further performs the following steps: Before the second vertical synchronization signal at a first moment, and the time difference between the first moment and the second vertical synchronization signal is less than or equal to a preset time threshold, it is determined that the first moment is after the second vertical synchronization signal.

[0032] In a third aspect, a computer-readable storage medium of the present application stores a computer program, which, when executed by a processor in an electronic device, causes the electronic device to perform the image processing method based on a vertical synchronization signal as in the first aspect and any possible implementation thereof.

[0033] In a fourth aspect, the present application provides a computer program product, which, when running on a computer, causes the computer to perform the image processing method based on a vertical synchronization signal as in the first aspect and any possible implementation thereof. The computer may be the above-mentioned electronic device.

[0034] It can be understood that the beneficial effects that can be achieved by the electronic device in any possible implementation of the above second aspect, the computer-readable storage medium in the third aspect, and the computer program product in the fourth aspect can refer to the beneficial effects in the first aspect and any possible implementation thereof, which will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1A FIG. is a schematic diagram of a vertical synchronization signal provided by an embodiment of the present application;

[0036] Figure 1B FIG. is another schematic diagram of a vertical synchronization signal provided by an embodiment of the present application;

[0037] Figure 2 FIG. is a software processing flow chart of an electronic device for displaying an image in response to a touch operation provided by an embodiment of the present application;

[0038] Figure 3 FIG. is a schematic diagram of the principle of layer drawing, rendering, composition, and image frame display provided by an embodiment of the present application;

[0039] Figure 4 FIG. is a schematic diagram of the principle of layer production and consumption of an electronic device provided by an embodiment of the present application;

[0040] Figure 5 FIG. is a schematic diagram of the change of layers in the first cache queue during the process of layer drawing, rendering, composition, and image frame refresh display of an electronic device provided by an embodiment of the present application;

[0041] Figure 6 FIG. is a first schematic diagram of image frame dropping provided by an embodiment of the present application;

[0042] Figure 7 Flowchart 1 of an image processing method based on a vertical synchronization signal provided by an embodiment of the present application;

[0043] Figure 8 Schematic diagram of image frame dropping provided by an embodiment of the present application Figure Two ;

[0044] Figure 9 Schematic diagram of image frame dropping provided by an embodiment of the present application Figure Three ;

[0045] Figure 10 Flowchart of an image processing method based on a vertical synchronization signal provided by an embodiment of the present application Figure Two ;

[0046] Figure 11 Schematic diagram of refreshing and displaying an image frame in advance at the screen refresh moment provided by an embodiment of the present application;

[0047] Figure 12 Schematic diagram of image frame dropping provided by an embodiment of the present application Figure Four ;

[0048] Figure 13 Another schematic diagram of refreshing and displaying an image frame in advance at the screen refresh moment provided by an embodiment of the present application;

[0049] Figure 14 Flowchart of an image processing method based on a vertical synchronization signal provided by an embodiment of the present application Figure Three ;

[0050] Figure 15 Schematic diagram of the change of image frames in the second buffer queue during the process of layer drawing, rendering, composition, and image frame refreshing and display of an electronic device provided by an embodiment of the present application;

[0051] Figure 16 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application;

[0052] Figure 17 Schematic diagram of the structure of a chip system provided by an embodiment of the present application. Detailed implementation manners

[0053] The technical scheme of the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Among them, in the description of the embodiment of the present application, the terms used in the following embodiments are only for the purpose of describing a specific embodiment, and are not intended to limit the present application. In addition, in order to facilitate the clear description of the technical scheme of the embodiment of the present application, in the embodiment of the present application, if the words "first", "second" and the like are used to distinguish the same items or similar items with basically the same functions and effects. It can be understood by those skilled in the art that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not limit necessarily different. And, in the description of the embodiment of the present application, unless otherwise specified, the meaning of "multiple" means two or more.

[0054] The embodiment of the present application provides an image processing method based on a vertical synchronization signal, which can be applied to an electronic device including a display screen (such as a touch screen). Through this method, the possibility of frame loss when the electronic device displays an image can be reduced, and the smoothness of the image displayed on the display screen can be ensured, thereby improving the user's visual experience.

[0055] Furthermore, even if frame loss is unavoidable, through this method, the electronic device can reduce the impact of frame loss on the screen display and ensure the smoothness of the screen display as much as possible to enhance the user's visual experience.

[0056] Exemplarily, the electronic device may be at least one of a mobile phone, a foldable electronic device, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle-mounted device, a smart home device, a smart city device, and the like including a display screen (such as a touch screen).

[0057] It can be understood that the embodiments of the present application do not impose any particular limitation on the specific type of the electronic device.

[0058] For the convenience of understanding the solution, the vertical synchronization signal in the technical field will be briefly introduced as follows. In the embodiments of the present application, the vertical synchronization signal includes vertical synchronization signal 1 (corresponding to the first vertical synchronization signal), vertical synchronization signal 2 (corresponding to the second vertical synchronization signal), and vertical synchronization signal 3 (corresponding to the third vertical synchronization signal).

[0059] Vertical synchronization signal 1: Such as the VSYNC_APP signal. This vertical synchronization signal 1 can be used to trigger the drawing of one or more layers and render the drawn layers. That is to say, the above vertical synchronization signal 1 can be used to trigger the UI thread to draw one or more layers, and the Render thread renders the one or more layers drawn by the UI thread. In the embodiments of the present application, this vertical synchronization signal 1 (such as the VSYNC_APP signal) is the first vertical synchronization signal.

[0060] Vertical synchronization signal 2: Such as the VSYNC_SF signal. This vertical synchronization signal 2 can be used to trigger the layer composition of the rendered one or more layers to obtain an image frame. That is to say, the above vertical synchronization signal 2 can be used to trigger the composition thread to perform layer composition on the one or more layers rendered by the Render thread to obtain an image frame. In the embodiments of the present application, this vertical synchronization signal 2 (such as the VSYNC_SF signal) is the second vertical synchronization signal.

[0061] Vertical synchronization signal 3: Such as the HW_VSYNC signal (also known as the VSYNC_TE signal). This vertical synchronization signal 3 can be used to trigger the hardware to refresh and display the image frame. In the embodiments of the present application, this vertical synchronization signal 3 (such as the HW_VSYNC signal or the VSYNC_TE signal) is the third vertical synchronization signal.

[0062] Among them, the vertical synchronization signal 3 is a hardware signal triggered by the display driver of the electronic device. In the embodiments of the present application, the signal period T3 of the vertical synchronization signal 3 is determined according to the screen refresh rate of the display screen of the electronic device.

[0063] Specifically, the signal period T3 of the vertical synchronization signal 3 is the reciprocal of the screen refresh rate of the display screen of the electronic device (for example, it can be an LCD or an OLED). Among them, the screen refresh rate of the electronic device can be the same as the frame rate of the display screen of the electronic device.

[0064] Therefore, the high frame rate of the electronic device can be understood as the high screen refresh rate. For example, the screen refresh rate and the frame rate of the display screen of the electronic device can be any one value among 60 Hertz (Hz), 70Hz, 75Hz, 80Hz, 90Hz, or 120Hz.

[0065] Exemplarily, if the screen refresh rate of the electronic device and the frame rate of the display screen are 60 Hz, then the signal period of the above vertical synchronization signal 3 is 1 / 60 = 0.01667 seconds (s) = 16.667 milliseconds (ms). If the screen refresh rate and the frame rate of the display screen are 90 Hz, then the signal period of the above vertical synchronization signal 3 is 1 / 90 = 0.0111 seconds = 11.11 milliseconds. If the screen refresh rate and the frame rate of the display screen are 120 Hz, then the signal period of the above vertical synchronization signal 3 is 1 / 120 = 0.008333 seconds = 8.33 milliseconds.

[0066] It should be noted that, in some embodiments, the electronic device may support multiple different frame rates. The frame rate of the electronic device can be switched between the above different frame rates. For example, the electronic device can switch the frame rate from 60 Hz to 120 Hz to increase the frame rate. The electronic device can also switch the frame rate from 120 Hz to 60 Hz to decrease the frame rate.

[0067] It can be understood that the frame rate in the embodiments of the present application is the frame rate currently used by the electronic device. That is, the signal period of the vertical synchronization signal 3 is the reciprocal of the frame rate currently used by the electronic device.

[0068] In addition, the vertical synchronization signal 3 in the embodiments of the present application is a periodic discrete signal. As Figure 1A and Figure 1B shown, a schematic diagram of a vertical synchronization signal is shown. Hereinafter, with reference to Figure 1A and Figure 1B the vertical synchronization signal 1, the vertical synchronization signal 2, and the vertical synchronization signal 3 will be described.

[0069] Referring to Figure 1A and Figure 1B , there will be a vertical synchronization signal 3 triggered by a hardware driver every other signal period. The vertical synchronization signal 1 and the vertical synchronization signal 2 are generated based on the vertical synchronization signal 3, that is, the vertical synchronization signal 3 can be the signal source of the vertical synchronization signal 1 and the vertical synchronization signal 2. Or, the vertical synchronization signal 1 and the vertical synchronization signal 2 are synchronized with the vertical synchronization signal 3. Therefore, generally speaking, the signal periods of the vertical synchronization signal 1 and the vertical synchronization signal 2 are the same as the signal period of the vertical synchronization signal 3. However, it should be noted that the phases of the vertical synchronization signal 1, the vertical synchronization signal 2, and the vertical synchronization signal 3 may be the same or may not be the same.

[0070] In the case where the phases are the same, for example, as Figure 1A shown, the signal period T1 of the vertical synchronization signal 1, the signal period T2 of the vertical synchronization signal 2, are the same as the signal period T3 of the vertical synchronization signal 3. And, as Figure 1AAs shown, the phases of vertical synchronization signal 1, vertical synchronization signal 2, and vertical synchronization signal 3 are the same. In this case, the periods between the vertical synchronization signals of each thread are the same, and each interval is one synchronization period. It can be understood that there is an interval of one synchronization period between vertical synchronization signal 1 and the corresponding vertical synchronization signal 2, and there is an interval of one synchronization period between vertical synchronization signal 2 and the corresponding vertical synchronization signal 3. As Figure 1A shown, the vertical synchronization signal 1 at time t1 and the vertical synchronization signal 2 at time t2 are separated by one synchronization period TZ. The vertical synchronization signal 2 at time t2 and the vertical synchronization signal 3 at time t3 are separated by one synchronization period TZ.

[0071] In the case of inconsistent phases, for example, as Figure 1B shown, the signal period T1 of vertical synchronization signal 1, the signal period T2 of vertical synchronization signal 2, and the signal period T3 of vertical synchronization signal 3 are the same. However, as Figure 1B shown, the phases of vertical synchronization signal 1, vertical synchronization signal 2, and vertical synchronization signal 3 are inconsistent. In this case, the periods between the vertical synchronization signals are not necessarily the same. That is, the periods between the corresponding vertical synchronization signal 1, vertical synchronization signal 2, and vertical synchronization signal 3 are not necessarily one synchronization period. As Figure 1B shown, the vertical synchronization signal 1 at time t1 and the vertical synchronization signal 2 at time t2 are separated by a first preset period. The vertical synchronization signal 2 at time t2 and the vertical synchronization signal 3 at time t3 are separated by a second preset period.

[0072] Among them, the first preset period and the second preset period are configured according to actual needs. The first preset period and the second preset period can be greater than the synchronization period or less than the synchronization period. And, in the embodiments of the present application, the first preset period can be equal to the second preset period, and the first preset period can also be not equal to the second preset period. It can be understood that the first preset period is the time reserved for drawing and rendering, and the second preset period is the time reserved for layer composition.

[0073] It can be understood that in the case of consistent phases, the first preset period = the second preset period = the synchronization period. That is, the time reserved for drawing and rendering and composition is the same. In the case of inconsistent phases, the time reserved for drawing and rendering and composition is not the same, and the reserved time for drawing and rendering and composition may be greater than one synchronization period or less than one synchronization period.

[0074] Hereinafter, for the convenience of understanding the solution, the embodiments of the present application mainly illustrate the image processing method based on vertical synchronization signals provided by the embodiments of the present application by taking the case of consistent phases (the first preset period = the second preset period = the synchronization period) as an example.

[0075] It can be understood that, in the actual implementation process, there may be a certain phase error between the vertical synchronization signal 1, the vertical synchronization signal 2, and the vertical synchronization signal 3 due to various factors (such as processing performance). It should be noted that when understanding the method of the embodiments of the present application, the above-mentioned phase error is ignored.

[0076] In summary, the above-mentioned vertical synchronization signal 1, vertical synchronization signal 2, and vertical synchronization signal 3 are all periodic discrete signals. For example, as Figure 1A shown in the figure and Figure 1B illustrated, there will be a vertical synchronization signal 1 every other signal period T1, there will be a vertical synchronization signal 2 every other signal period T2, and there will be a vertical synchronization signal 3 every other signal period T3. The signal periods of the above-mentioned vertical synchronization signal 1, vertical synchronization signal 2, and vertical synchronization signal 3 can all be referred to as the synchronization period TZ, that is, T1 = T2 = T3 = TZ. That is to say, the synchronization period TZ in the embodiments of the present application is the reciprocal of the screen refresh rate of the electronic device.

[0077] It should be noted that in different systems or architectures, the names of the vertical synchronization signals may be different. For example, in some systems or architectures, the name of the above-mentioned vertical synchronization signal (i.e., vertical synchronization signal 1) used to trigger the drawing of one or more layers may not be VSYNC_APP. However, no matter what the name of the vertical synchronization signal is, as long as it is a synchronization signal with a similar function and conforms to the technical concept of the method provided by the embodiments of the present application, it should be covered within the protection scope of the present application.

[0078] Moreover, in different systems or architectures, the definitions of the above-mentioned vertical synchronization signals may also be different. For example, in some other systems or architectures, the definition of the above-mentioned vertical synchronization signal 1 can be: the vertical synchronization signal 1 can be used to trigger the rendering of one or more layers; the definition of the vertical synchronization signal 2 can be: the vertical synchronization signal 2 can be used to trigger the generation of an image frame according to one or more layers; the definition of the vertical synchronization signal 3 can be: the vertical synchronization signal 3 can be used to trigger the display of the image frame. In the embodiments of the present application, the definition of the vertical synchronization signal is not limited. However, no matter what definition is made for the vertical synchronization signal, as long as it is a synchronization signal with a similar function and conforms to the technical concept of the method provided by the embodiments of the present application, it should be covered within the protection scope of the present application.

[0079] For the convenience of understanding, in the embodiments of the present application, here in combination with Figure 2 , taking the above display screen as a touch screen and the user's operation on the display screen as a touch operation as an example, the software processing flow of the electronic device from "the user's finger inputs a touch operation on the touch screen" to "the touch screen displays the image corresponding to the touch operation" is introduced.

[0080] AsFigure 2 As shown, the electronic device may include: a touch panel (TP) / TP driver 210, an Input framework (i.e., Input Framework) 220, a UI framework (i.e., UI Framework) 230, a Display framework (i.e., Display Framework) 240, and a hardware display module 250.

[0081] As Figure 2 shown, the software processing flow of the electronic device may include the following steps (1) - step (5).

[0082] Step (1): After the TP in the TP IC / TP driver 210 collects the touch operation of the user's finger on the TP of the electronic device, the TP driver reports the corresponding touch event to the Event Hub.

[0083] Step (2): The Input Reader thread of the Input framework 220 can read the touch event from the Event Hub, and then send the touch event to the Input Dispatcher thread; the Input Dispatcher thread uploads the touch event to the UI thread in the UI framework 230.

[0084] Step (3): The UI thread (such as Do Frame) in the UI framework 230 draws one or more layers corresponding to the touch event; the Render thread (such as Draw Frame) performs layer rendering on one or more layers. Among them, the above UI thread is a thread in the Central Processing Unit (CPU) of the electronic device. The Render thread is a thread in the GPU of the electronic device.

[0085] Step (4): The composition thread (Surface Flinger) in the Display framework 240 composes one or more drawn layers (i.e., one or more rendered layers) to obtain an image frame.

[0086] Step (5): The display driver of the hardware display module 250 Figure 2 As shown, the Liquid Crystal Display (LCD) driver can receive the composed image frame, and then the LCD displays the composed image frame. After the LCD displays the image frame, the image displayed by the LCD can be perceived by the human eye.

[0087] Generally, in response to a user's touch operation on the TP or a UI event, after the vertical synchronization signal 1 arrives, the UI framework can call the UI thread to draw one or more layers corresponding to the touch event, and then call the Render thread to render the one or more layers. Then, after the vertical synchronization signal 2 arrives, the Hardware Composer (HWC) can call the composition thread to perform layer composition on the drawn one or more layers (i.e., the rendered one or more layers) to obtain an image frame. Finally, after the vertical synchronization signal 3 arrives, the hardware display module can refresh and display the above image frame on the LCD. Among them, the above UI event can be triggered by a user's touch operation on the TP. Or, the UI event can be automatically triggered by the electronic device. For example, when the foreground application of the electronic device automatically switches the screen, the above UI event can be triggered. The foreground application is the application corresponding to the interface currently displayed on the display screen of the electronic device.

[0088] Among them, the TP can periodically detect the user's touch operation. After the TP detects the touch operation, it can wake up the above vertical synchronization signal 1 and vertical synchronization signal 2 to trigger the UI framework to perform layer drawing and rendering based on the vertical synchronization signal 1, and the hardware composition HWC to perform layer composition based on the vertical synchronization signal 2. Among them, the detection period for the TP to detect the touch operation is the same as the signal period T3 of the vertical synchronization signal 3 (such as HW_VSYNC).

[0089] It should be noted that the UI framework performs layer drawing and rendering periodically based on the vertical synchronization signal 1; the hardware composition HWC performs layer composition periodically based on the vertical synchronization signal 2; the LCD refreshes the image frame periodically based on the vertical synchronization signal 3.

[0090] As Figure 3 shown, the embodiments of the present application illustrate the process of the electronic device performing drawing, rendering, composition, and refreshing and displaying the image frame. Hereinafter, taking the vertical synchronization signal 1 as the VSYNC_APP signal, the vertical synchronization signal 2 as the VSYNC_SF signal, and the vertical synchronization signal 3 as the VSYNC_TE (i.e., HW_VSYNC) signal as an example, combined with Figure 3 a simple description of the process of the electronic device performing drawing, rendering, composition, and image frame display will be given.

[0091] Refer to Figure 3, the UI thread of the electronic device responds to the VSYNC_APP signal at time t1, executes "Drawing 1" to draw Layer 1, and then the Render thread executes "Rendering 1" to render this Layer 1; the composition thread of the electronic device responds to the VSYNC_SF signal at time t2, executes "Image Frame Composition 1" to perform layer composition on the above Layer 1 to obtain Image Frame 1; the LCD of the electronic device responds to the HW_VSYNC signal at time t3, executes "Image Frame Display 1" to refresh and display the above Image Frame 1.

[0092] For another example, as Figure 3 shown, the UI thread of the electronic device responds to the VSYNC_APP signal at time t2, executes "Drawing 2" to draw Layer 2, and then the Render thread executes "Rendering b" to render this Layer 2; the composition thread of the electronic device responds to the HW_VSYNC signal at time t3, executes "Image Frame Composition 2" to perform layer composition on the above Layer 2 to obtain Image Frame 2; the LCD of the electronic device responds to the HW_VSYNC signal at time t4, executes "Image Frame Display 2" to refresh and display the above Image Frame 2.

[0093] It can be seen that the layer drawn by the electronic device in response to the VSYNC_APP signal at time t1 corresponds to the VSYNC_SF signal at time t2 (one synchronization period (the first preset period) apart) for layer composition to obtain an image frame. And the image frame obtained by layer composition in response to the HW_VSYNC signal at time t2 corresponds to the HW_VSYNC signal at time t3 (one synchronization period (the second preset period) apart) for refresh display. Also, the layer drawn by the electronic device in response to the VSYNC_APP signal at time t2 corresponds to the VSYNC_SF signal at time t3 (one synchronization period (the first preset period) apart) for layer composition to obtain an image frame. And the image frame obtained by layer composition in response to the VSYNC_SF signal at time t3 corresponds to the HW_VSYNC signal at time t4 (one synchronization period (the second preset period) apart) for refresh display.

[0094] It can be understood that the VSYNC_APP signal at time t1 (such as the first vertical synchronization signal), the VSYNC_SF signal at time t2 (such as the second vertical synchronization signal), and the HW_VSYNC signal at time t3 (such as the third vertical synchronization signal) are corresponding vertical synchronization signals. For another example, the VSYNC_APP signal at time t2 (such as the first vertical synchronization signal), the VSYNC_SF signal at time t3 (such as the second vertical synchronization signal), and the HW_VSYNC signal at time t4 (such as the third vertical synchronization signal) are also corresponding vertical synchronization signals.

[0095] It should be noted that Figure 3The "Drawing 1" shown can be implemented in the CPU of the electronic device, and the "Rendering 1" can be implemented in the GPU of the electronic device. Figure 3 The "Drawing 2" shown can be implemented in the CPU of the electronic device, and the "Rendering 2" can be implemented in the GPU of the electronic device.

[0096] In some embodiments, the CPU of the electronic device can also perform rendering by the Render thread. It can be understood that the rendering performed by the Render thread in the CPU is the preparation before the GPU performs layer rendering on the drawn Layer 1 and Layer 2, and the "Rendering 1" and "Rendering 2" performed by the GPU are the formal layer rendering of the drawn Layer 1 and Layer 2 by the electronic device. That is to say, the drawing in the embodiments of the present application can include: the layer drawing performed by the UI thread and the preparation before the Render thread performs layer rendering on the layers drawn by the UI thread.

[0097] Figure 3 The process of the above-mentioned electronic device drawing, rendering, and compositing layers can constitute a graphics generation consumption model, as Figure 4 shown in the graphics generation consumption model 400. In this graphics generation consumption model 400, the UI thread and the Render thread (i.e., the renderer Render) of the electronic device act as producers to draw and render layers; the Render thread (i.e., the renderer Render) can save the layers that have completed the rendering preparation in the first cache queue and perform layer rendering on the layers in the first cache queue; the compositing thread (i.e., the compositor Surface Flinger) acts as a consumer, reads the layers from the first cache queue, performs layer composition on the read layers to obtain an image frame, and sends the image frame to the LCD (i.e., the display controller Display Controller) of the electronic device for display.

[0098] Among them, in the above-mentioned graphics generation consumption model, both the producer (such as the UI thread and the Render thread) and the consumer (such as the compositing thread) generate and consume layers according to the VSYNC signal.

[0099] Without jitter (i.e., no dropped frames), the production and consumption rates in the above-mentioned graphics generation consumption model are consistent. The producer (such as the Render thread) generates a layer (also known as frame data) every VSYNC period (such as the synchronization period TZ mentioned above) and puts it into the first buffer queue. The consumer (such as the composition thread) takes out a layer from the first buffer queue every VSYNC period (such as the synchronization period TZ mentioned above) for layer composition (also known as image frame composition). That is, the production period of the UI thread and the Render thread as producers is the same as the consumption period of the composition thread (i.e., Surface Flinger) as a consumer, both equal to the above-mentioned synchronization period TZ.

[0100] For example, as Figure 5 shown, at the tx moment in Figure 5 , the Render thread of the electronic device completes "Rendering 1". At this time, the Render thread can cache the rendered Layer 1 into the first buffer queue, that is, the producer produces a layer and caches the layer into the first buffer queue. That is, at the tx moment, the number of layers in the first buffer queue increases from 0 to 1 (i.e., 0->1). Subsequently, in response to Figure 5 the VSYNC_SF signal at the t2 moment shown, the composition thread of the electronic device can execute "Image Frame Composition 1" (also known as Layer Composition 1). At this time, the composition thread can read Layer 1 from the first buffer queue, that is, the consumer consumes a layer from the first buffer queue. That is, at the t2 moment, the number of layers in the first buffer queue decreases from 1 to 0 (i.e., 1->0).

[0101] Another example, at the ty moment in Figure 5 shown, the Render thread of the electronic device completes "Rendering 2". At this time, the Render thread can cache the rendered Layer 2 into the first buffer queue, that is, the producer produces a layer and caches the layer into the first buffer queue. That is, at the ty moment, the number of layers in the first buffer queue increases from 0 to 1 (i.e., 0->1).

[0102] Subsequently, in response to Figure 5 the VSYNC_SF signal at the t3 moment shown, the composition thread of the electronic device can execute "Image Frame Composition 2" (also known as Layer Composition 2). At this time, the composition thread can read Layer 2 from the first buffer queue, that is, the consumer consumes a layer from the first buffer queue. That is, at the t3 moment, the number of layers in the first buffer queue decreases from 1 to 0 (i.e., 1->0).

[0103] However, during the process of the electronic device drawing, rendering, synthesizing, and refreshing the display image frame in response to the above VSYNC_APP signal, VSYNC_SF signal, and HW_VSYNC signal, frame drops may occur due to the long duration of the application thread. Specifically, during the process of the display screen refreshing the display image frame, a blank image frame may be displayed. In this way, the coherence and smoothness of the display image on the display screen will be affected, thereby affecting the user's visual experience.

[0104] Specifically, the reason for the frame drop phenomenon in the display of the electronic device may be that the UI thread and the Render thread take too long to draw and render and cannot complete the drawing and rendering within one VSYNC cycle (such as the above synchronization cycle TZ).

[0105] In this way, the producer (such as the Render thread) cannot cache the drawn and rendered layer into the first cache queue on time. That is to say, the producer (such as the Render thread) will have at least one VSYNC cycle without caching frame data in the first cache queue. However, the consumer (such as the composition thread) will still take out a layer from the first cache queue for layer composition every other VSYNC cycle. However, since the producer does not cache the layer, the consumer cannot read the corresponding layer. Then, in this VSYNC cycle, layer composition cannot be performed to obtain an image frame, and this image frame cannot be refreshed and displayed. The display screen of the electronic device cannot be updated, and a frame drop phenomenon will occur. In this way, the coherence and smoothness of the display image on the display screen will be affected, thereby affecting the user's visual experience.

[0106] For example, as Figure 6 shown, the embodiment of the present application shows a schematic diagram of frame drop. Referring to Figure 6 , if the Render thread cannot complete "Rendering 2" before the arrival of time t3, it cannot cache Layer 2 into the first cache queue before time t3. Furthermore, at time t3, the number of frame data in the first cache queue is 0. Therefore, in response to the VSYNC_SF signal at time t3, the composition thread cannot read a layer from the first cache queue, so it cannot perform layer composition to obtain an image frame. Furthermore, at time t4, the display screen of the electronic device cannot refresh and display the image frame, and a frame drop phenomenon occurs.

[0107] At time tt after time t3, the Render thread completes "Rendering 2"; at this time, the number of layers in the first cache queue increases from 0 to 1 (i.e., 0 -> 1). In response to the VSYNC_SF signal at time t4 after time tt, the composition thread can read Layer 2 from the first cache queue, and the number of layers in the first cache queue decreases from 1 to 0 (i.e., 1 -> 0). The display screen of the electronic device can perform "Image Frame Display 2" to refresh and display the image frame at time t5.

[0108] As can be seen from Figure 6 the figure, during the synchronization period from time t4 to time t5, the display screen of the electronic device shows frame drops in the displayed image. By using the method of the embodiments of the present application, frame drops in the displayed image can be avoided, so as to prevent the display screen from showing a blank image for one frame. That is to say, by using the method of the embodiments of the present application, the possibility of frame drops when the electronic device displays an image can be reduced, and the smoothness of the image displayed on the display screen can be ensured, thereby enhancing the user's visual experience.

[0109] Furthermore, even if frame drops are inevitable, by using the method of the embodiments of the present application, the impact brought by frame drops can be minimized as much as possible, and the smoothness of the image displayed on the display screen can be ensured as much as possible.

[0110] It should be noted that the reason for frame drops in the image displayed by the electronic device may be that the electronic device cannot complete the drawing and rendering of one frame of layers within one frame (such as a synchronization period TZ), or it may be that the electronic device cannot complete the layer composition of one frame of layers within one frame (such as a synchronization period TZ).

[0111] In the following embodiments, mainly taking the case where the UI thread and the Render thread cannot complete the drawing and rendering of one frame of layers within one frame, resulting in frame drops in the image displayed by the electronic device as an example, the image processing method based on the vertical synchronization signal of the embodiments of the present application will be introduced.

[0112] The image processing method based on the vertical synchronization signal provided by the embodiments of the present application mainly determines the specific reason for possible frame drops based on the time sequence relationship between the first moment and the VSYNC_SF signal (i.e., the second vertical synchronization signal) and the HW_VSYNC signal (i.e., the third vertical synchronization signal). Furthermore, the electronic device adjusts the time points of layer composition and image frame refresh display based on this specific reason. For example, after missing the VSYNC_SF signal, if there is no image frame (the first image frame) to be refreshed and displayed in the HW_VSYNC signal corresponding to this VSYNC_SF signal with an interval of one synchronization period (the second preset period), the electronic device can decide to immediately perform layer composition, so that the image frame can be refreshed and displayed within the synchronization period of the corresponding HW_VSYNC signal, thereby avoiding frame drops of the image frame.

[0113] Alternatively, when the electronic device determines that frame drops have occurred, it can decide to refresh and display the image frame in advance, so as to minimize the impact brought by frame drops as much as possible, ensure the smoothness of the screen display as much as possible, and enhance the user's visual experience.

[0114] Among them, the above-mentioned first moment is the moment when the electronic device stores the first layer in the first cache queue after drawing and rendering the first layer in response to the VSYNC_APP signal (i.e., the first vertical synchronization signal). And it can be understood that the first vertical synchronization signal (VSYNC_APP signal), the second vertical synchronization signal (VSYNC_SF signal), and the third vertical synchronization signal (HW_VSYNC signal) in the embodiments of the present application are corresponding vertical synchronization signals.

[0115] For example, the corresponding VSYNC_APP signal, VSYNC_SF signal, and HW_VSYNC signal here can be Figure 1A , Figure 1B , Figure 3 , Figure 5 or Figure 6 the VSYNC_APP signal at time t1, the VSYNC_SF signal at time t2, and the HW_VSYNC signal (VSYNC_TE signal) at time t3 in Figure 1A , Figure 1B , Figure 3 , Figure 5 or Figure 6 the VSYNC_APP signal at time t2, the VSYNC_SF signal at time t3, and the HW_VSYNC signal (VSYNC_TE signal) at time t4 in

[0116] As Figure 7 shown, the embodiments of the present application show a flowchart of an image processing method based on a vertical synchronization signal. The method is applied to an electronic device including a display screen (such as a touch screen) and may include steps S701 - S707.

[0117] S701, the electronic device responds to the first vertical synchronization signal (VSYNC_APP signal), draws and renders the first layer of the first application, and caches the first layer in the first cache queue at the first moment.

[0118] When a first vertical synchronization signal (VSYNC_APP signal) arrives, the electronic device responds to this first vertical synchronization signal to draw the first layer of the first application. And the drawn first layer is cached in the first cache queue.

[0119] The embodiments of the present application record the moment when the electronic device caches the first layer as the first moment. Among them, the first application can be any application that needs to display a picture.

[0120] In some embodiments, the UI thread may draw a first layer in response to a first vertical synchronization signal (VSYNC_APP signal). Then, the Render thread prepares for rendering the first layer drawn by the UI thread and caches the first layer in a first cache queue. It should be noted that after the Render thread caches the first layer in the first cache queue, the Render thread may officially render the first layer cached in the first cache queue. After that, the composition thread may perform layer composition on the layer cached in the first cache queue (such as the first layer) to obtain an image frame (such as a second image frame).

[0121] S702, the electronic device determines whether the first moment is after a second vertical synchronization signal (VSYNC_SF signal).

[0122] Here, the second vertical synchronization signal is used to trigger layer composition and is a vertical synchronization signal corresponding to a synchronization period (a first preset period) after the first vertical synchronization signal in S701. If the electronic device determines that the first moment is after this second vertical synchronization signal (VSYNC_SF signal), the electronic device enters S703. If the electronic device determines that the first moment is before this second vertical synchronization signal (VSYNC_SF signal), the electronic device enters S706.

[0123] S703, the electronic device determines whether there is a first image frame to be refreshed and displayed in a third vertical synchronization signal (HW_VSYNC signal). Here, the third vertical synchronization signal is used to trigger the refresh and display of the image frame and is a vertical synchronization signal corresponding to a synchronization period (a second preset period) after the second vertical synchronization signal in S702. That is, it is a vertical synchronization signal corresponding to two synchronization periods (the sum of the first preset period and the second preset period) after the first vertical synchronization signal in S701.

[0124] If there is a first image frame to be refreshed and displayed in the third vertical synchronization signal, the electronic device ends the current process. Conversely, if there is no first image frame to be refreshed and displayed in the third vertical synchronization signal, the electronic device enters S704.

[0125] S704, at the first moment, the electronic device performs layer composition on the first layer in the first cache queue to obtain a second image frame and caches the second image frame in a second cache queue.

[0126] S705, the electronic device determines whether the first moment is before the third vertical synchronization signal (HW_VSYNC signal). If the first moment is before this third vertical synchronization signal (HW_VSYNC signal), the electronic device enters S707. If the first moment is after this third vertical synchronization signal (HW_VSYNC signal), the electronic device ends the current process.

[0127] S706. The electronic device synthesizes the first layer in the first buffer queue in response to the second vertical synchronization signal to obtain a second image frame, and caches the second image frame in the second buffer queue.

[0128] S707. The electronic device refreshes and displays the second image frame in the second buffer queue in response to the third vertical synchronization signal.

[0129] Specifically, if the electronic device determines that the first moment is before the second vertical synchronization signal (VSYNC_SF signal) corresponding to one synchronization period (the first preset period) after the first vertical synchronization signal (VSYNC_APP signal). In this case, it indicates that the electronic device has successfully completed the drawing and rendering of the layer (such as the first layer) and has stored it in the first buffer queue before the second vertical synchronization signal (VSYNC_SF signal) corresponding to one synchronization period (the first preset period) after the first vertical synchronization signal (VSYNC_APP signal) arrives.

[0130] That is, it indicates that the electronic device has successfully sent the layer (the first layer) to the synthesis thread before the second vertical synchronization signal (VSYNC_SF signal) corresponding to one synchronization period (the first preset period) after the first vertical synchronization signal (VSYNC_APP signal) arrives. Then, this layer (the first layer) can be smoothly synthesized when this second vertical synchronization signal (VSYNC_SF signal) arrives. Also, the electronic device can smoothly refresh and display the synthesized image frame when the third vertical synchronization signal (HW_VSYNC signal) corresponding to one synchronization period (the second preset period) after this second vertical synchronization signal (VSYNC_SF signal) arrives, that is, there will be no frame loss.

[0131] For example, as Figure 6 shown, the electronic device draws and renders layer 1 (the first layer) in response to the first vertical synchronization signal (VSYNC_APP signal) at time t1, and the electronic device stores layer 1 (the first layer) in the first buffer queue at time tx (the first moment). At this time, time tx (the first moment) is before the second vertical synchronization signal (VSYNC_SF signal) at the corresponding time t2, that is, the first moment in the above embodiment of the present application is before the second vertical synchronization signal (VSYNC_SF signal) corresponding to one synchronization period (the first preset period) after the first vertical synchronization signal (VSYNC_APP signal).

[0132] In this case, when the second vertical synchronization signal (VSYNC_SF signal) arrives at time t2, the electronic device can successfully perform "image frame composition 1" on layer 1 (the first layer) in response to this second vertical synchronization signal (VSYNC_SF signal) to obtain image frame 1 (the second image frame). And when the third vertical synchronization signal (HW_VSYNC signal) arrives at time t3, the electronic device can successfully perform "image frame display 1" to refresh and display image frame 1 (the second image frame) in response to this third vertical synchronization signal (HW_VSYNC signal).

[0133] Thus, the layer 1 (the first layer) rendered by the electronic device in response to the first vertical synchronization signal (VSYNC_APP signal) at time t1 can be successfully displayed within the synchronization period of the third vertical synchronization signal (HW_VSYNC signal) at time t3 corresponding to an interval of two synchronization periods (the sum of the first preset period and the second preset period), and there is no frame loss phenomenon in the electronic device.

[0134] Therefore, in the above case, the electronic device enters S706, waits for the arrival of the second vertical synchronization signal (VSYNC_SF signal) corresponding to an interval of one synchronization period (the first preset period) according to the normal process, and performs layer composition on the first layer in response to this second vertical synchronization signal (VSYNC_SF signal) to obtain the second image frame.

[0135] In some embodiments, the composition thread can read the first layer from the first buffer queue in response to this second vertical synchronization signal (VSYNC_SF signal) to perform layer composition to obtain the second image frame. Then, the electronic device enters S707, waits for the arrival of the corresponding third vertical synchronization signal (HW_VSYNC signal) according to the normal process, and refreshes and displays the second image frame in response to this third vertical synchronization signal (HW_VSYNC signal). In some embodiments, the display screen driver can read the second image frame from the second buffer queue and refresh and display it on the display screen (such as an LCD or an OLED).

[0136] If the electronic device determines that the first moment is after the second vertical synchronization signal (VSYNC_SF signal) corresponding to an interval of one synchronization period (the first preset period) of the first vertical synchronization signal (VSYNC_APP signal). In this case, it indicates that the electronic device has not successfully completed the drawing and rendering of the layer (such as the first layer) before the arrival of the second vertical synchronization signal (VSYNC_SF signal) corresponding to an interval of one synchronization period (the first preset period) of the first vertical synchronization signal (VSYNC_APP signal).

[0137] That is, it indicates that the electronic device fails to successfully send the layer (the first layer) to the composition thread before the arrival of the second vertical synchronization signal (VSYNC_SF signal) corresponding to one synchronization period (the first preset period) of the first vertical synchronization signal (VSYNC_APP signal).

[0138] Then, when the second vertical synchronization signal (VSYNC_SF signal) corresponding to one synchronization period (the first preset period) of the first vertical synchronization signal (VSYNC_APP signal) arrives, the composition thread cannot perform layer composition on the layer (the first layer) in response to this second vertical synchronization signal (VSYNC_SF signal) to obtain an image frame (such as the second image frame corresponding to the first layer). Furthermore, when the third vertical synchronization signal (HW_VSYNC signal) corresponding to two synchronization periods (the sum of the first preset period and the second preset period) of this first vertical synchronization signal (VSYNC_APP signal) arrives, the electronic device cannot refresh and display the corresponding image frame (the second image frame) in response to this third vertical synchronization signal (HW_VSYNC signal), resulting in frame loss. That is, the second image frame is lost, and the display screen of the electronic device does not display the second image frame within the corresponding synchronization period.

[0139] For example, as Figure 6 shown, the electronic device performs the drawing and rendering of layer 2 (the first layer) in response to the first vertical synchronization signal (VSYNC_APP signal) at time t2. However, because the drawing and rendering of layer 2 (the first layer) on the application side takes too long, that is, the drawing and rendering of layer 2 (the first layer) times out, the electronic device stores layer 2 (the first layer) in the first cache queue only at time tt (the first time).

[0140] At this time, time tt (the first time) is after the second vertical synchronization signal (VSYNC_SF signal) at the corresponding time t3, that is, the first time in the embodiments of the present application is after the second vertical synchronization signal (VSYNC_SF signal) corresponding to one synchronization period (the first preset period) of the first vertical synchronization signal (VSYNC_APP signal).

[0141] In this case, when the second vertical synchronization signal (VSYNC_SF signal) arrives at time t3, since there is no Layer 2 (the first layer) in the first buffer queue, the electronic device cannot read Layer 2 (the first layer), that is, the composition thread cannot read Layer 2 (the first layer), so the electronic device cannot perform layer composition on Layer 2 (the first layer) within the synchronization period TZ of this second vertical synchronization signal (VSYNC_SF signal). Furthermore, when the third vertical synchronization signal (HW_VSYNC signal) arrives at time t4, there is also no corresponding image frame 2 (the second image frame) to be refreshed and displayed. That is to say, in the synchronization period from time t4 to time t5, the display screen of the electronic device shows a frame drop phenomenon.

[0142] At this time, in order to avoid frame drops, in some embodiments, the electronic device can trigger the composition thread to immediately perform layer composition on the first layer (such as Figure 6 Layer 2) at the first moment (such as Figure 6 the tt moment).

[0143] However, based on the actual work distribution of the composition thread, the composition thread used for layer composition may not only perform layer composition on the first layer of the first application alone. That is to say, the composition thread for performing layer composition on the first layer of the first application may also be assigned to perform layer composition for other applications. For example, it is also assigned to compose the layers of the second application.

[0144] Then, if the first layer of the first application misses this second vertical synchronization signal (VSYNC_SF signal) due to drawing timeout and fails to perform layer composition in time, the corresponding composition thread may perform layer composition on the layers of other applications in response to this second vertical synchronization signal (VSYNC_SF signal).

[0145] That is, at the first moment when the first layer is cached, the corresponding composition thread may be performing layer composition on the layers of other applications in response to this second vertical synchronization signal (VSYNC_SF signal). Furthermore, as long as the image frame (i.e., the first image frame) obtained by the current layer composition is successfully stored in the second buffer queue. When the third vertical synchronization signal (HW_VSYNC signal) corresponding to a synchronization period (the second preset period) after this second vertical synchronization signal (VSYNC_SF signal) arrives, the electronic device will surely first read the image frame (the first image frame) of other applications from the second buffer queue to refresh the display.

[0146] Therefore, even if the electronic device immediately triggers layer composition on the first layer to obtain the second image frame at this first moment. This second image frame cannot be successfully refreshed and displayed when this third vertical synchronization signal (HW_VSYNC signal) arrives.

[0147] For example, as Figure 8 shown, the electronic device draws and renders layer 2 (the first layer) due to timeout, resulting in storing layer 2 (the first layer) in the first cache queue at time tt (the first moment). However, at this time, the second vertical synchronization signal (VSYNC_SF signal) at time t3 corresponding to time tt (the first moment) is being responded to by the electronic device to execute "image frame composition 3" to obtain image frame 3 (i.e., performing layer composition on the layers of other applications to obtain the first image frame).

[0148] Furthermore, after the third vertical synchronization signal (HW_VSYNC signal) at time t4 arrives, the electronic device will surely respond to this third vertical synchronization signal (HW_VSYNC signal) to execute "image frame display 3" to refresh and display image frame 3 (the first image frame). Therefore, even if the electronic device immediately triggers the execution of "image frame composition 2" at the first moment (time tt) to perform layer composition on layer 2 (the first layer), the obtained image frame 2 (the second image frame) of the electronic device will not be refreshed and displayed within the synchronization period TZ of the third vertical synchronization signal (HW_VSYNC signal) at time t4.

[0149] As Figure 8 shown, this image frame 2 (the second image frame) may be delayed until it is refreshed and displayed within the synchronization period TZ of the third vertical synchronization signal (HW_VSYNC signal) at time t5. Thus, even if the electronic device triggers immediate composition, it cannot avoid the loss of image frame 2 (the second image frame).

[0150] Therefore, if the electronic device determines that the first moment is after the second vertical synchronization signal (VSYNC_SF signal), it indicates that the electronic device can determine that immediate composition is needed to avoid frame loss. However, the electronic device still needs to further determine whether it can avoid frame loss by triggering immediate composition at present. That is, in this case, the electronic device needs to enter S703 to further determine whether there is a first image frame to be refreshed and displayed in the third vertical synchronization signal (HW_VSYNC signal).

[0151] If the electronic device determines that there is a first image frame to be refreshed and displayed in this third vertical synchronization signal (HW_VSYNC signal), for example, as the "image frame composition 3" shown above Figure 8 where the second vertical synchronization signal (VSYNC_SF signal) is responded to by the composition thread and layer composition is in progress. Then, even if the electronic device triggers immediate composition, it cannot avoid the loss of the second image frame. Therefore, in the embodiments of the present application, no additional processing is added, and the electronic device ends this processing flow.

[0152] If the electronic device determines that there is no first image frame to be refreshed and displayed in this third vertical synchronization signal (HW_VSYNC signal), then when the electronic device triggers immediate synthesis at this time, it can ensure that the second image frame is successfully displayed within the synchronization period TZ of this third vertical synchronization signal (HW_VSYNC signal). At this time, even if the rendering of the first layer by the electronic device times out, it will not cause the second image frame to be dropped, thus avoiding the frame drop problem.

[0153] Therefore, in this case, the electronic device enters S704. That is, at the first moment, the electronic device immediately performs layer synthesis on the first layer in the first buffer queue to obtain a second image frame, and caches the second image frame in the second buffer queue.

[0154] Specifically, at the first moment, the electronic device can immediately send an instruction to the synthesis thread, instructing the synthesis thread to immediately read the first layer from the first buffer queue for layer synthesis to obtain the second image frame corresponding to the first layer.

[0155] Meanwhile, after completing the layer synthesis of the second image frame, the second image frame is cached in the second buffer queue. Subsequently, in response to the third vertical synchronization signal (HW_VSYNC signal), the display driver reads the second image from the second buffer queue and refreshes and displays this second image frame on the display.

[0156] After the electronic device immediately completes the layer synthesis of the first layer in the first buffer queue at the first moment to obtain the corresponding second image frame and caches the second image frame in the second buffer queue, according to the normal process, the electronic device can refresh and display the second image frame in response to the corresponding third vertical synchronization signal (HW_VSYNC signal).

[0157] Thus, in the embodiment of the present application, when it is ensured that the corresponding third vertical synchronization signal can refresh and display the second image frame, immediate synthesis is triggered, thereby playing a role in avoiding frame drop of the second image frame.

[0158] However, in some other embodiments, even if the second image frame is immediately synthesized at the first moment and there is no first image frame to be refreshed and displayed in the third vertical synchronization signal, there is still a situation that may cause the second image frame to not be successfully refreshed and displayed when the third vertical synchronization signal (HW_VSYNC signal) arrives. That is, the first moment is after the third vertical synchronization signal (HW_VSYNC signal).

[0159] Because, if the electronic device completes rendering in response to the first vertical synchronization signal (VSYNC_APP) and caches the first layer at the first moment in the first cache queue, and this first moment has already missed the corresponding third vertical synchronization signal (HW_VSYNC signal), then even if the synthesis of the first layer is triggered immediately at the first moment, no matter how fast the layer synthesis speed of the synthesis thread is, it is in a state of missing this third vertical synchronization signal (HW_VSYNC signal). Therefore, the second image frame still cannot be displayed within the synchronization period TZ of this corresponding third vertical synchronization signal (HW_VSYNC signal), resulting in frame loss

[0160] For example, as Figure 9 shown, the electronic device times out in rendering layer 2 (the first layer) and caches layer 2 (the first layer) in the first cache queue only at time tw (the first moment). At this time, time tw is after time t4, so even if the electronic device immediately performs layer synthesis on layer 2 (the first layer) at time tw (the first moment) to obtain image frame 2 (the second image frame). This second image frame cannot be smoothly refreshed and displayed between time t4 and time t5. As Figure 9 shown, this image frame 2 (the second image frame) may still be delayed until it is refreshed and displayed between time t5 and time t6 (time t6 is the first moment after time t5, not shown in the figure). At this time, image frame 2 (the second image frame) is still a lost frame.

[0161] Therefore, in the embodiment of the present application, after the electronic device immediately synthesizes the second image frame at the first moment and caches it in the second cache queue, it can further determine whether the first moment is before this corresponding third vertical synchronization signal (HW_VSYNC signal). That is, after the electronic device executes S704, it enters S705 to determine whether the first moment is before the third vertical synchronization signal (HW_VSYNC signal).

[0162] If the first moment is after this corresponding third vertical synchronization signal (HW_VSYNC signal) (that is, the first moment is not before the corresponding third vertical synchronization signal (HW_VSYNC signal)). Then, the second image frame will inevitably be lost, so the embodiment of the present application does not add additional processing, and the electronic device ends this processing flow.

[0163] If the first moment is before this corresponding third vertical synchronization signal (HW_VSYNC signal), it means that the electronic device can, when the third vertical synchronization signal (HW_VSYNC signal) arrives, in response to this third vertical synchronization signal (HW_VSYNC signal), read the second image frame from the second cache queue and smoothly refresh and display this second image frame, thus ensuring no frame loss.

[0164] That is to say, if the first moment is before the corresponding third vertical synchronization signal (HW_VSYNC signal), the electronic device enters S707, waits for the corresponding third vertical synchronization signal (HW_VSYNC signal) to arrive according to the conventional process, and refreshes and displays the synthesized second image frame in response to this third vertical synchronization signal (HW_VSYNC signal). Thus, the second image frame is smoothly displayed without frame loss.

[0165] As Figure 10 shown, the embodiment of the present application shows a flowchart of another image processing method based on a vertical synchronization signal, which may include steps S1001 - S1008.

[0166] Compare Figure 10 with Figure 7 It can be seen that the embodiment of the present application adds S1008 to the process shown in Figure 7 The electronic device determines the screen refresh moment according to the predetermined frame rate, and refreshes and displays the second image frame in the second buffer queue at the screen refresh moment.

[0167] In the embodiment of the present application, for the inevitable frame loss situation where the first moment is after the third vertical synchronization signal (HW_VSYNC signal), the embodiment of the present application tries to display the second image frame as early as possible by adding the screen refresh moment, thereby reducing the display impact caused by frame loss and ensuring the smoothness of the display screen as much as possible.

[0168] It should be noted that the specific implementation of S1001 - S1007 in the embodiment of the present application can refer to the specific implementation of S701 - S707 in the above embodiment. The specific implementation methods and principles are the same, and the embodiment of the present application will not elaborate on this.

[0169] Hereinafter, the embodiment of the present application mainly elaborates on Figure 10 S1008 in

[0170] Specifically, if the first moment is after the third vertical synchronization signal (HW_VSYNC signal), then no matter how fast the layer composition of the first layer is, the second image frame cannot be displayed within the synchronization period TZ corresponding to this third vertical synchronization signal (HW_VSYNC signal). For example, as Figure 9 shown, the specific analysis of this situation can refer to the above analysis of Figure 9 and will not be elaborated here.

[0171] However, although the second image frame missed the refresh display when this third vertical synchronization signal (HW_VSYNC signal) arrived at this time, the second image frame has been successfully cached in the second cache queue and is waiting for the display screen driver to read and refresh the display. It can be understood that in the embodiment of the present application, the second image frame is in a state where it can be read and refreshed for display at any time.

[0172] Therefore, in order to reduce the impact of frame loss, display the second image frame as soon as possible, and prevent the impact on the refresh display of subsequent image frames. In the embodiment of the present application, a new screen refresh moment is determined according to a predetermined frame rate to refresh and display the lost second image frame in advance. That is to say, the electronic device reads the second image frame in the second cache queue at the new screen refresh moment for refresh display.

[0173] Among them, the predetermined frame rate in the embodiment of the present application is greater than the frame rate of the display screen of the electronic device. The predetermined frame rate can be set according to the display requirements and the capabilities supported by the display hardware. And the screen refresh moment when the electronic device reads the second image frame in the second cache queue for refresh display is after the third vertical synchronization signal (HW_VSYNC signal) and before the fourth vertical synchronization signal (HW_VSYNC signal). Among them, this fourth vertical synchronization signal (HW_VSYNC signal) is also a vertical synchronization signal used to trigger the display of the image frame, and the fourth vertical synchronization signal (HW_VSYNC signal) is after the third vertical synchronization signal (HW_VSYNC signal) and is separated from this third vertical synchronization signal (HW_VSYNC signal) by a synchronization period. It can be understood that since both the third vertical synchronization signal and the fourth vertical synchronization signal are HW_VSYNC signals, there is a fixed interval of one synchronization period between the third vertical synchronization signal and the fourth vertical synchronization signal, that is, the reciprocal of the screen refresh rate.

[0174] That is to say, the screen refresh moment determined in the embodiment of the present application is between the third vertical synchronization signal (HW_VSYNC signal) and the fourth vertical synchronization signal (HW_VSYNC signal). That is, the screen refresh moment is between this missed third vertical synchronization signal (HW_VSYNC signal) and the next third vertical synchronization signal (HW_VSYNC signal) corresponding to this third vertical synchronization signal (HW_VSYNC signal). For example, the screen refresh moment is between the t4 moment (the third vertical synchronization signal) and the t5 moment (the fourth vertical synchronization signal, that is, the next third vertical synchronization signal corresponding to the t4 moment).

[0175] Generally speaking, an electronic device (such as a display driver) determines the refresh time of an image frame (such as a second image frame) according to the frame rate of the display screen (i.e., the screen refresh rate). That is to say, when the rates of the producer and the consumer are consistent and there is no frame loss, usually one image frame is refreshed and displayed within one synchronization period on the display driver side.

[0176] For example, for a frame rate of 60HZ, one image frame is refreshed and displayed every 16.667 milliseconds, that is, the time interval between the refresh times of the image frames is 6.667 milliseconds. For a frame rate of 90HZ, one image frame is refreshed and displayed every 11.11 milliseconds, that is, the time interval between the refresh times of the image frames is 11.11 milliseconds. And for a frame rate of 120HZ, one image frame is refreshed and displayed every 8.33 milliseconds, that is, the time interval between the refresh times of the image frames is 8.83 milliseconds.

[0177] It can be seen from this that the higher the frame rate of the display screen of the electronic device (i.e., the screen refresh rate), the shorter the time interval for the image frame to be refreshed and displayed. Therefore, in the case of frame loss, a new screen refresh time can be determined by a predetermined frame rate that is higher than the frame rate of the display screen of the electronic device (i.e., the screen refresh rate). It can be understood that since the predetermined frame rate is higher than the frame rate of the display screen of the electronic device (i.e., the screen refresh rate), the number of screen refresh times corresponding to the predetermined frame rate is more and more compact compared to the screen refresh times corresponding to the frame rate of the display screen of the electronic device (i.e., the screen refresh rate). Furthermore, when the electronic device refreshes and displays the second image frame at the screen refresh time corresponding to the predetermined frame rate, the effect of displaying the second image frame in advance can be achieved.

[0178] At the same time, in the embodiments of the present application, the electronic device only refreshes and displays the second image frame at the screen refresh time between the third vertical synchronization signal (HW_VSYNC signal) and the fourth vertical synchronization signal (HW_VSYNC signal), so as to ensure that the second image frame can be refreshed and displayed within the synchronization period TZ of this corresponding third vertical synchronization signal (HW_VSYNC signal), so that an image frame can also be displayed within the original synchronization period TZ with frame loss.

[0179] It can be seen from this that although the display of the second image frame is a bit delayed compared to the case without frame loss, compared to refreshing and displaying at the next vertical synchronization signal, the second image frame is displayed in advance. Therefore, overall, the time of the delayed display of the second image frame is shortened, and the time of the unsmooth picture is reduced, so that the smoothness of the picture display can be restored as soon as possible, and the influence caused by frame loss can be reduced.

[0180] For example, as Figure 11 shown, taking the frame rate of the display screen as 120HZ and the predetermined frame rate as 360HZ as an example, the refresh display process in the embodiments of the present application will be described.

[0181] like Figure 11 As shown, if the frame rate of the display screen of the electronic device is 120HZ, then the original screen refresh time is Figure 11 The time interval between each image refresh time at 120HZ is 8.83 milliseconds. According to the conventional refresh display process, because the first time is after the t4 time, the image frame 2 (the second image frame) cannot be refreshed and displayed at the t4 time. The image frame 2 (the second image frame) will be delayed by 8.83 milliseconds and will not be refreshed and displayed until the t5 time (such as Figure 9 The image frames shown in display 2).

[0182] However, in the embodiment of the present application, the electronic device determines that the first time is after time t4, and then determines a new screen refresh time according to the predetermined frame rate 360HZ. Figure 11 As shown, the t1 moment, ta moment, tb moment, t2 moment, tc moment, td moment, t3 moment, te moment, tf moment, t4 moment, tg moment, th moment and t5 moment are all the screen refresh moments under 360HZ. It can be understood that the specific number of new screen refresh moments is determined according to the predetermined frame rate. Figure 11 This is only an exemplary illustration and does not constitute a limitation. At the same time, these picture refresh moments are picture refresh moments under 360 Hz, so the time interval between these picture refresh moments is 1 / 360=0.00277 seconds=2.77 milliseconds.

[0183] Therefore, a corresponding screen refresh time is added between time t4 (third vertical synchronization signal) and time t5 (fourth vertical synchronization signal), such as Figure 11 The tg time and th time are shown.

[0184] Furthermore, if the first moment is after moment t4, the electronic device misses the third vertical synchronization signal (HW_VSYNC signal) in response to moment t4 and does not refresh the displayed image frame 2 (such as the second image frame). In the embodiment of the present application, the electronic device can refresh the displayed image frame 2 (such as the second image frame) again at moment tg. At this time, although image frame 2 (such as the second image frame) is delayed by tg-t4 milliseconds, it can be displayed t5-tg milliseconds earlier than when it is displayed at moment t5.

[0185] For example, at 360HZ, image frame 2 (such as the second image frame) can be displayed at the earliest about 2.7ms after time t4. Compared with not displaying it in advance and displaying it at time t5, image frame 2 (such as the second image frame) can be displayed at most about 5.4ms in advance, thereby reducing the impact of frame loss.

[0186] It should be noted that increasing new frame refresh times at a predetermined frame rate in the embodiments of the present application does not change the original frame rate of the display screen. It can be understood that after the early display of the dropped second image frame is completed in the embodiments of the present application, the electronic device will still perform the processes of drawing, rendering, compositing, and refreshing the display according to the original frame rate (i.e., the screen refresh rate) of the display screen. For example, the frame rate of the display screen is always 120HZ and will not be changed to 360HZ.

[0187] In some embodiments, in addition to the second image frame not being able to be smoothly refreshed and displayed after the first moment after the third vertical synchronization signal (HW_VSYNC signal) (for example, the first moment tw is after the t4 moment), it is also possible that the layer composition of the layer by the composition thread times out, resulting in missing the third vertical synchronization signal (HW_VSYNC signal) and being unable to smoothly refresh and display the second image frame within the synchronization period TZ of the third vertical synchronization signal (HW_VSYNC signal).

[0188] That is, the first moment is before the third vertical synchronization signal (HW_VSYNC signal), but the second moment when the electronic device caches the second image frame into the second cache queue is after the third vertical synchronization signal (HW_VSYNC signal), which will also result in missing the third vertical synchronization signal (HW_VSYNC signal) and being unable to smoothly refresh and display the second image frame within the synchronization period TZ of the third vertical synchronization signal (HW_VSYNC signal).

[0189] As Figure 12 shown, the electronic device caches layer 2 (the first layer) into the first cache queue at the tw moment (the first moment) before the t4 moment. At the tw moment (the first moment), the electronic device reads layer 2 (the first layer) from the first cache queue and performs "image frame composition 2" to obtain image frame 2 (the second image frame).

[0190] However, at this time, the composition thread only completes the layer composition and caching of image frame 2 (the second image frame) at the tv moment (the second moment). However, the tv moment (the second moment) is after the t4 moment, thus also missing the third vertical synchronization signal (HW_VSYNC signal) at the t4 moment, resulting in image frame 2 (the second image frame) being delayed and refreshed and displayed at the t5 moment. In this case, image frame 2 (the second image frame) is not smoothly displayed at the t4 moment, resulting in inevitable frame loss.

[0191] To reduce the impact of frame loss caused by the timeout of the synthesis thread during synthesis, in the embodiments of the present application, for the situation where the first moment is before the third vertical synchronization signal (HW_VSYNC signal), but the second moment (i.e., the moment when the electronic device caches the second image frame in the second buffer queue) is after the third vertical synchronization signal (HW_VSYNC signal), the new screen refresh moment can also be determined according to a predetermined frame rate. Furthermore, at the new screen refresh moment, the electronic device refreshes and displays the second image frame in the second buffer queue.

[0192] For example, as Figure 13 shown. At time tw (the first moment), the electronic device reads Layer 2 (the first layer) from the first buffer queue and executes "Image Frame Synthesis 2", and obtains Image Frame 2 (the second image frame) at time tv (the second moment) and caches it in the second buffer queue. At this time, although the electronic device misses the third vertical synchronization signal (HW_VSYNC signal) at time t4, the electronic device can re-refresh and display Image Frame 2 (the second image frame) at time tg (the screen refresh moment). Thus, the electronic device advances the refresh display of Image Frame 2 (the second image frame) to time tg (the screen refresh moment), reducing the impact of frame loss.

[0193] As Figure 14 shown, for the situation in the embodiments of the present application where the first moment is before the third vertical synchronization signal (HW_VSYNC signal), but the second moment is after the third vertical synchronization signal (HW_VSYNC signal), the embodiments of the present application show a flowchart of another image processing method based on the vertical synchronization signal, including steps S1401 - S1408.

[0194] It can be understood that this situation in the embodiments of the present application is similar to the situation where the first moment is after the third vertical synchronization signal (HW_VSYNC signal), and their processing principles are the same. Therefore, Figure 14 the specific implementation of each step in Figure 7 and Figure 10 the specific implementation of each step in, the embodiments of the present application will not elaborate on this again.

[0195] In some embodiments, there are multiple screen refresh moments at which the electronic device can re-refresh and display the second image frame determined according to the predetermined frame rate. For example, Figure 11 and Figure 13 the tg moment and th moment in. In the embodiments of the present application, for the time sequence of the screen refresh moments in order of time, the first screen refresh moment is called the first refresh moment. At least one screen refresh moment after the first screen refresh moment is called the second refresh moment.

[0196] Because the first refresh moment and at least one second refresh moment are not determined by changing the frame rate of the display screen, but are determined based on a predetermined frame rate. Therefore, the electronic device may fail to refresh the display again. Generally speaking, the display is successfully refreshed again at the first refresh moment. However, in order to ensure as much as possible that the second image frame is successfully refreshed and displayed in advance, the embodiments of the present application may refresh and display the second image frame again at each screen refresh moment (including the first refresh moment and the second refresh moment) based on a traversal method.

[0197] Specifically, the electronic device, in chronological order, first refreshes and displays the second image frame in the second buffer queue at the first refresh moment. If the second image frame is successfully refreshed and displayed at the first refresh moment, the electronic device stops refreshing and displaying the second image frame. If the second image frame fails to be refreshed and displayed at the first refresh moment, the electronic device then, in chronological order, traverses and refreshes and displays the second image frame in advance at each second refresh moment in turn until the second image frame is successfully refreshed and displayed. That is to say, during the process of traversing the second refresh moment, once the electronic device successfully refreshes and displays the second image frame at a certain second refresh moment, indicating that the second image frame is successfully displayed in advance, the electronic device ends the process of refreshing and displaying the second image frame in advance, that is, stops traversing.

[0198] Or, after all the screen refresh moments have been traversed, the electronic device can also stop traversing. For example, Figure 11 For example, that is, if the electronic device fails to successfully refresh and display the second image frame at both the tg moment and the th moment, the electronic device stops traversing, and then refreshes and displays the second image frame when the third vertical synchronization signal (HW_VSYNC signal) arrives at the t5 moment.

[0199] Thus, the embodiments of the present application can refresh and display the second image frame again by traversing the screen refresh moment, and can ensure as much as possible that the second image frame is successfully refreshed and displayed.

[0200] In some embodiments, since the composition thread needs a certain amount of processing time to perform layer composition, in order to avoid as much as possible that the composition thread fails to complete the composition of the image frame before the third vertical synchronization signal (HW_VSYNC signal) arrives.

[0201] For the case where the first moment is before the second vertical synchronization signal (VSYNC_SF signal), but the time difference between the first moment and the second vertical synchronization signal is less than or equal to a preset time threshold, the embodiments of the present application regard this first moment as after the second vertical synchronization signal. Among them, the preset time threshold can be set according to experience and the actual composition time required by the composition thread. For example, the preset time threshold can be 1 millisecond, 2 milliseconds, etc.

[0202] Taking 2 milliseconds as an example, that is to say, except for the first moment after the second vertical synchronization signal (VSYNC_SF signal), for those first moments within 2 milliseconds before the second vertical synchronization signal (VSYNC_SF signal), they are all regarded as the first moment after the second vertical synchronization signal (VSYNC_SF signal). For example, the moment of the second vertical synchronization signal (VSYNC_SF signal) is the 5th millisecond. Then, all the first moments after the 3rd millisecond are the first moments after the second vertical synchronization signal (VSYNC_SF signal) at the 5th millisecond.

[0203] In some embodiments, generally when there is no abnormality (such as frame loss), the rate at which the display driver reads and sends the image frame for display is also consistent with the generation and consumption rates. It can be understood that the consumer (synthesis thread) synthesizes one image frame every VSYNC cycle (such as the above-mentioned synchronization cycle TZ) and caches it into the second cache queue, and the display driver fetches one image frame from the second cache queue every other VSYNC cycle (such as the above-mentioned synchronization cycle TZ) and refreshes and displays it on the display screen.

[0204] Therefore, for whether there is a first image frame to be refreshed and displayed for the third vertical synchronization signal (HW_VSYNC signal) in the above embodiments, in addition to the case where the corresponding second vertical synchronization signal (VSYNC_SF signal) is responded with a first image frame being synthesized, there may also be another case.

[0205] That is, currently (i.e., the first moment), the second cache queue includes a first image frame that can be refreshed and displayed. Because, if the second cache queue includes a first image frame waiting to be refreshed and displayed at the first moment, it means that when the third vertical synchronization signal (HW_VSYNC signal) arrives, the electronic device will surely respond to this third vertical synchronization signal (HW_VSYNC signal) and first refresh and display the first image frame waiting to be displayed in the second cache queue.

[0206] Thus, the first image frame that is still being synthesized or waiting to be synthesized at this time cannot be refreshed and displayed when this third vertical synchronization signal (HW_VSYNC signal) arrives. Therefore, this application embodiment also regards this situation as the third vertical synchronization signal (HW_VSYNC signal) having a first image frame that can be refreshed.

[0207] For example, as Figure 15 shown, if the electronic device finishes rendering the drawing of layer 1 (the first layer) and caches layer 2 (the first layer) in the first cache queue at the tt moment (the first moment) after the t3 moment. That is, the electronic device processes the drawing of layer 1 (the first layer) with a timeout. At this time, the number of image frames in the first cache queue increases from 0 to 1 (i.e., 0->1).

[0208] Meanwhile, if the second buffer queue currently includes the first image frame that can be refreshed and displayed. That is, at time tt, a frame of the first image frame (illustrated as "1") is stored in the second buffer queue.

[0209] Then, if the electronic device triggers the immediate synthesis of layer 2 (the first layer) at this time, obtaining the image frame 2 (the second image frame), and this image frame 2 (the second image frame) is cached in the second buffer queue. At this time, the number of image frames in the second buffer queue will increase from 1 to 2 (i.e., 1 -> 2).

[0210] Furthermore, when the third vertical synchronization signal (HW_VSYNC signal) arrives at time t4, if the display driver reads the image frame from the second buffer queue for refreshing and displaying, following the first-in, first-out characteristic of the queue, the dequeue order of the image frame 2 (the second image frame) is after the first image frame. The display driver will necessarily read the first image frame instead of this immediately synthesized image frame 2 (the second image frame). At this time, at time t4, the number of image frames in the second buffer queue will decrease from 2 to 1 (i.e., 2 -> 1).

[0211] And this image frame 2 (the second image frame) in the second buffer queue may be refreshed and displayed at time t5. That is, at time t5, the number of image frames in the second buffer queue will decrease from 1 to 0 (i.e., 1 -> 0).

[0212] It can be seen that in this case, even if the electronic device immediately synthesizes the image frame 2 (the second image frame). This image frame 2 (the second image frame) will not be sent for display at time t4. For the second image frame, it is still in a state of frame loss. Therefore, when at the first moment (time tt), there are other first image frames waiting to be refreshed and sent for display cached in the second buffer queue, the embodiment of the present application also regards that there are first image frames that can be refreshed in the third vertical synchronization signal (HW_VSYNC signal).

[0213] In summary, to determine whether there are first image frames to be refreshed and displayed in the third vertical synchronization signal (HW_VSYNC signal), it is necessary to determine whether there is layer synthesis for other layers of other applications in response to the second vertical synchronization signal (VSYNC_SF signal). At the same time, it is also necessary to determine whether there are first image frames to be displayed in the second buffer queue.

[0214] Only when there is no layer synthesis in response to the second vertical synchronization signal (VSYNC_SF signal) (that is, no synthesis is performed on the layers of any other applications), and there are no first image frames to be refreshed and displayed in the second buffer queue, can it be determined that there are no first image frames to be refreshed and displayed in the third vertical synchronization signal (HW_VSYNC signal).

[0215] Such asFigure 16 As shown, a schematic structural diagram of the above electronic device is shown.

[0216] The electronic device 1600 may include a processor 1610, an external memory interface 1620, an internal memory 1621, a universal serial bus (USB) connector 1630, a charging management module 1640, a power management module 1641, a battery 1642, an antenna 1, an antenna 2, a mobile communication module 1650, a wireless communication module 1660, an audio module 1670, a speaker 1670A, a receiver 1670B, a microphone 1670C, a headphone interface 1670D, a sensor module 1680, a key 1690, a motor 1691, an indicator 1692, a camera module 1693, a display screen 1694, and a subscriber identification module (SIM) card interface 1695, etc. The sensor module 1680 may include a pressure sensor 1680A, a gyroscope sensor 1680B, a barometric pressure sensor 1680C, a magnetic sensor 1680D, an acceleration sensor 1680E, a distance sensor 1680F, a proximity light sensor 1680G, a fingerprint sensor 1680H, a temperature sensor 1680J, a touch sensor 1680K, an ambient light sensor 1680L, a bone conduction sensor 1680M, etc.

[0217] It can be understood that the structure schematically shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 1600. In other embodiments of the present application, the electronic device 1600 may include more or fewer components than shown, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0218] The processor 1610 may include one or more processing units. For example, the processor 1610 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors. The processor 1610 can generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions.

[0219] A memory may also be provided in the processor 1610 for storing instructions and data. In some embodiments, the memory in the processor 1610 may be a cache memory. This memory can save the instructions or data that the processor 1610 has used or uses frequently. If the processor 1610 needs to use this instruction or data, it can directly call it from this memory. This avoids repeated accesses, reduces the waiting time of the processor 1610, and thus improves the efficiency of the system.

[0220] In some embodiments, the processor 1610 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc. The processor 1610 can be connected to modules such as a touch sensor, an audio module, a wireless communication module, a display screen, a camera module, etc. through at least one of the above interfaces.

[0221] It can be understood that the interface connection relationships among the modules illustrated in the embodiments of the present application are only illustrative descriptions and do not constitute a structural limitation on the electronic device 1600. In other embodiments of the present application, the electronic device 1600 may also adopt different interface connection manners in the above embodiments, or a combination of multiple interface connection manners.

[0222] The external memory interface 1620 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 1600. The external memory card communicates with the processor 1610 through the external memory interface 1620 to implement the data storage function. For example, files such as music and videos are saved in the external memory card, or files such as music and videos are transferred from the electronic device to the external memory card.

[0223] The internal memory 1621 can be used to store computer-executable program codes, and the executable program codes include instructions. The internal memory 1621 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area can store data created during the use of the electronic device 1600 (such as audio data, phone book, etc.). In addition, the internal memory 1621 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 1610 executes various functional methods or data processing of the electronic device 1600 by running the instructions stored in the internal memory 1621 and / or the instructions stored in the memory provided in the processor.

[0224] The USB connector 1630 is an interface that conforms to the USB standard specification and can be used to connect the electronic device 1600 and peripheral devices, specifically, it can be a Mini USB connector, a Micro USB connector, a USB Type C connector, etc. The USB connector 1630 can be used to connect a charger to charge the electronic device 1600, and can also be used to connect other electronic devices to implement data transmission between the electronic device 1600 and other electronic devices. It can also be used to connect headphones to output the audio stored in the electronic device through the headphones. This connector can also be used to connect other electronic devices, such as VR devices, etc. In some embodiments, the standard specification of the universal serial bus can be USB1.x, USB2.0, USB3.x, and USB4.

[0225] The charging management module 1640 is used to receive the charging input from the charger. While charging the battery 1642, the charging management module 1640 can also supply power to the electronic device through the power management module 1641.

[0226] The power management module 1641 is used to connect the battery 1642, the charging management module 1640, and the processor 1610. The power management module 1641 receives the inputs from the battery 1642 and / or the charging management module 1640, and supplies power to the processor 1610, the internal memory 1621, the display screen 1694, the camera module 1693, the wireless communication module 1660, etc. In some other embodiments, the power management module 1641 can also be disposed in the processor 1610. In some other embodiments, the power management module 1641 and the charging management module 1640 can also be disposed in the same device.

[0227] The wireless communication function of the electronic device 1600 can be implemented by the antenna 1, the antenna 2, the mobile communication module 1650, the wireless communication module 1660, the modulation and demodulation processor, and the baseband processor, etc.

[0228] The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 1600 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example, the antenna 1 can be multiplexed as the diversity antenna of the wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0229] The mobile communication module 1650 can provide wireless communication solutions such as 2G / 3G / 4G / 5G, etc. applied to the electronic device 1600. The mobile communication module 1650 can include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 1650 can receive electromagnetic waves from the antenna 1, filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 1650 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves through the antenna 1 and radiate it out.

[0230] The modulation and demodulation processor can include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Subsequently, the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 1670A, the receiver 1670B, etc.), or displays an image or video through the display screen 1694.

[0231] The wireless communication module 1660 can provide solutions for wireless communication applied to the electronic device 1600, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), Bluetooth low energy (BLE), ultra wide band (UWB), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The wireless communication module 1660 can be one or more devices integrating at least one communication processing module. The wireless communication module 1660 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signals, and sends the processed signals to the processor 1610. The wireless communication module 1660 can also receive the signals to be sent from the processor 1610, perform frequency modulation and amplification on them, and convert them into electromagnetic waves through the antenna 2 for radiation.

[0232] In some embodiments, antenna 1 of electronic device 1600 is coupled to mobile communication module 1650, and antenna 2 is coupled to wireless communication module 1660, so that electronic device 1600 can communicate with a network and other electronic devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS), and / or satellite based augmentation systems (SBAS).

[0233] Electronic device 1600 may implement a display function through a GPU, display screen 1694, and an application processor, etc. The GPU is a microprocessor for image processing, and is connected to display screen 1694 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 1610 may include one or more GPUs, which execute program instructions to generate or change display information.

[0234] The display screen 1694 is used to display images, videos, etc. The display screen 1694 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 1600 may include one or more display screens 1694.

[0235] The electronic device 1600 can implement the camera function through the camera module 1693, the ISP, the video codec, the GPU, the display screen 1694, the application processor AP, the neural network processor NPU, etc. The camera module 1693 can be used to collect the color image data and depth data of the shooting object. The ISP can be used to process the color image data collected by the camera module 1693. The digital signal processor is used to process digital signals and can also process other digital signals. The video codec is used to compress or decompress digital videos. The electronic device 1600 can support one or more video codecs. In this way, the electronic device 1600 can play or record videos in multiple encoding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0236] In some embodiments, the CPU, GPU, or NPU in the processor 1610 can process the color image data and depth data collected by the camera module 1693. The NPU is a neural-network (NN) computing processor. By learning from the biological neural network structure, such as learning from the transmission mode between human brain neurons, it can quickly process the input information and can also continuously self-learn. Through the NPU, applications such as intelligent cognition of the electronic device 1600 can be realized, such as: image recognition, face recognition, voice recognition, text understanding, etc.

[0237] In some embodiments, the electronic device 1600 may include one or more camera modules 1693. Specifically, the electronic device 1600 may include one front camera module 1693 and one rear camera module 1693. The camera module 1693 may be composed of a color camera module and a 3D sensing module. The camera module 1693 may also be composed of two or more cameras.

[0238] The electronic device 1600 may implement audio functions through the audio module 1670, the speaker 1670A, the receiver 1670B, the microphone 1670C, the headphone jack 1670D, and the application processor, etc. For example, music playback, recording, etc.

[0239] The audio module 1670 is used to convert digital audio information into an analog audio signal for output, and is also used to convert analog audio input into digital audio signals. The audio module 1670 can also be used for encoding and decoding audio signals. The speaker 1670A, also known as the "loudspeaker", is used to convert an audio electrical signal into a sound signal. The receiver 1670B, also known as the "earpiece", is used to convert an audio electrical signal into a sound signal. The microphone 1670C, also known as the "microphone", "transmitter", is used to convert a sound signal into an electrical signal. The headphone jack 1670D is used to connect a wired headphone.

[0240] The pressure sensor 1680A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 1680A may be disposed on the display screen 1694. There are many types of pressure sensors 1680A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor may include at least two parallel plates with conductive materials. When a force acts on the pressure sensor 1680A, the capacitance between the electrodes changes. The electronic device 1600 determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 1694, the electronic device 1600 detects the intensity of the touch operation according to the pressure sensor 1680A. The electronic device 1600 can also calculate the position of the touch based on the detection signal of the pressure sensor 1680A. In some embodiments, touch operations with the same touch position but different touch operation intensities may correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, the instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, the instruction to create a new short message is executed.

[0241] The touch sensor 180K, also referred to as a "touch control device". The touch sensor 180K can be disposed on the display screen 1694. The touch sensor 180K and the display screen 1694 form a touch screen, also referred to as a "touch control screen". The touch sensor 180K is used to detect a touch operation acting thereon or in its vicinity. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 1694. In some other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 1600, at a different position from where the display screen 1694 is located.

[0242] The button 1690 can include a power-on button, a volume button, etc. The button 1690 can be a mechanical button. It can also be a touch button. The electronic device 1600 can receive button inputs and generate key signal inputs related to the user settings and function control of the electronic device 1600. The motor 1691 can generate a vibration prompt. The motor 1691 can be used for incoming call vibration prompts and can also be used for touch vibration feedback. The indicator 1692 can be an indicator light and can be used to indicate the charging state, power change, and can also be used to indicate messages, missed calls, notifications, etc. The SIM card interface 1695 is used to connect the SIM card. The SIM card can be in contact with and separated from the electronic device 1600 by inserting or removing the SIM card from the SIM card interface 1695. The electronic device 1600 can support one or more SIM card interfaces. The SIM card interface 1695 can support Nano SIM cards, Micro SIM cards, SIM cards, etc.

[0243] Another embodiment of this application provides an electronic device, including: a display screen, one or more processors, and a memory. The display screen and the memory are respectively coupled to the processor; one or more computer program codes are stored in the memory, and the computer program codes include computer instructions; when the processor executes the computer instructions, the electronic device implements the image processing method based on the vertical synchronization signal described in any of the above embodiments.

[0244] Another embodiment of this application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor in an electronic device, the electronic device implements the image processing method based on the vertical synchronization signal described in any of the above embodiments.

[0245] The embodiments of this application also provide a computer program product. When the computer program product runs on a computer, the computer executes each function or step in the above method embodiments.

[0246] The embodiments of this application also provide a chip system, such as Figure 17As shown, the chip system 1700 includes at least one processor 1701 and at least one interface circuit 1702. The processor 1701 and the interface circuit 1702 can be interconnected by a line. For example, the interface circuit 1702 can be used to receive signals from other devices (such as the memory of a computer). For another example, the interface circuit 1702 can be used to send signals to other devices (such as the processor 1701).

[0247] Exemplarily, the interface circuit 1702 can read the instructions stored in the memory and send the instructions to the processor 1701. When the instructions are executed by the processor 1701, the computer can execute each step in the above embodiments. Of course, the chip system can also include other discrete devices, and the embodiments of the present application do not make specific limitations on this.

[0248] Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and brevity of description, only the above division of each functional module is used for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0249] In several embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0250] The units described as separate components may or may not be physically separated. The components displayed as units can be one physical unit or multiple physical units, that is, they can be located in one place, or can be distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0251] In addition, in each embodiment of the present application, each functional unit can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0252] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods of the embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0253] The above content is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An image processing method based on a vertical synchronization signal, characterized in that Applied to an electronic device, the method includes: In response to a first vertical synchronization signal, draw and render a first layer of a first application, and cache the first layer into a first cache queue at a first moment; The first moment is after a second vertical synchronization signal, and there is no first image frame to be refreshed and displayed in a third vertical synchronization signal. At the first moment, perform layer composition on the first layer in the first cache queue to obtain a second image frame, and cache the second image frame in a second cache queue; Wherein, the second vertical synchronization signal is used to trigger layer composition, the second vertical synchronization signal is after the first vertical synchronization signal, and is separated from the first vertical synchronization signal by a first preset period; The third vertical synchronization signal is used to trigger image frame refresh and display, the third vertical synchronization signal is after the second vertical synchronization signal, and is separated from the second vertical synchronization signal by a second preset period.

2. The method according to claim 1, wherein The method further includes: The first moment is before the third vertical synchronization signal. In response to the third vertical synchronization signal, refresh and display the second image frame in the second cache queue.

3. The method according to claim 2, wherein The method further includes: The first moment is after the third vertical synchronization signal. Refresh and display the second image frame in the second cache queue at a screen refresh moment; wherein, the screen refresh moment is after the third vertical synchronization signal and before a fourth vertical synchronization signal; wherein, the fourth vertical synchronization signal is used to trigger image frame display, and the fourth vertical synchronization signal is after the third vertical synchronization signal and is separated from the third vertical synchronization signal by a synchronization period.

4. The method according to claim 3, wherein Before refreshing and displaying the second image frame in the second cache queue at the screen refresh moment, the method further includes: Determine the screen refresh moment according to a predetermined frame rate; wherein, the predetermined frame rate is greater than the frame rate of the display screen of the electronic device.

5. The method according to claim 1 or 2, characterized in that, Cache the second image frame in the second cache queue at a second moment; the method further includes: The second moment is after the third vertical synchronization signal. Refresh and display the second image frame in the second cache queue at a screen refresh moment; wherein, the screen refresh moment is after the third vertical synchronization signal and before a fourth vertical synchronization signal; the screen refresh moment is determined according to a predetermined frame rate.

6. The method according to any one of claims 2-5, characterized in that The screen refresh moment includes a first refresh moment and at least one second refresh moment; the second refresh moment is after the first refresh moment; Refreshing and displaying the second image frame in the second cache queue at the screen refresh moment includes: At the first screen refresh moment, refresh and display the second image frame in the second cache queue; If the refresh and display fails at the first screen refresh moment, traverse the at least one second screen refresh moment in chronological order, and refresh and display the second image frame at each second screen refresh moment until the refresh and display is successful or the traversal is completed.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: No layer composition is performed in response to the second vertical synchronization signal; and, at the first moment, when there is no first image frame to be refreshed and displayed in the second buffer queue, it is determined that there is no first image frame to be refreshed and displayed in the third vertical synchronization signal.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: Before the second vertical synchronization signal at the first moment, in response to the second vertical synchronization signal, layer composition is performed on the first layer in the first buffer queue to obtain a second image frame, and the second image frame is cached in the second buffer queue; In response to the third vertical synchronization signal, the second image frame in the second buffer queue is refreshed and displayed.

9. The method according to any one of claims 1-8, characterized in that, The method further includes: Before the second vertical synchronization signal at the first moment, and when the time difference between the first moment and the second vertical synchronization signal is less than or equal to a preset time threshold, it is determined that the first moment is after the second vertical synchronization signal.

10. The method according to any one of claims 1-9, characterized in that, The first preset period is equal to one synchronization period, and the second preset period is equal to one synchronization period.

11. An electronic device, characterized in that, It includes: A display screen, one or more processors and a memory, where the display screen and the memory are respectively coupled to the processor; One or more computer program codes are stored in the memory, and the computer program codes include computer instructions; when the processor executes the computer instructions, the electronic device is caused to execute the image processing method based on vertical synchronization signals according to any one of claims 1-10.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor of the electronic device, the electronic device is caused to execute the image processing method based on vertical synchronization signals according to any one of claims 1-10.

Images