Image processing method based on vertical synchronization signal, electronic device and storage medium

Abstract

This application relates to the field of image processing and display technology, and provides an image processing method, electronic device, and storage medium based on a vertical synchronization signal. The method includes: the electronic device responding to a first vertical synchronization signal to render and cache a first layer in a first buffer queue at a first moment. At the first moment, after a second vertical synchronization signal and when there is no first image frame to be refreshed and displayed in the third vertical synchronization signal, the electronic device immediately triggers layer compositing of the first layer to obtain a second image frame. The second vertical synchronization signal follows the first vertical synchronization signal and is spaced at a first preset period, while the third vertical synchronization signal follows the second vertical synchronization signal and is spaced at a second preset period. Therefore, by triggering immediate compositing when there is a delay in rendering the first layer and the corresponding third vertical synchronization signal can refresh and display the second image frame, frame dropping can be avoided, ensuring smooth image display.

Description

Image processing method, electronic device and storage medium based on vertical synchronization signal

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on January 2, 2024, with application number 202410009794.1 and invention name “Image processing method, electronic device and storage medium based on vertical synchronization signal”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of image processing and display technology, and in particular to an image processing method, electronic device, and storage medium based on a vertical synchronization signal. Background Art

[0003] With the advancement of electronic technology, the performance of various electronic devices (such as mobile phones) is improving. Consumers are also increasingly demanding the human-computer interaction performance of electronic products. Among them, the visual consistency of the display content of electronic devices is a key human-computer interaction performance.

[0004] Currently, ensuring that electronic devices display images without frame drops is one of the prerequisites for ensuring the consistency of the displayed images. However, existing application threads may cause frame drops due to excessive rendering time, affecting the smoothness of the displayed image and poor user visual experience. Summary of the Invention

[0005] The embodiments of the present application provide an image processing method, an electronic device, and a storage medium based on a vertical synchronization signal, which are used to solve the problem of frame loss in the display of an electronic device and ensure the smoothness of image display.

[0006] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, a method for image processing based on a vertical synchronization signal is provided, which is applied to electronic devices. The vertical synchronization signal includes a first vertical synchronization signal for triggering drawing, a second vertical synchronization signal for triggering layer synthesis, and a third vertical synchronization signal for user-triggered refresh of the display. The second vertical synchronization signal follows the first vertical synchronization signal and is separated from the first vertical synchronization signal by a first preset period; the third vertical synchronization signal follows the second vertical synchronization signal and is separated from the second vertical synchronization signal by a second preset period; that is, the period between the third vertical synchronization signal and the first vertical synchronization signal after the first vertical synchronization signal is the sum of the first preset period and the second preset period. The method includes:

[0008] When the first vertical synchronization signal arrives, the electronic device draws and renders the first layer of the first application in response to the first vertical synchronization signal, and simultaneously caches the drawn and rendered first layer in the first cache queue at the first moment. Then, if the first moment of caching the first layer is after the second vertical synchronization signal, and there is no first image frame to be refreshed and displayed when the third vertical synchronization signal arrives, the electronic device immediately triggers layer synthesis of the first layer in the first cache queue at the first moment to obtain a second image frame, and caches the second image frame in the second cache queue.

[0009] It can be seen that, on the one hand, when the electronic device times out in drawing and rendering the first layer in response to the first vertical synchronization signal, and there is no first image frame to be refreshed and displayed corresponding to the third vertical synchronization signal, that is, the first moment is after the second vertical synchronization signal, causing the first layer to miss the second vertical synchronization signal for layer synthesis but the corresponding third vertical synchronization signal has not been missed, the electronic device of the present application immediately triggers layer synthesis of the first image, thereby avoiding frame loss in the case of timed drawing and rendering, and ensuring the smoothness of image display.

[0010] In one possible implementation of the first aspect, due to a timeout in rendering the first layer, the third vertical sync signal may be missed. In this case, even if the first layer is immediately composited, the corresponding second image frame cannot be refreshed and displayed at this third vertical sync signal, resulting in frame loss. Therefore, to minimize the impact of frame loss, in this case, the second image frame can be refreshed and displayed in advance.

[0011] That is, the image processing method based on a vertical synchronization signal may further include: refreshing and displaying the second image frame in the second cache queue at a screen refresh moment after a third vertical synchronization signal at a first moment; wherein the screen refresh moment is after the third vertical synchronization signal and before a fourth vertical synchronization signal; wherein the fourth vertical synchronization signal is used to trigger image frame display, and the fourth vertical synchronization signal is after the third vertical synchronization signal and is separated from the third vertical synchronization signal by one synchronization cycle. Thus, after missing the third vertical synchronization signal, the second image frame is refreshed as early as possible before the fourth vertical synchronization signal, and the display of the second image frame is delayed, thereby minimizing the impact of frame loss.

[0012] In another possible implementation of the first aspect, that is, when the first moment is before the third vertical synchronization signal, the electronic device can generally refresh and display the second image frame normally when the third vertical synchronization signal arrives. Therefore, the image processing method based on the vertical synchronization signal may further include: when the first moment is before the third vertical synchronization signal, the electronic device refreshes and displays the second image frame in the second cache queue in response to the third vertical synchronization signal.

[0013] In one possible implementation of the first aspect, the image refresh time can be determined based on a predetermined frame rate. The predetermined frame rate is greater than the frame rate of a display screen of the electronic device. Furthermore, the number of image refresh times determined based on the predetermined frame rate is greater and more frequent than the number of image refresh times determined based on the frame rate of the display screen, thereby ensuring that the second image frame can be refreshed in advance.

[0014] In one possible implementation of the first aspect, the second image frame is cached in the second cache queue at the second moment. Generally speaking, in addition to the first layer rendering timeout causing the inability to refresh the display on the third vertical synchronization signal, a layer composition timeout may also cause the inability to refresh the display on the third vertical synchronization signal. That is, if the second moment occurs after the third vertical synchronization signal, the display may also be unable to refresh on the third vertical synchronization signal, resulting in frame loss.

[0015] Therefore, to avoid frame dropout in this situation, the image processing method based on vertical synchronization signals may further include: refreshing and displaying the second image frame in the second buffer queue at a screen refresh time after the third vertical synchronization signal at the second moment. The screen refresh time in this aspect is also after the third vertical synchronization signal and before the fourth vertical synchronization signal. Furthermore, it can also be determined based on a predetermined frame rate that is greater than the frame rate of the display screen, which will not be further described here.

[0016] In a possible implementation of the first aspect, because there are usually many screen refresh moments, in order to ensure that the second image frame can be successfully refreshed and displayed in advance, the second image frame can be refreshed and displayed in advance by traversing the screen refresh moments. Therefore, the screen refresh moments can include a first refresh moment and at least one second refresh moment; the second refresh moment is after the first refresh moment. Based on this, refreshing and displaying the second image frame in the second cache queue at the screen refresh moment may include: refreshing and displaying the second image frame in the second cache queue at the first screen refresh moment; if the refresh display fails at the first screen refresh moment, traversing at least one second screen refresh moment in chronological order, refreshing and displaying the second image frame at each second screen refresh moment until the refresh display is successful or the traversal is completed.

[0017] In a possible implementation of the first aspect, whether the third vertical synchronization signal indicates a first image frame to be refreshed and displayed can be determined in two situations: one is that a first image frame is being synthesized in response to the second vertical synchronization signal; the other is that a first image frame to be refreshed and displayed exists in the second cache queue.

[0018] Both of the above situations will cause the first image frame to be refreshed and displayed before the second image frame by the third vertical synchronization signal, causing the second image frame to be delayed until the fourth vertical synchronization signal is refreshed and displayed, resulting in frame loss.

[0019] Therefore, the electronic device does not perform layer synthesis in response to the second vertical synchronization signal, and at the first moment, there is no first image frame to be refreshed and displayed in the second cache queue. The electronic device can determine that there is no first image frame to be refreshed and displayed in the third vertical synchronization signal.

[0020] In another possible implementation of the first aspect, if the first moment occurs before the second vertical synchronization signal, it indicates that the third vertical synchronization signal will not be missed. Therefore, the electronic device can routinely wait for the second vertical synchronization signal to arrive, and then, in response to the second vertical synchronization signal, perform layer synthesis on the first layer in the first cache queue to obtain the second image frame, and then cache the second image frame in the second cache queue. Similarly, the electronic device can routinely wait for the third vertical synchronization signal to arrive, and then, in response to the third vertical synchronization signal, refresh and display the second image frame in the second cache queue.

[0021] In one possible implementation of the first aspect, because layer compositing requires a certain amount of processing time, even if the first moment coincides with the second vertical synchronization signal, the third vertical synchronization signal may be missed due to the time required for layer compositing. Furthermore, to minimize this situation, in this aspect, even if the first moment precedes the second vertical synchronization signal, as long as the time difference between the first moment and the second vertical synchronization signal is less than or equal to a preset time threshold, the first moment can be considered to be after the second vertical synchronization signal.

[0022] In a second aspect, the present application provides an electronic device comprising: one or more processors and a memory, the memory being coupled to the processor; one or more computer program codes stored in the memory, the computer program codes comprising computer instructions; when the processor executes the computer instructions, the electronic device performs the following steps:

[0023] In response to a first vertical synchronization signal, draw and render the first layer of the first application, and cache the first layer in a first cache queue at a first moment; the first moment is after the second vertical synchronization signal, and there is no first image frame to be refreshed and displayed by a third vertical synchronization signal, perform layer synthesis on the first layer in the first cache queue at the first moment to obtain a second image frame, and cache the second image frame in a second cache queue.

[0024] Among them, the second vertical synchronization signal is used to trigger layer synthesis, and the second vertical synchronization signal follows the first vertical synchronization signal and is separated from the first vertical synchronization signal by a first preset period; the third vertical synchronization signal is used to trigger image frame refresh display, and the third vertical synchronization signal follows the second vertical synchronization signal and is separated from the second vertical synchronization signal by a second preset period.

[0025] In a possible implementation of the second aspect, when the above-mentioned computer instructions are executed by the processor, the electronic device further performs the following steps: at a first moment before the third vertical synchronization signal, in response to the third vertical synchronization signal, refreshes and displays the second image frame in the second cache queue.

[0026] In a possible implementation of the second aspect, when the above-mentioned computer instructions are executed by the processor, the electronic device further performs the following steps: at the first moment after the third vertical synchronization signal, the second image frame in the second cache queue is refreshed and displayed at the screen refresh moment; wherein the screen refresh moment is after the third vertical synchronization signal and before the fourth vertical synchronization signal; wherein the fourth vertical synchronization signal is used to trigger the image frame display, and the fourth vertical synchronization signal is after the third vertical synchronization signal and is separated from the third vertical synchronization signal by one synchronization cycle.

[0027] In a possible implementation of the second aspect, when the above-mentioned computer instructions are executed by the processor, the electronic device further performs the following steps: determining the screen refresh time according to a predetermined frame rate; wherein the predetermined frame rate is greater than the frame rate of the display screen of the electronic device.

[0028] In a possible implementation of the second aspect, when the above-mentioned computer instructions are executed by the processor, the electronic device further performs the following steps: at the second moment after the third vertical synchronization signal, the second image frame in the second cache queue is refreshed and displayed at the screen refresh moment; wherein the screen refresh moment is after the third vertical synchronization signal and before the fourth vertical synchronization signal; the screen refresh moment is determined according to a predetermined frame rate.

[0029] In a possible implementation of the second aspect, the screen refresh moment includes a first refresh moment and at least one second refresh moment; the second refresh moment is after the first refresh moment. When the above computer instructions are executed by the processor, the electronic device further performs the following steps: at the first screen refresh moment, refreshing and displaying the second image frame in the second cache queue; if refreshing and displaying fail at the first screen refresh moment, traversing at least one second screen refresh moment in chronological order, refreshing and displaying the second image frame at each second screen refresh moment until refreshing and displaying succeed or traversal is complete.

[0030] In a possible implementation of the second aspect, when the above-mentioned computer instructions are executed by the processor, the electronic device further performs the following steps: no layer synthesis is performed in response to the second vertical synchronization signal; and, at the first moment, there is no first image frame to be refreshed and displayed in the second cache queue, and it is determined that there is no first image frame to be refreshed and displayed in response to the third vertical synchronization signal.

[0031] In a possible implementation of the second aspect, when the above-mentioned computer instructions are executed by the processor, the electronic device further performs the following steps: at a first moment before the second vertical synchronization signal, in response to the second vertical synchronization signal, perform layer synthesis on the first layer in the first cache queue to obtain a second image frame; in response to a third vertical synchronization signal, refresh and display the second image frame in the second cache queue.

[0032] In a possible implementation of the second aspect, when the above-mentioned computer instructions are executed by the processor, the electronic device further performs the following steps: the first moment is before the second vertical synchronization signal, and the time difference between the first moment and the second vertical synchronization signal is less than or equal to a preset time threshold, and the first moment is determined to be after the second vertical synchronization signal.

[0033] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor in an electronic device, the electronic device executes an image processing method based on a vertical synchronization signal as in the first aspect and any possible implementation thereof.

[0034] In a fourth aspect, the present application provides a computer program product that, when executed on a computer, causes the computer to execute the image processing method based on a vertical synchronization signal according to the first aspect and any possible implementation thereof. The computer may be the electronic device described above.

[0035] It can be understood that the beneficial effects that can be achieved by the electronic device of any possible implementation of the second aspect, the computer-readable storage medium of the third aspect, and the computer program product of the fourth aspect can be referred to as the beneficial effects in the first aspect and any possible implementation thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] FIG1A is a schematic diagram of a vertical synchronization signal provided by an embodiment of the present application;

[0037] FIG1B is a schematic diagram of another vertical synchronization signal provided by an embodiment of the present application;

[0038] FIG2 is a flowchart of software processing for an electronic device to display an image in response to a touch operation according to an embodiment of the present application;

[0039] FIG3 is a schematic diagram of the principles of layer drawing, rendering, synthesis, and image frame display provided by an embodiment of the present application;

[0040] FIG4 is a schematic diagram showing the production and consumption principles of a layer of an electronic device provided in an embodiment of the present application;

[0041] FIG5 is a schematic diagram of changes in layers in a first cache queue during layer drawing, rendering, synthesis, and image frame refresh and display processes of an electronic device provided by an embodiment of the present application;

[0042] FIG6 is a first schematic diagram of an image frame loss according to an embodiment of the present application;

[0043] FIG7 is a flowchart of a method for image processing based on a vertical synchronization signal according to an embodiment of the present application;

[0044] FIG8 is a second schematic diagram of an image frame loss according to an embodiment of the present application;

[0045] FIG9 is a third schematic diagram of an image frame loss according to an embodiment of the present application;

[0046] FIG10 is a second flowchart of an image processing method based on a vertical synchronization signal provided by an embodiment of the present application;

[0047] FIG11 is a schematic diagram of refreshing a display image frame in advance at a screen refresh time according to an embodiment of the present application;

[0048] FIG12 is a fourth schematic diagram of an image frame loss according to an embodiment of the present application;

[0049] FIG13 is a schematic diagram of another embodiment of the present application for refreshing a display image frame in advance at a screen refresh time;

[0050] FIG14 is a third flowchart of an image processing method based on a vertical synchronization signal provided by an embodiment of the present application;

[0051] FIG15 is a schematic diagram of changes in image frames in a second cache queue during layer drawing, rendering, synthesis, and image frame refresh and display processes of an electronic device provided by an embodiment of the present application;

[0052] FIG16 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0053] FIG17 is a schematic structural diagram of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly and completely describe the technical solutions of the embodiments of the present application. Among them, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to limit the present application.

[0055] In addition, to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, if the words "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that the words "first" and "second" do not limit the quantity and execution order, and the words "first" and "second" do not necessarily mean different. Also, in the description of the embodiments of the present application, unless otherwise specified, the meaning of "plurality" means two or more.

[0056] The present invention provides an image processing method based on vertical synchronization signals, which can be applied to electronic devices including display screens (such as touch screens). This method can reduce the possibility of frame dropout when displaying images on the electronic device, ensure smooth display of images on the display screen, and thus enhance the user's visual experience.

[0057] Furthermore, even if frame loss is unavoidable, the electronic device can reduce the impact of frame loss on the screen display through this method, ensuring the smoothness of the screen display as much as possible to enhance the user's visual experience.

[0058] Exemplarily, the electronic device may be at least one of a mobile phone, a foldable electronic device, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, an in-vehicle device, a smart home device, a smart city device, and the like that includes a display screen (such as a touch screen).

[0059] It can be understood that the embodiments of the present application do not impose any special restrictions on the specific type of the electronic device.

[0060] To facilitate understanding of the solution, the following is a brief introduction to the vertical synchronization signal in this technical field. In the embodiment of the present application, the vertical synchronization signal includes vertical synchronization signal 1 (corresponding to the first vertical synchronization signal), vertical synchronization signal 2 (corresponding to the second vertical synchronization signal), and vertical synchronization signal 3 (corresponding to the third vertical synchronization signal).

[0061] Vertical synchronization signal 1: such as the VSYNC_APP signal. The vertical synchronization signal 1 can be used to trigger the drawing of one or more layers and render the drawn layers. That is, the above-mentioned vertical synchronization signal 1 can be used to trigger the UI thread to draw one or more layers, and the Render thread renders the one or more layers drawn by the UI thread. In an embodiment of the present application, the vertical synchronization signal 1 (such as the VSYNC_APP signal) is the first vertical synchronization signal.

[0062] Vertical synchronization signal 2: such as the VSYNC_SF signal. This vertical synchronization signal 2 can be used to trigger layer compositing of one or more rendered layers to obtain an image frame. In other words, the vertical synchronization signal 2 can be used to trigger the compositing thread to perform layer compositing on one or more layers rendered by the render thread to obtain an image frame. In this embodiment of the present application, this vertical synchronization signal 2 (such as the VSYNC_SF signal) is the second vertical synchronization signal.

[0063] Vertical synchronization signal 3: such as the HW_VSYNC signal (also known as the VSYNC_TE signal). This vertical synchronization signal 3 can be used to trigger the hardware to refresh the displayed image frame. In the embodiment of the present application, this vertical synchronization signal 3 (such as the HW_VSYNC signal or the VSYNC_TE signal) is the third vertical synchronization signal.

[0064] The vertical synchronization signal 3 is a hardware signal triggered by the display screen driver of the electronic device. In the embodiment of the present application, the signal period T3 of the vertical synchronization signal 3 is determined according to the screen refresh rate of the display screen of the electronic device.

[0065] Specifically, the signal period T3 of the vertical synchronization signal 3 is the reciprocal of the screen refresh rate of the display screen of the electronic device (eg, LCD or OLED), wherein the screen refresh rate of the electronic device may be the same as the frame rate of the display screen of the electronic device.

[0066] Therefore, the high frame rate of an electronic device can be understood as a high screen refresh rate. For example, the screen refresh rate and the frame rate of the display screen of the electronic device can be any value among 60 Hz, 70 Hz, 75 Hz, 80 Hz, 90 Hz or 120 Hz.

[0067] For example, if the screen refresh rate and the frame rate of the display screen of the electronic device are 60 Hz, then the signal period of the vertical synchronization signal 3 is 1 / 60 = 0.01667 seconds (s) = 16.667 milliseconds (ms). If the screen refresh rate and the frame rate of the display screen are 90 Hz, then the signal period of the vertical synchronization signal 3 is 1 / 90 = 0.0111 seconds = 11.11 milliseconds. If the screen refresh rate and the frame rate of the display screen are 120 Hz, then the signal period of the vertical synchronization signal 3 is 1 / 120 = 0.008333 seconds = 8.33 milliseconds.

[0068] It should be noted that in some embodiments, the electronic device may support multiple different frame rates. The frame rate of the electronic device can be switched between the above different frame rates. For example, the electronic device can switch the frame rate from 60Hz to 120Hz to increase the frame rate. The electronic device can also switch the frame rate from 120Hz to 60Hz to reduce the frame rate.

[0069] It is understandable that the frame rate in the embodiment of the present application is the frame rate currently used by the electronic device, that is, the signal period of the vertical synchronization signal 3 is the inverse of the frame rate currently used by the electronic device.

[0070] In addition, the vertical synchronization signal 3 in the embodiment of the present application is a periodic discrete signal. As shown in Figures 1A and 1B, a schematic diagram of a vertical synchronization signal is shown. Below, vertical synchronization signal 1, vertical synchronization signal 2, and vertical synchronization signal 3 are described in conjunction with Figures 1A and 1B.

[0071] Referring to Figures 1A and 1B, a vertical synchronization signal 3 triggered by a hardware driver is generated every other signal cycle. Vertical synchronization signal 1 and vertical synchronization signal 2 are generated based on vertical synchronization signal 3, that is, vertical synchronization signal 3 can be the signal source of vertical synchronization signal 1 and vertical synchronization signal 2. Alternatively, vertical synchronization signal 1 and vertical synchronization signal 2 are synchronized with vertical synchronization signal 3. Therefore, generally speaking, the signal cycle of vertical synchronization signal 1 and vertical synchronization signal 2 is the same as the signal cycle of vertical synchronization signal 3. However, it should be noted that the phases of vertical synchronization signal 1, vertical synchronization signal 2, and vertical synchronization signal 3 may be consistent or inconsistent.

[0072] In the case of phase consistency, for example, as shown in FIG1A , the signal period T1 of the vertical synchronization signal 1, the signal period T2 of the vertical synchronization signal 2, and the signal period T3 of the vertical synchronization signal 3 are the same. Furthermore, as shown in FIG1A , the phases of the vertical synchronization signal 1, the vertical synchronization signal 2, and the vertical synchronization signal 3 are consistent. In this case, the periods between the vertical synchronization signals of each thread are the same, and are all separated by one synchronization period. It can be understood that the vertical synchronization signal 1 is separated from the corresponding vertical synchronization signal 2 by one synchronization period, and the vertical synchronization signal 2 is separated from the corresponding vertical synchronization signal 3 by one synchronization period. As shown in FIG1A , the vertical synchronization signal 1 at time t1 and the vertical synchronization signal 2 at time t2 are separated by one synchronization period TZ. The vertical synchronization signal 2 at time t2 and the vertical synchronization signal 3 at time t3 are separated by one synchronization period TZ.

[0073] In the case of phase inconsistency, for example, as shown in FIG1B , the signal period T1 of vertical synchronization signal 1, the signal period T2 of vertical synchronization signal 2, and the signal period T3 of vertical synchronization signal 3 are the same. However, as shown in FIG1B , the phases of vertical synchronization signal 1, vertical synchronization signal 2, and vertical synchronization signal 3 are inconsistent. In this case, the periods between the vertical synchronization signals are not necessarily the same. That is, the periods between the corresponding vertical synchronization signals 1, vertical synchronization signal 2, and vertical synchronization signal 3 are not necessarily one synchronization period. As shown in FIG1B , the first preset period separates the vertical synchronization signal 1 at time t1 and the vertical synchronization signal 2 at time t2. The second preset period separates the vertical synchronization signal 2 at time t2 and the vertical synchronization signal 3 at time t3.

[0074] The first preset period and the second preset period are configured according to actual needs. The first preset period and the second preset period can be greater than or less than the synchronization period. Furthermore, in the embodiment of the present application, the first preset period can be equal to the second preset period, or the first preset period can be different from the second preset period. It is understood that the first preset period is the time reserved for drawing and rendering, and the second preset period is the time reserved for layer synthesis.

[0075] It is understood that when the phases are consistent, the first preset period = the second preset period = the synchronization period. That is, the time reserved for rendering and synthesis is the same. When the phases are inconsistent, the time reserved for rendering and synthesis is different, and the reserved rendering time and synthesis time may be greater than or less than one synchronization period.

[0076] In the following, to facilitate understanding of the solution, the embodiment of the present application mainly takes the phase consistency situation (first preset period = second preset period = synchronization period) as an example to illustrate the image processing method based on the vertical synchronization signal provided by the embodiment of the present application.

[0077] It is understandable that, in actual implementation, due to various factors (such as processing performance), there may be certain phase errors between vertical synchronization signal 1, vertical synchronization signal 2, and vertical synchronization signal 3. It should be noted that when understanding the method of the embodiment of the present application, the above phase errors are ignored.

[0078] In summary, the vertical synchronization signal 1, vertical synchronization signal 2, and vertical synchronization signal 3 are all periodic discrete signals. For example, as shown in Figures 1A and 1B, there is a vertical synchronization signal 1 every signal period T1, a vertical synchronization signal 2 every signal period T2, and a vertical synchronization signal 3 every signal period T3. The signal periods of the vertical synchronization signal 1, vertical synchronization signal 2, and vertical synchronization signal 3 can all be referred to as synchronization periods TZ, that is, T1 = T2 = T3 = TZ. In other words, the synchronization period TZ in the embodiment of the present application is the inverse of the screen refresh rate of the electronic device.

[0079] It should be noted that the name of the vertical synchronization signal may be different in different systems or architectures. For example, in some systems or architectures, the name of the vertical synchronization signal (i.e., vertical synchronization signal 1) used to trigger the drawing of one or more layers may not be VSYNC_APP. However, no matter what the name of the vertical synchronization signal is, as long as it is a synchronization signal with similar functions and conforms to the technical ideas of the method provided in the embodiment of this application, it should be covered within the scope of protection of this application.

[0080] Moreover, in different systems or architectures, the definition of the above-mentioned vertical synchronization signal may also be different. For example, in other systems or architectures, the definition of the above-mentioned vertical synchronization signal 1 may be: vertical synchronization signal 1 can be used to trigger the rendering of one or more layers; the definition of vertical synchronization signal 2 may be: vertical synchronization signal 2 can be used to trigger the generation of image frames based on one or more layers; the definition of vertical synchronization signal 3 may be: vertical synchronization signal 3 can be used to trigger the display of image frames. In the embodiment of the present application, the definition of the vertical synchronization signal is not limited. However, no matter what definition is made of the vertical synchronization signal, as long as it is a synchronization signal with similar functions and conforms to the technical ideas of the method provided in the embodiment of the present application, it should be covered within the scope of protection of the present application.

[0081] For ease of understanding, the embodiment of the present application is combined with Figure 2 here, taking the above-mentioned display screen as a touch screen and the user's operation on the display screen as a touch operation as an example, to introduce the software processing flow of the electronic device from "the user's finger inputs a touch operation on the touch screen" to "the touch screen displays the image corresponding to the touch operation".

[0082] As shown in FIG2 , the electronic device may include: a touch panel (TP) / TP driver (Driver) 210 , an Input Framework (i.e., Input Framework) 220 , a UI Framework (i.e., UI Framework) 230 , a Display Framework (i.e., Display Framework) 240 and a hardware display module 250 .

[0083] As shown in FIG2 , the software processing flow of the electronic device may include the following steps (1) to (5).

[0084] Step (1): After the TP in the TP IC / TP driver 210 collects the touch operation of the user's finger on the TP of the electronic device, the TP driver reports the corresponding touch event to the Event Hub.

[0085] Step (2): The Input Reader thread of the Input framework 220 can read the touch event from the Event Hub and then send the touch event to the Input Dispatcher thread; the Input Dispatcher thread uploads the touch event to the UI thread in the UI framework 230.

[0086] Step (3): The UI thread (e.g., Do Frame) in the UI framework 230 draws one or more layers corresponding to the touch event; the rendering thread (e.g., Draw Frame) renders the one or more layers. The UI thread is a thread in the electronic device's central processing unit (CPU). The rendering thread is a thread in the electronic device's GPU.

[0087] Step (4): The synthesis thread (Surface Flinger) in the Display framework 240 performs layer synthesis on the drawn one or more layers (ie, the rendered one or more layers) to obtain an image frame.

[0088] Step (5): The display screen driver of the hardware display module 250, as shown in FIG2 , receives the synthesized image frame and then displays the synthesized image frame on the LCD. After the LCD displays the image frame, the image displayed on the LCD can be perceived by the human eye.

[0089] Generally speaking, in response to the user's touch operation or UI event on the TP, the UI framework can call the UI thread to draw one or more layers corresponding to the touch event after the vertical synchronization signal 1 arrives, and then call the Render thread to render the one or more layers. Then, the hardware synthesis (Hardware Composer, HWC) can call the synthesis thread to perform layer synthesis on the drawn one or more layers (that is, one or more layers after rendering) to obtain an image frame after the vertical synchronization signal 2 arrives. Finally, the hardware display module can refresh and display the above image frame on the LCD after the vertical synchronization signal 3 arrives. The above UI event can be triggered by the user's touch operation on the TP. Alternatively, the UI event can be automatically triggered by the electronic device. For example, when the foreground application of the electronic device automatically switches the screen, the above UI event can be triggered. The foreground application is the application corresponding to the interface currently displayed on the display screen of the electronic device.

[0090] The TP can periodically detect user touch operations. After detecting a touch operation, the TP can wake up the aforementioned vertical synchronization signal 1 and vertical synchronization signal 2 to trigger the UI framework to draw and render layers based on vertical synchronization signal 1, and the hardware synthesis HWC to perform layer synthesis based on vertical synchronization signal 2. The detection period of the TP for detecting touch operations is the same as the signal period T3 of vertical synchronization signal 3 (such as HW_VSYNC).

[0091] It should be noted that the UI framework periodically draws and renders layers based on vertical synchronization signal 1; the hardware synthesis HWC periodically synthesizes layers based on vertical synchronization signal 2; and the LCD periodically refreshes image frames based on vertical synchronization signal 3.

[0092] As shown in Figure 3, an embodiment of the present application illustrates the process of an electronic device executing drawing, rendering, synthesis, and refreshing a display image frame. Below, taking the above-mentioned vertical synchronization signal 1 being the VSYNC_APP signal, vertical synchronization signal 2 being the VSYNC_SF signal, and vertical synchronization signal 3 being the VSYNC_TE (i.e., HW_VSYNC) signal as an example, the process of an electronic device executing drawing, rendering, synthesis, and image frame display is briefly described in conjunction with Figure 3.

[0093] Referring to Figure 3, the UI thread of the electronic device responds to the VSYNC_APP signal at time t1, executes "Draw 1" to draw layer 1, and then the Render thread executes "Render 1" to render layer 1; the synthesis thread of the electronic device responds to the VSYNC_SF signal at time t2, executes "Image frame synthesis 1" to perform layer synthesis on the above layer 1 to obtain image frame 1; the LCD of the electronic device responds to the HW_VSYNC signal at time t3, executes "Image frame display 1" to refresh and display the above image frame 1.

[0094] For another example, as shown in FIG3 , the UI thread of the electronic device responds to the VSYNC_APP signal at time t2, executes "Draw 2" to draw layer 2, and then the Render thread executes "Render b" to render layer 2; the synthesis thread of the electronic device responds to the HW_VSYNC signal at time t3, executes "Image frame synthesis 2" to perform layer synthesis on the above-mentioned layer 2 to obtain image frame 2; the LCD of the electronic device responds to the HW_VSYNC signal at time t4, executes "Image frame display 2" to refresh and display the above-mentioned image frame 2.

[0095] Thus, the layer drawn by the electronic device in response to the VSYNC_APP signal at time t1 is synthesized with the VSYNC_SF signal at time t2, which is one synchronization cycle (first preset cycle) apart, to obtain an image frame. The image frame synthesized in response to the HW_VSYNC signal at time t2 is refreshed and displayed in response to the HW_VSYNC signal at time t3, which is one synchronization cycle (second preset cycle) apart. Furthermore, the layer drawn by the electronic device in response to the VSYNC_APP signal at time t2 is synthesized with the VSYNC_SF signal at time t3, which is one synchronization cycle (first preset cycle) apart, to obtain an image frame. The image frame synthesized in response to the VSYNC_SF signal at time t3 is refreshed and displayed in response to the HW_VSYNC signal at time t4, which is one synchronization cycle (second preset cycle) apart.

[0096] It can be understood that the VSYNC_APP signal at time t1 (e.g., the first vertical synchronization signal), the VSYNC_SF signal at time t2 (e.g., the second vertical synchronization signal), and the HW_VSYNC signal at time t3 (e.g., the third vertical synchronization signal) are corresponding vertical synchronization signals. For another example, the VSYNC_APP signal at time t2 (e.g., the first vertical synchronization signal), the VSYNC_SF signal at time t3 (e.g., the second vertical synchronization signal), and the HW_VSYNC signal at time t4 (e.g., the third vertical synchronization signal) are also corresponding vertical synchronization signals.

[0097] It should be noted that "Drawing 1" shown in FIG3 can be implemented in the CPU of the electronic device, and "Rendering 1" can be implemented in the GPU of the electronic device. "Drawing 2" shown in FIG3 can be implemented in the CPU of the electronic device, and "Rendering 2" can be implemented in the GPU of the electronic device.

[0098] In some embodiments, the electronic device's CPU can also perform rendering by a Render thread. It's understood that the rendering performed by the Render thread in the CPU is the GPU's preparation for rendering the drawn layers 1 and 2, while the GPU's execution of "Render 1" and "Render 2" is the electronic device's formal layer rendering of the drawn layers 1 and 2. In other words, the rendering in the embodiments of the present application can include: layer drawing performed by the UI thread and the Render thread's preparation for layer rendering of the layers drawn by the UI thread.

[0099] The process of drawing, rendering, and synthesizing layers by the electronic device in FIG3 can constitute a graphics generation consumption model, such as the graphics generation consumption model 400 shown in FIG4 . In the graphics generation consumption model 400, the UI thread and the Render thread (i.e., the renderer Render) of the electronic device act as producers to draw and render layers; the Render thread (i.e., the renderer Render) can save the layers that have completed rendering preparation in the first cache queue and perform layer rendering on the layers in the first cache queue; the synthesis thread (i.e., the synthesizer Surface Flinger) acts as a consumer to read layers from the first cache queue, perform layer synthesis on the read layers to obtain image frames, and send the image frames to the LCD (i.e., the display controller Display Controller) of the electronic device for display.

[0100] In the above-mentioned graphics generation and consumption model, both producers (such as UI threads and Render threads) and consumers (such as synthesis threads) generate and consume layers based on the VSYNC signal.

[0101] In the aforementioned graphics generation and consumption model, the production and consumption rates remain consistent, provided there are no lags (i.e., no frame drops). The producer (e.g., the Render thread) generates a frame (also known as frame data) every VSYNC cycle (e.g., the synchronization period TZ) and places it into the first cache queue. The consumer (e.g., the Composition thread) retrieves a frame from the first cache queue every VSYNC cycle (e.g., the synchronization period TZ) for layer synthesis (also known as image frame synthesis). This means that the production cycle of the UI thread and Render thread as producers is the same as the consumption cycle of the Composition thread (i.e., Surface Flinger) as a consumer, both equal to the synchronization period TZ.

[0102] For example, as shown in Figure 5, at time tx in Figure 5, the Render thread of the electronic device completes "Rendering 1". At this time, the Render thread can cache the rendered layer 1 to the first cache queue, that is, the producer produces a frame layer and caches the layer to the first cache queue. That is, at time tx, the number of layers in the first cache queue increases from 0 to 1 (i.e., 0->1). Subsequently, the electronic device responds to the VSYNC_SF signal at time t2 shown in Figure 5, and the synthesis thread of the electronic device can execute "image frame synthesis 1" (also called layer synthesis 1). At this time, the synthesis thread can read layer 1 from the first cache queue, that is, the consumer consumes a frame layer from the first cache queue. That is, at time t2, the number of layers in the first cache queue decreases from 1 to 0 (i.e., 1->0).

[0103] For example, at time ty shown in Figure 5, the electronic device's Render thread completes "Render 2." At this point, the Render thread caches the rendered layer 2 in the first cache queue. This means the producer has produced a frame of layers and cached it in the first cache queue. That is, at time ty, the number of layers in the first cache queue increases from 0 to 1 (i.e., 0->1).

[0104] Subsequently, in response to the VSYNC_SF signal at time t3 shown in Figure 5 , the electronic device's composition thread may execute "Image Frame Composition 2" (also known as Layer Composition 2). At this point, the composition thread may read Layer 2 from the first cache queue, meaning that the consumer consumes a frame of layers from the first cache queue. In other words, at time t3, the number of layers in the first cache queue decreases from 1 to 0 (i.e., 1->0).

[0105] However, when electronic devices draw, render, synthesize, and refresh display image frames in response to the aforementioned VSYNC_APP, VSYNC_SF, and HW_VSYNC signals, frame drops may occur due to the application thread taking a long time. Specifically, a blank frame may appear while the display is refreshing the image frame. This can affect the consistency and smoothness of the displayed image, thereby affecting the user's visual experience.

[0106] Specifically, the reason why the electronic device displays images with frame drops may be that the UI thread and Render thread take too long to draw and render, and cannot be completed within one VSYNC cycle (such as the synchronization cycle TZ mentioned above).

[0107] In this way, the producer (such as the Render thread) cannot cache the rendered layer into the first cache queue on time. That is to say, the producer (such as the Render thread) will not cache the frame data in the first cache queue for at least one VSYNC cycle. However, the consumer (such as the synthesis thread) will still take out a frame of layer from the first cache queue for layer synthesis every other VSYNC cycle. However, because the producer did not cache the layer, the consumer cannot read the corresponding layer. Then this VSYNC cycle cannot perform layer synthesis to obtain the image frame, and cannot refresh and display the image frame. The display screen of the display screen cannot be updated, and frame loss will occur. This will affect the continuity and smoothness of the image displayed on the display screen, thereby affecting the user's visual experience.

[0108] For example, as shown in FIG6 , an embodiment of the present application shows a schematic diagram of frame loss. Referring to FIG6 , the Render thread cannot complete "Rendering 2" before the arrival of time t3, and therefore cannot cache layer 2 to the first cache queue before time t3. Furthermore, at time t3, the number of frame data in the first cache queue is 0. Therefore, in response to the VSYNC_SF signal at time t3, the synthesis thread cannot read the layer from the first cache queue, and thus cannot perform layer synthesis to obtain an image frame. Furthermore, at time t4, the display screen of the electronic device cannot refresh the displayed image frame, and thus a frame loss phenomenon occurs.

[0109] At time tt, after time t3, the Render thread completes "Render 2"; at this point, the number of layers in the first cache queue increases from 0 to 1 (i.e., 0 -> 1). In response to the VSYNC_SF signal at time t4, after time tt, the Compositing Thread can read layer 2 from the first cache queue, and the number of layers in the first cache queue decreases from 1 to 0 (i.e., 1 -> 0). The electronic device's display can then execute "Image Frame Display 2" to refresh the displayed image frame at time t5.

[0110] As shown in Figure 6, during the synchronization period between time t4 and time t5, the electronic device's display screen displays frame drops. However, the method of the present embodiment can avoid frame drops in the displayed image, thereby preventing the display screen from displaying a blank frame. In other words, the method of the present embodiment can reduce the likelihood of frame drops when the electronic device displays images, ensuring smooth display of images on the display screen, thereby improving the user's visual experience.

[0111] Furthermore, even if frame loss is unavoidable, the impact of frame loss can be reduced as much as possible through the method of the embodiment of the present application, and the smoothness of the image displayed on the display screen can be guaranteed as much as possible.

[0112] It should be noted that the reason for frame loss in the image displayed by an electronic device may be that the electronic device is unable to complete the drawing and rendering of a frame layer within one frame (such as one synchronization period TZ), or it may be that the electronic device is unable to complete the layer synthesis of a frame layer within one frame (such as one synchronization period TZ).

[0113] In the following embodiments, the image processing method based on the vertical synchronization signal in the embodiment of the present application is mainly introduced by taking the case where the UI thread and the Render thread cannot complete the drawing and rendering of a frame layer within one frame, resulting in frame loss of the image displayed by the electronic device as an example.

[0114] The image processing method based on the vertical synchronization signal provided in the embodiment of the present application mainly determines the specific reason for possible frame loss based on the time sequence relationship between the first moment and the VSYNC_SF signal (i.e., the second vertical synchronization signal) and the HW_VSYNC signal (i.e., the third vertical synchronization signal). Furthermore, the electronic device adjusts the time point of layer synthesis and image frame refresh display based on the specific reason. For example, after missing the VSYNC_SF signal, if the HW_VSYNC signal corresponding to the VSYNC_SF signal that is one synchronization cycle (the second preset cycle) does not have an image frame to be refreshed and displayed (the first image frame), the electronic device can decide to perform layer synthesis immediately. In this way, the image frame can be refreshed and displayed within the synchronization cycle of the corresponding HW_VSYNC signal, thereby avoiding frame loss.

[0115] Alternatively, when the electronic device determines that a frame has been lost, it can decide to refresh the displayed image frame in advance, thereby minimizing the impact of frame loss and ensuring smooth display as much as possible to enhance the user's visual experience.

[0116] The first moment is when the electronic device draws and renders the first layer in response to the VSYNC_APP signal (i.e., the first vertical synchronization signal) and stores the first layer in the first cache queue. It is understood that the first vertical synchronization signal (VSYNC_APP signal), the second vertical synchronization signal (VSYNC_SF signal), and the third vertical synchronization signal (HW_VSYNC signal) in the embodiment of the present application are corresponding vertical synchronization signals.

[0117] For example, the corresponding VSYNC_APP signal, VSYNC_SF signal, and HW_VSYNC signal herein may correspond to the VSYNC_APP signal at time t1, the VSYNC_SF signal at time t2, and the HW_VSYNC signal (VSYNC_TE signal) at time t3 in FIG. 1A , FIG. 1B , FIG. 3 , FIG. 5 , or FIG. 6 Alternatively, the corresponding VSYNC_APP signal, VSYNC_SF signal, and HW_VSYNC signal herein may also correspond to the VSYNC_APP signal at time t2, the VSYNC_SF signal at time t3, and the HW_VSYNC signal (VSYNC_TE signal) at time t4 in FIG. 1A , FIG. 1B , FIG. 3 , FIG. 5 , or FIG.

[0118] As shown in FIG7 , an embodiment of the present application shows a flowchart of an image processing method based on a vertical synchronization signal. The method is applied to an electronic device including a display screen (such as a touch screen) and may include steps S701 - S707 .

[0119] S701: The electronic device draws and renders a first layer of a first application in response to a first vertical synchronization signal (VSYNC_APP signal), and caches the first layer in a first cache queue at a first moment.

[0120] When a first vertical synchronization signal (VSYNC_APP signal) arrives, the electronic device renders the first layer of the first application in response to the first vertical synchronization signal, and caches the rendered first layer in a first cache queue.

[0121] In the embodiment of the present application, the time when the electronic device caches the first layer is recorded as the first time. The first application can be any application that needs to display a screen.

[0122] In some embodiments, the UI thread can respond to the first vertical synchronization signal (VSYNC_APP signal) to draw the first layer. Then, the Render thread prepares to render the first layer drawn by the UI thread and caches the first layer to the first cache queue. It should be noted that after the Render thread caches the first layer to the first cache queue, the Render thread can formally render the first layer cached in the first cache queue. Afterwards, the synthesis thread can perform layer synthesis on the layers cached in the first cache queue (such as the first layer) to obtain an image frame (such as the second image frame).

[0123] S702: The electronic device determines whether the first moment is after the second vertical synchronization signal (VSYNC_SF signal).

[0124] The second vertical synchronization signal here is used to trigger layer compositing and is a vertical synchronization signal corresponding to the first vertical synchronization signal in S701, which is separated by one synchronization period (the first preset period). If the electronic device determines that the first moment is after this second vertical synchronization signal (VSYNC_SF signal), the electronic device proceeds to S703. If the electronic device determines that the first moment is before this second vertical synchronization signal (VSYNC_SF signal), the electronic device proceeds to S706.

[0125] At step S703, the electronic device determines whether a third vertical synchronization signal (HW_VSYNC signal) contains a first image frame to be refreshed and displayed. The third vertical synchronization signal is used to trigger image frame refresh and display, and is a vertical synchronization signal that is one synchronization cycle (the second preset cycle) apart from the second vertical synchronization signal at step S702. This means that the third vertical synchronization signal is two synchronization cycles (the sum of the first preset cycle and the second preset cycle) apart from the first vertical synchronization signal at step S701.

[0126] If the third vertical synchronization signal contains a first image frame to be refreshed and displayed, the electronic device ends the current process. Otherwise, if the third vertical synchronization signal contains no first image frame to be refreshed and displayed, the electronic device proceeds to S704.

[0127] S704: At a first moment, the electronic device performs layer synthesis on the first layer in the first cache queue to obtain a second image frame, and caches the second image frame in a second cache queue.

[0128] At step S705, the electronic device determines whether the first moment is before the third vertical synchronization signal (HW_VSYNC signal). If the first moment is before the third vertical synchronization signal (HW_VSYNC signal), the electronic device proceeds to step S707. If the first moment is after the third vertical synchronization signal (HW_VSYNC signal), the electronic device terminates the current processing flow.

[0129] S706 , the electronic device performs layer synthesis on the first layer in the first cache queue to obtain a second image frame in response to the second vertical synchronization signal, and caches the second image frame in the second cache queue.

[0130] S707 , the electronic device refreshes and displays the second image frame in the second cache queue in response to the third vertical synchronization signal.

[0131] Specifically, if the electronic device determines that the first moment is before the second vertical synchronization signal (VSYNC_SF signal) corresponding to the first vertical synchronization signal (VSYNC_APP signal) that is one synchronization cycle (first preset cycle) apart, in this case, it indicates that the electronic device has successfully completed the drawing and rendering of the layer (such as the first layer) and has stored it in the first cache queue before the second vertical synchronization signal (VSYNC_SF signal) corresponding to the first synchronization cycle (first preset cycle) apart from the first vertical synchronization signal (VSYNC_APP signal) arrives.

[0132] That is, it indicates that the electronic device has successfully sent the layer (first layer) to the synthesis thread before the arrival of the second vertical synchronization signal (VSYNC_SF signal) corresponding to the first vertical synchronization signal (VSYNC_APP signal) with an interval of one synchronization cycle (first preset cycle). Then, the layer (first layer) can be smoothly synthesized when the second vertical synchronization signal (VSYNC_SF signal) arrives. In addition, the electronic device can also smoothly refresh the display of the synthesized image frame when the third vertical synchronization signal (HW_VSYNC signal) corresponding to the second vertical synchronization signal (VSYNC_SF signal) with a interval of one synchronization cycle (second preset cycle) arrives, that is, there will be no frame loss.

[0133] For example, as shown in Figure 6, the electronic device draws and renders layer 1 (the first layer) in response to the first vertical synchronization signal (VSYNC_APP signal) at time t1, and the electronic device stores layer 1 (the first layer) in the first cache queue at time tx (the first time). At this time, time tx (the first time) is before the second vertical synchronization signal (VSYNC_SF signal) at the corresponding time t2, that is, the first time in the embodiment of the present application is before the second vertical synchronization signal (VSYNC_SF signal) corresponding to one synchronization cycle (the first preset cycle) after the first vertical synchronization signal (VSYNC_APP signal).

[0134] In this case, when the second vertical synchronization signal (VSYNC_SF signal) arrives at time t2, the electronic device can respond to this second vertical synchronization signal (VSYNC_SF signal) and successfully perform "Image Frame Synthesis 1" on layer 1 (the first layer) to obtain image frame 1 (the second image frame). Furthermore, when the third vertical synchronization signal (HW_VSYNC signal) arrives at time t3, the electronic device can respond to this third vertical synchronization signal (HW_VSYNC signal) and successfully perform "Image Frame Display 1" to refresh and display image frame 1 (the second image frame).

[0135] Therefore, the electronic device draws the rendered layer 1 (first layer) in response to the first vertical synchronization signal (VSYNC_APP signal) at time t1, and can successfully display it within the synchronization period of the third vertical synchronization signal (HW_VSYNC signal) at time t3 corresponding to an interval of two synchronization periods (the sum of the first preset period and the second preset period), and the electronic device does not experience frame loss.

[0136] Therefore, in the above case, the electronic device enters S706, waits for the arrival of the second vertical synchronization signal (VSYNC_SF signal) corresponding to an interval of one synchronization cycle (first preset cycle) according to the conventional process, and performs layer synthesis on the first layer in response to this second vertical synchronization signal (VSYNC_SF signal) to obtain the second image frame.

[0137] In some embodiments, the composition thread can respond to this second vertical synchronization signal (VSYNC_SF signal) by reading the first layer from the first cache queue and performing layer composition to obtain the second image frame. The electronic device then proceeds to S707 and waits for the corresponding third vertical synchronization signal (HW_VSYNC signal) according to the conventional process. In response to this third vertical synchronization signal (HW_VSYNC signal), the second image frame is refreshed and displayed. In some embodiments, the display driver can read the second image frame from the second cache queue and refresh the display (such as an LCD or OLED) for display.

[0138] If the electronic device determines that the first moment is after the second vertical synchronization signal (VSYNC_SF signal) corresponding to one synchronization cycle (first preset cycle) after the first vertical synchronization signal (VSYNC_APP signal), in this case, it indicates that the electronic device has not successfully completed the drawing and rendering of the layer (such as the first layer) before the second vertical synchronization signal (VSYNC_SF signal) corresponding to one synchronization cycle (first preset cycle) after the first vertical synchronization signal (VSYNC_APP signal) arrives.

[0139] That is, it indicates that the electronic device has not successfully sent the layer (first layer) to the synthesis thread before the second vertical synchronization signal (VSYNC_SF signal) corresponding to the first vertical synchronization signal (VSYNC_APP signal) arrives after one synchronization period (first preset period).

[0140] Then, the synthesis thread cannot respond to the second vertical synchronization signal (VSYNC_SF signal) when the second vertical synchronization signal (VSYNC_SF signal) corresponding to the first vertical synchronization signal (VSYNC_APP signal) arrives one synchronization cycle (first preset cycle) apart. Compositing the layer (first layer) to obtain an image frame (such as the second image frame corresponding to the first layer) in response to this second vertical synchronization signal (VSYNC_SF signal). Furthermore, the electronic device cannot respond to the third vertical synchronization signal (HW_VSYNC signal) when the third vertical synchronization signal (HW_VSYNC signal) corresponding to the first vertical synchronization signal (VSYNC_APP signal) arrives two synchronization cycles (the sum of the first preset cycle and the second preset cycle) apart. This causes frame loss. That is, the second image frame is lost, and the display screen of the electronic device does not display the second image frame within the corresponding synchronization cycle.

[0141] For example, as shown in Figure 6, the electronic device draws and renders layer 2 (the first layer) in response to the first vertical synchronization signal (VSYNC_APP signal) at time t2. However, because the application side takes too long to draw and render layer 2 (the first layer), that is, the drawing and rendering of layer 2 (the first layer) times out, the electronic device does not store layer 2 (the first layer) in the first cache queue until time tt (the first time).

[0142] At this time, the tt moment (the first moment) is after the second vertical synchronization signal (VSYNC_SF signal) at the corresponding t3 moment, that is, the above-mentioned first moment in the embodiment of the present application is after the second vertical synchronization signal (VSYNC_SF signal) corresponding to the first vertical synchronization signal (VSYNC_APP signal) which is one synchronization cycle (first preset cycle).

[0143] In this case, when the second vertical synchronization signal (VSYNC_SF signal) arrives at time t3, because there is no layer 2 (first layer) in the first cache queue, the electronic device cannot read layer 2 (first layer), that is, the synthesis thread cannot read layer 2 (first layer), and thus the electronic device cannot perform layer synthesis on layer 2 (first layer) within the synchronization period TZ of this second vertical synchronization signal (VSYNC_SF signal). Furthermore, when the third vertical synchronization signal (HW_VSYNC signal) arrives at time t4, there is no corresponding image frame 2 (second image frame) that can be refreshed and displayed. In other words, during the synchronization period from time t4 to time t5, the display screen of the electronic device displays an image with frame loss.

[0144] At this time, in order to avoid frame loss, in some embodiments, the electronic device can trigger the synthesis thread to immediately perform layer synthesis on the first layer (layer 2 in Figure 6) at a first moment (such as moment tt in Figure 6).

[0145] However, based on the actual work allocation of compositing threads, the compositing thread used for layer compositing may not only composite the first layer of the first application. In other words, the compositing thread that composites the first layer of the first application may also be assigned to composite layers for other applications. For example, it may also be assigned to composite the layers of a second application.

[0146] Then, if the first layer of the first application misses the second vertical synchronization signal (VSYNC_SF signal) due to drawing timeout and fails to perform layer synthesis in time, the corresponding synthesis thread may synthesize the layers of other applications in response to the second vertical synchronization signal (VSYNC_SF signal).

[0147] That is, at the first moment of caching the first layer, the corresponding synthesis thread may be performing layer synthesis on the layers of other applications in response to this second vertical synchronization signal (VSYNC_SF signal). Furthermore, as long as the image frame (i.e., the first image frame) obtained by the current layer synthesis is successfully stored in the second cache queue, when the third vertical synchronization signal (HW_VSYNC signal) corresponding to the second vertical synchronization signal (VSYNC_SF signal) arrives with a synchronization cycle (the second preset cycle) interval, the electronic device will inevitably read the image frame (first image frame) of other applications from the second cache queue first to refresh the display.

[0148] Therefore, even if the electronic device immediately triggers the layer synthesis of the first layer to obtain the second image frame at the first moment, the second image frame cannot be refreshed and displayed smoothly when the third vertical synchronization signal (HW_VSYNC signal) arrives.

[0149] For example, as shown in Figure 8, the electronic device times out while rendering layer 2 (the first layer), resulting in layer 2 (the first layer) being stored in the first cache queue at time tt (the first moment). However, at this time, the second vertical synchronization signal (VSYNC_SF signal) at time t3 corresponding to time tt (the first moment) is responded to by the electronic device, which is executing "image frame synthesis 3" to obtain image frame 3 (i.e., performing layer synthesis on layers of other applications to obtain the first image frame).

[0150] Furthermore, upon the arrival of the third vertical synchronization signal (HW_VSYNC signal) at time t4, the electronic device will inevitably execute "Image Frame Display 3" in response to this third vertical synchronization signal (HW_VSYNC signal) to refresh and display Image Frame 3 (the first image frame). Therefore, even if the electronic device immediately triggers "Image Frame Synthesis 2" to perform layer synthesis on Layer 2 (the first layer) at the first moment (tt), the resulting Image Frame 2 (the second image frame) will not be refreshed and displayed within the synchronization period TZ of the third vertical synchronization signal (HW_VSYNC signal) at time t4.

[0151] As shown in FIG8 , this image frame 2 (the second image frame) may be delayed until the refresh display is within the synchronization period TZ of the third vertical synchronization signal (HW_VSYNC signal) at time t5. Therefore, even if the electronic device triggers immediate synthesis, the loss of image frame 2 (the second image frame) is inevitable.

[0152] Therefore, if the electronic device determines that the first moment is after the second vertical synchronization signal (VSYNC_SF signal), it indicates that the electronic device can determine that immediate synthesis is needed to avoid frame loss. However, the electronic device needs to further determine whether frame loss can be avoided by triggering immediate synthesis. That is, in this case, the electronic device needs to proceed to S703 to further determine whether the third vertical synchronization signal (HW_VSYNC signal) contains a first image frame to be refreshed and displayed.

[0153] If the electronic device determines that the third vertical synchronization signal (HW_VSYNC signal) contains a first image frame to be refreshed and displayed, for example, as shown in "Image Frame Synthesis 3" in FIG8 , that is, the second vertical synchronization signal (VSYNC_SF signal) is responded to by the synthesis thread as indicating that layer synthesis is in progress, then even if the electronic device triggers immediate synthesis, the second image frame cannot be avoided from being dropped. Therefore, in this embodiment of the application, no additional processing is performed, and the electronic device ends the current processing flow.

[0154] If the electronic device determines that the third vertical synchronization signal (HW_VSYNC signal) does not contain a first image frame to be refreshed and displayed, then the electronic device triggers immediate synthesis to ensure that the second image frame is displayed smoothly within the synchronization period TZ of the third vertical synchronization signal (HW_VSYNC signal). At this time, even if the electronic device times out when rendering the first layer, it will not cause the second image frame to be dropped, thus avoiding the frame drop problem.

[0155] Therefore, in this case, the electronic device proceeds to S704. That is, at the first moment, the electronic device immediately performs layer synthesis on the first layer in the first cache queue to obtain a second image frame, and caches the second image frame in the second cache queue.

[0156] Specifically, at the first moment, the electronic device can immediately send an instruction to the synthesis thread, instructing the synthesis thread to immediately read the first layer from the first cache queue for layer synthesis to obtain a second image frame corresponding to the first layer.

[0157] At the same time, after completing the layer synthesis of the second image frame, the second image frame is cached in the second cache queue. Subsequently, the display driver responds to the third vertical synchronization signal (HW_VSYNC signal) to read the second image from the second cache queue and refresh the display to display the second image frame.

[0158] The electronic device immediately completes layer synthesis of the first layer in the first cache queue at the first moment to obtain the corresponding second image frame, and after caching the second image frame to the second cache queue, according to the conventional process, the electronic device can refresh and display the second image frame in response to the corresponding third vertical synchronization signal (HW_VSYNC signal).

[0159] Therefore, the embodiment of the present application triggers immediate synthesis while ensuring that the corresponding third vertical synchronization signal can refresh and display the second image frame, thereby avoiding frame loss of the second image frame.

[0160] However, in other embodiments, even if the second image frame is synthesized immediately at the first moment and there is no first image frame to be refreshed and displayed when the third vertical synchronization signal is received, there is still a situation in which the second image frame cannot be refreshed and displayed smoothly when the third vertical synchronization signal (HW_VSYNC signal) arrives. That is, when the first moment is after the third vertical synchronization signal (HW_VSYNC signal).

[0161] Because, if the electronic device completes rendering in response to the first vertical synchronization signal (VSYNC_APP) and caches the first layer in the first cache queue, it has already missed the corresponding third vertical synchronization signal (HW_VSYNC signal). Even if the first layer is immediately synthesized at the first moment, no matter how fast the synthesis thread synthesizes the layers, it will still miss the third vertical synchronization signal (HW_VSYNC signal). Therefore, the second image frame still cannot be displayed within the synchronization period TZ of the corresponding third vertical synchronization signal (HW_VSYNC signal), resulting in frame loss.

[0162] For example, as shown in Figure 9, the electronic device times out in drawing and rendering layer 2 (the first layer), and only caches layer 2 (the first layer) in the first cache queue at time tw (the first moment). At this time, time tw is after time t4, so even if the electronic device immediately performs layer synthesis on layer 2 (the first layer) at time tw (the first moment) to obtain image frame 2 (the second image frame). This second image frame cannot be refreshed and displayed smoothly between time t4 and time t5. As shown in Figure 9, this image frame 2 (the second image frame) may still be delayed to refresh and display between time t5 and time t6 (time t6 is the first moment after time t5, not shown in the figure). At this time, image frame 2 (the second image frame) is still a lost frame.

[0163] Therefore, in the embodiment of the present application, after the electronic device immediately synthesizes the second image frame at the first moment and caches it in the second cache queue, it can further determine whether the first moment is before the corresponding third vertical synchronization signal (HW_VSYNC signal). That is, after executing S704, the electronic device proceeds to S705 to determine whether the first moment is before the third vertical synchronization signal (HW_VSYNC signal).

[0164] If the first moment is after the corresponding third vertical synchronization signal (HW_VSYNC signal) (that is, the first moment is not before the corresponding third vertical synchronization signal (HW_VSYNC signal)), then the second image frame will inevitably be lost. Therefore, in the embodiment of the present application, no additional processing is added, and the electronic device ends the current processing flow.

[0165] If the first moment is before the corresponding third vertical synchronization signal (HW_VSYNC signal), it means that the electronic device can respond to the third vertical synchronization signal (HW_VSYNC signal) when the third vertical synchronization signal (HW_VSYNC signal) arrives, read the second image frame from the second cache queue and smoothly refresh and display the second image frame, thereby ensuring that no frame is lost.

[0166] That is, if the first moment is before the corresponding third vertical synchronization signal (HW_VSYNC signal), the electronic device proceeds to S707, waits for the corresponding third vertical synchronization signal (HW_VSYNC signal) to arrive according to the conventional process, and refreshes and displays the synthesized second image frame in response to the third vertical synchronization signal (HW_VSYNC signal). As a result, the second image frame is displayed smoothly without frame loss.

[0167] As shown in FIG10 , an embodiment of the present application shows a flowchart of another image processing method based on a vertical synchronization signal, which may include steps S1001 - S1008 .

[0168] Comparing FIG10 with FIG7 , it can be seen that the embodiment of the present application adds S1008 to the process shown in FIG7 . The electronic device determines the screen refresh time according to the predetermined frame rate, and refreshes and displays the second image frame in the second cache queue at the screen refresh time.

[0169] In an embodiment of the present application, in response to the inevitable frame loss situation in which the first moment is after the third vertical synchronization signal (HW_VSYNC signal), the embodiment of the present application increases the screen refresh moment to display the second image frame as early as possible, thereby reducing the display impact caused by the frame loss and ensuring the smoothness of the screen display as much as possible.

[0170] It should be noted that the specific implementation of S1001-S1007 in the embodiment of the present application can refer to the specific implementation of S701-S707 in the above embodiment. The specific implementation method and principle are the same, and the embodiment of the present application will not be repeated here.

[0171] Hereinafter, the embodiment of the present application mainly describes S1008 in FIG. 10 in detail.

[0172] Specifically, if the first moment occurs after the third vertical synchronization signal (HW_VSYNC signal), then no matter how fast the first layer is composited, the second image frame cannot be displayed within the synchronization period TZ corresponding to the third vertical synchronization signal (HW_VSYNC signal). For example, as shown in Figure 9, a detailed analysis of this situation can be referred to the analysis of Figure 9 above and will not be repeated here.

[0173] However, although the second image frame missed the refresh display when the third vertical synchronization signal (HW_VSYNC signal) arrived, the second image frame has been successfully cached in the second cache queue, waiting to be read by the display driver for refresh display. It can be understood that in this embodiment of the application, the second image frame is in a state where it can be read and refreshed at any time.

[0174] Therefore, in order to reduce the impact of frame loss, the second image frame is displayed as quickly as possible to prevent it from affecting the refresh display of subsequent image frames. In this embodiment of the present application, a new image refresh time is determined based on a predetermined frame rate to refresh and display the second image frame that has been lost in advance. In other words, the electronic device reads the second image frame in the second cache queue at the new image refresh time for refresh display.

[0175] Among them, the predetermined frame rate in the embodiment of the present application is greater than the frame rate of the display screen of the electronic device. The predetermined frame rate can be set according to the display requirements and the capabilities that the display hardware can support. In addition, the screen refresh moment when the electronic device reads the second image frame in the second cache queue for refresh display is after the third vertical synchronization signal (HW_VSYNC signal) and before the fourth vertical synchronization signal (HW_VSYNC signal). Among them, this fourth vertical synchronization signal (HW_VSYNC signal) is also a vertical synchronization signal for triggering the display of the image frame, and the fourth vertical synchronization signal (HW_VSYNC signal) is after the third vertical synchronization signal (HW_VSYNC signal) and is separated from the third vertical synchronization signal (HW_VSYNC signal) by one synchronization cycle. It can be understood that since the third vertical synchronization signal and the fourth vertical synchronization signal are both HW_VSYNC signals, the third vertical synchronization signal and the fourth vertical synchronization signal are fixedly separated by a synchronization cycle, which is the inverse of the screen refresh rate.

[0176] That is, the screen refresh time determined by the embodiment of the present application is between the third vertical synchronization signal (HW_VSYNC signal) and the fourth vertical synchronization signal (HW_VSYNC signal). That is, the screen refresh time is between the missed third vertical synchronization signal (HW_VSYNC signal) and the next third vertical synchronization signal (HW_VSYNC signal) corresponding to the third vertical synchronization signal (HW_VSYNC signal). For example, the screen refresh time is between time t4 (the third vertical synchronization signal) and time t5 (the fourth vertical synchronization signal, i.e., the next third vertical synchronization signal corresponding to time t4).

[0177] Generally speaking, electronic devices (such as display drivers) determine the refresh timing of image frames (such as the second image frame) based on the display's frame rate (i.e., screen refresh rate). In other words, if the producer and consumer rates are consistent and there are no frame drops, the display driver will typically refresh the display at least once within a synchronization period.

[0178] For example, a 60Hz frame rate refreshes and displays an image frame every 16.667 milliseconds, meaning the frame refresh interval is 6.667 milliseconds. A 90Hz frame rate refreshes and displays an image frame every 11.11 milliseconds, meaning the frame refresh interval is 11.11 milliseconds. A 120Hz frame rate refreshes and displays an image frame every 8.33 milliseconds, meaning the frame refresh interval is 8.83 milliseconds.

[0179] It can be seen that the higher the frame rate of the electronic device display screen (i.e., the screen refresh rate), the shorter the time interval for refreshing and displaying the image frame. Therefore, in the case of frame loss, a new picture refresh moment can be determined by a predetermined frame rate that is higher than the frame rate of the electronic device display screen (i.e., the screen refresh rate). It can be understood that since the predetermined frame rate is higher than the frame rate of the electronic device display screen (i.e., the screen refresh rate), the picture refresh moments corresponding to the predetermined frame rate are more numerous and more compact than the picture refresh moments corresponding to the frame rate of the electronic device display screen (i.e., the screen refresh rate). Furthermore, the electronic device refreshes and displays the second image frame at the picture refresh moment corresponding to the predetermined frame rate, thereby achieving the effect of displaying the second image frame in advance.

[0180] At the same time, in an embodiment of the present application, the electronic device refreshes and displays the second image frame only at the screen refresh moment between the third vertical synchronization signal (HW_VSYNC signal) and the fourth vertical synchronization signal (HW_VSYNC signal), thereby ensuring that the second image frame can be refreshed and displayed within the synchronization period TZ of the corresponding third vertical synchronization signal (HW_VSYNC signal), so that the image frame can also be displayed within the synchronization period TZ where the frame was originally lost.

[0181] As can be seen, although the display of the second image frame is slightly delayed compared to the case where no frame is dropped, the second image frame is displayed earlier than when the next vertical sync signal is refreshed. Therefore, the overall delay in displaying the second image frame is shortened, and the duration of image lag is reduced, thus quickly restoring image smoothness and reducing the impact of frame drop.

[0182] For example, as shown in FIG11 , taking the frame rate of the display screen as 120 Hz and the predetermined frame rate as 360 Hz as an example, the refresh display process in the embodiment of the present application is described.

[0183] As shown in Figure 11, if the frame rate of the display screen of the electronic device is 120HZ, then the original screen refresh moments are t1, t2, t3, t4 and t5 in Figure 11. The time interval between each screen refresh moment under 120HZ is 8.83 milliseconds. According to the conventional refresh display process, because the first moment is after t4, image frame 2 (the second image frame) cannot be refreshed and displayed at t4. Image frame 2 (the second image frame) will be delayed by 8.83 milliseconds and will not be refreshed and displayed until t5 (such as image frame display 2 shown in Figure 9).

[0184] However, in the embodiment of the present application, when the electronic device determines that the first moment is after the moment t4, it determines the new screen refresh moment according to the predetermined frame rate 360HZ. As shown in Figure 11, the moments t1, ta, tb, t2, tc, td, t3, te, tf, t4, tg, th and t5 are all screen refresh moments at 360HZ. It can be understood that the specific number of new screen refresh moments is determined according to the predetermined frame rate. Figure 11 is only an exemplary illustration and does not constitute a limitation. At the same time, these screen refresh moments are screen refresh moments at 360HZ, so the time interval between these screen refresh moments is 1 / 360 = 0.00277 seconds = 2.77 milliseconds.

[0185] Therefore, corresponding screen refresh times are added between time t4 (third vertical synchronization signal) and time t5 (fourth vertical synchronization signal), such as time tg and time th shown in FIG11 .

[0186] Furthermore, if the first moment is after moment t4, the electronic device misses the third vertical synchronization signal (HW_VSYNC signal) in response to moment t4 and does not refresh and display image frame 2 (such as the second image frame). In an embodiment of the present application, the electronic device can refresh and display image frame 2 (such as the second image frame) again at moment tg. At this time, although image frame 2 (such as the second image frame) is delayed by tg-t4 milliseconds, it can be displayed t5-tg milliseconds earlier than if it were displayed at moment t5.

[0187] For example, at 360HZ, image frame 2 (such as the second image frame) can be displayed at the earliest about 2.7ms after time t4. Compared with not displaying it in advance and displaying it at time t5, image frame 2 (such as the second image frame) can be displayed at most about 5.4ms in advance, thereby reducing the impact of frame loss.

[0188] It should be noted that the embodiment of the present application adds a new image refresh time by presetting the frame rate, and does not change the original frame rate of the display screen. It is understandable that after the embodiment of the present application completes the early display of the second image frame that was lost, the electronic device will still perform the drawing, rendering, synthesis, and refresh display process according to the original frame rate of the display screen (i.e., the screen refresh rate). For example, the frame rate of the display screen has always been 120HZ and will not be changed to 360HZ.

[0189] In some embodiments, in addition to the fact that the first moment is after the third vertical synchronization signal (HW_VSYNC signal) (for example, the first moment tw is after moment t4), which causes the second image frame to be unable to be refreshed and displayed smoothly, it is also possible that the third vertical synchronization signal (HW_VSYNC signal) is missed due to the timeout of the synthesis thread in synthesizing the layer, making it impossible to refresh and display the second image frame smoothly within the synchronization period TZ of the third vertical synchronization signal (HW_VSYNC signal).

[0190] That is, the first moment is before the third vertical synchronization signal (HW_VSYNC signal), but the second moment when the electronic device caches the second image frame to the second cache queue is after the third vertical synchronization signal (HW_VSYNC signal), which will also cause the third vertical synchronization signal (HW_VSYNC signal) to be missed and the second image frame cannot be refreshed and displayed smoothly within the synchronization period TZ of the third vertical synchronization signal (HW_VSYNC signal).

[0191] As shown in Figure 12, at time tw (the first time) before time t4, the electronic device caches layer 2 (the first layer) in the first cache queue. At time tw (the first time), the electronic device reads layer 2 (the first layer) from the first cache queue and performs "Image Frame Synthesis 2" to obtain image frame 2 (the second image frame).

[0192] However, the compositing thread only completes layer compositing and caching for frame 2 (the second image frame) at time tv (the second time). However, time tv (the second time) is after time t4, and thus misses the third vertical synchronization signal (HW_VSYNC signal) at time t4, causing frame 2 (the second image frame) to be delayed in refreshing and displaying at time t5. This also results in frame 2 (the second image frame) not being successfully displayed at time t4, resulting in unavoidable frame drop.

[0193] To reduce the impact of frame drops caused by composition thread timeout, in this embodiment of the present application, for the situation where the first moment is before the third vertical synchronization signal (HW_VSYNC signal), but the second moment (i.e., the moment when the electronic device caches the second image frame in the second cache queue) is after the third vertical synchronization signal (HW_VSYNC signal), a new image refresh moment can also be determined by a predetermined frame rate. Furthermore, the electronic device refreshes and displays the second image frame in the second cache queue at the new image refresh moment.

[0194] For example, as shown in Figure 13. At time tw (the first moment), the electronic device reads layer 2 (the first layer) from the first cache queue to perform "image frame synthesis 2", and obtains image frame 2 (the second image frame) at time tv (the second moment) and caches it in the second cache queue. At this time, although the electronic device missed the third vertical synchronization signal (HW_VSYNC signal) at time t4, the electronic device can refresh and display image frame 2 (the second image frame) at time tg (the screen refresh moment). As a result, the electronic device refreshes and displays image frame 2 (the second image frame) in advance at time tg (the screen refresh moment), reducing the impact of frame loss.

[0195] As shown in Figure 14, for the situation in an embodiment of the present application where the first moment is before the third vertical synchronization signal (HW_VSYNC signal), but the second moment is after the third vertical synchronization signal (HW_VSYNC signal), the embodiment of the present application shows a flowchart of another image processing method based on the vertical synchronization signal, including steps S1401-S1408.

[0196] It is understandable that this situation in the embodiment of the present application is similar to the situation at the first moment after the third vertical synchronization signal (HW_VSYNC signal), and the processing principles are the same. Therefore, the specific implementation of each step in Figure 14 can refer to the specific implementation of each step in Figure 7 and Figure 10, and the embodiment of the present application will not be repeated here.

[0197] In some embodiments, the electronic device may determine, based on a predetermined frame rate, multiple screen refresh moments at which the second image frame can be refreshed and displayed. For example, the tg moment and the th moment in Figures 11 and 13 . In the embodiments of the present application, the first screen refresh moment in the chronological order of the screen refresh moments is referred to as the first refresh moment. At least one screen refresh moment after the first screen refresh moment is referred to as the second refresh moment.

[0198] Because the first refresh moment and at least one second refresh moment are not determined by changing the frame rate of the display screen, but are determined based on a predetermined frame rate. Therefore, the electronic device may fail to refresh the display. Generally speaking, the display will be refreshed successfully at the first refresh moment. However, in order to ensure that the second image frame is refreshed and displayed successfully in advance as much as possible, the embodiment of the present application can refresh the display of the second image frame at each screen refresh moment (including the first refresh moment and the second refresh moment) based on a traversal method.

[0199] Specifically, the electronic device first refreshes and displays the second image frame in the second cache queue in advance at the first refresh moment in chronological order. If the second image frame is successfully refreshed and displayed at the first refresh moment, the electronic device stops refreshing and displaying the second image frame. If the second image frame fails to be refreshed and displayed at the first refresh moment, the electronic device then traverses in chronological order and refreshes and displays the second image frame in advance at each second refresh moment until the second image frame is successfully refreshed and displayed. That is to say, in the process of traversing the second refresh moments, once the electronic device successfully refreshes and displays the second image frame at a certain second refresh moment, indicating that the second image frame is successfully displayed in advance, the electronic device ends the process of refreshing and displaying the second image frame in advance, that is, stops traversing.

[0200] Alternatively, the electronic device may stop traversing after all image refresh moments have been traversed. For example, taking FIG11 as an example, if the electronic device fails to successfully refresh and display the second image frame at both time tg and time th, the electronic device stops traversing and then refreshes and displays the second image frame when the third vertical synchronization signal (HW_VSYNC signal) arrives at time t5.

[0201] Therefore, the embodiment of the present application can ensure that the second image frame is refreshed and displayed successfully as much as possible by traversing the screen refresh time to refresh the second image frame.

[0202] In some embodiments, since it takes a certain amount of processing time for the composition thread to perform layer composition, it is necessary to avoid as much as possible that the composition thread fails to complete the composition of the image frame before the third vertical synchronization signal (HW_VSYNC signal) arrives.

[0203] In the embodiment of the present application, if the first moment is before the second vertical synchronization signal (VSYNC_SF signal), but the time difference between the first moment and the second vertical synchronization signal is less than or equal to a preset time threshold, the first moment is considered to be after the second vertical synchronization signal. The preset time threshold can be set based on experience and the actual synthesis time required by the synthesis thread. For example, the preset time threshold can be 1 millisecond, 2 milliseconds, etc.

[0204] Taking 2 milliseconds as an example, except for the first moment after the second vertical synchronization signal (VSYNC_SF signal), all first moments within 2 milliseconds before the second vertical synchronization signal (VSYNC_SF signal) are considered to be the first moment after the second vertical synchronization signal (VSYNC_SF signal). For example, if the second vertical synchronization signal (VSYNC_SF signal) occurs at 5 milliseconds, then all first moments after 3 milliseconds are considered to be the first moments after the second vertical synchronization signal (VSYNC_SF signal) at 5 milliseconds.

[0205] In some embodiments, in the absence of exceptions (e.g., frame loss), the rate at which the display driver reads and displays image frames is consistent with the rate at which frames are generated and consumed. This can be understood as follows: the consumer (compositing thread) synthesizes one image frame per VSYNC cycle (e.g., the aforementioned synchronization period TZ) and caches it in the second cache queue, while the display driver retrieves one image frame from the second cache queue and refreshes it for display on the display screen every VSYNC cycle (e.g., the aforementioned synchronization period TZ).

[0206] Therefore, regarding whether there is a first image frame to be refreshed and displayed in response to the third vertical synchronization signal (HW_VSYNC signal) in the above embodiment, in addition to the situation that the corresponding second vertical synchronization signal (VSYNC_SF signal) is responded to with the first image frame being synthesized, there may be another situation.

[0207] That is, currently (i.e., at the first moment), the second cache queue includes the first image frame that can be refreshed and displayed. Because, at the first moment, if the second cache queue includes the first image frame waiting to be refreshed and displayed, then when the third vertical synchronization signal (HW_VSYNC signal) arrives, the electronic device will inevitably respond to the third vertical synchronization signal (HW_VSYNC signal) and refresh and display the first image frame waiting to be displayed in the second cache queue first.

[0208] Therefore, the first image frame that is still being synthesized or waiting to be synthesized cannot be refreshed when the third vertical synchronization signal (HW_VSYNC signal) arrives. Therefore, this situation is also regarded as the first image frame that can be refreshed when the third vertical synchronization signal (HW_VSYNC signal) arrives in this embodiment of the application.

[0209] For example, as shown in Figure 15, if the electronic device completes rendering of layer 1 (the first layer) and caches layer 2 (the first layer) in the first cache queue at time tt (the first moment) after time t3, the electronic device has timed out to complete layer 1 (the first layer). At this point, the image frame count in the first cache queue increases from 0 to 1 (i.e., 0 -> 1).

[0210] Meanwhile, if the second buffer queue currently includes the first image frame that can be refreshed and displayed, that is, at time tt, the second buffer queue stores one first image frame (shown as "1").

[0211] Then, if the electronic device triggers to immediately synthesize layer 2 (the first layer), image frame 2 (the second image frame) is obtained and cached in the second cache queue, the number of image frames in the second cache queue will increase from 1 to 2 (i.e., 1->2).

[0212] Then, when the third vertical synchronization signal (HW_VSYNC) arrives at time t4, the display driver reads image frames from the second cache queue for refresh. Since the queue follows the first-in-first-out principle, frame 2 (the second image frame) is dequeued after frame 1. Therefore, the display driver must read frame 1 instead of the immediately synthesized frame 2 (the second image frame). At this point, the number of image frames in the second cache queue at time t4 decreases from 2 to 1 (i.e., 2->1).

[0213] The image frame 2 (the second image frame) in the second cache queue may be refreshed and displayed at time t5. That is, at time t5, the image frame in the second cache queue will be reduced from 1 to 0 (i.e., 1->0).

[0214] It can be seen that in this case, even if the electronic device immediately synthesizes image frame 2 (the second image frame), this image frame 2 (the second image frame) will not be displayed at time t4. For the second image frame, it is still a frame loss state. Therefore, when at the first moment (time tt), there are other first image frames waiting to be refreshed and displayed in the second cache queue, and the embodiment of the present application is also regarded as the presence of a first image frame that can be refreshed by the third vertical synchronization signal (HW_VSYNC signal).

[0215] In summary, determining whether the third vertical synchronization signal (HW_VSYNC signal) contains a first image frame to be refreshed and displayed requires determining whether other layers of other applications are being composited in response to the second vertical synchronization signal (VSYNC_SF signal). Furthermore, it is necessary to determine whether the second buffer queue contains a first image frame to be displayed.

[0216] Only when no layer synthesis is performed in response to the second vertical synchronization signal (VSYNC_SF signal) (i.e., no synthesis is performed on the layers of any other applications) and there is no first image frame to be refreshed and displayed in the second cache queue, can it be determined that there is no first image frame to be refreshed and displayed in the third vertical synchronization signal (HW_VSYNC signal).

[0217] FIG16 shows a schematic structural diagram of the electronic device mentioned above.

[0218] The electronic device 1600 may include a processor 1610, an external memory interface 1620, an internal memory 1621, a universal serial bus (USB) connector 1630, a charging management module 1640, a power management module 1641, a battery 1642, an antenna 1, an antenna 2, a mobile communication module 1650, a wireless communication module 1660, an audio module 1670, a speaker 1670A, a receiver 1670B, a microphone 1670C, an earphone interface 1670D, a sensor module 1680, a button 1690, a motor 1691, an indicator 1692, a camera module 1693, a display screen 1694, and a subscriber identification module (SIM) card interface 1695, etc.

[0219] Among them, the sensor module 1680 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0220] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 1600. In other embodiments of the present application, the electronic device 1600 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0221] Processor 1610 may include one or more processing units, for example, an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors. Processor 1610 may generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.

[0222] Processor 1610 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 1610 may be a cache memory. This memory can store instructions or data that have been used or are frequently used by processor 1610. When processor 1610 needs to use the instruction or data, it can directly access it from this memory. This avoids duplicate accesses, reduces processor 1610 latency, and thus improves system efficiency.

[0223] In some embodiments, the processor 1610 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface. The processor 1610 may be connected to modules such as a touch sensor, an audio module, a wireless communication module, a display screen, and a camera module through at least one of the above interfaces.

[0224] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 1600. In other embodiments of the present application, the electronic device 1600 may also adopt a different interface connection method from the above embodiment, or a combination of multiple interface connection methods.

[0225] External memory interface 1620 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of electronic device 1600. The external memory card communicates with processor 1610 via external memory interface 1620 to implement data storage functions. For example, files such as music and videos can be saved on the external memory card or transferred from the electronic device to the external memory card.

[0226] The internal memory 1621 can be used to store computer executable program code, which includes instructions. The internal memory 1621 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 1600 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 1621 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 1610 executes various functional methods or data processing of the electronic device 1600 by running instructions stored in the internal memory 1621 and / or instructions stored in a memory provided in the processor.

[0227] The USB connector 1630 is an interface that complies with USB standard specifications and can be used to connect the electronic device 1600 and peripheral devices. Specifically, it can be a Mini USB connector, a Micro USB connector, a USB Type C connector, etc. The USB connector 1630 can be used to connect a charger to enable the charger to charge the electronic device 1600, and can also be used to connect other electronic devices to enable data transmission between the electronic device 1600 and other electronic devices. It can also be used to connect headphones to output audio stored in the electronic device through the headphones. The connector can also be used to connect other electronic devices, such as VR devices. In some embodiments, the standard specifications of the universal serial bus can be USB1.x, USB2.0, USB3.x and USB4.

[0228] The charging management module 1640 is used to receive charging input from a charger. While charging the battery 1642 , the charging management module 1640 can also provide power to the electronic device through the power management module 1641 .

[0229] The power management module 1641 is used to connect the battery 1642, the charging management module 1640, and the processor 1610. The power management module 1641 receives input from the battery 1642 and / or the charging management module 1640 and provides power to the processor 1610, the internal memory 1621, the display 1694, the camera module 1693, and the wireless communication module 1660. In other embodiments, the power management module 1641 may also be provided in the processor 1610. In other embodiments, the power management module 1641 and the charging management module 1640 may also be provided in the same device.

[0230] The wireless communication function of the electronic device 1600 can be implemented through antenna 1, antenna 2, mobile communication module 1650, wireless communication module 1660, modem processor and baseband processor.

[0231] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 1600 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0232] Mobile communication module 1650 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for electronic device 1600. Mobile communication module 1650 may include at least one filter, switch, power amplifier, low-noise amplifier (LNA), etc. Mobile communication module 1650 can receive electromagnetic waves from antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modem processor for demodulation. Mobile communication module 1650 can also amplify the signals modulated by the modem processor and convert them into electromagnetic waves for radiation via antenna 1.

[0233] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium- or high-frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs sound signals through an audio device (including but not limited to the speaker 1670A, the receiver 1670B, etc.) or displays images or videos through the display screen 1694.

[0234] The wireless communication module 1660 can provide wireless communication solutions for the electronic device 1600, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), Bluetooth low energy (BLE), ultra wide band (UWB), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. The wireless communication module 1660 can be one or more devices that integrate at least one communication processing module. The wireless communication module 1660 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 1610. The wireless communication module 1660 can also receive the signal to be transmitted from the processor 1610, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0235] In some embodiments, antenna 1 of electronic device 1600 is coupled to mobile communication module 1650, and antenna 2 is coupled to wireless communication module 1660, so that electronic device 1600 can communicate with a network and other electronic devices via wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0236] Electronic device 1600 can implement display functions using a GPU, display screen 1694, and an application processor. A GPU is a microprocessor for image processing that connects display screen 1694 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 1610 may include one or more GPUs that execute program instructions to generate or modify display information.

[0237] Display screen 1694 is used to display images, videos, and the like. Display screen 1694 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-OLED, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 1600 may include one or more display screens 1694.

[0238] The electronic device 1600 can realize the camera function through the camera module 1693, ISP, video codec, GPU, display screen 1694, application processor AP, neural network processor NPU, etc. The camera module 1693 can be used to collect color image data and depth data of the photographed object. The ISP can be used to process the color image data collected by the camera module 1693. The digital signal processor is used to process digital signals and can also process other digital signals. The video codec is used to compress or decompress digital video. The electronic device 1600 can support one or more video codecs. In this way, the electronic device 1600 can play or record videos in multiple encoding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0239] In some embodiments, the CPU, GPU, or NPU in the processor 1610 can process the color image data and depth data collected by the camera module 1693. The NPU is a neural network (NN) computing processor that quickly processes input information by drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, and can also continuously self-learn. The NPU can realize applications such as intelligent cognition of the electronic device 1600, such as image recognition, face recognition, voice recognition, and text understanding.

[0240] In some embodiments, electronic device 1600 may include one or more camera modules 1693. Specifically, electronic device 1600 may include one front camera module 1693 and one rear camera module 1693. Camera module 1693 may be composed of a color camera module and a 3D sensing module. Camera module 1693 may also be composed of two or more cameras.

[0241] The electronic device 1600 can implement audio functions such as music playback and recording through the audio module 1670, the speaker 1670A, the receiver 1670B, the microphone 1670C, the headphone jack 1670D, and the application processor.

[0242] Audio module 1670 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. Audio module 1670 can also be used to encode and decode audio signals. Speaker 1670A, also known as a "speaker," is used to convert audio electrical signals into sound signals. Receiver 1670B, also known as a "handset," is used to convert audio electrical signals into sound signals. Microphone 1670C, also known as a "microphone," is used to convert sound signals into electrical signals. Headphone jack 1670D is used to connect wired headphones.

[0243] The pressure sensor is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor can be located on the display screen 1694. There are many types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor can include at least two parallel plates made of conductive material. When force is applied to the pressure sensor, the capacitance between the electrodes changes. Electronic device 1600 determines the intensity of the pressure based on this change in capacitance. When a touch operation is applied to the display screen 1694, electronic device 1600 detects the intensity of the touch operation using the pressure sensor. Electronic device 1600 can also calculate the location of the touch based on the detection signal from the pressure sensor. In some embodiments, touch operations applied to the same touch location but with different touch operation intensities can correspond to different operation instructions. For example, when a touch operation with an intensity less than a first pressure threshold is applied to a short message application icon, an instruction to view short messages is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to a short message application icon, an instruction to create a new short message is executed.

[0244] A touch sensor, also known as a "touch device," can be provided on the display screen 1694. The touch sensor and the display screen 1694 form a touch screen, also known as a "touch screen." The touch sensor is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to an application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 1694. In other embodiments, the touch sensor can also be provided on the surface of the electronic device 1600, at a location different from that of the display screen 1694.

[0245] The buttons 1690 may include a power button, a volume button, etc. The buttons 1690 may be mechanical buttons. Alternatively, they may be touch buttons. The electronic device 1600 may receive button inputs and generate key signal inputs related to the user settings and function controls of the electronic device 1600. The motor 1691 may generate vibration prompts. The motor 1691 may be used for incoming call vibration prompts or for touch vibration feedback. The indicator 1692 may be an indicator light that may be used to indicate the charging status, power changes, messages, missed calls, notifications, etc. The SIM card interface 1695 is used to connect a SIM card. The SIM card may be connected to or disconnected from the electronic device 1600 by inserting or removing the SIM card interface 1695. The electronic device 1600 may support one or more SIM card interfaces. The SIM card interface 1695 may support Nano SIM cards, Micro SIM cards, SIM cards, etc.

[0246] Another embodiment of the present application provides an electronic device comprising: a display screen, one or more processors, and a memory. The display screen and the memory are each coupled to the processor; the memory stores one or more computer program codes, each comprising computer instructions; when the processor executes the computer instructions, the electronic device implements the image processing method based on a vertical synchronization signal described in any of the above embodiments.

[0247] Another embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor in an electronic device, the electronic device implements the image processing method based on the vertical synchronization signal recorded in any of the above embodiments.

[0248] The embodiment of the present application further provides a computer program product, which, when executed on a computer, enables the computer to execute the functions or steps in the above method embodiment.

[0249] The present application also provides a chip system, as shown in FIG17 . The chip system 1700 includes at least one processor 1701 and at least one interface circuit 1702. The processor 1701 and the interface circuit 1702 can be interconnected via a line. For example, the interface circuit 1702 can be used to receive signals from other devices (such as a computer memory). For another example, the interface circuit 1702 can be used to send signals to other devices (such as the processor 1701).

[0250] For example, the interface circuit 1702 can read instructions stored in the memory and send the instructions to the processor 1701. When the instructions are executed by the processor 1701, the computer can execute the various steps in the above embodiment. Of course, the chip system can also include other discrete devices, which are not specifically limited in the embodiments of the present application.

[0251] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0252] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0253] Units described as separate components may or may not be physically separate, and components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0254] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0255] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0256] The above content is only a specific embodiment of this application, but the scope of protection of this application is not limited to this. Any changes or replacements within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. An image processing method based on a vertical synchronization signal, characterized in that Applied to an electronic device, the method includes: In response to a first vertical synchronization signal, draw and render a first layer of a first application, and cache the first layer in a first cache queue at a first moment; The first moment is after a second vertical synchronization signal, and there is no first image frame to be refreshed and displayed in a third vertical synchronization signal. At the first moment, perform layer composition on the first layer in the first cache queue to obtain a second image frame, and cache the second image frame in a second cache queue; Wherein, the second vertical synchronization signal is used to trigger layer composition, the second vertical synchronization signal is after the first vertical synchronization signal, and is spaced from the first vertical synchronization signal by a first preset period; The third vertical synchronization signal is used to trigger image frame refresh and display, the third vertical synchronization signal is after the second vertical synchronization signal, and is spaced from the second vertical synchronization signal by a second preset period.

2. The method according to claim 1, wherein The method further includes: The first moment is before the third vertical synchronization signal. In response to the third vertical synchronization signal, refresh and display the second image frame in the second cache queue.

3. The method according to claim 2, characterized in that, The method further includes: The first moment is after the third vertical synchronization signal. Refresh and display the second image frame in the second cache queue at a screen refresh moment; wherein, the screen refresh moment is after the third vertical synchronization signal and before a fourth vertical synchronization signal; wherein, the fourth vertical synchronization signal is used to trigger image frame display, and the fourth vertical synchronization signal is after the third vertical synchronization signal and is spaced from the third vertical synchronization signal by a synchronization period.

4. The method according to claim 3, characterized in that, Before refreshing and displaying the second image frame in the second cache queue at the screen refresh moment, the method further includes: Determine the screen refresh moment according to a predetermined frame rate; wherein, the predetermined frame rate is greater than the frame rate of the display screen of the electronic device.

5. The method according to claim 1 or 2, characterized in that, Cache the second image frame in the second cache queue at a second moment; the method further includes: The second moment is after the third vertical synchronization signal. Refresh and display the second image frame in the second cache queue at a screen refresh moment; wherein, the screen refresh moment is after the third vertical synchronization signal and before a fourth vertical synchronization signal; the screen refresh moment is determined according to a predetermined frame rate.

6. The method according to any one of claims 2 - 5, characterized in that, The screen refresh moment includes a first refresh moment and at least one second refresh moment; the second refresh moment is after the first refresh moment; Refreshing and displaying the second image frame in the second cache queue at the screen refresh moment includes: At the first screen refresh moment, refresh and display the second image frame in the second cache queue; If the refresh and display fails at the first screen refresh moment, traverse the at least one second screen refresh moment in chronological order, and refresh and display the second image frame at each second screen refresh moment until the refresh and display is successful or the traversal is completed.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: No layer composition is performed in response to the second vertical synchronization signal; and at the first moment, when there is no first image frame to be refreshed and displayed in the second buffer queue, it is determined that there is no first image frame to be refreshed and displayed in the third vertical synchronization signal.

8. The method according to any one of claims 1-7, characterized in that The method further includes: Before the second vertical synchronization signal at the first moment, in response to the second vertical synchronization signal, layer composition is performed on the first layer in the first buffer queue to obtain a second image frame, and the second image frame is cached in the second buffer queue; In response to the third vertical synchronization signal, the second image frame in the second buffer queue is refreshed and displayed.

9. The method according to any one of claims 1 to 8, characterized in that, The method further includes: Before the second vertical synchronization signal at the first moment, and when the time difference between the first moment and the second vertical synchronization signal is less than or equal to a preset time threshold, it is determined that the first moment is after the second vertical synchronization signal.

10. The method according to any one of claims 1-9, characterized in that, The first preset period is equal to one synchronization period, and the second preset period is equal to one synchronization period.

11. An electronic device, characterized in that, It includes: A display screen, one or more processors and a memory, where the display screen and the memory are respectively coupled to the processor; One or more computer program codes are stored in the memory, and the computer program codes include computer instructions; when the processor executes the computer instructions, the electronic device is caused to execute the image processing method based on vertical synchronization signals as described in any one of claims 1-10.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor of the electronic device, the electronic device is caused to execute the image processing method based on vertical synchronization signals as described in any one of claims 1-10.

Images