Display method and electronic device

Abstract

The present application discloses a display method and an electronic device, the method comprising: receiving a first touch operation input by a user; if the touch scene of the first touch operation is a hands-off operation scene, performing a drawing and rendering operation on a first image frame within a first configuration duration; performing a synthesis operation on the first image frame after drawing and rendering within a second configuration duration; receiving a second touch operation input by a user; if the touch scene of the second touch operation is a hands-on operation scene, performing a drawing and rendering operation on the second image frame within a third configuration duration in the hands-on operation scene; performing a synthesis operation on the second image frame after drawing and rendering within a fourth configuration duration in the hands-on operation scene; wherein the third configuration duration is less than the first configuration duration, and / or the fourth configuration duration is less than the second configuration duration. In this way, different working durations are configured for different touch operation scenes, which can better adapt to different needs in different application scenes.

Description

[0001] This application is a divisional application of the invention application with an application date of January 28, 2023, Chinese application number 202310041624.7, and invention name “A display method and electronic device”. Technical Field

[0002] The present application belongs to the technical field of display systems, and in particular, relates to a display method and an electronic device. Background Art

[0003] Tracking performance is a key indicator for measuring touch experience. The shorter the tracking response time, the better the tracking performance. Tracking response time refers to the time required for an electronic device to display the page corresponding to the touch operation on the display screen after receiving the user's touch operation.

[0004] At present, for some touch screen operation scenarios, there is still a problem of poor hand tracking, that is, when the electronic device displays content, there is often a problem of display lag, which affects the user experience. Summary of the invention

[0005] The present application provides a display method and an electronic device for solving the problem of poor chirality.

[0006] In a first aspect, the present application provides a display method, which is applied to an electronic device, and the method includes: receiving a first touch operation input by a user; if the touch scenario of the first touch operation is a hand-off operation scenario, in response to a received first application vertical synchronization signal, within a first configuration time length in the hand-off operation scenario, performing a drawing and rendering operation on a first image frame; in response to a received first synthetic vertical synchronization signal, within a second configuration time length in the hand-off operation scenario, performing a synthesis operation on the first image frame after drawing and rendering; wherein the first synthetic vertical synchronization signal and the first application vertical synchronization signal are separated by the first configuration time length; in response to a received first hardware vertical synchronization signal, performing a display operation on the synthesized first image frame; wherein the first hardware vertical synchronization signal and the first synthetic vertical synchronization signal are separated by the second configuration time length; receiving a first touch operation input by a user; a second touch operation; if the touch scenario of the second touch operation is a hands-free operation scenario, then in response to the received second application vertical synchronization signal, within the third configuration time length in the hands-free operation scenario, a drawing and rendering operation is performed on the second image frame; in response to the received second synthetic vertical synchronization signal, within the fourth configuration time length in the hands-free operation scenario, a synthesis operation is performed on the second image frame after drawing and rendering; wherein the second synthetic vertical synchronization signal and the second application vertical synchronization signal are separated by the third configuration time length; in response to the received second hardware vertical synchronization signal, a display operation is performed on the synthesized second image frame; the second hardware vertical synchronization signal and the second synthetic vertical synchronization signal are separated by the fourth configuration time length; wherein the third configuration time length is less than the first configuration time length, and / or the fourth configuration time length is less than the second configuration time length.

[0007] In this way, different working durations are configured for different touch operation scenarios, which can better adapt to different needs in different application scenarios. Specifically, in the scenario of hands-on operation, the configuration duration (including drawing duration, and / or synthesis duration) can be adjusted to make the configuration duration in the hands-on operation scenario shorter than the configuration duration in the hands-off operation scenario. In this way, based on the adjusted configuration duration, the drawing rendering, synthesis and display operations are performed, which can reduce touch delay, improve hand tracking, and improve user experience.

[0008] In one possible implementation, the hands-on operation scenario includes a hands-on sliding operation scenario and a hands-on pressing operation scenario.

[0009] In one implementable manner, the method also includes: if the touch scenario of the first touch operation is a hands-off operation scenario, determining first configuration information corresponding to the hands-off operation scenario, the first configuration information including the first configuration duration and the second configuration duration; if the touch scenario of the second touch operation is a hands-on operation scenario, determining second configuration information corresponding to the hands-on operation scenario, the second configuration information including the third configuration duration and the fourth configuration duration.

[0010] In one implementable manner, the second application vertical synchronization signal and the third application vertical synchronization signal are spaced apart by a first time length, the second application vertical synchronization signal and the fourth application vertical synchronization signal are spaced apart by a second time length, the third application vertical synchronization signal is the previous application vertical synchronization signal of the second application vertical synchronization signal, and the fourth application vertical synchronization signal is the next application vertical synchronization signal of the second application vertical synchronization signal; the second synthetic vertical synchronization signal and the third synthetic vertical synchronization signal are spaced apart by a third time length, the second synthetic vertical synchronization signal and the fourth synthetic vertical synchronization signal are spaced apart by a fourth time length, the third synthetic vertical synchronization signal is the previous synthetic vertical synchronization signal of the second synthetic vertical synchronization signal, and the fourth synthetic vertical synchronization signal is the next synthetic vertical synchronization signal of the second synthetic vertical synchronization signal; wherein, the first time length is different from the second time length, and / or the third time length is different from the fourth time length.

[0011] In this way, the configuration duration may be modified by adjusting the offset of the applied vertical signal and / or the synthesized vertical signal.

[0012] In one implementable manner, if the touch scenario of the second touch operation is a hands-free operation scenario, the method further includes: generating the second application vertical synchronization signal based on the second hardware vertical synchronization signal and the first signal offset in the hands-free operation scenario; generating the second synthetic vertical synchronization signal based on the second hardware vertical synchronization signal and the second signal offset in the hands-free operation scenario.

[0013] In one implementable manner, the method further includes: obtaining a historical drawing time for executing the drawing and rendering operation, and a historical synthesis time for executing the synthesis operation; determining the third configuration time and the fourth configuration time according to the historical drawing time and the historical synthesis time; the third configuration time is greater than or equal to the maximum drawing time in the historical drawing time, and the fourth configuration time is greater than or equal to the maximum synthesis time in the historical synthesis time.

[0014] In this way, the third configuration duration and the fourth configuration duration determined based on historical statistical data can better ensure that the drawing rendering, synthesis and display operations are completed on time within the third configuration duration and the fourth configuration duration.

[0015] In one achievable manner, the method further includes: determining a chirality level corresponding to the hands-free operation scenario; and determining the third configuration duration and the fourth configuration duration based on the chirality level, wherein different chirality levels correspond to different configuration durations.

[0016] In this way, the present application further subdivides the hand-free sliding operation so that different hand-free sliding operation scenarios correspond to different configuration durations, which can better adapt to the hand tracking requirements in different sliding operation scenarios and further improve the user experience.

[0017] In one implementable manner, the non-hands-off operation scenario includes a finger-on-hand operation scenario and a stylus-on-hand operation scenario, and the stylus-on-hand operation scenario corresponds to a higher chirality level than the finger-on-hand operation scenario; wherein the third configuration time length corresponding to the stylus-on-hand operation scenario is shorter than the third configuration time length corresponding to the finger-on-hand operation scenario, and / or the fourth configuration time length corresponding to the stylus-on-hand operation scenario is shorter than the fourth configuration time length corresponding to the finger-on-hand operation scenario.

[0018] In an implementable manner, if the touch scenario of the second touch operation is a hands-free operation scenario, the method further includes: when there are multiple cached frames in a cache queue for caching rendered image frames, and the first cached frame at the top of the cache queue meets the frame loss condition, determining a first preset display sending time and a second preset display sending time corresponding to the first cached frame; the first preset display sending time is a preset time for performing a display sending operation on the first cached frame, and the second preset display sending time is a preset display sending time corresponding to a second synthetic vertical synchronization signal; when the first preset display sending time is earlier than the second preset display sending time, determining a third preset display sending time corresponding to the second cached frame; the third preset display sending time is a preset time for performing a display sending operation on the second cached frame, and the second cached frame is a cached frame in the cache queue located after the first cached frame; when the third preset display sending time is the same as the second preset display sending time, or when the third preset display sending time is earlier than the second preset display sending time, discarding the first cached frame and performing a synthesis operation on the second cached frame.

[0019] In this way, the first cache frame can be discarded and the second cache frame can be used to participate in the synthesis operation, so as to display the image on the display screen as quickly as possible without the user feeling any lag.

[0020] In one implementable embodiment, the method further includes: when the third preset display time is later than the second preset display time, and it is determined that the interval between the currently received second synthetic vertical synchronization signal and the previous second synthetic vertical synchronization signal is less than one Vsync cycle, or when the interval between the second application vertical synchronization signal used to trigger the drawing and rendering operation on the second cache frame and the previous second application vertical synchronization signal is less than one Vsync cycle, discarding the first cache frame and performing the synthesis operation on the second cache frame.

[0021] In this way, the accumulation of cache frames due to misjudgment of the cache status can be avoided.

[0022] In one achievable manner, the method further includes: when the first preset display time is the same as the second preset display time, or when the third preset display time is later than the second preset display time, performing a synthesis operation on the first cache frame.

[0023] In one implementable method, the frame loss conditions include: the timestamp attribute corresponding to the first preset display time of the first cached frame is an automatic timestamp, the current refresh rate is greater than 60Hz, and the time interval from the last time the image frame in the cache queue was discarded is greater than a preset time interval threshold.

[0024] In a second aspect, the present application provides another display method, which is applied to an electronic device, the method comprising: receiving a second touch operation input by a user; if the touch scenario of the second touch operation is a hand-free operation scenario, then when there are multiple cached frames in a cache queue for caching the rendered image frame, and the first cached frame at the top of the cache queue meets the frame loss condition, determining a first preset display sending time and a second preset display sending time corresponding to the first cached frame; the first preset display sending time is a preset time for performing a display sending operation on the first cached frame, and the second preset display sending time is a preset display sending time corresponding to a second synthetic vertical synchronization signal; when the first preset display sending time is earlier than the second preset display sending time, determining a third preset display sending time corresponding to the second cached frame; the third preset display sending time is a preset time for performing a display sending operation on the second cached frame, and the second cached frame is a cached frame in the cache queue located after the first cached frame; when the third preset display sending time is the same as the second preset display sending time, or when the third preset display sending time is earlier than the second preset display sending time, discarding the first cached frame and performing a synthesis operation on the second cached frame.

[0025] In this way, the first cache frame can be discarded and the second cache frame can be used to participate in the synthesis operation, so as to display the image on the display screen as quickly as possible without the user feeling any lag.

[0026] In one implementable embodiment, the method further includes: when the third preset display time is later than the second preset display time, and it is determined that the interval between the currently received second synthetic vertical synchronization signal and the previous second synthetic vertical synchronization signal is less than one Vsync cycle, or when the interval between the second application vertical synchronization signal used to trigger the drawing and rendering operation on the second cache frame and the previous second application vertical synchronization signal is less than one Vsync cycle, discarding the first cache frame and performing the synthesis operation on the second cache frame.

[0027] In this way, the accumulation of cache frames due to misjudgment of the cache status can be avoided.

[0028] In one achievable manner, the method further includes: when the first preset display time is the same as the second preset display time, or when the third preset display time is later than the second preset display time, performing a synthesis operation on the first cache frame.

[0029] In one implementable method, the frame loss conditions include: the timestamp attribute corresponding to the first preset display time of the first cached frame is an automatic timestamp, the current refresh rate is greater than 60Hz, and the time interval from the last time the image frame in the cache queue was discarded is greater than a preset time interval threshold.

[0030] In a third aspect, the present application provides a display device, which may include:

[0031] A first receiving module, used for receiving a first touch operation input by a user;

[0032] A first drawing and rendering module is used to, when the touch scene of the first touch operation is a hand-off operation scene, respond to a received first application vertical synchronization signal and perform a drawing and rendering operation on a first image frame within a first configured time length in the hand-off operation scene;

[0033] A first synthesis module is configured to perform a synthesis operation on the first image frame after drawing and rendering within a second configuration time length in the hands-off operation scenario in response to a received first synthesis vertical synchronization signal; wherein the first synthesis vertical synchronization signal and the first application vertical synchronization signal are separated by the first configuration time length;

[0034] A first display module, configured to perform a display operation on the synthesized first image frame in response to a received first hardware vertical synchronization signal; wherein the first hardware vertical synchronization signal and the first synthesized vertical synchronization signal are separated by the second configuration time length;

[0035] A second receiving module, used for receiving a second touch operation input by a user;

[0036] A second drawing and rendering module is used for, when the touch scene of the second touch operation is a hands-on operation scene, responding to a received second application vertical synchronization signal, performing a drawing and rendering operation on the second image frame within a third configured time length in the hands-on operation scene;

[0037] A second synthesis module is configured to perform a synthesis operation on the second image frame after drawing and rendering within a fourth configuration time length in the hands-free operation scenario in response to the received second synthesis vertical synchronization signal; wherein the second synthesis vertical synchronization signal and the second application vertical synchronization signal are separated by the third configuration time length;

[0038] A second display module is used to perform a display operation on the synthesized second image frame in response to a received second hardware vertical synchronization signal; the second hardware vertical synchronization signal and the second synthesized vertical synchronization signal are separated by the fourth configuration time length; wherein the third configuration time length is less than the first configuration time length, and / or the fourth configuration time length is less than the second configuration time length.

[0039] Among them, a display device provided in the third aspect can be used to execute any display method in the first aspect.

[0040] In a fourth aspect, the present application provides another display device, the device comprising a sixth determination module, configured to determine a first preset display sending time and a second preset display sending time corresponding to the first cached frame when there are multiple cached frames in a cache queue for caching rendered image frames and a first cached frame at the top of the cache queue meets a frame loss condition; the first preset display sending time is a preset time for performing a display sending operation on the first cached frame, and the second preset display sending time is a preset display sending time corresponding to a second synthetic vertical synchronization signal;

[0041] a seventh determination module, configured to determine that the second cache frame corresponds to a third preset display sending time when the first preset display sending time is earlier than the second preset display sending time; the third preset display sending time is a preset time for performing a display sending operation on the second cache frame, and the second cache frame is a cache frame in the cache queue that is located after the first cache frame;

[0042] The second synthesis module is used to discard the first cache frame and perform a synthesis operation on the second cache frame when the third preset display time is the same as the second preset display time, or when the third preset display time is earlier than the second preset display time.

[0043] Among them, a display device provided in the fourth aspect can be used to execute any display method in the second aspect.

[0044] In a fifth aspect, the present application provides an electronic device, comprising a memory and a processor; the memory and the processor are coupled; the memory is used to store computer program code, the computer program code comprises computer instructions, and when the processor executes the computer instructions, the electronic device executes a method as described in any one of the first aspect or the second aspect.

[0045] In a sixth aspect, the present application provides a computer storage medium, in which a computer program or instruction is stored. When the computer program or instruction is executed, the method as described in any one of the first aspect or the second aspect is executed.

[0046] In a seventh aspect, the present application provides a computer program product, comprising a computer-readable code, which, when executed in an electronic device, enables the electronic device to implement the method described in any one of the first aspect or the second aspect.

[0047] It should be understood that the beneficial effects of the second to seventh aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 A framework diagram of an image display process provided in an embodiment of the present application;

[0050] Figure 2 A schematic diagram of touch delay provided in an embodiment of the present application;

[0051] Figure 3 A schematic diagram of an image display process in a click operation scenario provided in an embodiment of the present application;

[0052] Figure 4A schematic diagram of an image display process in another click operation scenario provided in an embodiment of the present application;

[0053] Figure 5 A schematic diagram of another image display process provided in an embodiment of the present application;

[0054] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0055] Figure 7 A software structure block diagram of an electronic device provided in an embodiment of the present application;

[0056] Figure 8 A flow chart of a display method provided in Embodiment 1 of the present application;

[0057] Fig. 9 A module interaction diagram of a display method provided in Example 1 of the present application;

[0058] Fig.10 A scene diagram of a user sliding operation on a display screen provided in an embodiment of the present application;

[0059] Fig.11 A flowchart of a display method in a hands-free sliding operation scenario provided in Embodiment 1 of the present application;

[0060] Fig.12 A flowchart of a display method in another scenario of sliding operation without releasing the hand provided in the first embodiment of the present application;

[0061] Fig.13 A schematic diagram of a display process provided by an embodiment of the present application;

[0062] Fig.14 A statistical graph of the actual working time of a rendering and synthesis operation provided in an embodiment of the present application;

[0063] Fig.15 A flow chart of a display method provided in Embodiment 2 of the present application;

[0064] Fig.16 A schematic diagram of an image display process provided in Embodiment 2 of the present application;

[0065] Fig.17 A schematic diagram of another image display process provided in Embodiment 2 of the present application;

[0066] Fig.18 A schematic diagram of the structure of a display device provided in an embodiment of the present application;

[0067] Fig.19 A schematic diagram of the structure of another display device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0069] The technical solution of the present application is described below in conjunction with the accompanying drawings.

[0070] In the description of this application, unless otherwise specified, "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, "at least one" means one or more, "at least two" means two or more, and "multiple" also means two or more. The words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not necessarily limit them to be different.

[0071] It should be noted that, in this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplarily" or "for example" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0072] In order to facilitate understanding of the technical solution of the present application, the application scenarios of the technical solution provided by the present application are first exemplified below.

[0073] like Figure 1 As shown, when the user touches the touch screen TP, the pressure sensor in the touch screen TP detects the pressure value and uploads the pressure value to the algorithm library. The algorithm library converts the pressure value into a coordinate value (also called point reporting data), and then sends the coordinate value to the Input module. The Input module can resample the coordinate value to obtain the resampled coordinate value. When the application receives the Vsync-APP signal (also called the application vertical synchronization signal), the application obtains the resampled coordinate value for image drawing and rendering; when the Vsync-SF signal (also called the synthetic vertical synchronization signal) is received, the synthesis module synthesizes the rendered image; when the Vsync-HW signal (also called the hardware vertical synchronization signal) is received, the synthesis module sends the synthesized image to the display LCD for display.

[0074] After an electronic device receives a touch operation input by a user, the time required for the page corresponding to the touch operation to be displayed on the display screen is called the hand tracking response time, also known as touch latency.

[0075] Combination Figure 1 and Figure 2 As shown, the touch delay mainly includes the following four delays: The first delay T1 represents the delay from the user touching the touch screen TP ( Figure 2 Point A) to generate reporting data ( Figure 2 The second part of the delay T2 represents the time it takes to transmit the reporting data to the input subsystem and wait for the reporting data to be consumed ( Figure 2 The third part of the delay T3 represents the time consumed by taking out the reporting point data, rendering, synthesis and sending to display ( Figure 2 The fourth part of the delay T4 represents the time consumed by the synthesized image display ( Figure 2 The time that the image (point E in the middle) is spent on the display.

[0076] Figure 2 The whole process from the user touching the display screen to the screen corresponding to the touch operation being displayed on the display screen is shown. It should be understood that optimizing any of the above four delays can improve the hand tracking performance. The method provided in the embodiment of the present application aims to optimize the above third delay T3.

[0077] The processing flow corresponding to the third part of the delay T3 is further introduced below.

[0078] For ease of understanding, firstly, some related concepts in the processing flow corresponding to the third part delay T3 are introduced.

[0079] 1. Frame: refers to the smallest unit of a single picture in the interface display. A frame can be understood as a still picture. Displaying multiple connected frames quickly and continuously can create the illusion of motion. Frame rate refers to the number of frames that refresh an image in 1 second. It can also be understood as the number of times the graphics processor in an electronic device refreshes the screen per second. A high frame rate can produce smoother and more realistic animations. The more frames per second, the smoother the displayed action will be.

[0080] It should be noted that before the interface displays a frame, it usually needs to go through processes such as drawing, rendering, and synthesis.

[0081] 2. Frame drawing: refers to the picture drawing of the display interface. The display interface can be composed of one or more views, each view can be drawn by the visual control of the view system, each view is composed of subviews, and a subview corresponds to a small widget in the view, for example, one of the subviews corresponds to a symbol in the picture view.

[0082] 3. Frame rendering: It is to color the drawn view or add 3D effects, etc. For example, 3D effects can be lighting effects, shadow effects, and texture effects.

[0083] 4. Frame synthesis: It is the process of synthesizing multiple or more rendered views into a display interface.

[0084] 5. Sending for display: It is the operation of sending the synthesized view to the display module (such as the display driver chip DDIC).

[0085] The interface display process corresponding to the third part of the delay T3 is described below.

[0086] In order to improve the smoothness of display and reduce the occurrence of display freezes and other phenomena, electronic devices generally display based on vertical synchronization (Vertical Synchronization, Vsync) signals to synchronize the image drawing, rendering, synthesis and display processes.

[0087] The Vsync signal is a periodic signal, and the Vsync signal period can be set according to the screen refresh rate. For example, when the screen refresh rate is 60Hz, the Vsync signal period can be 16.6ms, that is, the electronic device generates a control signal every 16.6ms to trigger the Vsync signal period. For another example, when the screen refresh rate is 90Hz, the Vsync signal period can be 11.1ms, that is, the electronic device generates a control signal every 11.1ms to trigger the Vsync signal period.

[0088] It should be noted that the Vsync signal is divided into a hardware Vsync signal (also called a Vsync-HW signal) and a software Vsync signal. The Vsync-HW signal is generated by the screen LCD, and the synthesis module converts the Vsync-HW signal into a software signal: Vsync-APP signal and Vsync-SF signal. The Vsync-APP signal is used to trigger the execution of drawing and rendering operations. The Vsync-SF signal is used to trigger the execution of synthesis operations. The Vsync-HW signal is used to trigger the execution of display operations.

[0089] It should also be noted that the system in current electronic devices pre-configures sufficient drawing time for applications and sufficient synthesis time for synthesis modules to perform drawing rendering and synthesis operations. Specifically, when the synthesis module converts the Vsync-HW signal into a software signal, the offset of the Vsync-APP signal and the Vsync-SF signal can be adjusted so that the Vsync-APP signal, the Vsync-SF signal and the Vsync-HW signal are sent at different positions in each Vsync cycle, so that the drawing rendering and synthesis operations meet their respective configured durations.

[0090] For example, Figure 3 As shown in FIG. 1 , when the synthesis module converts the Vsync-HW signal into a software signal, the offset of the Vsync-APP signal is set to 1ms, and the offset of the Vsync-SF signal is set to -2ms. In this way, the sending rhythm of the Vsync-APP signal, the Vsync-SF signal, and the Vsync-HW signal is as follows: Figure 3 shown.

[0091] Currently, based on the offset of the Vsync-APP signal and the offset of the Vsync-SF signal, the method for calculating the drawing duration configured for the drawing rendering operation and the composition duration configured for the composition operation is as follows:

[0092] The drawing time T is calculated according to the following first relation: app , calculate the synthesis time T according to the following second relationship: sf .

[0093] The first relation is: T app =T te +(sfOffset-appOffset);

[0094] The second relationship is: T sf =T te –sfOffset.

[0095] Among them, T app Indicates the drawing duration configured for the drawing rendering operation, T sf Represents the composition duration configured for the composition operation, T te Indicates the Vsync cycle, appOffset indicates the offset of the Vsync-APP signal, and sfOffset indicates the offset of the Vsync-SF signal.

[0096] It should be noted that, according to the existing calculation method, if the T calculated according to the following first relationship is app Less than T te , then in the calculated T app Add a T on top of that te .

[0097] For example, appOffset = 1ms, sfOffset = -2ms, T te = 11.1ms as an example. According to the first relationship, T app =11.1+(-2-1)=8.1ms, T sf =11.1–(-2)=13.1ms. app =8.1ms<11.1ms, therefore, Tapp =8.1+11.1=19.2ms.

[0098] In this way, Figure 3 As shown, in response to the Vsync-APP signal 11, the application performs a rendering operation on frame 1, and the rendered frame 1 waits for the Vsync-SF signal (i.e., the Vsync-SF signal) which is 19.2ms apart from the Vsync-APP signal 11. Figure 3 That is, in response to the Vsync-SF signal 22, the synthesis module takes out the rendered frame 1 for consumption and performs synthesis operation. Similarly, the synthesized frame 1 has to wait until the Vsync-HW signal (i.e., the Vsync-SF signal 22) with an interval of 13.1ms from the Vsync-SF signal 22 arrives. Figure 3 In other words, in response to the Vsync-HW signal 33, the display module performs a display operation on the synthesized frame 1 and refreshes the frame 1 on the screen. In this way, it takes about three Vsync cycles from rendering the frame 1 to sending the frame 1 to the display module for display.

[0099] At present, for any touch operation scenario of the user, the electronic device uses the corresponding drawing duration and synthesis duration pre-set by the above method to perform drawing rendering, synthesis and display operations. This leads to the following problem: the above pre-set configuration duration may be able to meet the requirements of drawing in some application scenarios (such as click operation scenarios). However, for scenarios that require hand tracking, such as dragging pages and other scenarios where the user does not let go of the hand, the drawing rendering, synthesis and display operations are still performed based on such configuration duration, which may cause poor hand tracking. For example, the above Figure 3 As shown, it takes about three Vsync cycles for a frame of image to be rendered and sent to the display module for display. The image display efficiency is poor, the touch response is slow, and the hand tracking is poor, which seriously affects the user's application experience in the scenario of sliding with hands.

[0100] like Figure 5 As shown, the rendered image frame can be cached in the cache queue of the synthesis module, waiting to be consumed by the synthesis module. After receiving the Vsync-SF signal, the synthesis module sequentially takes out the cache frames from the cache queue to perform synthesis operations.

[0101] In some abnormal situations, such as the failure to complete the rendering of frame 1 in the first Vsync cycle, two cached frames may be accumulated in the cache queue in the second Vsync cycle. When there are multiple cached frames in the cache queue, the synthesis module takes out the cached frames from the cache queue for synthesis operations in the order in which the cached frames are queued. For example, in the first Vsync cycle, the application starts to draw and render frame 4, and the cache queue includes frames 2 and 3 after drawing and rendering. In the second Vsync cycle, the application starts to draw and render frame 5, and the synthesis module takes out the first frame 2 from the cache queue for synthesis operations, and the cache queue includes frames 3 and 4 after drawing and rendering. And so on, in the fifth Vsync cycle, frame 4 is displayed on the screen. It can be seen that compared with the case where there are no cached frames in the cache queue, when there are two cached frames in the cache queue, the image will be delayed by two Vsync cycles.

[0102] Therefore, the number of cached frames accumulated in the cache queue will also affect the third part of the delay T3.

[0103] In summary, the third part of the delay T3 can be expressed by the following third relationship: T3 = T 配 +M*T te .

[0104] Among them, T 配 Indicates the total configuration time from the start of rendering to the display operation; M represents the number of cached frames accumulated in the cache queue, and M is a positive integer greater than or equal to 0.

[0105] In order to optimize the third part of the delay T3, an embodiment of the present application provides a display method. The display method can reduce the third part of the delay T3 by optimizing the configuration time and / or reducing the number of cache frames accumulated in the cache queue, thereby improving the hand tracking performance.

[0106] A display method provided in an embodiment of the present application can be applied to any electronic device with a touch screen, such as a mobile phone, a watch, a tablet computer, etc.

[0107] For example, the specific structure of the electronic device can be found in Figure 6 Schematic diagram of the structure of the electronic device shown.

[0108] like Figure 6As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0109] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0110] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0111] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0112] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0113] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0114] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0115] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0116] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0117] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0118] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0119] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0120] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0121] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0122] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0123] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0124] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0125] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0126] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0127] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0128] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0129] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0130] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0131] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0132] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0133] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0134] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0135] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0136] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0137] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0138] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0139] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.

[0140] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0141] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0142] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A. A plurality of speakers 170A can be provided in the electronic device 100. For example, a speaker 170A can be provided on the top of the electronic device 100, and a speaker 170A can also be provided on the bottom, and so on.

[0143] The receiver 170B, also called a "receiver", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the receiver 170B can be placed close to the human ear to receive the voice. In some embodiments, the speaker 170A and the receiver 170B can also be set as one component, which is not limited in the present application.

[0144] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0145] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0146] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0147] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

[0148] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0149] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover can be automatically unlocked.

[0150] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0151] The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0152] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0153] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0154] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0155] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of a processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 performs a boost on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

[0156] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0157] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 180M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 180M can also be set in an earphone and combined into a bone conduction earphone. The audio module 170 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 180M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M to realize a heart rate detection function.

[0158] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0159] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0160] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0161] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0162] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present invention, the Android system of the layered architecture is taken as an example to exemplify the software structure of the electronic device 100.

[0163] Figure 7 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0164] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.

[0165] The application layer can include a series of application packages.

[0166] like Figure 7 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0167] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0168] like Figure 7 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0169] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0170] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0171] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0172] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0173] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0174] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0175] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0176] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0177] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0178] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0179] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0180] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0181] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0182] A 2D graphics engine is a drawing engine for 2D drawings.

[0183] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0184] The display method provided by the present application is described below.

[0185] Embodiment 1

[0186] The display method provided in the first embodiment of the present application reduces the third part delay T3 by optimizing the configuration time, thereby improving the hand tracking performance.

[0187] Figure 4 For Figure 3 The corresponding display process diagram is shown below. Figure 4 As shown, the applicant found that in the actual execution process, in most cases, there is a large amount of idle time in the drawing time configured for the drawing and rendering operation. For example, although the drawing time configured for the drawing and rendering operation is 19.2ms, the actual time consumed for drawing and rendering a frame of image is only 8.6ms, and there is an idle time of 10.6ms. Similarly, although the synthesis time configured for the synthesis operation is 13.1ms, the actual time consumed for synthesizing a frame of image is only 8.1ms, and there is an idle time of 5.2ms.

[0188] In view of this, the display method provided in the first embodiment of the present application can, in order to improve the tracking performance of the hands-free sliding scene, shorten the working time configured for drawing rendering and synthesis operations after confirming that the current scene is a hands-free operation scene, speed up the touch response, and refresh the image to the screen as soon as possible.

[0189] Figure 8 A flow chart of a display method provided in Embodiment 1 of the present application, such as Figure 8 As shown, the display method provided in the first embodiment of the present application may include the following steps:

[0190] Step S101: receiving a first touch operation input by a user.

[0191] Step S102, if the touch scenario of the first touch operation is a hand-off operation scenario, in response to the received first Vsync-APP signal (ie, the first application vertical synchronization signal), a drawing rendering operation is performed on the first image frame within a first configured duration in the hand-off operation scenario.

[0192] Step S103 , in response to the received first Vsync-SF signal (ie, the first synthesized vertical synchronization signal), a synthesis operation is performed on the rendered first image frame within a second configured duration in the hands-off operation scenario.

[0193] Step S104 , in response to the received first Vsync-HW signal (ie, the first hardware vertical synchronization signal), performing a display operation on the synthesized first image frame.

[0194] The embodiments of the present application include two touch operation scenarios, one is a hand-off operation scenario, such as a click operation scenario. The other is a hand-on operation scenario, which is also called a hand-following operation scenario. Among them, the hand-on operation scenario can include a hand-on sliding operation scenario and a hand-on pressing operation scenario.

[0195] The first configuration duration refers to the drawing duration configured for the rendering operation in the hands-off operation scenario; the second configuration duration refers to the synthesis duration configured for the synthesis operation in the hands-off operation scenario. The first Vsync-SF signal and the first Vsync-APP signal are separated by the first configuration duration, and the first Vsync-HW signal and the first Vsync-SF signal are separated by the second configuration duration. Figure 3 and Figure 4 As shown, the first configuration duration is 19.2 ms and the second configuration duration is 13.1 ms.

[0196] like Figure 3As shown, taking the display process of frame 1 as an example, in response to the received first Vsync-APP signal 11, a drawing and rendering operation is performed on frame 1 to obtain the drawn and rendered frame 1. Afterwards, when the first Vsync-SF signal 22 is received, which is separated from the first Vsync-APP signal 11 by a first matching time length (i.e., 19.2ms), a synthesis operation is performed on the drawn and rendered frame 1 to obtain the synthesized frame 1. Next, when the first Vsync-HW signal 33 is received, which is separated from the first Vsync-SF signal 22 by a second matching time length (i.e., 13.1ms), a display sending operation is performed on the synthesized frame 1 to brush the image corresponding to frame 1 onto the screen.

[0197] It should be noted that the first Vsync-HW signal 33 is located after the first Vsync-SF signal 22 , and the first Vsync-SF signal 22 is located after the first Vsync-APP signal 11 .

[0198] Step S105: receiving a second touch operation input by the user.

[0199] Step S106, if the touch scenario of the second touch operation is a hands-on operation scenario, in response to the received second Vsync-APP signal (i.e., the second application vertical synchronization signal), a drawing and rendering operation is performed on the second image frame within a third configured time length in the hands-on operation scenario.

[0200] Step S107 , in response to the received second Vsync-SF signal (ie, the second synthesized vertical synchronization signal), a synthesis operation is performed on the rendered second image frame within a fourth configured duration in the hands-free operation scenario.

[0201] Step S108 , in response to the received second Vsync-HW signal (ie, the second hardware vertical synchronization signal), performing a display operation on the synthesized second image frame.

[0202] The third configuration duration refers to the drawing duration configured for the drawing and rendering operation in the hands-free operation scenario; the fourth configuration duration refers to the synthesis duration configured for the synthesis operation in the hands-free operation scenario. The second Vsync-SF signal and the second Vsync-APP signal are separated by the third configuration duration, and the second Vsync-HW signal and the second Vsync-SF signal are separated by the fourth configuration duration. The third configuration duration is less than the first configuration duration, and / or the fourth configuration duration is less than the second configuration duration. For example, Fig.11 and Fig.12 As shown, the first configuration duration is 11.1 ms and the second configuration duration is 13.1 ms.

[0203] like Fig.11As shown, taking the display process of frame 0 as an example, in response to the received second Vsync-APP signal 11, a drawing and rendering operation is performed on frame 0 to obtain the drawn and rendered frame 0. Afterwards, when a second Vsync-SF signal 22 is received which is spaced apart from the second Vsync-APP signal 11 by a third matching time length (i.e., 11.1ms), a synthesis operation is performed on the drawn and rendered frame 0 to obtain the synthesized frame 0. Next, when a second Vsync-HW signal 33 which is spaced apart from the second Vsync-SF signal 22 by a fourth matching time length (i.e., 13.1ms), a display sending operation is performed on the synthesized frame 0 to brush the image corresponding to frame 0 onto the screen.

[0204] It should be noted that the second Vsync-HW signal 33 is located after the second Vsync-SF signal 22 , and the second Vsync-SF signal 22 is located after the second Vsync-APP signal 11 .

[0205] like Fig.12 As shown in the figure, in the hands-on operation scenario (such as the hands-on sliding operation scenario), when the drawing and rendering operation is performed based on the drawing duration corresponding to the hands-on operation scenario, the idle time is 2.5ms, and when the synthesis operation is performed based on the synthesis duration corresponding to the hands-on operation scenario, the idle time is 5.2ms. Figure 4 The display process shown in the figure shortens the idle time of drawing and rendering in the hands-free operation scenario, thereby shortening the duration of the entire display process and improving the hand tracking performance. Specifically, in the hands-free operation scenario, a frame of image needs to go through about three Vsync cycles before it can be output to the display screen for display; in the hands-free sliding operation scenario, a frame of image needs to go through about two Vsync cycles before it can be output to the display screen for display. In this way, compared to Figure 4 In the hands-off operation scenario, the present application can refresh the image to the display screen one Vsync cycle in advance in the hands-on sliding operation scenario, thereby improving the hand tracking performance in the hands-on sliding operation scenario.

[0206] The display method provided in the embodiment of the present application configures different working durations for different touch operation scenarios, which can better adapt to different requirements in different application scenarios. For example, the hand-off operation scenario can be an operation in which the user clicks on the touch screen, and the hand-on operation scenario can be an operation in which the user slides on the touch screen without leaving the hand, such as dragging and displaying icons on the desktop. In general, compared with the click operation, the user has higher requirements for the hand-following performance of the hand-on sliding operation. Therefore, in the embodiment of the present application, the total configuration duration of the corresponding drawing and synthesis in the hand-on sliding operation scenario can be made shorter than the total configuration duration of the drawing and synthesis in the click operation scenario. In this way, in the click operation scenario, based on the longer configuration duration, the drawing rendering, synthesis and display operations are performed, which neither affects the user's application experience nor reduces the load of the electronic device. In the hand-on sliding operation scenario, based on the shorter configuration duration, the drawing rendering, synthesis and display operations are performed, which can reduce the touch delay, improve the hand-following performance, and improve the user experience.

[0207] Combine the following Fig. 9 The interaction between the modules involved in the method provided in the first embodiment of the present application is described. The modules involved include: an input module, an application, a synthesis module and a display module.

[0208] Step S201: When the electronic device receives a touch operation input by a user, the Input module may determine the input event type.

[0209] There are three types of input events: Down event, Move event, and Up event. Down event indicates that the screen is being pressed, Move event indicates that the screen is being slid, and Up event indicates that the screen is being left.

[0210] Fig.10 A scene diagram of a user sliding operation on a display screen provided in an embodiment of the present application, Fig.10 The path of a user sliding on the display screen is exemplarily shown through position 1, position 2, position 3, position 4 and position 5, wherein position 1, position 2, position 3, position 4 and position 5 represent corresponding coordinate positions on the display screen.

[0211] The user starts pressing the display screen from position 1, and then slides down in a straight line until he lifts his hand off the display screen at position 5. It should be noted that the user passes through multiple positions in the process of sliding down in a straight line. Fig.10 Only positions 2, 3 and 4 are shown as examples.

[0212] When the user performs a sliding operation, the Input module can determine the input event type corresponding to each touch point. For example, Fig.10The input event type corresponding to position 1 is Down, the input event type corresponding to position 5 is Up, and the input event types between positions 1 and 5 are all Move, such as the input event types corresponding to positions 2, 3, and 4 are all Move. It should be understood that before position 5, the operations performed by the user are all sliding operations without releasing the finger, and at position 5, the user's finger leaves the screen.

[0213] It should be noted that the Input module can not only be used to determine the input event type, but also to obtain coordinate information corresponding to the touch operation and send the coordinate information to the application for drawing and rendering operations.

[0214] Step S202: When the input event type is Down or Up, the Input module sends the corresponding input event type information to the synthesis module.

[0215] For example, if the Input module determines that the current input event type is a Down event, the Input module can send a first indication message to the synthesis module, and the first indication message is used to indicate that the currently received input event is a Down event. For another example, if the Input module determines that the current input event type is an Up event, the Input module can send a second indication message to the synthesis module, and the second indication message is used to indicate that the currently received input event is an Up event.

[0216] When the Input module determines that the current input event type is a Move event, it may not send any indication information to the synthesis module.

[0217] Step S203: In response to the Vsync-SF signal, the synthesis module determines the current touch scene according to the currently received input event type.

[0218] It should be noted that each time the synthesis module determines the current touch scene, the synthesis module may receive only one Down event, the synthesis module may receive only one Up event, or the synthesis module may receive two events, one Down event and one Up event. In another case, the synthesis module does not receive any event.

[0219] Correspondingly, in response to the Vsync-SF signal, if the synthesis module currently only receives a Down event, the synthesis module determines that the current touch scene is a hands-on operation scene, for example, a hands-on sliding operation scene or a hands-on pressing operation scene; if the synthesis module currently receives two events (a Down event and an Up event), the synthesis module determines that the current touch scene is a hands-off operation scene, for example, a click operation scene; if the synthesis module currently only receives an Up event, the synthesis module determines that the current touch scene is a hands-off operation scene.

[0220] Step S204: The synthesis module calls configuration information corresponding to the current touch scene.

[0221] The configuration information may include a drawing duration configured for drawing rendering, a composition duration configured for composition, a Vsync-APP signal offset, and a Vsync-SF signal offset.

[0222] Exemplarily, the hands-off operation scenario corresponds to the first configuration information, the first configuration information may include a first configuration duration and a second configuration duration, and the first configuration information may also include a first offset and a second offset. The first offset is the Vsync-APP signal offset corresponding to the hands-off operation scenario, and the second offset is the Vsync-SF signal offset corresponding to the hands-off operation scenario.

[0223] Exemplarily, the hands-free operation scenario corresponds to the second configuration information, the second configuration information may include a third configuration duration and a fourth configuration duration, and the second configuration information may also include a third offset and a fourth offset. The third offset is the Vsync-APP signal offset corresponding to the hands-free operation scenario, and the fourth offset is the Vsync-SF signal offset corresponding to the hands-free operation scenario.

[0224] Among them, the third configuration duration can be less than the first configuration duration and the fourth configuration duration is the same as the second configuration duration, or the fourth configuration duration can be less than the second configuration duration and the third configuration duration is the same as the first configuration duration, or the third configuration duration can be less than the first configuration duration and the fourth configuration duration can be less than the second configuration duration. As long as the total configuration duration of drawing, rendering, and synthesis in the hands-on sliding operation scenario is less than the total configuration duration of drawing, rendering, and synthesis in the hands-off sliding operation scenario, it will be fine.

[0225] Correspondingly, the third offset may be different from the first offset, or the fourth offset may be different from the second offset, or the third offset may be different from the first offset and the fourth offset may be different from the second offset.

[0226] Step S205 : The synthesis module adjusts the Vsync-APP signal and the Vsync-SF signal based on the Vsync-APP signal offset and the Vsync-SF signal offset in the configuration information.

[0227] Step S206: The synthesis module sends the Vsync-APP signal with the offset adjusted to the application.

[0228] Step S207 : The application performs a drawing and rendering operation in response to the Vsync-APP signal after the offset is adjusted.

[0229] Step S208: The application sends the rendered image to the buffer queue of the synthesis module.

[0230] Step S209 : The synthesis module performs a synthesis operation in response to the Vsync-SF signal having a drawing duration interval with the adjusted Vsync-APP signal.

[0231] Step S2010: The synthesis module sends the synthesized image to the buffer queue of the display module.

[0232] Step S2011 : the display module performs a display operation in response to a Vsync-HW signal having a synthesis time interval with the adjusted Vsync-SF signal.

[0233] For example, in step S203, it is determined that the current touch scene is the touch-and-slide scene, and the offset of the Vsync-APP signal in the configuration information corresponding to the touch-and-slide scene is -2ms, and the offset of the Vsync-SF signal is -2ms. Fig.11 As shown, the synthesis module adjusts the Vsync-APP signal and the Vsync-SF signal based on the Vsync-APP signal offset and the Vsync-SF signal offset corresponding to the hands-free sliding scene in the above step S205. The transmission rhythm of the adjusted Vsync-APP signal, Vsync-SF signal and Vsync-HW signal is as follows: Fig.11 shown.

[0234] Next, step S206 to step S2011 perform drawing rendering, synthesis and display sending operations based on the adjusted Vsync-APP signal and Vsync-SF signal sending rhythm, as well as the drawing duration and synthesis duration corresponding to the hand-free sliding scene, until the current input event type received by the synthesis module is Up. When the current input event type received by the synthesis module is Up, the synthesis module calls the configuration information corresponding to the hand-free operation scene again, and subsequently performs drawing rendering, synthesis and display sending operations with the Vsync-APP signal and Vsync-SF signal sending rhythm corresponding to the hand-free sliding operation scene, and the configuration duration corresponding to the hand-free sliding scene.

[0235] Fig.12 For Fig.11 The corresponding display process diagram. Fig.11 and 12As shown, in the scenario of sliding operation without leaving the hand, the drawing time in the configuration information is 11.1ms, the synthesis time in the configuration information is 13.1ms, the Vsync-APP signal offset is -2ms, and the Vsync-SF signal offset is -2ms. In this way, the drawing, rendering, synthesis and display time of a frame of image can be reduced to 22.1ms. In other words, in the scenario of sliding operation without leaving the hand, the present application can output a frame of image to the display screen for display after two Vsync cycles. In this way, compared with Figure 4 The image can be refreshed to the display screen one Vsync cycle in advance, thereby improving the hand tracking in the scenario of sliding operation without taking the hand away.

[0236] For example, Fig.13 As shown, in the display process of frame 1, the synthesis module adopts the configuration information corresponding to the hands-off operation scenario, wherein the drawing duration configured for the drawing rendering operation is 19.2ms, and the synthesis duration configured for the synthesis operation is 13.1ms. In this way, in response to the Vsync-APP signal 11, frame 1 is drawn and rendered; in response to the Vsync-SF signal 22, a synthesis operation is performed on frame 1; and in response to the Vsync-HW signal 33, a display operation is performed on the synthesized frame 1. Among them, the interval between the Vsync-APP signal 11 and the Vsync-SF signal 22 meets the drawing duration of 19.2ms corresponding to the hands-off operation scenario, and the interval between the Vsync-SF signal 22 and the Vsync-HW signal 33 meets the synthesis duration of 13.1ms corresponding to the hands-off operation scenario.

[0237] The offset of the Vsync-APP signal and / or the offset of the Vsync-SF signal will cause the corresponding configuration duration to change, thereby modifying the drawing duration and the synthesis duration.

[0238] Assuming that a Down event sent by the Input module is received at point M between the Vsync-APP signal 12 and the Vsync-APP signal 13, when the Vsync-SF signal 23 is received, the synthesis module adjusts the sending rhythm of the Vsync-APP signal and the Vsync-SF signal in the subsequent Vsync cycle based on the configuration information corresponding to the hands-free sliding operation scenario. In other words, the synthesis module adjusts the starting time of the subsequent Vsync-APP signal and the Vsync-SF signal based on the configuration information corresponding to the hands-free sliding operation scenario, and periodically sends the Vsync-APP signal and the Vsync-SF signal at the re-determined starting time. Fig.13As shown, the offset of the Vsync-APP signal 14 after the Vsync-SF signal 23 is -2ms, and the offset of the Vsync-SF signal 24 is -2ms. Since the offset of the Vsync-APP signal changes before and after the adjustment, the interval between the first Vsync-APP signal (Vsync-APP signal 14) after the offset adjustment and the previous Vsync-APP signal (Vsync-APP signal 13) is no longer a Vsync cycle (11.1ms). Fig.13 , the interval between the Vsync-APP signal 13 and the Vsync-APP signal 14 (i.e., the first duration) is equal to the sum of the interval between the Vsync-SF signal 23 and the Vsync-SF signal 24 and the interval between the Vsync-APP signal 13 and the Vsync-SF signal 23, that is, the interval between the Vsync-APP signal 13 and the Vsync-APP signal 14 is equal to 19.2ms. In other words, the first duration (19.2ms) of the interval between the Vsync-APP signal 14 and the Vsync-APP signal 13 is different from the second duration (11.1ms) of the interval between the Vsync-APP signal 14 and the Vsync-APP signal 15. Correspondingly, since the offset of the Vsync-SF signal does not change before and after the adjustment, the third duration (11.1ms) of the interval between the Vsync-SF signal 24 and the Vsync-SF signal 23 after the offset is adjusted is the same as the fourth duration (11.1ms) of the interval between the Vsync-SF signal 24 and the Vsync-SF signal 25.

[0239] Next, the application performs drawing and rendering operations based on the drawing duration corresponding to the hands-free sliding operation scenario, and the synthesis module performs synthesis and display operations based on the synthesis duration corresponding to the hands-free sliding operation scenario. For example, in response to the Vsync-APP signal 14, draw and render frame 4; in response to the Vsync-SF signal 25, perform a synthesis operation on frame 4; in response to the Vsync-HW signal 36, perform a display operation on the synthesized frame 4. Among them, by offsetting the Vsync-APP signal, the interval between the Vsync-APP signal 14 and the Vsync-SF signal 25 meets the drawing duration of 11.1ms corresponding to the hands-free operation scenario, and the interval between the Vsync-SF signal 25 and the Vsync-HW signal 36 meets the synthesis duration of 13.1ms corresponding to the hands-free operation scenario.

[0240] Combination Fig.13It can be seen that before receiving the user's hand-free sliding operation, it takes about 3 Vsync cycles to output frame 1 to the display module for display. After receiving the user's hand-free sliding operation, frame 4 can be output to the display module for display after about 2 Vsync cycles, so that the image can be refreshed to the display screen one Vsync cycle in advance, thereby improving the hand tracking performance in the hand-free sliding operation scenario.

[0241] It should be noted that the above embodiment is only an example of adjusting the Vsync-APP signal offset in the scenario of sliding operation without leaving the hand, and does not limit the offset adjustment method in different operation scenarios. For example, in different operation scenarios, the Vsync-APP signal offset and the Vsync-SF signal offset are different. For another example, in different operation scenarios, the Vsync-APP signal offset is the same, and the Vsync-SF signal offset is different.

[0242] The embodiments of the present application do not limit the method of configuring the drawing duration for the drawing and rendering operation and the method of configuring the synthesis duration for the synthesis operation in the hands-free operation scenario.

[0243] In one implementable manner, a historical drawing time for executing the drawing and rendering operation and a historical synthesis time for executing the synthesis operation are obtained; then, based on the historical drawing time and the historical synthesis time, a drawing time configured for the drawing and rendering operation and a synthesis time configured for the synthesis operation are determined; wherein the drawing time configured for the drawing and rendering operation is greater than or equal to a maximum drawing time in the historical drawing time, and the synthesis time configured for the synthesis operation is greater than or equal to a maximum synthesis time in the historical synthesis time.

[0244] First of all, it should be noted that the historical drawing duration refers to the actual duration consumed for each drawing and rendering operation, and the historical synthesis duration refers to the actual duration consumed for each synthesis operation.

[0245] Among them, all the historical drawing durations counted include the upper limit of the historical drawing duration (i.e., the maximum drawing duration) and the lower limit of the historical drawing duration (i.e., the minimum drawing duration). Similarly, all the historical synthesis durations counted include the upper limit of the historical synthesis duration (i.e., the maximum synthesis duration) and the lower limit of the historical synthesis duration (i.e., the minimum synthesis duration). However, the upper limit of the historical drawing duration and the upper limit of the historical synthesis duration may include abnormal points. Therefore, after counting the historical drawing duration and the historical synthesis duration, the abnormal points can be firstly eliminated using the abnormal point elimination algorithm, and then the upper limit of the historical drawing duration and the upper limit of the historical synthesis duration can be determined from the remaining normal points.

[0246] For example, Fig.14The historical drawing duration of the rendering operation performed on frames 1 to 40 and the historical synthesis duration of the synthesis operation performed on frames 1 to 40 are shown. Fig.14 As shown in the figure, the upper limit of the historical drawing duration is 13.0ms ( Fig.13 Point C in the middle), the lower limit of the historical drawing duration is 8.0ms ( Fig.13 The upper limit of the historical synthesis duration is 8.5ms ( Fig.13 The lower limit of the historical synthesis duration is 8.0ms ( Fig.13 point B2).

[0247] However, from Fig.14 It can be seen that the historical drawing time basically fluctuates around 8.0ms, and point C is obviously an abnormal point. Therefore, we can first eliminate point C, and then determine the upper limit of the historical drawing time to be 8.2ms among the remaining normal points ( Fig.13 point A1 in the middle).

[0248] Based on the above statistical results, it can be determined that the drawing time configured for the drawing rendering operation should be greater than 8.2ms, and the synthesis time configured for the synthesis operation should be greater than 8.5ms. For example, the drawing time configured for the hand-free sliding operation scenario can be 11.1ms, and the synthesis time configured for the hand-free sliding operation scenario can be 13.1ms. In this way, the total configuration time for the hand-free sliding operation scenario is 24.2ms.

[0249] The offset of the Vsync-APP signal and the Vsync-SF signal may be set so that the drawing duration configured for the drawing and rendering operation is 11.1 ms and the synthesis duration configured for the synthesis operation is 13.1 ms.

[0250] Exemplarily, taking the Vsync period of 11.1 ms as an example, the offset of the Vsync-APP signal is set to -2 ms, and the offset of the Vsync-SF signal is set to -2 ms.

[0251] In this way, T can be calculated according to the above first relationship: app =11.1+(-2)-(-2)=11.1ms. According to the second relational expression above, T sf =11.1-(-2)=13.1ms.

[0252] It should be noted that, in the embodiments of the present application, app It can be less than 11.1ms. That is to say, in the embodiment of the present application, after calculating T app If the calculated T appAdd a T on top of that te . In order to achieve the purpose of shortening the configuration time in the sliding operation field without letting go of the hand.

[0253] It should be noted that the embodiment of the present application does not limit the form of the configuration information. For example, the configuration information may include the drawing duration configured for the drawing and rendering operation and the synthesis duration configured for the synthesis operation (for example, the configuration information is T app =11.1ms, T sf =13.1ms). For another example, the configuration information may also include an offset of the Vsync-APP signal and an offset of the Vsync-SF signal (e.g., the configuration information is appOffset=-2ms, sfOffset=-2ms). For another example, the configuration information may also include a drawing duration configured for a drawing rendering operation, a composition duration configured for a composition operation, an offset of the Vsync-APP signal, and an offset of the Vsync-SF signal.

[0254] It should also be noted that, in the configuration information of different touch operation scenarios, at least one of the values ​​of the drawing duration configured for the drawing and rendering operation and the synthesis duration configured for the synthesis operation is different. Similarly, in the configuration information of different touch operation scenarios, at least one of the values ​​of the offset of the Vsync-APP signal and the offset of the Vsync-SF signal is different.

[0255] In addition, since the drawing duration configured for the drawing and rendering operation and the synthesis duration configured for the synthesis operation are determined based on historical statistical data, and the image processing situation of the electronic device changes in real time, if the drawing, rendering, synthesis and display operations are always performed based on the same configuration duration, it may happen that the image drawing and rendering cannot be completed within the configured drawing duration, or it may happen that the image synthesis cannot be completed within the configured synthesis duration. To avoid this situation, the embodiment of the present application can regularly detect the actual situation of performing the drawing, rendering, synthesis and display operations based on the current configuration duration.

[0256] Exemplarily, the number of times the drawing and rendering operation cannot be completed within the currently configured drawing time under the drawing time currently configured for the drawing and rendering operation, and the number of times the synthesis operation cannot be completed within the configured drawing time under the synthesis time currently configured for the synthesis operation can be determined at every preset time interval; when the number of times the corresponding operation cannot be completed within the corresponding configured time reaches a preset number threshold, the values ​​of the drawing time configured for the drawing and rendering operation and the synthesis time configured for the synthesis operation are re-determined. Then, based on the re-configured drawing time and synthesis time, the drawing, rendering, synthesis and display operations are performed.

[0257] To further improve the user experience, the embodiments of the present application may further subdivide the hands-free sliding operation scenarios so that different hands-free sliding operation scenarios correspond to different configuration durations.

[0258] In one possible implementation, different hand-free sliding operations may be divided into following chirality levels. Thus, after receiving the user's hand-free sliding operation, the following chirality level corresponding to the hand-free sliding operation may be determined first, and then the configured drawing duration and synthesis duration may be determined based on the following chirality level, wherein different following chirality levels correspond to different configured durations.

[0259] Exemplarily, the hand-free sliding operation can be specifically divided into a finger-free sliding operation scenario and a stylus-free sliding operation scenario. The finger-free sliding operation scenario refers to a scenario in which the finger performs a hand-free sliding operation on the display screen, and the stylus-free sliding operation scenario refers to a scenario in which the stylus performs a hand-free sliding operation on the display screen. Among them, the chirality requirement for the stylus to perform a hand-free sliding operation on the display screen is higher than the chirality requirement for the finger to perform a hand-free sliding operation on the display screen. In this way, the embodiments of the present application can classify the specific scenarios of the hand-free sliding operation into levels of chirality requirements. For example, the chirality requirement level for the hand-free sliding operation of the stylus on the display screen is level one, and the chirality requirement level for the finger-free sliding operation on the display screen is level two, wherein the level one chirality requirement level is higher than the level two chirality requirement level.

[0260] Further, different levels of chirality requirements correspond to different configuration durations, for example, the third configuration duration corresponding to the stylus-on-hand operation scenario is shorter than the third configuration duration corresponding to the finger-on-hand operation scenario, and / or the fourth configuration duration corresponding to the stylus-on-hand operation scenario is shorter than the fourth configuration duration corresponding to the finger-on-hand operation scenario. For example, the total configuration duration of the third configuration duration and the fourth configuration duration corresponding to the hand-on-hand sliding operation scenario with a level of chirality requirement of one is 11.1ms. The total configuration duration of the third configuration duration and the fourth configuration duration corresponding to the hand-on-hand sliding operation scenario with a level of chirality requirement of two is 24.2ms. In this way, when it is determined that the hand-on-hand sliding operation is a finger-on-hand sliding operation on the display screen, the image drawing, rendering, synthesis and display sending operations can be completed within 24.2ms. Correspondingly, when it is determined that the hand-on-hand sliding operation is a finger-on-hand sliding operation on the display screen, the image drawing, rendering, synthesis and display sending operations can be completed within 11.1ms. In this way, compared with the operation scenario where the finger does not leave the hand, the stylus does not leave the hand operation scenario can further advance one Vsync cycle to refresh the image to the display screen. This can better adapt to the tracking requirements in different operation scenarios where the finger does not leave the hand, and improve the user experience.

[0261] The present application does not limit the method for determining whether the hand-free sliding operation is triggered by a finger or a stylus. For example, the touch screen temperature corresponding to the sliding track during the hand-free sliding operation can be detected. When the touch screen temperature is higher than the temperature threshold, the current hand-free sliding operation is determined to be a finger-triggered sliding operation; when the touch screen temperature is lower than or equal to the temperature threshold, the current hand-free sliding operation is determined to be a stylus-triggered sliding operation.

[0262] It should be noted that the present application embodiment only uses the object that triggers the hand-free sliding operation as a finger or a stylus to divide the chirality requirement levels for exemplary explanation, and does not limit the division of chirality requirement levels. For example, the chirality requirement levels can also be divided according to the interface attributes of the hand-free sliding operation. For example, the chirality requirement for the interface of a game application interface is level one, and the chirality requirement for the interface of a video application interface is level two.

[0263] In summary, the display method provided in the first embodiment of the present application performs drawing rendering, synthesis and display sending operations based on the configuration duration corresponding to the hands-off operation scenario in the case of a hands-off operation scenario; and performs drawing rendering, synthesis and display sending operations based on the configuration duration corresponding to the hands-on operation scenario in the case of a hands-on operation scenario. In this way, by configuring different durations for different touch operation scenarios, different requirements in different application scenarios can be better adapted. Specifically, in the scenario of hands-on sliding operation, drawing rendering, synthesis and display sending operations are performed based on a relatively short configuration duration, which can reduce touch delay, improve hand tracking in the hands-on sliding operation scenario, and improve user experience.

[0264] It should be noted that when the electronic device performs a frame rate switching operation or uses the GPU to perform a synthesis operation, the load on the electronic device is large. If in this case, based on the configuration duration corresponding to the non-hands sliding operation scenario, the drawing rendering, synthesis and display sending operations are performed, the application or synthesis module may not be able to complete their respective tasks on time. Therefore, before performing drawing rendering, synthesis and display sending operations based on the configuration duration corresponding to the non-hands sliding operation scenario, the present application can first determine whether a frame rate switching operation is being performed at the current moment; if the frame rate switching operation is not performed at the current moment, determine whether the GPU is being used to perform the synthesis operation at the current moment; if the GPU is not used to perform the synthesis operation at the current moment, in response to the non-hands sliding operation, the drawing rendering, synthesis and display sending operations are performed based on the configuration duration corresponding to the non-hands sliding operation scenario.

[0265] Embodiment 2

[0266] The display method provided in the second embodiment of the present application reduces the third part delay T3 by optimizing the number of cache frames accumulated in the cache queue, thereby improving the hand tracking performance.

[0267] Fig.15 A flow chart of a display control method provided in Embodiment 2 of the present application is shown in FIG. Fig.15 As shown, the display method provided in the first embodiment of the present application may include the following steps:

[0268] Step S301, receiving a second touch operation input by a user;

[0269] Step S302, if the touch scene of the second touch operation is a hands-on operation scene, and the number of cached frames in the cache queue for caching rendered image frames is multiple, determine whether the cached frames in the cache queue meet the frame loss condition.

[0270] In an embodiment of the present application, the cache queue refers to the cache queue of the synthesis module, that is, the cache queue used to cache the image frames after drawing and rendering. For example, the cache queue includes a first cache frame, a second cache frame and a third cache frame, wherein the first cache frame is the image frame at the first position in the cache queue.

[0271] like Figure 5 As shown, in the first Vsync cycle, the cache queue includes two cache frames, namely frame 2 and frame 3 after drawing and rendering, wherein frame 2 is ranked first in the cache queue and frame 3 is ranked after frame 2. Therefore, in the first Vsync cycle, frame 2 is the first cache frame in the cache queue and frame 3 is the second cache frame.

[0272] In the embodiment of the present application, frame loss means that when there are cache frame buffers piled up in the cache queue, the first cache frame can be discarded at an appropriate time and the second cache frame can be used to participate in the synthesis operation, so as to achieve the purpose of displaying the image on the display screen as soon as possible.

[0273] In one implementable manner, the embodiment of the present application may first determine whether the first cache frame satisfies the frame loss condition; if the first cache frame satisfies the frame loss condition, then determine whether to use the first cache frame or the second cache frame to participate in the synthesis operation. If it is determined that the second cache frame is used to participate in the synthesis operation, the first cache frame is discarded. The embodiment of the present application does not limit the frame loss method. For example, the frame determined not to be displayed may be deleted, or the frame determined not to be displayed may be overwritten.

[0274] Among them, the frame loss conditions may include that the timestamp attribute of the first cached frame is an automatic timestamp, the current refresh rate is greater than 60Hz, and the time interval between the last time an image frame in the cache queue was discarded is greater than a preset time interval threshold, wherein the first timestamp attribute is used to characterize the first preset display time of the first cached frame.

[0275] The timestamp attribute of the cached frame can be divided into automatic timestamp and manual timestamp. Manual timestamp refers to the display time of the specified cached frame, that is, the display time of the cached frame cannot be changed. For example, the timestamp attribute of the cached frame corresponding to the video content is generally a manual timestamp, that is, the display time of each frame in the video cannot be changed at will. For example, specify that the first cached frame is displayed at the 1st second, and specify that the second cached frame is displayed at the 5th second. Automatic timestamp refers to a timestamp that is automatically generated according to the queue order without special specification.

[0276] According to human factor analysis, the frame loss duration that users can feel as a freeze is 25-58ms, which means: when the refresh rate is 60Hz, if a frame is lost, the frame spacing increases to 33.2ms, which will make the user feel the freeze; when the refresh rate is 90Hz, if a frame is lost, the frame spacing increases to 22.2ms, which will not make the user feel the freeze; when the refresh rate is 120Hz, if a frame is lost, the frame spacing increases to 16.6ms, which will not make the user feel the freeze. Therefore, in order to avoid the phenomenon of freeze, the embodiment of the present application sets a refresh rate greater than 60Hz as one of the frame loss conditions. For example, the frame loss scheme can be executed when the refresh rate is 90Hz or 120Hz.

[0277] Combined with the above analysis, it can be seen that if two image frames are discarded continuously in a relatively short period of time, obvious jamming will occur. Therefore, in the embodiment of the present application, the time interval from the last time an image frame in the cache queue was discarded is greater than a preset time interval threshold as one of the frame discard conditions. For example, the time interval from the last time an image frame in the cache queue was discarded must be greater than 100ms, that is, within 100ms, two image frames cannot be discarded continuously.

[0278] The frame loss condition may also include that the first cached frame belongs to an image frame of a whitelist application. For example, the whitelist application includes a video application and a game application. Thus, if the first cached frame belongs to an image frame in a video application or a game application, it is determined that the first cached frame belongs to an image frame of a whitelist application.

[0279] It should be noted that the embodiment of the present application does not limit the order of judging the frame loss conditions such as the timestamp attribute of the first cached frame, whether the current refresh rate is greater than 60Hz, and whether the time interval from the last time an image frame in the cache queue was discarded is greater than a preset time interval threshold.

[0280] Exemplarily, it is possible to first determine whether the timestamp attribute of the first cached frame is an automatic timestamp. If the timestamp attribute of the first cached frame is an automatic timestamp, it is further determined whether the current refresh rate is greater than 60 Hz. If the current refresh rate is greater than 60 Hz, it is possible to further determine whether the time interval from the last time the image frame in the cache queue was discarded is greater than a preset time interval threshold. If the time interval from the last time the image frame in the cache queue was discarded is greater than the preset time interval threshold, it is possible to further determine whether the first cached frame belongs to an image frame of a whitelist application. If the first cached frame belongs to an image frame of a whitelist application, the subsequent step S303 can be executed.

[0281] Step S303: when the first cached frame in the cache queue meets the frame loss condition, determine the cache state of the first cached frame.

[0282] The embodiment of the present application divides the cache states of the cached frames into three types, namely early, current and late.

[0283] In one implementable manner, the cache status of the first cached frame can be determined in the following manner: determine the first expected display time (also referred to as the first preset display time) and the second expected display time (also referred to as the second preset display time) corresponding to the first cached frame. The first preset display time is a preset time for performing a display operation on the first cached frame, and the second preset display time is a preset display time corresponding to the Vsync-SF signal. It can also be understood that the first preset display time is the expected display time corresponding to the Vsync-APP signal used to indicate the execution of a drawing and rendering operation on the first cached frame, and the second preset display time is the expected display time corresponding to the current Vsync-SF signal. between If the first preset display time is earlier than the second preset display time, the cache state of the first cache frame is determined to be late. If the first preset display time is later than the second preset display time, the cache state of the first cache frame is determined to be early. If the first preset display time is the same as the second preset display time, the cache state of the first cache frame is determined to be current.

[0284] First, the first preset display time and the second preset display time are explained as follows.

[0285] like Fig.16 As shown, each Vsync-APP signal corresponds to a first preset display time, and each Vsync-SF signal also corresponds to a second preset display time. Fig.16The first preset display time corresponding to the Vsync-APP signal 11 is T2, the first preset display time corresponding to the Vsync-APP signal 12 is T3, and the first preset display time corresponding to the Vsync-APP signal 13 is T4. Similarly, Fig.16 The second preset display sending time corresponding to the Vsync-SF signal 23 is T3, the second preset display sending time corresponding to the Vsync-SF signal 24 is T4, and the second preset display sending time corresponding to the Vsync-SF signal 25 is T5.

[0286] That is to say, the first preset display time corresponding to the Vsync-APP signal refers to the preset display time for drawing and rendering the image frame under the guidance of the Vsync-APP signal. The second preset display time corresponding to the Vsync-SF signal refers to the preset display time for synthesizing the image frame under the guidance of the Vsync-SF signal.

[0287] In the embodiment of the present application, the preset time for performing the display sending operation on the first cache frame is called the first preset display sending time, the preset time for performing the display sending operation on the second cache frame is called the third preset display sending time, and the preset time for performing the display sending operation on the third cache frame is called the fourth preset display sending time.

[0288] In one possible implementation, the rendered image frame drawn in response to the Vsync-APP signal may carry a timestamp, which is used to indicate the first preset display time corresponding to the image frame. Fig.16 As shown, the timestamp of drawing the rendered frame 0 in response to the Vsync-APP signal 11 is T2, the timestamp of drawing the rendered frame 1 in response to the Vsync-APP signal 12 is T3, and the timestamp of drawing the rendered frame 2 in response to the Vsync-APP signal 13 is T4.

[0289] like Fig.16 As shown, taking the current Vsync-SF signal as the Vsync-SF signal 23 as an example, determining the cache state of the cached frame in the cache queue is described.

[0290] When the Vsync-SF signal 22 is received, the application has not yet completed the drawing and rendering of frame 0. Therefore, in response to the Vsync-SF signal 22, the synthesis module cannot obtain the drawn and rendered frame 0, and the cache queue is empty. When the Vsync-SF signal 23 arrives, the application completes the drawing and rendering of frames 0, 1, and 2. In this way, when the Vsync-SF signal 23 is received, the cache queue includes three cache frames, frame 0, frame 1, and frame 2. Among them, frame 0 is the first cache frame, frame 1 is the second cache frame, and frame 2 is the third cache frame. Among them, the second preset display time of frame 0, frame 1, and frame 2 is T3, the first preset display time corresponding to frame 0 is T2, the third preset display time corresponding to frame 1 is T3, and the fourth preset display time corresponding to frame 2 is T4. In this way, the first preset display time T2 corresponding to frame 0 is earlier than the second preset display time T3, so the cache state of frame 0 is late; the third preset display time T3 corresponding to frame 1 is the same as the second preset display time T3, so the cache state of frame 1 is current; the fourth preset display time T4 corresponding to frame 2 is later than the second preset display time T3, so the cache state of frame 2 is early.

[0291] The cache state of the first cache frame may include the following situations:

[0292] Case 1: The cache state of the first cached frame is late.

[0293] When the cache state of the first cache frame is late, the following steps S304 to S306 may be performed.

[0294] Step S304: determine the cache status of the second cache frame.

[0295] The method for determining the cache state of the second cache frame may refer to the above description of step S303, which will not be described again here.

[0296] Step S305 , when the cache state of the second cache frame is late or current, determine to discard the first cache frame, and perform a synthesis operation on the second cache frame in the current Vsync cycle.

[0297] For example, Fig.16 As shown, in the third Vsync cycle, in response to the Vsync-SF signal 23, the late state frame 0 can be discarded, and frame 1 can be used to participate in this synthesis operation. In this way, frame 1 can be displayed on the display screen one Vsync cycle in advance.

[0298] Step S306 : when the cache state of the second cache frame is early, determine to perform a synthesis operation on the first cache frame in the current Vsync cycle.

[0299] The cached frame in the early state indicates that the frame is rendered one Vsync cycle in advance, so it can wait until the next Vsync cycle to participate in synthesis.

[0300] It should be noted that when the electronic device is actually running, the Vsync-APP signal or the second Vyscn signal may be disordered. For example, when switching the frame rate, the electronic device needs to recalibrate the Vsync cycle. In this way, the Vsync-APP signal or the Vyscn-SF signal may be received in advance. If the cache state of the cache frame is still determined according to the above method when the Vsync-APP signal or the Vyscn-SF signal is received in advance, it may cause the subsequent cache frames to accumulate. For example, it is possible that the cache frame that should have been in the current state is mistakenly judged as the early state. In other words, the cache frame that should have been taken out for consumption and synthesis processing in the current Vsync cycle will be mistakenly judged to be taken out for consumption and synthesis processing in the next Vsync cycle, which will cause the subsequent cache frames to accumulate.

[0301] For example, Fig.17 As shown, the frame rate corresponding to the first Vsync cycle is 90Hz, at which the Vsync cycle is 11.1ms, the interval between the Vsync-APP signal 11 and the Vsync-APP signal 12 is 11.1ms, and the interval between the Vsync-SF signal 21 and the Vsync-SF signal 22 is 11.1ms. In the second Vsync cycle, since the frame rate is switched from 90Hz to 120Hz, the Vsync-SF signal 23 is received in advance, so that the interval between the Vsync-SF signal 22 and the Vsync-SF signal 23 is less than 11.1ms.

[0302] If the rendering of frame 0 is completed before the Vsync-SF signal 23, the first preset display time of frame 0 is still T2 corresponding to the 90Hz frame rate, and the second preset display time of frame 0 becomes TA corresponding to the 120Hz frame rate, and TA is between T1 and T2. In this way, since the first preset display time T2 of frame 0 is later than the second preset display time TA of frame 0, the cache state of frame 0 is judged to be early. In fact, the cache state of frame 0 should be current.

[0303] Based on the above analysis, in the case where the cache state of the second cache frame is early, the embodiment of the present application can also adopt the following solution: the synthesis module first determines whether the Vsync-APP signal or Vsync-SF signal is received in advance; in the current Vsync cycle, if the Vsync-APP signal or Vsync-SF signal is received in advance, the first cache frame is discarded, and it is determined that the second cache frame is used to synthesize the image frame in the current Vsync cycle. In this way, the accumulation of cache frames due to misjudgment of the cache state can be avoided.

[0304] Case 2: The cache state of the first cache frame is current.

[0305] When the cache state of the first cache frame is current, the following step S307 may be performed.

[0306] Step S307 , determining to perform a synthesis operation on the first cache frame in the current Vsync period.

[0307] If the cache state of the first cache frame is current, then the cache state of the cache frame following the first cache frame should be early. Therefore, when it is determined that the cache state of the first cache frame is current, it can be directly determined to use the first cache frame to synthesize the image frame without continuing to judge the cache state of the subsequent cache frames.

[0308] Case 3: The cache state of the first cached frame is early.

[0309] When the cache state of the first cache frame is early, the following step S308 may be performed.

[0310] Step S308: determine to perform a synthesis operation on the first buffered frame in the next Vsync cycle.

[0311] If the cache state of the first cache frame is early, then the cache state of the cache frame following the first cache frame should be early. Therefore, when it is determined that the cache state of the first cache frame is early, it can be directly determined that the first cache frame is used to synthesize the image frame in the next Vsync cycle, and there is no need to continue to determine the cache state of the subsequent cache frames. In other words, when it is determined that the cache state of the first cache frame is early, the Vsync-SF signal of this Vsync cycle can be idle, and no synthesis operation is performed on any cache frame.

[0312] Similarly, in order to avoid the problem of cache frame accumulation due to the erroneous judgment of the cache frame that should be in the current state as the early state. When the cache state of the first cache frame is early, the following solution can also be adopted: first determine whether the Vsync-APP signal or the Vsync-SF signal is received in advance in the current Vsync cycle; in the case where the Vsync-APP signal or the Vsync-SF signal is received in advance in the current Vsync cycle, determine to use the first cache frame to synthesize the image frame in the current Vsync cycle. In this way, the problem of cache frame accumulation due to misjudgment of the cache state can be avoided.

[0313] It should be noted that the embodiment of the present application can use the above method to determine whether each cached frame in the cache queue meets the frame loss condition and the cache status of the cached frame. For example, the order of the cached frame ranked first in the cache queue to the last cache can be traversed to perform the step of determining whether each cached frame in the cache queue meets the frame loss condition and the cache status of the cached frame.

[0314] In summary, the display method provided in the second embodiment of the present application determines the timing of frame drop based on the frame drop condition and the cache status of the cache frame in the hands-free operation scenario, so as to output and display the image as quickly as possible and improve hand tracking.

[0315] The various method embodiments described in this document may be independent solutions or may be combined according to internal logic, and these solutions all fall within the protection scope of this application.

[0316] The above describes in detail the method provided by the embodiment of the present application. Fig.18 and Fig.19 The device provided in the embodiment of the present application is described in detail. It should be understood that the description of the device embodiment corresponds to the description of the method embodiment, so the contents not described in detail can be referred to the method embodiment above, and for the sake of brevity, they will not be repeated here.

[0317] The above embodiments introduce the display method provided by the present application. It is understandable that, in order to realize the above functions, the display device includes a hardware structure and / or software module corresponding to each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0318] The embodiment of the present application can divide the functional modules of the display device according to the above method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0319] Fig.18 is a schematic diagram of a display device provided in an embodiment of the present application. In one embodiment, the electronic device can Fig.18 The software device shown in the figure realizes the corresponding functions. Fig.18 As shown, the display device may include:

[0320] A first receiving module 401 is used to receive a first touch operation input by a user;

[0321] A first drawing and rendering module 402 is used for, when the touch scene of the first touch operation is a hand-off operation scene, responding to a received first application vertical synchronization signal, performing a drawing and rendering operation on a first image frame within a first configured time length in the hand-off operation scene;

[0322] A first synthesis module 403 is configured to perform a synthesis operation on the first image frame after drawing and rendering within a second configuration time length in the hands-off operation scenario in response to the received first synthesis vertical synchronization signal; wherein the first synthesis vertical synchronization signal and the first application vertical synchronization signal are separated by the first configuration time length;

[0323] The first display module 404 is configured to perform a display operation on the synthesized first image frame in response to a received first hardware vertical synchronization signal; wherein the first hardware vertical synchronization signal and the first synthesized vertical synchronization signal are separated by the second configuration time length;

[0324] The second receiving module 405 is used to receive a second touch operation input by a user;

[0325] A second drawing and rendering module 406 is configured to, when the touch scenario of the second touch operation is a hands-on operation scenario, perform a drawing and rendering operation on the second image frame within a third configured time length in the hands-on operation scenario in response to a received second application vertical synchronization signal;

[0326] A second synthesis module 407 is configured to perform a synthesis operation on the second image frame after drawing and rendering within a fourth configuration time length in the hands-free operation scenario in response to the received second synthesis vertical synchronization signal; wherein the second synthesis vertical synchronization signal and the second application vertical synchronization signal are separated by the third configuration time length;

[0327] The second display module 408 is used to perform a display operation on the synthesized second image frame in response to a received second hardware vertical synchronization signal; the second hardware vertical synchronization signal and the second synthesized vertical synchronization signal are separated by the fourth configuration time length; wherein the third configuration time length is less than the first configuration time length, and / or the fourth configuration time length is less than the second configuration time length.

[0328] In a possible implementation, the hands-on operation scenario includes a hands-on sliding operation scenario and a hands-on pressing operation scenario.

[0329] A possible implementation also includes:

[0330] A first determining module, configured to determine, when the touch scenario of the first touch operation is a hand-off operation scenario, first configuration information corresponding to the hand-off operation scenario, wherein the first configuration information includes the first configuration duration and the second configuration duration;

[0331] The second determination module is used to determine second configuration information corresponding to the hands-free operation scenario when the touch scenario of the second touch operation is a hands-free operation scenario, wherein the second configuration information includes the third configuration duration and the fourth configuration duration.

[0332] In a possible implementation, the second application vertical synchronization signal and the third application vertical synchronization signal are spaced apart by a first time length, the second application vertical synchronization signal and the fourth application vertical synchronization signal are spaced apart by a second time length, the third application vertical synchronization signal is the previous application vertical synchronization signal of the second application vertical synchronization signal, and the fourth application vertical synchronization signal is the next application vertical synchronization signal of the second application vertical synchronization signal;

[0333] The second synthesized vertical synchronization signal and the third synthesized vertical synchronization signal are separated by a third time length, the second synthesized vertical synchronization signal and the fourth synthesized vertical synchronization signal are separated by a fourth time length, the third synthesized vertical synchronization signal is the previous synthesized vertical synchronization signal of the second synthesized vertical synchronization signal, and the fourth synthesized vertical synchronization signal is the next synthesized vertical synchronization signal of the second synthesized vertical synchronization signal;

[0334] The first duration is different from the second duration, and / or the third duration is different from the fourth duration.

[0335] A possible implementation also includes:

[0336] A first generating module, configured to generate the second application vertical synchronization signal based on the second hardware vertical synchronization signal and the first signal offset in the hands-free operation scenario;

[0337] The second generating module is used to generate the second synthetic vertical synchronization signal based on the second hardware vertical synchronization signal and the second signal offset in the hands-free operation scenario.

[0338] A possible implementation also includes:

[0339] An acquisition module, used for acquiring a historical drawing time for executing the drawing and rendering operation, and a historical synthesis time for executing the synthesis operation;

[0340] A third determining module is used to determine the third configuration duration and the fourth configuration duration according to the historical drawing duration and the historical synthesis duration; the third configuration duration is greater than or equal to the maximum drawing duration in the historical drawing durations, and the fourth configuration duration is greater than or equal to the maximum synthesis duration in the historical synthesis durations.

[0341] A possible implementation also includes:

[0342] A fourth determination module is used to determine a chirality level corresponding to the hands-on operation scenario;

[0343] The fifth determining module is configured to determine the third configuration duration and the fourth configuration duration based on the chirality level, wherein different chirality levels correspond to different configuration durations.

[0344] In a possible implementation, the hand-on-hold operation scenario includes a finger-on-hold operation scenario and a stylus-on-hold operation scenario, and the stylus-on-hold operation scenario corresponds to a higher chirality level than the finger-on-hold operation scenario.

[0345] Among them, the third configuration time length corresponding to the stylus pen never leaves hand operation scenario is shorter than the third configuration time length corresponding to the finger never leaves hand operation scenario, and / or the fourth configuration time length corresponding to the stylus pen never leaves hand operation scenario is shorter than the fourth configuration time length corresponding to the finger never leaves hand operation scenario.

[0346] A possible implementation also includes:

[0347] a sixth determination module, configured to determine, when there are multiple cached frames in a cache queue for caching rendered image frames and a first cached frame at the top of the cache queue meets a frame loss condition, a first preset display sending time and a second preset display sending time corresponding to the first cached frame; the first preset display sending time is a preset time for performing a display sending operation on the first cached frame, and the second preset display sending time is a preset display sending time corresponding to a second synthetic vertical synchronization signal;

[0348] a seventh determination module, configured to determine that the second cache frame corresponds to a third preset display sending time when the first preset display sending time is earlier than the second preset display sending time; the third preset display sending time is a preset time for performing a display sending operation on the second cache frame, and the second cache frame is a cache frame in the cache queue that is located after the first cache frame;

[0349] The second synthesis module is used to discard the first cache frame and perform a synthesis operation on the second cache frame when the third preset display time is the same as the second preset display time, or when the third preset display time is earlier than the second preset display time.

[0350] In one possible implementation, the second synthesis module is further used to discard the first cached frame and perform a synthesis operation on the second cached frame when the third preset display time is later than the second preset display time and it is determined that the interval between the currently received second synthesized vertical synchronization signal and the previous second synthesized vertical synchronization signal is less than one Vsync cycle, or when the interval between the second application vertical synchronization signal used to trigger the drawing and rendering operation on the second cached frame and the previous second application vertical synchronization signal is less than one Vsync cycle.

[0351] In a possible implementation, the second synthesis module is further used to perform a synthesis operation on the first cache frame when the first preset display time is the same as the second preset display time, or when the third preset display time is later than the second preset display time.

[0352] In one possible implementation, the frame loss conditions include: the timestamp attribute corresponding to the first preset display time of the first cached frame is an automatic timestamp, the current refresh rate is greater than 60Hz, and the time interval from the last time the image frame in the cache queue was discarded is greater than a preset time interval threshold.

[0353] Fig.19 This is a schematic diagram of a display device provided in an embodiment of the present application. In one embodiment, the electronic device can Fig.19 The hardware devices shown in the figure implement the corresponding functions. Fig.19As shown, the device 1000 may include a transceiver 1001, a processor 1002 and a memory 1003. Among them, the transceiver 1001 may be used to receive a user's touch operation. The processor 1002 may include one or more processing units, for example: the processor 1002 may include an application processor, a modem processor, a graphics processor, an image signal processor, a controller, a video codec, a digital signal processor, a baseband processor, and / or a neural network processor, etc. Among them, different processing units may be independent devices or integrated in one or more processors. The memory 1003 is coupled to the processor 1002 for storing various software programs and / or multiple sets of instructions. The memory 1003 may include a volatile memory and / or a non-volatile memory.

[0354] The device 1000 can execute the operations performed in the above method embodiments.

[0355] For example, in an optional embodiment of the present application, the transceiver 1001 is used to: receive a first touch operation input by a user, and receive a second touch operation input by a user. The processor 1002 is used to: if the touch scenario of the first touch operation is a hands-off operation scenario, then in response to the received first application vertical synchronization signal, perform a drawing and rendering operation on the first image frame within a first configuration duration in the hands-off operation scenario; in response to the received first synthetic vertical synchronization signal, perform a synthesis operation on the first image frame after drawing and rendering within a second configuration duration in the hands-off operation scenario; wherein the first synthetic vertical synchronization signal and the first application vertical synchronization signal are separated by the first configuration duration; in response to the received first hardware vertical synchronization signal, perform a display operation on the synthesized first image frame; wherein the first hardware vertical synchronization signal and the first synthetic vertical synchronization signal are separated by the second configuration duration; if the touch scenario of the second touch operation is a hands-on operation scenario, then the processor 1002 is used to: if the touch scenario of the first touch operation is a hands-off operation scenario, then in response to the received first application vertical synchronization signal, perform a display operation on the first image frame after drawing and rendering within a second configuration duration in the hands-off operation scenario; wherein the first synthetic vertical synchronization signal and the first application vertical synchronization signal are separated by the second configuration duration; wherein the first hardware vertical synchronization signal and the first synthetic vertical synchronization signal are separated by the second configuration duration; wherein the second touch scenario of the second touch operation is a hands-on operation scenario, then the processor 1002 is used to: operation scenario, then in response to the received second application vertical synchronization signal, a drawing and rendering operation is performed on the second image frame within a third configuration time length in the hands-free operation scenario; in response to the received second synthetic vertical synchronization signal, a synthesis operation is performed on the second image frame after drawing and rendering within a fourth configuration time length in the hands-free operation scenario; wherein the second synthetic vertical synchronization signal and the second application vertical synchronization signal are separated by the third configuration time length; in response to the received second hardware vertical synchronization signal, a display operation is performed on the synthesized second image frame; the second hardware vertical synchronization signal and the second synthetic vertical synchronization signal are separated by the fourth configuration time length; wherein the third configuration time length is less than the first configuration time length, and / or the fourth configuration time length is less than the second configuration time length.

[0356] In a possible implementation, the hands-on operation scenario includes a hands-on sliding operation scenario and a hands-on pressing operation scenario.

[0357] In one possible implementation, the processor 1002 is further used to: if the touch scenario of the first touch operation is a hands-off operation scenario, determine the first configuration information corresponding to the hands-off operation scenario, the first configuration information including the first configuration duration and the second configuration duration; if the touch scenario of the second touch operation is a hands-on operation scenario, determine the second configuration information corresponding to the hands-on operation scenario, the second configuration information including the third configuration duration and the fourth configuration duration.

[0358] In a possible implementation, the second application vertical synchronization signal and the third application vertical synchronization signal are spaced apart by a first time length, the second application vertical synchronization signal and the fourth application vertical synchronization signal are spaced apart by a second time length, the third application vertical synchronization signal is the previous application vertical synchronization signal of the second application vertical synchronization signal, and the fourth application vertical synchronization signal is the next application vertical synchronization signal of the second application vertical synchronization signal; the second synthetic vertical synchronization signal and the third synthetic vertical synchronization signal are spaced apart by a third time length, the second synthetic vertical synchronization signal and the fourth synthetic vertical synchronization signal are spaced apart by a fourth time length, the third synthetic vertical synchronization signal is the previous synthetic vertical synchronization signal of the second synthetic vertical synchronization signal, and the fourth synthetic vertical synchronization signal is the next synthetic vertical synchronization signal of the second synthetic vertical synchronization signal; wherein, the first time length is different from the second time length, and / or the third time length is different from the fourth time length.

[0359] In one possible implementation, the processor 1002 is also used to: generate the second application vertical synchronization signal based on the second hardware vertical synchronization signal and the first signal offset in the hands-free operation scenario; generate the second synthetic vertical synchronization signal based on the second hardware vertical synchronization signal and the second signal offset in the hands-free operation scenario.

[0360] In a possible implementation, the processor 1002 is further used to: obtain a historical drawing time for executing the drawing and rendering operation, and a historical synthesis time for executing the synthesis operation; determine the third configuration time and the fourth configuration time according to the historical drawing time and the historical synthesis time; the third configuration time is greater than or equal to the maximum drawing time in the historical drawing time, and the fourth configuration time is greater than or equal to the maximum synthesis time in the historical synthesis time.

[0361] In one possible implementation, processor 1002 is further used to: determine a chirality level corresponding to the hands-free operation scenario; and determine the third configuration duration and the fourth configuration duration based on the chirality level, wherein different chirality levels correspond to different configuration durations.

[0362] In a possible implementation, the hand-not-leaving-hand operation scenario includes a finger-not-leaving-hand operation scenario and a stylus-not-leaving-hand operation scenario, and the stylus-not-leaving-hand operation scenario corresponds to a higher chirality level than the finger-not-leaving-hand operation scenario; wherein the third configuration time length corresponding to the stylus-not-leaving-hand operation scenario is shorter than the third configuration time length corresponding to the finger-not-leaving-hand operation scenario, and / or the fourth configuration time length corresponding to the stylus-not-leaving-hand operation scenario is shorter than the fourth configuration time length corresponding to the finger-not-leaving-hand operation scenario.

[0363] In a possible implementation, the processor 1002 is further used to: when there are multiple cached frames in a cache queue for caching rendered image frames, and the first cached frame at the top of the cache queue meets the frame loss condition, determine a first preset display sending time and a second preset display sending time corresponding to the first cached frame; the first preset display sending time is a preset time for performing a display sending operation on the first cached frame, and the second preset display sending time is a preset display sending time corresponding to a second synthetic vertical synchronization signal; when the first preset display sending time is earlier than the second preset display sending time, determine a third preset display sending time corresponding to the second cached frame; the third preset display sending time is a preset time for performing a display sending operation on the second cached frame, and the second cached frame is a cached frame in the cache queue that is located after the first cached frame; when the third preset display sending time is the same as the second preset display sending time, or when the third preset display sending time is earlier than the second preset display sending time, discard the first cached frame and perform a synthesis operation on the second cached frame.

[0364] In one possible implementation, the processor 1002 is further used to: when the third preset display time is later than the second preset display time, and it is determined that the interval between the currently received second synthetic vertical synchronization signal and the previous second synthetic vertical synchronization signal is less than one Vsync cycle, or when the interval between the second application vertical synchronization signal used to trigger the drawing and rendering operation on the second cache frame and the previous second application vertical synchronization signal is less than one Vsync cycle, discard the first cache frame and perform the synthesis operation on the second cache frame.

[0365] In a possible implementation, the processor 1002 is further used to: perform a synthesis operation on the first cache frame when the first preset display time is the same as the second preset display time, or when the third preset display time is later than the second preset display time.

[0366] In one possible implementation, the frame loss conditions include: the timestamp attribute corresponding to the first preset display time of the first cached frame is an automatic timestamp, the current refresh rate is greater than 60Hz, and the time interval from the last time the image frame in the cache queue was discarded is greater than a preset time interval threshold.

[0367] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0368] It should be noted that the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor can be combined and performed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0369] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM (DR RAM). It should be noted that the memory of the apparatus and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0370] According to the method provided in the embodiments of the present application, the embodiments of the present application also provide a computer program product, which includes: a computer program or instructions, when the computer program or instructions are run on a computer, the computer executes the method of any one of the embodiments of the method.

[0371] According to the method provided in the embodiments of the present application, the embodiments of the present application also provide a computer storage medium, which stores a computer program or instructions. When the computer program or instructions are run on a computer, the computer executes the method of any one of the embodiments of the method.

[0372] Those of ordinary skill in the art will appreciate that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0373] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0374] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0375] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0376] In addition, each functional module in each embodiment of the present application may be integrated into one processing unit, or each module may exist physically separately, or two or more modules may be integrated into one unit.

[0377] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0378] The devices, computer storage media, and computer program products provided in the above-mentioned embodiments of the present application are all used to execute the methods provided above. Therefore, the beneficial effects that can be achieved can refer to the corresponding beneficial effects of the methods provided above, and will not be repeated here.

[0379] It should be understood that in each embodiment of the present application, the execution order of each step should be determined by its function and internal logic, and the size of the sequence number of each step does not mean the order of execution and does not limit the implementation process of the embodiment.

[0380] Each part of this specification is described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the device, computer storage medium, and computer program product, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0381] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0382] The above-described embodiments of the present application do not constitute a limitation on the protection scope of the present application.

Claims

1. A display method, characterized in that: Applied to electronic equipment, the method comprises: Receiving a second touch operation input by the user; If the touch scenario of the second touch operation is a hands-on operation scenario, and there are multiple cached frames in the cache queue for caching the rendered image frames, and the first cache frame at the top of the cache queue meets the frame loss condition, determine a first preset display sending time and a second preset display sending time corresponding to the first cache frame, the frame loss condition includes that the time interval with the last time the image frame in the cache queue is discarded is greater than a preset time interval threshold; the first preset display sending time is a preset time for performing a display sending operation on the first cache frame, and the second preset display sending time is a preset display sending time corresponding to a second synthetic vertical synchronization Vsync-SF signal; In the case where the first preset display sending time is earlier than the second preset display sending time, determining that the second cache frame corresponds to a third preset display sending time; the third preset display sending time is a preset time for performing a display sending operation on the second cache frame, and the second cache frame is a cache frame in the cache queue that is located after the first cache frame; When the third preset display time is the same as the second preset display time, or when the third preset display time is earlier than the second preset display time, the first cache frame is discarded and a synthesis operation is performed on the second cache frame.

2. The method according to claim 1, characterized in that The method further comprises: When the third preset display time is later than the second preset display time, and it is determined that the interval between the currently received second synthetic vertical synchronization signal and the previous second synthetic vertical synchronization signal is less than one Vsync cycle, or when the interval between the second application vertical synchronization Vsync-APP signal used to trigger the drawing and rendering operation on the second cache frame and the previous second application vertical synchronization signal is less than one Vsync cycle, the first cache frame is discarded and the synthesis operation is performed on the second cache frame.

3. The method according to claim 1, characterized in that The method further comprises: When the first preset display sending time is the same as the second preset display sending time, or when the third preset display sending time is later than the second preset display sending time, a synthesis operation is performed on the first cache frame.

4. The method according to claim 1, characterized in that: The frame loss condition also includes: a timestamp attribute corresponding to a first preset display time of the first cached frame is an automatic timestamp, and a current refresh rate is greater than 60 Hz.

5. The method according to claim 2, characterized in that: Before receiving the second touch operation input by the user, the method further includes: Receiving a first touch operation input by a user; If the touch scenario of the first touch operation is a hands-off operation scenario, in response to the received first application vertical synchronization Vsync-APP signal, perform a drawing rendering operation on the first image frame within a first configured time length in the hands-off operation scenario; In response to the received first synthetic vertical synchronization Vsync-SF signal, performing a synthesis operation on the first image frame after drawing and rendering within a second configuration time length in the hands-off operation scenario; wherein the first synthetic vertical synchronization signal and the first application vertical synchronization signal are separated by the first configuration time length; In response to the received first hardware vertical synchronization Vsync-HW signal, a display operation is performed on the synthesized first image frame; wherein the first hardware vertical synchronization signal and the first synthesized vertical synchronization signal are separated by the second configuration time length.

6. The method according to claim 5, characterized in that After receiving the second touch operation input by the user, the method further includes: If the touch scenario of the second touch operation is a hands-on operation scenario, in response to the received second application vertical synchronization signal, perform a drawing rendering operation on the second image frame within a third configured time length in the hands-on operation scenario; In response to the received second synthetic vertical synchronization signal, performing a synthesis operation on the second image frame after drawing and rendering within a fourth configuration time length in the hands-free operation scenario; wherein the second synthetic vertical synchronization signal and the second application vertical synchronization signal are separated by the third configuration time length; In response to the received second hardware vertical synchronization Vsync-HW signal, performing a display operation on the synthesized second image frame; the second hardware vertical synchronization signal and the second synthesized vertical synchronization signal are separated by the fourth configuration time length; Among them, the third configuration duration is shorter than the first configuration duration, and / or the fourth configuration duration is shorter than the second configuration duration.

7. The method according to any one of claims 1 to 6, characterized in that: The hands-on operation scenarios include hands-on sliding operation scenarios and hands-on pressing operation scenarios.

8. The method according to claim 6, characterized in that The method further comprises: If the touch scenario of the first touch operation is a hands-off operation scenario, determining first configuration information corresponding to the hands-off operation scenario, the first configuration information including the first configuration duration and the second configuration duration; If the touch scenario of the second touch operation is a hands-on operation scenario, second configuration information corresponding to the hands-on operation scenario is determined, where the second configuration information includes the third configuration duration and the fourth configuration duration.

9. The method according to claim 6, characterized in that The second application vertical synchronization signal and the third application vertical synchronization signal are separated by a first time length, the second application vertical synchronization signal and the fourth application vertical synchronization signal are separated by a second time length, the third application vertical synchronization signal is the previous application vertical synchronization signal of the second application vertical synchronization signal, and the fourth application vertical synchronization signal is the next application vertical synchronization signal of the second application vertical synchronization signal; The second synthesized vertical synchronization signal and the third synthesized vertical synchronization signal are separated by a third time length, the second synthesized vertical synchronization signal and the fourth synthesized vertical synchronization signal are separated by a fourth time length, the third synthesized vertical synchronization signal is the previous synthesized vertical synchronization signal of the second synthesized vertical synchronization signal, and the fourth synthesized vertical synchronization signal is the next synthesized vertical synchronization signal of the second synthesized vertical synchronization signal; The first duration is different from the second duration, and / or the third duration is different from the fourth duration.

10. The method according to claim 6, characterized in that If the touch scene of the second touch operation is a hands-on operation scene, the method further includes: generating the second application vertical synchronization signal based on the second hardware vertical synchronization signal and the first signal offset in the hands-free operation scenario; The second synthesized vertical synchronization signal is generated based on the second hardware vertical synchronization signal and the second signal offset in the hands-free operation scenario.

11. The method according to claim 6, characterized in that The method further comprises: Obtaining a historical drawing time for executing the drawing and rendering operation, and a historical synthesis time for executing the synthesis operation; The third configuration duration and the fourth configuration duration are determined according to the historical drawing duration and the historical synthesis duration; the third configuration duration is greater than or equal to the maximum drawing duration in the historical drawing durations, and the fourth configuration duration is greater than or equal to the maximum synthesis duration in the historical synthesis durations.

12. The method according to claim 6, characterized in that The method further comprises: Determining a chirality level corresponding to the hands-free operation scenario; The third configuration duration and the fourth configuration duration are determined based on the chirality level, wherein different chirality levels correspond to different configuration durations.

13. The method according to claim 12, characterized in that The non-hands-off operation scenario includes a finger-on-hand operation scenario and a stylus-on-hand operation scenario, and the stylus-on-hand operation scenario corresponds to a higher chirality level than the finger-on-hand operation scenario; Among them, the third configuration time length corresponding to the stylus pen never leaves hand operation scenario is shorter than the third configuration time length corresponding to the finger never leaves hand operation scenario, and / or the fourth configuration time length corresponding to the stylus pen never leaves hand operation scenario is shorter than the fourth configuration time length corresponding to the finger never leaves hand operation scenario.

14. An electronic device, characterized in that: It comprises a memory and a processor; the memory and the processor are coupled; the memory is used to store computer program code, the computer program code comprises computer instructions, and when the processor executes the computer instructions, the electronic device executes the method as described in any one of claims 1-13.

15. A computer storage medium, characterized in that The computer storage medium stores a computer program or instruction. When the computer program or instruction is executed, the method according to any one of claims 1 to 13 is executed.

Images