Updating method, device and storage medium for displaying image

Abstract

Embodiments of the present disclosure relate to a display image updating method and device and a storage medium. The method is applied to a terminal device. The method comprises: detecting a touch event for a display interface, and when the terminal device is in a first application state, performing application drawing processing on at least one first type layer corresponding to the touch event based on a type of the touch event and a target trigger time of application drawing processing corresponding to the type; after the application drawing processing on the at least one first type layer is completed, triggering layer composition processing on the at least one first type layer to obtain a target image frame; and updating a display image of the display interface based on the target image frame.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, but not limited to the field of wireless communication technology, and particularly to a method, apparatus and storage medium for updating a displayed image. Background Technology

[0002] With the rapid development of terminal device technology, more and more terminal devices are equipped with touch screens and display systems. The processing of data to be displayed in the display system mainly includes: application drawing processing, layer compositing processing, and hardware refresh display processing. In order to avoid display lag and improve display effect, the display system synchronizes the various processes through the system vertical synchronization signal.

[0003] For a single frame of an image, the process from application rendering to final screen display requires approximately three system vertical synchronization signal cycles. Even on high refresh rate (e.g., 120Hz) terminal devices, the latency for a single frame is about 24ms. In some application scenarios requiring high smoothness and responsiveness (e.g., stylus applications), noticeable stuttering can occur, resulting in a poor user experience. Summary of the Invention

[0004] This disclosure provides a method, apparatus, and storage medium for updating a displayed image.

[0005] The first aspect of this disclosure provides a method for updating a displayed image, applied to a terminal device, wherein the method includes:

[0006] When a touch event is detected on the display interface and the terminal device is in a first application state, application drawing processing is performed on at least one first type layer corresponding to the touch event based on the type of the touch event and the target trigger time of the application drawing processing corresponding to the type.

[0007] After the application rendering process of the at least one first-type layer is completed, the layer compositing process of the at least one first-type layer is triggered to obtain the target image frame;

[0008] Based on the target image frame, the display image of the display interface is updated.

[0009] In the above scheme, the step of performing application drawing processing on at least one first-type layer corresponding to the touch event based on the type of the touch event and the target trigger time of the application drawing processing corresponding to the type includes:

[0010] If the type of the touch event indicates that the touch event is a click event, based on the detection time of the click event, at least one first-type layer corresponding to the click event is applied to the drawing process.

[0011] In the above scheme, the step of performing application drawing processing on at least one first-type layer corresponding to the touch event based on the type of the touch event and the target trigger time of the application drawing processing corresponding to the type of the touch event includes:

[0012] If the type of the touch event indicates that the touch event is a swipe event, at a preset time before the next first synchronization signal is generated, at least one first type layer corresponding to the swipe event is applied to the drawing process; wherein, the first synchronization signal is the system vertical synchronization signal that triggers the application drawing process.

[0013] In the above scheme, the method further includes:

[0014] Obtain the maximum time required to apply drawing and layer compositing processing to the first type of layer;

[0015] The preset time is determined based on the difference between the maximum duration and the signal period of the first synchronization signal.

[0016] In the above scheme, the step of triggering layer compositing processing for the at least one first-type layer after the application rendering process of the at least one first-type layer is completed to obtain the target image frame includes:

[0017] After the application drawing process of at least one first-type layer is completed, the at least one first-type layer after the application drawing process is written into the layer composition cache;

[0018] Triggered by the first write operation of the first type of layer, layer compositing processing is performed on at least one first type of layer in the layer compositing cache to obtain the target image frame.

[0019] In the above scheme, the method further includes:

[0020] Triggered by the first synchronization signal, at least one second-type layer is applied to the obtained layer for drawing, and the at least one second-type layer after the application drawing is written to the layer composition cache; wherein, the second-type layer is generated by the application event of the application currently running on the terminal device.

[0021] Based on the signal period of a second synchronization signal prior to the second write operation of the second type of image, it is determined whether there is layer compositing processing for the first type of layer within the signal period of the second synchronization signal; the second synchronization signal is the system vertical synchronization signal that triggers the layer compositing processing.

[0022] The triggering of the first write operation based on the first type of layer, performing layer compositing processing on at least one first type of layer in the layer compositing cache to obtain the target image frame, includes:

[0023] If there is a layer compositing process for the first type of layer, within a signal period of a second synchronization signal after the second write operation, based on the triggering of the first write operation, layer compositing process is performed on the at least one first type of layer and the at least one second type of layer in the layer compositing cache to obtain the target image frame.

[0024] In the above scheme, the method further includes:

[0025] If there is no layer compositing process for the first type of layer, based on the triggering of the next second synchronization signal, layer compositing process is performed on at least one second type of layer in the layer compositing cache to obtain the target image frame.

[0026] In the above scheme, before performing application drawing processing on at least one first-type layer corresponding to the touch event based on the type of the touch event and the target trigger time of the application drawing processing corresponding to the type, the method further includes:

[0027] In response to a launch operation for any application on the display interface, obtain the window identifier corresponding to the application;

[0028] Determine whether the window identifier corresponding to the application includes a preset identifier;

[0029] If the window identifier corresponding to the application includes a preset identifier, the terminal device is determined to be in a first application state, and at least one first-type layer generated by the touch event is obtained.

[0030] In the above scheme, the method further includes:

[0031] If the window identifier corresponding to the application does not include the preset identifier, the terminal device is determined to be in a second application state.

[0032] When a touch event is detected on the display interface and the terminal device is in the second application state, the application drawing process is performed on at least one first type layer corresponding to the touch event based on the triggering of the first synchronization signal.

[0033] Triggered by the second synchronization signal, at least one first-class layer after application rendering is subjected to layer compositing.

[0034] A second aspect of this disclosure provides an image updating apparatus applied to a terminal device, wherein the apparatus includes:

[0035] The first processing module is configured to detect a touch event for the display interface and, when the terminal device is in a first application state, perform application drawing processing on at least one first type layer corresponding to the touch event based on the type of the touch event and the target trigger time of the application drawing processing corresponding to the type.

[0036] The second processing module is configured to trigger layer compositing processing on the at least one first-type layer after the application drawing processing of the at least one first-type layer is completed, so as to obtain the target image frame.

[0037] The update module is configured to update the display image of the display interface based on the target image frame.

[0038] In the above scheme, the first processing module is configured as follows:

[0039] If the type of the touch event indicates that the touch event is a click event, based on the detection time of the click event, at least one first-type layer corresponding to the click event is applied to the drawing process.

[0040] In the above scheme, the first processing module is configured as follows:

[0041] If the type of the touch event indicates that the touch event is a swipe event, at a preset time before the next first synchronization signal is generated, at least one first type layer corresponding to the swipe event is applied to the drawing process; wherein, the first synchronization signal is the system vertical synchronization signal that triggers the application drawing process.

[0042] In the above scheme, the first processing module is configured as follows:

[0043] Obtain the maximum time required to apply drawing and layer compositing processing to the first type of layer;

[0044] The preset time is determined based on the difference between the maximum duration and the signal period of the first synchronization signal.

[0045] In the above scheme, the second processing module is configured as follows:

[0046] After the application drawing process of at least one first-type layer is completed, the at least one first-type layer after the application drawing process is written into the layer composition cache;

[0047] Triggered by the first write operation of the first type of layer, layer compositing processing is performed on at least one first type of layer in the layer compositing cache to obtain the target image frame.

[0048] In the above scheme, the first processing module is configured as follows:

[0049] Triggered by the first synchronization signal, at least one second-type layer is applied to the obtained layer for drawing, and the at least one second-type layer after the application drawing is written to the layer composition cache; wherein, the second-type layer is generated by the application event of the application currently running on the terminal device.

[0050] Based on the signal period of a second synchronization signal prior to the second write operation of the second type of image, it is determined whether there is layer compositing processing for the first type of layer within the signal period of the second synchronization signal; the second synchronization signal is the system vertical synchronization signal that triggers the layer compositing processing.

[0051] The second processing module is configured as follows:

[0052] If there is a layer compositing process for the first type of layer, within a signal period of a second synchronization signal after the second write operation, based on the triggering of the first write operation, layer compositing process is performed on the at least one first type of layer and the at least one second type of layer in the layer compositing cache to obtain the target image frame.

[0053] In the above scheme, the second processing module is configured as follows:

[0054] If there is no layer compositing process for the first type of layer, based on the triggering of the next second synchronization signal, layer compositing process is performed on at least one second type of layer in the layer compositing cache to obtain the target image frame.

[0055] In the above scheme, the first processing module is configured as follows:

[0056] In response to a launch operation for any application on the display interface, obtain the window identifier corresponding to the application;

[0057] Determine whether the window identifier corresponding to the application includes a preset identifier;

[0058] If the window identifier corresponding to the application includes a preset identifier, the terminal device is determined to be in a first application state, and at least one first-type layer generated by the touch event is obtained.

[0059] In the above scheme, the first processing module is configured as follows:

[0060] If the window identifier corresponding to the application does not include the preset identifier, the terminal device is determined to be in a second application state.

[0061] When a touch event is detected on the display interface and the terminal device is in the second application state, the application drawing process is performed on at least one first type layer corresponding to the touch event based on the triggering of the first synchronization signal.

[0062] The second processing module is configured as follows:

[0063] Triggered by the second synchronization signal, at least one first-class layer after application rendering is subjected to layer compositing.

[0064] A third aspect of this disclosure provides an image updating apparatus, comprising:

[0065] processor;

[0066] Memory configured to store executable instructions;

[0067] When the processor executes the executable instructions, it performs the update method for providing the displayed image as described in the first aspect above.

[0068] A fourth aspect of this disclosure provides a non-transitory computer-readable storage medium, wherein when instructions in the storage medium are executed by a processor of a display image updating device, the display image updating device is able to perform the display image updating method provided in the first aspect.

[0069] In this embodiment of the present disclosure, when a touch event is detected on the display interface and the terminal device is in a first application state, a target trigger time corresponding to the type of touch event is determined according to the type of touch event. At the target trigger time, application drawing processing of at least one first type layer generated by the touch event is triggered. Thus, application drawing processing is performed in a timely manner based on different types of touch events generated during the touch process of the display interface, without waiting for the triggering of the first synchronization signal, reducing the delay in the application drawing processing process.

[0070] Furthermore, after the application rendering process of at least one first-type layer is completed, the layer compositing process of the at least one first-type layer after the application rendering process is directly triggered to obtain the target image frame; there is no need to wait for the second synchronization signal, which reduces the delay in the layer compositing process, thereby reducing the time spent from detecting the touch event to displaying the image frame generated by the touch event on the display interface, greatly reducing the touch latency of the terminal device and improving the user experience.

[0071] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the embodiments of this disclosure. Attached Figure Description

[0072] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the embodiments of the invention.

[0073] Figure 1 This is a flowchart illustrating a method for updating a displayed image according to an exemplary embodiment. Figure 1 ;

[0074] Figure 2 This is a flowchart illustrating a method for updating a displayed image according to an exemplary embodiment. Figure 2 ;

[0075] Figure 3 This is a schematic diagram illustrating the processing of the image frame corresponding to the Down event, as shown by related technologies;

[0076] Figure 4 This is a schematic diagram illustrating the processing of an image frame corresponding to a Down event according to an exemplary embodiment;

[0077] Figure 5 This is a schematic diagram illustrating the processing of the image frame corresponding to the Move event, as shown by related technologies;

[0078] Figure 6 This is a schematic diagram illustrating the processing of an image frame corresponding to a Down event according to an exemplary embodiment;

[0079] Figure 7 This is a schematic diagram illustrating the structure of an image updating device according to an exemplary embodiment;

[0080] Figure 8 This is a block diagram illustrating a terminal device according to an exemplary embodiment. Detailed Implementation

[0081] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of the present invention.

[0082] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the embodiments disclosed herein. The singular forms “a,” “the,” and “the” used herein are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0083] It should be understood that although the terms first, second, third, etc., may be used to describe various information in embodiments of this disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of embodiments of this disclosure, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0084] This disclosure provides a method for updating a displayed image, such as... Figure 1 As shown, Figure 1 This is a flowchart illustrating a method for updating a displayed image according to an exemplary embodiment. Figure 1 The method includes:

[0085] Step S101: When a touch event is detected on the display interface and the terminal device is in the first application state, application drawing processing is performed on at least one first type layer corresponding to the touch event based on the type of the touch event and the target trigger time of the application drawing processing corresponding to the type.

[0086] Step S102: After the application drawing process of the at least one first type of layer is completed, the layer compositing process of the at least one first type of layer is triggered to obtain the target image frame.

[0087] Step S103: Update the display image of the display interface based on the target image frame.

[0088] The method for updating the displayed image provided in this disclosure can be applied to terminal devices including touch screens. Here, the terminal device may include smartphones, tablets, or laptops, etc.

[0089] In step S101, during the use of the terminal device, after detecting a touch event generated by a touch operation on the display interface of the terminal device, it can be determined whether the current terminal device is in the first application state. If the current terminal device is in the first application state, the target trigger time of the application drawing process corresponding to the type of the touch event is determined according to the type of the touch event, and at the target trigger time, application drawing processing is performed on at least one first type layer generated based on the touch event.

[0090] Here, the first application state can be: the application currently running in the foreground of the terminal device is the target application. The target application is a pre-set application with high responsiveness requirements, such as a drawing application or a document application.

[0091] By acquiring the operating information of the terminal device, it can be determined whether the terminal device is in a first application state based on the operating information. The operating information of the terminal device includes at least: information about the currently running application.

[0092] The types of touch events can include: Down events, Move events, and Up events. It is understood that a user's touch operation on the display interface generates a sequence of events. This sequence refers to a series of events generated from the moment the touch object touches the display interface until the moment the touch object leaves the display interface. That is, an event sequence begins with a Down event, includes a variable number of Move events, and ends with an Up event.

[0093] In this embodiment of the present disclosure, after detecting a touch event, the type of the touch event is determined; and the target trigger time for triggering application rendering processing corresponding to the type of touch event is obtained; here, the target trigger time is different for different types of touch events.

[0094] It should be noted that, within the Android platform, after a touch event is detected, it is usually necessary to wait for the next first synchronization signal. The first synchronization signal triggers the application drawing processing of multiple first-type layers generated by the touch event. After the application drawing processing is completed, it is necessary to wait for the next second synchronization signal. The second synchronization signal triggers the layer compositing processing of the multiple first-type layers after the application drawing processing to obtain an image frame. Finally, under the trigger of the third synchronization signal, the image currently displayed on the display interface is refreshed based on the image frame.

[0095] Here, the first synchronization signal is the system vertical synchronization signal that triggers application rendering processing, namely the Vsync-app signal; the second synchronization signal is the system vertical synchronization signal that triggers layer compositing processing, namely the Vsync-sf signal; and the third synchronization signal is the system vertical synchronization signal that triggers hardware refresh processing, namely the Vsync-hw signal.

[0096] It should be noted that the first synchronization signal and the second synchronization signal are generated based on the third synchronization signal; that is, the third synchronization signal can be the signal source of the first and second synchronization signals; or, the first synchronization signal, the second synchronization signal, and the third synchronization signal are synchronized. Therefore, the signal periods of the first and second synchronization signals are the same as the signal period of the third synchronization signal, and their phases are consistent. However, in practical applications, due to various factors (e.g., the different processing performances of different components within the terminal device), there is an offset between the first synchronization signal, the second synchronization signal, and the third synchronization signal, and they will not be perfectly aligned.

[0097] In other words, from the detection of a touch event to the display of the image frame generated by the touch event, the process requires three signal cycles of the system vertical synchronization signal; this results in significant touch latency, leading to a poor user experience in some application scenarios with high responsiveness requirements (e.g., drawing scenarios).

[0098] In related technologies, in order to reduce touch latency, after detecting a Down event, drawing processing is applied directly to at least one first-type layer generated by the Down event without waiting for the triggering of the first synchronization signal. However, this method does not process the subsequently generated first synchronization signal, which will cause the image frame drawn in advance to be the same as the image frame drawn by the subsequently generated first synchronization signal, resulting in a one-frame stutter in the display interface. Furthermore, this method does not process the Move event, so the improvement effect on the touch latency of the display interface is limited.

[0099] According to the embodiments of this disclosure, different target trigger times for triggering application rendering processing are set based on different types of touch events. When a touch event is detected, at least one first-type layer generated by the touch event is passed to the currently running target application. The target application performs application rendering processing on at least one first-type layer generated by the touch event based on the trigger time of the target event, without waiting for the trigger of the first synchronization signal, thereby reducing the touch latency in the entire touch process.

[0100] In step S102, after the application rendering process of at least one first-type layer is completed, the application passes the at least one first-type layer after the application rendering process to Surface Flinger, and triggers Surface Flinger to perform layer compositing processing on the at least one first-type layer after the application rendering process to obtain the target image frame.

[0101] It should be noted that in the relevant technology, after the application completes the application drawing process of the at least one first type of layer, it will pass the at least one first type of layer after the application drawing process to Surface Flinger. However, Surface Flinger needs to wait for the next second synchronization signal. Under the trigger of the second synchronization signal, it will perform layer compositing process on the at least one first type of layer after the application drawing process.

[0102] To reduce touch latency, this embodiment of the present disclosure directly triggers the Surface Flinger to perform layer compositing on the at least one first-type layer after completing the application drawing process for at least one first-type layer, without waiting for the second synchronization signal.

[0103] In step S103, after obtaining the target image frame, the terminal device refreshes the display interface under the trigger of the next third synchronization signal to display the target image frame on the display interface.

[0104] Here, the third synchronization signal is a hardware signal triggered by the display screen driver of the terminal device, and the signal period of the third synchronization signal is determined according to the screen refresh rate of the terminal device's display screen. Specifically, the signal period of the third synchronization signal is the reciprocal of the screen refresh rate of the terminal device's display screen.

[0105] For example, the screen refresh rate of the terminal device's display screen can be any value such as 60Hz, 70Hz, 75Hz, or 80Hz. Taking a screen refresh rate of 60Hz as an example, the signal period of the third synchronization signal is 16.667ms. It should be noted that the terminal device may support multiple different screen refresh rates, and the signal period of the third synchronization signal is determined by the screen refresh rate currently used by the terminal device.

[0106] In this embodiment of the present disclosure, when a touch event is detected on the display interface and the terminal device is in a first application state, a target trigger time corresponding to the type of touch event is determined according to the type of touch event. At the target trigger time, application drawing processing of at least one first type layer generated by the touch event is triggered. Thus, application drawing processing is performed in a timely manner based on different types of touch events generated during the touch process of the display interface, without waiting for the triggering of the first synchronization signal, reducing the delay in the application drawing processing process.

[0107] Furthermore, after the application rendering process of at least one first-type layer is completed, the layer compositing process of the at least one first-type layer after the application rendering process is directly triggered to obtain the target image frame; there is no need to wait for the second synchronization signal, which reduces the delay in the layer compositing process, thereby reducing the time spent from detecting the touch event to displaying the image frame generated by the touch event on the display interface, greatly reducing the touch latency of the terminal device and improving the user experience.

[0108] In some embodiments, step S101, which involves performing application drawing processing on at least one first-type layer corresponding to the touch event based on the type of the touch event and the target trigger time of the application drawing processing corresponding to the type, includes:

[0109] If the type of the touch event indicates that the touch event is a click event, based on the detection time of the click event, at least one first-type layer corresponding to the click event is applied to the drawing process.

[0110] In this embodiment of the disclosure, if the currently detected touch event is a Down event, the target trigger time corresponding to the Down event can be determined as the detection time when the Down event is detected.

[0111] Therefore, when the Down event is detected and the terminal device is in the first application state, at least one first-type layer generated by the Down event is passed to the currently running target application, and the target application is triggered to perform application drawing processing on the at least one first-type layer.

[0112] It should be noted that in the relevant technology, when the terminal device detects a Down event, it will pass at least one first-type layer generated by the Down event to the target application. After receiving the at least one first-type layer, the target application needs to wait for a first synchronization signal. Under the trigger of the first synchronization signal, it will perform application drawing processing on the at least one first-type layer.

[0113] In this embodiment of the disclosure, after a Down event is detected, at least one first-type layer corresponding to the Down event is passed to the target application, and based on this passing operation, the target application is triggered to perform application drawing processing on the at least one first-type layer without waiting for the first synchronization signal, thereby reducing the delay in the application drawing processing process corresponding to the Down event.

[0114] In some embodiments, step S101, which involves performing application drawing processing on at least one first-type layer corresponding to the touch event based on the type of the touch event and the target trigger time of the application drawing processing corresponding to the type, includes:

[0115] If the type of the touch event indicates that the touch event is a swipe event, at a preset time before the next first synchronization signal is generated, at least one first type layer corresponding to the swipe event is applied to the drawing process; wherein, the first synchronization signal is the system vertical synchronization signal that triggers the application drawing process.

[0116] In this embodiment of the disclosure, if the currently detected touch event is a Move event, the target trigger time corresponding to the Move event can be determined to be a preset time before the next first synchronization signal is generated after the Move event is detected.

[0117] Here, the preset time can be set according to actual needs. For example, the target trigger time of the Move event can be 2ms before the next first synchronization signal is generated.

[0118] It should be noted that in the relevant technology, when the terminal device detects a Move event, it does not directly pass at least one first-type layer generated by the Move event to the target application. Instead, after detecting the Move event, it needs to wait for the next first synchronization signal. Under the trigger of the first synchronization signal, it passes at least one first-type layer generated by the Move event to the target application, so that the target application can perform application drawing processing on the at least one first-type layer after receiving it.

[0119] Even if, in the embodiments of this disclosure, after the target application completes the application rendering process, the Surface Flinger is directly triggered to perform layer compositing on at least one first-type layer after the application rendering process without waiting for the second synchronization signal, it is still difficult to guarantee that the application rendering process and layer compositing process for at least one first-type layer generated by the Move event are completed within the signal cycle of one first synchronization signal.

[0120] Therefore, for Move events, the present disclosure embodiment can advance the processing time of the application drawing processing of at least one first type layer generated by the Move event, that is, at a preset time before the next first synchronization signal after the Move event is detected, the at least one first type layer generated by the Move event is passed to the target application, triggering the target application to perform application drawing processing on the at least one first type layer generated by the Move event.

[0121] After the target application completes the application rendering process, the Surface Flinger is directly triggered to perform layer compositing on at least one first-type layer after the application rendering process, thereby completing the application rendering and layer compositing of at least one first-type layer generated by the Move event within the signal cycle of a first synchronization signal, reducing the touch latency of the terminal device.

[0122] In some embodiments, the method further includes:

[0123] Obtain the maximum time required to apply drawing and layer compositing processing to the first type of layer;

[0124] The preset time is determined based on the difference between the maximum duration and the signal period of the first synchronization signal.

[0125] In this embodiment of the disclosure, a first duration required for applying drawing processing to the first type of layer and a second duration required for layer compositing processing to the first type of layer are obtained respectively, and the maximum duration is determined based on the first duration and the second duration.

[0126] Here, based on experimental testing, the first time required for applying drawing processing to the first type of layer and the second time required for layer compositing processing can be obtained respectively.

[0127] Specific experimental testing methods can be set according to actual conditions, and this disclosure does not limit them. For example, a robotic arm can be used to touch the display interface, and the Android system tracing (Systrace) tool can be used to capture the first timing information of the terminal device applying drawing processing to at least one layer, and the second timing information of the layer compositing processing to at least one layer; and based on multiple first timing information, the first duration required for applying drawing processing to the first type of layer is determined; based on multiple second timing information, the second duration required for layer compositing processing to the first type of layer is determined, thereby determining the maximum duration.

[0128] Here, Systrace is a tool for performance analysis of the Android operating system of a terminal device. Systrace's functions include tracking the system's input / output operations, kernel work queues, CPU load, and the operating status of various subsystems of the Android operating system.

[0129] In some embodiments, the processing time for application rendering and layer compositing can be obtained by acquiring multiple historical image frames, and the maximum time required for application rendering and layer compositing of the first type of layer can be determined based on the processing time of the multiple historical image frames.

[0130] After determining the maximum duration required for application rendering and layer compositing, the difference between the maximum duration and the signal period of the first synchronization signal is determined. Based on the difference, the preset time is determined. Thus, at the preset time before the next first synchronization signal is generated after the Move event is detected, application rendering is performed on at least one first-type layer generated by the Move event. After the application rendering is completed, layer compositing is directly triggered on the at least one first-type layer, so that the application rendering and layer compositing of at least one first-type layer generated by the Move event can be completed within the signal period of one first synchronization signal, reducing the touch latency of the terminal device.

[0131] In some embodiments, step S102, which involves triggering a layer compositing process for the at least one first-type layer after the application rendering process of the at least one first-type layer is completed, to obtain a target image frame, includes:

[0132] After the application drawing process of at least one first-type layer is completed, the at least one first-type layer after the application drawing process is written into the layer composition cache;

[0133] Triggered by the first write operation of the first type of layer, layer compositing processing is performed on at least one first type of layer in the layer compositing cache to obtain the target image frame.

[0134] In this embodiment of the disclosure, after the target application completes the application drawing processing of at least one first type layer generated by the touch event, it writes the at least one first type layer after the application drawing processing into the layer compositing cache; after Surface Flinger detects the first write operation to the layer compositing cache, based on the triggering of the first write operation, it reads the first type layer written by the first write operation from the layer compositing cache, performs layer compositing processing on the first type layer, and obtains the target image frame.

[0135] It should be noted that in related technologies, Surface Flinger typically reads the first type of layer from the layer compositing cache and performs layer compositing processing on the first type of layer upon triggering a second synchronization signal. If Surface Flinger detects the first write operation to the layer compositing cache only after the previous second synchronization signal has been generated, Surface Flinger needs to wait for approximately one second synchronization signal cycle before it can perform layer compositing processing on the first type of layer written by the first write operation.

[0136] To reduce the waiting time during the layer compositing process, this embodiment uses the write time of the first write operation to the layer compositing cache as the trigger time for triggering the layer compositing process. This allows Surface Flinger to directly perform layer compositing on the first type of layer written by the first write operation after detecting the first write operation to the layer compositing cache, without waiting for the generation of the next second synchronization signal, thus significantly reducing the processing delay of the layer compositing process.

[0137] In some embodiments, the method further includes:

[0138] Triggered by the first synchronization signal, at least one second-type layer is applied to the obtained layer for drawing, and the at least one second-type layer after the application drawing is written to the layer composition cache; wherein, the second-type layer is generated by the application event of the application currently running on the terminal device.

[0139] Based on the signal period of a second synchronization signal prior to the second write operation of the second type of image, it is determined whether there is layer compositing processing for the first type of layer within the signal period of the second synchronization signal; the second synchronization signal is the system vertical synchronization signal that triggers the layer compositing processing.

[0140] The triggering of the first write operation based on the first type of layer, performing layer compositing processing on at least one first type of layer in the layer compositing cache to obtain the target image frame, includes:

[0141] If there is a layer compositing process for the first type of layer, within a signal period of a second synchronization signal after the second write operation, based on the triggering of the first write operation, layer compositing process is performed on the at least one first type of layer and the at least one second type of layer in the layer compositing cache to obtain the target image frame.

[0142] In this embodiment of the disclosure, the layers to be applied for drawing and layer composition processing include: a first type of layer and / or a second type of layer.

[0143] It should be noted that the second type of layer is generated by application events of the application currently running on the terminal device; for example, when the terminal device is in the first application state, when the stylus establishes a connection with the terminal device, a connection establishment prompt window will pop up on the display interface; the connection establishment prompt window image contains at least one second type of layer generated by the connection establishment event.

[0144] When an application event is detected and the terminal device is in a first application state, at least one second-type layer generated by the application event is acquired, and the device waits for the generation of the next first synchronization signal; and based on the triggering of the next synchronization signal, the at least one second-type layer is subjected to application drawing processing, and the at least one second-type layer after application drawing processing is written into the layer composition cache.

[0145] It should be noted that since the second type of layer is triggered by application events, there is some delay in the application rendering and layer compositing processes of the second type of layer. This delay does not affect the image display effect on the interface. Therefore, the application rendering process for the second type of layer can be triggered based on the first synchronization signal, and the image compositing process can be triggered based on the second synchronization signal. In other words, the timing of the application rendering and image compositing processes for the first type of layer on the terminal device is different from the timing of the application rendering and image compositing processes for the second type of layer.

[0146] After Surface Flinger detects a second write operation to the layer compositing cache, it determines whether there is layer compositing processing for the first type of layer within the signal period of a second synchronization signal prior to the second write operation.

[0147] If it is determined that there is layer compositing processing of the first type of layer within the signal period, in the signal period of the next second synchronization signal after the second write operation to the layer compositing cache is detected, the first write operation to the layer compositing cache is detected, and when the first write operation is detected, layer compositing processing is triggered on at least one first type of layer and at least one second type of layer stored in the layer compositing cache to obtain the target image frame.

[0148] It should be noted that when the terminal device is in the first application state, that is, during the operation of the target application on the terminal device, there will be times when both the first type of layer and the second type of layer need to be image composited. For example, when drawing on the display interface with a stylus, a prompt message from another application pops up; at this time, both the first type of layer generated by the stylus drawing and the second type of layer generated by the prompt message need to be composited.

[0149] In order to ensure that the layer compositing process of the first type of layer is prioritized when the terminal device is in the first application state, after writing at least one second type of layer after the application drawing process into the layer compositing cache, it can be determined whether there is layer compositing process for the first type of layer within the signal period of a second synchronization signal before the second writing operation.

[0150] If there is layer compositing processing for the first type of layer, Surface Flinger can delay layer compositing processing for the second type of layer. After detecting the first write operation, based on the first write operation, Surface Flinger is triggered to perform image compositing processing on the first type of layer and the second type of layer in the layer compositing cache together to obtain the target image frame.

[0151] Understandably, within the signal cycle of a second synchronization signal after Surface Flinger completes image compositing processing of at least one first-type layer corresponding to the nth touch event and obtains the nth frame target image, Surface Flinger detects a second write operation to write at least one second-type layer to the layer compositing buffer. If Surface Flinger performs layer compositing processing on the second-type layer based on the processing timing of the layer compositing processing of the second-type layer, that is, based on the triggering of the second synchronization signal, it may cause the target image frame obtained by the layer compositing processing of at least one first-type layer corresponding to the (n+1)th touch event to be refreshed and displayed on the display interface with a delay of one signal cycle of the third synchronization signal, thereby causing touch delay and display screen stuttering.

[0152] Therefore, in this embodiment of the present disclosure, after detecting the second write operation, Surface Flinger needs to first determine whether there is image compositing processing for at least one first-type layer corresponding to the nth touch event within the signal period of a second synchronization signal before the second write operation. If there is image compositing processing for at least one first-type layer corresponding to the nth touch event, the layer compositing processing for the second-type layer can be delayed. And within the signal period of a second synchronization signal after the second write operation, based on the triggering of the first write operation for at least one first-type layer corresponding to the n+1th touch event, layer compositing processing is performed on at least one first-type layer and at least one second-type layer corresponding to the n+1th touch event in the layer compositing cache to obtain the target image frame. In this way, when both the first-type layer and the second-type layer need to undergo image compositing processing, the touch delay and display screen stuttering caused by layer compositing processing are reduced, and the user experience is improved.

[0153] In some embodiments, the method further includes:

[0154] If there is no layer compositing process for the first type of layer, based on the triggering of the next second synchronization signal, layer compositing process is performed on at least one second type of layer in the layer compositing cache to obtain the target image frame.

[0155] In this embodiment of the disclosure, after Surface Flinger detects a second write operation to the layer compositing cache, it determines whether there is a layer compositing process for the first type of layer within the signal period of a second synchronization signal prior to the second write operation.

[0156] If there is no layer compositing process for the first type of layer within the signal period, layer compositing process can be performed on at least one second type of layer in the layer compositing cache based on the triggering of the next second synchronization signal after the second write operation to obtain the target image frame.

[0157] It is understandable that if Surface Flinger does not detect layer compositing processing of the first type of layer within the signal cycle of the second synchronization signal before the second write operation, it means that the touch device is not in contact with the display interface within the signal cycle of the second synchronization signal before the second write operation; therefore, the first write operation will not occur within the signal cycle of the second synchronization signal after the second write operation, and layer compositing processing of the first type of layer is not required.

[0158] At this time, based on the triggering of the next second synchronization signal after the second write operation, at least one second type of layer in the layer compositing cache can be processed to obtain the target image frame; and under the triggering of the third synchronization signal, the target image frame is refreshed and displayed on the display interface.

[0159] In some embodiments, before performing application drawing processing on at least one first-type layer corresponding to the touch event based on the type of the touch event and the target trigger time of the application drawing processing corresponding to the type, the method further includes:

[0160] In response to a launch operation for any application on the display interface, obtain the window identifier corresponding to the application;

[0161] Determine whether the window identifier corresponding to the application includes a preset identifier;

[0162] If the window identifier corresponding to the application includes a preset identifier, the terminal device is determined to be in a first application state, and at least one first-type layer generated by the touch event is obtained.

[0163] In this embodiment of the disclosure, the terminal device receives a user's selection operation for the application icon of any application displayed on the display interface, launches the application corresponding to the application icon, and in response to the launch operation of the application, obtains the window identifier corresponding to the application.

[0164] Here, the window identifier may include: the application window's number, name, etc. It is understood that each open application window on the display interface has a corresponding window identifier, which indicates the application currently running in the foreground on the terminal device.

[0165] The preset identifier can be a pre-set window identifier for the target application; here, the target application may include: drawing applications, document applications, etc.

[0166] By comparing the window identifier of the application with the preset identifier, it is determined whether the window identifier of the application contains the preset identifier; if the window identifier of the application contains the preset identifier, it indicates that the application running in the foreground of the terminal device is the target application, that is, the terminal device is in the first application state.

[0167] It is understood that, upon detecting a touch event, this embodiment of the present disclosure can obtain the window identifier corresponding to the application, determine whether the application currently running in the foreground of the terminal device is the target application based on the window identifier, and when the application currently running in the foreground of the terminal device is the target application, i.e., the terminal device is in the first application state, obtain at least one first type layer generated by the touch event, so as to perform application drawing processing on at least one first type layer corresponding to the touch event based on the type of the touch event and the target trigger time of the application drawing processing corresponding to the type, and after the application drawing processing is completed, trigger the layer compositing processing of the at least one first type layer to obtain the target image frame.

[0168] This embodiment of the disclosure determines whether the terminal device is in a first application state with high responsiveness requirements by obtaining the window identifier. If the terminal device is in the first application state, when performing application drawing processing and layer compositing processing on at least one first type of image generated by the touch event, it does not need to wait for the triggering of the first synchronization signal and the second synchronization signal, thereby effectively reducing the touch latency in the whole process and improving the user experience.

[0169] In some embodiments, the method further includes:

[0170] If the window identifier corresponding to the application does not include the preset identifier, the terminal device is determined to be in a second application state.

[0171] When a touch event is detected on the display interface and the terminal device is in the second application state, the application drawing process is performed on at least one first type layer corresponding to the touch event based on the triggering of the first synchronization signal.

[0172] Triggered by the second synchronization signal, at least one first-class layer after application rendering is subjected to layer compositing.

[0173] In this embodiment of the disclosure, the window identifier of the application is compared with the preset identifier to determine whether the window identifier of the application contains the preset identifier; if the window identifier corresponding to the application does not contain the preset identifier, it indicates that the application running in the foreground of the terminal device is not the target application, that is, the terminal device is in the second application state.

[0174] Understandably, considering that the terminal device is in the second application state, even if there is a touch delay, the touch delay will not affect the display effect of the terminal device because the responsiveness requirements of the second application state are lower. Therefore, when the terminal device is in the second application state, the application rendering process and layer composition process of at least one first type layer corresponding to the touch event can be started according to the triggering of the first synchronization signal and the second synchronization signal, respectively.

[0175] This disclosure also provides a method for updating the displayed image, such as... Figure 2 As shown, Figure 2 This is a flowchart illustrating a method for updating a displayed image according to an exemplary embodiment. Figure 2 The method includes:

[0176] Step S201: In response to a launch operation for any application on the display interface, obtain the window identifier corresponding to the application; determine whether the window identifier corresponding to the application includes a preset identifier; if the window identifier corresponding to the application includes a preset identifier, determine that the terminal device is in a first application state.

[0177] In this example, the terminal device being in the first application state can be: the terminal device is in a stylus application scenario.

[0178] By obtaining the window identifier of the application, it can be determined whether the window identifier of the application includes the window identifier corresponding to the stylus application. If the window identifier of the application includes the window identifier corresponding to the stylus application, it is determined that the terminal device is in the first application state.

[0179] Understandably, stylus applications require high responsiveness. In the Android operating system, displaying a single frame of an image from a hand-drawn drawing to a screen display requires approximately three vertical synchronization signal cycles. Even if the terminal device's screen refresh rate is 120Hz, the delay for the next frame in a stylus application scenario is still about 24ms, severely degrading the user experience.

[0180] Step S202: A touch event for the display interface is detected. If the type of the touch event indicates that the touch event is a Down event, drawing processing is applied to at least one first-type layer corresponding to the Down event based on the detection time of the Down event.

[0181] In this example, when a touch event is detected and the terminal device is in a stylus application scenario, the type of the touch event can be determined first. If the type of the touch event indicates that the touch event is a Down event, at least one first-type layer generated by the Down event can be applied to the drawing process at the moment the Down event is detected.

[0182] Understandably, a touch event sequence refers to a series of events that occur from the moment the touch subject touches the screen until the moment the touch subject leaves the screen; a touch event sequence begins with a Down event, contains at least one Move event in the middle, and ends with an Up event.

[0183] like Figure 3 As shown, Figure 3 This is a schematic diagram illustrating the processing of an image frame corresponding to a Down event, as shown in related technologies. In these technologies, after detecting a Down event, the terminal device, upon triggering the next first synchronization signal, will pass at least one first-type layer generated by the Down event to the application for application rendering. That is, the application rendering processing for a Down event requires waiting for the first synchronization signal to be triggered. Here, the first synchronization signal is the Vsync-app signal.

[0184] like Figure 4 As shown, Figure 4 This is a schematic diagram illustrating the processing of an image frame corresponding to a Down event according to an exemplary embodiment. In this example, upon detecting a Down event, the application drawing process for at least one first-type layer corresponding to the Down event is immediately initiated, without waiting for the triggering of the first synchronization signal, thus reducing delays in the application drawing process.

[0185] Step S203: If the type of the touch event indicates that the touch event is a Move event, at a preset time before the next first synchronization signal is generated, at least one first type layer corresponding to the Move event is applied to the drawing process; wherein, the first synchronization signal is the system vertical synchronization signal that triggers the application drawing process.

[0186] In this example, when a touch event is detected and the terminal device is in a stylus application scenario, if the type of the touch event indicates that the touch event is a Move event, at least one first-type layer generated by the Move event can be applied to the drawing process at a preset time before the next first synchronization signal is generated.

[0187] like Figure 5 As shown, Figure 5 This is a schematic diagram illustrating the processing of an image frame corresponding to a Move event, as shown in related technologies. In these technologies, after detecting a Move event, the terminal device, upon triggering the next first synchronization signal, transmits at least one first-type layer generated by the Move event to the application for application rendering processing. Consequently, the application rendering processing and layer compositing processing of at least one first-type layer generated by the Move event cannot be completed within the signal cycle of one first synchronization signal.

[0188] like Figure 6 As shown, Figure 6 This is a schematic diagram illustrating the processing of an image frame corresponding to a Down event according to an exemplary embodiment. In this example, after detecting a Move event, at a preset time before the next first synchronization signal is generated, the application drawing processing of at least one first-type layer generated by the Move event is performed, that is, the application drawing processing of at least one first-type layer generated by the Move event is started in advance.

[0189] Here, the preset time can be 2ms.

[0190] In some embodiments, the maximum duration required for applying drawing and layer compositing to the first type of layer can be obtained; the preset time is determined based on the difference between the maximum duration and the signal period of the first synchronization signal.

[0191] It is understood that, based on the difference between the maximum duration and the signal period of the first synchronization signal, the processing time for applying drawing processing to at least one first-type layer generated by the Move event is determined in advance, so that the application drawing processing and layer compositing processing of at least one first-type layer generated by the Move event can be completed within the signal period of one first synchronization signal.

[0192] Step S204: After the application drawing process of at least one first type of layer is completed, the at least one first type of layer after the application drawing process is written into the layer composition cache; based on the triggering of the first write operation of the first type of layer, the at least one first type of layer in the layer composition cache is subjected to layer composition process to obtain the target image frame;

[0193] like Figure 4 and Figure 6As shown in the example, after completing the application drawing process of the first type of layer, at least one first type of layer after the application drawing process is passed to Surface Flinger, so that Surface Flinger can immediately perform layer compositing processing on the first type of layer after receiving it to obtain the target image frame.

[0194] It should be noted that, as Figure 3 and Figure 5 As shown, in related technologies, Surface Flinger needs to be triggered by a second synchronization signal (i.e., the Vsync-sf signal) to start the layer compositing process; however, in this example, the first write operation of the layer compositing cache triggers the layer compositing process of the first type of layer written by the first write operation, thereby realizing that after the application drawing process of the first type of layer is completed, the layer compositing process of the first type of image is immediately performed, reducing the delay in the layer compositing process stage.

[0195] Depend on Figure 3 and Figure 4 It is known that the processing of the image frame corresponding to the Down event in related technologies requires approximately 3-4 signal cycles of the vertical synchronization signal, while the processing of the image frame corresponding to the Down event in this example requires approximately 2-3 signal cycles of the vertical synchronization signal, reducing the delay by approximately one signal cycle of the vertical synchronization signal.

[0196] Depend on Figure 5 and Figure 6 It is known that the processing of the image frame corresponding to the Move event in related technologies requires approximately two vertical synchronization signal cycles, while the processing of the image frame corresponding to the Down event in this example can be completed within one vertical synchronization signal cycle, thus reducing the delay by approximately one vertical synchronization signal cycle.

[0197] In some embodiments, based on the triggering of a first synchronization signal, at least one second-type layer is applied to the obtained layer for drawing, and the at least one second-type layer after the application drawing is written to the layer composition cache; wherein, the second-type layer is generated by an application event of the application currently running on the terminal device.

[0198] Based on the signal period of a second synchronization signal prior to the second write operation of the second type of image, it is determined whether there is layer compositing processing for the first type of layer within the signal period of the second synchronization signal; the second synchronization signal is the system vertical synchronization signal that triggers the layer compositing processing.

[0199] If there is a layer compositing process for the first type of layer, within a signal period of a second synchronization signal after the second write operation, based on the triggering of the first write operation, layer compositing process is performed on the at least one first type of layer and the at least one second type of layer in the layer compositing cache to obtain the target image frame.

[0200] In this example, the terminal device contains two different vertical synchronization signal triggering mechanisms: one is for the first type of layer generated by the terminal device in the first application state; the other is for the second type of layer generated by the terminal device.

[0201] When the terminal device is in the first application state, and there are first-type layers and second-type layers being combined for layer compositing, the system vertical synchronization signal triggering mechanism corresponding to the first-type layer takes precedence, and the layer compositing process of the first-type layer is processed first.

[0202] After the second type of layer completes the application drawing process, the second type of layer is written into the layer composition cache. It is then determined whether there was a layer composition process for the first type of layer in the previous second synchronization signal cycle. If there was a layer composition process for the first type of layer in the previous second synchronization signal cycle, it means that a new first type of layer will be written into the layer composition cache in one second synchronization signal cycle when the second type of layer is composed.

[0203] Therefore, in order to reduce screen stuttering, even if the second synchronization signal arrives within a second synchronization signal cycle after the second write operation, the second type of layer will not be processed for layer compositing. Instead, when the first write operation is detected, the first type of layer and the second type of layer will be processed for layer compositing together under the trigger of the first write operation.

[0204] In other embodiments, if there is no layer compositing process for the first type of layer, layer compositing process is performed on at least one second type of layer in the layer compositing cache based on the triggering of the next second synchronization signal to obtain the target image frame.

[0205] In this example, if there is layer compositing processing for the first type of layer within the signal cycle of the previous second synchronization signal, it means that no new first type of layer will be written to the layer compositing cache within one second synchronization signal cycle of layer compositing processing for the second type of layer. Therefore, the triggering mechanism of the system vertical synchronization signal corresponding to the second type of layer can be directly adopted, that is, layer compositing processing for the second type of layer is performed under the trigger of the second synchronization signal.

[0206] Step S205: If the window identifier corresponding to the application does not include the preset identifier, determine that the terminal device is in a second application state; when a touch event is detected on the display interface and the terminal device is in the second application state, based on the triggering of the first synchronization signal, perform application drawing processing on at least one first type layer corresponding to the touch event; based on the triggering of the second synchronization signal, perform layer compositing processing on at least one first type layer after application drawing processing to obtain a target image frame;

[0207] In this example, the terminal device being in the second application state can be: the terminal device is not in a stylus application scenario.

[0208] Understandably, in the Android operating system, although it takes about three vertical synchronization signal cycles to display a frame of an image from a hand-drawn pen to the screen, the application scenarios of the terminal device do not have high requirements for responsiveness. Even if there is a delay, it will not affect the display effect of the terminal device. Therefore, when the terminal device is in the second application state, application drawing processing and layer compositing processing can be performed based on the triggering of the first synchronization signal and the second synchronization signal, respectively.

[0209] Step S206: Update the display image of the display interface based on the target image frame.

[0210] In this example, after obtaining the synthesized target image frame, the target image frame is refreshed and displayed on the display interface of the terminal device under the trigger of the third synchronization signal.

[0211] This disclosure also provides an image updating device. Figure 7 This is a schematic diagram illustrating the structure of an image updating device according to an exemplary embodiment, such as... Figure 7 As shown, the device is applied to a terminal device, and the device 100 includes:

[0212] The first processing module 101 is configured to detect a touch event for the display interface and, when the terminal device is in a first application state, perform application drawing processing on at least one first type layer corresponding to the touch event based on the type of the touch event and the target trigger time of the application drawing processing corresponding to the type.

[0213] The second processing module 102 is configured to trigger layer compositing processing on the at least one first-type layer after the application drawing processing of the at least one first-type layer is completed, so as to obtain the target image frame.

[0214] The update module 103 is configured to update the display image of the display interface based on the target image frame.

[0215] In some embodiments, the first processing module 101 is configured to:

[0216] If the type of the touch event indicates that the touch event is a click event, based on the detection time of the click event, at least one first-type layer corresponding to the click event is applied to the drawing process.

[0217] In some embodiments, the first processing module 101 is configured to:

[0218] If the type of the touch event indicates that the touch event is a swipe event, at a preset time before the next first synchronization signal is generated, at least one first type layer corresponding to the swipe event is applied to the drawing process; wherein, the first synchronization signal is the system vertical synchronization signal that triggers the application drawing process.

[0219] In some embodiments, the first processing module 101 is configured to:

[0220] Obtain the maximum time required to apply drawing and layer compositing processing to the first type of layer;

[0221] The preset time is determined based on the difference between the maximum duration and the signal period of the first synchronization signal.

[0222] In some embodiments, the second processing module 102 is configured to:

[0223] After the application drawing process of at least one first-type layer is completed, the at least one first-type layer after the application drawing process is written into the layer composition cache;

[0224] Triggered by the first write operation of the first type of layer, layer compositing processing is performed on at least one first type of layer in the layer compositing cache to obtain the target image frame.

[0225] In some embodiments, the first processing module 101 is configured to:

[0226] Triggered by the first synchronization signal, at least one second-type layer is applied to the obtained layer for drawing, and the at least one second-type layer after the application drawing is written to the layer composition cache; wherein, the second-type layer is generated by the application event of the application currently running on the terminal device.

[0227] Based on the signal period of a second synchronization signal prior to the second write operation of the second type of image, it is determined whether there is layer compositing processing for the first type of layer within the signal period of the second synchronization signal; the second synchronization signal is the system vertical synchronization signal that triggers the layer compositing processing.

[0228] The second processing module 102 is configured as follows:

[0229] If there is a layer compositing process for the first type of layer, within a signal period of a second synchronization signal after the second write operation, based on the triggering of the first write operation, layer compositing process is performed on the at least one first type of layer and the at least one second type of layer in the layer compositing cache to obtain the target image frame.

[0230] In some embodiments, the second processing module 102 is configured to:

[0231] If there is no layer compositing process for the first type of layer, based on the triggering of the next second synchronization signal, layer compositing process is performed on at least one second type of layer in the layer compositing cache to obtain the target image frame.

[0232] In some embodiments, the first processing module 101 is configured to:

[0233] In response to a launch operation for any application on the display interface, obtain the window identifier corresponding to the application;

[0234] Determine whether the window identifier corresponding to the application includes a preset identifier;

[0235] If the window identifier corresponding to the application includes a preset identifier, the terminal device is determined to be in a first application state, and at least one first-type layer generated by the touch event is obtained.

[0236] In some embodiments, the first processing module 101 is configured to:

[0237] If the window identifier corresponding to the application does not include the preset identifier, the terminal device is determined to be in a second application state.

[0238] When a touch event is detected on the display interface and the terminal device is in the second application state, the application drawing process is performed on at least one first type layer corresponding to the touch event based on the triggering of the first synchronization signal.

[0239] The second processing module 102 is configured as follows:

[0240] Triggered by the second synchronization signal, at least one first-class layer after application rendering is subjected to layer compositing.

[0241] Figure 8 This is a block diagram illustrating a terminal device according to an exemplary embodiment. For example, the terminal device 800 may be a mobile phone, a mobile computer, etc.

[0242] Reference Figure 8 The terminal device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0243] Processing component 802 typically controls the overall operation of terminal device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0244] Memory 804 is configured to store various types of data to support the operation of device 800. Examples of this data include instructions for any application or method operating on terminal device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0245] Power supply component 806 provides power to various components of terminal device 800. Power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to terminal device 800.

[0246] Multimedia component 808 includes a screen that provides an output interface between the terminal device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0247] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when terminal device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0248] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0249] Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of terminal device 800. For example, sensor assembly 814 can detect the on / off state of device 800, the relative positioning of components such as the display and keypad of terminal device 800, changes in position of terminal device 800 or a component of terminal device 800, the presence or absence of user contact with terminal device 800, orientation or acceleration / deceleration of terminal device 800, and temperature changes of terminal device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0250] Communication component 816 is configured to facilitate wired or wireless communication between terminal device 800 and other devices. Terminal device 800 can access wireless networks based on communication standards, such as Wi-Fi, 4G, or 5G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0251] In an exemplary embodiment, the terminal device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0252] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of a terminal device 800 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0253] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0254] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for updating a displayed image, applied to a terminal device, wherein, The method includes: When a touch event is detected on the display interface and the terminal device is in a first application state, application drawing processing is performed on at least one first type layer corresponding to the touch event based on the type of the touch event and the target trigger time of the application drawing processing corresponding to the type. After the application rendering process of the at least one first-type layer is completed, the layer compositing process of the at least one first-type layer is triggered to obtain the target image frame; Based on the target image frame, the display image of the display interface is updated; After the application rendering process of the at least one first-type layer is completed, a layer compositing process for the at least one first-type layer is triggered to obtain the target image frame, including: After the application drawing process of at least one first-type layer is completed, the at least one first-type layer after the application drawing process is written into the layer composition cache; Triggered by the first write operation of the first type of layer, layer compositing processing is performed on at least one first type of layer in the layer compositing cache to obtain the target image frame.

2. The method according to claim 1, wherein, The step of performing application drawing processing on at least one first-type layer corresponding to the touch event based on the type of the touch event and the target trigger time of the application drawing processing corresponding to the type includes: If the type of the touch event indicates that the touch event is a click event, based on the detection time of the click event, at least one first-type layer corresponding to the click event is applied to the drawing process.

3. The method according to claim 1, wherein, The application rendering process, based on the type of the touch event and the target trigger time of the application rendering process corresponding to the type of the touch event, performs application rendering processing on at least one first-type layer corresponding to the touch event, including: If the type of the touch event indicates that the touch event is a swipe event, at a preset time before the next first synchronization signal is generated, at least one first type layer corresponding to the swipe event is applied to the drawing process; wherein, the first synchronization signal is the system vertical synchronization signal that triggers the application drawing process.

4. The method according to claim 3, wherein, The method further includes: Obtain the maximum time required to apply drawing and layer compositing processing to the first type of layer; The preset time is determined based on the difference between the maximum duration and the signal period of the first synchronization signal.

5. The method according to claim 1, wherein, The method further includes: Triggered by the first synchronization signal, at least one second-type layer is applied to the obtained layer for drawing, and the at least one second-type layer after the application drawing is written to the layer composition cache; wherein, the second-type layer is generated by the application event of the application currently running on the terminal device. Based on the signal period of a second synchronization signal prior to the second write operation of the second type of image, it is determined whether there is layer compositing processing for the first type of layer within the signal period of the second synchronization signal; the second synchronization signal is the system vertical synchronization signal that triggers the layer compositing processing. The triggering of the first write operation based on the first type of layer, performing layer compositing processing on at least one first type of layer in the layer compositing cache to obtain the target image frame, includes: If there is a layer compositing process for the first type of layer, within a signal period of a second synchronization signal after the second write operation, based on the triggering of the first write operation, layer compositing process is performed on the at least one first type of layer and the at least one second type of layer in the layer compositing cache to obtain the target image frame.

6. The method according to claim 5, wherein, The method further includes: If there is no layer compositing process for the first type of layer, based on the triggering of the next second synchronization signal, layer compositing process is performed on at least one second type of layer in the layer compositing cache to obtain the target image frame.

7. The method according to claim 1, wherein, Before performing application drawing processing on at least one first-type layer corresponding to the touch event based on the type of the touch event and the target trigger time of the application drawing processing corresponding to the type, the method further includes: In response to a launch operation for any application on the display interface, obtain the window identifier corresponding to the application; Determine whether the window identifier corresponding to the application includes a preset identifier; If the window identifier corresponding to the application includes a preset identifier, the terminal device is determined to be in a first application state, and at least one first-type layer generated by the touch event is obtained.

8. The method according to claim 7, wherein, The method further includes: If the window identifier corresponding to the application does not include the preset identifier, the terminal device is determined to be in a second application state. When a touch event is detected on the display interface and the terminal device is in the second application state, the application drawing process is performed on at least one first type layer corresponding to the touch event based on the triggering of the first synchronization signal. Triggered by the second synchronization signal, at least one first-class layer after application rendering is subjected to layer compositing.

9. An image updating device, applied to a terminal device, wherein, include: The first processing module is configured to detect a touch event for the display interface and, when the terminal device is in a first application state, perform application drawing processing on at least one first type layer corresponding to the touch event based on the type of the touch event and the target trigger time of the application drawing processing corresponding to the type. The second processing module is configured to trigger layer compositing processing on the at least one first-type layer after the application drawing processing of the at least one first-type layer is completed, so as to obtain the target image frame. The step of triggering layer compositing processing for the at least one first-type layer after the application drawing processing of the at least one first-type layer is completed to obtain a target image frame includes: writing the at least one first-type layer after the application drawing processing of the at least one first-type layer into a layer compositing cache after the application drawing processing of the at least one first-type layer is completed; performing layer compositing processing on the at least one first-type layer in the layer compositing cache based on the triggering of the first write operation of the first first-type layer to obtain the target image frame; and an update module is configured to update the display image of the display interface based on the target image frame.

10. An image updating device, wherein, include: processor; Memory configured to store executable instructions; The processor is configured to, when executing executable instructions stored in the memory, implement the method for updating the display image according to any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of a display image updating device, enable the display image updating device to perform the display image updating method according to any one of claims 1 to 8.

Images