Display processing method and device, storage medium and terminal equipment

Abstract

The invention discloses a display processing method and device, a storage medium and terminal equipment, and relates to the technical field of display, and the method comprises the steps: obtaining original frame data of a frame displayed in an interface scrolling stage; determining frame time consumption data according to the original frame data, the frame time consumption data being time consumption data of a frame display process; and carrying out lagging analysis according to the frame time consumption data to obtain a lagging condition in the interface rolling stage so as to carry out rolling display optimization on the interface according to the lagging condition. According to the method, the optimization effect and the optimization reliability of interface scrolling display in the interface scrolling scene can be improved.

Description

Technical Field

[0001] This application relates to the field of display technology, specifically to a display processing method, apparatus, storage medium, and terminal device. Background Technology

[0002] With the development of terminal devices, rich and clear UI interfaces and smooth user experience are important indicators of user experience. Currently, in order to achieve clearer UI interfaces, terminal devices are gradually incorporating features such as 4K UI and more complex interface layouts and animations. In scenarios with rapid screen changes and frequent screen scrolling, this leads to higher frame rendering and rendering times, resulting in more severe stuttering in the interface display during scrolling scenarios.

[0003] Currently, relevant optimization solutions typically analyze and optimize for stuttering based on all raw frame data provided by the system (such as Android). However, these solutions often neglect the scrolling phase of the interface display process, resulting in inaccurate analysis of stuttering during scrolling and poor optimization of the scrolling display. Furthermore, using all raw frame data for calculation and analysis can easily cause additional time consumption, negatively impacting the original scrolling display process and leading to weak optimization reliability. Summary of the Invention

[0004] This application provides a display processing solution that can effectively improve the optimization effect and reliability of interface scrolling display in interface scrolling scenarios.

[0005] The embodiments of this application provide the following technical solutions: According to one embodiment of this application, a display processing method includes: acquiring raw frame data of frames displayed during an interface scrolling phase; determining frame time consumption data based on the raw frame data, wherein the frame time consumption data is the time consumption data of the frame display process; performing stuttering analysis based on the frame time consumption data to obtain stuttering conditions during the interface scrolling phase, so as to optimize the scrolling display of the interface based on the stuttering conditions.

[0006] In some embodiments of this application, the step of performing stutter analysis based on the frame time consumption data to obtain the stuttering situation in the interface scrolling stage includes: determining abnormal time consumption frames in the interface scrolling stage based on the frame time consumption data; when the proportion of the abnormal time consumption frames in the interface scrolling stage is greater than a predetermined proportion, the stuttering situation is determined to be stuttering.

[0007] In some embodiments of this application, the frame time consumption data includes single-frame drawing time and single-frame rendering time; the step of determining the time-abnormal frame in the interface scrolling stage based on the frame time consumption data includes: determining the frame corresponding to the single-frame drawing time or the single-frame rendering time that is greater than a predetermined time limit as the time-abnormal frame.

[0008] In some embodiments of this application, after performing stuttering analysis based on the frame time consumption data to obtain the stuttering situation during the interface scrolling stage, the method further includes: if the stuttering situation is stuttering, analyzing the frame time consumption data corresponding to the time-abnormal frames to obtain time consumption distribution data; determining the target optimization scheme set for the interface based on the time consumption distribution data; and using the target optimization scheme set to optimize the scrolling display of the interface.

[0009] In some embodiments of this application, the frame time consumption data includes animation time consumption, buffer request time consumption, buffer commit time consumption, single-frame drawing time consumption, and single-frame rendering time consumption; the step of analyzing the frame time consumption data corresponding to the time-consuming abnormal frames to obtain time consumption distribution data includes: dividing the animation time consumption by the single-frame drawing time consumption to obtain the animation time consumption ratio; dividing the buffer request time consumption by the single-frame rendering time consumption to obtain the request time consumption ratio; dividing the buffer commit time consumption by the single-frame rendering time consumption to obtain the commit time consumption ratio; the time consumption distribution data includes the animation time consumption ratio, the request time consumption ratio, and the commit time consumption ratio.

[0010] In some embodiments of this application, the target optimization scheme set includes a scheme for optimizing the display of a first set of display stages when the interface is scrolling. The first set of display stages includes one or more stages such as a buffer request stage, a buffer submission stage, and an animation playback stage. Determining the target optimization scheme set of the interface based on the time consumption distribution data includes: determining that the first set of display stages includes the animation playback stage when the first average percentage of the animation time consumption ratio of all time-consuming abnormal frames in the interface scrolling stage is greater than a first preset threshold; determining that the first set of display stages includes the buffer request stage when the second average percentage of the request time consumption ratio of all time-consuming abnormal frames in the interface scrolling stage is greater than a second preset threshold; and determining that the first set of display stages includes the buffer submission stage when the third average percentage of the submission time consumption ratio of all time-consuming abnormal frames in the interface scrolling stage is greater than a third preset threshold.

[0011] In some embodiments of this application, when the first display stage set includes the buffer submission stage, the step of using the target optimization scheme set to optimize the scrolling display of the interface includes: selecting elements to be clipped from the elements included in the interface; determining the region position of the clipping area of ​​the element to be clipped; and saving the region position of the clipping area so that the clipping area can be clipped according to the region position for rendering when the interface is scrolling.

[0012] In some embodiments of this application, determining the region location of the cropping area of ​​the element to be cropped includes: determining the invisible region of the element to be cropped as the cropping area; calculating the region location of the invisible region to obtain the region location of the cropping area.

[0013] In some embodiments of this application, when the first display stage set includes the buffer application stage, the step of using the target optimization scheme set to optimize the scrolling display of the interface includes: determining the number of buffers corresponding to the interface; and expanding the number of buffers.

[0014] In some embodiments of this application, when the first display stage set includes the animation playback stage, the step of using the target optimization scheme set to optimize the scrolling display of the interface includes: determining whether the animation closing condition is met; if the animation closing condition is met, then closing the animation displayed on the interface during the scrolling display.

[0015] In some embodiments of this application, determining whether the animation closing condition is met includes: playing an animation closing prompt message; and determining that the animation closing condition is met when a consent to close instruction information is received.

[0016] In some embodiments of this application, before obtaining the original frame data of the frames displayed during the interface scrolling phase, the method further includes: obtaining a set of prediction optimization schemes corresponding to the interface, wherein the set of prediction optimization schemes is obtained by predicting and analyzing historical optimization data of the interface scrolling display using a preset prediction analysis model, wherein the preset prediction analysis model is a pre-trained machine learning model; and using the set of prediction optimization schemes to optimize the interface scrolling display.

[0017] According to one embodiment of this application, a display processing apparatus includes: an acquisition module, configured to: acquire raw frame data of frames displayed during an interface scrolling phase; a determination module, configured to: determine frame time consumption data based on the raw frame data, wherein the frame time consumption data is the time consumption data of the frame display process; and an analysis module, configured to: perform stuttering analysis based on the frame time consumption data to obtain stuttering conditions during the interface scrolling phase, so as to optimize the scrolling display of the interface based on the stuttering conditions.

[0018] According to another embodiment of this application, a storage medium stores a computer program thereon, which, when executed by the processor of a terminal device, causes the terminal device to perform the method described in the embodiments of this application.

[0019] According to another embodiment of this application, a terminal device may include: a memory storing a computer program; and a processor reading the computer program stored in the memory to execute the methods described in the embodiments of this application.

[0020] According to another embodiment of this application, a computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a terminal device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the terminal device to perform the methods provided in the various optional implementations described in the embodiments of this application.

[0021] In this embodiment, the original frame data of the frames displayed during the interface scrolling phase is obtained; based on the original frame data, frame time data is determined, wherein the frame time data is the time data consumed during the frame display process; based on the frame time data, stuttering analysis is performed to obtain the stuttering situation during the interface scrolling phase, so as to optimize the scrolling display of the interface based on the stuttering situation.

[0022] In this embodiment of the application, by identifying the interface scrolling stage and obtaining the original frame data of the frames displayed in the interface scrolling stage, and converting it into frame time data in the interface scrolling stage, and performing stuttering analysis based on the frame time data in the interface scrolling stage, the analysis of stuttering in the interface scrolling stage is more accurate, resulting in better optimization effect and shorter calculation and analysis time. This avoids negative impact on the original scrolling display process and effectively improves the optimization effect and reliability of interface scrolling display in interface scrolling scenarios. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart of a display processing method according to an embodiment of this application is shown.

[0025] Figure 2 A flowchart illustrating the stage identification process according to an embodiment of this application is shown.

[0026] Figure 3 A flowchart illustrating a stuttering determination process according to an embodiment of this application is shown.

[0027] Figure 4 A flowchart illustrating a display optimization process according to an embodiment of this application is shown.

[0028] Figure 5 A flowchart illustrating the element trimming process according to one embodiment of this application is shown.

[0029] Figure 6 A flowchart of a buffer expansion process according to an embodiment of this application is shown.

[0030] Figure 7 An animation closing flowchart according to an embodiment of this application is shown.

[0031] Figure 8 A block diagram of a display processing apparatus according to an embodiment of this application is shown.

[0032] Figure 9 A block diagram of a terminal device according to an embodiment of this application is shown. Detailed Implementation

[0033] The present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments provided herein are merely illustrative of the present disclosure and are not intended to limit the present disclosure. Furthermore, the embodiments provided below are some embodiments for implementing the present disclosure, and not all embodiments for implementing the present disclosure. Unless otherwise specified, the technical solutions described in the embodiments of the present disclosure can be implemented in any combination. It should be noted that, in the embodiments of this disclosure, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a method or apparatus that includes a list of elements includes not only the elements expressly described, but also other elements not expressly listed, or elements inherent to implementing the method or apparatus. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other related elements (e.g., steps in the method or units in the apparatus, such as portions of circuitry, processors, programs, or software, etc.) in the method or apparatus that includes that element. For example, the display processing method provided in the embodiments of this disclosure includes a series of steps, but the display processing method provided in the embodiments of this disclosure is not limited to the steps described. Similarly, the display processing apparatus provided in the embodiments of this disclosure includes a series of units, but the apparatus provided in the embodiments of this disclosure is not limited to the units explicitly described, but may also include units that need to be set up for obtaining relevant information or processing based on information. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of this disclosure. It is understood that in the specific implementation of this application, relevant data is involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards.

[0034] With the development of terminal devices, rich and clear UI interfaces and smooth user experience are important indicators of user experience. Currently, in order to achieve clearer UI interfaces, terminal devices are gradually incorporating features such as 4K UI and more complex interface layouts and animations. In scenarios with rapid screen changes and frequent screen scrolling, this leads to higher frame rendering and rendering times, resulting in more severe stuttering in the interface display during scrolling scenarios.

[0035] Currently, relevant optimization solutions typically analyze and optimize for stuttering based on all raw frame data provided by the system (such as Android). However, these solutions often neglect the scrolling phase of the interface display process, resulting in inaccurate analysis of stuttering during scrolling and poor optimization of the scrolling display. Furthermore, using all raw frame data for calculation and analysis can easily cause additional time consumption, negatively impacting the original scrolling display process and leading to weak optimization reliability.

[0036] To address these issues, this application provides a display processing solution that can effectively improve the optimization effect and reliability of interface scrolling display in scrolling scenarios.

[0037] The following is a detailed description of the relevant embodiments of the display processing solution provided in this application.

[0038] Figure 1 A flowchart illustrating a display processing method according to an embodiment of this application is shown. The execution subject of this display processing method can be a terminal device such as a television (TV), mobile phone, computer, smartwatch, virtual device, or in-vehicle device. For example, in one specific embodiment of this application, the execution subject of the display processing method is specifically located in a television. The television may include a processor and a memory, and the memory stores a computer program. Therefore, the processor in the television can read the computer program stored in the memory to execute the methods of the various embodiments of this application.

[0039] like Figure 1 As shown, the display processing method may include steps S110 to S130.

[0040] Step S110: Obtain the raw frame data of the frames displayed during the interface scrolling phase; Step S120: Determine the frame time data based on the original frame data. The frame time data is the time consumed during the frame display process. Step S130: Perform stuttering analysis based on frame time data to obtain stuttering conditions during the interface scrolling phase, and optimize the scrolling display of the interface based on the stuttering conditions.

[0041] During the interface display process, the interface scrolling phase is monitored and identified. The interface scrolling phase refers to the interface display period from the moment scrolling begins to the moment scrolling ends. For example, for a television interface, the trigger events (Input events) of the remote control's directional keys (such as DPAD_UP / DPAD_DOWN) can be listened to to identify a single interface scrolling phase. For example, pressing a directional key can be considered as the start of scrolling, and releasing a directional key or reaching the end of a timer can be considered as the end of scrolling.

[0042] The system (such as Android) records all raw frame data of the interface display process (all raw frame data includes the raw frame data of all frames displayed during the interface display process). The raw frame data may include frame data such as the start and end times of animation playback, the start and end times of buffer allocation, etc. In this embodiment, based on identifying the interface scrolling stage, a portion of the raw frame data of the interface scrolling stage is further obtained from the all raw frame data (the portion of raw frame data includes the raw frame data of the frames displayed during the interface scrolling stage).

[0043] Furthermore, based on the original frame data of the frames displayed during the interface scrolling phase, the frame time consumption data during the interface scrolling phase is determined, that is, the original frame data is converted into frame time consumption data. The frame time consumption data is the time consumption data of the frame display process. The frame time consumption data can include multiple time consumption data of the frame display process, such as animation time consumption, buffer request time consumption, buffer submission time consumption, single frame drawing time consumption, single frame rendering time consumption, etc.

[0044] Analyzing stuttering based on frame time data during the scrolling phase has several advantages. First, it effectively considers the scrolling phase, leading to a more accurate analysis of stuttering during this phase. This allows for better optimization of the scrolling display based on the stuttering situation. Second, using only frame time data for calculation and analysis avoids the extra time spent using all raw frame data, which could negatively impact the original scrolling process and improve optimization reliability.

[0045] Therefore, in this embodiment of the application, by identifying the interface scrolling stage and obtaining the original frame data of the frames displayed in the interface scrolling stage, and converting it into frame time data in the interface scrolling stage, and performing stuttering analysis based on the frame time data in the interface scrolling stage, the analysis of stuttering in the interface scrolling stage is more accurate, resulting in better optimization effect and shorter calculation and analysis time, avoiding negative impact on the original scrolling display process, and effectively improving the optimization effect and reliability of interface scrolling display in interface scrolling scenarios.

[0046] The following description Figure 1 Further optional specific embodiments are provided for each step performed during the display processing in the example.

[0047] In one embodiment, step S110, obtaining the original frame data of the frames displayed during the interface scrolling phase, includes: monitoring the first moment when the interface starts scrolling and the second moment when the scrolling ends, determining the time period between the first moment and the second moment as the interface scrolling phase; and obtaining the original frame data of the frames displayed during the interface scrolling phase.

[0048] The rolling state machine can monitor the first moment when rolling starts and the second moment when rolling ends, and the timer can be used to detect idle time and determine when to stop rolling.

[0049] For example, in one instance, it can be done as follows Figure 2Steps S210 to S2110 show the identification of the interface scrolling stage. Step S210: A key press is received; Step S220: Determine if a key press is pre-defined; if it is, it indicates a key press that triggers interface scrolling, and further execute step S230. Step S230: Check the scrolling state; Step S240: Determine if the scrolling is dynamic, i.e., determine if the scrolling state is dynamic, where dynamic scrolling means the interface is currently scrolling; if it is dynamic, execute step S250; otherwise, execute step S260. Step S250: Cancel the previous timer; Step S260: Set to dynamic scrolling and start a new timer; Step S270: Read the current system time as the first time; Step S280: Determine if the timer has been reached (e.g., whether it has reached 300ms); if not, execute step S290, wait for the next pre-defined key press or the timer to reach its limit; Step S2100: If the timer has reached its limit, read the current time as the second time; Step S2110: Set to a stopped state. Therefore, the first moment when scrolling begins and the second moment when scrolling ends can be determined, and the time period between the first moment and the second moment is defined as one interface scrolling phase.

[0050] See Figure 3 In one embodiment, step S130 involves performing stutter analysis based on frame time consumption data to obtain stuttering conditions during the interface scrolling phase, including: step S131, determining abnormal time consumption frames during the interface scrolling phase based on frame time consumption data; step S132, determining that stuttering occurs when the proportion of abnormal time consumption frames during the interface scrolling phase is greater than a predetermined proportion.

[0051] During the interface scrolling phase, multiple frames are displayed. Frames with abnormal single-frame drawing or rendering times are identified as time-abnormal frames. The ratio of the total number of time-abnormal frames (BadFrameCount) to the total number of frames (FrameCount) displayed during the interface scrolling phase represents the percentage of time-abnormal frames. FrameCount. When the percentage of frames with abnormally long execution times exceeds a predetermined percentage (e.g., 10%), it can be accurately determined that a stutter occurred during the scrolling phase of the interface.

[0052] Furthermore, in one embodiment, the frame time consumption data includes single-frame drawing time and single-frame rendering time; based on the frame time consumption data, determining abnormal frames in the interface scrolling stage includes: determining frames corresponding to single-frame drawing time or single-frame rendering time that exceed a predetermined time limit as abnormal frames.

[0053] For the i-th frame displayed during the interface scrolling phase, its single-frame rendering time doFrameCost can be determined based on its original frame data. i and single-frame rendering time drawFrameCosti In this context, the time consumed by drawing a single frame is equal to the time difference between the time when drawing is completed and the time when drawing begins, and the time consumed by rendering a single frame is equal to the time difference between the time when rendering is completed and the time when rendering begins.

[0054] If the time taken to draw or render a single frame of the i-th frame is greater than the predetermined time limit, it indicates that the drawing or rendering process of the i-th frame takes too long. Frames whose single-frame drawing or rendering time exceeds the predetermined time limit can be accurately identified as frames with abnormal time consumption.

[0055] For further details, please refer to [link / reference]. Figure 4 In one embodiment, after performing stuttering analysis based on frame time consumption data to obtain the stuttering situation during the interface scrolling stage, the method may further include: step S140, if the stuttering situation is stuttering, analyzing the frame time consumption data corresponding to the abnormal time consumption frame to obtain time consumption distribution data; step S150, determining the target optimization scheme set for the interface based on the time consumption distribution data; step S160, using the target optimization scheme set to optimize the scrolling display of the interface.

[0056] If stuttering occurs, it indicates that scrolling display optimization is needed to fix the stuttering. In this case, by analyzing the frame time data corresponding to the abnormally timed frames, time distribution data is obtained. This time distribution data reflects the distribution of time consumption at each stage of the frame time data. Based on this time distribution data, the stages requiring display optimization can be accurately determined, and thus the corresponding target optimization scheme set can be accurately identified. Using the target optimization scheme set to optimize the scrolling display of the interface can effectively improve the scrolling display.

[0057] In one embodiment, the frame time consumption data includes animation time consumption, buffer request time consumption, buffer commit time consumption, single-frame drawing time consumption, and single-frame rendering time consumption. Analysis is performed on the frame time consumption data corresponding to frames with abnormal time consumption to obtain time consumption distribution data, including: dividing the animation time consumption by the single-frame drawing time to obtain the animation time consumption percentage; dividing the buffer request time consumption by the single-frame rendering time to obtain the request time consumption percentage; and dividing the buffer commit time consumption by the single-frame rendering time to obtain the commit time consumption percentage. The time consumption distribution data includes the animation time consumption percentage, the request time consumption percentage, and the commit time consumption percentage.

[0058] For the i-th frame displayed during the interface scrolling phase, its animation duration (animationCost) can also be determined based on its original frame data. i Buffer request time dequeueBufferCost i Buffer commit time queueBufferCost i Single-frame rendering time doFrameCost iand single-frame rendering time drawFrameCost i Animation time is equal to the time difference between the end of the animation and the start of the animation. Buffer request time is equal to the time difference between the completion of buffer request and the start of buffer request. Buffer commit time is equal to the time difference between the completion of buffer commit and the start of buffer commit. Single frame drawing time is equal to the time difference between the completion of drawing and the start of drawing. Single frame rendering time is equal to the time difference between the completion of rendering and the start of rendering.

[0059] For the i-th frame, the percentage of animation time consumed animationCost i / doFrameCost i Request time percentage dequeueBufferCost i / drawFrameCost i Submission time percentage queueBufferCost i / drawFrameCost i The time consumption distribution data of the i-th frame includes the proportion of animation time consumption, the proportion of request time consumption, and the proportion of submission time consumption. This time consumption distribution data can reflect the time consumption distribution of the three main stages: animation playback stage, buffer allocation stage, and buffer submission stage. Based on this, the stages that need to be optimized in display can be accurately determined, and the corresponding target optimization scheme set can be accurately obtained.

[0060] The display of each frame during the interface scrolling phase can further include many sub-phases. In this embodiment of the application, one or more of the buffer allocation phase (dequeuebuffer phase), buffer submission phase (queueBuffer phase), and animation playback phase (animation phase) are optimized for display. The applicant found that this has a good optimization effect on the interface scrolling display in scrolling display scenarios, and can effectively avoid adverse effects on the interface scrolling display.

[0061] Furthermore, in one embodiment, the target optimization scheme set includes schemes for optimizing the display of a first set of display stages during scrolling of the interface. The first set of display stages includes one or more stages such as a buffer allocation stage, a buffer submission stage, and an animation playback stage. Determining the target optimization scheme set of the interface based on time consumption distribution data may specifically include: If the average first percentage of the animation time of all time-consuming abnormal frames in the interface scrolling phase is greater than the first preset threshold, then the first display phase set is determined to include the animation playback phase; if the average second percentage of the request time of all time-consuming abnormal frames in the interface scrolling phase is greater than the second preset threshold, then the first display phase set is determined to include the buffer allocation phase; if the average third percentage of the submission time of all time-consuming abnormal frames in the interface scrolling phase is greater than the third preset threshold, then the first display phase set is determined to include the buffer submission phase.

[0062] First percentage average The second percentage average The third percentage average , where N is the total number of all time-consuming abnormal frames during the interface scrolling phase.

[0063] Furthermore, .

[0064] in, Let be the first display stage set, 'a' be the first preset threshold, 'b' be the third preset threshold, and 'c' be the second preset threshold. When the average percentage of the first proportion is greater than the first preset threshold 'a', the first display stage set will include 1 (animation playback stage); when the average percentage of the second proportion is greater than the second preset threshold 'c', the first display stage set will include 3 (buffer allocation stage); when the average percentage of the third proportion is greater than the third preset threshold 'b', the first display stage set will include 2 (buffer submission stage).

[0065] The values ​​of a, b, and c can be set according to the actual situation. This application does not impose any special restrictions on them. For example, in one example, a, b, and c are all 0.7.

[0066] Furthermore, in one embodiment, when the first display stage set includes a buffer submission stage, optimizing the scrolling display of the interface using a target optimization scheme set may include: selecting elements to be clipped from the elements included in the interface; determining the region position of the clipping area of ​​the element to be clipped; and saving the region position of the clipping area so that the clipping area can be used for rendering after being clipped according to the region position when the interface is scrolling.

[0067] When the first display stage set includes a buffer submission stage, the target optimization scheme set includes the "element pruning" scheme that optimizes the display of that buffer submission stage. Using the "element pruning" scheme, firstly, elements to be pruned are selected from the elements included in the interface. Then, the location of the pruning region of the element to be pruned is determined and saved. Subsequently, when the interface scrolls, the pruning region can be pruned based on its location, and the remaining element region can be used for rendering.

[0068] It is understandable that when the first display stage set does not include the buffer commit stage, the target optimization scheme set will not include the "element pruning" scheme that performs display optimization on the buffer commit stage, nor will it use the "element pruning" scheme for pruning.

[0069] After each frame is drawn, drawing instructions are sent to the GPU for rendering. By cropping a portion of the elements to be cropped, the remaining area is used by producers (such as rendering threads) to fill the rendering buffer, reducing the rendering time for each frame and preventing excessively long rendering times. The buffer submission phase (queueBuffer phase) submits the rendering-filled buffer to the compositing queue. Reducing rendering time speeds up the submission of the rendering-filled buffer to the compositing queue, thereby reducing the buffer submission time in the buffer submission phase and lowering the probability of stuttering during screen scrolling. The compositing queue is used by consumers (SurfaceFlinger) for compositing and display.

[0070] Furthermore, in one embodiment, determining the region location of the clipping area of ​​the element to be clipped includes: determining the invisible region of the element to be clipped as the clipping region; calculating the region location of the invisible region to obtain the region location of the clipping region.

[0071] The interface involves the overlapping rendering of elements (such as rectangular elements and rounded rectangle elements). Therefore, some elements will have invisible areas obscured by their parent elements. These invisible areas are not visible to the user but are still included in the rendering process. By identifying elements obscured by their parent elements as those to be clipped, and saving the location of these invisible areas as the clipping area, the user-visible area (the remaining element area) can be obtained by clipping the invisible areas during the interface's scrolling display. This approach does not affect the user experience and reduces rendering time, reliably reducing buffer commit time during the buffer commit phase.

[0072] Optionally, in other embodiments, determining the location of the cropping region of the element to be cropped may include: determining the user-specified element as the element to be cropped and determining the specified region of the user-specified element to be cropped as the cropping region.

[0073] See Figure 5 In one example, the process of calculating and obtaining the location of the cropped area may include steps S310 to S3110.

[0074] Step S310: Traverse the rendering elements, i.e., traverse the elements to be rendered in the interface from top to bottom; Step S320: Determine whether the traversed element is a clipping element. A clipping element can be an element with an invisible area that is obscured by the upper element; Step S330: Determine whether the clipping element is transparent; if not (i.e., opaque), proceed to Step S340: Add a clipping label, i.e., label the opaque clipping element as an "element to be clipped", which can avoid clipping transparent elements and causing abnormal interface display; Step S350: Traverse in reverse, i.e., traverse the elements from bottom to top; Step S360: Create clipping region A, i.e., create the region corresponding to the traversed element as clipping region A; Step S370: Calculate the intersection of the upper element (i.e., the element in the layer above the traversed element) and clipping region A in the interface; Step S380: Determine if the intersection is empty? If step S390 is not executed, save the key-value pairs, that is, use the traversed elements as keys and the intersection as values, and save them in key-value pair form; step S3110, determine if the traversed elements have labels; if so, execute step S3110, the union of the elements and A, that is, the key-value pairs of the traversed elements (the intersection is the location of the clipping region) and A (the area of ​​the remaining elements after clipping) are saved together. In this way, it can be determined that elements that are obscured by the upper-layer elements and are not transparent are identified as elements to be clipped, and the location of the invisible area is saved as the location of the clipping region.

[0075] Furthermore, in one embodiment, when the first display stage set includes a buffer allocation stage, a target optimization scheme set is used to optimize the scrolling display of the interface, including: determining the number of buffers corresponding to the interface; and expanding the number of buffers.

[0076] When the first display stage set includes a buffer allocation stage (dequeuebuffer stage), the target optimization scheme set includes the "buffer adjustment" scheme for display optimization of that buffer allocation stage. Using the "buffer adjustment" scheme, the number of buffers corresponding to the interface is first determined (i.e., the total number of buffers waiting to be rotated in the buffer queue), and then the number of buffers corresponding to the interface is expanded to obtain the expanded number of buffers.

[0077] CPU drawing and GPU rendering are usually performed asynchronously. CPU drawing and GPU rendering rely on buffer queues for data exchange. Among them, the buffer allocation time in the buffer allocation phase (dequeuebuffer phase) is an important indicator affecting whether the buffer can be submitted to the GPU in a timely manner to start rendering. A long buffer allocation time will result in long idle time for the GPU and it will be unable to perform high-throughput rendering work.

[0078] Expanding the number of buffers increases the total number of buffers in the corresponding buffer queue of the interface, thereby improving the efficiency of requesting buffers from the buffer queue and reducing the buffer request time in the buffer request stage (dequeuebuffer stage). This allows for more efficient request of buffers to be submitted to the GPU to start rendering, thus reducing the probability of stuttering when the interface scrolls.

[0079] Furthermore, in one embodiment, the number of expanded buffers can be limited to less than or equal to a predetermined first number (e.g., 5) to avoid excessive memory resource consumption.

[0080] For example, in one instance, the process of expanding the number of buffers may include Figure 6 The steps S410 to S450 are shown. In step S410, the first display stage set includes a buffer allocation stage (dequeuebuffer stage); in step S420, the number of buffers corresponding to the interface is determined; in step S430, it is determined whether the number of buffers is less than a predetermined first number (e.g., 5). If so, proceed to step S440 to allocate an expanded number of buffers so that the expanded number of buffers is equal to the predetermined first number.

[0081] In addition, in some embodiments, when the first display stage set does not include the buffer allocation stage (dequeuebuffer stage), the target optimization scheme set does not include the "buffer adjustment" scheme for display optimization of the buffer allocation stage. In this case, if the number of buffers corresponding to the interface is greater than a predetermined second number (such as 3), the number of buffers can be adjusted to the predetermined second number to avoid invalid resource occupation.

[0082] Furthermore, in one embodiment, when the first display stage set includes an animation playback stage, using a target optimization scheme set to optimize the scrolling display of the interface may include: determining whether the animation closing condition is met; if the animation closing condition is met, then closing the animation displayed on the interface during scrolling display.

[0083] When the first display stage set includes an animation playback stage, the target optimization scheme set includes the "animation off" scheme, which optimizes the display of that animation playback stage. Using the "animation off" scheme, if the animation off condition is met, the scrolling animation displayed when the interface is scrolling is turned off, and subsequently, the scrolling animation applied during frame rendering is canceled.

[0084] When the interface scrolls, the application may use custom scrolling animations. Excessively long scrolling animations can cause the rendering process to time out, which in turn blocks the rendering process, resulting in stuttering during screen scrolling. By disabling the animation displayed during screen scrolling when the conditions for disabling animations are met, the time consumed during the animation playback phase (i.e., animation duration) can be reduced, thereby lowering the probability of stuttering during screen scrolling.

[0085] Furthermore, in one embodiment, determining whether the animation shutdown condition is met may include: playing an animation shutdown prompt message; and determining that the animation shutdown condition is met when a shutdown consent instruction is received.

[0086] An animated closing prompt message (such as "Lag detected, scrolling animation will be turned off soon") can be played via voice or pop-up window. Users can choose whether to agree to turn it off. If they agree, the terminal device will receive an instruction message indicating that the animation closing conditions are met. Then, the terminal device will determine that the animation closing conditions are met and turn off the animation displayed when the interface is scrolling. This can further avoid affecting the user experience or needs.

[0087] Furthermore, in some embodiments, before displaying the animation closing prompt message, it can be determined whether "the target optimization scheme set includes the animation playback stage for a consecutive number of times" and "more than a predetermined time has passed since the user last disagreed to close the animation". If "the target optimization scheme set includes the animation playback stage for a consecutive number of times" and "more than a predetermined time has passed since the user last disagreed to close the animation", then the animation closing prompt message is displayed, which can further improve the user experience.

[0088] For example, in one instance, a process employing an "animated shutdown" scheme might include... Figure 7 Steps S510 to S590 are shown below. Step S510: The first display stage set includes an animation playback stage. Step S520: The number of occurrences of the animation playback stage is animationCount = animationCount + 1. Step S530: Determine if animationCount is greater than a predetermined number. If yes (i.e., "the target optimization scheme set includes an animation playback stage for a consecutive predetermined number of times"), proceed to step S540: Determine if the time elapsed since the user last disagreed to close the animation exceeds a predetermined duration. If yes, proceed to step S550: Play the animation close prompt message. Proceed to step S560: Determine if the user agrees. If yes, proceed to step S570: Close the animation. If no, proceed to step S580: Record the time of this disagreement. Step S590: End.

[0089] The foregoing content describes relevant embodiments of "analyzing stuttering based on the raw frame data of the frames displayed during the interface scrolling phase, and dynamically optimizing the scrolling display in real time based on the stuttering situation." The following further describes how, at a predetermined time before the interface starts scrolling, the terminal device can further employ a set of predictive optimization schemes to optimize the scrolling display of the interface in advance.

[0090] In one embodiment, before obtaining the original frame data of the frames displayed during the interface scrolling phase, the method further includes obtaining a set of prediction optimization schemes corresponding to the interface. The set of prediction optimization schemes is obtained by predicting and analyzing historical optimization data of the interface scrolling display using a preset prediction analysis model. The preset prediction analysis model is a pre-trained machine learning model. The set of prediction optimization schemes is used to optimize the interface scrolling display.

[0091] For the user interface, historical optimization data on scrolling display optimizations is collected. This historical optimization data effectively reflects the past optimization status during the scrolling phase. By using a pre-defined predictive analysis model to predict and analyze the historical optimization data, a set of effective predictive optimization schemes for scrolling display optimization can be obtained. The pre-defined predictive analysis model can be deployed on terminal devices or in the cloud.

[0092] The preset predictive analysis model is a pre-trained machine learning model, which can be a large model, a decision tree model, a neural network model, or other machine learning models.

[0093] When the interface is created or at a predetermined time before the interface starts scrolling, the terminal device can obtain a set of prediction optimization schemes output by a preset predictive analysis model, and can use the set of prediction optimization schemes to optimize the scrolling display of the interface. This allows for effective scrolling display optimization in advance before the interface scrolls, resulting in a better scrolling display effect.

[0094] The preset predictive analysis model is a pre-trained machine learning model. In one example, the preset predictive analysis model can be a large model that has been fine-tuned and trained beforehand. In other examples, the preset predictive analysis model can be other machine learning models such as deep neural networks or decision trees.

[0095] Therefore, at a predetermined point in time, either when the interface is created or before it begins scrolling, the terminal device can pre-optimize the scrolling display using a predictive optimization scheme set. This predictive optimization scheme set is derived from historical optimization data of the interface's scrolling display using a pre-defined predictive analysis model, resulting in a good optimization effect for the scrolling display. Consequently, the overall optimization effect of the interface scrolling display can be effectively improved. Furthermore, after pre-optimizing the scrolling display using the predictive optimization scheme set, further dynamic real-time optimization can be performed by judging the lag situation. Combining pre-optimization with dynamic real-time optimization can further enhance the overall scrolling display optimization effect.

[0096] Furthermore, in one embodiment, the historical optimization data includes set data corresponding to multiple historical optimization scheme sets; before obtaining the predicted optimization scheme set corresponding to the interface, it may further include: using a preset predictive analysis model to perform predictive analysis based on the set data corresponding to multiple historical optimization scheme sets to obtain a score for each historical optimization scheme set; and determining the historical optimization scheme set corresponding to the score that meets the predetermined conditions as the predicted optimization scheme set.

[0097] For the interface, historical optimization data on scrolling display optimizations are collected. This historical optimization data effectively reflects the historical optimization status during the interface scrolling phase. The historical optimization data includes set data corresponding to multiple historical optimization scheme sets. Each historical optimization scheme set can include schemes for optimizing the display of the interface during the third display phase set during scrolling. The third display phase set can include one or more phases such as the buffer allocation phase, buffer submission phase, and animation playback phase.

[0098] A pre-defined predictive analysis model is used to perform predictive analysis based on the aggregate data corresponding to multiple historical optimization scheme sets, obtaining a score for each historical optimization scheme set. The higher the score, the better the optimization effect on the interface scrolling display. The historical optimization scheme sets corresponding to scores that meet predetermined conditions are determined as the predictive optimization scheme sets, thus obtaining the predictive optimization scheme sets that effectively optimize the interface scrolling display. The predetermined condition can be the highest score; that is, the historical optimization scheme set with the highest score can be determined as the predictive optimization scheme set.

[0099] Furthermore, after each set of target optimization solutions for the interface is determined, both the previously determined sets and the currently determined sets can be compiled into historical optimization solution sets, thus continuously collecting these historical optimization solution sets. A pre-set predictive analysis model can then be used to dynamically predict and analyze based on the continuously collected historical optimization solution set data, thereby refreshing the predictive optimization solution set and further improving its optimization effect.

[0100] Furthermore, in one embodiment, the aggregate data includes the frequency of occurrence of each historical optimization scheme set and the average rendering time after applying each historical optimization scheme set; predictive analysis is performed based on the aggregate data corresponding to multiple historical optimization scheme sets to obtain the score of each historical optimization scheme set, including: using a weighted calculation layer, based on the function W S = We perform weight calculations to obtain the set weights W corresponding to each set of historical optimization schemes S. S A scoring layer is used, based on a function. The scores are calculated to obtain the scores corresponding to each set of historical optimization solutions S. Where α, β, and θ are the model parameters of the preset predictive analysis model. This refers to the number of times the historical optimal solution set S appears. This refers to the average rendering time after applying the historical optimization scheme set S. Given inputs x and θ, S represents the probability of S occurring, where x = [i, S], and x refers to the combination of interface i and the set of historical optimization schemes S.

[0101] The dataset includes the number of occurrences of each historical optimization scheme set S when optimizing the scrolling display for interface i. And the average rendering time after applying each historical optimization scheme set S. Specifically, for each historical optimization scheme set S, after each time the historical optimization scheme set S is applied to optimize the scrolling display of interface i, the single-frame rendering time (DoFrameCost) of the i-th frame displayed in one scrolling phase of interface i can be calculated. i The average time taken to render a single frame during the scrolling phase of this interface is the average time taken to render a single frame (n frames). Then, the number of occurrences of the historical optimization solution set S. ,mean The set of historical optimization schemes S applied in the second application can then exist. The average time for each different single rendering session The average time taken for a single drawing is the average drawing time after applying the historical optimization scheme set S. Multiple historical optimization scheme sets S can be used to calculate their respective average drawing times.

[0102] The pre-trained model parameters in the preset predictive analysis model include α, β, and θ. Among them, x refers to the combination of interface i and the historical optimization scheme set S. x is input into the preset predictive analysis model in the form of feature vectors. Let S be the probability of occurrence of the set of historical optimization solutions given inputs x and θ. In the pre-defined predictive analysis model, the weight calculation layer is based on the function W. S = We perform weight calculations to obtain the set weights W corresponding to each set of historical optimization schemes S. S The scoring layer is based on the function. The scores are calculated to obtain the scores corresponding to each set of historical optimization solutions S. .score It can accurately reflect the optimization effect of the historical optimization scheme set S on the interface scrolling display.

[0103] Furthermore, in one embodiment, the predictive optimization scheme set includes a scheme for optimizing the display of a second set of display stages when the interface is scrolling. The second set of display stages includes one or more stages such as a buffer allocation stage, a buffer submission stage, and an animation playback stage. Optimizing the scrolling display of the interface using the predictive optimization scheme set may include: using a scheme from the predictive optimization scheme set to optimize the display of each stage in the second set of display stages when the interface is scrolling.

[0104] The display of each frame during the interface scrolling phase can further include many sub-phases. In this embodiment of the application, one or more of the buffer allocation phase (dequeuebuffer phase), buffer submission phase (queueBuffer phase), and animation playback phase (animation phase) are optimized for display. The applicant found that this has a good optimization effect on the interface scrolling display in scrolling display scenarios, and can effectively avoid adverse effects on the interface scrolling display.

[0105] The predictive optimization scheme set includes schemes for optimizing the display of the second display stage set during the scrolling display of the interface. This second display stage set includes one or more stages such as the buffer allocation stage, buffer submission stage, and animation playback stage, with each stage having a corresponding scheme. By employing the schemes from the predictive optimization scheme set, display optimization can be performed separately for each stage within the second display stage set during the interface scrolling display, thereby achieving a good optimization effect for the interface scrolling display.

[0106] Furthermore, in some embodiments, the prediction optimization scheme set may include schemes for display optimization at all stages of the interface during scrolling, and not just limited to the buffer allocation stage, buffer submission stage, and animation playback stage.

[0107] Furthermore, in one embodiment, when the second display stage set includes a buffer submission stage, a predictive optimization scheme is adopted to perform display optimization for each stage in the second display stage set when the interface is scrolling. This may include: selecting elements to be clipped from the elements included in the interface; determining the region position of the clipping area of ​​the elements to be clipped; and saving the region position of the clipping area so that it can be used for rendering after clipping the region according to the region position when the interface is scrolling.

[0108] When the second display stage set includes a buffer submission stage, the prediction optimization scheme set includes the "element pruning" scheme for display optimization of that buffer submission stage. The "element pruning" scheme first filters out elements to be pruned from the elements included in the interface. Then, it determines and saves the location of the pruning region for each element. Later, when the interface scrolls, the pruning region can be pruned based on its location, and the remaining element area can be used for rendering.

[0109] It is understandable that when the second display stage set does not include the buffer submission stage, the prediction optimization scheme set will not include the "element pruning" scheme that performs display optimization on the buffer submission stage, nor will it use the "element pruning" scheme for pruning.

[0110] After each frame is drawn, drawing instructions are sent to the GPU for rendering. By cropping a portion of the elements to be cropped, the remaining area is used by producers (such as rendering threads) to fill the rendering buffer, reducing the rendering time for each frame and preventing excessively long rendering times. The buffer submission phase (queueBuffer phase) submits the rendering-filled buffer to the compositing queue. Reducing rendering time speeds up the submission of the rendering-filled buffer to the compositing queue, thereby reducing the buffer submission time in the buffer submission phase and lowering the probability of stuttering during screen scrolling. The compositing queue is used by consumers (SurfaceFlinger) for compositing and display.

[0111] Furthermore, in one embodiment, determining the region location of the clipping area of ​​the element to be clipped may include: determining the invisible region of the element to be clipped as the clipping region; calculating the region location of the invisible region to obtain the region location of the clipping region.

[0112] The interface involves the overlapping rendering of elements (such as rectangular elements and rounded rectangle elements). Therefore, some elements will have invisible areas obscured by their parent elements. These invisible areas are not visible to the user but are still included in the rendering process. By identifying elements obscured by their parent elements as those to be clipped, and saving the location of these invisible areas as the clipping area, the user-visible area (the remaining element area) can be obtained by clipping the invisible areas during the interface's scrolling display. This approach does not affect the user experience and reduces rendering time, reliably reducing buffer commit time during the buffer commit phase.

[0113] Optionally, in other embodiments, determining the location of the cropping region of the element to be cropped may include: determining the user-specified element as the element to be cropped and determining the specified region of the user-specified element to be cropped as the cropping region.

[0114] Furthermore, in one embodiment, when the second display stage set includes a buffer application stage, a prediction optimization scheme is adopted to perform display optimization for each stage in the second display stage set when the interface is scrolling, including: determining the number of buffers corresponding to the interface; and expanding the number of buffers.

[0115] When the second display stage set includes a buffer allocation stage (dequeuebuffer stage), the prediction optimization scheme set includes the "buffer adjustment" scheme for display optimization of that buffer allocation stage. Using the "buffer adjustment" scheme, the number of buffers corresponding to the interface is first determined (i.e., the total number of buffers waiting to be rotated in the buffer queue), and then the number of buffers corresponding to the interface is expanded to obtain the expanded number of buffers.

[0116] CPU drawing and GPU rendering are usually performed asynchronously. CPU drawing and GPU rendering rely on buffer queues for data exchange. Among them, the buffer allocation time in the buffer allocation phase (dequeuebuffer phase) is an important indicator affecting whether the buffer can be submitted to the GPU in a timely manner to start rendering. A long buffer allocation time will result in long idle time for the GPU and it will be unable to perform high-throughput rendering work.

[0117] Expanding the number of buffers increases the total number of buffers in the corresponding buffer queue of the interface, thereby improving the efficiency of requesting buffers from the buffer queue and reducing the buffer request time in the buffer request stage (dequeuebuffer stage). This allows for more efficient request of buffers to be submitted to the GPU to start rendering, thus reducing the probability of stuttering when the interface scrolls.

[0118] Furthermore, in one embodiment, the number of expanded buffers can be limited to less than or equal to a predetermined first number (e.g., 5) to avoid excessive memory resource consumption. Additionally, in some embodiments, when the second display stage set does not include a buffer allocation stage (dequeuebuffer stage), the prediction optimization scheme set does not include a "buffer adjustment" scheme that performs display optimization on that buffer allocation stage.

[0119] Furthermore, in one embodiment, when the second display stage set includes an animation playback stage, a predictive optimization scheme is adopted to perform display optimization for each stage in the second display stage set when the interface is scrolling, including: determining whether the animation closing condition is met; if the animation closing condition is met, then closing the animation displayed when the interface is scrolling.

[0120] When the second display stage set includes an animation playback stage, the prediction optimization scheme set includes the "animation off" scheme, which optimizes the display of that animation playback stage. Using the "animation off" scheme, if the animation off condition is met, the scrolling animation displayed when the interface is scrolling is turned off, and subsequently, the scrolling animation applied during frame rendering is canceled.

[0121] When the interface scrolls, the application may use custom scrolling animations. Excessively long scrolling animations can cause the rendering process to time out, which in turn blocks the rendering process, resulting in stuttering during screen scrolling. By disabling the animation displayed during screen scrolling when the conditions for disabling animations are met, the time consumed during the animation playback phase (i.e., animation duration) can be reduced, thereby lowering the probability of stuttering during screen scrolling.

[0122] Furthermore, in one embodiment, determining whether the animation shutdown condition is met may include: playing an animation shutdown prompt message; and determining that the animation shutdown condition is met when a shutdown consent instruction is received.

[0123] An animated closing prompt message (such as "Lag detected, scrolling animation will be turned off soon") can be played via voice or pop-up window. Users can choose whether to agree to turn it off. If they agree, the terminal device will receive an instruction message indicating that the animation closing conditions are met. Then, the terminal device will determine that the animation closing conditions are met and turn off the animation displayed when the interface is scrolling. This can further avoid affecting the user experience or needs.

[0124] Furthermore, in some embodiments, before displaying the animation playback shutdown prompt message, it can be determined whether "the prediction optimization scheme set includes the animation playback stage for a consecutive number of times" and "more than a predetermined time has passed since the user last disagreed to close the animation". If "the prediction optimization scheme set includes the animation playback stage for a consecutive number of times" and "more than a predetermined time has passed since the user last disagreed to close the animation", then the animation playback shutdown prompt message is displayed, which can further improve the user experience.

[0125] To facilitate better implementation of the display processing method provided in the embodiments of this application, the embodiments of this application also provide a display processing apparatus based on the above-described display processing method. The meanings of the terms used are the same as in the above-described display processing method, and specific implementation details can be found in the descriptions in the method embodiments. Figure 8 A block diagram of a display processing apparatus according to an embodiment of this application is shown.

[0126] like Figure 8 As shown, the display processing device 600 may include: an acquisition module 610, which can be used to: acquire the original frame data of the frames displayed during the interface scrolling phase; a determination module 620, which can be used to: determine the frame time consumption data based on the original frame data, wherein the frame time consumption data is the time consumption data of the frame display process; and an analysis module 630, which can be used to: perform stuttering analysis based on the frame time consumption data to obtain the stuttering situation during the interface scrolling phase, so as to optimize the scrolling display of the interface based on the stuttering situation.

[0127] In some embodiments of this application, when performing stuttering analysis based on the frame time consumption data to obtain the stuttering situation in the interface scrolling stage, the analysis module can be used to: determine the time-consuming abnormal frames in the interface scrolling stage based on the frame time consumption data; when the frame proportion of the time-consuming abnormal frames in the interface scrolling stage is greater than a predetermined proportion, then determine that the stuttering situation has occurred.

[0128] In some embodiments of this application, the frame time consumption data includes single-frame drawing time and single-frame rendering time; when determining abnormal frames in the interface scrolling stage based on the frame time consumption data, the analysis module can be used to: determine the frames corresponding to the single-frame drawing time or the single-frame rendering time that are greater than a predetermined time limit as the abnormal frames.

[0129] In some embodiments of this application, after performing stuttering analysis based on the frame time consumption data to obtain the stuttering situation during the interface scrolling stage, an optimization module is further included, which can be used to: if the stuttering situation is stuttering, analyze the frame time consumption data corresponding to the time-abnormal frame to obtain time consumption distribution data; determine the target optimization scheme set for the interface based on the time consumption distribution data; and use the target optimization scheme set to optimize the scrolling display of the interface.

[0130] In some embodiments of this application, the frame time consumption data includes animation time consumption, buffer request time consumption, buffer commit time consumption, single-frame drawing time consumption, and single-frame rendering time consumption. When analyzing the frame time consumption data corresponding to the time consumption abnormal frames to obtain time consumption distribution data, the optimization module can be used to: divide the animation time consumption by the single-frame drawing time consumption to obtain the animation time consumption ratio; divide the buffer request time consumption by the single-frame rendering time consumption to obtain the request time consumption ratio; divide the buffer commit time consumption by the single-frame rendering time consumption to obtain the commit time consumption ratio; the time consumption distribution data includes the animation time consumption ratio, the request time consumption ratio, and the commit time consumption ratio.

[0131] In some embodiments of this application, the target optimization scheme set includes a scheme for optimizing the display of a first set of display stages when the interface is scrolling. The first set of display stages includes one or more stages such as a buffer request stage, a buffer submission stage, and an animation playback stage. When determining the target optimization scheme set of the interface based on the time consumption distribution data, the optimization module can be used to: determine that the first set of display stages includes the animation playback stage when the first average proportion of the animation time consumption ratio of all time-consuming abnormal frames in the interface scrolling stage is greater than a first preset threshold; determine that the first set of display stages includes the buffer request stage when the second average proportion of the request time consumption ratio of all time-consuming abnormal frames in the interface scrolling stage is greater than a second preset threshold; and determine that the first set of display stages includes the buffer submission stage when the third average proportion of the submission time consumption ratio of all time-consuming abnormal frames in the interface scrolling stage is greater than a third preset threshold.

[0132] In some embodiments of this application, when the first display stage set includes the buffer submission stage, when the target optimization scheme set is used to optimize the scrolling display of the interface, the optimization module can be used to: filter out elements to be clipped from the elements included in the interface; determine the region position of the clipping area of ​​the element to be clipped; save the region position of the clipping area so that the clipping area can be clipped according to the region position and used for rendering when the interface is scrolling.

[0133] In some embodiments of this application, when determining the region position of the cropping region of the element to be cropped, the optimization module can be used to: determine the invisible region of the element to be cropped as the cropping region; calculate the region position of the invisible region to obtain the region position of the cropping region.

[0134] In some embodiments of this application, when the first display stage set includes the buffer application stage, when the target optimization scheme set is used to optimize the scrolling display of the interface, the optimization module can be used to: determine the number of buffers corresponding to the interface; and expand the number of buffers.

[0135] In some embodiments of this application, when the first display stage set includes the animation playback stage, when the target optimization scheme set is used to optimize the scrolling display of the interface, the optimization module can be used to: determine whether the animation closing condition is met; if the animation closing condition is met, then turn off the animation displayed on the interface when the interface is scrolling.

[0136] In some embodiments of this application, when determining whether the animation closing condition is met, the optimization module can be used to: play an animation closing prompt message; and when receiving an indication to agree to close the animation, determine that the animation closing condition is met.

[0137] In some embodiments of this application, before obtaining the original frame data of the frames displayed during the interface scrolling phase, a pre-repair module is further included, which can be used to: obtain a set of prediction optimization schemes corresponding to the interface, wherein the set of prediction optimization schemes is obtained by predicting and analyzing historical optimization data of the interface scrolling display using a preset prediction analysis model, wherein the preset prediction analysis model is a pre-trained machine learning model; and use the set of prediction optimization schemes to optimize the interface scrolling display.

[0138] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0139] Furthermore, embodiments of this application also provide a terminal device, such as... Figure 9 As shown, Figure 9 A block diagram of a terminal device according to an embodiment of this application is shown, specifically: The terminal device may include components such as a processor 701 with one or more processing cores, a memory 702 with one or more computer-readable storage media, a power supply 703, and an input unit 704. Those skilled in the art will understand that... Figure 9 The terminal device structure shown does not constitute a limitation on the terminal device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 701 is the control center of the terminal device, connecting various parts of the computer device via various interfaces and lines. It executes various functions and processes data by running or executing software programs and / or modules stored in the memory 702, and by calling data stored in the memory 702. Optionally, the processor 701 may include one or more processing cores; preferably, the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user page, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 701.

[0140] The memory 702 can be used to store software programs and modules. The processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 702 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 702 may also include a memory controller to provide the processor 701 with access to the memory 702.

[0141] The terminal device also includes a power supply 703 that supplies power to the various components. Preferably, the power supply 703 can be logically connected to the processor 701 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 703 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0142] The terminal device may also include an input unit 704, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0143] Although not shown, the terminal device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 701 in the terminal device can load the executable files corresponding to the processes of one or more computer programs into the memory 702 according to the following instructions, and the processor 701 runs the computer programs stored in the memory 702, thereby realizing the various functions in the foregoing embodiments of this application.

[0144] For example, processor 701 can perform the following: obtain the original frame data of the frames displayed during the interface scrolling phase; determine the frame time consumption data based on the original frame data, wherein the frame time consumption data is the time consumption data of the frame display process; perform stuttering analysis based on the frame time consumption data to obtain the stuttering situation during the interface scrolling phase, so as to optimize the scrolling display of the interface based on the stuttering situation.

[0145] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by a computer program, or by a computer program controlling related hardware. The computer program can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0146] Therefore, embodiments of this application also provide a storage medium storing a computer program that can be loaded by a processor to execute the steps in any of the methods provided in embodiments of this application.

[0147] The storage medium can be a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0148] Since the computer program stored in the storage medium can execute the steps of any of the methods provided in the embodiments of this application, the beneficial effects that the methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0149] According to another embodiment of this application, a computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a terminal device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the terminal device to perform the methods provided in the various optional implementations described in the embodiments of this application.

[0150] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0151] It should be understood that this application is not limited to the embodiments described above and shown in the accompanying drawings, but various modifications and changes can be made without departing from its scope.

Claims

1. A display processing method, characterized in that, include: Get the raw frame data of the frames displayed during the interface scrolling phase; Based on the original frame data, the frame time consumption data is determined, which is the time consumption data of the frame display process; Based on the frame time data, a stuttering analysis is performed to obtain the stuttering situation during the interface scrolling phase, so as to optimize the scrolling display of the interface according to the stuttering situation.

2. The method according to claim 1, characterized in that, The step of performing stutter analysis based on the frame time data to obtain the stuttering situation during the interface scrolling phase includes: Based on the frame time consumption data, identify the time-consuming abnormal frames in the interface scrolling phase; If the proportion of time-consuming abnormal frames during the interface scrolling phase is greater than a predetermined proportion, then the stuttering situation is determined to be stuttering.

3. The method according to claim 2, characterized in that, The frame time consumption data includes the single-frame drawing time and the single-frame rendering time; determining the time-consuming abnormal frames in the interface scrolling phase based on the frame time consumption data includes: The frames whose single-frame drawing time or single-frame rendering time exceeds the predetermined limit are identified as the time-abnormal frames.

4. The method according to claim 1, characterized in that, After performing stuttering analysis based on the frame time data to obtain the stuttering situation during the interface scrolling phase, the method further includes: If the lag is a lag, analyze the frame time data corresponding to the abnormal time frame to obtain time distribution data; Determine the target optimization scheme set for the interface based on the time consumption distribution data; The target optimization scheme set is used to optimize the scrolling display of the interface.

5. The method according to claim 4, characterized in that, The frame time consumption data includes animation time consumption, buffer request time consumption, buffer commit time consumption, single frame drawing time consumption, and single frame rendering time consumption. The step of analyzing the frame time consumption data corresponding to the time-consuming abnormal frames to obtain time consumption distribution data includes: Divide the animation time by the single-frame rendering time to obtain the animation time percentage; Divide the buffer request time by the single-frame rendering time to obtain the request time percentage. Divide the buffer submission time by the single-frame rendering time to obtain the submission time percentage; the time distribution data includes the animation time percentage, the request time percentage and the submission time percentage.

6. The method according to claim 5, characterized in that, The target optimization scheme set includes schemes for optimizing the display of the interface during scrolling, wherein the first display stage set includes one or more stages such as buffer allocation stage, buffer submission stage, and animation playback stage; The step of determining the target optimization scheme set for the interface based on the time consumption distribution data includes: When the average first percentage of the animation time consumption of all time-consuming abnormal frames in the interface scrolling phase is greater than the first preset threshold, it is determined that the first display phase set includes the animation playback phase. When the average of the second percentage of the request time of all time-consuming abnormal frames in the interface scrolling phase is greater than the second preset threshold, it is determined that the buffer application phase is included in the first display phase set. When the average third percentage of the submission time of all time-consuming abnormal frames in the interface scrolling phase is greater than the third preset threshold, it is determined that the first display phase set includes the buffer submission phase.

7. The method according to claim 6, characterized in that, When the first display stage set includes the buffer submission stage, the step of optimizing the scrolling display of the interface using the target optimization scheme set includes: Select the elements to be cropped from the elements included in the interface; Determine the location of the clipping region for the element to be clipped; The location of the cropped area is saved so that it can be cropped based on the location when the interface is scrolling and then used for rendering.

8. The method according to claim 7, characterized in that, Determining the location of the cropping region for the element to be cropped includes: The invisible area of ​​the element to be clipped is defined as the clipping area; Calculate the location of the invisible area to obtain the location of the cropped area.

9. The method according to claim 6, characterized in that, When the first display stage set includes the buffer allocation stage, the step of optimizing the scrolling display of the interface using the target optimization scheme set includes: Determine the number of buffers corresponding to the interface; The number of buffers is increased.

10. The method according to claim 6, characterized in that, When the first display stage set includes the animation playback stage, the step of optimizing the scrolling display of the interface using the target optimization scheme set includes: Determine if the animation shutdown conditions are met; If the animation shutdown condition is met, then the animation displayed on the interface during scrolling is turned off.

11. The method according to claim 10, characterized in that, Determining whether the animation shutdown condition is met includes: The animation playback closed notification message; When a message indicating agreement to close is received, it is determined that the conditions for closing the animation are met.

12. The method according to any one of claims 1 to 11, characterized in that, Before acquiring the raw frame data of the frames displayed during the interface scrolling phase, the method further includes: Obtain the prediction optimization scheme set corresponding to the interface. The prediction optimization scheme set is obtained by predicting and analyzing historical optimization data of the interface scrolling display optimization using a preset prediction analysis model. The preset prediction analysis model is a pre-trained machine learning model. The predicted optimization scheme set is used to optimize the scrolling display of the interface.

13. A display processing device, characterized in that, include: The acquisition module is used to: acquire the raw frame data of the frames displayed during the interface scrolling phase; The determining module is used to: determine frame time data based on the original frame data, wherein the frame time data is the time data consumed during the frame display process; The analysis module is used to: perform stuttering analysis based on the frame time data to obtain the stuttering situation during the interface scrolling stage, and optimize the scrolling display of the interface based on the stuttering situation.

14. A storage medium, characterized in that, It stores a computer program that, when executed by the processor of the terminal device, causes the terminal device to perform the method described in any one of claims 1 to 12.

15. A terminal device, characterized in that, include: Memory, which stores computer programs; A processor reads a computer program stored in memory to execute the method according to any one of claims 1 to 12.

Images