Display processing method and device, storage medium and terminal equipment

Abstract

The invention discloses a display processing method and device, a storage medium and terminal equipment, and relates to the technical field of display, and the method comprises the steps: obtaining a prediction optimization scheme set corresponding to an interface, the prediction optimization scheme set is obtained by performing prediction analysis on historical optimization data obtained by performing rolling display optimization on the interface by adopting a preset prediction analysis model, and the preset prediction analysis model is a pre-trained machine learning model; and carrying out scrolling display optimization on the interface by adopting the prediction optimization scheme set. According to the method, the optimization effect of interface scrolling display in the interface scrolling scene can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of display technology, specifically to a display processing method, apparatus, storage medium, and terminal device. Background Technology

[0002] With the development of terminal devices, rich and clear UI interfaces and smooth user experience are important indicators of user experience. Currently, in order to achieve clearer UI interfaces, terminal devices are gradually incorporating features such as 4K UI and more complex interface layouts and animations. In scenarios with rapid screen changes and frequent screen scrolling, this leads to higher frame rendering and rendering times, resulting in more severe stuttering in the interface display during scrolling scenarios.

[0003] Currently, relevant optimization solutions typically analyze stuttering and optimize display based on all raw frame data of the interface display process provided by the system (such as Android). However, these solutions often ignore the interface scrolling phase during the display process, and the display optimization is lagging. Therefore, the optimization effect on the interface scrolling display in scrolling scenarios is poor. Summary of the Invention

[0004] This application provides a display processing solution that can effectively improve the optimization effect of interface scrolling display in interface scrolling scenarios.

[0005] The embodiments of this application provide the following technical solutions: According to one embodiment of this application, a display processing method includes: obtaining a set of predictive optimization schemes corresponding to an interface, wherein the set of predictive optimization schemes is obtained by predictive analysis of historical optimization data of the interface for scrolling display optimization using a preset predictive analysis model, and the preset predictive analysis model is a pre-trained machine learning model; and using the set of predictive optimization schemes to optimize the scrolling display of the interface.

[0006] In some embodiments of this application, the historical optimization data includes set data corresponding to multiple historical optimization scheme sets; before obtaining the predicted optimization scheme set corresponding to the interface, the method further includes: using the preset predictive analysis model to perform predictive analysis based on the set data corresponding to the multiple historical optimization scheme sets to obtain a score for each historical optimization scheme set; and determining the historical optimization scheme set corresponding to the score that meets predetermined conditions as the predicted optimization scheme set.

[0007] In some embodiments of this application, the set data includes the occurrence frequency of each of the historical optimization scheme sets and the average drawing time after applying each of the historical optimization scheme sets; the step of performing predictive analysis based on the set data corresponding to the multiple historical optimization scheme sets to obtain a score for each of the historical optimization scheme sets includes: using a weighted calculation layer, based on the function W S = Weight calculations are performed to obtain the set weight W corresponding to each set of historical optimization schemes S. S A scoring layer is used, based on a function. The scores are calculated to obtain the scores corresponding to each set of historical optimization solutions S. Wherein, α, β, and θ are the model parameters of the preset predictive analysis model. This refers to the number of times the historical optimization scheme set S appears. This refers to the average rendering time after applying the historical optimization scheme set S. Given inputs x and θ, S is the probability of occurrence, where x = [i, S], and x refers to the combination of interface i and the set of historical optimization schemes S.

[0008] In some embodiments of this application, the predictive optimization scheme set includes a scheme for optimizing the display of the interface during scrolling, wherein the first display stage set includes one or more stages such as buffer allocation stage, buffer submission stage, and animation playback stage; the step of using the predictive optimization scheme set to optimize the scrolling display of the interface includes: using the schemes in the predictive optimization scheme set to optimize the display of each stage in the first display stage set during the scrolling display of the interface.

[0009] In some embodiments of this application, when the first display stage set includes the buffer submission stage, the adoption of the prediction optimization scheme set scheme to perform display optimization for each stage in the first display stage set when the interface is scrolling includes: selecting elements to be clipped from the elements included in the interface; determining the region position of the clipping region of the element to be clipped; and saving the region position of the clipping region so that the clipping region can be used for rendering after being clipped according to the region position when the interface is scrolling.

[0010] In some embodiments of this application, determining the region location of the cropping area of ​​the element to be cropped includes: determining the invisible region of the element to be cropped as the cropping area; calculating the region location of the invisible region to obtain the region location of the cropping area.

[0011] In some embodiments of this application, when the first display stage set includes the buffer application stage, the step of adopting the prediction optimization scheme set scheme to perform display optimization for each stage in the first display stage set when the interface is scrolling includes: determining the number of buffers corresponding to the interface; and expanding the number of buffers.

[0012] In some embodiments of this application, when the first display stage set includes the animation playback stage, the adoption of the prediction optimization scheme set scheme to perform display optimization for each stage in the first display stage set when the interface is scrolling includes: determining whether the animation closing condition is met; if the animation closing condition is met, then closing the animation displayed when the interface is scrolling.

[0013] In some embodiments of this application, determining whether the animation closing condition is met includes: playing an animation closing prompt message; and determining that the animation closing condition is met when a consent to close instruction information is received.

[0014] In some embodiments of this application, after optimizing the scrolling display of the interface using the predicted optimization scheme set, the method further includes: obtaining the original frame data of the frames displayed on the interface during the interface scrolling phase; determining frame time consumption data based on the original frame data, wherein the frame time consumption data is the time consumption data of the frame display process; performing stuttering analysis based on the frame time consumption data to obtain the stuttering situation during the interface scrolling phase, so as to optimize the scrolling display of the interface based on the stuttering situation.

[0015] In some embodiments of this application, after performing stuttering analysis based on the frame time consumption data to obtain the stuttering situation during the interface scrolling stage, the method further includes: if the stuttering situation is stuttering, analyzing the frame time consumption data corresponding to the time-abnormal frames to obtain time consumption distribution data; determining the target optimization scheme set for the interface based on the time consumption distribution data; and using the target optimization scheme set to optimize the scrolling display of the interface.

[0016] In some embodiments of this application, the target optimization scheme set includes a scheme for optimizing the display of the second display stage set when the interface is scrolling, the second display stage set including one or more stages such as buffer allocation stage, buffer submission stage and animation playback stage; the step of using the target optimization scheme set to optimize the scrolling display of the interface includes: using the schemes in the target optimization scheme set to optimize the display of each stage in the second display stage set when the interface is scrolling.

[0017] According to one embodiment of this application, a display processing apparatus includes: an acquisition module, configured to: acquire a set of predictive optimization schemes corresponding to an interface, wherein the set of predictive optimization schemes is obtained by predictive analysis of historical optimization data of the interface for scrolling display optimization using a preset predictive analysis model, wherein the preset predictive analysis model is a pre-trained machine learning model; and an optimization module, configured to: optimize the scrolling display of the interface using the set of predictive optimization schemes.

[0018] According to another embodiment of this application, a storage medium stores a computer program thereon, which, when executed by the processor of a terminal device, causes the terminal device to perform the method described in the embodiments of this application.

[0019] According to another embodiment of this application, a terminal device may include: a memory storing a computer program; and a processor reading the computer program stored in the memory to execute the methods described in the embodiments of this application.

[0020] According to another embodiment of this application, a computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a terminal device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the terminal device to perform the methods provided in the various optional implementations described in the embodiments of this application.

[0021] In this embodiment of the application, a set of prediction optimization schemes corresponding to the interface is obtained. The set of prediction optimization schemes is obtained by predicting and analyzing historical optimization data of the interface for scrolling display optimization using a preset prediction analysis model. The preset prediction analysis model is a pre-trained machine learning model. The set of prediction optimization schemes is used to optimize the scrolling display of the interface.

[0022] In this embodiment of the application, the terminal device can pre-optimize the scrolling display of the interface using a predictive optimization scheme set at a predetermined time before the interface starts scrolling, either when the interface is created or before scrolling begins. Furthermore, the predictive optimization scheme set is obtained by predicting and analyzing historical optimization data of the interface scrolling display using a preset predictive analysis model, resulting in a good optimization effect for the interface scrolling display. Therefore, the overall optimization effect of the interface scrolling display can be effectively improved in scrolling scenarios. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart of a display processing method according to an embodiment of this application is shown.

[0025] Figure 2 A flowchart illustrating a scheme prediction process according to an embodiment of this application is shown.

[0026] Figure 3 A flowchart illustrating the scheme scoring process according to an embodiment of this application is shown.

[0027] Figure 4 An optimization and data management flowchart according to an embodiment of this application is shown.

[0028] Figure 5 A block diagram of a display processing apparatus according to an embodiment of this application is shown.

[0029] Figure 6 A block diagram of a terminal device according to an embodiment of this application is shown. Detailed Implementation

[0030] The present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments provided herein are merely illustrative of the present disclosure and are not intended to limit the present disclosure. Furthermore, the embodiments provided below are some embodiments for implementing the present disclosure, and not all embodiments for implementing the present disclosure. Unless otherwise specified, the technical solutions described in the embodiments of the present disclosure can be implemented in any combination. It should be noted that, in the embodiments of this disclosure, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a method or apparatus that includes a list of elements includes not only the elements expressly described, but also other elements not expressly listed, or elements inherent to implementing the method or apparatus. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other related elements (e.g., steps in the method or units in the apparatus, such as portions of circuitry, processors, programs, or software, etc.) in the method or apparatus that includes that element. For example, the display processing method provided in this embodiment includes a series of steps, but the display processing method provided in this embodiment is not limited to the steps described. Similarly, the display processing apparatus provided in this embodiment includes a series of units, but the apparatus provided in this embodiment is not limited to the units explicitly described, and may also include units that need to be set up for obtaining relevant information or processing based on information. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of this disclosure. It is understood that in the specific implementation of this application, relevant data is involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards.

[0031] With the development of terminal devices, rich and clear UI interfaces and smooth user experience are important indicators of user experience. Currently, in order to achieve clearer UI interfaces, terminal devices are gradually incorporating features such as 4K UI and more complex interface layouts and animations. In scenarios with rapid screen changes and frequent screen scrolling, this leads to higher frame rendering and rendering times, resulting in more severe stuttering in the interface display during scrolling scenarios.

[0032] Currently, relevant optimization solutions typically analyze stuttering and optimize display based on all raw frame data of the interface display process provided by the system (such as Android). However, these solutions often ignore the interface scrolling phase during the display process, and the display optimization is lagging. Therefore, the optimization effect on the interface scrolling display in scrolling scenarios is poor.

[0033] To address these issues, this application provides a display processing solution that can effectively improve the optimization effect of interface scrolling display in scrolling scenarios.

[0034] The following is a detailed description of the relevant embodiments of the display processing solution provided in this application.

[0035] Figure 1A flowchart illustrating a display processing method according to an embodiment of this application is shown. The execution subject of this display processing method can be a terminal device such as a television (TV), mobile phone, computer, smartwatch, virtual device, or in-vehicle device. For example, in one specific embodiment of this application, the execution subject of the display processing method is specifically located in a television. The television may include a processor and a memory, and the memory stores a computer program. Therefore, the processor in the television can read the computer program stored in the memory to execute the methods of the various embodiments of this application.

[0036] like Figure 1 As shown, the display processing method may include steps S110 to S120.

[0037] Step S110: Obtain the prediction optimization scheme set corresponding to the interface. The prediction optimization scheme set is obtained by predicting and analyzing historical optimization data of the interface scrolling display optimization using a preset prediction analysis model. The preset prediction analysis model is a pre-trained machine learning model. Step S120: Use the predictive optimization scheme set to optimize the scrolling display of the interface.

[0038] For the user interface, historical optimization data on scrolling display optimizations is collected. This historical optimization data effectively reflects the past optimization status during the scrolling phase. By using a pre-defined predictive analysis model to predict and analyze the historical optimization data, a set of effective predictive optimization schemes for scrolling display optimization can be obtained. The pre-defined predictive analysis model can be deployed on terminal devices or in the cloud.

[0039] The preset predictive analysis model is a pre-trained machine learning model, which can be a large model, a decision tree model, a neural network model, or other machine learning models.

[0040] When the interface is created or at a predetermined time before the interface starts scrolling, the terminal device can obtain a set of prediction optimization schemes output by a preset predictive analysis model, and can use the set of prediction optimization schemes to optimize the scrolling display of the interface. This allows for effective scrolling display optimization in advance before the interface scrolls, resulting in a better scrolling display effect.

[0041] The preset predictive analysis model is a pre-trained machine learning model. In one example, the preset predictive analysis model can be a large model that has been fine-tuned and trained beforehand. In other examples, the preset predictive analysis model can be other machine learning models such as deep neural networks or decision trees.

[0042] In summary, using the method described in this embodiment, the terminal device can pre-optimize the scrolling display of the interface by employing a predictive optimization scheme set at a predetermined time, either when the interface is created or before the interface begins scrolling. Furthermore, the predictive optimization scheme set is obtained by predicting and analyzing historical optimization data of the interface's scrolling display using a preset predictive analysis model, resulting in a good optimization effect for the interface's scrolling display. Therefore, overall, it can effectively improve the optimization effect of the interface scrolling display in scrolling scenarios.

[0043] The following description Figure 1 Further optional specific embodiments are provided for each step performed during the display processing in the example.

[0044] See Figure 2 In one embodiment, the historical optimization data includes set data corresponding to multiple historical optimization scheme sets; before obtaining the predicted optimization scheme set corresponding to the interface, it may further include: step S210, using a preset predictive analysis model to perform predictive analysis based on the set data corresponding to multiple historical optimization scheme sets to obtain the score of each historical optimization scheme set; and step S220, determining the historical optimization scheme set corresponding to the score that meets the predetermined conditions as the predicted optimization scheme set.

[0045] For the interface, historical optimization data on scrolling display optimizations are collected. This historical optimization data effectively reflects the historical optimization status during the interface scrolling phase. The historical optimization data includes set data corresponding to multiple historical optimization scheme sets. Each historical optimization scheme set can include schemes for optimizing the display of the interface during the third display phase set during scrolling. The third display phase set can include one or more phases such as the buffer allocation phase, buffer submission phase, and animation playback phase.

[0046] A pre-defined predictive analysis model is used to perform predictive analysis based on the aggregate data corresponding to multiple historical optimization scheme sets, obtaining a score for each historical optimization scheme set. The higher the score, the better the optimization effect on the interface scrolling display. The historical optimization scheme sets corresponding to scores that meet predetermined conditions are determined as the predictive optimization scheme sets, thus obtaining the predictive optimization scheme sets that effectively optimize the interface scrolling display. The predetermined condition can be the highest score; that is, the historical optimization scheme set with the highest score can be determined as the predictive optimization scheme set.

[0047] Furthermore, after each set of target optimization solutions for the interface is determined, both the previously determined sets and the currently determined sets can be compiled into historical optimization solution sets, thus continuously collecting these historical optimization solution sets. A pre-set predictive analysis model can then be used to dynamically predict and analyze based on the continuously collected historical optimization solution set data, thereby refreshing the predictive optimization solution set and further improving its optimization effect.

[0048] For further details, please refer to [link / reference]. Figure 3 In one embodiment, the set data includes the occurrence frequency of each historical optimization scheme set and the average drawing time after applying each historical optimization scheme set; in step S210, predictive analysis is performed based on the set data corresponding to multiple historical optimization scheme sets to obtain the score of each historical optimization scheme set, including: step S211, using a weight calculation layer, according to function W S = We perform weight calculations to obtain the set weights W corresponding to each set of historical optimization schemes S. S Step S212: Employ a scoring layer based on the function. The scores are calculated to obtain the scores corresponding to each set of historical optimization solutions S. Where α, β, and θ are the model parameters of the preset predictive analysis model. This refers to the number of times the historical optimal solution set S appears. This refers to the average rendering time after applying the historical optimization scheme set S. Given inputs x and θ, S represents the probability of S occurring, where x = [i, S], and x refers to the combination of interface i and the set of historical optimization schemes S.

[0049] The dataset includes the number of occurrences of each historical optimization scheme set S when optimizing the scrolling display for interface i. And the average rendering time after applying each historical optimization scheme set S. Specifically, for each historical optimization scheme set S, after each time the historical optimization scheme set S is applied to optimize the scrolling display of interface i, the single-frame rendering time (DoFrameCost) of the i-th frame displayed in one scrolling phase of interface i can be calculated. i The average time taken to render a single frame during the scrolling phase of this interface is the average time taken to render a single frame (n frames). Then, the number of occurrences of the historical optimization solution set S. ,mean The set of historical optimization schemes S applied in the second application can then exist. The average time for each different single rendering session The average time taken for a single drawing is the average drawing time after applying the historical optimization scheme set S. Multiple historical optimization scheme sets S can be used to calculate their respective average drawing times.

[0050] The pre-trained model parameters in the preset predictive analysis model include α, β, and θ. Among them, x refers to the combination of interface i and the historical optimization scheme set S. x is input into the preset predictive analysis model in the form of feature vectors. Let S be the probability of occurrence of the set of historical optimization solutions given inputs x and θ. In the pre-defined predictive analysis model, the weight calculation layer is based on the function W. S = We perform weight calculations to obtain the set weights W corresponding to each set of historical optimization schemes S. S The scoring layer is based on the function. The scores are calculated to obtain the scores corresponding to each set of historical optimization solutions S. .score It can accurately reflect the optimization effect of the historical optimization scheme set S on the interface scrolling display.

[0051] Furthermore, in one embodiment, the predictive optimization scheme set includes a scheme for optimizing the display of a first set of display stages when the interface is scrolling. The first set of display stages includes one or more stages such as a buffer allocation stage, a buffer submission stage, and an animation playback stage. Optimizing the scrolling display of the interface using the predictive optimization scheme set may include: using a scheme from the predictive optimization scheme set to optimize the display of each stage in the first set of display stages when the interface is scrolling.

[0052] The display of each frame during the interface scrolling phase can further include many sub-phases. In this embodiment of the application, one or more of the buffer allocation phase (dequeuebuffer phase), buffer submission phase (queueBuffer phase), and animation playback phase (animation phase) are optimized for display. The applicant found that this has a good optimization effect on the interface scrolling display in scrolling display scenarios, and can effectively avoid adverse effects on the interface scrolling display.

[0053] The predictive optimization scheme set includes schemes for optimizing the display of the first set of display stages during the scrolling display of the interface. This first set of display stages includes one or more stages such as the buffer allocation stage, buffer submission stage, and animation playback stage, with each stage having a corresponding scheme. By employing the schemes in the predictive optimization scheme set, display optimization can be performed separately for each stage in the first set of display stages during the scrolling display of the interface, thereby achieving a good optimization effect on the scrolling display.

[0054] Furthermore, in some embodiments, the prediction optimization scheme set may include schemes for display optimization at all stages of the interface during scrolling, and not just limited to the buffer allocation stage, buffer submission stage, and animation playback stage.

[0055] Furthermore, in one embodiment, when the first display stage set includes a buffer submission stage, a predictive optimization scheme is adopted to perform display optimization for each stage in the first display stage set when the interface is scrolling. This may include: selecting elements to be clipped from the elements included in the interface; determining the region position of the clipping area of ​​the elements to be clipped; and saving the region position of the clipping area so that it can be used for rendering after clipping the region according to the region position when the interface is scrolling.

[0056] When the first display stage set includes a buffer submission stage, the prediction optimization scheme set includes the "element pruning" scheme for display optimization of that buffer submission stage. Using the "element pruning" scheme, firstly, elements to be pruned are selected from the elements included in the interface. Then, the location of the pruning region of the element to be pruned is determined and saved. Subsequently, when the interface scrolls, the pruning region can be pruned based on its location, and the remaining element region can be used for rendering.

[0057] It is understandable that when the first display stage set does not include the buffer submission stage, the prediction optimization scheme set will not include the "element pruning" scheme that performs display optimization on the buffer submission stage, nor will it use the "element pruning" scheme for pruning.

[0058] After each frame is drawn, drawing instructions are sent to the GPU for rendering. By cropping a portion of the elements to be cropped, the remaining area is used by producers (such as rendering threads) to fill the rendering buffer, reducing the rendering time for each frame and preventing excessively long rendering times. The buffer submission phase (queueBuffer phase) submits the rendering-filled buffer to the compositing queue. Reducing rendering time speeds up the submission of the rendering-filled buffer to the compositing queue, thereby reducing the buffer submission time in the buffer submission phase and lowering the probability of stuttering during screen scrolling. The compositing queue is used by consumers (SurfaceFlinger) for compositing and display.

[0059] Furthermore, in one embodiment, determining the region location of the clipping area of ​​the element to be clipped may include: determining the invisible region of the element to be clipped as the clipping region; calculating the region location of the invisible region to obtain the region location of the clipping region.

[0060] The interface involves the overlapping rendering of elements (such as rectangular elements and rounded rectangle elements). Therefore, some elements will have invisible areas obscured by their parent elements. These invisible areas are not visible to the user but are still included in the rendering process. By identifying elements obscured by their parent elements as those to be clipped, and saving the location of these invisible areas as the clipping area, the user-visible area (the remaining element area) can be obtained by clipping the invisible areas during the interface's scrolling display. This approach does not affect the user experience and reduces rendering time, reliably reducing buffer commit time during the buffer commit phase.

[0061] Optionally, in other embodiments, determining the location of the cropping region of the element to be cropped may include: determining the user-specified element as the element to be cropped and determining the specified region of the user-specified element to be cropped as the cropping region.

[0062] Furthermore, in one embodiment, when the first display stage set includes a buffer application stage, a prediction optimization scheme is adopted to perform display optimization for each stage in the first display stage set when the interface is scrolling, including: determining the number of buffers corresponding to the interface; and expanding the number of buffers.

[0063] When the first display stage set includes a buffer allocation stage (dequeuebuffer stage), the prediction optimization scheme set includes the "buffer adjustment" scheme for display optimization of that buffer allocation stage. Using the "buffer adjustment" scheme, the number of buffers corresponding to the interface is first determined (i.e., the total number of buffers waiting to be rotated in the buffer queue), and then the number of buffers corresponding to the interface is expanded to obtain the expanded number of buffers.

[0064] CPU drawing and GPU rendering are usually performed asynchronously. CPU drawing and GPU rendering rely on buffer queues for data exchange. Among them, the buffer allocation time in the buffer allocation phase (dequeuebuffer phase) is an important indicator affecting whether the buffer can be submitted to the GPU in a timely manner to start rendering. A long buffer allocation time will result in long idle time for the GPU and it will be unable to perform high-throughput rendering work.

[0065] Expanding the number of buffers increases the total number of buffers in the corresponding buffer queue of the interface, thereby improving the efficiency of requesting buffers from the buffer queue and reducing the buffer request time in the buffer request stage (dequeuebuffer stage). This allows for more efficient request of buffers to be submitted to the GPU to start rendering, thus reducing the probability of stuttering when the interface scrolls.

[0066] Furthermore, in one embodiment, the number of expanded buffers can be limited to less than or equal to a predetermined first number (e.g., 5) to avoid excessive memory resource consumption. Additionally, in some embodiments, when the first display stage set does not include a buffer allocation stage (dequeuebuffer stage), the prediction optimization scheme set does not include a "buffer adjustment" scheme that performs display optimization on that buffer allocation stage.

[0067] Furthermore, in one embodiment, when the first display stage set includes an animation playback stage, a predictive optimization scheme is adopted to optimize the display of each stage in the first display stage set when the interface is scrolling, including: determining whether the animation closing condition is met; if the animation closing condition is met, then closing the animation displayed when the interface is scrolling.

[0068] When the first display stage set includes an animation playback stage, the prediction optimization scheme set includes the "animation off" scheme, which optimizes the display of that animation playback stage. Using the "animation off" scheme, if the animation off condition is met, the scrolling animation displayed when the interface is scrolling is turned off, and subsequently, the scrolling animation applied during frame rendering is canceled.

[0069] When the interface scrolls, the application may use custom scrolling animations. Excessively long scrolling animations can cause the rendering process to time out, which in turn blocks the rendering process, resulting in stuttering during screen scrolling. By disabling the animation displayed during screen scrolling when the conditions for disabling animations are met, the time consumed during the animation playback phase (i.e., animation duration) can be reduced, thereby lowering the probability of stuttering during screen scrolling.

[0070] Furthermore, in one embodiment, determining whether the animation shutdown condition is met may include: playing an animation shutdown prompt message; and determining that the animation shutdown condition is met when a shutdown consent instruction is received.

[0071] An animated closing prompt message (such as "Lag detected, scrolling animation will be turned off soon") can be played via voice or pop-up window. Users can choose whether to agree to turn it off. If they agree, the terminal device will receive an instruction message indicating that the animation closing conditions are met. Then, the terminal device will determine that the animation closing conditions are met and turn off the animation displayed when the interface is scrolling. This can further avoid affecting the user experience or needs.

[0072] Furthermore, in some embodiments, before displaying the animation playback shutdown prompt message, it can be determined whether "the prediction optimization scheme set includes the animation playback stage for a consecutive number of times" and "more than a predetermined time has passed since the user last disagreed to close the animation". If "the prediction optimization scheme set includes the animation playback stage for a consecutive number of times" and "more than a predetermined time has passed since the user last disagreed to close the animation", then the animation playback shutdown prompt message is displayed, which can further improve the user experience.

[0073] The foregoing section described relevant embodiments for pre-optimization of the interface scrolling display using a predictive optimization scheme set. When the interface belongs to an application, the training data of the interface in that application can be cleared after the application is updated / uninstalled. The training data may include the aforementioned predictive optimization scheme set, historical optimization data, etc. After the application is updated or reinstalled, historical optimization data can be collected again to analyze and determine a new predictive optimization scheme set.

[0074] For example, in one example, such as Figure 4 As shown, step S310 involves creating the interface; step S320 involves determining if a predictive optimization scheme set exists; if yes (i.e., a predictive optimization scheme set corresponding to the interface exists), then step S330 is executed; if no (i.e., a predictive optimization scheme set corresponding to the interface does not exist), then step S360 is executed; step S330 involves pre-optimization, specifically, using the predictive optimization scheme set to optimize the scrolling display of the interface. Step S340 involves determining whether the application will be updated / uninstalled; if yes, then step S350 is executed to clear the training data of the interface in the application. Step S360 involves not performing pre-optimization.

[0075] In one embodiment, after optimizing the scrolling display of the interface using a predictive optimization scheme set, the method may further include: obtaining the original frame data of the frames displayed on the interface during the scrolling phase; determining the frame time consumption data based on the original frame data, wherein the frame time consumption data is the time consumption data of the frame display process; performing stuttering analysis based on the frame time consumption data to obtain the stuttering situation during the scrolling phase, so as to optimize the scrolling display of the interface based on the stuttering situation.

[0076] During the interface display process, the interface scrolling phase is monitored and identified. The interface scrolling phase refers to the interface display period from the moment scrolling begins to the moment scrolling ends. For example, for a television interface, the individual interface scrolling phase can be identified by listening to the trigger events (Input events) of the remote control's directional keys (such as DPAD_UP / DPAD_DOWN). For example, pressing a directional key can be considered as the start of scrolling, and releasing a directional key can be considered as the end of scrolling.

[0077] The system (such as Android) records all raw frame data of the interface display process (all raw frame data includes the raw frame data of all frames displayed during the interface display process). The raw frame data may include frame data such as the start and end times of animation playback, the start and end times of buffer allocation, etc. In this embodiment, based on identifying the interface scrolling stage, a portion of the raw frame data of the interface scrolling stage is further obtained from the all raw frame data (the portion of raw frame data includes the raw frame data of the frames displayed during the interface scrolling stage).

[0078] Furthermore, based on the original frame data of the frames displayed during the interface scrolling phase, the frame time consumption data during the interface scrolling phase is determined, that is, the original frame data is converted into frame time consumption data. The frame time consumption data is the time consumption data of the frame display process, which may include multiple time consumption data of the frame display process such as animation time consumption, buffer request time consumption, buffer commit time consumption, etc.

[0079] Analyzing stuttering based on frame latency data during the scrolling phase offers several advantages. First, it effectively considers the scrolling process, leading to more accurate analysis of stuttering and resulting in better optimization of the scrolling display. Second, using only frame latency data avoids the extra time required to use all raw frame data, thus improving the reliability of the optimization. Furthermore, after optimizing the scrolling display using a predictive optimization scheme set, further real-time optimization of the scrolling display can be performed based on stuttering assessments.

[0080] In one embodiment, stuttering analysis is performed based on frame time consumption data to obtain stuttering conditions during the interface scrolling phase, including: determining abnormal time consumption frames during the interface scrolling phase based on frame time consumption data; when the proportion of abnormal time consumption frames during the interface scrolling phase is greater than a predetermined proportion, stuttering is determined to have occurred.

[0081] During the interface scrolling phase, multiple frames are displayed. Frames with abnormal single-frame drawing or rendering times are identified as time-abnormal frames. The ratio of the total number of time-abnormal frames (BadFrameCount) to the total number of frames (FrameCount) displayed during the interface scrolling phase represents the percentage of time-abnormal frames. FrameCount. When the percentage of frames with abnormally long execution times exceeds a predetermined percentage (e.g., 10%), it can be accurately determined that a stutter occurred during the scrolling phase of the interface.

[0082] Furthermore, in one embodiment, the frame time consumption data includes single-frame drawing time and single-frame rendering time; based on the frame time consumption data, determining abnormal frames in the interface scrolling stage includes: determining frames corresponding to single-frame drawing time or single-frame rendering time that exceed a predetermined time limit as abnormal frames.

[0083] For the i-th frame displayed during the interface scrolling phase, its single-frame rendering time doFrameCost can be determined based on its original frame data. i and single-frame rendering time drawFrameCost i In this context, the time consumed by drawing a single frame is equal to the time difference between the time when drawing is completed and the time when drawing begins, and the time consumed by rendering a single frame is equal to the time difference between the time when rendering is completed and the time when rendering begins.

[0084] If the time taken to draw or render a single frame of the i-th frame is greater than the predetermined time limit, it indicates that the drawing or rendering process of the i-th frame takes too long. Frames whose single-frame drawing or rendering time exceeds the predetermined time limit can be accurately identified as frames with abnormal time consumption.

[0085] Furthermore, in one embodiment, after performing stuttering analysis based on frame time consumption data to obtain the stuttering situation during the interface scrolling stage, the method may further include: if the stuttering situation is stuttering, analyzing the frame time consumption data corresponding to the abnormal time consumption frames to obtain time consumption distribution data; determining the target optimization scheme set for the interface based on the time consumption distribution data; and using the target optimization scheme set to optimize the scrolling display of the interface.

[0086] If stuttering occurs, it indicates that scrolling display optimization is needed to fix the stuttering. In this case, by analyzing the frame time data corresponding to the abnormally timed frames, time distribution data is obtained. This time distribution data reflects the distribution of time consumption at each stage of the frame time data. Based on this time distribution data, the stages requiring display optimization can be accurately determined, and thus the corresponding target optimization scheme set can be accurately identified. Using the target optimization scheme set to optimize the scrolling display of the interface can effectively improve the scrolling display.

[0087] In one embodiment, the frame time consumption data includes animation time consumption, buffer request time consumption, buffer commit time consumption, single-frame drawing time consumption, and single-frame rendering time consumption. Analysis is performed on the frame time consumption data corresponding to frames with abnormal time consumption to obtain time consumption distribution data, including: dividing the animation time consumption by the single-frame drawing time to obtain the animation time consumption percentage; dividing the buffer request time consumption by the single-frame rendering time to obtain the request time consumption percentage; and dividing the buffer commit time consumption by the single-frame rendering time to obtain the commit time consumption percentage. The time consumption distribution data includes the animation time consumption percentage, the request time consumption percentage, and the commit time consumption percentage.

[0088] For the i-th frame displayed during the interface scrolling phase, its animation duration (animationCost) can also be determined based on its original frame data. i Buffer request time dequeueBufferCost i Buffer commit time queueBufferCost i Single-frame rendering time doFrameCost i and single-frame rendering time drawFrameCost i Animation time is equal to the time difference between the end of the animation and the start of the animation. Buffer request time is equal to the time difference between the completion of buffer request and the start of buffer request. Buffer commit time is equal to the time difference between the completion of buffer commit and the start of buffer commit. Single frame drawing time is equal to the time difference between the completion of drawing and the start of drawing. Single frame rendering time is equal to the time difference between the completion of rendering and the start of rendering.

[0089] For the i-th frame, the percentage of animation time consumed animationCost i / doFrameCost i Request time percentage dequeueBufferCost i / drawFrameCost i Submission time percentage queueBufferCost i / drawFrameCost i The time consumption distribution data of the i-th frame includes the proportion of animation time consumption, the proportion of request time consumption, and the proportion of submission time consumption. This time consumption distribution data can reflect the time consumption distribution of the three main stages: animation playback stage, buffer allocation stage, and buffer submission stage. Based on this, the stages that need to be optimized in display can be accurately determined, and the corresponding target optimization scheme set can be accurately obtained.

[0090] The display of each frame during the interface scrolling phase can further include many sub-phases. In this embodiment of the application, one or more of the buffer allocation phase (dequeuebuffer phase), buffer submission phase (queueBuffer phase), and animation playback phase (animation phase) are optimized for display. The applicant found that this has a good optimization effect on the interface scrolling display in scrolling display scenarios, and can effectively avoid adverse effects on the interface scrolling display.

[0091] Furthermore, in one embodiment, the target optimization scheme set includes schemes for optimizing the display of a second display stage set during scrolling of the interface. The second display stage set includes one or more stages such as a buffer allocation stage, a buffer submission stage, and an animation playback stage. Determining the target optimization scheme set of the interface based on time consumption distribution data may specifically include: If the average first percentage of the animation time of all time-consuming abnormal frames in the interface scrolling phase is greater than the first preset threshold, then the second display phase set is determined to include the animation playback phase; if the average second percentage of the request time of all time-consuming abnormal frames in the interface scrolling phase is greater than the second preset threshold, then the second display phase set is determined to include the buffer allocation phase; if the average third percentage of the submission time of all time-consuming abnormal frames in the interface scrolling phase is greater than the third preset threshold, then the second display phase set is determined to include the buffer submission phase.

[0092] First percentage average The second percentage average The third percentage average , where N is the total number of all time-consuming abnormal frames during the interface scrolling phase.

[0093] Furthermore, .

[0094] in, Let be the set of the second display stages, where 'a' is the first preset threshold, 'b' is the third preset threshold, and 'c' is the second preset threshold. When the average percentage of the first proportion is greater than the first preset threshold 'a', the second display stage set will include 1 (animation playback stage); when the average percentage of the second proportion is greater than the second preset threshold 'c', the second display stage set will include 3 (buffer allocation stage); and when the average percentage of the third proportion is greater than the third preset threshold 'b', the second display stage set will include 2 (buffer submission stage).

[0095] The values ​​of a, b, and c can be set according to the actual situation. This application does not impose any special restrictions on them. For example, in one example, a, b, and c are all 0.7.

[0096] Furthermore, in one embodiment, the target optimization scheme set includes a scheme for optimizing the display of a second set of display stages when the interface is scrolling. The second set of display stages includes one or more stages such as a buffer allocation stage, a buffer submission stage, and an animation playback stage. The step of using the target optimization scheme set to optimize the scrolling display of the interface may specifically include: using the schemes in the target optimization scheme set to optimize the display of each stage in the second set of display stages when the interface is scrolling.

[0097] Furthermore, in one embodiment, when the second display stage set includes a buffer submission stage, the target optimization scheme set is used to perform display optimization for each stage in the second display stage set when the interface is scrolling. This may include: selecting elements to be clipped from the elements included in the interface; determining the region position of the clipping area of ​​the elements to be clipped; and saving the region position of the clipping area so that it can be used for rendering after clipping the region according to the region position when the interface is scrolling.

[0098] When the second display stage set includes a buffer submission stage, the target optimization scheme set includes the "element pruning" scheme for display optimization of that buffer submission stage. The "element pruning" scheme first filters out elements to be pruned from the elements included in the interface. Then, it determines and saves the location of the pruning region for each element. Later, when the interface scrolls, the pruning region can be pruned based on its location, and the remaining element area can be used for rendering.

[0099] It is understandable that when the second display stage set does not include the buffer submission stage, the target optimization scheme set will not include the "element pruning" scheme that performs display optimization on the buffer submission stage, nor will it use the "element pruning" scheme for pruning.

[0100] After each frame is drawn, drawing instructions are sent to the GPU for rendering. By cropping a portion of the elements to be cropped, the remaining area is used by producers (such as rendering threads) to fill the rendering buffer, reducing the rendering time for each frame and preventing excessively long rendering times. The buffer submission phase (queueBuffer phase) submits the rendering-filled buffer to the compositing queue. Reducing rendering time speeds up the submission of the rendering-filled buffer to the compositing queue, thereby reducing the buffer submission time in the buffer submission phase and lowering the probability of stuttering during screen scrolling. The compositing queue is used by consumers (SurfaceFlinger) for compositing and display.

[0101] Furthermore, in one embodiment, determining the region location of the clipping area of ​​the element to be clipped includes: determining the invisible region of the element to be clipped as the clipping region; calculating the region location of the invisible region to obtain the region location of the clipping region.

[0102] The interface involves the overlapping rendering of elements (such as rectangular elements and rounded rectangle elements). Therefore, some elements will have invisible areas obscured by their parent elements. These invisible areas are not visible to the user but are still included in the rendering process. By identifying elements obscured by their parent elements as those to be clipped, and saving the location of these invisible areas as the clipping area, the user-visible area (the remaining element area) can be obtained by clipping the invisible areas during the interface's scrolling display. This approach does not affect the user experience and reduces rendering time, reliably reducing buffer commit time during the buffer commit phase.

[0103] Optionally, in other embodiments, determining the location of the cropping region of the element to be cropped may include: determining the user-specified element as the element to be cropped and determining the specified region of the user-specified element to be cropped as the cropping region.

[0104] Furthermore, in one embodiment, when the second display stage set includes a buffer application stage, the target optimization scheme set is adopted to perform display optimization for each stage in the second display stage set when the interface is scrolling. This may include: determining the number of buffers corresponding to the interface; and expanding the number of buffers.

[0105] When the second display stage set includes a buffer allocation stage (dequeuebuffer stage), the target optimization scheme set includes the "buffer adjustment" scheme for display optimization of that buffer allocation stage. Using the "buffer adjustment" scheme, the number of buffers corresponding to the interface is first determined (i.e., the total number of buffers waiting to be rotated in the buffer queue), and then the number of buffers corresponding to the interface is expanded to obtain the expanded number of buffers.

[0106] CPU drawing and GPU rendering are usually performed asynchronously. CPU drawing and GPU rendering rely on buffer queues for data exchange. Among them, the buffer allocation time in the buffer allocation phase (dequeuebuffer phase) is an important indicator affecting whether the buffer can be submitted to the GPU in a timely manner to start rendering. A long buffer allocation time will result in long idle time for the GPU and it will be unable to perform high-throughput rendering work.

[0107] Expanding the number of buffers increases the total number of buffers in the corresponding buffer queue of the interface, thereby improving the efficiency of requesting buffers from the buffer queue and reducing the buffer request time in the buffer request stage (dequeuebuffer stage). This allows for more efficient request of buffers to be submitted to the GPU to start rendering, thus reducing the probability of stuttering when the interface scrolls.

[0108] Furthermore, in one embodiment, the number of expanded buffers can be limited to be less than or equal to a predetermined first number (e.g., 5) to avoid excessive memory resource consumption. Additionally, in some embodiments, when the second display stage set does not include a buffer allocation stage (dequeuebuffer stage), the target optimization scheme set does not include a "buffer adjustment" scheme for display optimization of that buffer allocation stage. In this case, if the number of buffers corresponding to the interface is greater than a predetermined second number (e.g., 3), the number of buffers can be adjusted to that predetermined second number, thereby avoiding invalid resource consumption.

[0109] Furthermore, in one embodiment, when the second display stage set includes an animation playback stage, the target optimization scheme set is adopted to optimize the display for each stage in the second display stage set when the interface is scrolling. This may include: determining whether the animation closing condition is met; if the animation closing condition is met, then closing the animation displayed when the interface is scrolling.

[0110] When the second display stage set includes an animation playback stage, the target optimization scheme set includes the "animation off" scheme, which optimizes the display of that animation playback stage. Using the "animation off" scheme, if the animation off condition is met, the scrolling animation displayed when the interface is scrolling is turned off, and subsequently, the scrolling animation applied during frame rendering is canceled.

[0111] When the interface scrolls, the application may use custom scrolling animations. Excessively long scrolling animations can cause the rendering process to time out, which in turn blocks the rendering process, resulting in stuttering during screen scrolling. By disabling the animation displayed during screen scrolling when the conditions for disabling animations are met, the time consumed during the animation playback phase (i.e., animation duration) can be reduced, thereby lowering the probability of stuttering during screen scrolling.

[0112] Furthermore, in one embodiment, determining whether the animation shutdown condition is met may include: playing an animation shutdown prompt message; and determining that the animation shutdown condition is met when a shutdown consent instruction is received.

[0113] An animated closing prompt message (such as "Lag detected, scrolling animation will be turned off soon") can be played via voice or pop-up window. Users can choose whether to agree to turn it off. If they agree, the terminal device will receive an instruction message indicating that the animation closing conditions are met. Then, the terminal device will determine that the animation closing conditions are met and turn off the animation displayed when the interface is scrolling. This can further avoid affecting the user experience or needs.

[0114] Furthermore, in some embodiments, before displaying the animation closing prompt message, it can be determined whether "the target optimization scheme set includes the animation playback stage for a consecutive number of times" and "more than a predetermined time has passed since the user last disagreed to close the animation". If "the target optimization scheme set includes the animation playback stage for a consecutive number of times" and "more than a predetermined time has passed since the user last disagreed to close the animation", then the animation closing prompt message is displayed, which can further improve the user experience.

[0115] To facilitate better implementation of the display processing method provided in the embodiments of this application, the embodiments of this application also provide a display processing apparatus based on the above-described display processing method. The meanings of the terms used are the same as in the above-described display processing method, and specific implementation details can be found in the descriptions in the method embodiments. Figure 5 A block diagram of a display processing apparatus according to an embodiment of this application is shown.

[0116] like Figure 5 As shown, the display processing device 400 may include: an acquisition module 410, which can be used to: acquire a set of prediction optimization schemes corresponding to the interface, wherein the set of prediction optimization schemes is obtained by predicting and analyzing historical optimization data of the interface for scrolling display optimization using a preset prediction analysis model, wherein the preset prediction analysis model is a pre-trained machine learning model; and an optimization module 420, which can be used to: optimize the scrolling display of the interface using the set of prediction optimization schemes.

[0117] In some embodiments of this application, the historical optimization data includes set data corresponding to multiple historical optimization scheme sets; before obtaining the predicted optimization scheme set corresponding to the interface, the device further includes a prediction module that can be used to: use the preset prediction analysis model to perform prediction analysis based on the set data corresponding to the multiple historical optimization scheme sets to obtain a score for each historical optimization scheme set; and determine the historical optimization scheme set corresponding to the score that meets the predetermined conditions as the predicted optimization scheme set.

[0118] In some embodiments of this application, the set data includes the occurrence frequency of each of the historical optimization scheme sets and the average drawing time after the application of each of the historical optimization scheme sets; when performing predictive analysis based on the set data corresponding to the multiple historical optimization scheme sets to obtain the score of each of the historical optimization scheme sets, the prediction module can be used to: employ a weight calculation layer, based on the function W S = Weight calculations are performed to obtain the set weight W corresponding to each set of historical optimization schemes S. S A scoring layer is used, based on a function. The scores are calculated to obtain the scores corresponding to each set of historical optimization solutions S. Wherein, α, β, and θ are the model parameters of the preset predictive analysis model. This refers to the number of times the historical optimization scheme set S appears. This refers to the average rendering time after applying the historical optimization scheme set S. Given inputs x and θ, S is the probability of occurrence, where x = [i, S], and x refers to the combination of interface i and the set of historical optimization schemes S.

[0119] In some embodiments of this application, the predictive optimization scheme set includes a scheme for optimizing the display of a first set of display stages when the interface is scrolling. The first set of display stages includes one or more stages such as a buffer allocation stage, a buffer submission stage, and an animation playback stage. When the predictive optimization scheme set is used to optimize the scrolling display of the interface, the optimization module can be used to: use the schemes in the predictive optimization scheme set to optimize the display of each stage in the first set of display stages when the interface is scrolling.

[0120] In some embodiments of this application, when the first display stage set includes the buffer submission stage, when the prediction optimization scheme set is adopted to optimize the display of each stage in the first display stage set during the interface scrolling display, the optimization module can be used to: filter out elements to be clipped from the elements included in the interface; determine the region position of the clipping area of ​​the element to be clipped; save the region position of the clipping area so that the clipping area can be used for rendering after being clipped according to the region position during the interface scrolling display.

[0121] In some embodiments of this application, when determining the region position of the cropping region of the element to be cropped, the optimization module can be used to: determine the invisible region of the element to be cropped as the cropping region; calculate the region position of the invisible region to obtain the region position of the cropping region.

[0122] In some embodiments of this application, when the first display stage set includes the buffer application stage, when the prediction optimization scheme set is adopted to optimize the display of each stage in the first display stage set when the interface is scrolling, the optimization module can be used to: determine the number of buffers corresponding to the interface; and expand the number of buffers.

[0123] In some embodiments of this application, when the first display stage set includes the animation playback stage, when the prediction optimization scheme set is adopted to optimize the display of each stage in the first display stage set when the interface is scrolling, the optimization module can be used to: determine whether the animation closing condition is met; if the animation closing condition is met, then turn off the animation displayed when the interface is scrolling.

[0124] In some embodiments of this application, when determining whether the animation closing condition is met, the optimization module can be used to: play an animation closing prompt message; and when receiving an indication to agree to close the animation, determine that the animation closing condition is met.

[0125] In some embodiments of this application, after the predicted optimization scheme set is used to optimize the scrolling display of the interface, the device further includes an analysis module for: acquiring the original frame data of the frames displayed on the interface during the interface scrolling phase; determining frame time consumption data based on the original frame data, wherein the frame time consumption data is the time consumption data of the frame display process; performing stuttering analysis based on the frame time consumption data to obtain the stuttering situation during the interface scrolling phase, so as to optimize the scrolling display of the interface based on the stuttering situation.

[0126] In some embodiments of this application, after performing stuttering analysis based on the frame time consumption data to obtain the stuttering situation during the interface scrolling stage, the device further includes a repair module for: if the stuttering situation is stuttering, analyzing the frame time consumption data corresponding to the time-abnormal frames to obtain time consumption distribution data; determining the target optimization scheme set for the interface based on the time consumption distribution data; and using the target optimization scheme set to optimize the scrolling display of the interface.

[0127] In some embodiments of this application, the target optimization scheme set includes a scheme for optimizing the display of the second display stage set when the interface is scrolling. The second display stage set includes one or more stages such as buffer allocation stage, buffer submission stage, and animation playback stage. When the target optimization scheme set is used to optimize the scrolling display of the interface, the repair module is used to: use the schemes in the target optimization scheme set to optimize the display of each stage in the second display stage set when the interface is scrolling.

[0128] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0129] Furthermore, embodiments of this application also provide a terminal device, such as... Figure 6 As shown, Figure 6 A block diagram of a terminal device according to an embodiment of this application is shown, specifically: The terminal device may include components such as a processor 501 with one or more processing cores, a memory 502 with one or more computer-readable storage media, a power supply 503, and an input unit 504. Those skilled in the art will understand that... Figure 6 The terminal device structure shown does not constitute a limitation on the terminal device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 501 is the control center of the terminal device, connecting various parts of the computer device via various interfaces and lines. It executes various functions and processes data by running or executing software programs and / or modules stored in the memory 502, and by calling data stored in the memory 502. Optionally, the processor 501 may include one or more processing cores; preferably, the processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user page, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 501.

[0130] The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.

[0131] The terminal device also includes a power supply 503 that supplies power to the various components. Preferably, the power supply 503 can be logically connected to the processor 501 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 503 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0132] The terminal device may also include an input unit 504, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0133] Although not shown, the terminal device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 501 in the terminal device can load the executable files corresponding to the processes of one or more computer programs into the memory 502 according to the following instructions, and the processor 501 runs the computer programs stored in the memory 502, thereby realizing the various functions in the foregoing embodiments of this application.

[0134] For example, processor 501 can perform the following steps: obtain a set of prediction optimization schemes corresponding to the interface, wherein the set of prediction optimization schemes is obtained by predicting and analyzing historical optimization data of the interface for scrolling display optimization using a preset prediction analysis model, wherein the preset prediction analysis model is a pre-trained machine learning model; and use the set of prediction optimization schemes to optimize the scrolling display of the interface.

[0135] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by a computer program, or by a computer program controlling related hardware. The computer program can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0136] Therefore, embodiments of this application also provide a storage medium storing a computer program that can be loaded by a processor to execute the steps in any of the methods provided in embodiments of this application.

[0137] The storage medium can be a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0138] Since the computer program stored in the storage medium can execute the steps of any of the methods provided in the embodiments of this application, the beneficial effects that the methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0139] According to another embodiment of this application, a computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a terminal device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the terminal device to perform the methods provided in the various optional implementations described in the embodiments of this application.

[0140] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0141] It should be understood that this application is not limited to the embodiments described above and shown in the accompanying drawings, but various modifications and changes can be made without departing from its scope.

Claims

1. A display processing method, characterized in that, include: Obtain the prediction optimization scheme set corresponding to the interface. The prediction optimization scheme set is obtained by predicting and analyzing historical optimization data of the interface scrolling display optimization using a preset prediction analysis model. The preset prediction analysis model is a pre-trained machine learning model. The predicted optimization scheme set is used to optimize the scrolling display of the interface.

2. The method according to claim 1, characterized in that, The historical optimization data includes set data corresponding to multiple historical optimization scheme sets; Before obtaining the prediction optimization scheme set corresponding to the interface, the method further includes: Using the preset predictive analysis model, predictive analysis is performed based on the set data corresponding to the multiple historical optimization scheme sets to obtain the scores of each historical optimization scheme set; Furthermore, the set of historical optimization schemes corresponding to the scores that meet the predetermined conditions is determined as the set of predicted optimization schemes.

3. The method according to claim 2, characterized in that, The data set includes the number of occurrences of each of the historical optimization scheme sets and the average drawing time after each of the historical optimization scheme sets is applied; The step of performing predictive analysis based on the set data corresponding to the multiple historical optimization scheme sets to obtain a score for each historical optimization scheme set includes: A weighted calculation layer is used, based on the function W. S = Weight calculations are performed to obtain the set weight W corresponding to each set of historical optimization schemes S. S ; A scoring layer is used, based on the function The scores are calculated to obtain the scores corresponding to each set of historical optimization solutions S. ; Wherein, α, β, and θ are the model parameters of the preset predictive analysis model. This refers to the number of times the historical optimization scheme set S appears. This refers to the average rendering time after applying the historical optimization scheme set S. Given inputs x and θ, S is the probability of occurrence, where x = [i, S], and x refers to the combination of interface i and the set of historical optimization schemes S.

4. The method according to claim 1, characterized in that, The predictive optimization scheme set includes schemes for optimizing the display of the interface during scrolling, wherein the first display stage set includes one or more stages such as buffer allocation stage, buffer submission stage, and animation playback stage. The step of using the predicted optimization scheme set to optimize the scrolling display of the interface includes: The prediction optimization scheme is adopted to optimize the display of each stage in the first set of display stages when the interface is scrolling.

5. The method according to claim 4, characterized in that, When the first display stage set includes the buffer submission stage, the adoption of the prediction optimization scheme set scheme, performing display optimization for each stage in the first display stage set when the interface is scrolling, includes: Select the elements to be cropped from the elements included in the interface; Determine the location of the clipping region for the element to be clipped; The location of the cropped area is saved so that it can be cropped based on the location when the interface is scrolling and then used for rendering.

6. The method according to claim 5, characterized in that, Determining the location of the cropping region for the element to be cropped includes: The invisible area of ​​the element to be clipped is defined as the clipping area; Calculate the location of the invisible area to obtain the location of the cropped area.

7. The method according to claim 4, characterized in that, When the first display stage set includes the buffer allocation stage, the adoption of the prediction optimization scheme set scheme, performing display optimization for each stage in the first display stage set when the interface is scrolling, includes: Determine the number of buffers corresponding to the interface; The number of buffers is increased.

8. The method according to claim 4, characterized in that, When the first display stage set includes the animation playback stage, the adoption of the prediction optimization scheme set scheme, performing display optimization for each stage in the first display stage set when the interface is scrolling, includes: Determine if the animation shutdown conditions are met; If the animation shutdown condition is met, then the animation displayed on the interface during scrolling is turned off.

9. The method according to claim 8, characterized in that, Determining whether the animation shutdown condition is met includes: The animation playback closed notification message; When a message indicating agreement to close is received, it is determined that the conditions for closing the animation are met.

10. The method according to any one of claims 1 to 9, characterized in that, After optimizing the scrolling display of the interface using the predicted optimization scheme set, the method further includes: Obtain the raw frame data of the frames displayed during the interface scrolling phase; Based on the original frame data, the frame time consumption data is determined, which is the time consumption data of the frame display process; Based on the frame time data, a stuttering analysis is performed to obtain the stuttering situation during the interface scrolling phase, so as to optimize the scrolling display of the interface according to the stuttering situation.

11. The method according to claim 10, characterized in that, After performing stuttering analysis based on the frame time data to obtain the stuttering situation during the interface scrolling phase, the method further includes: If the lag is a lag, analyze the frame time data corresponding to the abnormal time frame to obtain time distribution data; Determine the target optimization scheme set for the interface based on the time consumption distribution data; The target optimization scheme set is used to optimize the scrolling display of the interface.

12. The method according to claim 11, characterized in that, The target optimization scheme set includes schemes for optimizing the display of the second display stage set when the interface is scrolling. The second display stage set includes one or more stages such as buffer allocation stage, buffer submission stage, and animation playback stage. The step of optimizing the scrolling display of the interface using the target optimization scheme set includes: The target optimization scheme is adopted to optimize the display of each stage in the second display stage set when the interface is scrolling.

13. A display processing device, characterized in that, include: The acquisition module is used to: acquire the set of prediction optimization schemes corresponding to the interface, wherein the set of prediction optimization schemes is obtained by predicting and analyzing historical optimization data of the interface for scrolling display optimization using a preset prediction analysis model, and the preset prediction analysis model is a pre-trained machine learning model. The optimization module is used to: optimize the scrolling display of the interface using the predicted optimization scheme set.

14. A storage medium, characterized in that, It stores a computer program that, when executed by the processor of the terminal device, causes the terminal device to perform the method described in any one of claims 1 to 12.

15. A terminal device, characterized in that, include: Memory, which stores computer programs; A processor reads a computer program stored in memory to execute the method according to any one of claims 1 to 12.

Images