Application running method and device, electronic equipment and storage medium

Abstract

The invention relates to an application running method and device, electronic equipment and a storage medium. The method comprises the following steps: determining a target application of which the power consumption is to be improved and a target layer of which the refresh rate of the target application is to be improved; identifying the type of the target layer by using a decision tree model; when the type of the target layer belongs to the power consumption adjustable layer, adjusting the refresh rate value of the target layer; determining the refresh rate value of the terminal screen based on the adjusted refresh rate value of the target layer; and running the target application based on the refresh rate value of the terminal screen. In addition, the application running method can be executed by using an artificial intelligence model. Therefore, the refresh rate of the system can be automatically adjusted without manual intervention, and corresponding refresh rates do not need to be manually set for different APPs one by one, so that the repeated and tedious setting process is avoided, and the setting of the refresh rate is more intelligent and convenient.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and more specifically, to an application running method, device, electronic device, and storage medium. Background Art

[0002] When the terminal is in the adaptive refresh rate (Refresh Rate) mode, the refresh rate of the system, that is, the refresh rate of the terminal screen, is determined by the refresh rate (frame persecond, fps) and area weight of each layer in the multiple layers (layers) contained in the application (Application, APP). For example, when the area weights of the multiple layers contained in the APP meet the preset requirements and the refresh rates of the multiple layers contained in the APP are inconsistent, the refresh rate of the system generally takes the highest value among the multiple refresh rates. For example, the refresh rate of most video sources may be less than 30 frames per second, while the refresh rate of some software bullet screen scrolling may be as high as 120 frames per second. When the area weights of the video layer and the bullet screen layer meet the preset conditions, the system refresh rate will be forced to maintain at 120 frames per second, but in fact, the video to be played does not need such a high refresh rate at all. At the same time, since the system refresh rate is forced to be increased, the terminal will work at a higher current, which will cause a substantial increase in the power consumption of the terminal and cause excessive waste of resources.

[0003] In the related art, in order to avoid the waste of resources caused by the increase of terminal power consumption, the developer mainly tests different apps and then manually limits the refresh rate of each app to a lower value on the server side. However, this method requires the developer to test different apps one by one and then set the corresponding refresh rate for different apps one by one on the server side. The workload of setting the refresh rate is large and cumbersome, and it is not smart enough. Summary of the invention

[0004] The present disclosure provides an application running method, device, electronic device and storage medium to at least solve the problem in the above-mentioned related technologies that the refresh rate setting is labor-intensive and cumbersome and not intelligent enough.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an application running method, comprising: determining a target application whose power consumption needs to be improved and a target layer whose refresh rate needs to be improved; identifying the type of the target layer using a decision tree model; adjusting the refresh rate value of the target layer when the type of the target layer belongs to a power consumption adjustable layer; determining the refresh rate value of a terminal screen based on the adjusted refresh rate value of the target layer; and running the target application based on the refresh rate value of the terminal screen.

[0006] Optionally, adjusting the refresh rate value of the target layer includes: adjusting the refresh rate value of the target layer based on at least one of a center position relationship between the target layer and the terminal screen, an area relationship between the target layer and the terminal screen, and a type of the target layer.

[0007] Optionally, the adjusting the refresh rate value of the target layer includes: determining a preset minimum refresh rate value corresponding to the target layer based on at least one of a center position relationship between the target layer and the terminal screen, an area relationship between the target layer and the terminal screen, and a type of the target layer; and adjusting the refresh rate value of the target layer based on the preset minimum refresh rate value corresponding to the target layer and an actual sampling refresh rate value of the target layer.

[0008] Optionally, after running the target application based on the refresh rate value of the terminal screen, it also includes: obtaining power consumption and performance status data of the terminal; optimizing the refresh rate value of the target layer through a reinforcement learning model based on the power consumption and performance status data of the terminal; determining the optimized refresh rate value of the terminal screen based on the optimized refresh rate value of the target layer; and running the target application based on the optimized refresh rate value of the terminal screen.

[0009] Optionally, the refresh rate value of the target layer is optimized based on the power consumption and performance status data of the terminal through a reinforcement learning model, including: optimizing the refresh rate value of the target layer based on the power consumption and performance status data of the terminal and a preset state action table of the reinforcement learning model to obtain the optimized refresh rate value of the target layer, wherein the preset state action table includes multiple states, each of the states corresponds to multiple refresh rate adjustment action types, and each of the refresh rate adjustment action types corresponds to its own state action penalty value.

[0010] Optionally, determining the refresh rate value of the terminal screen based on the adjusted refresh rate value of the target layer includes: determining the refresh rate value of the terminal screen based on the adjusted refresh rate value of the target layer and actual sampling refresh rate values ​​of other application layers included in the target application.

[0011] Optionally, determining the target application whose power consumption needs to be improved includes: acquiring historical behavior characteristic data of foreground operation of each application installed in the terminal; and determining the application whose historical behavior characteristic data exceeds a predetermined range as the target application.

[0012] Optionally, determining the target layer of the target application whose refresh rate needs to be improved includes: while the target application is running, obtaining running behavior characteristic data of the target application; and when the running behavior characteristic data exceeds a preset range, determining that the application layer with the highest actual sampling refresh rate value among multiple application layers included in the target application is the target layer.

[0013] Optionally, the use of a decision tree model to identify the type of the target layer includes: obtaining layer display behavior characteristic data of the target layer of the target application within a predetermined time period; and inputting the layer display behavior characteristic data into the decision tree model to identify the type of the target layer.

[0014] Optionally, obtaining the layer display behavior characteristic data of the target layer of the target application within a predetermined time period includes: collecting the layer display behavior characteristic data of the target layer at preset time intervals within the predetermined time period, and obtaining multiple groups of layer display behavior characteristic data corresponding to the target layer.

[0015] Optionally, inputting the layer display behavior characteristic data into the decision tree model to identify the type of the target layer includes: inputting each group of layer display behavior characteristic data of the multiple groups of layer display behavior characteristic data corresponding to the target layer into the decision tree model respectively, and obtaining identification results corresponding to each group of layer display behavior characteristic data; when the multiple groups of identification results corresponding to the multiple groups of layer display behavior characteristic data all indicate that the target layer is the power consumption adjustable layer, determining that the target layer is the power consumption adjustable layer.

[0016] Optionally, the layer display behavior characteristic data includes at least one of the following items: the position of the target layer on the Z-axis, the percentage of the area of ​​the target layer to the area of ​​the terminal screen, the transparency of the target layer, whether the target layer corresponds to a specific keyword, whether the target layer belongs to a preset surface view layer type, the content concentration of the target layer, the horizontal scrolling step concentration of the target layer, the refresh rate value of the target layer, and the content flow of the target layer.

[0017] Optionally, the application layer is pre-divided into multiple types according to power consumption, and at least one predetermined type among the multiple types is set as the power consumption adjustable layer.

[0018] Optionally, the multiple types include a first type, a second type and a third type, and the first type and the second type are set as the power consumption adjustable layer; wherein the first type is an application layer for displaying the substantial content of the target application; the second type is an application layer for displaying comment information related to the substantial content; and the third type is an application layer type included in the application except the first type and the second type.

[0019] According to a second aspect of an embodiment of the present disclosure, there is provided an application running device, comprising: a target layer determination module, configured to determine a target application whose power consumption is to be improved and a target layer whose refresh rate of the target application is to be improved; an identification module, configured to identify the type of the target layer using a decision tree model; an adjustment module, configured to adjust the refresh rate value of the target layer when the type of the target layer belongs to a power consumption adjustable layer; a first determination module, configured to determine the refresh rate value of a terminal screen based on the adjusted refresh rate value of the target layer; and a first running module, configured to run the target application based on the refresh rate value of the terminal screen.

[0020] Optionally, the adjustment module is configured to adjust the refresh rate value of the target layer based on at least one of a center position relationship between the target layer and the terminal screen, an area relationship between the target layer and the terminal screen, and a type of the target layer.

[0021] Optionally, the adjustment module is configured to: determine a preset minimum refresh rate value corresponding to the target layer based on at least one of a center position relationship between the target layer and the terminal screen, an area relationship between the target layer and the terminal screen, and a type of the target layer; and adjust the refresh rate value of the target layer based on the preset minimum refresh rate value corresponding to the target layer and an actual sampling refresh rate value of the target layer.

[0022] Optionally, the application running device also includes: a status data acquisition module, configured to acquire the power consumption and performance status data of the terminal; an optimization module, configured to optimize the refresh rate value of the target layer based on the power consumption and performance status data of the terminal through a reinforcement learning model; a second determination module, configured to determine the optimized refresh rate value of the terminal screen based on the optimized refresh rate value of the target layer; and a second running module, configured to run the target application based on the optimized refresh rate value of the terminal screen.

[0023] Optionally, the optimization module is configured to: optimize the refresh rate value of the target layer based on the power consumption and performance status data of the terminal and the preset state action table of the reinforcement learning model, and obtain the optimized refresh rate value of the target layer, wherein the preset state action table includes multiple states, each of the states corresponds to multiple refresh rate adjustment action types, and each of the refresh rate adjustment action types corresponds to its own state action penalty value.

[0024] Optionally, the first determination module is configured to determine the refresh rate value of the terminal screen based on the adjusted refresh rate value of the target layer and actual sampled refresh rate values ​​of other application layers included in the target application.

[0025] Optionally, the target layer determination module is configured to: obtain historical behavior characteristic data of foreground operation of each application installed in the terminal; and determine the application whose historical behavior characteristic data exceeds a predetermined range as the target application.

[0026] Optionally, the target layer determination module is configured to: obtain running behavior characteristic data of the target application while the target application is running; and when the running behavior characteristic data exceeds a preset range, determine that the application layer with the highest actual sampling refresh rate value among the multiple application layers included in the target application is the target layer.

[0027] Optionally, the identification module is configured to: obtain layer display behavior characteristic data of the target layer of the target application within a predetermined time period; and input the layer display behavior characteristic data into the decision tree model to identify the type of the target layer.

[0028] Optionally, the identification module is configured to: collect layer display behavior characteristic data of the target layer at preset time intervals within the predetermined time period, and obtain multiple groups of layer display behavior characteristic data corresponding to the target layer.

[0029] Optionally, the identification module is configured to: input each group of layer display behavior characteristic data among the multiple groups of layer display behavior characteristic data corresponding to the target layer into the decision tree model respectively, and obtain the identification results corresponding to each group of layer display behavior characteristic data; when the multiple groups of identification results corresponding to the multiple groups of layer display behavior characteristic data all indicate that the target layer is the power consumption adjustable layer, determine that the target layer is the power consumption adjustable layer.

[0030] Optionally, the layer display behavior characteristic data includes at least one of the following items: the position of the target layer on the Z-axis, the percentage of the area of ​​the target layer to the area of ​​the terminal screen, the transparency of the target layer, whether the target layer corresponds to a specific keyword, whether the target layer belongs to a preset surface view layer type, the content concentration of the target layer, the horizontal scrolling step concentration of the target layer, the refresh rate value of the target layer, and the content flow of the target layer.

[0031] Optionally, the application layer is pre-divided into multiple types according to power consumption, and at least one predetermined type among the multiple types is set as the power consumption adjustable layer.

[0032] Optionally, the multiple types include a first type, a second type and a third type, and the first type and the second type are set as the power consumption adjustable layer; wherein the first type is an application layer for displaying the substantial content of the target application; the second type is an application layer for displaying comment information related to the substantial content; and the third type is an application layer type included in the application except the first type and the second type.

[0033] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement an application running method according to the present disclosure.

[0034] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute an application running method according to the present disclosure.

[0035] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0036] The present disclosure can use a decision tree model to identify the type of target layer whose refresh rate is to be improved, and when the type of the target layer belongs to the power consumption adjustable layer, the refresh rate value of the target layer can be automatically adjusted, thereby affecting the change in the refresh rate value of the terminal screen. In this way, it is possible to avoid the system refresh rate being forced to increase, thereby avoiding the phenomenon that the terminal's operating current is high due to the system refresh rate being forced to increase, thereby avoiding the phenomenon that the terminal power consumption is excessive, and more resources can be saved. In addition, since the present disclosure does not need to manually set the corresponding refresh rate for different APPs one by one, that is, the entire process does not require manual intervention, it avoids repeated and cumbersome setting processes, making the refresh rate setting more intelligent and convenient.

[0037] Furthermore, the applications can be screened by the historical behavior feature data of the foreground running of the applications, and then the applications worth optimizing can be obtained. This avoids the phenomenon of ineffective optimization of a large number of applications that have no optimization value, and can save optimization resources.

[0038] Furthermore, since multiple sets of collected data are used for identification, the probability of misjudgment can be greatly reduced, and the recognition accuracy of the target layer type can be guaranteed.

[0039] Furthermore, by adjusting the refresh rate value of the target layer in combination with features of multiple dimensions, the accuracy of the refresh rate value adjustment can be ensured.

[0040] Furthermore, by adjusting the refresh rate value of the target layer based on the preset minimum refresh rate value and the actual sampled refresh rate value, the objectivity of the refresh rate value adjustment can be ensured. Moreover, since the determined refresh rate value is a range interval, the phenomenon that the system refresh rate is forced to be increased by directly using the actual sampled frame rate of the target layer as its final refresh rate value can be avoided, thereby avoiding excessive power consumption of the terminal and saving resources.

[0041] Furthermore, when determining the refresh rate value of the terminal screen, the adjusted refresh rate value of the target layer and the actual sampled refresh rate values ​​of other application layers are comprehensively considered, which can not only ensure that the power consumption of the terminal is reduced, but also ensure that the performance of the terminal remains basically unchanged.

[0042] Furthermore, since the converged Q table is obtained through a large amount of training, it has guiding significance for the optimization process of the actual refresh rate value, and can ensure that the refresh rate value obtained after optimization achieves a good balance between terminal power consumption and terminal performance.

[0043] Furthermore, since the preset state-action table of the reinforcement learning model records the state-action penalty value corresponding to each action type in multiple action types in each state, it is only necessary to use the current state of the terminal to query in the trained Q table to find the best action to be performed in the current state. Moreover, the optimal refresh rate value that balances the terminal power consumption and terminal performance can be found through a limited number of queries. Running the target application based on the optimal refresh rate value can greatly reduce the power consumption of the terminal. At the same time, it can also ensure that the video playback quality is not affected, that is, it can ensure a smooth viewing experience and avoid flickering and freezing on the terminal screen.

[0044] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0046] Figure 1 is a schematic diagram showing an interface of a video APP displaying bullet comments while playing a video according to an exemplary embodiment of the present disclosure;

[0047] Figure 2 is a schematic diagram of an interface showing that a video APP according to an exemplary embodiment of the present disclosure turns off the bullet screen function while playing a video;

[0048] Figure 3 is a schematic diagram showing a barrage layer according to an exemplary embodiment of the present disclosure;

[0049] Figure 4 is a flowchart illustrating an application running method according to an exemplary embodiment of the present disclosure;

[0050] Figure 5 is a schematic diagram showing contents included in a view hierarchy of a bullet screen layer according to an exemplary embodiment of the present disclosure;

[0051] Figure 6 is a schematic diagram showing five consecutive frames of display data content of a bullet screen layer according to an exemplary embodiment of the present disclosure;

[0052] Figure 7 is a schematic diagram showing normalization of display content of a row in a certain frame of a bullet screen layer according to an exemplary embodiment of the present disclosure;

[0053] Figure 8 is a schematic diagram showing normalization of display content of the 64th line in the 1st frame and the 2nd frame of the bullet screen layer according to an exemplary embodiment of the present disclosure;

[0054] Fig. 9 is an original scatter plot showing the horizontal scrolling step of the bullet screen layer according to an exemplary embodiment of the present disclosure;

[0055] Fig.10 is a scatter diagram showing a horizontal scroll step correction of a bullet screen layer according to an exemplary embodiment of the present disclosure;

[0056] Fig.11 is a schematic diagram showing characteristic parameters used by different application layers according to an exemplary embodiment of the present disclosure;

[0057] Fig.12 is a schematic diagram showing a training data set of a decision tree model according to an exemplary embodiment of the present disclosure;

[0058] Fig.13is a schematic diagram showing a first-level root node of a decision tree according to an exemplary embodiment of the present disclosure;

[0059] Fig.14 is a partial node schematic diagram showing the first layer and the second layer of a decision tree model according to an exemplary embodiment of the present disclosure;

[0060] Fig.15 is a schematic diagram showing some nodes included in a complete decision tree model according to an exemplary embodiment of the present disclosure;

[0061] Fig.16 is a schematic diagram showing the difference in performance scores between a special layer and a general high-traffic layer according to an exemplary embodiment of the present disclosure;

[0062] Fig.17 is a schematic diagram showing the segmentation of current and the level of current factor dc according to an exemplary embodiment of the present disclosure;

[0063] Fig.18 is a schematic diagram showing segmenting of performance scores and levels of performance score factors df according to an exemplary embodiment of the present disclosure;

[0064] Fig.19 is a schematic diagram showing action types according to an exemplary embodiment of the present disclosure;

[0065] Fig. 20 is a schematic diagram showing a single-step penalty table according to an exemplary embodiment of the present disclosure;

[0066] Fig.21 is a schematic diagram showing a Q table in an initialization state according to an exemplary embodiment of the present disclosure;

[0067] Fig. 22 is a schematic diagram showing Sarsa Learning (S, A) state transition according to an exemplary embodiment of the present disclosure;

[0068] Fig.23 is a schematic diagram showing a Sarsa Learning Q table after updating according to an exemplary embodiment of the present disclosure;

[0069] Fig.24 is a schematic diagram showing a Q value in the S(1,1) state according to an exemplary embodiment of the present disclosure;

[0070] Fig.25 is a block diagram illustrating an application execution device according to an exemplary embodiment of the present disclosure;

[0071] Fig.26 is a block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0072] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0073] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation methods described in the following examples do not represent all implementation methods consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the attached claims.

[0074] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0075] At present, various types of APPs emerge in an endless stream, and each type of APP can realize its unique functions. For example, video APPs can be used to play videos, live broadcast APPs can be used for live broadcasts, game APPs can be used to provide game services for users, and so on. In order to enhance the user's interactive experience and create a network environment atmosphere for multi-user viewing and discussion in real time, the above-mentioned types of APPs generally have the function of "bullet screen". "Bullet screen" can refer to the commentary subtitles that pop up when watching videos / live broadcasts / playing games on the Internet, and the bullet screen is generally displayed on the video interface / live broadcast interface / game interface. Further, there are some phenomena: the more popular the video / live broadcast, the more historical bullet screens uploaded and distributed by fans, so that there is bullet screen scrolling throughout the video playback; the user's viewing habit is often to watch the video content while occasionally looking at the bullet screen content, so the subtle changes in the bullet screen display will not be easily noticed; the bullet screen function is more practical, which can strengthen the viewing atmosphere of fans and thus increase the popularity. Among them, the "fans" involved in this disclosure specifically refer to a user group that likes a certain person or a certain thing.

[0076] It should be noted that an APP may include multiple application layers, and the functions of each application layer are different from each other. For example, for a video APP, it may include at least a video layer and a barrage layer, and the barrage layer may be overlaid on the video layer.

[0077] When the terminal device is running an APP, it generally works in "adaptive refresh rate mode", that is, the refresh rate of the terminal screen is determined by the refresh rate (fps) and area ratio (weight) of each layer in the multiple layers contained in the APP. For example, as mentioned above, for video APPs, the refresh rate of its video layer is generally 30 frames per second, while the refresh rate of its bullet screen layer may reach 120 frames per second. When the area ratios of the video layer and the bullet screen layer meet the preset conditions, the refresh rate of the terminal screen, that is, the system refresh rate, will be forced to maintain at 120 frames per second, but in fact, the video to be played does not need such a high refresh rate at all. At this time, since the refresh rate value of the terminal screen is forced to be increased, the load of the central processing unit (CPU) / graphics processing unit (GPU) / liquid crystal display (LCD) used for rendering synthesis will increase, resulting in a substantial increase in the power consumption of the terminal, which will cause the terminal to heat up, freeze, and lose power quickly.

[0078] For example, if a video APP opens the "bullet screen" function when playing a video, the operating current of the terminal can reach 530mA, and the display content of the APP when playing the video can be as follows: Figure 1 See Figure 1 , Figure 1 1 is a schematic diagram showing an interface of a video APP displaying bullet screen while playing a video according to an exemplary embodiment of the present disclosure. At this time, 4 rows of bullet screens are suspended and displayed above the playback interface, and these bullet screens scroll horizontally from right to left.

[0079] At this time, if the "bullet screen" function is turned off, the operating current of the terminal becomes 326mA. The display interface after turning off the "bullet screen" function is as follows: Figure 2 See Figure 2 , Figure 2 1 is a schematic diagram showing an interface of a video APP according to an exemplary embodiment of the present disclosure turning off the bullet screen function while playing a video. At this time, only the video content is displayed in the playback interface, and the words "bullet screen is turned off" are displayed at the top of the playback interface.

[0080] visible, Figure 1The playback interface with the barrage function turned on in Figure 2 Compared with the playback interface with the barrage function turned off, the only difference is that a transparent barrage layer is covered on the video layer. Figure 3 is a schematic diagram showing a bullet screen layer according to an exemplary embodiment of the present disclosure. Figure 3 , 4 lines of barrage content are displayed in the upper half of the barrage layer.

[0081] It should be noted that the working current of the terminal is proportional to the refresh rate value of the terminal screen, that is, the larger the refresh rate value of the terminal screen, the higher the working current of the terminal and the greater the power consumption of the terminal. As mentioned above, when the video APP contains both the video layer and the bullet screen layer, if the refresh rate (fps) of the bullet screen layer is the highest and the area is also large, and the terminal screen also supports such a high refresh rate, then the system refresh rate is close to the refresh rate of the bullet screen layer, that is, at this time, the scrolling frame rate of the bullet screen layer actually determines the final refresh rate of the terminal screen. And, as mentioned above, when the terminal is running the video APP, if the bullet screen function is turned on, its working current is 530mA; if the bullet screen function is turned off, its working current is 326mA. It can be seen that when the bullet screen function is turned on, the working current of the terminal will be much higher than that when the bullet screen function is not turned on, and the increase is 62.6%, which is a very large increase. Therefore, the existing "adaptive refresh rate mode" will cause the terminal screen to maintain a high refresh rate level, which will lead to unnecessary waste of resources.

[0082] In the prior art, in order to solve the high power consumption problem caused by turning on the bullet screen function, developers mainly test different apps and then manually limit the refresh rate of each app to a lower value on the server side. This leads to the following problems:

[0083] 1. Each user has different demands. That is, for the same type of APP, different users may have APPs developed by different manufacturers installed on their terminals. This requires developers to verify the refresh rate value of each APP one by one and set it on the server side. The operation process is relatively cumbersome.

[0084] 2. For the non-playing interface of the APP, such as the menu interface of the APP, artificially limiting the refresh rate to a lower value will cause the user to experience a freeze when sliding the non-playing interface, that is, the non-playing interface does not slide smoothly.

[0085] 3. For video apps, if the video source itself is a high-frame rate video, the APP will be forced to play at a low frame rate, which will result in the advantages of high-frame rate video not being brought into play, and users will not be able to enjoy high-quality viewing.

[0086] In order to solve the above-mentioned problems existing in the related art, the application operation method, device, electronic device and storage medium provided by the present disclosure can use a decision tree model to identify the type of target layer whose refresh rate is to be improved, and when the type of the target layer belongs to the power consumption adjustable layer, the refresh rate value of the target layer can be automatically adjusted, thereby affecting the change of the refresh rate value of the terminal screen. In this way, it is possible to avoid the system refresh rate being forced to increase, thereby avoiding the phenomenon that the terminal's working current is high due to the system refresh rate being forced to increase, thereby avoiding the phenomenon that the terminal power consumption is excessive, and more resources can be saved. In addition, since the present disclosure does not need to manually set the corresponding refresh rate for different APPs one by one, that is, the entire process does not require manual intervention, thus avoiding repeated and cumbersome setting processes, making the refresh rate setting more intelligent and convenient.

[0087] Next, we will refer to Figures 4 to 26 The application running method, application running device, electronic device and storage medium according to the present disclosure are described in detail.

[0088] Figure 4 is a flowchart illustrating an application execution method according to an exemplary embodiment of the present disclosure.

[0089] Reference Figure 4 In step 401, the target application whose power consumption is to be improved and the target layer whose refresh rate is to be improved may be determined. The target application may include multiple application layers, each of which is used to implement its unique functions. For example, if the target application is a video application, it may at least include a barrage layer, a video layer, and the like. The barrage layer is mainly used to display barrage, and the video layer is mainly used to display video images. Moreover, the refresh rates corresponding to different application layers are generally different from each other.

[0090] According to an exemplary embodiment of the present disclosure, historical behavior characteristic data of foreground operation of each application installed in the terminal may be acquired, and then an application whose historical behavior characteristic data exceeds a predetermined range may be determined as a target application.

[0091] It should be noted that after the terminal is turned on, a thread can be automatically started, that is, a system service can be automatically created to monitor and sample the relevant feature data of each APP used in the foreground, and record it. The above-mentioned "historical behavior feature data" may include at least one of the following items: historical total operation time (FG use time), historical total power consumption (FG app battery drain), historical total power consumed by the video-related service module of the application (FGVideo battery drain), and historical average operation current (FG app average current). Among them, "FG" means "foreground".

[0092] Furthermore, you can add a listener to the Activity Manager Service to monitor the switching of the foreground app, and compare the difference in battery data (batterystats) when the app enters the foreground and exits the foreground to obtain the above "historical behavior feature data". For example, you can execute the following command: Runtime.exec("adb shell dumpsys batterystats").

[0093] Among them, "batterystats" is a function supported by Android itself. The principle is that the device manufacturer needs to add a power_profile.xml to the system. This power_profile.xml will define the standard measurement value of the current consumed by each hardware component or software module of each manufacturer. When the relevant module or process interface is called by the system, the battery data service (batterystats service) will automatically convert a standard power consumption value according to its usage time and other rules. The advantage of this method is that it does not need to be measured in real time, so there is no additional load.

[0094] For example, for a certain app, the following data can be sampled (running in the foreground for 20 minutes):

[0095] “When entering the foreground:

[0096] UID u0a345:0.0178fg:0.00700(cpu=0.0107(394ms)cpu:fg=0.00700system_services=0.00706sensors=0.0000526(1s 113ms))

[0097] When exiting the foreground:

[0098] UID u0a345:63.8fg:4.91(screen=48.0(19m 36s 158ms)cpu=4.66(8m50s553ms)cpu:fg=4.66(24s 744ms)audio=3.65(19m 12s 546ms)video=7.01(19m20s630ms)system_services=0.198sensors=0.0555(19m 35s 413ms)wifi=0.255(5s322ms)wifi:fg=0.255)”

[0099] The power consumption value of the app in the bright screen state can be calculated based on the difference in battery data when the app enters the foreground and exits the foreground. For example, based on the above sampling results, it can be known that the total power consumed by the app from the time it enters the foreground is 0.0178mAh, the total power consumed by the app from the time it leaves the foreground is 63.8mAh, the total power consumed by the video-related service module from the time the app leaves the foreground is 7.01mAh, and so on.

[0100] For example, you can get the following behavioral characteristics data for the app in the foreground within 20 minutes:

[0101] Total historical running time = 20 minutes

[0102] Total power consumed in history = 63.8-0.0178 = 63.7822 mAh

[0103] The total historical power consumed by the video-related service module of the application = 7.01-0 = 7.01 mAh

[0104] Because our optimization target is the app refresh rate, the data here is when the app is in the foreground and the screen is on. The power consumption of the app in the background has nothing to do with the display refresh rate, so it does not need to be considered.

[0105] Generally speaking, when playing a video, the video module consumes power ("video battery drain") and the audio module consumes power ("audio battery drain"). "Video module consumes power" means that the app calls services such as video encoding and decoding / data flow. For example, during video playback, processes such as media.codec / media.swcodec / media.hwcode / MediaServer will be called, thereby triggering the calculation of their usage time, which is finally converted into standard power. If these media services are not triggered, the video module of the APP generally consumes 0 or very little power (audio playback generally only consumes power from the audio module), so it can be used to distinguish between ordinary applications and video applications.

[0106] For each APP installed in the terminal, a copy of the above data can be recorded. If it is used multiple times in the foreground, the data for each day can be accumulated. For example, only the usage data for the last 7 days can be retained, that is, the user's app usage preferences and battery consumption behavior data in the last week. In addition, it can be scheduled every night when the terminal is idle (Idle) to decide which apps are our optimization targets based on these data. In this way, it will not hinder the normal use of users, and can also filter out the applications that need to be optimized.

[0107] The total usage time, total power consumption, and total power consumption of the video-related service module (video) for each app in the past 7 days are accumulated as follows:

[0108]

[0109]

[0110]

[0111] Among them, “app total use time” means the total usage time of the APP within 7 days, “app use time” means the usage time of the APP in one day, “app total battery drain” means the total power consumed by the APP in 7 days, “app battery drain” means the power consumed by the APP in one day, “app total video drain” means the total power consumed by the video-related service modules of the APP in 7 days, and “app video drain” means the power consumed by the video-related service modules of the APP in one day.

[0112] In addition, generally speaking, for video apps, the average current (app average current) must be above 300mA to be worth optimizing. Therefore, it is necessary to calculate the average current of the app to exclude apps that have been used for a long time and have a low average current despite a high total power consumption:

[0113]

[0114] Assume that the predetermined range, that is, the reference threshold value of each parameter is as follows:

[0115] Apps that users frequently use should meet the following requirements:

[0116] TH app total use time ≥5 hours

[0117] Apps that consume more power should meet the following requirements:

[0118] TH app total battery drain ≥1500mAh

[0119] Apps that deserve improved power consumption should meet the following requirements:

[0120] TH app average current ≥300mA

[0121] Apps with video-related service modules that consume a lot of power should meet the following requirements:

[0122] TH app total video drain ≥36mAh

[0123] It should be noted that if the app objects to be optimized are not limited to the video category, you do not need to consider TH app total video drain This condition. In this way, all apps exceeding the above threshold can be selected, that is, APP = {a1, a2, a3, ... aN}, and these selected apps are high-current video / live broadcast / game apps that are worth optimizing in the subsequent process.

[0124] In this way, applications can be screened based on their historical behavior feature data when running in the foreground, and applications that are worth optimizing can be obtained. This avoids the phenomenon of ineffective optimization of a large number of applications that have no optimization value, and can save optimization resources.

[0125] According to an exemplary embodiment of the present disclosure, while the target application is running, the running behavior characteristic data of the target application can be obtained. In the case where the running behavior characteristic data exceeds a preset range, the application layer with the highest corresponding actual sampling refresh rate value among the multiple application layers included in the target application can be determined as the target layer. Among them, the "running behavior characteristic data" can include at least one of the following items: "running average current THapp average current ”, “The total running power consumed by the video-related service modules of the application is TH app video total drain ", "The current refresh rate value of the terminal screen is TH system refresh rate ”.

[0126] For example, the average current (Averagecurrent) of the target application during playback, the total power consumed by the video-related service modules of the application (video drain), and the current refresh rate of the terminal screen can be collected every 5 seconds. Assuming that the collection is performed continuously for about 30 times, the values ​​of these three parameters can be entered into the next step when they all meet the following conditions:

[0127] TH app average current ≥300mA

[0128] TH app video total drain ≥3mAh(30*5seconds)

[0129] TH system refresh rate ≥30Hz

[0130] It should be noted that if the target app to be optimized is not limited to video apps, there is no need to determine the TH app video total drain Furthermore, if any of the above three parameters does not meet the preset condition, detection may be performed again after a period of time, such as a few minutes, until the above three parameters all meet their respective preset conditions.

[0131] Furthermore, the surface layer (SurfaceLayer) feature data of the target application when it is playing can be collected at regular intervals, for example, every 5 seconds. This step of detection is mainly used to confirm the layer distribution when the target application is playing. Taking ordinary scenarios as an example, generally speaking, if there is a visible surface view layer (Surface View Layer) with large traffic, then even if its own refresh rate value is not the highest refresh rate, the refresh rate changes of other layers will greatly affect its power consumption changes. In this case, we call these "other layers" "Body Surface View Layer".

[0132] Take the example of a video app with a bullet screen function. Generally speaking, an Android app will use a surface view (SurfaceView) / texture view (TextureView) for real-time video playback, and another surface view / texture view to cover the video or parallel for real-time scrolling of bullet screen. Each surface view / texture view is independently refreshed by directly communicating with the display synthesis system (Surface Flinger), so at least two display layers (display layers) will be formed here, namely the video layer and the bullet screen layer, and the refresh speed of each layer, namely the frame rate (fps), is only related to the update speed of its own display content.

[0133] When there are two application layers and their respective refresh rates are inconsistent, the system refresh rate, that is, the refresh rate of the terminal screen, generally takes the higher value between the two. Therefore, for low frame rate videos, the refresh rate of the video layer is low, and the refresh rate of the bullet screen layer is high. The system refresh rate will depend on the refresh rate of the bullet screen layer. Therefore, by appropriately reducing the refresh rate of the bullet screen layer, the system refresh rate can be reduced, thereby achieving the purpose of power saving; for high frame rate videos, the refresh rate of the video layer is high, and adjusting the refresh rate of the bullet screen layer alone will not reduce the system refresh rate, and thus will not affect the experience of high frame rate video playback.

[0134] Further, return to reference Figure 1 , Figure 1 The following is a schematic diagram of the interface that displays bullet comments while playing a video. You can use "adb shell dumpsys SurfaceFlinger-list|grep com.tencent.qqlive" to obtain Figure 1 The data of multiple application layers of the video APP shown in FIG. The positions of the multiple application layers are arranged according to the vertical relationship of the Z axis or the coverage area.

[0135] Assumptions Figure 1 The video APP shown in FIG. 4 includes a total of 4 application layers, and the data obtained from the 4 application layers can be:

[0136] "*Layer 0xb40000744e318000(SurfaceView / / PS: bullet screen layer

[0137] [com.tencent.qqlive / com.tencent.qqlive.ona.activity.VideoDetailActivity]@0(BLAST)#606)

[0138] *Layer 0xb40000744e16b000(SurfaceView / / PS: video layer

[0139] [com.tencent.qqlive / com.tencent.qqlive.ona.activity.VideoDetailActivity]@0(BLAST)#632)

[0140] *Layer 0xb40000744e0c1000 / / PS: video menu layer

[0141] (com.tencent.qqlive / com.tencent.qqlive.ona.activity.VideoDetailActivity$_24703#601)

[0142] *Layer 0xb40000744e2e0000(Bounds for / / PS:activity main UI

[0143] com.tencent.qqlive / com.tencent.qqlive.ona.activity.VideoDetailActivity@0#604)”

[0144] The display content of the video layer (Video Layer) used to display the video playback content is as follows: Figure 2 As shown, the relevant data of the video layer is as follows:

[0145] (Video layer FPS = 24)

[0146] *Layer 0xb40000744e16b000(SurfaceView

[0147] [com.tencent.qqlive / com.tencent.qqlive.ona.activity.VideoDetailActivity]@0(BLAST)#632)

[0148] Among them, the display content of the barrage layer (Barrage Layer) used to display the rolling content of the barrage is as follows Figure 3 As shown, the relevant data of the bullet screen layer is as follows:

[0149] (layer FPS=120)

[0150] *Layer 0xb40000744e318000(SurfaceView

[0151] [com.tencent.qqlive / com.tencent.qqlive.ona.activity.VideoDetailActivity]@0(BLAST)#606)

[0152] Other characteristic parameters of each application layer can also be collected from dumpsys: the position of the application layer on the Z axis [Layout Position (Z direction and X, Y location)], the area of ​​the application layer (Layoutarea), the transparency of the application layer [Layer Transparency (blend / alpha / isOpaque)], the transparent color of the application layer (Layer Transparent Color), the visibility of the application layer (Layer visibility), etc.

[0153] Through the data of the four application layers obtained above, it can be determined that the current video APP playback interface has two surface view layers, namely: @0(BLAST)#606 and @0(BLAST)#632.

[0154] Generally speaking, the video layer is based on Surface View, and the game layer is also based on GL_Surface View. These two SurfaceView-based application layers are likely to be high traffic layers. If there are other visible application layers with higher refresh rates in the app at the same time, then this application layer may also be a high traffic layer.

[0155] The refresh speed of the above application layers, that is, the frame rate (fps), can also be collected from the display synthesis system at the same time. In addition, the collected frame rate of the application layer is generally a decimal. The frame rate of the application layer can be rounded up to the refresh rate (Refresh Rate) supported by the terminal screen, and the refresh rate value obtained after rounding is used as the refresh rate value corresponding to the application layer. Exemplarily, the refresh rate supported by the terminal screen, that is, the available refresh rate, can include 7 levels, namely: [10Hz, 24Hz, 30Hz, 48Hz, 60Hz, 96Hz, 120Hz]. The refresh rate of the terminal screen can only be adjusted and switched within the refresh rate range it supports.

[0156] For example, Figure 1The refresh rates of the four application layers included in the video APP shown in FIG. 1 , that is, the sampling results of the frame rate (fps) can be as follows:

[0157]

[0158]

[0159]

[0160]

[0161] It can be seen that by rounding up the refresh rate of the application layer, that is, the frame rate, the refresh rates of the above four application layers are: [120Hz, 24Hz, 10Hz, 0Hz]. Among the above four application layers, there are two surface view layers, whose refresh rate values ​​are 120Hz and 24Hz respectively; the refresh rate values ​​of the other two application layers are 10Hz and 0Hz respectively, and the refresh rate values ​​of different application layers vary greatly.

[0162] As mentioned above, it can be determined that the application layer with the highest actual sampling refresh rate value among the multiple application layers contained in the target application is the target layer, and the actual sampling frame rates corresponding to the above-mentioned four application layers, that is, the actual sampling refresh rate values ​​are: 101.02Hz, 23.73Hz, 7.12Hz, and 0Hz, respectively. Therefore, it can be determined that the barrage layer with the highest actual sampling refresh rate value among these four application layers is the above-mentioned target layer.

[0163] In step 402, a decision tree model may be used to identify the type of the target layer.

[0164] According to an exemplary embodiment of the present disclosure, layer display behavior feature data of a target layer of a target application within a predetermined time period may be obtained, and then the layer display behavior feature data may be input into a decision tree model to identify the type of the target layer. The decision tree model may be, but is not limited to, a C4.5 decision tree model.

[0165] According to an exemplary embodiment of the present disclosure, the "layer display behavior characteristic data" may include at least one of the following items: the position of the target layer on the Z axis (Layer Z Position), the percentage of the area of ​​the target layer to the area of ​​the terminal screen (Layer Size), the transparency of the target layer (Layer Transparency), whether the target layer corresponds to a specific keyword (Layer Surface View Keyword), whether the target layer belongs to a preset surface view layer type (Layer SurfaceView Type), the content concentration of the target layer (Layer Content Density), the horizontal scroll step concentration of the target layer (Layer Horizontal Scroll Step Density), the refresh rate value of the target layer (Layer RefreshRate), and the content traffic of the target layer (Layer Content Traffic).

[0166] Furthermore, you can first use "adb shell dumpsys activity top" to collect the view tree (ViewTree.) information. Among them, "adb shell dumpsys activity top" can be a function that comes with Android. Taking the aforementioned Layer#606 (barrage layer) as an example, the content of its view hierarchy (View Hierarchy) is specifically shown in the attached Figure 5 , Figure 5 It is a schematic diagram showing the contents included in the view hierarchy (View Hierarchy) of the bullet screen layer according to an exemplary embodiment of the present disclosure.

[0167] Reference Figure 5 , from the view tree of the barrage layer, the characteristic parameters of the barrage layer can be collected: whether it contains specific keywords (Layer Surface View Keyword), the position on the Z axis (Layout Positionion), and the area (Layoutarea). The reason for collecting "Layer Surface View Keyword" here is that many apps currently integrate and use several open source barrage code library software development kits (barrage code library SDK). The view names related to barrage in these SDKs generally contain the following string: "danmake / Danmu / Tanmu / bullet / barrage / ..." Therefore, the characteristic parameter of "whether it contains specific keywords" can be used as a judgment condition for determining whether a certain application layer is a special layer (SpecialLayer), that is, the barrage layer.

[0168] Furthermore, the display data content of 5 consecutive frames of the target layer can also be collected. For example, the display data content can be collected using the Android built-in "adb shell vndservice call display.qservice 21i32 5i32 1i32 1", or the data can be directly collected from the display synthesis system.

[0169] Taking Layer#606 (bullet screen layer) as an example, the layer surface buffer content of 5 consecutive frames can be as follows Figure 6 As shown, Figure 6 is a schematic diagram showing five consecutive frames of display data content of a bullet screen layer according to an exemplary embodiment of the present disclosure. Figure 6 The five frames of the collected bullet screen layer are numbered as frame 1, frame 2, frame 3, frame 4, and frame 5. And, Figure 6 The lower right corner of the figure also shows the Barrage Scroll Rule: From right to left Scroll; Each time Scroll x pixels, and the actual scrolling of these 5 frames: This Image frame 1-5 each scroll 4~5 pixels.

[0170] Based on the layer surface buffer content, the following characteristic parameters of the target layer can be calculated: content density (Layer Content Density), horizontal scroll step density (Layer Horizontal Scroll Step Density), content traffic (Layer Content Traffic).

[0171] Among them, the content concentration can be calculated by the following formula:

[0172]

[0173] "pxiels count not transparent color " indicates the number of opaque pixels, "pxiels count all ” indicates the total number of pixels.

[0174] Assume that all transparent points are white and all non-transparent points are normalized to black. Figure 72 is a schematic diagram showing the normalization of display content of a row in a certain frame of a bullet screen layer according to an exemplary embodiment of the present disclosure. Figure 7 , pxiels count in this row not transparent color =8, pxiels count all =2316, here it is assumed that each row of the LCD has 2316 pixels.

[0175] Based on the number of all opaque pixels in all rows of the image frame and the number of all pixels in the image frame, the content concentration of the image frame can be calculated. Assume that the calculated content concentrations of the above five image frames of the bullet screen layer are:

[0176] Content Density barrage 1-5 ={3.607%, 3.597%, 3.608%, 3.579%, 3.590%}

[0177] It should be noted that the frame content concentration of the video layer is generally 82.8248% (the video layer belongs to the traditional high traffic layer (Ordinary High Traffic Layer), and the content concentration of the Ordinary High Traffic Layer is generally greater than 60%), that is:

[0178] Content Density video =82.8248%

[0179] Generally speaking, the content concentration of the barrage layer will not exceed 20%.

[0180] The calculation method of the horizontal scrolling step concentration is as follows:

[0181] We can first count the starting position of each line in the first frame that is not transparent, then count the starting position of each line in the second frame that is not transparent, and then calculate the difference of the starting position of each line for statistical analysis. Here we assume that all transparent points are white and all non-transparent points are normalized to black. Assume that there is non-transparent color in the 64th line in both the first and second frames of the bullet chat layer. The scatter plots of the 64th line in these two frames are as follows: Figure 8 As shown, Figure 8 is a schematic diagram showing normalization of display content of line 64 in the 1st and 2nd frames of the bullet screen layer according to an exemplary embodiment of the present disclosure.

[0182] Reference Figure 8, we can get the coordinates of the first non-transparent pixel in the 64th row in the first frame as x=4; the coordinates of the first non-transparent pixel in the 64th row in the second frame as x=0. Then the scrolling distance of the 64th row between the two frames is 0-4=-4. By analogy, we can calculate the scrolling distance of each row in all the rows contained in the image frame.

[0183] In each of the image frames 1 to 5 of the above bullet-screen layer, there are only 4 lines of text or icons, corresponding to 197 lines of actual pixels, and the rest are transparent. By collecting these lines, we can calculate the horizontal scrolling step distribution of the image frame 2 of the above bullet-screen layer relative to the image frame 1. Fig. 9 is an original scatter plot showing the horizontal scrolling step of the bullet screen layer according to an exemplary embodiment of the present disclosure. Fig. 9 The series of numbers in the horizontal direction represent the serial numbers of the rows contained in the image frame, and the series of numbers in the vertical direction represent the size of the horizontal scrolling step.

[0184] Because bullet comments usually scroll from right to left, the step length can only be negative. Positive numbers usually appear when new scrolling content or other small amounts of background noise data are added to the frame, and in fact, the data with positive step lengths does not need to be considered. Therefore, all data with zero step lengths and positive integers can be removed, and the horizontal scroll step length scatter plot becomes as follows: Fig.10 shown. Fig.10 is a scatter plot showing horizontal scroll step correction of a barrage layer according to an exemplary embodiment of the present disclosure.

[0185] Reference Fig.10 , it can be counted that about 86% of the steps fall on "-5", so the horizontal scrolling step of frame 2 relative to frame 1 is -5. Similarly, the horizontal scrolling step and concentration between other frames can be obtained. For example, it can be obtained that the horizontal scrolling step of frame 3 relative to frame 2 is -4, and the concentration is 87%; the horizontal scrolling step of frame 4 relative to frame 3 is -5, and the concentration is 88%; the horizontal scrolling step of frame 5 relative to frame 4 is -4, and the concentration is 85%. It should be noted that if the probability of there being two adjacent steps between two frames is similar, for example, 40% of the steps fall on -4, and 45% of the steps fall on -5, then it can be considered that 85% of the steps between these two frames fall on -4.5.

[0186] In this way, the horizontal scrolling step of the five image frames 1 to 5 collected from the bullet screen layer is:

[0187]

[0188] The horizontal scroll step concentration is:

[0189]

[0190] The above process of calculating the horizontal scrolling step and the concentration of the horizontal scrolling step only needs to traverse the pixels once for cumulative statistics, which is very simple and fast, and does not require a complex image recognition process. Generally speaking, the concentration of the horizontal scrolling step of the bullet screen layer is greater than 65%, while the concentration of the horizontal scrolling step of the non-bullet screen layer is generally very small.

[0191] It should be noted that in addition to scrolling horizontally from right to left, the bullet screen can also scroll vertically. For example, it can also scroll vertically from bottom to top. In this case, when calculating the vertical scrolling step and the vertical scrolling step concentration, you only need to change it to calculating the vertical scrolling step by column. The principle is the same as the horizontal scrolling step, so I will not go into details here.

[0192] Among them, content traffic can be calculated by the following method:

[0193] For the five image frames 1 to 5 sampled from the bullet screen layer, we can perform MD5 fast operations on all the data in each frame in turn, and then obtain a 64-bit MD5 value:

[0194]

[0195] Then, the md5 duplication of these five frames can be calculated. The calculation method of duplication is: sort the five md5 values, then check how many different values ​​there are, and finally divide the number of different values ​​by the total number of frames to get the content flow, that is:

[0196] content traffic=((Num of different Md5) / 5)*100%

[0197] Generally speaking, the higher the content flow, the greater the difference in power consumption of the application layer at different refresh rates, which means the impact on power consumption will be relatively obvious. The content flow of the ordinary high-flow layer is generally greater than 60%.

[0198] Furthermore, the calculation method of the content concentration of the Body SurfaceView Layer can be:

[0199] If there are surface view layers other than the layer with the highest refresh rate, then the content concentration of all surface view layers can be calculated and the maximum value can be taken:

[0200]

[0201] The calculation method of Body Surface View Layer content flow can be:

[0202] If there is a surface view layer (Surface View Layer) in addition to the highest refresh rate layer, then calculate the content flow of all surface view layers and take the maximum value among them:

[0203]

[0204] It should be noted that if the target layer, that is, the layer with the highest refresh rate is not the surface view layer, the two parameters "Max body Surface View Content Density" and "Max body Surface View Content Traffic" must also be entered into the subsequent decision tree model as one of the influencing factors for identifying the target layer type; if the target layer, that is, the layer with the highest refresh rate is the surface view layer, the two parameters "Max body Surface View Content Density" and "Max body Surface View Content Traffic" will not be entered into the subsequent decision tree model, that is, at this time these two parameters will not be used as influencing factors for identifying the target layer type.

[0205] According to an exemplary embodiment of the present disclosure, the application layer may be pre-divided into a plurality of types according to power consumption, and a predetermined at least one type among the plurality of types may be set as a power consumption adjustable layer.

[0206] The decision tree model may be, but is not limited to, a C4.5 decision tree model.

[0207] According to an exemplary embodiment of the present disclosure, the application layer can be pre-divided into multiple types according to the power consumption, for example, but not limited to, a high power consumption layer (also referred to as a high flow rate) and a low power consumption layer (also referred to as a low flow rate). In addition, the high power consumption layer and the low power consumption layer can be further classified according to the display content, for example, the high power consumption layer can be classified into, but not limited to, a content layer and a comment information layer. For example, the high power consumption layer can be classified into a first type and a second type, that is, the first type can be an application layer for displaying the substantial content of the target application, which can also be referred to as an ordinary high flow layer. Exemplarily, it can be a video layer (Video Layer) / game layer (Game Layer); the second type can be an application layer for displaying comment information related to the substantial content, which can be, for example, a bullet screen layer. These two types can be set as power consumption adjustable layers because of their relatively high current consumption. In addition, the low power consumption layer can be used as the third type, which can also be referred to as an ordinary low flow layer. Usually, the current consumption caused by this type is relatively small, so it can not be set as a power consumption adjustable layer. Of course, the classification of the application layer is not limited to the above examples, and can also be divided into types according to actual power consumption or functional status. The above classification method is only an exemplary description.

[0208] When using a decision tree model to identify the type of application layer, the characteristic parameters used by different application layers may be different. For example, Fig.11 is a schematic diagram showing characteristic parameters used by different application layers according to an exemplary embodiment of the present disclosure. Fig.11 Although the special layer and the ordinary high traffic layer are both adjustable high power layers, their corresponding identification feature parameters are different and may overlap partially. As an unadjustable layer, the ordinary low traffic layer has the following identification rule: if an application layer is identified as not belonging to the above two types: special layer or ordinary high traffic layer, it belongs to the ordinary low traffic layer.

[0209] First, a decision tree model needs to be established. By automatically testing hundreds of known samples and sampling feature data, a training data set for the decision tree model can be formed. Fig.12 is a schematic diagram showing a training data set of a decision tree model according to an exemplary embodiment of the present disclosure. Fig.12Each application layer can correspond to multiple features, including but not limited to the following items: the position of the layer layout on the Z axis (Layer Z Position), the percentage of the area of ​​the application layer layout to the screen area (Layer Size), the transparency of the application layer layout (Layer Transparency), whether it contains keywords (Layer Surface View Keyword), whether it is a surface view layer (Layer Surface View Type), content concentration (Layer Content Density), the maximum content concentration in other surface view layers (Max bodySurfaceView Layer Content Density), horizontal scroll step concentration (Layer Horizontal ScrollStep Density), application layer refresh rate (Layer Refresh Rate), content traffic (Layer ContentTraffic), and the maximum content traffic in other surface view layers (Max Body SurfaceView Layer ContentTraffic).

[0210] Among them, "Layer Z Position" can indicate the position of the layer layout on the Z axis, which can be divided into 3 levels: 0, 1, and 2. Among them, "0" can indicate the top, "1" can indicate one level below level 0, and "2" can indicate one level below level 1. Generally, the "Layer Z Position" of the bullet chat layer is the smallest, that is, the position of the bullet chat layer is generally at the top layer.

[0211] "Layer Size" can indicate the percentage of the area of ​​the application layer layout to the screen area, which can be divided into 3 levels: <28%, 28-72%, >72%. Generally, the "Layer Size" of the bullet screen layer is larger.

[0212] “Layer size” = (layer width*layer height) / (screen width*screen height)*100%, where “layer width” represents the width of the application layer, “layer height” represents the height of the application layer, “screen width” represents the width of the terminal screen, and “screen height” represents the height of the terminal screen.

[0213] "Layer Transparency" can indicate the transparency of the application layer layout, which can be divided into 5 levels: 0%, <50%, 50-70%, 70-99%, 100%. Among them, 0% can indicate completely opaque, and 100% can indicate completely transparent. Generally, the "Layer Transparency" of the bullet comment layer is the largest.

[0214] "Layer Surface View Keyword" can indicate whether the layer view contains keywords such as danmake / Danmu / Tanmu / bullet / barrage. 0 can indicate not containing, and 1 can indicate containing.

[0215] "Layer Surface View Type" can indicate whether the corresponding application layer is a Surface ViewLayer. 0 can indicate no, and 1 can indicate yes. Generally speaking, the probability that the surface view layer is an Ordinary High Traffic Layer is relatively high.

[0216] "Layer Content Density" refers to the content concentration, which can be divided into 5 levels: <10%, 10-25%, 25-50%, 50%-75%, >75%. The Content Density of the ordinary bullet comment layer is smaller, and the Content Density of the ordinary high traffic layer is larger.

[0217] “Max body SurfaceView Layer Content Density” can be divided into 5 levels, and the specific division method is similar to “Layer Content Density”.

[0218] "Layer Horizontal Scroll Step Density" is the concentration of horizontal scrolling steps, which can be divided into 5 levels: <10%, 10-25%, 25-50%, 50-70%, >70%. Generally, the "Layer Horizontal Scroll Step Density" of the bullet screen layer is larger.

[0219] "Layer Refresh Rate" is the refresh rate of the application layer, which can be divided into 5 levels: <=24, 30, 48, 60, >=96. Generally, the "Layer Refresh Rate" of the bullet screen layer is larger.

[0220] "Layer Content Traffic" refers to content traffic, which can be divided into five levels: <10%, 10-25%, 25-50%, 50-70%, >70%. Generally, the "Layer Content Traffic" of the Ordinary High Traffic Layer is larger.

[0221] “Max Body Surface View Layer Content Traffic” can be divided into 5 levels, and the specific division method is similar to “Layer Content Traffic”.

[0222] In the present disclosure, the C4.5 entropy gain rate can be used to determine the impact of each factor on the final decision result. Assume that the total number of samples (cases) is 300. Among them, the number of samples with a barrage layer (has barrage) is 100, and the number of samples without a barrage layer (no barrage) is 200. "has barrage" can be called a positive example, and "no barrage" can be called a negative example.

[0223] First, the information entropy calculation formula can be shown as follows:

[0224]

[0225] The total entropy value of the initial root node can be calculated based on the above information entropy calculation formula:

[0226]

[0227] Then, the entropy gain of each feature such as “the position of the layer layout on the Z axis”, “the percentage of the area of ​​the application layer layout to the screen area”, and “the transparency of the application layer layout” can be calculated.

[0228] For example, the entropy gain of "the position of the layer layout on the Z axis" can be calculated first. It is assumed that there are 150 samples with the characteristic value of "0", 80 samples with the characteristic value of "1", and 70 samples with the characteristic value of "2" in the "position of the layer layout on the Z axis". It is also assumed that there are 90 positive examples and 60 negative examples in the samples with the characteristic value of 0; 30 positive examples and 50 negative examples in the samples with the characteristic value of 1; and 5 positive examples and 65 negative examples in the samples with the characteristic value of 2.

[0229] The entropy value of the attribute Layer Z Position = 0 is:

[0230]

[0231] The entropy value of the attribute Layer Z Position = 1 is:

[0232]

[0233] The entropy value of the attribute Layer Z Position = 2 is:

[0234]

[0235] Then the entropy gain of the attribute Layer Z Position is:

[0236]

[0237] Similarly, the entropy gain of other attributes can be calculated as:

[0238] Layer Size=g2; Layer Transparency=g3;

[0239] Layer Surface View Keyword=g4; Layer Content Density=g5;

[0240] Layer Horizontal Scroll Step Density=g6; Layer FPS=g7.

[0241] Theoretically, the attribute with the largest entropy gain can be used as the first-level root node of the decision tree, and then further subdivided. However, since entropy gain actually has a preference for attributes with a large number of possible values, that is, the more samples it has, the greater the possible entropy gain value. Therefore, the entropy gain rate can be used to select the first-level root node. The specific method is as follows:

[0242] First, calculate the average entropy gain of all attributes:

[0243]

[0244] Then, find the corresponding attribute whose entropy gain is greater than the average entropy gain value from all attributes. Assume that the entropy gain of Layer ZPosition (g1) and the entropy gain of Layer Content Density (g5) are greater than the average entropy gain value g0.

[0245] Next, we can calculate the entropy gain rate of the "Layer layout position on the Z axis":

[0246] First, the intrinsic value IV(D1) of "the position of the layer layout on the Z axis" can be calculated. Since there are 150 samples with an eigenvalue of 0, 80 samples with an eigenvalue of 1, and 70 samples with an eigenvalue of 2 in "the position of the layer layout on the Z axis", therefore:

[0247]

[0248] Then, the entropy gain rate of "Layer layout position on the Z axis" can be calculated:

[0249]

[0250] Similarly, the entropy gain rate of “content concentration” can be calculated: r2.

[0251] Next, the attribute with the larger entropy gain rate can be selected as the root node of the first layer of the decision tree. If r1=r2, then one of the "Layer layout position on the Z axis" and "content concentration" can be selected as the root node of the first level. Fig.13 is a schematic diagram showing a first-level root node of a decision tree according to an exemplary embodiment of the present disclosure. Fig.13 "Layer layout position on the Z axis" is the first-layer root node of the decision tree model, which contains a total of 3 branches, namely, the branch with eigenvalue equal to 0, the branch with eigenvalue equal to 1, and the branch with eigenvalue equal to 2.

[0252] After getting the root node of the first layer (the position of the Layer layout on the Z axis), you can select the branch and root nodes of the second layer. Return to reference Fig.13 , which contains 3 branches in total. Let's look at the first branch, which is the branch with eigenvalue equal to 0. Under this branch, the sample set contains 150 samples, of which 90 are positive examples and 60 are negative examples. Therefore, the total entropy value of the branch root node under this branch can be calculated:

[0253]

[0254] Similarly, the entropy gain of the following attributes can be calculated:

[0255] Layer Size=g21; Layer Transparency=g22;

[0256] Layer Surface View Keyword=g23; Layer Content Density=g24;

[0257] Layer Horizontal Scroll Step Density=g25;Layer FPS=g26

[0258] It should be noted that there is no need to calculate the entropy gain of "the position of the layer layout on the Z axis" here.

[0259] Similarly, the average entropy gain of all attributes in this layer can be calculated:

[0260]

[0261] Similarly, we can find out the attributes that are greater than the average entropy gain value g20 from all the attributes of this layer. Assume that the entropy gain of LayerTransparency (g22), the entropy gain of Layer Content Density (g24), and the entropy gain of Layer HorizontalScroll Step Density (g25) are greater than the average entropy gain value g20. Similarly, we can find out the attribute with the largest entropy gain rate among these three attributes. Assuming that the attribute with the largest entropy gain rate is "content concentration", we can select it as the branch root node of this branch.

[0262] Fig.14 FIG. 1 is a schematic diagram showing some nodes of the first layer and the second layer of the decision tree model according to an exemplary embodiment of the present disclosure. Fig.14 , “Content concentration” is selected as the branch root node of the second layer of the decision tree, and 5 branches are refined under this branch root node.

[0263] By analogy, following similar calculation methods, a complete decision tree model can be established based on the training data set. Fig.15 FIG. 1 is a schematic diagram showing some nodes included in a complete decision tree model according to an exemplary embodiment of the present disclosure. Fig.15 , in each layer of the decision tree, only one branch is selected to expand downward. In fact, each branch of each layer of the decision tree model can expand downward to more branches, that is, the decision tree model is actually a relatively large structure. In order to avoid redundancy, Fig.15 Only some nodes in the decision tree model are shown. Moreover, after using all attributes for expansion, each of the multiple branches contained in the last branch root node will point to its corresponding category, that is, has barrage or no barrage.

[0264] In this way, the target layer's layer display behavior feature data can be used to perform step-by-step screening in the decision tree model. When the bottom layer of the decision tree model is screened, the target layer's layer display behavior feature data finally falls into which branch, and the category corresponding to the branch is the type of the target layer, that is, either has barrage or does not have barrage.

[0265] According to an exemplary embodiment of the present disclosure, the layer display behavior characteristic data of the target layer can be collected at preset time intervals within a predetermined time period to obtain multiple sets of layer display behavior characteristic data corresponding to the target layer. Exemplarily, the layer display behavior characteristic data of the target layer can be collected every 5 seconds for 30 consecutive times, and a total of 30 sets of layer display behavior characteristic data can be obtained.

[0266] According to an exemplary embodiment of the present disclosure, each set of layer display behavior characteristic data in the multiple sets of layer display behavior characteristic data corresponding to the target layer can be input into the decision tree model respectively to obtain the recognition result corresponding to each set of layer display behavior characteristic data. Exemplarily, 30 sets of layer display behavior characteristic data can be input into the decision tree model respectively to obtain the recognition result corresponding to each set of layer display behavior characteristic data.

[0267] Next, when multiple groups of recognition results corresponding to multiple groups of layer display behavior characteristic data all indicate that the target layer is a power consumption adjustable layer, the target layer can be determined to be a power consumption adjustable layer. Exemplarily, when 30 groups of recognition results corresponding to 30 groups of layer display behavior characteristic data all indicate that the target layer is a power consumption adjustable layer, the target layer can be determined to be a power consumption adjustable layer.

[0268] It should be noted that if only one set of layer display behavior feature data is used to identify the type of the target layer, misjudgment may occur. Therefore, the present disclosure can collect multiple sets of layer display behavior feature data, and only when multiple sets of recognition results corresponding to the multiple sets of layer display behavior feature data indicate that the type of the target layer is the same type, will the type of the target layer be determined to be the same type. In this way, since multiple sets of collected data are used for identification, the probability of misjudgment can be greatly reduced, and the accuracy of identification of the target layer type can be guaranteed.

[0269] In step 403, when the type of the target layer is a power-adjustable layer, the refresh rate value of the target layer can be adjusted. That is, when the type of the target layer is an adjustable high-power layer (special layer, ordinary high-traffic layer), the refresh rate value of the target layer can be adjusted.

[0270] It should be noted that in the prior art, when the terminal is in the adaptive refresh rate mode, the system refresh rate is voted by the refresh speed and area ratio of each layer at present, and its basic principle is shown in the following formula:

[0271]

[0272]

[0273] Among them, "Layer weight " indicates the area ratio of the application layer, that is, the area weight, "Layer area" indicates the area of ​​the application layer, "Screen area" indicates the area of ​​the terminal screen, and "TH layer " indicates the area ratio threshold, "RefreshRate system " " indicates the system refresh rate, "Layer FPS" indicates the refresh rate of the application layer, and "LCD Display Models" indicates the refresh rate supported by the LCD display.

[0274] The first formula means that if the overall refresh rate of the system is to be affected, the area weight of the application layer must exceed the threshold; the second formula means that the final refresh rate of the system depends on the highest refresh rate of all application layers that meet the area ratio threshold, and this refresh rate must also be within the refresh rate range supported by the LCD display. For example, the screen of a certain terminal supports seven refresh rates: [10Hz, 24Hz, 30Hz, 48Hz, 60Hz, 96Hz, 120Hz].

[0275] As mentioned earlier, Figure 1 The refresh rate values ​​of the four application layers in the video APP shown in the figure are [120Hz, 24Hz, 10Hz, 0Hz] respectively. The application layer with the highest refresh rate is the bullet screen layer, and its refresh rate is 120Hz. If the refresh rate value of the bullet screen layer participating in the vote is not interfered, the system refresh rate value is:

[0276] Refresh Rate systembefore =max AllLayerweight≥TH {120,24,10}=120Hz

[0277] It can be seen that at this time, the system refresh rate is forced to be raised to a high refresh rate, which will cause excessive power consumption of the terminal and cause a waste of resources.

[0278] Therefore, according to an exemplary embodiment of the present disclosure, the refresh rate value of the target layer may be adjusted based on at least one of the center position relationship between the target layer and the terminal screen, the area relationship between the target layer and the terminal screen, and the type of the target layer.

[0279] It should be noted that, from the perspective of visual experience, the closer the center of the application layer is to the center of the screen, the higher its refresh rate can be; or, the larger the ratio of the area of ​​the application layer to the area of ​​the terminal screen, that is, the larger the area of ​​the application layer, the higher the refresh rate of the application layer can be; or, the refresh rate of the application layer varies depending on the type of application layer. For example, the refresh rate of the bullet screen layer is generally higher than that of the video layer, and so on. In this way, by adjusting the refresh rate value of the target layer in combination with the characteristics of multiple dimensions, the accuracy of the refresh rate value adjustment can be guaranteed.

[0280] According to an exemplary embodiment of the present disclosure, the preset minimum refresh rate value corresponding to the target layer can be determined based on at least one of the center position relationship between the target layer and the terminal screen, the area relationship between the target layer and the terminal screen, and the type of the target layer.

[0281] Exemplarily, taking the relationship between the target layer and the center position of the terminal screen as an example, the following corresponding relationship can be set:

[0282] The distance between the center position of the application layer (Layer center position) and the center position of the screen (Screen center) does not exceed 1 / 3 of the height of the application layer, and the minimum refresh rate of the application layer can be set to 48Hz;

[0283] If the center of the application layer is more than 1 / 3 of the height of the application layer from the center of the screen, the minimum refresh rate of the application layer can be set to 30 Hz.

[0284] Then, the refresh rate value of the target layer can be adjusted based on the preset minimum refresh rate value corresponding to the target layer and the actual sampled refresh rate value of the target layer. Figure 1 Among the four application layers contained in the video APP shown in , the target layer is the bullet screen layer, and its actual sampling frame rate is 101.02Hz, that is, its actual sampling refresh rate value is 101.02Hz, and the refresh rate value supported by the terminal screen obtained by rounding up the actual sampling frame rate is 120Hz. Assuming that the distance between the center position of the target layer and the center position of the terminal screen is less than 1 / 3 of the application layer height, the preset minimum refresh rate value corresponding to the target layer can be 48Hz. At this time, it can be determined that the refresh rate value range of the target layer is: [48Hz, 60Hz, 96Hz, 120Hz].

[0285] In this way, by adjusting the refresh rate value of the target layer based on the preset minimum refresh rate value and the actual sampled refresh rate value, the objectivity of the refresh rate value adjustment can be guaranteed. In addition, since the determined refresh rate value is a range interval, it can avoid the phenomenon that the system refresh rate is forced to be increased by directly using the actual sampled frame rate of the target layer as its final refresh rate value, thereby avoiding excessive power consumption of the terminal and saving resources.

[0286] In step 404, a refresh rate value of the terminal screen may be determined based on the adjusted refresh rate value of the target layer.

[0287] According to an exemplary embodiment of the present disclosure, the refresh rate value of the terminal screen may be determined based on the adjusted refresh rate value of the target layer and the actual sampled refresh rate values ​​of other application layers included in the target application.

[0288] For example, the left end point 48Hz in the refresh rate value range of the target layer [48Hz, 60Hz, 96Hz, 120Hz] can be taken as the refresh rate value of the target layer participating in the vote, that is, the display synthesis system can be notified that the refresh rate value of the target layer is specified as 48Hz. Figure 1 The refresh rate values ​​of three of the four application layers in the video APP shown in the figure meet the threshold, and are [48Hz, 24Hz, 10Hz] respectively. Therefore, when the display synthesis system calculates the overall refresh rate of the system, it will automatically change the current system final refresh rate to 48Hz:

[0289]

[0290] It can be seen that in the present disclosure, it is actually not necessary to adjust the adjustable high power consumption layer itself. It is only necessary to tell the display synthesis system what the refresh rate of the application layer is, which will directly affect the final system refresh rate calculated by the display synthesis system. In addition, when determining the refresh rate value of the terminal screen, the adjusted refresh rate value of the target layer and the actual sampled refresh rate values ​​of other application layers are comprehensively considered, which can ensure that the power consumption of the terminal is reduced and the performance of the terminal is basically unchanged.

[0291] In step 405, the target application may be run based on the refresh rate value of the terminal screen.

[0292] It should be noted that when the refresh rate of the adjustable high power consumption layer is lowered, the terminal power consumption is reduced due to the decrease in refresh rate, but the decrease in refresh rate will also cause changes in the performance of the application layer. How to balance the benefits of power consumption and performance and determine the most appropriate refresh rate is another problem that needs to be solved.

[0293] After adjusting the refresh rates of two different adjustable high power consumption layers, the present disclosure shows that although their final improvement results are both a decrease in operating current, there are certain differences in their improved performance evaluations. Fig.16 FIG. 2 is a schematic diagram showing the difference in performance scores between a special layer and a general high-traffic layer according to an exemplary embodiment of the present disclosure. Fig.16For special layers, the evaluation can be based on layer fluency; for ordinary high traffic layers, the evaluation can be based on layer framerate stability.

[0294] Here, we will take the special layer (bullet screen layer) as an example to explain the calculation of the performance score. Among them, "bullet screen scrolling smoothness" (layer Fluency) specifically refers to whether flickering can be felt during the scrolling process, that is, scroll flicker (scroll flicker). "Bullet screen scrolling smoothness" is mainly measured and evaluated based on two feature data, which actually determine the display quality of the bullet screen:

[0295] Layer Horizontal Scroll Step

[0296] Layer Horizontal Scroll Distance per second

[0297] The calculation process of the horizontal scrolling step is specifically described in step 402, which will not be repeated here. In addition, the absolute value of the horizontal scrolling step can be taken. For example, assuming the horizontal scrolling step is -4, its absolute value is: |-4|=4.

[0298] The "horizontal scroll distance per second" can be calculated using the following formula:

[0299] scroll distance=|scroll step*refresh rate|

[0300] It can be seen that if the horizontal scrolling step is too small, although the scrolling is smooth, the low refresh rate will cause the horizontal scrolling distance per second to be too short or too slow; if the horizontal scrolling step is too large, the scrolling is not smooth, and the high refresh rate will cause the horizontal scrolling distance per second to be too long or too fast. After many tests, the best bullet screen scrolling experience is: the horizontal scrolling step is between 3 and 6; the horizontal scrolling distance per second is about 180. If it is too large, it is faster, and if it is too small, it is slower.

[0301] Therefore, we can define the score of the fluency of the bullet screen (with a value of 0 to 1):

[0302] If the horizontal scrolling step is 3 to 6,

[0303] If the horizontal scroll step is less than 3,

[0305] If the horizontal scroll step is > 6,

[0307] Where a = 0.5, b = 0.1. If the score is less than 0, it can be assigned a value of 0, which is generally above 0.75. In this way, the value of the fluency of the bullet screen can be quantified from 0 to 1, where 1 is the best score.

[0308] Next, we will take the common high traffic layer as an example to illustrate the calculation of performance score. Here, we can define the score of frame rate stability (Layer framerate stability) (with a value of 0 to 1):

[0309]

[0310] Among them, Layer Framedrop is the frame drop rate of the application layer, which can be sampled from the display synthesis system; Layer Orignial FPS is the average frame rate of the target layer before optimizing the refresh rate value of the target layer; Layercurrent FPS can be the sampled frame rate of the target layer after optimizing the refresh rate value of the target layer, and further, Layer current FPS can be the sampled frame rate of the target layer after each optimization of the refresh rate value of the target layer. In this way, the score of the frame rate stability of the application layer can be quantified into 0~1, where 1 is the best score. Generally speaking, a low refresh rate of the application layer may lead to an increase in the frame drop rate, which in turn leads to a low frame rate stability score.

[0311] For the two adjustable high power consumption layers, whether they are special layers or ordinary high traffic layers, the optimization of their refresh rate values ​​is learned based on the power consumption score (Power Score) and performance score (Performance Score).

[0312] Below, we take the bullet screen layer as an example to illustrate the optimization learning process. The learning process of the ordinary high-traffic layer is similar, except that the performance score type needs to be selected as frame rate stability.

[0313] First, a SARSA-Learning training model can be established. The process of establishing a SARSA-Learning training model can include six parts: 1. state definition; 2. action definition A; 3. single-step reward (or single-step penalty) R; 4. discounted action value function Q; 5. iterative training process; 6. strategy function.

[0314] 1. Status definition:

[0315] Assume that the average sampling current of the terminal before the refresh rate value of the target layer is optimized is rc, the average performance score of the terminal is rf, and the sampling period is 5s; Assume that the target average current tc = p*rc, and the target average performance score tf = q*rf. Where p and q are proportional coefficients; Assume that cc is the average sampling current after each optimization of the refresh rate value of the target layer, cf is the average performance score after each optimization, dc is the current factor, and df is the performance score factor.

[0316] Then the factor levels of the features can be defined as:

[0317] dc=f1(cc)=f1(cc,rc,tc)

[0318] df=f2(cf)=f2(cf,rf,tf)

[0319] So we can define the state S, namely:

[0320] S=s(dc,df)=s(f1(cc),f2(cf))

[0321] The average sampling current cc after each optimization is based on the target average current tc and the average sampling current rc of the terminal before the initial optimization. Assuming that the current segment size That is, the current can be segmented based on this size. Here, dc can be defined as 5 levels, and the smaller the dc, the greater the current gain. Fig.17 2 is a schematic diagram showing the current segmentation and the level of the current factor dc according to an exemplary embodiment of the present disclosure. Fig.17 , the current is divided into 5 sections, and the current factor dc contains 5 levels in total.

[0322] The average performance score cf after each optimization can be referenced by the target average performance score tf. Assuming that the segment size of the performance score bf = 0.1, the performance score can be segmented based on this size. Here, the performance score factor df can be defined as 4 levels. The smaller the df, the greater the performance score gain. df is actually a suppression factor that can balance the power consumption gain and performance gain. Fig.18It is a schematic diagram showing the segmentation of performance scores and the levels of the performance score factor df according to an exemplary embodiment of the present disclosure. Referring to Fig.18 , the performance score is divided into 4 segments in total, and the performance score factor df includes 4 levels in total.

[0323] When optimizing the refresh rate value of the target layer, the current can be reduced by up to 40%. Therefore, the aforementioned p and q coefficients can be defined as 0.6 and 1. That is, without changing the performance score, reducing the current consumption by 40% can be used as the regulation target.

[0324] Still taking the Figure 1 shown video APP as an example, assuming:

[0325]

[0326] The state binary tuple S = s(dc, df) = s(f1(cc), f2(cf))

[0327] It should be noted that when determining the current state before the initial optimization target layer's refresh rate, since the refresh rate has not started to be optimized yet, it can be considered that cc = rc and cf = rf = tf. At this time, referring back to Fig.17 , since the value of cc at this time falls within the range of rc - bc < cc ≤ rc, it can be determined that at the current state, that is, before the initial optimization of the target layer's refresh rate, dc = 2. Referring back to Fig.18 , since the value of cf at this time falls within the range of cf ≥ tf, it can be determined that at the current state, that is, before the initial optimization of the target layer's refresh rate, df = 0. Therefore, the current state of the terminal is:

[0328] s(dc, df) = s(f1(530), f2(1.0)) = s(2, 0)

[0329] Assume that after optimizing the refresh rate value of the target layer, the current cc = 416 mA and cf = 0.95. Then at this time:

[0330] rc - bc = 530 mA - 106 mA = 424 mA; tc = 0.6 × 530 = 318 mA;

[0331] tf = 1.0 * 1.0 = 1.0; tf - bf = 1.0 - 0.1 = 0.9

[0332] Referring back to Fig.17 , since the current cc = 416 mA falls within the range of (318, 424], that is, within the range of (tc, rc - bc], it can be determined that after this optimization of the target layer's refresh rate value, dc = 1. Referring back to Fig.18, since cf=0.95 falls into the interval (0.9,1.0], that is, falls into the interval (tf-bf, tf], it can be determined that df=1 after the refresh rate value of the target layer is optimized this time. Therefore, the state of the terminal after the refresh rate value of the target layer is optimized this time is:

[0333] s(dc,df)=s(f1(416),f2(0.95))=s(1,1)

[0334] 2. Action definition A:

[0335] In step 404, the refresh rate value switching range of the target layer is determined to be: [48 Hz, 60 Hz, 96 Hz, 120 Hz]. The "action definition" in the present disclosure refers to adjusting the refresh rate value by one level, and a total of 3 action types can be defined. Fig.19 2 is a schematic diagram showing the action types according to an exemplary embodiment of the present disclosure. Fig.19 , the three action types are: maintain the current refresh rate, reduce the refresh rate by one level, and adjust the refresh rate up by one level.

[0336] 3. Single-step reward (or single-step penalty) R:

[0337] Here we can use the minimization penalty value function, and the single-step penalty can be defined using a table, that is, in a certain state s(dc,df), there is a preset penalty value R. Fig. 20 is a schematic diagram showing a single-step penalty table according to an exemplary embodiment of the present disclosure. Fig. 20 In each state s(dc, df), there is a corresponding preset penalty value R. For example, R(0,0)=0; R(4,3)=10, and so on.

[0338] It should be noted that when the performance score decreases significantly, for example, when df=3, the penalty value is relatively high regardless of how much the current decreases; if the performance score remains unchanged, the penalty value is gradually increased according to the increase in current.

[0339] As mentioned above, before the refresh rate value of the target layer is initially optimized, the terminal is in the state S(2,0), and the corresponding single-step penalty R value is R(S(2,0))=R(2,0)=2, indicating that the refresh rate value of the target layer can be further optimized.

[0340] 4. Discounted action value function Q;

[0341] Q=Q(S,A)=Q(dc,df,A), the Q table can be initialized to an all-zero matrix. Fig.21is a schematic diagram showing a Q table in an initialization state according to an exemplary embodiment of the present disclosure. Fig.21 , in each state s(dc,df), there are 3 actions (Action), each action corresponds to a discount value, that is, the state action penalty value. For example, Q(0,0,0) = 0. The purpose of learning is to continuously calculate the Q value of the corresponding state in the actual test until it reaches the convergence state, that is, until Fig.21 The floating range of the discount value in each grid is within the preset range. After the Q table learning is completed, in actual use, it can be used to guide the adjustment of the refresh rate value of the target layer until the refresh rate value reaches the optimal state.

[0342] 5. Iterative training process;

[0343] The SARSA-Learning iterative update formula of this model can be expressed as:

[0344] Q(S,A)=Q(S,A)+a*(R(S′,A′)+r*Q(S′,A′)-Q(S,A))

[0345] Among them, S = s (dc, df) represents the current state;

[0346] A represents the action taken, a is the learning rate, and r is the discount factor;

[0347] S′=s(dc′,df′) represents the new state reached after taking action A in state S;

[0348] A′ represents the action selected in the new state;

[0349] R(S′,A′) is the single-step return (single-step reward, single-step penalty) obtained by reaching the new state;

[0350] Q(S,A) is the reward value (state-action penalty value) of taking action A in the current state S.

[0351] Assume that a = 0.3, r = 0.8. As mentioned above, the terminal is in the state S(2,0) before the refresh rate value of the target layer is initially optimized. Assume that in this state S(2,0), 5 actions A are selected and different states are reached respectively. Fig. 22 is a schematic diagram showing Sarsa Learning (S, A) state transition according to an exemplary embodiment of the present disclosure.

[0352] The following Q values ​​can be calculated separately:

[0353] At the initial state S(2,0), after executing action A=1, we reach the new state S′(1,1), and the next action is also A′=1, then:

[0354] Q(2,0,1)=Q(2,0,1)+0.3*(R(1,1)+0.8*Q(1,1,1)-Q(2,0,1))=0+

[0355] 0.3*(2+0.8*0-0)=0.6;

[0356] When S(1,1), after executing action A=1, we reach the new state S′(0,1), and the next action is also A′=1, then:

[0357] Q(1,1,1)=Q(1,1,1)+0.3*(R(0,1)+0.8*Q(0,1,1)-Q(1,1,1))=0+

[0358] 0.3*(1+0.8*0-0)=0.3;

[0359] When S(0,1), after executing action A=1, we reach the new state S′(0,2), and the next action is A′=0, then:

[0360] Q(0,1,1)=Q(0,1,1)+0.3*(R(0,2)+0.8*Q(0,2,0)-Q(0,1,1))=0+

[0361] 0.3*(3+0.8*0-0)=0.9;

[0362] When S(0,2), after executing action A=0, the new state is still S′(0,2), and the next action is A′=2, then:

[0363] Q(0,2,0)=Q(0,2,0)+0.3*(R(0,2)+0.8*Q(0,2,2)-Q(0,2,1))=0+

[0364] 0.3*(3+0.8*0-0)=0.9.

[0365] After the above five actions, the data in the Q table is updated. Fig.23 is a schematic diagram showing the Sarsa Learning Q table after updating according to an exemplary embodiment of the present disclosure. Fig.23 At this time, the value of Q(2,0,1) is updated from 0 to 0.6; the value of Q(1,1,1) is updated from 0 to 0.3; the value of Q(0,1,1) is updated from 0 to 0.9; the value of Q(0,2,0) is updated from 0 to 0.9.

[0366] Similarly, after continuous iterative updates, the Q table data above will be gradually filled and iteratively converged, that is, the data changes very little, that is, under each state-action Q(S,A), it will converge to a final state-action penalty value.

[0367] 6. Strategy function.

[0368] The strategy function here is used to guide the Q table that has been trained for a period of time to select the next action type when it reaches a new state, that is, to explore how to get the optimal training result. The strategy function used here should be both exploratory and greedy.

[0369] It should be noted that the ε-greedy method can be used to obtain the random number p:

[0370] p=Rand()(0≤p<1)

[0371] Assuming a threshold value is ε, then:

[0372] When p <= ε, the selected action type can be the best action, that is, the action type with the smallest corresponding state-action penalty value;

[0373] When p>ε, an action type can be randomly selected for exploration.

[0374] For example, suppose that after a period of training, the Q value in the S(1,1) state is as follows Fig.24 shown. Fig.24 is a schematic diagram showing the Q value in the S(1,1) state according to an exemplary embodiment of the present disclosure. Fig.24 , there are 3 Q values ​​(state action penalty value) in the S(1,1) state, which are 0.15, 0.35 and 0.25 respectively.

[0375] When training again and entering this state S(1,1), assuming ε = 0.3, take the random number p = Rand()

[0376] If p = 0.2, since p = 0.2 < = ε = 0.3, the action type with the smallest corresponding state-action penalty value can be selected, that is, the action type A = 0 with a corresponding state-action penalty value of 0.15 can be selected;

[0377] If p = 0.5, since p = 0.5> ε = 0.3, we can randomly select an action type, that is, A = Rand(2) = ∪(0,1,2), that is, we can randomly select an action type from the three action types [0,1,2].

[0378] It should be noted that when the training iteration reaches a certain level, the strategy of p <= ε can be directly adopted, that is, the action type with the smallest penalty value of the corresponding state action can be directly adopted, that is, the min A＝0,1,2 The action type corresponding to {Q(S,A)}.

[0379] According to an exemplary embodiment of the present disclosure, the power consumption and performance status data s(dc, df) of the terminal can be obtained. Then, based on the power consumption and performance status data s(dc, df) of the terminal, the refresh rate value of the target layer can be optimized through a reinforcement learning model, that is, through a trained Q table. Next, based on the optimized refresh rate value of the target layer, the optimized refresh rate value of the terminal screen can be determined. Then, based on the optimized refresh rate value of the terminal screen, the target application can be run.

[0380] In this way, since the converged Q table is obtained through a large amount of training, it has guiding significance for the optimization process of the actual refresh rate value, and can ensure that the refresh rate value obtained after optimization achieves a good balance between terminal power consumption and terminal performance.

[0381] According to an exemplary embodiment of the present disclosure, the refresh rate value of the target layer can be optimized based on the power consumption and performance status data s(dc, df) of the terminal and the preset state action table of the reinforcement learning model to obtain the optimized refresh rate value of the target layer. Among them, the preset state action table, that is, the Q table, can contain multiple states s(dc, df), and each state can correspond to multiple refresh rate adjustment action types. Exemplarily, each state can correspond to 3 refresh rate adjustment action types, namely: (maintain refresh rate) 0, (adjust the refresh rate downward by one level) 1, (adjust the refresh rate upward by one level) 2. Each refresh rate adjustment action type can correspond to its own state action penalty value (Q value).

[0382] Exemplarily, as mentioned above, the state of the terminal before the refresh rate value of the target layer is initially optimized is S(2,0). Assume that the refresh rate value of the target layer before the refresh rate value is initially optimized is 48Hz, the average sampling current of the terminal before the initial optimization is rc, and the average performance score of the terminal before the initial optimization is rf.

[0383] First, based on the initial state S(2,0) and the trained Q-table, we can search the trained Q-table for the action type with the smallest state-action penalty value among the three action types under the state S(dc,df)=S(2,0). Assuming that the action type with the smallest state-action penalty value is 2, that is, "adjust the refresh rate value upward by one level", we can then increase the refresh rate value by one level based on 48Hz according to action type 2.

[0384] It should be noted that when adjusting the refresh rate value of the target layer, it is necessary to adjust between the refresh rate levels supported by the terminal screen. As mentioned above, a certain terminal supports 7 refresh rate levels, namely: [10Hz, 24Hz, 30Hz, 48Hz, 60Hz, 96Hz, 120Hz]. Therefore, the refresh rate value of the target layer can only be selected and adjusted between these 7 refresh rate levels. As mentioned above, if the refresh rate value is increased by one level from 48Hz, the refresh rate value of the target layer will be increased from 48Hz to 60Hz.

[0385] Then, the average sampling current cc and the average performance score cf of the terminal after the refresh rate value of the target layer is adjusted this time can be obtained. Then, based on the average sampling current rc of the terminal before the initial optimization, the average performance score rf of the terminal before the initial optimization, the average sampling current cc of the terminal after the refresh rate value of the target layer is adjusted this time, the average performance score cf of the terminal after the refresh rate value of the target layer is adjusted this time, Fig.17 The schematic diagram of current segmentation and current factor dc level is shown, Fig.18 The schematic diagram of segmenting the performance score and the level of the performance score factor df is shown to determine the state S′(dc, df) of the terminal after the refresh rate value of the target layer is adjusted this time.

[0386] Next, based on the state S′(dc, df) and the trained Q-table, the action type with the smallest corresponding state action penalty value (Q value) among the three action types under the state S′(dc, df) can be found from the trained Q-table. Assuming that the action type with the smallest corresponding state action penalty value is 2, that is, "adjust the refresh rate value by one level", then according to the action type 2, the refresh rate value can be increased by one level on the basis of 60Hz. At this time, the refresh rate value of the target layer is adjusted to 96Hz.

[0387] Then, the average sampling current cc and the average performance score cf of the terminal after the refresh rate value of the target layer is adjusted this time can be obtained. Then, based on the average sampling current rc of the terminal before the initial optimization, the average performance score rf of the terminal before the initial optimization, the average sampling current cc of the terminal after the refresh rate value of the target layer is adjusted this time, the average performance score cf of the terminal after the refresh rate value of the target layer is adjusted this time, Fig.17 The schematic diagram of current segmentation and current factor dc level is shown, Fig.18 The schematic diagram of segmenting the performance score and the level of the performance score factor df is shown, and the state S″(dc, df) of the terminal after adjusting the refresh rate value of the target layer is determined. And so on, until the current state S of the terminal is determined. 当前(dc, df) and the trained Q table, the current state S found from the trained Q table 当前 The action type with the smallest state action penalty value among the three action types under (dc, df) is "maintaining the refresh rate unchanged". At this time, the refresh rate value of the target layer obtained is the optimal refresh rate value that balances the terminal power consumption and terminal performance.

[0388] In this way, since the preset state-action table of the reinforcement learning model records the state-action penalty value corresponding to each action type in multiple action types in each state in multiple states, it is only necessary to use the current state of the terminal to query in the trained Q table to find the best action to be performed in the current state. Moreover, the optimal refresh rate value that balances the terminal power consumption and terminal performance can be found through a limited number of queries. Running the target application based on the optimal refresh rate value can greatly reduce the power consumption of the terminal. At the same time, it can also ensure that the video playback quality is not affected, that is, it can ensure a smooth viewing experience and avoid flickering and freezing on the terminal screen.

[0389] As mentioned earlier, Figure 1 The refresh rate of the video layer in the four application layers of the video APP shown in the figure is 24Hz, and the refresh rate of the bullet screen layer is 120Hz. According to the prior art, the system refresh rate is:

[0390] System Refresh Rate=max{24,120}=120Hz, the operating current of the terminal is very high.

[0391] In the present disclosure, assuming that the refresh rate value of the target layer obtained by the final optimization is 48 Hz, when the frame rate of the video source is 24 Hz, System Refresh Rate = max{24,48} ​​= 48 Hz. Compared with the prior art, the current is improved a lot, and the performance remains unchanged.

[0392] When the frame rate of the video source is 30Hz, System Refresh Rate=max{30,48}=48Hz. Compared with the existing technology, the current is improved a lot, and the performance remains unchanged.

[0393] When the frame rate of the video source is 60 Hz, System Refresh Rate = max{60,48} = 60 Hz. Compared with the existing technology, the current is improved and the performance remains unchanged.

[0394] When the frame rate of the video source is 120 Hz, System Refresh Rate = max{120,48} = 120 Hz. Compared with the prior art, the performance remains unchanged.

[0395] It should be noted that in the related art, manually forcibly setting the refresh rate of the app will change the playback quality of the video. In the present disclosure, the setting of the refresh rate does not change the playback quality of the video, but only reduces the high current problem caused by it. However, if the video source is a high frame rate video, the terminal screen will still run in high frame rate mode, but at this time it is not the cause of the increased current. From the experimental results, while ensuring that the performance remains basically unchanged, the power consumption and heat generation of the SpecialLayer (bullet screen layer) can generally be reduced by about 20% to 30%, and the power consumption of the Ordinary HighTraffic Layer can be reduced by about 8% to 20%.

[0396] Fig.25 is a block diagram illustrating an application execution device according to an exemplary embodiment of the present disclosure.

[0397] Reference Fig.25 , the device 2500 may include a target layer determination module 2501, an identification module 2502, an adjustment module 2503, a first determination module 2504 and a first operation module 2505.

[0398] The target layer determination module 2501 can determine the target application whose power consumption needs to be improved and the target layer whose refresh rate needs to be improved. The target application can include multiple application layers, each of which is used to implement its unique function.

[0399] According to an exemplary embodiment of the present disclosure, the target layer determination module 2501 can obtain the historical behavior feature data of the foreground operation of each application installed in the terminal, and then determine the application whose historical behavior feature data exceeds the predetermined range as the target application. In this way, the application can be screened by the historical behavior feature data of the foreground operation of the application, and then the application worth optimizing can be obtained. This avoids the phenomenon of ineffective optimization for a large number of applications that have no optimization value, and can save optimization resources.

[0400] According to an exemplary embodiment of the present disclosure, while the target application is running, the target layer determination module 2501 can obtain the running behavior characteristic data of the target application. In the case where the running behavior characteristic data exceeds a preset range, the target layer determination module 2501 can determine that the application layer with the highest actual sampling refresh rate value among the multiple application layers included in the target application is the target layer. Among them, the "running behavior characteristic data" can include at least one of the following items: "running average current TH app averagecurrent ”, “The total running power consumed by the video-related service modules of the application is TH appvideototaldrain ", "The current refresh rate value of the terminal screen is TH systemrefreshrate ”.

[0401] The identification module 2502 is mainly used to identify the type of the target layer using a decision tree model.

[0402] According to an exemplary embodiment of the present disclosure, the identification module 2502 is specifically used to obtain layer display behavior feature data of the target layer of the target application within a predetermined time period, and then input the layer display behavior feature data into a decision tree model to identify the type of the target layer. The decision tree model may be, but is not limited to, a C4.5 decision tree model.

[0403] According to an exemplary embodiment of the present disclosure, the “layer display behavior characteristic data” may include at least one of the following items:

[0404] The position of the target layer on the Z axis, the percentage of the target layer's area to the terminal screen's area, the target layer's transparency, whether the target layer corresponds to a specific keyword, whether the target layer belongs to a preset surface view layer type, the target layer's content concentration, the target layer's horizontal scrolling step concentration, the target layer's refresh rate, and the target layer's content flow.

[0405] According to an exemplary embodiment of the present disclosure, the application layer may be pre-divided into a plurality of types according to power consumption, and a predetermined at least one type among the plurality of types may be set as a power consumption adjustable layer.

[0406] According to an exemplary embodiment of the present disclosure, the application layer can be pre-divided into multiple types according to the power consumption, for example, but not limited to, a high power consumption layer (also referred to as a high flow rate) and a low power consumption layer (also referred to as a low flow rate). In addition, the high power consumption layer and the low power consumption layer can be further classified according to the display content, for example, the high power consumption layer can be classified into, but not limited to, a content layer and a comment information layer. For example, the high power consumption layer can be classified into a first type and a second type, that is, the first type can be an application layer for displaying the substantial content of the target application, which can also be referred to as an ordinary high flow layer. Exemplarily, it can be a video layer (Video Layer) / game layer (Game Layer); the second type can be an application layer for displaying comment information related to the substantial content, which can be, for example, a bullet screen layer. These two types can be set as power consumption adjustable layers because of their relatively high current consumption. In addition, the low power consumption layer can be used as the third type, which can also be referred to as an ordinary low flow layer. Usually, the current consumption caused by this type is relatively small, so it can not be set as a power consumption adjustable layer. Of course, the classification of the application layer is not limited to the above examples, and can also be divided into types according to actual power consumption or functional status. The above classification method is only an exemplary description.

[0407] According to an exemplary embodiment of the present disclosure, the identification module 2502 may collect layer display behavior characteristic data of the target layer at preset time intervals within a predetermined time period to obtain multiple sets of layer display behavior characteristic data corresponding to the target layer. For example, the layer display behavior characteristic data of the target layer may be collected every 5 seconds for 30 consecutive times, and a total of 30 sets of layer display behavior characteristic data may be obtained.

[0408] According to an exemplary embodiment of the present disclosure, the identification module 2502 may also input each set of layer display behavior characteristic data in the multiple sets of layer display behavior characteristic data corresponding to the target layer into the decision tree model to obtain the identification result corresponding to each set of layer display behavior characteristic data. For example, 30 sets of layer display behavior characteristic data may be input into the decision tree model to obtain the identification result corresponding to each set of layer display behavior characteristic data.

[0409] Next, when multiple groups of recognition results corresponding to multiple groups of layer display behavior feature data all indicate that the target layer is a power-adjustable layer, the target layer can be determined to be a power-adjustable layer. Exemplarily, when 30 groups of recognition results corresponding to 30 groups of layer display behavior feature data all indicate that the target layer is a power-adjustable layer, the target layer can be determined to be a power-adjustable layer. In this way, since multiple groups of collected data are used for identification, the probability of misjudgment can be greatly reduced, and the recognition accuracy of the target layer type can be guaranteed.

[0410] When the type of the target layer belongs to the power consumption adjustable layer, the adjustment module 2503 can adjust the refresh rate value of the target layer. That is, when the type of the target layer belongs to the adjustable high power consumption layer (special layer, ordinary high traffic layer), the refresh rate value of the target layer can be adjusted.

[0411] According to an exemplary embodiment of the present disclosure, the adjustment module 2503 can adjust the refresh rate value of the target layer based on at least one of the center position relationship between the target layer and the terminal screen, the area relationship between the target layer and the terminal screen, and the type of the target layer.

[0412] It should be noted that, from the perspective of visual experience, the closer the center of the application layer is to the center of the screen, the higher its refresh rate can be; or, the larger the ratio of the area of ​​the application layer to the area of ​​the terminal screen, that is, the larger the area of ​​the application layer, the higher the refresh rate of the application layer can be; or, the refresh rate of the application layer varies depending on the type of application layer. For example, the refresh rate of the bullet screen layer is generally higher than that of the video layer, and so on. In this way, by adjusting the refresh rate value of the target layer in combination with the characteristics of multiple dimensions, the accuracy of the refresh rate value adjustment can be guaranteed.

[0413] According to an exemplary embodiment of the present disclosure, the adjustment module 2503 can determine the preset minimum refresh rate value corresponding to the target layer based on at least one of the center position relationship between the target layer and the terminal screen, the area relationship between the target layer and the terminal screen, and the type of the target layer. Then, the adjustment module 2503 can adjust the refresh rate value of the target layer based on the preset minimum refresh rate value corresponding to the target layer and the actual sampling refresh rate value of the target layer. In this way, by adjusting the refresh rate value of the target layer based on the preset minimum refresh rate value and the actual sampling refresh rate value, the objectivity of the refresh rate value adjustment can be guaranteed. In addition, since the determined refresh rate value is a range interval, it can avoid the phenomenon that the system refresh rate is forced to be pulled up by directly taking the actual sampling frame rate of the target layer as its final refresh rate value, thereby avoiding excessive power consumption of the terminal and saving resources.

[0414] The first determination module 2504 may determine a refresh rate value of the terminal screen based on the adjusted refresh rate value of the target layer.

[0415] According to an exemplary embodiment of the present disclosure, the first determination module 2504 can determine the refresh rate value of the terminal screen based on the adjusted refresh rate value of the target layer and the actual sampled refresh rate values ​​of other application layers included in the target application. In this way, when determining the refresh rate value of the terminal screen, the adjusted refresh rate value of the target layer and the actual sampled refresh rate values ​​of other application layers are comprehensively considered, which can ensure that the power consumption of the terminal is reduced and the performance of the terminal is basically unchanged.

[0416] The first running module 2505 may run the target application based on the refresh rate value of the terminal screen.

[0417] According to an exemplary embodiment of the present disclosure, the status data acquisition module can acquire the power consumption and performance status data s(dc, df) of the terminal. Then, the optimization module can optimize the refresh rate value of the target layer based on the power consumption and performance status data s(dc, df) of the terminal through a reinforcement learning model, that is, through a trained Q table. Next, the second determination module can determine the optimized refresh rate value of the terminal screen based on the optimized refresh rate value of the target layer. Then, the second operation module can run the target application based on the optimized refresh rate value of the terminal screen.

[0418] In this way, since the converged Q table is obtained through a large amount of training, it has guiding significance for the optimization process of the actual refresh rate value, and can ensure that the refresh rate value obtained after optimization achieves a good balance between terminal power consumption and terminal performance.

[0419] According to an exemplary embodiment of the present disclosure, the optimization module can optimize the refresh rate value of the target layer based on the power consumption and performance status data s(dc, df) of the terminal and the preset state action table of the reinforcement learning model to obtain the optimized refresh rate value of the target layer. Among them, the preset state action table, that is, the Q table, can contain multiple states s(dc, df), and each state can correspond to multiple refresh rate adjustment action types. Exemplarily, each state can correspond to 3 refresh rate adjustment action types, namely: (maintain refresh rate) 0, (adjust the refresh rate by one level downward) 1, (adjust the refresh rate by one level upward) 2. Each refresh rate adjustment action type can correspond to its own state action penalty value (Q value).

[0420] In this way, since the preset state-action table of the reinforcement learning model records the state-action penalty value corresponding to each action type in multiple action types in each state in multiple states, it is only necessary to use the current state of the terminal to query in the trained Q table to find the best action to be performed in the current state. Moreover, the optimal refresh rate value that balances the terminal power consumption and terminal performance can be found through a limited number of queries. Running the target application based on the optimal refresh rate value can greatly reduce the power consumption of the terminal. At the same time, it can also ensure that the video playback quality is not affected, that is, it can ensure a smooth viewing experience and avoid flickering and freezing on the terminal screen.

[0421] Fig.26 is a block diagram illustrating an electronic device 2600 according to an exemplary embodiment of the present disclosure.

[0422] Reference Fig.26 The electronic device 2600 includes at least one memory 2601 and at least one processor 2602. The at least one memory 2601 stores instructions. When the instructions are executed by the at least one processor 2602, the application running method according to the exemplary embodiment of the present disclosure is executed.

[0423] As an example, the electronic device 2600 may be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above instructions. Here, the electronic device 2600 is not necessarily a single electronic device, but may also be any device or circuit capable of executing the above instructions (or instruction sets) individually or in combination. The electronic device 2600 may also be part of an integrated control system or system manager, or may be configured as a portable electronic device interconnected with a local or remote (e.g., via wireless transmission) interface.

[0424] In the electronic device 2600, the processor 2602 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0425] The processor 2602 may execute instructions or codes stored in the memory 2601, wherein the memory 2601 may also store data. Instructions and data may also be sent and received over a network via a network interface device, wherein the network interface device may employ any known transmission protocol.

[0426] The memory 2601 may be integrated with the processor 2602, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. In addition, the memory 2601 may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory 2601 and the processor 2602 may be operationally coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor 2602 can read files stored in the memory.

[0427] In addition, the electronic device 2600 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device 2600 may be connected to each other via a bus and / or a network.

[0428] Further, an artificial intelligence model may be used to execute the application running method executed by the electronic device 2600. At this time, at least one of the multiple modules included in the electronic device 2600 may be implemented by the AI ​​model. Functions associated with AI may be executed by non-volatile memory, volatile memory, and a processor.

[0429] The processor may include one or more processors. In this case, the one or more processors may be general-purpose processors, such as a central processing unit (CPU), an application processor (AP), etc., a processor used only for graphics (such as a graphics processing unit (GPU), a visual processing unit (VPU) and / or an AI-specific processor (such as a neural processing unit (NPU)).

[0430] One or more processors control the processing of input data according to predefined operating rules or artificial intelligence (AI) models stored in non-volatile memory and volatile memory. As an example, the predefined operating rules or artificial intelligence models can be provided by training or learning. Here, providing by learning means that by applying a learning algorithm to multiple learning data, a predefined operating rule or AI model with desired characteristics is formed. Learning can be performed in the device itself that executes AI according to an embodiment, and / or can be implemented by a separate server / device / system.

[0431] As an example, an artificial intelligence model can be composed of multiple neural network layers. Each layer has multiple weight values, and layer operations are performed by calculations of the previous layer and operations on multiple weight values. Examples of neural networks include, but are not limited to, convolutional neural networks (CNNs), deep neural networks (DNNs), recurrent neural networks (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), generative adversarial networks (GANs), and deep Q networks.

[0432] A learning algorithm is a method of using multiple learning data to train a predetermined target device (e.g., a robot) to enable, allow, or control the target device to make a determination or prediction. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0433] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium may also be provided, and when the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to execute the above-mentioned application running method. Examples of computer-readable storage media here include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0434] According to the application operation method, device, electronic device and storage medium disclosed in the present invention, a decision tree model can be used to identify the type of target layer whose refresh rate is to be improved, and when the type of the target layer belongs to the power consumption adjustable layer, the refresh rate value of the target layer can be automatically adjusted, thereby affecting the change in the refresh rate value of the terminal screen. In this way, it is possible to avoid the system refresh rate being forced to increase, thereby avoiding the phenomenon that the terminal's operating current is high due to the system refresh rate being forced to increase, thereby avoiding the phenomenon that the terminal power consumption is excessive, and more resources can be saved. In addition, since the present invention does not need to manually set the corresponding refresh rate for different APPs one by one, that is, the entire process does not require manual intervention, thus avoiding repeated and cumbersome setting processes, making the refresh rate setting more intelligent and convenient.

[0435] Furthermore, the applications can be screened by the historical behavior feature data of the foreground running of the applications, and then the applications worth optimizing can be obtained. This avoids the phenomenon of ineffective optimization of a large number of applications that have no optimization value, and can save optimization resources.

[0436] Furthermore, since multiple sets of collected data are used for identification, the probability of misjudgment can be greatly reduced, and the recognition accuracy of the target layer type can be guaranteed.

[0437] Furthermore, by adjusting the refresh rate value of the target layer in combination with features of multiple dimensions, the accuracy of the refresh rate value adjustment can be ensured.

[0438] Furthermore, by adjusting the refresh rate value of the target layer based on the preset minimum refresh rate value and the actual sampled refresh rate value, the objectivity of the refresh rate value adjustment can be ensured. Moreover, since the determined refresh rate value is a range interval, the phenomenon that the system refresh rate is forced to be increased by directly using the actual sampled frame rate of the target layer as its final refresh rate value can be avoided, thereby avoiding excessive power consumption of the terminal and saving resources.

[0439] Furthermore, when determining the refresh rate value of the terminal screen, the adjusted refresh rate value of the target layer and the actual sampled refresh rate values ​​of other application layers are comprehensively considered, which can not only ensure that the power consumption of the terminal is reduced, but also ensure that the performance of the terminal remains basically unchanged.

[0440] Furthermore, since the converged Q table is obtained through a large amount of training, it has guiding significance for the optimization process of the actual refresh rate value, and can ensure that the refresh rate value obtained after optimization achieves a good balance between terminal power consumption and terminal performance.

[0441] Furthermore, since the preset state-action table of the reinforcement learning model records the state-action penalty value corresponding to each action type in multiple action types in each state, it is only necessary to use the current state of the terminal to query in the trained Q table to find the best action to be performed in the current state. Moreover, the optimal refresh rate value that balances the terminal power consumption and terminal performance can be found through a limited number of queries. Running the target application based on the optimal refresh rate value can greatly reduce the power consumption of the terminal. At the same time, it can also ensure that the video playback quality is not affected, that is, it can ensure a smooth viewing experience and avoid flickering and freezing on the terminal screen.

[0442] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

[0443] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An application running method, characterized in that: include: Determine a target application whose power consumption is to be improved and a target layer whose refresh rate is to be improved; Using a decision tree model to identify the type of the target layer; When the type of the target layer is a power consumption adjustable layer, adjusting the refresh rate value of the target layer; Determining a refresh rate value of a terminal screen based on the adjusted refresh rate value of the target layer; The target application is run based on the refresh rate value of the terminal screen.

2. The application running method according to claim 1, characterized in that: The adjusting the refresh rate value of the target layer includes: The refresh rate value of the target layer is adjusted based on at least one of a central position relationship between the target layer and the terminal screen, an area relationship between the target layer and the terminal screen, and a type of the target layer.

3. The application running method according to claim 2, characterized in that: The adjusting the refresh rate value of the target layer includes: Determine a preset minimum refresh rate value corresponding to the target layer based on at least one of a central position relationship between the target layer and the terminal screen, an area relationship between the target layer and the terminal screen, and a type of the target layer; The refresh rate value of the target layer is adjusted based on the preset minimum refresh rate value corresponding to the target layer and the actual sampling refresh rate value of the target layer.

4. The application running method according to claim 1, characterized in that: After running the target application based on the refresh rate value of the terminal screen, the method further includes: Acquiring power consumption and performance status data of the terminal; Based on the power consumption and performance status data of the terminal, the refresh rate value of the target layer is optimized through a reinforcement learning model; Determining an optimized refresh rate value of the terminal screen based on the optimized refresh rate value of the target layer; The target application is run based on the optimized refresh rate value of the terminal screen.

5. The application running method according to claim 4, characterized in that: The optimizing the refresh rate value of the target layer based on the power consumption and performance status data of the terminal through a reinforcement learning model includes: Based on the power consumption and performance status data of the terminal and the preset state action table of the reinforcement learning model, the refresh rate value of the target layer is optimized to obtain the optimized refresh rate value of the target layer, wherein the preset state action table includes multiple states, each of the states corresponds to multiple refresh rate adjustment action types, and each of the refresh rate adjustment action types corresponds to its own state action penalty value.

6. The application running method according to claim 1, characterized in that: The step of determining a refresh rate value of a terminal screen based on the adjusted refresh rate value of the target layer includes: The refresh rate value of the terminal screen is determined based on the adjusted refresh rate value of the target layer and the actual sampled refresh rate values ​​of other application layers included in the target application.

7. The application running method according to claim 1, characterized in that: The determining of the target application whose power consumption needs to be improved includes: Acquire historical behavior characteristic data of foreground operation of each application installed in the terminal; The application whose historical behavior characteristic data exceeds a predetermined range is determined as the target application.

8. The application running method according to claim 1, characterized in that: The determining a target layer of the target application whose refresh rate is to be improved includes: While the target application is running, acquiring running behavior characteristic data of the target application; When the running behavior characteristic data exceeds a preset range, it is determined that the application layer with the highest corresponding actual sampling refresh rate value among the multiple application layers included in the target application is the target layer.

9. The application running method according to claim 1, characterized in that: The using a decision tree model to identify the type of the target layer includes: Acquire layer display behavior characteristic data of the target layer of the target application within a predetermined time period; The layer display behavior characteristic data is input into the decision tree model to identify the type of the target layer.

10. The application running method according to claim 9, characterized in that: The obtaining the layer display behavior characteristic data of the target layer of the target application within a predetermined time period includes: The layer display behavior characteristic data of the target layer is collected at preset time intervals within the predetermined time period to obtain multiple groups of layer display behavior characteristic data corresponding to the target layer.

11. The application running method according to claim 10, characterized in that: The step of inputting the layer display behavior characteristic data into the decision tree model to identify the type of the target layer comprises: Inputting each set of layer display behavior characteristic data among the multiple sets of layer display behavior characteristic data corresponding to the target layer into the decision tree model respectively, and obtaining a recognition result corresponding to each set of layer display behavior characteristic data; In a case where the multiple groups of identification results corresponding to the multiple groups of layer display behavior characteristic data all indicate that the target layer is the power consumption adjustable layer, it is determined that the target layer is the power consumption adjustable layer.

12. The application running method according to any one of claims 9 to 11, characterized in that: The layer display behavior characteristic data includes at least one of the following items: The position of the target layer on the Z axis, the percentage of the area of ​​the target layer to the area of ​​the terminal screen, the transparency of the target layer, whether the target layer corresponds to a specific keyword, whether the target layer belongs to a preset surface view layer type, the content concentration of the target layer, the horizontal scrolling step concentration of the target layer, the refresh rate value of the target layer, and the content flow of the target layer.

13. The application running method according to claim 1, characterized in that: The application layer is pre-divided into multiple types according to power consumption, and at least one predetermined type among the multiple types is set as the power consumption adjustable layer.

14. The application running method according to claim 13, characterized in that: The multiple types include a first type, a second type, and a third type, and the first type and the second type are set as the power consumption adjustable layer; Wherein, the first type is an application layer for displaying substantial content of the target application; The second type is an application layer for displaying comment information related to the substantive content; The third type is an application layer type included in the application except the first type and the second type.

15. An application running device, characterized in that: include: A target layer determination module is configured to determine a target application whose power consumption is to be improved and a target layer whose refresh rate is to be improved; An identification module, configured to identify the type of the target layer using a decision tree model; An adjustment module, configured to adjust the refresh rate value of the target layer when the type of the target layer is a power consumption adjustable layer; A first determining module is configured to determine a refresh rate value of a terminal screen based on the adjusted refresh rate value of the target layer; The first running module is configured to run the target application based on the refresh rate value of the terminal screen.

16. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the application running method according to any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the application running method as claimed in any one of claims 1 to 14.

Images