Frame loss prediction method and device, electronic equipment and computer readable storage medium

Abstract

The invention discloses a frame loss prediction method and device, electronic equipment and a computer readable storage medium, which can predict a frame loss risk in a frame drawing task so as to reduce frame loss. The method comprises the steps of determining corresponding previous state data of a target task in a previous frame under the condition that current frame drawing is started; the target task represents a task related to frame loss in a frame drawing task; performing frame loss prediction by using the frame loss prediction model and the previous state data, and determining a frame loss prediction result corresponding to the current frame; wherein the frame loss prediction model is obtained by counting at least one historical state data set of the target task in at least one historical statistical window and fitting according to the at least one state data set; each of the at least one statistical window includes at least one frame.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a frame loss prediction method, device, electronic device and computer-readable storage medium. Background Art

[0002] In the current Linux kernel, the load tracking algorithm is mainly used to predict the future load of the system based on the past load of the system, so as to evaluate future risks. However, the current load tracking algorithm does not perform load prediction for frame drawing tasks, so it is impossible to predict or perceive the risk of frame loss in frame drawing tasks, and thus it is impossible to take corresponding measures in time to reduce frame loss, which leads to an increase in the frame loss rate. Summary of the invention

[0003] The embodiments of the present application are intended to provide a frame loss prediction method, device, electronic device, and computer-readable storage medium.

[0004] The technical solution of this application is implemented as follows:

[0005] In a first aspect, a frame loss prediction method is provided, comprising:

[0006] In the case of starting the current frame drawing, determining the previous state data corresponding to the target task in the previous frame; the target task represents the task related to the frame loss in the frame drawing tasks;

[0007] Performing frame loss prediction using a frame loss prediction model and previous state data to determine a frame loss prediction result corresponding to the current frame;

[0008] The frame loss prediction model is obtained by statistically analyzing at least one historical state data set of the target task in at least one historical statistical window and fitting according to the at least one state data set; each of the at least one statistical window includes at least one frame.

[0009] In a second aspect, a frame loss prediction device is provided, comprising:

[0010] A determination module, used to determine the previous state data corresponding to the target task in the previous frame when starting the current frame drawing; the target task represents the task related to the frame loss in the frame drawing tasks;

[0011] A prediction module is used to perform frame loss prediction using a frame loss prediction model and previous state data to determine a frame loss prediction result corresponding to the current frame; wherein the frame loss prediction model is obtained by counting at least one historical state data set of the target task in at least one historical statistical window and fitting according to the at least one state data set; each of the at least one statistical window includes at least one frame.

[0012] In a third aspect, an electronic device is provided, comprising: a memory and a processor; wherein:

[0013] The memory is used to store executable instructions.

[0014] The processor is used to implement the frame loss prediction method provided in the present application when executing the executable instructions stored in the memory.

[0015] In a fourth aspect, a computer-readable storage medium is provided, storing executable instructions for causing a processor to implement the frame loss prediction method provided in the present application when executing the instructions.

[0016] In a fifth aspect, a computer program product is provided, including a computer program or instructions, which, when executed by a processor, implements the frame loss prediction method provided in the present application.

[0017] The present application provides a frame loss prediction method, device, electronic device and computer-readable storage medium, the method comprising: determining the previous state data corresponding to the target task in the previous frame when the current frame drawing is started; the target task represents the task related to the frame loss in the frame drawing task; using the frame loss prediction model and the previous state data to predict the frame loss, and determining the frame loss prediction result corresponding to the current frame; wherein the frame loss prediction model is obtained by counting at least one historical state data set of the target task in at least one historical statistical window, and fitting according to at least one state data set; each statistical window in at least one statistical window includes at least one frame. In this way, the frame loss prediction model is fitted by counting at least one historical state data set in at least one statistical window of the target task related to the frame loss, so that the characteristics of the state data in the target task under the frame loss condition can be identified by the frame loss prediction model, so that according to the previous state data corresponding to the target task in the previous frame, the frame loss prediction can be performed on the current frame to obtain the frame loss prediction result corresponding to the current frame. The embodiment of the present application realizes the prediction of the risk of frame loss according to the characteristics of the frame drawing task related to the frame loss, so that corresponding measures can be taken based on the frame loss prediction result to reduce the frame loss. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of a scheme flow of a frame loss prediction method provided in an embodiment of the present application;

[0019] Figure 2 A schematic diagram of the frame loss probability corresponding to different execution times of the UI Thread provided in an embodiment of the present application;

[0020] Figure 3 A schematic diagram of a frame loss prediction method provided in an embodiment of the present application Figure 1 ;

[0021] Figure 4 A schematic diagram of a process of fitting a frame loss prediction model and performing frame loss prediction in a frame loss prediction method provided in an embodiment of the present application;

[0022] Figure 5 A schematic diagram of a frame loss prediction method provided in an embodiment of the present application Figure 2 ;

[0023] Figure 6 A schematic diagram of a process of fitting a frame loss prediction model in a frame loss prediction method provided in an embodiment of the present application;

[0024] Figure 7 A schematic diagram of a frame loss prediction method provided in an embodiment of the present application Figure 3 ;

[0025] Figure 8 A schematic diagram of a flow chart of applying a frame loss prediction method provided in an embodiment of the present application to an actual scenario;

[0026] Fig. 9 A schematic diagram of the structure of a frame loss prediction device provided in an embodiment of the present application;

[0027] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0029] In current Linux kernels, such as the Linux kernel of the Android system, the prediction of future risks is mainly achieved through the load tracking algorithm, which predicts the load of the central processing unit (CPU) to achieve load balancing and DVFS. The load prediction algorithm predicts the CPU load in the next load prediction window based on the CPU load in several past load prediction windows, thereby providing a basis for the CPU's dynamic voltage and frequency scaling (Dynamic Voltage and Frequency Scaling, DVFS) and task migration.

[0030] Frame drawing is an important factor that directly affects the user experience in Android devices. The current related technologies usually optimize frame drawing by predicting the load of the next frame based on the local principle that the content between consecutive frames is similar. For example, as shown in formula (1), it is as follows:

[0031] S(n+1)=a·L(n)+(1-a)·S(n) Formula (1)

[0032] Among them, L(n) is the actual load when generating the nth frame, S(n) is the predicted load of the nth frame, and a is the smoothing factor. The larger the value of a, the smaller the impact of historical data. By predicting the load of the next frame, the CPU frequency and the core selection of drawing-related tasks are dynamically adjusted to achieve the purpose of making the frame drawing time close to the expected frame interval.

[0033] However, the current load tracking algorithm treats all tasks equally, and counting the load of all tasks will bring a large overhead. Therefore, the current CPU does not perform special processing on tasks related to frame drawing when scheduling tasks. This results in the CPU being unable to perceive this risk even when the drawing of a frame cannot be completed in time. In addition, the window used for load prediction is close to the frame interval, but the two are not equivalent. This means that based on the current implementation of the load prediction algorithm, it is impossible to predict the load for the frame, and the CPU load situation cannot well reflect the risk in the frame drawing process. There is no absolute correlation between the two.

[0034] Furthermore, on current Android devices, the Linux kernel does not have a convenient method to determine whether the current frame is dropped. Currently, the above prediction method based on exponential smoothing is only a prediction of the load, and cannot predict the risks in the drawing process, and has little effect in complex and changeable user scenarios. This is because users frequently switch applications during use, and it is difficult to make accurate load predictions using chain prediction models, which have large errors in actual use. At the same time, this method predicts based on frames, and when risks occur within the frame, it cannot make timely adjustments. The above method is implemented on a device with a fixed refresh rate, which cannot meet the needs of current devices that support dynamic refresh rates. At the same time, the current related technology prioritizes tasks on small cores to save power when drawing frames, which can no longer meet performance requirements.

[0035] In summary, the current load tracking algorithm is not optimized for frame drawing and cannot timely predict the risk of frame loss during drawing.

[0036] The embodiments of the present application provide a frame loss prediction method, apparatus, and device, which can conveniently identify risky frames based on the status data of tasks related to frame loss, and use the identification results for CPU task scheduling and DVFS. The frame loss prediction method of the embodiments of the present application is applied to electronic devices. In some embodiments, the electronic devices can be implemented as various types of user terminals such as laptops, tablet computers, desktop computers, set-top boxes, mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), etc. Exemplarily, the electronic devices may include smartphones, tablet computers, laptop computers, desktop computers, smart speakers, smart watches, etc. equipped with intelligent operating systems such as Android systems. The specific selection is based on the actual situation, and the embodiments of the present application are not limited.

[0037] The main program flow of this application is as follows Figure 1 As shown, first, the Android frame drawing process is analyzed, and tasks closely related to frame loss are found as target tasks, and the data closely related to frame loss in the target tasks are determined as the status data corresponding to the target tasks. After that, data is collected on the device under test, and the risk frame recognition model is fitted by collecting the status data generated by the target task during operation. Then, the status information set of historical frames is collected in real time when the device is running, and the risk of the current frame is predicted according to the risk frame recognition model. Finally, the CPU task scheduling and DVFS are optimized according to the frame loss risk flag obtained by the model to achieve power consumption savings while meeting performance requirements.

[0038] The basic process of Android frame drawing usually includes: 1. Request layout: measure the size of each view to be drawn in order to meet the layout requirements of drawing; 2. Layout: determine the position of each view in the page; 3. Drawing: call the drawing method to draw each view and get the drawing result; 4. Synthesis: synthesize all the drawing results to display on the screen. During use, the device will continuously generate graphics content in the form of frames and present it to the display hardware. This process is called the graphics pipeline. The work of the graphics pipeline is completed by the CPU and the graphics processing unit (GPU). If the frame drawing time exceeds the length of one frame, it is considered to cause frame loss.

[0039] In actual applications, different application scenarios may have different graphics pipelines, but the graphics pipeline is executed by several tasks. In this application, the task responsible for the main work of the graphics pipeline is taken as the target task. For example, the target task may include the main thread (or UI thread, UIThread) responsible for processing UI operations and drawing interfaces in the Android system, and / or the rendering thread (Render Thread) responsible for processing rendering-related tasks in the Android system, the core component SurfaceFlinger (SF) responsible for graphics display in the Android operating system, etc. The specific selection is made according to the actual situation, and the embodiments of this application are not limited.

[0040] When determining the target task, it is necessary to determine the state data corresponding to the target task. In the target task, there may be many kinds of data related to frame drawing performance, for example, it may include the execution time of the task, the number of instructions, the number of cycles, etc. The present application proposes to determine the state data corresponding to the target task based on correlation and locality. Among them, correlation refers to the correlation between data and frame loss. If the distribution of the value of a certain data in the two cases of frame loss and no frame loss is significantly different, the data has a high correlation with the frame loss. Locality refers to the closeness of the values ​​of data between consecutive frames. If the difference between the values ​​of a certain data between adjacent frames is close to 0, it can be considered that the data has good locality. The present application can combine correlation and locality to determine the state data corresponding to the target task. For example, if a certain data generated by the target task has a high correlation with frame loss and good locality, the state data can be determined as state data.

[0041] Next, taking UI Thread as an example, we analyze the relevance and locality of UI Thread to determine the state data corresponding to UIThread, as follows:

[0042] 1. Analyze the correlation and locality between UI Thread and drawing method execution time.

[0043] In the Android system, frame loss information needs to be obtained from the framework layer, while the main thread, such as the UI Thread, runs in a different layer, such as the kernel layer. In order to analyze the correlation between the main thread and frame loss, we can first analyze the drawing method called by the main thread in the framework layer, determine the correlation between the execution time of the drawing method in the framework layer and the execution time of the main thread, and determine the correlation between the execution time of the drawing method and the frame loss information in the same framework layer. In this way, we can determine the correlation between the execution time of the main thread and the frame loss information.

[0044] The drawing method includes a method for processing frame drawing that runs in the main thread. Exemplarily, the drawing method may include the doFrame function in the Android system. The doFrame function is a core function executed in the Framework layer of the Android system to process the drawing of each frame, and is used to execute the request layout, layout, drawing operations, synthesis and display processes in the above-mentioned frame drawing process. The work of the doFrame function is mainly completed by the UI Thread using CPU resources. It can be seen that the longer the execution time of the doFrame function, the more CPU computing power is required for frame drawing.

[0045] The main thread UI Thread performs related work in the Linux kernel. There is no concept of frames in the kernel. Instead, the arrival of frames is determined by synchronization signals. Exemplarily, the synchronization signal may include a hardware VSYNC signal. In order to analyze the correlation between the execution time of the UIThread and doFrame functions, the interval between the two synchronization signals may be regarded as a frame, and the execution time of the doFrame function and the UI Thread may be counted in units of frames, and the error between the two within a frame may be calculated. Through actual statistics and calculations, the embodiment of the present application finds that within the duration of a frame FrameInterval, the execution time T of the doFrame function doFrame The execution time T of UI Thread UI The error between error(error=(T doFrame -T UI ) / FrameInterval) is close to 6%, which shows that the execution time of the doFrame function is highly correlated with the execution time of the UI Thread.

[0046] Moreover, by calculating the error of the execution time of the doFrame function between adjacent frames, the present application found that the execution time of the doFrame function between adjacent frames is close to 0%, which indicates that the execution time of the doFrame function has a strong locality, and further indicates that the execution time of the UI Thread, which has a high correlation with the execution time of the doFrame function, has a strong locality.

[0047] 2. Analyze the correlation between frame drops during UI Thread execution time.

[0048] Based on this, since both the frame loss information and the execution time of the doFrame function can be obtained in the framework layer of the Android system, by counting the frame loss information in the framework layer and the execution time of the doFrame function, the correlation between the execution time of the doFrame function and the frame loss can be analyzed. For example, by sorting multiple frames (such as 100 frames) executed by the doFrame function according to the execution time of the doFrame function, the frame loss probability corresponding to different execution times can be obtained as follows: Figure 2 As shown. Figure 2 It can be seen that frame loss is more concentrated in the last 15% of frames with longer doFrame function execution time, which shows that the correlation between doFrame function execution time and frame loss is high. Since the correlation between doFrame function execution time and UI Thread execution time is high, it can be explained that the correlation between UI Thread execution time and frame loss is high.

[0049] In summary, the correlation between UI Thread execution time and frame loss is high and has strong locality. Therefore, the execution time of the target task UI Thread can be used as state data. In this way, the target task such as UI Thread can be used as the main observation target, and the state data of the target task such as UI Thread execution time can be counted in each frame. By fitting the data, the potential frame loss risk of the currently drawing frame can be predicted.

[0050] Based on the above analysis, see Figure 3 , Figure 3 A schematic diagram of a frame loss prediction method provided in an embodiment of the present application Figure 1 ,include:

[0051] S101 . When starting the current frame drawing, determine the previous state data corresponding to the target task in the previous frame.

[0052] In S101, the target task represents the task related to frame loss in the frame drawing task. Exemplarily, the target task may include the target task determined by the above analysis process, such as UI thread, Render Thread, or SurfaceFlinger, etc.; the state data may include the data generated by the target task determined by the above analysis process, which is highly correlated with frame loss and has strong locality, such as the running time of UI Thread, the number of instructions per cycle (IPC) of UI Thread and Render Thread, the number of buffers in the buffer queue (bufferqueue) of SurfaceFlinger, etc. The specific selection is made according to the actual situation, and the embodiments of the present application are not limited.

[0053] In the embodiment of the present application, the electronic device regards the time interval between two adjacent synchronization signals as a frame, and records the time when each synchronization signal arrives as the end time of the previous frame and the start time of the next frame, that is, determining to enter or start drawing of the current frame. Exemplarily, the synchronization signal may include the above-mentioned hardware synchronization signal HW_VSYNC, or may also include signals characterizing the target task entering the queue, leaving the queue, and adding and removing CPUs, etc. The specific selection is made according to the actual situation, and the embodiment of the present application is not limited.

[0054] In the embodiment of the present application, starting the current frame drawing indicates that the current frame is in a state to be drawn. Whether the current frame will be lost can be predicted based on the state data of the previous frame in the past. In the case where the current frame drawing is started according to the synchronization signal, the state data generated by the target task in the previous frame is counted as the previous state data.

[0055] S102: Utilize the frame loss prediction model and the previous state data to perform frame loss prediction, and determine a frame loss prediction result corresponding to the current frame.

[0056] Considering the uncertainty of user operations and the complexity of application scenarios, the embodiment of the present application collects statistics of the historical status data set of the target task through a sliding window to determine the limit or threshold of the frame loss risk according to the statistical historical status data set.

[0057] In the embodiment of the present application, the frame loss prediction model is obtained by counting at least one historical state data set of the target task in at least one historical statistical window and fitting according to the at least one state data set. Each of the at least one statistical window includes at least one frame.

[0058] Exemplarily, information can be counted with n frames as a historical statistical window, and N historical statistical windows can be set. Wherein, n and N are both greater than or equal to 1. In each historical statistical window, when the synchronization signal arrives, the state data of the previous frame is counted, and a local array is used to save the state data of each frame in the historical statistical window, so as to obtain a state data set corresponding to each historical statistical window, and then obtain at least one historical state data set.

[0059] It is understandable that at least one historical state data set represents the state data generated when the target task is running in the historical time. According to the at least one historical state data set, data fitting is performed, and the frame loss prediction model obtained can represent the numerical distribution of the state data when the target task is running in the historical time. Based on the above analysis, the numerical distribution of the state data is obviously different in the two cases of frame loss and no frame loss. Therefore, according to the frame loss prediction model, the boundary or threshold between frame loss and no frame loss can be determined, and then the frame loss prediction can be performed in combination with the previous state data of the previous frame as the frame loss prediction result corresponding to the current frame.

[0060] For example, Figure 4 As shown, when the HW_VSYNC signal arrives, the state data of the target task (such as UI Thread, SF, etc.) during operation between the current HW_VSYNC signal and the previous HW_VSYNC signal is collected to obtain multiple state data sets, and a frame loss prediction model is fitted according to the obtained multiple state data sets. The frame loss prediction result corresponding to the current frame is predicted according to the frame loss prediction model, and the frame loss prediction result corresponding to the current frame is output in the form of setting the risk flag of the current frame.

[0061] For example, a flag bit 1 may be used to represent a frame loss prediction result with a high risk of frame loss, and a flag bit 0 may be used to represent a frame loss prediction result with a low risk of frame loss. Frame loss prediction results represented by other data forms may also be included, which are specifically selected according to actual conditions and are not limited in the embodiments of the present application.

[0062] For example, the parameters of the offline fitted frame loss prediction model can be put into the Linux kernel for online identification of the frame loss risk of each frame. A risk frame flag is used to record the prediction result of the frame loss prediction model. If the result calculated after substituting the last state data of the previous frame into the frame loss prediction model is close to 1, it means that the current frame has a greater risk of frame loss, and the risk frame flag is set to 1. Otherwise, it means that the current frame has a small risk of frame loss, and the risk frame flag is set to 0.

[0063] It can be understood that the embodiment of the present application fits a frame loss prediction model by counting at least one historical state data set in at least one statistical window of the target task related to frame loss. Therefore, the frame loss prediction model can be used to identify the characteristics of the state data in the target task under frame loss conditions. In this way, according to the previous state data corresponding to the target task in the previous frame, the frame loss prediction of the current frame can be performed to obtain the frame loss prediction result corresponding to the current frame. The embodiment of the present application implements frame loss risk prediction based on the characteristics of the frame drawing task related to frame loss, so that corresponding improvement measures can be taken based on the obtained frame loss prediction result to reduce frame loss.

[0064] In some embodiments, the present application predicts the risk flag of each frame in the Linux kernel, and offline statistics the actual frame loss of each frame in the Linux framework layer, and statistically calculates the prediction accuracy through offline analysis and comparison. Experimental results show that the comprehensive average prediction accuracy of the present application in the test scenario can reach 94%.

[0065] In some embodiments, in S102, the frame loss prediction model and the previous state data are used to perform frame loss prediction. Before determining the frame loss prediction result corresponding to the current frame, the frame loss prediction result may be determined by Figure 5 The process of S001-S002 shown in the figure is used to fit the frame loss prediction model, as follows:

[0066] S001. Count at least one historical status data set of a target task in at least one historical statistical window.

[0067] Among them, at least one state data set corresponds to at least one historical statistical window. In each historical statistical window, the electronic device can collect the historical state data of the target task when the synchronization signal arrives, save them in a global array for easy use, and output them as a log file through a tool for offline analysis. The historical state data may include one or more predetermined state data that is highly correlated with frame loss and has locality and is generated when the target task is running.

[0068] S002. According to at least one historical state data set, a frame loss prediction model is obtained by fitting.

[0069] In the statistical process, for each historical statistical window, at the last frame of the historical statistical window, the state data of all frames in the historical statistical window are integrated into a window frame loss model, and according to the window frame loss model of each historical statistical window, a comprehensive frame loss prediction model is fitted in combination with a preset fitting strategy. Exemplarily, the process of fitting a frame loss prediction model according to N historical statistical windows containing n frames can be as follows: Figure 6 shown.

[0070] The preset fitting strategy can be determined according to the needs of the actual application, and the embodiment of the present application does not specifically limit it. Here, a method for fitting according to the time sequence is provided as follows:

[0071] At least one weight corresponding to at least one historical statistical window is determined according to the time sequence of at least one historical statistical window; and a frame loss prediction model is obtained by fitting according to at least one weight and at least one historical state data set.

[0072] That is, you can set the weight coefficient to focus on closer windows to achieve fast response in high-load scenarios.

[0073] In some embodiments, the historical state data set includes: the historical execution time of each frame of the target task in at least one frame of the historical statistical window, for example, the historical execution time of each frame of the UI Thread in at least one frame of the historical statistical window. After the frame loss prediction model is obtained by fitting, the numerical distribution of the historical execution time of the UI Thread in at least one historical statistical window can be determined according to the frame loss prediction model, and then the boundary or threshold for distinguishing between high risk of frame loss and low risk of frame loss can be determined.

[0074] In some embodiments, a target execution time threshold corresponding to a frame loss prediction model is determined based on at least one historical execution time of each historical state data set in at least one historical state data set; the target execution time threshold is determined by sorting the historical execution time of at least one historical state data set and obtaining the historical execution time at a preset position.

[0075] Here, each historical status data set includes at least one historical execution time, and all historical execution times in at least one historical status data set are uniformly sorted, wherein the sorting order may be the order of the historical execution time from large to small, or may be the order of the historical execution time from small to large. Among all the historical execution times that are sorted from large to small or from small to large, the target execution time threshold is determined based on the historical execution time at a preset position. Exemplarily, the target execution time threshold may include the historical execution time at the 85th position among all the historical execution times that are sorted from small to large, or may include the historical execution time at the 15th position among all the historical execution times that are sorted from large to small.

[0076] Accordingly, the previous state data includes: the execution time of the target task in the previous frame; when performing frame loss prediction according to the frame loss prediction model, the process of S103 may include:

[0077] When the execution time in the previous frame is greater than or equal to the target execution time threshold, a frame loss prediction result representing a high risk of frame loss is determined; when the execution time in the previous frame is equal to the target execution time threshold, a frame loss prediction result representing a low risk of frame loss is determined.

[0078] Exemplarily, based on the aforementioned analysis of UI Thread execution time and frame loss, the UI Thread execution time is sorted, and the probability of frame loss in the last 15% of frames with long UI Thread execution time is higher. Therefore, all historical execution times contained in at least one historical state data set can be sorted, and the UI Thread execution time at the 85% position (equivalent to the preset sorting position) with long historical execution time can be obtained as the target execution time threshold, which is used for subsequent online judgment of frame loss. If the execution time of the UI Thread in the previous frame is less than the UI Thread execution time at the 85% position in the frame loss prediction model (equivalent to the target execution time threshold), it can be considered that the current frame may not lose frames (low risk of frame loss), otherwise it is considered that frame loss may occur (high risk of frame loss).

[0079] It can be understood that the embodiment of the present application conducts an in-depth analysis of the status data generated by the target task, fits a frame loss prediction model through at least one historical status data generated by the target task, and uses it for frame loss prediction, thereby achieving targeted frame loss prediction and frame loss risk identification for frame drawing tasks.

[0080] In some embodiments, the target task runs on a target processor cluster of multiple processor clusters in the system; after determining the frame loss prediction result, it can also be performed as follows Figure 7 As shown, execute S201-S202 as follows:

[0081] S201. Calculate a first load corresponding to a target processor cluster and a second load corresponding to a non-target processor cluster among a plurality of processor clusters.

[0082] The non-target processor cluster is a processor cluster other than the target processor cluster among the multiple processors, that is, a processor cluster that does not run the target task. The first load includes the load corresponding to each processor in the target processor cluster, and the second load includes the load corresponding to each processor in the non-target processor cluster.

[0083] Currently, the load prediction algorithm of the Linux kernel predicts the CPU load based on the load prediction window. However, the load prediction window and the frame interval are very close, but not completely the same, resulting in the current load prediction algorithm being unable to accurately predict the CPU load of each frame.

[0084] The embodiment of the present application redesigns the load calculation method, and calculates the load corresponding to each processor in the target processor cluster as the first load based on at least one operating frequency run by each processor in the target processor cluster within a frame time and the time running at each operating frequency of at least one operating frequency, as well as the maximum operating frequency and maximum computing power corresponding to the target processor cluster; calculates the load corresponding to each processor in the non-target processor cluster as the second load based on at least one operating frequency run by each processor in the non-target processor cluster within a frame time and the time running at each operating frequency of at least one operating frequency, as well as the maximum operating frequency and maximum computing power corresponding to the non-target processor cluster.

[0085] In the embodiment of the present application, when the synchronization signal arrives, the load of the processor in the previous frame is calculated to obtain the first load and the second load. For example, for each processor CPU, as shown in formula (2), as follows:

[0086]

[0087] In formula (2), freq max and max_capacity are the maximum operating frequency and maximum computing power that the CPU can achieve. T represents the load calculation window preset in the embodiment of the present application, representing the interval between the last load calculation and the current load calculation, and T is less than or equal to the duration of one frame. t is the CPU operating frequency freq within time T. curr The actual running time. curr The processor is operating at frequency freq curr It should be noted that in the embodiment of the present application, the CPU operating frequency is adjusted accordingly after the load calculation, so the operating frequency freq within the T time is curr The CPU operating frequency can be load calculated and adjusted when outputting the frame loss prediction result corresponding to the current frame, or it can be load calculated and frequency adjusted periodically according to the above load calculation window to achieve real-time and dynamic load balancing and DVFS.

[0088] According to formula (2), for a certain CPU, the load average value of the CPU at all operating frequencies within one frame time is calculated, as shown in formula (3), as follows:

[0089] load avg =avg(load1,load2,...,load m ),m∈[1,M] (3)

[0090] Among them, load1 to load mIt is the load calculated when the CPU runs at each of the m operating frequencies in the last frame (the event interval between the last synchronization signal and the current synchronization signal), where m is at most the number of frequencies in the cluster where the processor is located (such as the target processor cluster or the non-target processor cluster). avg It is the average load value of the CPU at all operating frequencies within one frame time.

[0091] For each processor in the target processor cluster, the load average value of each processor in the target processor cluster when running at m operating frequencies in the previous frame can be calculated according to the process of formula (2)-formula (3), which is used as the load corresponding to each processor in the target processor cluster, thereby obtaining the first load.

[0092] For each processor in the non-target processor cluster, the load average value of each processor in the non-target processor cluster when running at m operating frequencies in the previous frame can be calculated according to the process of formula (2)-formula (3), which is used as the load corresponding to each processor in the non-target processor cluster, thereby obtaining the second load.

[0093] It should be noted that for different processor clusters, the maximum operating frequency, maximum computing power and number of frequency points that the CPU can achieve may be the same or different.

[0094] In this way, frame-based CPU load calculation is implemented, and the CPU load for each frame can be predicted.

[0095] S202: Adjust operating frequencies of multiple processor clusters according to the frame loss prediction result, the first load, and the second load.

[0096] In some embodiments, the frame loss risk predicted by the frame loss prediction model may be combined with a frame-based CPU load tracking algorithm for CPU task scheduling and DVFS to reduce the frame loss risk and save power consumption.

[0097] For example, after identifying the risk flag of each frame, the frequency can be increased or decreased in a targeted manner to control the execution efficiency of CPU rendering-related work, thereby achieving the purpose of ensuring normal completion of rendering and reducing power consumption overhead.

[0098] In some embodiments, the original CPU DVFS method can be limited by setting a frequency range of a processor cluster.

[0099] At present, there are usually three CPU clusters in the big.LITTLE multi-core architecture, namely small cores, large cores and super large cores. This application has counted the execution time of UI Thread on each cluster and found that its execution time on the large core cluster is about 80%. According to relevant knowledge, frame rendering related threads should be executed more in the same cluster to better guarantee performance and save energy consumption. Therefore, in the embodiment of this application, UI Thread is executed more on the large core cluster. Therefore, when frame loss occurs, UI Thread may run on a non-small core cluster. At present, the CPU frequency set for non-small core clusters is usually smaller than that for frame drawing tasks, while the CPU frequency set for small core clusters is suitable for frame drawing. Most of the UI Thread runs on the small core cluster when there is no possibility of frame loss. If the CPU frequency of the small core cluster is reduced at this time, it will force UI Thread to schedule on the non-small core cluster. Therefore, when the frame loss prediction model outputs a high risk of frame loss, a higher frequency can be set for the cluster that is running the UI Thread (large core cluster or super large core cluster), and a reasonable frequency range can be set for the cluster that is not running the UI Thread (small core cluster or large core cluster) according to the load of each CPU in its cluster, and the frequency range of the small core cluster can be used for the subsequent low risk of frame loss, thereby preventing the CPU frequency from being too high and causing the UIThread to be scheduled to the small core cluster. When the risk of frame loss is low, the large core cluster can set a reasonable frequency range according to the load of each CPU in its cluster, without limiting the super large core to ensure performance.

[0100] In some embodiments, when the frame loss prediction result indicates a high risk of frame loss, a first operating frequency range corresponding to the target processor cluster is determined based on the first load, the preset processor highest frequency point and the preset processor maximum computing power; the operating frequency of the processors in the target processor cluster is adjusted based on the first operating frequency range; load prediction is performed on a non-target processor cluster through a load prediction algorithm to determine a non-target predicted load; a second operating frequency range corresponding to the non-target processor cluster is determined based on the second load and the non-target predicted load; and the operating frequency of the processors in the non-target processor cluster is adjusted based on the second operating frequency range.

[0101] In some embodiments, according to formula (4) to formula (6), when the frame loss prediction result indicates a high risk of frame loss, the first operating frequency range corresponding to the target processor cluster can be determined according to the first load, the highest frequency point of the preset processor, and the maximum computing power of the preset processor, as follows:

[0102] load final_a =max(load a ) (4)

[0103]

[0104] In formula (4), load a represents the first load, that is, the load corresponding to each processor in the target processor cluster. That is to say, according to the load corresponding to each processor in the target processor cluster, the maximum load is determined as the load corresponding to the target processor cluster. final_a The first operating frequency range is calculated according to the load corresponding to the target processor cluster by using formula (5) and formula (6). In formula (5), f max Represents the highest frequency point that the processor can select, capacity max Represents the maximum available computing power of the CPU in the processor cluster. In formula (6), FrameInterval is the preset frame interval, and FrameDeadline represents the limit of the safe frame. For example, the FrameDeadline of each frame rendering can be set to 65% of the frame interval, or it can be set to other values ​​according to actual needs, which is not specifically limited in the embodiments of the present application. The first operating frequency range is: [freq1_min, freq1_max]. The operating frequency of the processor in the target processor cluster can be adjusted to the first operating frequency range to increase the operating frequency of the target processor cluster and reduce the risk of frame loss.

[0105] In some embodiments, the non-target predicted load includes a predicted load corresponding to each processor in the non-target processor cluster obtained by performing load prediction on each processor in the non-target processor cluster through a load prediction algorithm. The process of determining the second operating frequency range corresponding to the non-target processor cluster based on the second load and the non-target predicted load includes: determining a corresponding load error range corresponding to the non-target processor cluster based on the second load and the non-target predicted load; determining the second operating frequency range based on the second load and the load error range corresponding to the non-target processor cluster.

[0106] In some embodiments, when the frame loss prediction result indicates a high risk of frame loss, the error corresponding to each processor in the non-target processor cluster can be calculated according to the load corresponding to each processor in the non-target processor cluster and the predicted load, as shown in formula (7), as follows:

[0107]

[0108] Among them, load b ' is the predicted load corresponding to a processor in the non-target processor cluster, load b is the load corresponding to the processor in the non-target processor cluster, and error2 is the error corresponding to the processor in the non-target processor cluster.

[0109] In some embodiments, based on formula (7), the load error range [error 2] corresponding to the non-target processor cluster can be determined according to the maximum value max(error2) and the minimum value min(error2) of the error corresponding to each processor in the non-target processor cluster. min_2 ,error max_2 ], where error min_2 =min(error2),error max_2 =max(error2).

[0110] In some embodiments, when the frame loss prediction result indicates a high risk of frame loss, the process of determining the second operating frequency range according to the load error range corresponding to the second load and the non-target processor cluster can be as shown in formula (8) and formula (9), as follows:

[0111]

[0112] According to the second load, the maximum value of the load corresponding to each processor in the non-target processor cluster can be determined as the load corresponding to the non-target processor cluster. final_b , load final_b =max(load b ). According to formula (8) and formula (9), when the frame loss prediction result indicates a high risk of frame loss, the second operating frequency range [freq2_min, freq2_max] corresponding to the non-target processor cluster can be determined.

[0113] It should be noted that for different processor clusters, the highest frequency point f that the processor can select max , the maximum available computing capacity of the largest CPU max Can be the same or different.

[0114] In some embodiments, after determining the second operating frequency range corresponding to the non-target processor cluster, the operating frequency of the processors in the non-target processor cluster can be maintained within the second operating frequency range when the frame loss prediction result changes from a high risk of frame loss to a low risk of frame loss.

[0115] That is to say, for a low-risk non-target processor cluster (such as a small core cluster), the [freq2_min, freq2_max] set for the processor cluster when the risk is high can be directly used.

[0116] In some embodiments, when the frame loss prediction result indicates a low risk of frame loss, a load prediction algorithm is used to perform load prediction on the target processor cluster to determine a target predicted load; a first operating frequency range corresponding to the target processor cluster is determined based on the first load and the target predicted load; and the operating frequency of the processors in the target processor cluster is adjusted based on the first operating frequency range. The target predicted load includes the predicted load corresponding to each processor in the target processor cluster.

[0117] Exemplarily, when the frame loss prediction result indicates a low risk of frame loss, the first operating frequency range corresponding to the target processor cluster may be determined according to the first load and the target predicted load as shown in formula (10) and formula (11), as follows:

[0118]

[0119] In the case where the frame loss prediction result indicates a high risk of frame loss, the load corresponding to each processor in the target processor cluster can be used to predict the frame loss. a With the predicted load a ', calculate the error error1 corresponding to each processor in the non-target processor cluster, as shown in formula (12), as follows:

[0120]

[0121] Based on formula (12), the load error range [error1] corresponding to the target processor cluster can be determined according to the maximum value max(error1) and minimum value min(error1) of the error corresponding to each processor in the target processor cluster. min_1 ,error max_1 ], where error min_1 =min(error1),error max_1 =max(error1).

[0122] According to the above process, the frequency ranges corresponding to the target processor cluster under high risk of frame loss, the non-target processor cluster under high risk of frame loss, the target processor cluster under low risk of frame loss, and the non-target processor cluster under low risk of frame loss can be cut off according to the frame loss prediction results, and corresponding frequency adjustments can be made according to the frequency ranges.

[0123] It is understandable that the embodiments of the present application can adapt to different scenarios and different drawing methods, and can judge the risk of frame loss between the current frame and the next frame in real time with very low load in the Android kernel, and the recognition result has high accuracy. In addition, it can be used for various situations such as accurate frequency adjustment and migration of graphics pipeline tasks, limiting the original CPUDVFS, and achieving the effect of optimizing performance while keeping power consumption basically unchanged.

[0124] Based on the above embodiments, the present application provides a frame loss prediction method applied to actual scenarios, such as Figure 8 shown.

[0125] For each historical statistical window of at least one historical statistical window, when the synchronization signal arrives, the state data of the target task in the previous frame of the historical statistical window is collected, so as to update the state data set corresponding to the historical statistical window (such as the historical execution time set of the UI Thread). In the last frame of each historical statistical window, the window frame loss model corresponding to each historical statistical window is fitted according to the state data set corresponding to each historical statistical window. If the last frame of the statistical period is reached, the frame loss prediction model is fitted according to the window frame loss model corresponding to at least one historical statistical window, and the frame loss prediction is performed according to the frame loss prediction model. If the last frame of the statistical period is not reached, the state data set of the current record can be saved first. Online identification is performed according to the state data set of the current record, the flag position of the high-risk frame is set to 1, and the flag position of the high-risk frame is set to 0, and then CPU optimization is performed based on the risk frame flag. Exemplarily, frequency adjustment is performed according to the method in S201-S202 provided in the embodiment of the present application.

[0126] It is understandable that the CPU DVFS optimization method based on UI Thread can limit the original CPU DVFS in real time according to the operation status of UI Thread at an extremely low load only in the Linux kernel. The current Android system does not make good use of UI Thread. UI Thread can more conveniently indicate the frame loss information that was originally only available in the framework layer in the kernel, thereby realizing the frame loss prediction or frame loss risk identification of frame drawing. Moreover, UI Thread, as a key task in the graphics pipeline, has not been allocated enough CPU computing power by the Linux system. The embodiment of the present application optimizes UI Thread in a targeted manner through a further frequency adjustment process, which can achieve the effect of optimizing performance while keeping power consumption basically unchanged.

[0127] To implement the method of the embodiment of the present application, based on the same inventive concept, the embodiment of the present application also provides a frame loss prediction device, such as Fig. 9 As shown, the frame loss prediction device 1 includes:

[0128] The determination module 11 is used to determine the previous state data corresponding to the target task in the previous frame when the current frame drawing is started; the target task represents the task related to the frame loss in the frame drawing task;

[0129] The prediction module 12 is used to use the frame loss prediction model and the previous state data to perform frame loss prediction and determine the frame loss prediction result corresponding to the current frame; wherein the frame loss prediction model is obtained by counting at least one historical state data set of the target task in at least one historical statistical window and fitting according to the at least one state data set; each statistical window in the at least one statistical window includes at least one frame.

[0130] In some embodiments, the at least one state data set corresponds to the at least one historical statistical window, and the frame loss prediction device 1 also includes: a fitting module, the fitting module is used to use the frame loss prediction model and the previous state data to perform frame loss prediction, and before determining the frame loss prediction result corresponding to the current frame, count at least one historical state data set of the target task in the at least one historical statistical window; the at least one state data set corresponds to the at least one historical statistical window; according to the at least one historical state data set, the frame loss prediction model is fitted.

[0131] In some embodiments, the fitting module is further used to determine at least one weight corresponding to the at least one historical statistical window according to the time sequence of the at least one historical statistical window; and to fit the frame loss prediction model according to the at least one weight and the at least one historical status data set.

[0132] In some embodiments, the historical status data set includes: the historical execution time of the target task in each frame of at least one frame in the historical statistical window; the prediction module 12 is also used to determine the target execution time threshold corresponding to the frame loss prediction model after fitting the frame loss prediction model according to at least one historical execution time of each historical status data set in the at least one historical status data set; the target execution time threshold is determined by sorting the historical execution time of at least one historical status data set and obtaining the historical execution time at a preset position.

[0133] In some embodiments, the previous state data includes: the execution time of the target task in the previous frame; the prediction module 12 is also used to determine a frame loss prediction result representing a high risk of frame loss when the execution time in the previous frame is greater than or equal to the target execution time threshold; and to determine a frame loss prediction result representing a low risk of frame loss when the execution time in the previous frame is equal to the target execution time threshold.

[0134] In some embodiments, the target task runs on a target processor cluster of multiple processor clusters in the system; the fitting module is also used to determine at least one weight corresponding to the at least one historical statistical window according to the time sequence of the at least one historical statistical window; and the frame loss prediction model is fitted according to the at least one weight and the at least one historical execution time threshold.

[0135] In some embodiments, the previous state data includes: the execution time of the target task in the previous frame; the prediction module 12 is also used to determine a frame loss prediction result representing a high risk of frame loss when the execution time in the previous frame is greater than or equal to a target execution time threshold; the target execution time threshold is determined based on the at least one historical execution time threshold; when the execution time in the previous frame is equal to the target execution time threshold, determine a frame loss prediction result representing a low risk of frame loss.

[0136] In some embodiments, the target task runs on a target processor cluster among multiple processor clusters in the system; the frame loss prediction device also includes: an adjustment module; the adjustment module is used to calculate a first load corresponding to the target processor cluster and a second load corresponding to a non-target processor cluster among the multiple processor clusters after determining the frame loss prediction result; and adjust the operating frequencies of the multiple processor clusters according to the frame loss prediction result, the first load and the second load.

[0137] In some embodiments, the adjustment module is also used to calculate the load corresponding to each processor in the target processor cluster as the first load based on at least one operating frequency at which each processor in the target processor cluster runs within a frame time and the time it runs at each operating frequency of the at least one operating frequency, as well as the maximum operating frequency and maximum computing power corresponding to the target processor cluster; and calculate the load corresponding to each processor in the non-target processor cluster as the second load based on at least one operating frequency at which each processor in the non-target processor cluster runs within a frame time and the time it runs at each operating frequency of the at least one operating frequency, as well as the maximum operating frequency and maximum computing power corresponding to the non-target processor cluster.

[0138] In some embodiments, the adjustment module is also used to determine a first operating frequency range corresponding to the target processor cluster based on the first load, the preset processor highest frequency point and the preset processor maximum computing power when the frame loss prediction result indicates a high risk of frame loss; adjust the operating frequency of the processors in the target processor cluster based on the first operating frequency range; perform load prediction on the non-target processor cluster through a load prediction algorithm to determine a non-target predicted load; determine a second operating frequency range corresponding to the non-target processor cluster based on the second load and the non-target predicted load; and adjust the operating frequency of the processors in the non-target processor cluster based on the second operating frequency range.

[0139] In some embodiments, the second load includes the load corresponding to each processor in the non-target processor cluster; the non-target predicted load includes the predicted load corresponding to each processor in the non-target processor cluster; the adjustment module is further used to determine the load error range based on the second load and the non-target predicted load; and determine the second operating frequency range based on the second load and the load error range.

[0140] In some embodiments, the adjustment module is also used to maintain the operating frequency of the processors in the non-target processor cluster within the second operating frequency range after determining the second operating frequency range corresponding to the non-target processor cluster, when the frame loss prediction result changes from a high risk of frame loss to a low risk of frame loss.

[0141] In some embodiments, the adjustment module is also used to, when the frame loss prediction result indicates a low risk of frame loss, perform load prediction on the target processor cluster through a load prediction algorithm to determine a target predicted load; determine a first operating frequency range corresponding to the target processor cluster based on the first load and the target predicted load; and adjust the operating frequency of the processor in the target processor cluster based on the first operating frequency range.

[0142] It should be noted that the description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present invention, please refer to the description of the method embodiment of the present invention for understanding.

[0143] In practical applications, the above-mentioned frame loss prediction device can be an electronic device that implements the frame loss prediction function, or a chip applied to an electronic device. In the present application, the device can implement the functions of multiple units through software, hardware, or a combination of software and hardware, so that the device can execute the frame loss prediction method provided in any of the above embodiments. And the technical effects of each technical solution of the device can refer to the technical effects of the corresponding technical solutions in the frame loss prediction method, and this application will not repeat them one by one.

[0144] Based on the hardware implementation of each unit in the above frame loss prediction device, the embodiment of the present application also provides an electronic device, such as Fig.10 As shown, the electronic device 90 includes: a processor 901 and a memory 902 configured to store a computer program that can be run on the processor;

[0145] The processor 901 is configured to execute the method steps in the aforementioned embodiment when running a computer program.

[0146] Of course, in practical applications, Fig.10 As shown, the various components in the electronic device 90 are coupled together through a bus system 903. It can be understood that the bus system 903 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 903 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are marked as bus system 903 in the figure.

[0147] In practical applications, the processor may be at least one of an application-specific integrated circuit (ASIC), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a controller, a microcontroller, and a microprocessor. It is understandable that for different devices, the electronic device used to implement the functions of the processor may also be other, and the embodiments of the present application do not specifically limit this.

[0148] The above-mentioned memory can be a volatile memory (volatile memory), such as a random access memory (RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD); or a combination of the above-mentioned types of memory, and provide instructions and data to the processor.

[0149] Optionally, the electronic device 90 may be a chip, and the chip may further include an input interface, wherein the processor may control the input interface to communicate with other devices or chips, and specifically, may obtain information or data sent by other devices or chips.

[0150] Optionally, the chip may further include an output interface, wherein the processor may control the output interface to communicate with other devices or chips, and specifically, may output information or data to other devices or chips.

[0151] In an exemplary embodiment, the embodiment of the present application further provides a computer-readable storage medium, such as a memory including a computer program, and the computer program can be executed by a processor of an electronic device to complete the steps of the aforementioned method.

[0152] An embodiment of the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of any one of the methods described in the embodiments of the present application are implemented.

[0153] Optionally, the computer program product can be applied to the electronic device in the embodiments of the present application, and the computer program instructions enable the computer to execute the corresponding processes implemented by the electronic device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0154] The present application also provides a computer program. Optionally, the computer program can be applied to the electronic device in the present application embodiment. When the computer program is run on a computer, the computer executes the corresponding process implemented by the electronic device in each method of the present application embodiment. For the sake of brevity, it will not be repeated here.

[0155] It should be understood that in the embodiments of the present application, related data such as user information is involved. When the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0156] It should be understood that the terms used in the present application are only for the purpose of describing specific embodiments, but are not intended to limit the present application. The singular forms of "a", "said" and "the" used in the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in this article refers to and includes any or all possible combinations of one or more associated listed items. The expressions "having", "may have", "include" and "include", or "may include" and "may include" in this application can be used to indicate the presence of corresponding features (e.g., elements such as numerical values, functions, operations or components), but the presence of additional features is not excluded.

[0157] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other and are not necessarily used to describe a specific order or sequence. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information.

[0158] The technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0159] In the several embodiments provided in the present application, it should be understood that the disclosed methods, devices and equipment can be implemented in other ways. The embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0160] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0161] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0162] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A frame loss prediction method, characterized in that: The method comprises: In the case of starting the current frame drawing, determining the previous state data corresponding to the target task in the previous frame; the target task represents the task related to the frame loss in the frame drawing tasks; Performing frame loss prediction using a frame loss prediction model and previous state data to determine a frame loss prediction result corresponding to the current frame; The frame loss prediction model is obtained by statistically analyzing at least one historical state data set of the target task in at least one historical statistical window and fitting according to the at least one state data set; each of the at least one statistical window includes at least one frame.

2. The method according to claim 1, characterized in that Before performing frame loss prediction using the frame loss prediction model and the previous state data to determine the frame loss prediction result corresponding to the current frame, the method further includes: Counting at least one historical status data set of the target task in at least one historical statistical window; the at least one status data set corresponds to the at least one historical statistical window; A frame loss prediction model is obtained by fitting according to the at least one historical status data set.

3. The method according to claim 2, characterized in that The fitting of the frame loss prediction model according to the at least one historical state data set includes: Determining at least one weight corresponding to the at least one historical statistical window according to the time sequence of the at least one historical statistical window; The frame loss prediction model is obtained by fitting according to the at least one weight and the at least one historical state data set.

4. The method according to claim 2 or 3, characterized in that: The historical state data set includes: the historical execution time of the target task in each frame of at least one frame in the historical statistical window; after the frame loss prediction model is obtained by fitting, the method further includes: According to at least one historical execution time of each historical state data set in the at least one historical state data set, the target execution time threshold corresponding to the frame loss prediction model is determined; the target execution time threshold is determined by sorting the historical execution time of at least one historical state data set and obtaining the historical execution time at a preset position.

5. The method according to claim 4, characterized in that The previous state data includes: the execution time of the target task in the previous frame; the frame loss prediction is performed using the frame loss prediction model and the previous state data to determine the frame loss prediction result corresponding to the current frame, including: When the execution time in the previous frame is greater than or equal to the target execution time threshold, determining a frame loss prediction result indicating a high risk of frame loss; When the execution time in the previous frame is equal to the target execution time threshold, a frame loss prediction result indicating a low risk of frame loss is determined.

6. The method according to any one of claims 1 to 3 or claim 5, characterized in that: The target task is run on a target processor cluster of multiple processor clusters in the system; after determining the frame loss prediction result, the method further includes: Calculating a first load corresponding to the target processor cluster and a second load corresponding to a non-target processor cluster among the multiple processor clusters; The operating frequencies of the multiple processor clusters are adjusted according to the frame loss prediction result, the first load, and the second load.

7. The method according to claim 6, characterized in that The calculating the first load corresponding to the target processor cluster and the second load corresponding to the non-target processor clusters among the multiple processor clusters includes: Calculate the load corresponding to each processor in the target processor cluster as the first load according to at least one operating frequency at which each processor in the target processor cluster runs within a frame time, the time at which each processor runs at the at least one operating frequency, and the maximum operating frequency and maximum computing power corresponding to the target processor cluster; According to at least one operating frequency at which each processor in the non-target processor cluster runs within a frame time and the time it runs at each operating frequency of the at least one operating frequency, as well as the maximum operating frequency and maximum computing power corresponding to the non-target processor cluster, the load corresponding to each processor in the non-target processor cluster is calculated as the second load.

8. The method according to claim 6, characterized in that The adjusting the operating frequencies of the multiple processor clusters according to the frame loss prediction result, the first load, and the second load includes: When the frame loss prediction result indicates a high risk of frame loss, determining a first operating frequency range corresponding to the target processor cluster according to the first load, the highest frequency point of the preset processor, and the maximum computing power of the preset processor; adjusting the operating frequency of the processors in the target processor cluster according to the first operating frequency range; Performing load prediction on the non-target processor cluster by using a load prediction algorithm to determine a non-target predicted load; Determining a second operating frequency range corresponding to the non-target processor cluster according to the second load and the non-target predicted load; The operating frequencies of the processors in the non-target processor cluster are adjusted according to the second operating frequency range.

9. The method according to claim 8, characterized in that The second load includes a load corresponding to each processor in the non-target processor cluster; the non-target predicted load includes a predicted load corresponding to each processor in the non-target processor cluster; and determining a second operating frequency range corresponding to the non-target processor cluster according to the second load and the non-target predicted load includes: determining a load error range according to the second load and the non-target predicted load; The second operating frequency range is determined according to the second load and the load error range.

10. The method according to claim 8 or 9, characterized in that: After determining the second operating frequency range corresponding to the non-target processor cluster, the method further includes: When the frame loss prediction result changes from a high risk of frame loss to a low risk of frame loss, the operating frequency of the processors in the non-target processor cluster is maintained within the second operating frequency range.

11. The method according to claim 10, characterized in that The adjusting the operating frequencies of the multiple processor clusters according to the frame loss prediction result, the first load, and the second load includes: When the frame loss prediction result indicates a low risk of frame loss, performing load prediction on the target processor cluster by using a load prediction algorithm to determine a target predicted load; Determining a first operating frequency range corresponding to the target processor cluster according to the first load and the target predicted load; The operating frequencies of the processors in the target processor cluster are adjusted according to the first operating frequency range.

12. A frame loss prediction device, characterized in that: The device comprises: A determination module, used to determine the previous state data corresponding to the target task in the previous frame when starting the current frame drawing; the target task represents the task related to the frame loss in the frame drawing tasks; A prediction module is used to perform frame loss prediction using a frame loss prediction model and previous state data to determine a frame loss prediction result corresponding to the current frame; wherein the frame loss prediction model is obtained by counting at least one historical state data set of the target task in at least one historical statistical window and fitting according to the at least one state data set; each of the at least one statistical window includes at least one frame.

13. An electronic device, characterized in that: include: A memory and a processor; wherein, The memory is used to store executable instructions. The processor is used to implement the method according to any one of claims 1-11 when executing the executable instructions stored in the memory.

14. A computer-readable storage medium, characterized in that: Executable instructions are stored, which are used to cause a processor to execute and implement the method according to any one of claims 1 to 11.

15. A computer program product, characterized in that The method comprises a computer program or an instruction, which, when executed by a processor, implements the method according to any one of claims 1 to 11.

Images