Frame loss determination method and related apparatus

Abstract

The application provides a frame loss determination method and related device. The method includes: analyzing frame rate data of a played video to determine a normal frame interval corresponding interval range of the frame rate data and a normal frame rate of the video; in a case where a frame interval between a current frame image and a previous frame image exceeds the interval range, determining a frame drawing capability required by the electronic device to maintain the normal frame rate based on a frame interval between each piece of frame rate data and a previous piece of frame rate data; and in a case where the frame drawing capability required by the electronic device exceeds the interval range corresponding to the normal frame interval, determining that a frame loss phenomenon occurs in the video progress. By implementing the method, the electronic device can determine the frame loss phenomenon in the video progress in a timely manner, and intervene in the frame loss phenomenon in a timely manner before the video progress produces a significant freezing phenomenon due to the frame loss, thereby reducing the frame loss phenomenon in the video progress and improving user experience.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a method and apparatus for determining frame loss. Background Technology

[0002] In daily use, users frequently play videos on electronic devices such as desktop computers and laptops. Currently, some electronic devices reduce power consumption by sacrificing some performance. However, during video playback, if the performance provided is insufficient to meet the demands of the device's current operating scenario, frame drops may occur, resulting in stuttering and lag, leading to a poor user experience.

[0003] Current electronic devices cannot detect frame drops in video in a timely manner, and therefore cannot intervene promptly to reduce their occurrence. Therefore, how to promptly identify the occurrence of frame drops is an urgent problem to be solved. Summary of the Invention

[0004] The purpose of this application is to provide a method and related apparatus for determining frame drops. By implementing this method, electronic devices can promptly detect frame drops in a video process and intervene before the video process experiences significant stuttering due to frame drops, thereby reducing frame drops and improving the user experience.

[0005] The aforementioned and other objectives will be achieved through the features described in the independent claims. Further implementations are illustrated in the dependent claims, the specification, and the drawings.

[0006] In a first aspect, this application provides a method for determining dropped frames, comprising: determining a target number of frames, a maximum frame interval, and a minimum frame interval based on multiple frame rate data of a first video, wherein the target number of frames is the average number of images displayed per second when an electronic device plays the first video, and the minimum frame interval and the maximum frame interval are used to determine whether the frame interval between two adjacent frame data is abnormal; if the frame interval between the first frame rate data and the second frame rate data is greater than the maximum frame interval, determining the target frame interval corresponding to each frame rate data after the second frame rate data, wherein the target frame interval is based on the target number of frames and each frame rate data and the maximum frame interval. The frame interval of the second frame rate data is determined; the first frame rate data is the frame rate data obtained after acquiring the multiple frame rate data, and the second frame rate data is the preceding frame rate data adjacent to the first frame rate data; if the frame interval between the third frame rate data and the fourth frame rate data is greater than the target frame interval corresponding to the fourth frame rate data, and the target frame interval corresponding to the third frame rate data is less than the minimum frame interval, the first video frame drop is determined, and the fourth frame rate data is the frame rate data obtained after acquiring the multiple frame rate data, and the fourth frame rate data is the preceding frame rate data adjacent to the third frame rate data.

[0007] In this method, the multiple frame rate data are the frame rate data corresponding to multiple consecutive frames of images in the first video. Each frame rate data contains a corresponding frame generation timestamp. The frame generation timestamp of any frame rate data can be approximated as the display time of the image corresponding to that frame rate data on the screen of the electronic device. The interval between the frame generation timestamps of any two adjacent frame rate data (i.e., the frame interval between these two frame rate data) can be approximated as the interval between the display times of the images corresponding to these two frame rate data on the screen of the electronic device.

[0008] Understandably, after acquiring the multiple frame rate data, the electronic device can calculate the frame interval between every two adjacent frame rate data points. These frame intervals are highly likely to be a set of fluctuating values. The electronic device can use the average of the frame intervals between every two adjacent frame rate data points as the target frame number (the target frame number can be considered as the frame rate of the first video when the electronic device plays the first video normally). Based on statistical principles, it can use the standard deviation and the target frame interval to calculate the normal fluctuation range of the frame interval values ​​of adjacent frame rate data points. The upper limit of this normal fluctuation range is the maximum frame interval, and the lower limit is the minimum frame interval.

[0009] If the frame interval between the frame rate data obtained after the multiple frame rate data and the previous frame rate data does not fall within the normal fluctuation range, the electronic device can determine that the playback of the first video may be abnormal. Therefore, when the frame interval between the first frame rate data and the previous frame rate data is greater than the maximum frame interval, the electronic device can determine that the video played in the first process may experience frame drops later.

[0010] In reality, the most important criterion for an electronic device to determine whether video playback is experiencing frame drops lies in its ability to maintain the video's frame rate at the target frame rate. In other words, does the electronic device have the capability to generate a frame image with the same frame rate as the target integer within 1 second? Therefore, the fact that the frame interval between the first and second frame rate data is greater than the maximum frame interval is only the primary condition for the electronic device to trigger a judgment on whether the video is experiencing frame drops. Subsequent processes also require determining whether the electronic device has the capability to maintain the frame rate at the target frame rate, i.e., whether the electronic device can continuously output f for 1 second. tgt Frame image.

[0011] Here, we assume the target frame count is f. tgt If, after obtaining the first frame rate and discovering that the frame interval between the first frame rate data and the second frame rate data is greater than the maximum frame interval, the electronic device can assume that it can still maintain a frame rate of the target frame number f. tgt Furthermore, the electronic device can use the image corresponding to the first frame rate data as the first frame image output within a 1-second duration. Therefore, subtracting the time spent displaying the first frame image from 1 second gives the time used to display the subsequent frames (f...). tgt -1) The available time for the corresponding image in the frame data; similarly, subtract the time used for the first and second frames from 1 second to get the time used to display the subsequent (f) frames. tgt -2) The available time for the image corresponding to each frame rate data point, and so on. Specifically, the ratio of the subsequent available time to the remaining number of frames for each frame rate data point is the target frame interval corresponding to that frame rate data point.

[0012] Understandably, for the first frame rate data and the frame rate data acquired after the first frame rate data, the smaller the target frame interval corresponding to the frame rate data, the more difficult it is for the electronic device to maintain the frame rate at the target frame rate. Therefore, in this method, if there is a third frame rate data, and the preceding frame rate data adjacent to the third frame rate data is the fourth frame rate data, and if the frame interval between the third frame rate data and the fourth frame rate data is greater than the target frame interval corresponding to the fourth frame rate data, and the frame interval corresponding to the third frame rate data is less than the minimum frame interval, then the electronic device can determine that the current frame rendering capability (i.e., the current performance supply level of the electronic device) cannot maintain the frame rate of the first video played in the first process at the value corresponding to the target integer, and the electronic device can determine that the playback of the first video will experience frame drops. By implementing this method, the electronic device can promptly detect frame drops in the video process and intervene in the frame drops in time before the video process experiences obvious stuttering due to frame drops, thereby reducing frame drops in the video process and improving the user experience.

[0013] In conjunction with the first aspect, in one possible implementation, before determining the target number of frames, the maximum frame interval, and the minimum frame interval of the first video based on multiple frame rate data of the first video, the method includes: acquiring multiple frame rate data of the first video when the maximum available power supplied by the CPU of the electronic device is greater than a first threshold.

[0014] To ensure that the target frame count, maximum frame interval, and minimum frame interval calculated by the electronic device based on the multiple frame rate data accurately reflect the actual frame interval during normal playback of the first video, the electronic device needs to guarantee that there are no dropped frames during the playback of the first video while the multiple frame rate data are being generated. Therefore, the electronic device can complete the acquisition of the multiple frame rate data under high-performance operating conditions. For example, the electronic device can perform the aforementioned operation to acquire the multiple frame rate data when the CPU's maximum power supply is sufficiently high; understandably, when the CPU's maximum power supply is sufficiently high, the electronic device's performance is adequate, and frame drops are less likely to occur during the playback of the first video.

[0015] In conjunction with the first aspect, in one possible implementation, before determining the target number of frames, the maximum frame interval, and the minimum frame interval based on multiple frame rate data of the first video, the method further includes: determining at least one process playing the video from multiple processes, and obtaining the process number of each process in the at least one process playing the video, wherein the at least one process playing the video includes a first process, and the first process is the process playing the first video; obtaining multiple instruction information of the first process based on the process number of the first process, wherein any one of the multiple instruction information contains a cache number storing the arbitrary instruction information; and using multiple consecutive instruction information in the instruction information whose cache numbers are all the first cache number as multiple frame data of the first video.

[0016] It should be understood that electronic devices may process multiple video playback processes simultaneously, and multiple videos may be playing simultaneously within the same process. However, in electronic devices, the frame rate data corresponding to images in different videos (even if multiple videos are playing the same video) are stored in different image processing caches. Therefore, in this embodiment, the electronic device can first filter the process IDs of the video playback processes. After obtaining the ID belonging to the first process, it uses the process ID to obtain all frame rate data (instruction information) of the first process, and determines the frame rate data with the same cache ID as the frame rate data of the same video according to the image cache ID contained in the frame rate data. Then, the electronic device can use multiple consecutive frame rate data with the same image cache ID as the first cache ID as the aforementioned first frame rate data. Further, after obtaining the multiple frame rate data, the electronic device can arrange the frame rate data according to the order of the timestamps in the frame rate data to facilitate the subsequent calculation of the frame interval between adjacent frame rate data.

[0017] In conjunction with the first aspect, in one possible implementation, after determining the first video frame loss, the method further includes: increasing the maximum available power of the CPU or the maximum operating frequency of the CPU; and / or increasing the operating power of the GPU or the operating frequency of the GPU.

[0018] Understandably, if an electronic device lacks sufficient frame rendering capabilities, and without further intervention, it will skip scenes that should be displayed and not show them on the screen to prevent the video from becoming out of sync with the audio during playback. While this maintains synchronization between the video and audio, users may still notice that the video is discontinuous. If the frame drop rate is too high, users may even experience noticeable stuttering during video playback.

[0019] Therefore, in this embodiment, when the electronic device's frame rendering capability is insufficient, the electronic device can increase its performance supply to reduce or avoid frame dropping during subsequent video playback.

[0020] Optionally, the electronic device can increase the CPU's maximum available power PL1 (Package Long-duration Power Limit) or maximum operating frequency (Max Frequency); alternatively, the electronic device can also adjust the maximum duration for which the CPU power briefly exceeds the power limit (i.e., the CPU's maximum available power) to improve the CPU's command processing performance.

[0021] Optionally, the electronic device increases the maximum available power or maximum operating frequency of the CPU by a margin greater than the second threshold.

[0022] Optionally, the electronic devices can also increase the GPU's maximum power or maximum frequency to improve the GPU's rendering performance.

[0023] Optionally, the electronic device can also adjust the time rate that the CPU can allocate on Draw Calls to improve command processing performance.

[0024] Optionally, for images with the same rendering state, the electronic device can package the instruction information corresponding to multiple images with the same rendering state into a single instruction. Then, the electronic device only needs to call the Draw Call to execute one instruction.

[0025] Optionally, the electronic device may also choose other methods to increase the supply of device performance, such as increasing the speed of CPU write commands and / or increasing the rendering speed of GPU, which is not limited in this application.

[0026] In conjunction with the first aspect, in one possible implementation, after determining that the first video has lost frames, the method further includes: stopping the acquisition of frame rate data of the first video; after a first duration, continuing to acquire frame rate data of the first video; and determining whether the first video has lost frames based on the frame rate data of the first video acquired after the first duration.

[0027] Since adjusting the performance supply level of electronic devices takes time, in one optional implementation, after the electronic device reports the determination that frame dropping will occur in the video to the system, the electronic device can pause acquiring the frame rate data of the currently playing video and stop calculating the frame rendering capability required for each video. After several seconds (e.g., five seconds), the electronic device can then resume acquiring the frame rate data of the currently playing video and continue calculating the frame rendering capability required for each video using the aforementioned method.

[0028] Optionally, the electronic device may not update the target frame count, target frame interval, maximum frame interval, and minimum frame interval. Instead, it can set the target frame interval corresponding to the first frame rate data acquired after 5 seconds as the maximum frame interval. If the frame interval between a subsequent frame rate data and the previous frame rate data exceeds the maximum frame interval, it can calculate the remaining frame count and target frame interval for each frame rate data using the frame intervals of adjacent frame rate data, thereby determining if the video is experiencing frame drops. Alternatively, after the performance supply level increases, the electronic device can reselect multiple frame data from the first video acquired after 5 seconds to update the target frame count, target frame interval, maximum frame interval, and minimum frame interval. Afterward, the electronic device can continue to acquire frame rate data from the first video and determine whether the first video continues to experience frame drops according to the aforementioned operations. In this way, even in an operating environment where the performance supply level is constantly changing, the electronic device can still promptly determine if frame drops will occur during video playback and intervene in the frame drops before the video process experiences significant stuttering due to frame drops, reducing frame drops and improving the user experience.

[0029] In conjunction with the first aspect, in one possible implementation, the display process of the multiple frames of images corresponding to the multiple frame rate data on the screen of the electronic device is not interrupted.

[0030] Furthermore, the acquisition of the aforementioned multiple frame rate data takes a certain amount of time. If the display of the image in the first video on the screen is interrupted during the acquisition process, the frame interval between the last frame rate data before the interruption and the first frame rate data after the resumption of display may be particularly long. This may cause the target frame count, maximum frame interval, and minimum frame interval calculated based on the multiple frame rate data to not accurately reflect the frame rate situation when the first video is playing normally, which may further affect the accuracy of the frame loss result judgment. Therefore, in this embodiment, if the display of the image in the first video on the screen is interrupted during the acquisition of the aforementioned multiple frame rate data, the electronic device needs to stop acquiring the frame rate data and discard the previously acquired frame rate data. After the image of the first video continues to be displayed continuously on the screen, the acquisition of the multiple frame rate data of the first video will resume. It should be noted that "the display of the image on the screen is interrupted" can include the electronic device stopping the playback of the first video, and the situation where the playback window of the first video on the screen is completely obscured by other windows. For example, if the amount of the multiple frame rate data is 60, the electronic device may need to take 2-3 seconds to successfully collect the 60 frame rate data corresponding to the 60 frames in the first video. If the electronic device only collects the above 30 frame rate data of 30 frames in the first video and the user pauses the first video, the electronic device can stop collecting the frame rate data of the first video and re-collect the 60 frame rate data corresponding to the above 60 frames after the user continues to play the first video (excluding the above 30 frame rate data collected previously).

[0031] In conjunction with the first aspect, in one possible implementation, the target number of frames is determined based on the average frame interval of the first video, the maximum frame interval and the minimum frame interval are determined based on the frame interval standard deviation, the average frame interval is the average of the frame intervals between all adjacent frame rate data in the plurality of frame rate data, and the frame interval standard deviation is the standard deviation of the frame intervals between all adjacent frame rate data in the plurality of frame rate data.

[0032] Here, it is assumed that the number of frame rate data points is N or more. The average frame interval (IFS) calculated based on these frame rate data points is then... avg for:

[0033]

[0034] Among them, IFS a，b This represents the time interval between the frame generation timestamp of frame rate data a and the frame generation timestamp of frame rate data b. Frame rate data a and frame rate data b are two adjacent frame rate data.

[0035] The electronic device can calculate the frame rate when sending the first video in the first process to the display, that is, the number of images that the electronic device needs to draw per second, which is also the target frame number f. tgt Its calculation method can be expressed as:

[0036]

[0037] in, This indicates the floor function. This represents the largest integer less than X.

[0038] It's easy to understand that the frame rate of the first video during normal playback is f. tgt The first video's playback frame rate is no less than f tgt In this case, the video that the user observes is basically smooth, so as not to affect the user's viewing experience.

[0039] In addition, based on IFS avg The standard deviation (IFS) of the frame intervals of the multiple frame rate data can be calculated. std Its calculation method can be expressed as:

[0040]

[0041] Furthermore, based on the aforementioned IFS std The maximum frame interval (IFS) can be calculated. max and minimum frame interval (IFS) min Its calculation method can be expressed as:

[0042] IFS min =IFS avg -M×IFS std ;

[0043] IFS max =IFS avg +M×IFS std ;

[0044] Where M can be any value from 1 to 10, and "×" represents multiplication; specifically, the value of M can be 3.

[0045] As is understandable, standard deviation is a quantitative indicator used to characterize the degree to which each data point in a statistically analyzed group deviates from the mean. Mathematically, the M-times-standard-deviation method is a commonly used data processing method for identifying outliers. It involves two concepts: determining the "allowable range" for each sample based on the size of the population sample standard deviation; and identifying sample values ​​that exceed the "allowable range" as outliers. The size of the population sample standard deviation is related to the definition of outliers. Generally, three times the sample standard deviation is considered the allowable deviation range for the sample; that is, if a sample exceeds three times the standard deviation, that sample is considered an outlier.

[0046] In a second aspect, embodiments of this application provide an electronic device, the electronic device comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, and the one or more processors invoking the computer instructions to cause the electronic device to perform the method in the first aspect or any possible implementation of the first aspect.

[0047] Thirdly, a chip system is provided, the chip system being applied to an electronic device, the chip system including one or more processors, the processors being configured to invoke computer instructions to cause the electronic device to perform a method as described in the first aspect or any possible implementation thereof.

[0048] Fourthly, a computer program product containing instructions, when run on an electronic device, causes the electronic device to perform the method as described in the first aspect or any possible implementation thereof.

[0049] Fifthly, a computer-readable storage medium is provided, including instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in the first aspect or any possible implementation thereof. Attached Figure Description

[0050] Figure 1 A flowchart illustrating a frame loss determination method provided in an embodiment of this application;

[0051] Figure 2 A schematic diagram illustrating a process for filtering a currently playing video, provided as an embodiment of this application;

[0052] Figure 3 An architecture diagram of an ETW system provided in this application embodiment;

[0053] Figure 4 This application provides a schematic diagram illustrating a process for determining frame rate parameters based on frame rate data.

[0054] Figure 5 A comparison diagram of image display time intervals under different playback states provided in an embodiment of this application;

[0055] Figure 6 A schematic diagram illustrating a frame-by-frame calculation process for determining the rendering capability of a target frame, provided as an embodiment of this application;

[0056] Figure 7 This application provides a schematic diagram of a scenario where the CPU and GPU work together.

[0057] Figure 8 This application provides a structural diagram of an electronic device. Detailed Implementation

[0058] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0059] To facilitate understanding, the relevant terms involved in the embodiments of this application will be introduced below.

[0060] (1) Performance and power consumption of electronic devices

[0061] Performance refers to the properties and functions of a substance. For electronic devices such as computers and tablets, the most important evaluation indicator is the processing speed, which is the number of instructions the electronic device can execute per second, usually described as "millions of instructions per second". Understandably, the processing speed of an electronic device is further reflected in whether the device runs smoothly and whether there are any noticeable lags; the higher the performance of the electronic device, the less frequent the lags, and vice versa.

[0062] Power consumption refers to the amount of energy consumed by an electronic device per unit of time. Specifically, for electronic devices, power consumption refers to the electrical energy consumed per unit of time, measured in watts (W). Even in standby mode, electronic devices still consume a certain amount of electrical energy (unless the power is cut off).

[0063] To balance performance and power consumption, some electronic devices can adaptively adjust their performance limits based on the performance demands of the current operating scenario, sacrificing some performance to reduce power consumption. When an electronic device experiences noticeable lag, it indicates that its current performance limit is insufficient to meet the performance requirements of the current operating scenario. Therefore, the device can adjust its performance limit to reduce the frequency of lag. Specifically, for desktop and laptop computers, since the central processing unit (CPU) is the core of the system's computation and control, the CPU's maximum available power and maximum operating frequency determine the device's performance limit. Taking the maximum available CPU power as an example, generally speaking, for microprocessors of the same series, a higher maximum available CPU power generally indicates a better performance limit for the electronic device.

[0064] Therefore, for the electronic device provided in this application, when a stuttering phenomenon (such as playback stuttering caused by dropped video frames) is determined to occur during the operation of the electronic device, the electronic device can appropriately increase the maximum CPU power supply and / or the maximum CPU operating frequency to improve the performance of the electronic device, thereby reducing the frequency of stuttering and improving the user experience.

[0065] (2) Frame dropping

[0066] Frame dropping, also known as frame skipping, occurs when one or more frames are skipped and not displayed on the screen of an electronic device during video playback. Videos played by electronic devices are composed of many rapidly displayed images, each of which is called a frame.

[0067] Frame dropping is common in video playback, such as online videos and real-time video transmissions. Generally, the time an image from a video remains in the human brain is about 0.1 to 0.4 seconds. Most videos currently operate at a frame rate of 18-24 frames per second, meaning 18-24 frames are played per second to create the visual effect of continuous visuals. When frame dropping is not severe—for example, if only one frame out of 18 images played per second is skipped—the user may not notice it. However, if frame dropping is more severe, such as if two or three consecutive frames are skipped within a second, the user will clearly perceive a discontinuity in the screen.

[0068] (3) Frame rate data

[0069] For the processes that electronic devices need to process, the producers in the system will generate several corresponding instruction messages (events) for the processes so that the consumers in the system can obtain and execute these instructions.

[0070] In this application, for any process that is playing a video, there is a producer in the system (such as DirectX-Graphics-Infrastructure (DXGI) or Direct-3D (D3D)) that generates image processing instructions for the process. These image processing instructions can be obtained by the electronic device from its own cache or log file. In this application, these image processing instructions can be referred to as "frame rate data".

[0071] Specifically, since video playback involves electronic devices sequentially drawing and displaying images frame by frame on the screen, each frame in a playing video corresponds to a frame rate data entry. Each frame rate data entry contains a process ID, a frame ID, a frame generation timestamp, and an image processing buffer ID. Frame rate data with the same process ID and image processing buffer ID belong to the same video file. For the same video file, the frame rate data corresponding to an earlier-generated frame will be displayed on the screen earlier by the electronic device.

[0072] In this application, the timestamp contained in the frame rate data can be approximated as the time when the image corresponding to the frame rate data is displayed on the screen.

[0073] (4) Frame interval

[0074] A video file contains multiple frames, each with a corresponding timestamp. During video playback, the order in which these frames are displayed on the screen is determined by their respective timestamps—images with earlier timestamps are displayed before those with later timestamps. The interval between the frame generation timestamps of any two frames can be called the interframe space (IFS) of these two frames. Correspondingly, the time difference between the display of any two frames on the screen can be approximated as the interframe space between their corresponding frame rate data.

[0075] In this application, the average frame interval between any two adjacent frames in a video, corresponding to multiple consecutive images, can be referred to as the average frame interval (IFS) of these multiple frames. avg .

[0076] Furthermore, based on the average frame interval of these multiple frame rate data points, the standard deviation of the frame interval between adjacent frame rate data points (IFS) can be calculated.std Furthermore, based on the frame interval standard deviation (IFS) of adjacent frame rate data, the estimated maximum value (IFS) of the frame interval between subsequent adjacent frame rate data can be calculated. max And the estimated minimum value IFS min ,in:

[0077] IFS min =IFS avg -3×IFS std ;

[0078] IFS max =IFS avg +3×IFS std ;

[0079] Understandably, in statistics, for a set of values ​​exhibiting linear fluctuations, the three-standard-deviation method can be used to identify values ​​with significant differences (i.e., large fluctuations). However, in this application, in adjacent displayed images, the frame intervals of the corresponding frame rate data for two sets of adjacent images (e.g., the first frame and the second image, and the second frame and the third frame) may also be different. Therefore, the frame intervals between the corresponding frame rate data of these images can also be considered as a set of linearly fluctuating values. The estimated highest value (IFS) of the frame intervals of the aforementioned multiple frame rate data is calculated. max And the estimated minimum value IFS min Subsequently, if two adjacent images are displayed, their frame interval must be greater than IFS. max Or less than IFS min This indicates that the frame interval of the corresponding frame rate data of these two images is significantly different from the frame interval of the previous frame rate data, which also indicates that the video file may have encountered an anomaly during subsequent playback.

[0080] (5) Performance Counter

[0081] Performance counters, also known as performance monitors, are system functions provided by the operating system. They collect and analyze performance data related to applications, services, and drivers in real time to identify system bottlenecks, monitor component performance, and ultimately help users optimize system resources. Performance counters can monitor multiple performance objects. Typical performance objects in a system can include processors, processes, threads, network communications (such as TCP, UDP, ICMP, IP, etc.), and system services (such as ACS / RSVP services).

[0082] Specifically, the performance counters include a graphics processing unit (GPU) engine counter, which contains a utilization percentage module. For each process, this module monitors the count values ​​of the video decoding module and the video processing module in real time. When both of these count values ​​are greater than 0, it indicates that the corresponding process is currently playing video.

[0083] In daily use, users often play videos on electronic devices such as desktop computers and laptops. However, when the performance provided by the device is insufficient to meet the performance requirements of the current operating scenario, the video may experience frame drops, resulting in stuttering and lag.

[0084] Current electronic devices have difficulty uniformly monitoring frame drops at the operating system level and cannot promptly identify frame drops in video. As a result, they often cannot intervene in the frame drop quadrant in a timely manner, and often only respond after the user has clearly felt the video stuttering, which greatly damages the user experience.

[0085] To address the aforementioned shortcomings, this application provides a method for determining frame drops. This method analyzes the frame rate data of the played video to determine the range corresponding to the normal frame interval. When the frame interval between a subsequent frame and the previous frame exceeds this range, the method determines the frame rendering capability required by the electronic device to maintain a normal frame rate based on the frame interval between each frame rate data point and the previous frame rate data point. When the frame rendering capability required by the electronic device exceeds the range corresponding to the normal frame interval, it is determined that frame drops exist in the video process. Implementing this method allows the electronic device to promptly detect frame drops in the video process and intervene before the video process experiences noticeable stuttering due to frame drops, thereby reducing frame drops and improving the user experience.

[0086] like Figure 1 As shown, the frame loss determination method provided in this application may include the following steps:

[0087] S101, The electronic device determines the process number of the first process that is playing the video.

[0088] The aforementioned electronic devices can be mobile phones, in-vehicle devices (such as on-board units (OBUs)), tablets, computers with data transceiver capabilities (such as laptops, PDAs, etc.), mobile internet devices (MIDs), terminals in smart cities, terminals in smart homes, terminal devices in 5G networks, or terminal devices in future evolved public land mobile networks (PLMNs), etc. It is understood that this application does not limit the specific form of the aforementioned electronic devices.

[0089] Optionally, the aforementioned electronic device may currently have multiple processes running, which may include one or more processes that are playing video and processes that are not playing video. The first process mentioned above can be any one of the one or more processes that are playing video.

[0090] Specifically, electronic devices can monitor the performance data of the video decoding and video processing modules through the utilization percentage module in the GPU engine counters of the system's performance counters. Processes whose corresponding counts for both modules are greater than 0 are identified as those currently playing video. Understandably, if a process's utilization of the video decoding and video processing modules is greater than 0, it means that the process is using these modules to perform operations such as encoding, decoding, transcoding, color space conversion, and pixel conversion on the video; therefore, this process is the one currently playing video.

[0091] Figure 2 This illustrates the process by which an electronic device, based on the utilization rates of the video decoding and video processing modules across all processes, selects the process currently playing video from all processes. For example... Figure 2 As shown, the processes currently pending on the electronic device may include Figure 2 The diagram shows multiple processes, p1, p2, p3, ..., pn, etc., and at least one of these processes is currently playing video. For any one of these processes, the electronic device can query the performance counters to determine the utilization rate of the video decoding module and the video processing module for that process. Processes with a utilization rate greater than 0 for both the video decoding and video processing modules will be identified as... Figure 2 The processes play1, play2, ..., playn are the processes currently playing the video. The first process mentioned above can be any one of the processes play1 through playn.

[0092] It should be noted that when filtering out the processes that are playing a video, the electronic device will simultaneously obtain and save the process IDs of these processes, so that the frame rate data of the corresponding processes can be obtained later based on these process IDs.

[0093] S102, the electronic device acquires N frame rate data of the first process and determines the target number of frames, target frame interval, maximum frame interval and minimum frame interval based on the N frame rate data.

[0094] The aforementioned N frame rate data points refer to the frame rate data corresponding to N consecutive frames of the first video played by the first process. Optionally, the specific value of N can be 60.

[0095] After the electronic device determines that the first process is the process playing video from among multiple processes to be processed, the electronic device can obtain the instruction information corresponding to the first process (i.e., the event with the process number of the first process) from the event tracing system (ETW). This instruction information includes instruction information related to image processing.

[0096] It should be noted that ETW is a general event tracing system provided by the operating system of electronic devices to monitor the performance of systems under load. Figure 3 The diagram shows the architecture of ETW. In the operating system 30, the entire ETW system 301 can be composed of three parts: producers (including producers 303, 304, and 305), consumers 306, and controller 302.

[0097] Controller 302 is a controller. Its main tasks are twofold: first, to manage session control objects. It is responsible for creating session control objects in memory in response to producer registration requests, and also for sending the instruction information (events) recorded in the session control objects to consumers. Second, to manage producers, such as starting or stopping them. To avoid additional overhead, producers are not always running; they only begin working after being started.

[0098] A producer, or provider of instruction information (i.e., events), can be a system component, a driver, or a developed application. Before generating instruction information, the producer needs to register a session control object with the system. The electronic device allocates a corresponding buffer area for each session control object. Once the producer is started by the controller, it can begin sending instruction information to the buffer area in the corresponding session control object. Figure 3Taking producer 303 as an example, session control object 3011 is obtained by producer 303 through registration. When controller 302 starts producer 303, producer 303 will start generating instruction information and write it into the cache of session control object 3011, waiting to be obtained and executed by consumer 306.

[0099] Consumer 306 is used to receive instruction information (events) provided by one or more session control objects. Specifically, consumer 306 can receive instruction information (events) transmitted to the log file by the session control object, or it can receive instruction information (events) from the session control object in real time. When processing instruction information (events), consumer 306 can specify the start and end times for receiving instruction information (events), and the producer can also only transmit instruction information events generated within the time range.

[0100] In this application, most of the instruction information (events) related to image processing in the aforementioned first process are generated by the DXGI or D3D modules. That is, in the ETW system, DXGI and D3D can act as producers, generating some of the instruction information (events) required to complete the aforementioned first process. The electronic device can listen to the instruction information written into the ETW system by these two modules as producers, and obtain the frame rate data corresponding to the aforementioned first process from the acquired instruction information based on the process number of the first process.

[0101] It should be understood that within the same process, multiple videos may be played simultaneously, but the frame rate data corresponding to images in different videos will be stored in different image processing caches. Therefore, after obtaining the frame rate data belonging to the first process, the electronic device also needs to determine the frame rate data with the same image cache number as the frame rate data corresponding to images in the same video, according to the image cache number contained in the frame rate data. Furthermore, after obtaining multiple (≥N) frame rate data corresponding to multiple frames of images in the same video, the electronic device can arrange the frame rate data according to the chronological order of the timestamps in the frame rate data, and then select N consecutive frame rate data as the frame rate data corresponding to the N frames of images in the first process. In other words, after determining all video playback processes and obtaining the process numbers, the electronic device can obtain the corresponding frame rate data for any binary tuple (process number, image cache number), and from the multiple frame rate data obtained for the same binary tuple (process number, image cache number), select N consecutive frame rate data as the N frame rate data for the first process according to the chronological order of the timestamps contained in each frame rate data. For ease of explanation, the following explanation will use N frame rate data corresponding to the images in the first video played by the first process as an example.

[0102] Figure 4 The example illustrates the data information contained in each of the above N frame rate data, and the process by which the electronic device determines the target number of frames, the target frame interval, the maximum frame interval, and the minimum frame interval based on the above N frame rate data.

[0103] like Figure 4 As shown in (A), any one of the N frame rate data points contains a process ID (ProcessID), a frame ID (EventID), a frame generation timestamp (Timestamp), and an image processing cache ID (pSwapchain or pIDXGISwapChain). Frame rate data generated by DXGI is stored in a cache of type pSwapchain, while frame rate data generated by D3D is stored in a cache of type pIDXGISwapChain.

[0104] Figure 4 Example (B) illustrates the key information contained in the frame rate data with frame number 1 among the above N frame rate data. For example... Figure 4 As shown in (B), the frame rate data belongs to process number "14216", which can be the first process mentioned above. The frame generation timestamp contained in the frame rate data is "133287886730547216"; it should be noted that the frame generation timestamp in the frame rate data actually represents the generation time of the frame rate data, but it can be approximated as the time when it is consumed by the producer (that is, the time when the image corresponding to the frame rate data is displayed on the display screen). In some embodiments, in order to facilitate comparison of the chronological relationship of the timestamps, Figure 4 The frame generation timestamps contained in the (B) frame rate data can also be in other formats, and this application does not limit this. The image processing cache number where the frame rate data is stored is "0x173DDFACAC0". In addition to storing the frame rate data with frame number 1, the image processing cache can also store multiple other frame rate data, which correspond to multiple frames in the same video file.

[0105] Here it is assumed that the smaller the frame number, the earlier the frame rate data is generated. Figure 4 Table (C) shows a table obtained by arranging multiple frame rate data according to the order of timestamps in the frame rate data by an electronic device. Understandably, timestamp TS1 corresponds to an earlier time than timestamp TS2, TS2 corresponds to an earlier time than timestamp TS3, and so on. The timestamps contained in these frame rate data also determine when their corresponding images are ultimately displayed on the screen; the earlier the timestamp corresponds to the frame rate data, the earlier its corresponding image will be displayed on the screen by the electronic device.

[0106] Optionally, to facilitate comparison of the chronological order of timestamps, electronic devices may also convert the frame generation timestamps contained in the frame rate data into other formats. Figure 4 Table (D) shows a table obtained after an electronic device converts timestamps to other formats and arranges multiple frame rate data according to the chronological order of the corresponding timestamps. Figure 4 Taking the frame rate data with frame number 1 in the table shown in (D) as an example, let's assume that this frame rate data is... Figure 4 The frame rate data shown in (B) with frame number 1 shows that the original timestamp of this frame rate data was "133287886730547216". However, after being converted by the electronic device, its timestamp was converted to other formats, that is... Figure 4 The "168814255.7563" shown in (D) should be understood as follows: the conversion of the timestamp format does not change the actual time corresponding to the timestamp, nor does it change the temporal relationship between the timestamp and other timestamps.

[0107] After arranging the acquired frame rate data in chronological order, the electronic device can select N consecutive frame rate data points as the N frame rate data points for the first process. It should be understood that "consecutive" here means that the images corresponding to these N frame rate data points can constitute a segment of video in the video file played by the first process, and this segment of video has no dropped frames. To avoid frame drops in the video segment corresponding to these N frame rate data points, the electronic device can complete the acquisition of the N frame rate data points under high-performance operating conditions. For example, the electronic device can perform the aforementioned operation to acquire the N frame rate data points when the CPU's maximum power supply is sufficiently high; understandably, when the CPU's maximum power supply is sufficiently high, the electronic device's performance is adequate, and frame drops are less likely to occur during video playback.

[0108] After acquiring the aforementioned N frame rate data points, the electronic device can use this data to calculate the frame rendering capability required to complete the first process described above. In other words, without dropping frames, the electronic device needs to render (or display) how many images per second on average to the screen. Specifically, for example... Figure 4 As shown in (E), the electronic device can calculate the target number of frames, average frame interval, maximum frame interval, and minimum frame interval based on the frame generation timestamps contained in the above N frame rate data, wherein:

[0109] The average frame interval mentioned above is the average frame interval (IFS) of the above N frame rate data. avg That is, the average frame interval of the above N frame rate data can be calculated as follows:

[0110]

[0111] Among them, IFS a，b This represents the time interval between the frame generation timestamp of frame rate data 'a' and the frame generation timestamp of frame rate data 'b'. Frame rate data 'a' and frame rate data 'b' are two adjacent frames, for example... Figure 4 The two frame rate data in (D) are frame number 1 and frame number 2.

[0112] It's easy to understand that, based on the aforementioned average frame interval, the electronic device can calculate the frame rate when sending the first video in the first process to the display, that is, the number of images the electronic device needs to render per second, which is also the aforementioned target frame number f. tgt Its calculation method can be expressed as:

[0113]

[0114] in, This indicates the floor function. This represents the largest integer less than X.

[0115] The target number of frames f tgt This is calculated by the electronic device based on the aforementioned N frame rate data when no frame drops occur. Therefore, it is easy to understand that the frame rate of the first video during normal playback is f. tgt The first video's playback frame rate is no less than f tgt In this case, the video that the user observes is basically smooth, so as not to affect the user's viewing experience.

[0116] Furthermore, based on the aforementioned IFS avg The standard deviation (IFS) of the frame intervals of N frame rate data points can be calculated. std Its calculation method can be expressed as:

[0117]

[0118] Furthermore, based on the aforementioned IFS std The maximum frame interval (IFS) mentioned above can be calculated. max and minimum frame interval (IFS) min Its calculation method can be expressed as:

[0119] IFSS min =IFS avg -M×IFS std

[0120] IFS max =IFS avg +M×IFS std ;

[0121] Where M can be any value from 1 to 10, and "×" represents multiplication; specifically, the value of M can be 3.

[0122] As is understandable, standard deviation is a quantitative indicator used to characterize the degree to which each data point in a statistically analyzed group deviates from the mean. Mathematically, the M-times-standard-deviation method is a commonly used data processing method for identifying outliers. It involves two concepts: determining the "allowable range" for each sample based on the size of the population sample standard deviation; and identifying sample values ​​that exceed the "allowable range" as outliers. The size of the population sample standard deviation is related to the definition of outliers. Generally, three times the sample standard deviation is considered the allowable deviation range for the sample; that is, if a sample exceeds three times the standard deviation, that sample is considered an outlier.

[0123] In this application, the frame interval of the image corresponding to each two adjacent frame rate data in the aforementioned N frame rate data may not be a fixed value, but rather a set of linearly fluctuating values. Therefore, in this application, after calculating the aforementioned target frame number, average frame interval, maximum frame interval, and minimum frame interval, the electronic device can... tgt Determine the frame rate of the video played by the first process (i.e., how many frames of images need to be displayed per second when playing the video), and set the target frame interval IFS as follows. avg The ideal (or standard) value for the time interval between each subsequent frame and the previous frame is determined, and the minimum frame interval (IFS) is defined as follows. min and maximum frame interval (IFS) max This serves as the basis for determining whether the frame interval between subsequently displayed adjacent images is abnormal. Specifically, the electronic device can use the aforementioned minimum frame interval (IFS) as a criterion. min and the aforementioned maximum frame interval (IFS) max The range formed by [IFS] min IFS max As the range of values ​​for the normal frame interval (i.e., the "allowable range" in the previous description), if the frame interval of two adjacent frames displayed subsequently does not fall within this range, it means that the frame interval of these two frames is significantly different from the frame interval between the previous adjacent frames, which means that the video playback process of the first process has become abnormal.

[0124] Understandably, if the video played in the first process is uninterrupted, DXG or D3D, as the producer, will continuously generate corresponding instruction information (events). Assume that the aforementioned N frame rate data points come from the first video played in the first process, and that the frame rate data generated by DXG or D3D based on this first video is stored in the first buffer. Then, when the number of frame rate data points generated based on the first video in the first buffer is less than N, the electronic device can temporarily disregard calculating the target frame number f. tgt Average Frame Interval (IFS) avg Maximum Frame Interval (IFS) max and minimum frame interval (IFS) min The electronic device selects N consecutive frame rate data points from the first buffer for calculation only when the number of frame rate data points generated based on the first video in the first buffer is greater than or equal to N.

[0125] In an optional implementation, the electronic device can determine the N consecutive frame rate data selected from the first buffer under high-performance supply conditions as the N frame rate data, and determine the target frame number f based on the N frame rate data. tgt Average Frame Interval (IFS) avg Maximum Frame Interval (IFS) max and minimum frame interval (IFS) min The value. Then, if the frame rate data subsequently obtained from the first cache falls within the interval [IFS] of the previous frame rate data, the frame interval between the previous frame rate data and the current frame rate data will be considered. min IFS max If the frame rate data subsequently obtained from the first buffer does not fall within the interval [IFS], the electronic device will not process it; min IFS max If the electronic device determines that an abnormality has occurred during the playback of the first video, it can then execute the subsequent step S103.

[0126] Furthermore, since electronic devices may reduce their performance output to lower power consumption (e.g., reduce the maximum power available to the CPU), this process may affect video playback in the first process. Therefore, in another optional implementation, the frame interval between the latest frame rate data stored in the first cache and the previous frame rate data is a normal frame interval (i.e., falling within [IFS]). min IFS max In the case of […], the electronic device can continuously select the most recently stored N frame rate data from the first buffer as the latest frame rate data is written to the first buffer to calculate and update the target frame number f. tgt Average Frame Interval (IFS) avg Maximum Frame Interval (IFS)max and minimum frame interval (IFS) min The value of N. For example, assuming the value of N is 60, when the electronic device is in high-performance supply mode, after DXGI or D3D stores the 60th frame rate data generated based on the first video into the first cache, the electronic device can use the 1st to 60th frame rate data in the first cache as the N frame rate data, and calculate the target frame number f based on the 1st to 60th frame rate data. tgt Average Frame Interval (IFS) avg Maximum Frame Interval (IFS) max and minimum frame interval (IFS) min The value; then, DXGI or D3D stores the 61st frame rate data generated based on the first video into the first buffer (here it is assumed that the time interval between the frame generation timestamp of the 61st frame rate data and the 60th frame rate data falls within the range calculated based on the 1st to 60th frame rate data [IFS]). min IFS max In the above-mentioned electronic device, the frame rate data from the 2nd to the 61st records in the first buffer can be used as the N frame rate data records, and the target frame number f can be calculated and updated based on the frame rate data from the 2nd to the 61st records. tgt Average Frame Interval (IFS) avg Maximum Frame Interval (IFS) max and minimum frame interval (IFS) min The value is maintained until the frame interval between the most recently written frame rate data and the previous frame rate data does not fall within the newly calculated range [IFS]. min IFS max If the electronic device determines that an abnormality has occurred during the playback of the first video, it can then stop using the N most recently stored frame rate data in the first cache to calculate and update the target frame number f. tgt Average Frame Interval (IFS) avg Maximum Frame Interval (IFS) max and minimum frame interval (IFS) min Instead of retrieving the value, the subsequent step S103 is executed.

[0127] Furthermore, the acquisition of the aforementioned N frame rate data takes a certain amount of time. Therefore, in an optional implementation, if the display of the images in the first video on the screen is interrupted during the acquisition of the aforementioned N frame rate data, the electronic device needs to stop acquiring the frame rate data and discard the previously acquired frame rate data. After the images of the first video continue to be displayed continuously on the screen, the acquisition of the N frame rate data of the first video will resume. It should be noted that "the display of the images on the screen is interrupted" can include the electronic device stopping playback of the first video, and the first video's playback window being completely obscured by other windows. For example, when N is 60, the electronic device may need 2-3 seconds to successfully acquire the 60 frame rate data corresponding to 60 frames in the first video. If the electronic device only acquires the 30 frame rate data corresponding to 30 frames in the first video, and the user pauses playback of the first video, the electronic device can stop acquiring the frame rate data of the first video and re-acquire the 60 frame rate data corresponding to the subsequent 60 frames after the user resumes playback of the first video (excluding the previously acquired 30 frame rate data).

[0128] S103. When the frame interval between the first frame rate data and the second frame rate data is greater than the maximum frame interval mentioned above, the electronic device sets the corresponding number of remaining frames for each frame rate data starting from the first frame rate data, and determines the target frame interval corresponding to each frame rate data based on the number of remaining frames corresponding to each frame rate data and the frame interval between each frame rate data and the previous frame rate data.

[0129] The first frame rate data mentioned above is the frame rate data obtained after the aforementioned N frame rate data, and it has the same process number and image processing buffer number as the aforementioned N frame rate data. The second frame rate data mentioned above is the preceding frame rate data adjacent to the second frame rate data. Specifically, the first frame rate data can be the first frame rate data acquired by the electronic device after the aforementioned N frame rate data, where the frame interval between adjacent frame rate data is greater than IFS. max Frame rate data.

[0130] Starting from the first frame rate data mentioned above, the remaining frame number f corresponding to the nth frame rate data obtained subsequently. res_n The calculation method is as follows:

[0131]

[0132] in, f represents the number of remaining frames corresponding to the previous frame rate data. res_1 That is, the number of remaining frames corresponding to the first frame rate, and f res_1 =f tgt -1.

[0133] Starting from the first frame rate data mentioned above, the target frame interval (IFS) corresponding to the nth frame rate data obtained subsequently. tgt_n The calculation method is as follows:

[0134]

[0135] Among them, IFS n To start counting from the first frame rate data mentioned above, the frame interval between the nth frame rate data and the previous frame rate data is IFS. n1 This refers to the first frame rate data mentioned above.

[0136] As explained above, when the frame interval between the subsequently acquired frame rate data and the previous frame rate data does not fall within the newly calculated interval [IFS]... min IFS max In this case, the electronic device can determine that the video playback may be malfunctioning. Therefore, when the frame interval between the first frame rate data and the previous frame rate data is greater than the currently calculated IFS... max In this case, the electronic device can determine that the video played in the first process may experience frame drops later.

[0137] It should be understood that it is not simply a matter of the frame interval between two subsequent frames of data being greater than the IFS. max At that time, the electronic device can determine that the first video being played in the first process will definitely experience frame drops; in fact, the most important basis for the electronic device to determine whether video playback is experiencing frame drops lies in whether the electronic device has the ability to maintain the video playback frame rate at f. tgt In other words, does the electronic device still have the capability to draw f within 1 second? tgt Frame image. For example, even if the frame interval between the first frame rate data and the second frame rate data is greater than IFS. max The frame rate data acquired subsequently may also contain frames with a frame interval significantly smaller than the IFS. avg or even smaller than IFS min The frame rate data, therefore even if the frame interval between the first frame rate data and the second frame rate data is greater than IFS. max Even under these circumstances, electronic devices may still be able to maintain the frame rate at the target frame rate f. tgt That is, electronic devices may still display f within 1 second. tgt At this point, the electronic device cannot determine whether frame drops will occur during the playback of the first video.

[0138] Therefore, the frame interval between the first frame rate data and the second frame rate data is greater than the currently calculated IFS. maxThis is only the primary condition for electronic devices to determine whether video frames are dropped. Subsequent processes also require determining whether the electronic device is capable of maintaining the frame rate at the target frame rate f. tgt In other words, can the electronic device maintain a continuous output of f for 1 second? tgt Frame image.

[0139] It should be noted that for frame rate data acquired after the aforementioned N frame rate data points, the smaller the corresponding target frame interval, the more likely the electronic device will maintain the frame rate at the target frame number f. tgt The greater the difficulty, the more challenging it will be.

[0140] by Figure 5 The first video shown is illustrated using a diagram illustrating the image display time interval. For ease of explanation, it is assumed that the average frame interval (IFS) calculated by the electronic device based on the aforementioned N frame rate data is... avg The target frame count is 0.041. tgt It consists of 24 frames.

[0141] Figure 5 (A) illustrates the process by which an electronic device displays 24 frames of images during the 1-second period (i.e., duration T0 is 1 second) between time t50 and t51 while playing the aforementioned first video. It is understandable that... Figure 5 In (A), the time when any frame of an image is displayed on the screen can be approximated as the frame generation timestamp of the frame rate data corresponding to that image; and the time interval between the display time of any frame of an image and the previous frame of an image can be considered as the frame interval between the frame rate data corresponding to that image and the frame rate data corresponding to the previous frame of an image. For example, the frame rate data corresponding to image P1 and image P0 (image P0 is the image displayed before image P1 and adjacent to image P1) Figure 5 The frame interval between the two frame rate data points (not shown in the image) is the frame interval between them. Figure 5 Similarly, the T1 duration shown in (A) is the frame interval between the frame rate data corresponding to image P2 and the two frame rate data corresponding to image P1. Figure 5 The T2 duration shown in (A) is used for the same reasoning.

[0142] Electronic devices can [do something] within 1 second (i.e., duration T0 is 1 second). Figure 5 Images P1 through P24, a total of 24 frames, from (A) onwards, are displayed sequentially on the screen of an electronic device. It should be noted that, ideally, the interval between the display times of any two frames can be 1 second (IFS). avg That is, the values ​​of T1-T24 are all IFS. avgHowever, in reality, when the first video is played normally, the time interval between the display times of two different frames on the screen is likely not exactly the same, but they are all within the range [IFS]. min IFS max [In the context of], for example, the duration of T1 can be different from the duration of T2, and the duration of T3 can also be different from the duration of T4, and there are instances in T1-T24 that are greater than IFS. avg The value also exists that is less than IFS. avg The value of . In this case, the electronic device can display 24 frames of images, from image P1 to image P24, on the screen within 1 second.

[0143] However, the interval between the times when two adjacent frames are displayed on the screen is significantly larger (greater than the maximum frame interval IFS). max If at least one of the two frames is drawn too slowly, the electronic device will need to draw the image at a much faster speed in subsequent drawing processes to ensure that it can display 24 frames on the screen in one second.

[0144] like Figure 5 As shown in (B) in the image, Figure 5 Images P1'-P24' shown in (B) can be images from the first video mentioned above, and the 24 frame rate data corresponding to images P1'-P24' can be frame rate data acquired after the aforementioned N frame rate data have been obtained. It is assumed here that during subsequent playback of the first video, the electronic device reduces its performance output to lower power consumption, resulting in images P1' and P0' (image P0' is the image displayed before and adjacent to image P1') being displayed differently. Figure 5 The time interval T1' between (not shown in the image) and the maximum frame interval IFS mentioned above, where T1' is greater than the maximum frame interval IFS mentioned above. max .

[0145] Since the time interval between images P1' and P0' is significantly large, if the electronic device wants to display 24 frames sequentially on the screen within a 1-second period from t50' to t51', it must display the subsequent 23 frames (i.e., images P2' to P24') within the remaining (1-T1') period. In other words, if the electronic device can maintain the frame rate of the first video at 28 frames per second during the period from t50' to t51', the average display time interval between any two adjacent images in the subsequent 23 frames must be (1-T1') / 23, which is the target frame interval corresponding to the aforementioned first frame rate data.

[0146] Understandably, the frame rate data corresponding to image P1' can be the first frame rate data mentioned above. Starting from the frame rate data corresponding to image P1, the display process of each frame will occupy a portion of the time interval t50'-t51', which will inevitably affect the time used to display subsequent images on the screen. In other words, the frame interval between the frame rate data corresponding to each subsequent frame and the frame rate data corresponding to the previous frame will affect the possibility of the electronic device maintaining a frame rate of 24 frames per second. For example, assuming the display time interval T2' between image P2' and image P1' is greater than (1-T1') / 23, it can be considered that the time spent by the electronic device to display image P2' on the screen exceeds the originally planned time. Therefore, in order to maintain the frame rate of the first video at 28 frames per second during the time interval t50'-t51', the electronic device needs to display the subsequent 22 frames (i.e., image P3'-image P24') on the screen within the remaining time interval (1-T1'-T2'). The average display time interval between any two adjacent images in these subsequent 22 frames needs to be (1-T1'-T2) / 22, which is the target frame interval corresponding to the frame rate data of image P2'. Because T2' is greater than (1-T1') / 23, simple mathematical reasoning shows that [(1-T1'-T2) / 22] < [(1-T1') / 23]. This means that the electronic device needs to complete the display process of subsequent images at a faster speed, which puts greater pressure on the performance of the electronic device.

[0147] S104. When the frame interval between the third frame rate data and the fourth frame rate data is greater than the target frame interval corresponding to the third frame rate data, and the target frame interval corresponding to the third frame rate data is less than the minimum frame interval mentioned above, the electronic device determines that the first video frame is dropped.

[0148] As can be seen from the foregoing description, for the aforementioned first frame rate data and the frame rate data acquired after the first frame rate data, the smaller the target frame interval corresponding to the frame rate data, the more difficult it is for the electronic device to maintain the frame rate at the target frame rate. Therefore, in this embodiment, if there exists a third frame rate data, and the preceding frame rate data adjacent to the third frame rate data is the aforementioned fourth frame rate data, if the frame interval between the third frame rate data and the fourth frame rate data is greater than the target frame interval corresponding to the fourth frame rate data, and the frame interval corresponding to the third frame rate data is less than the aforementioned minimum frame interval (IFS), min If the electronic device determines that its current frame rendering capability (i.e., the current performance level of the electronic device) is insufficient to maintain the frame rate of the first video played in the first process at f, then the electronic device can determine that the current frame rendering capability is insufficient to maintain the frame rate of the first video played in the first process at f. tgt .

[0149] First, the target frame interval corresponding to the fourth frame rate data mentioned above is the frame interval determined by the electronic device for subsequent frame rate data based on the sum of the frame intervals of all preceding frame rate data (from the first frame rate data to the fourth frame rate data) (i.e., the time required to output the images corresponding to these frame rate data). When the frame interval between the third and fourth frame rate data (i.e., the time required to output the image corresponding to the third frame rate data) is greater than the target frame interval corresponding to the fourth frame rate data, it indicates that the electronic device needs to further speed up the display of subsequent images. Second, IFS min This calculation, based on the aforementioned N frames of data, assumes that the electronic device has sufficient performance capabilities. This also means that even with sufficient performance capabilities, when playing the first video, the frame interval between two consecutive frames might not be reduced to IFS. min When the target frame interval corresponding to the third frame rate data is too small, it indicates that the frame intervals of the frame rate data preceding this frame rate data are too large. This is likely because the electronic device has reduced the performance allocation, for example, by reducing the maximum power that the CPU can provide in order to reduce power consumption. With reduced performance allocation, the electronic device is even less able to reduce the frame interval between subsequent frame rate data to less than IFS. min Therefore, when the target frame interval corresponding to the aforementioned third frame rate data is determined to be less than IFS... min Subsequently, the electronic device can determine that the current frame rendering capability is insufficient (i.e., the current electronic device cannot maintain the frame rate at f). tgt Therefore, the video will experience frame drops during subsequent playback.

[0150] Figure 6 The diagram illustrates the specific process by which an electronic device calculates the remaining number of frames and the target frame interval for each frame rate data point based on subsequently acquired frame rate data after acquiring the first frame rate data.

[0151] like Figure 6 As shown in (A), it is assumed that N is 60, and the average frame interval IFS is calculated by the electronic device based on the above N frame rate data. avg The target frame count is 0.041. tgt 24 frames, minimum frame interval (IFS) min The maximum frame interval (IFS) is 0.034. max The value is 0.048. This calculation result can be used by electronic devices based on frame rate data from frame number 1 to frame number 60. Figure 6 The frame rate data (not shown in the image) is calculated from the frame rate data of the electronic device based on frame number 8 (some frame rate data is shown in the image). Figure 6 The data is calculated from frame rate data up to frame number 67 (not shown in the image).

[0152] exist Figure 6 In (B), the frame interval between frame rate data with frame number 68 and frame rate data with frame number 67 is 0.051. Since 0.051 > IFS min =0.034, therefore, the frame rate data with frame number 68 can be the first frame rate data mentioned above, and the frame rate data with frame number 67 can be the second frame rate data mentioned above. As explained above, the electronic device will use the frame generation timestamp of the frame rate data with frame number 67 as the start time of a 1-second segment, and will use the image corresponding to the first frame rate data as the first frame image output by the electronic device in that 1-second segment. Furthermore, the electronic device will set a corresponding remaining frame number for each frame rate data starting from frame rate data with frame number 68, and determine the target frame interval for each frame rate data based on the remaining frame number for each frame rate data and the frame interval between each frame rate data and the previous frame rate data (the target frame interval for frame rate data before frame rate data with frame number 68 can all be set to 1 FS). max , that is, 0.048).

[0153] Taking frame rate data with frame number 68 as an example, since the frame interval between it and frame rate data with frame number 67 is 0.051, it means that the electronic device needs 0.051 seconds to display the image corresponding to frame rate data with frame number 68 on the screen. If the electronic device wants to maintain a frame rate of 24 frames / second, it needs to complete the display of at least 23 subsequent frames within (1-0.051) = 0.949 seconds. Therefore, it is not difficult to calculate that the corresponding average frame interval is 0.949 ÷ 23 ≈ 0.0413 seconds. Thus, the remaining duration corresponding to frame rate data with frame number 68 is 0.949 seconds, the remaining number of frames is 23, and the corresponding target frame interval (IFS) is... tgt_68 The interval is 0.0413 seconds. This is because the frame interval between frame number 68 and frame number 67 is greater than the target frame interval (IFS) corresponding to frame number 67. max =0.048 seconds, therefore, in the column "Does the current frame interval meet the target set by the previous frame?", the label value corresponding to the frame rate data with frame number 68 is 0, indicating that the current frame interval is greater than the target frame interval set by the electronic device based on the previous frame data. Furthermore, because the target frame interval IFS corresponding to the frame rate data with frame number 68 is... tgt_68 =0.0413 seconds is greater than the minimum frame interval (IFS) min =0.034 seconds, therefore, in the column "Does the target interval of the next frame meet the minimum frame interval value", the label value corresponding to the frame rate data with frame number 68 is 1.

[0154] Taking frame rate data with frame number 76 as an example, since the frame interval between it and frame rate data with frame number 75 is 0.051, it means that the electronic device needs 0.092 seconds to display the image corresponding to frame rate data with frame number 76 on the screen. If the electronic device wants to maintain a frame rate of 24 frames / second, it needs to complete the display of at least 15 subsequent frames within (0.643-0.092) = 0.551 seconds. Therefore, it is not difficult to calculate that the corresponding average frame interval is 0.551÷15≈0.0367 seconds. Thus, the remaining duration corresponding to frame rate data with frame number 76 is 0.551 seconds, the remaining number of frames is 15, and the corresponding target frame interval (IFS) is... tgt_76 The interval is 0.0367 seconds. This is because the frame interval between frame rate data with frame number 76 and frame number 75 is greater than the target frame interval (IFS) corresponding to frame rate data with frame number 75. tgt_76 =0.0402 seconds, therefore, in the column "Does the current frame interval meet the target set by the previous frame?", the label value corresponding to the frame rate data with frame number 76 is 0, indicating that the current frame interval is greater than the target frame interval set by the electronic device based on the previous frame data. Furthermore, because the target frame interval IFS corresponding to the frame rate data with frame number 76... tgt_76 =0.0367 seconds is greater than the minimum frame interval (IFS) min =0.034 seconds, therefore, in the column "Does the target interval of the next frame meet the minimum frame interval value", the tag value corresponding to the frame rate data with frame number 76 is also 1.

[0155] If the label values ​​for "Does the current frame interval meet the target set in the previous frame?" and "Does the target interval for the next frame meet the minimum frame interval value?" in the frame rate data are not both 0, the electronic device can continue to calculate the remaining number of frames and the corresponding target frame interval for subsequent frame rate data. Examples will not be provided here. The electronic device can determine that the current frame rendering capability is insufficient when the label values ​​for both "Does the current frame interval meet the target set in the previous frame?" and "Does the target interval for the next frame meet the minimum frame interval value?" in a certain frame rate data are both 0.

[0156] Reference Figure 6 Frame rate data with frame numbers 81 and 82 in (B) can be considered as the third frame rate data mentioned above, and therefore, frame rate data with frame number 81 can be considered as the fourth frame rate data mentioned above. Since the frame interval between frame rate data with frame number 82 and frame rate data with frame number 81 is 0.047, this frame interval is greater than the target frame interval (IFS) corresponding to frame rate data with frame number 81. tgt_81=0.0345 seconds, therefore, in the column "Does the current frame interval meet the target set in the previous frame?", the label value corresponding to the frame rate data with frame number 82 is 0. Furthermore, since the target frame interval (IFS) corresponding to the frame rate data with frame number 82 is... tgt_82 =0.0331 seconds is greater than the minimum frame interval (IFS) min =0.034 seconds, therefore, in the column "Does the target interval for the next frame meet the minimum frame interval value?", the tag value corresponding to the frame rate data with frame number 82 is also 0. Combining the foregoing explanation, it can be seen that when the frame interval between the third frame rate data and the fourth frame rate data is greater than the target interval corresponding to the fourth frame rate data, and the frame interval corresponding to the third frame rate data is less than the aforementioned minimum frame interval (IFS),... min If the electronic device determines that its current frame rendering capability (i.e., the current performance level of the electronic device) is insufficient to maintain the frame rate of the first video played in the first process at f, then the electronic device can determine that its current frame rendering capability (i.e., the current performance level of the electronic device) is insufficient to maintain the frame rate of the first video played in the first process at f. tgt Therefore, when the electronic device determines that 0.047 > IFS tgt_81 And IFS tgt_82 <IFS min Subsequently, the electronic device can determine that the current frame rendering capability is insufficient, and the video playback may experience frame drops.

[0157] Understandably, for all processes playing videos, and for all videos played by those processes, the electronic device can collect the corresponding frame rate data and determine, according to the steps described above, whether it can maintain the frame rate at the normal playback frame rate of the video. When the electronic device cannot maintain the normal playback frame rate of any video, it will determine that the current frame rendering capability is insufficient, meaning that at least one of the videos played by the electronic device will experience frame drops.

[0158] Optionally, when frame data is generated N1 seconds before the end of the 1-second duration or when there are N2 frames remaining, the electronic device may report to the system a determination that the device's frame rendering capability is insufficient and frame dropping is about to occur during video playback. This allows the electronic device to intervene in the frame dropping phenomenon in a timely manner. Specifically, the value of N1 can be 0.3, and the value of N2 can be 5.

[0159] S105. In the event of frame loss in the first video, the electronic device increases its performance provision.

[0160] Understandably, when an electronic device's frame rendering capability is insufficient, if the device doesn't intervene, it will skip frames that should be displayed to prevent audio and video from becoming out of sync during playback. This means discarding one or more frames, which is the phenomenon of frame dropping during video playback. While this maintains audio and video synchronization, users may noticeably perceive discontinuities in the video. With excessively high frame dropping frequencies, users may even experience noticeable stuttering during playback.

[0161] Figure 7 This illustrates the CPU and GPU operation flow of an electronic device during video playback.

[0162] like Figure 7 As shown, during the process of drawing images from a video file onto an electronic device screen, the CPU collects the necessary mesh information (including vertex position information, normal direction, vertex color, etc.) and texture information from the hard drive, sets the rendering state (such as transparency, double-sided or single-sided, using vertex shaders or fragment shading, etc.), and then calls the DrawCall method to write all the information (drawing instructions) into a buffer. Afterwards, the GPU reads the relevant drawing instructions from the buffer for rendering. When the CPU writes commands too slowly or the GPU renders too slowly, the frame interval between frame rate data increases, causing frame drops during video playback.

[0163] Therefore, in this embodiment of the application, when the frame rendering capability of the electronic device is insufficient, the electronic device can increase the performance supply to reduce or avoid frame dropping during subsequent video playback.

[0164] Optionally, the electronic device can increase the CPU's maximum available power PL1 (Package Long-duration Power Limit) or maximum operating frequency (Max Frequency); alternatively, the electronic device can also adjust the maximum duration for which the CPU power briefly exceeds the power limit (i.e., the CPU's maximum available power) to improve the CPU's command processing performance.

[0165] Optionally, the electronic devices can also increase the GPU's maximum power or maximum frequency to improve the GPU's rendering performance.

[0166] Optionally, the electronic device can also adjust the time rate that the CPU can allocate on Draw Calls to improve command processing performance.

[0167] Optionally, for images with the same rendering state, the electronic device can package the instruction information corresponding to multiple images with the same rendering state into a single instruction. Then, the electronic device only needs to call the Draw Call to execute one instruction.

[0168] Optionally, the electronic device may also choose other methods to increase the supply of device performance, such as increasing the speed of CPU write commands and / or increasing the rendering speed of GPU, which is not limited in this application.

[0169] Since adjusting the performance supply level of electronic devices takes time, in one optional implementation, after the electronic device reports the determination that frame drops may occur to the system, it can pause acquiring frame rate data of the currently playing video and stop calculating the frame rendering capability required for each video. After N3 seconds, the electronic device can then resume acquiring frame rate data of the currently playing video and continue calculating the frame rendering capability required for each video using the aforementioned method.

[0170] Optionally, the electronic device may not update the target frame number, target frame interval, maximum frame interval, and minimum frame interval. Instead, the electronic device may set the target frame interval corresponding to the first frame rate data (which is the frame rate data of the first video) acquired after N3 seconds to the aforementioned IFS. max And the frame interval between a subsequent frame rate data and the previous frame rate data is greater than IFS. max In this case, the remaining number of frames and the target frame interval corresponding to each frame rate data are calculated using the frame interval of adjacent frame rate data to determine if the video is experiencing frame drops. Alternatively, after the performance supply level is increased, the electronic device can reselect N frames of the first video acquired after the aforementioned N3 seconds to update the aforementioned target frame number, target frame interval, maximum frame interval, and minimum frame interval. Afterward, the electronic device can continue to acquire the frame rate data of the first video and determine whether the first video is experiencing frame drops according to the operations described in steps S103-S104. In this way, even in an operating environment where the performance supply level is constantly changing, the electronic device can still promptly determine that frame drops will occur during video playback and intervene in the frame drops in time before the video process causes obvious stuttering due to frame drops, thereby reducing frame drops in the video process and improving the user experience.

[0171] The electronic device provided in this application will now be described.

[0172] The electronic device may be a mobile phone, tablet computer, wearable device, in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), or dedicated camera (such as SLR camera, point-and-shoot camera), etc. This application does not limit the specific type of the electronic device.

[0173] Figure 8 The structure of the electronic device is shown as an example.

[0174] Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, audio module 170, speaker 170A, display screen 194, etc.

[0175] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0176] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0177] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0178] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0179] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0180] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, internal memory 121, display screen 194, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0181] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel.

[0182] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0183] Internal memory 121 can be used to store computer executable program code, which includes instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of electronic device 100 by running instructions stored in internal memory 121 and / or instructions stored in memory located in the processor.

[0184] Electronic device 100 can implement audio functions, such as playing video, through audio module 170, speaker 170A, display screen 194 and application processor.

[0185] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0186] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or make hands-free calls through the speaker 170A.

[0187] The memory in internal memory 121 and / or the memory in processor 110 can be used to store instruction information generated by the producer in the process, such as frame rate data generated by the producer DXGI module and / or D3D module in a process that is playing a video.

[0188] When the process of electronic device 100 includes a process in which a video is being played, processor 110 can, when the performance of electronic device is sufficiently high, collect N frame rate data corresponding to the first video in the process of playing video, and calculate the target number of frames, target frame interval, maximum frame interval and minimum frame interval of the corresponding video based on the N frame rate data.

[0189] The processor 110 is further configured to acquire frame rate data corresponding to the first video after acquiring the aforementioned N frame rate data. If the frame interval between the first frame rate data and the previous frame rate data is greater than the aforementioned maximum frame interval, starting from the first frame rate data, a corresponding remaining number of frames is set for each frame rate data, and a target frame interval is determined for each frame rate data based on the remaining number of frames corresponding to each frame rate data and the frame interval between each frame rate data and the previous frame rate data. If the frame interval between the second frame rate data and the third frame rate data is greater than the target frame interval corresponding to the second frame rate data, and the target frame interval corresponding to the second frame rate data is less than the aforementioned minimum frame interval, the processor 110 can determine that the current frame rendering capability is insufficient, and frame dropping will occur during the playback of the first video. The processor 110 is also configured to adjust the maximum available power of the processor 110 if it is determined that frame dropping will occur during video playback. For details, please refer to the aforementioned... Figure 1 The relevant explanations will not be repeated here.

[0190] This application also provides an electronic device, which includes one or more processors and a memory; wherein the memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the electronic device to perform the methods shown in the foregoing embodiments.

[0191] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0192] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0193] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for determining frame loss, characterized in that, include: Based on multiple frame rate data points from the first video, a target frame count, a maximum frame interval, and a minimum frame interval are determined. The target frame count is the average number of images displayed per second when the electronic device plays the first video. The minimum and maximum frame intervals are used to determine whether the frame interval between two adjacent frame rate data points is abnormal. The interval between the frame generation timestamps of any two frame rate data points is the frame interval between those two frame rate data points. If the frame interval between the first and second frame rate data points is greater than the maximum frame interval, a target frame interval is determined for each frame rate data point following the second frame rate data point. The target frame interval is the ratio of the remaining duration to the remaining number of frames. The remaining number of frames is the difference between the target frame count and the first frame count. The first frame count is the number of frame rate data points between each frame rate data point and the first frame rate data point. The remaining duration is determined by the frame interval between each frame rate data point and the second frame rate data point. The first frame rate data point is the frame rate data point acquired after the multiple frame rate data points are acquired, and the second frame rate data point is the preceding frame rate data point adjacent to the first frame rate data point. If the frame interval between the third frame rate data and the fourth frame rate data is greater than the target frame interval corresponding to the fourth frame rate data, and the target frame interval corresponding to the third frame rate data is less than the minimum frame interval, the first video frame is determined to be dropped. The fourth frame rate data is the frame rate data obtained after the multiple frame rate data are obtained, and the fourth frame rate data is the previous frame rate data adjacent to the third frame rate data.

2. The method according to claim 1, characterized in that, Before determining the target number of frames, maximum frame interval, and minimum frame interval of the first video based on multiple frame rate data of the first video, the method includes: When the maximum power that the CPU of the electronic device can supply is greater than a first threshold, multiple frame rate data of the first video are acquired.

3. The method according to claim 1 or 2, characterized in that, Before determining the target number of frames, maximum frame interval, and minimum frame interval of the first video based on multiple frame rate data of the first video, the method further includes: Determine at least one process playing a video from multiple processes, and obtain the process number of each process in the at least one process playing a video, wherein the at least one process playing a video includes a first process, and the first process is the process playing the first video; Based on the process ID of the first process, multiple instruction information of the first process is obtained, and each instruction information contains a cache ID that stores the instruction information. Multiple consecutive instruction messages with cache numbers all being the first cache number are used as multiple frame rate data of the first video.

4. The method according to claim 1 or 2, characterized in that, After determining the first video frame loss, the method further includes: Increase the maximum power supply of the CPU or the maximum operating frequency of the CPU; And / or increase GPU power consumption or GPU operating frequency.

5. The method according to claim 3, characterized in that, After determining the first video frame loss, the method further includes: Increase the maximum power supply of the CPU or the maximum operating frequency of the CPU; And / or increase GPU power consumption or GPU operating frequency.

6. The method according to any one of claims 1 or 2, characterized in that, After determining the first video frame loss, the method further includes: Stop acquiring frame rate data from the first video; After the first duration, continue acquiring the frame rate data of the first video. Whether the first video has dropped frames is determined based on the frame rate data of the first video obtained after the first duration.

7. The method according to claim 3, characterized in that, After determining the first video frame loss, the method further includes: Stop acquiring frame rate data from the first video; After the first duration, continue acquiring the frame rate data of the first video. Whether the first video has dropped frames is determined based on the frame rate data of the first video obtained after the first duration.

8. The method according to claim 1 or 2, characterized in that, The display of the multiple frames corresponding to the multiple frame rate data on the screen of the electronic device was not interrupted.

9. The method according to claim 1 or 2, characterized in that, The target number of frames is determined based on the average frame interval of the first video. The maximum and minimum frame intervals are determined based on the frame interval standard deviation. The average frame interval is the average of the frame intervals between all adjacent frame rate data in the multiple frame rate data. The frame interval standard deviation is the standard deviation of the frame intervals between all adjacent frame rate data in the multiple frame rate data.

10. An electronic device, characterized in that, The electronic device includes: one or more processors, memory, and a display screen; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1-9.

11. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the processors being used to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1-9.

12. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on an electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1-9.

Images