Frame loss determination method and related device

Abstract

The invention provides a frame loss determination method and a related device, and the method comprises the steps: determining an interval range corresponding to a normal frame interval of frame rate data and a normal frame rate of a video through analyzing the frame rate data of a played video; under the condition that the frame interval between a certain subsequent frame of image and a previous frame of image exceeds the interval range, determining the frame drawing capability required by the electronic equipment for keeping the normal frame rate based on the frame interval between each piece of frame rate data and the previous piece of frame rate data; and under the condition that the frame drawing capability required by the electronic equipment exceeds the interval range corresponding to the normal frame interval, determining that a frame loss phenomenon occurs in the video process. By implementing the method, the electronic equipment can determine the frame loss phenomenon in the video process in time, and intervenes the frame loss phenomenon in time before the video process generates an obvious lagging phenomenon due to frame loss, so that the frame loss phenomenon in the video process is reduced, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a frame loss determination method and related devices. Background Art

[0002] In daily use, users often use electronic devices such as desktop computers and laptops to play videos. At present, some electronic devices will reduce the power consumption of electronic devices by sacrificing some performance. However, in the process of playing videos, when the performance they can provide is not enough to meet the performance requirements of the current operation scenario of the electronic device, the video played by the electronic device is likely to drop frames, and the picture displayed by the electronic device will be stuck, which brings a very bad user experience to users.

[0003] Current electronic devices cannot monitor the frame loss phenomenon of the video in time, and therefore cannot intervene in the frame loss phenomenon during video playback in time, thereby reducing the occurrence of frame loss. Therefore, how to determine the occurrence of frame loss in time is an urgent problem to be solved. Summary of the invention

[0004] The purpose of this application is to provide a frame loss determination method and related devices. By implementing this method, the electronic device can promptly determine the frame loss phenomenon in the video process, and intervene in the frame loss phenomenon in time before the video process produces obvious freezes due to frame loss, thereby reducing the frame loss phenomenon in the video process and improving the user experience.

[0005] The above-mentioned and other objects are achieved by the features of the independent claims. Further implementations are presented in the dependent claims, the description and the drawings.

[0006] In a first aspect, the present application provides a method for determining frame loss, including: determining a target frame number, a maximum frame interval, and a minimum frame interval based on multiple frame rate data of a first video, wherein the target frame number is the average number of images displayed per second when the electronic device plays the first video, and the minimum frame interval and the maximum frame interval are used to determine whether the frame interval between two adjacent frame data is abnormal; when the frame interval between the first frame rate data and the second frame rate data is greater than the maximum frame interval, respectively determining the target frame interval corresponding to each frame rate data after the second frame rate data, and the target frame interval is based on the target frame number and the interval between each frame rate data and the The frame interval of the second frame rate data is determined; the first frame rate data is the frame rate data obtained after the multiple frame rate data are obtained, and the second frame rate data is the previous frame rate data adjacent to the first frame rate data; when the frame interval between the third frame rate data and the fourth frame rate data is greater than the target frame interval corresponding to the fourth frame rate data, and the target frame interval corresponding to the third frame rate data is less than the minimum frame interval, it is determined that the first video has lost a frame, and the fourth frame rate data is the frame rate data obtained after the multiple frame rate data are obtained, and the fourth frame rate data is the previous frame rate data adjacent to the third frame rate data.

[0007] In this method, the multiple pieces of frame rate data are frame rate data corresponding to the continuous multiple frames of images in the first video. Each piece of frame rate data includes a corresponding frame generation timestamp, and the frame generation timestamp of any piece of frame rate data can be approximately regarded as the display time of the image corresponding to the frame rate data on the screen of the electronic device, and the interval between the frame generation timestamps of any two adjacent frame rate data (i.e., the frame interval between the two pieces of frame rate data) can be approximately regarded as the interval of the display time of the image corresponding to the two pieces of frame rate data on the screen of the electronic device.

[0008] It is understandable that after acquiring the multiple frame rate data, the electronic device can calculate the frame interval between every two adjacent frame rate data, and these frame intervals are most likely a set of fluctuating values. The electronic device can use the average value of the frame interval between every two adjacent frame rate data in the multiple frame rate data as the target frame number (the target frame number can be considered as the frame rate of the first video when the electronic device plays the first video normally), and calculate the normal fluctuation range of the frame interval values ​​of adjacent frame rate data in the multiple frame rate data according to statistical principles using the standard deviation and the target frame interval. The upper limit value of the normal fluctuation range is the maximum frame interval, and the lower limit value is the minimum frame interval.

[0009] When the frame interval between the frame rate data acquired after the plurality of frame rate data and the previous frame rate data does not fall within the normal fluctuation range, the electronic device can determine that the playback of the first video may be abnormal. Therefore, when the frame interval between the first frame rate data and the previous frame rate data is greater than the maximum frame interval, the electronic device can determine that the video played by the first process may subsequently experience frame loss.

[0010] In fact, the most important basis for an electronic device to determine whether a video playback has dropped frames is whether the electronic device is capable of maintaining the frame rate of the video playback at the target frame number, that is, whether the electronic device is still capable of drawing a frame image with the same value as the target integer within 1 second. Therefore, the frame interval between the first frame rate data and the second frame rate data being greater than the maximum frame interval is only the primary condition for the electronic device to trigger the determination of whether the video has dropped frames. In the subsequent process, it is also necessary to determine whether the electronic device is capable of maintaining the frame rate at the target frame number, that is, whether the electronic device can still continuously output f within 1 second. tgt Frame image.

[0011] Here we assume that the target frame number is f tgt When the first frame rate is obtained and the frame interval between the first frame rate data and the second frame rate data is found to be greater than the maximum frame interval, the electronic device may assume that it can still maintain the frame rate at the upper target frame number f tgt , and the electronic device can use the image corresponding to the first frame rate data as the first frame image output within 1 second, then the time consumed for displaying the first frame image is subtracted from 1 second to obtain the time consumed for displaying the following (f tgt -1) frame data corresponding to the available time of the image; similarly, 1s is used to subtract the time used for the first frame image and the second frame image, which is the available time for displaying the following (f tgt -2) The available time of the image corresponding to the frame rate data, and so on. Specifically, the ratio of the subsequent available time corresponding to each frame rate data to the remaining number of frames is the target frame interval corresponding to the frame rate data.

[0012] It can be understood that for the first frame rate data and the frame rate data obtained after the first frame rate data, if the target frame interval corresponding to the frame rate data is smaller, the electronic device will have greater difficulty in maintaining the frame rate at the target frame number. Therefore, in this method, if there is a third frame rate data, and the previous frame rate data adjacent to the third frame rate data is the fourth frame rate data, if the frame interval between the third frame rate data and the fourth frame rate data is greater than the target frame interval corresponding to the fourth frame rate data, and the frame interval corresponding to the third frame rate data is less than the minimum frame interval, then the electronic device can determine that the current frame drawing capability (i.e., the current performance supply level of the electronic device) cannot keep the frame rate of the first video played by the first process to the value corresponding to the target integer, and the electronic device can determine that the first video will be played with frame loss. By implementing this method, the electronic device can promptly determine the frame loss phenomenon in the video process, and intervene in the frame loss phenomenon in time before the video process produces obvious jamming due to frame loss, reduce the frame loss phenomenon of the video process, and improve the user experience.

[0013] In combination with the first aspect, in a possible implementation, before determining the target frame number, maximum frame interval and minimum frame interval of the first video based on multiple frame rate data of the first video, the method includes: when the maximum power that can be supplied by the CPU of the electronic device is greater than a first threshold, obtaining multiple frame rate data of the first video.

[0014] In order to ensure that the target number of frames, the maximum frame interval, and the minimum frame interval calculated by the electronic device based on the multiple frame rate data can correctly reflect the actual frame interval status when the first video is played normally, the electronic device needs to ensure that during the generation of the multiple frame rate data, there is no frame loss in the playback of the first video. Therefore, the electronic device can complete the acquisition operation of the multiple frame rate data under a high-performance supply working state. For example, the electronic device can perform the aforementioned operation to acquire the multiple frame rate data when the maximum supply power of the CPU is large enough; it is understandable that when the maximum supply power of the CPU is large enough, the performance supply of the electronic device is sufficient, and the playback of the first video is not prone to frame loss.

[0015] In combination with the first aspect, in a possible implementation, before determining the target frame number, maximum frame interval and minimum frame interval based on multiple frame rate data of the first video, the method also includes: determining at least one process for playing the video from multiple processes, and obtaining the process number of each process in the at least one process for playing the video, the at least one process for playing the video includes a first process, and the first process is the process for playing the first video; obtaining multiple instruction information of the first process based on the process number of the first process, any one of the multiple instruction information includes a cache number for storing the any one instruction information; and using multiple consecutive instruction information whose cache numbers in the instruction information are all the first cache numbers as multiple frame data of the first video.

[0016] It should be understood that the electronic device may process multiple processes that are playing videos at the same time, and there may be multiple videos playing at the same time in the same process. However, in the electronic device, the frame rate data corresponding to the images in different videos (even if the multiple videos are all videos played by the same video) will be stored in different image processing caches. Therefore, in this embodiment, the electronic device can first screen the process number of the process in which the electronic device is playing the video, and after obtaining the number belonging to the first process, use the process number of the first process to obtain all the frame rate data (instruction information) of the first process, and determine the frame rate data with the same cache number as the frame rate data of the same video according to the image cache number contained in the frame rate data. Afterwards, the electronic device can use the continuous multiple frame rate data whose image cache number is the first cache number as the above-mentioned first frame rate data. Further, after obtaining the multiple frame rate data, the electronic device can arrange the frame rate data according to the order of the timestamps in the frame rate data to facilitate the subsequent calculation of the frame interval between adjacent frame rate data.

[0017] In combination with the first aspect, in a possible implementation, after determining that the first video has lost a frame, the method further includes: increasing the maximum power that can be supplied by the CPU or the maximum operating frequency of the CPU; and / or increasing the GPU operating power or the GPU operating frequency.

[0018] It is understandable that when the electronic device's frame drawing capability is insufficient, if the electronic device does not perform other interventions, in order to prevent the video picture and sound from being out of sync during subsequent playback, the electronic device will skip the picture that should have been displayed and not display it on the screen. Although this can keep the video picture and sound synchronized, the user may also clearly feel that the picture in the video will be incoherent. When the frame loss frequency is too high, the user may even feel that the video playback is obviously stuck.

[0019] Therefore, in this embodiment, when the frame drawing capability of the electronic device is insufficient, the electronic device can increase the performance supply level to reduce or avoid frame loss during subsequent video playback.

[0020] Optionally, the electronic device can increase the maximum power supply PL1 (Package Long-duration Power Limit) or the maximum operating frequency (Max Frequency) of the CPU; optionally, the electronic device can also adjust the maximum duration of the CPU power temporarily breaking through the power consumption wall (i.e., the maximum power supply of the CPU) to improve the command processing performance of the CPU.

[0021] Optionally, the electronic device increases the maximum supplyable power or the maximum operable frequency of the CPU by a magnitude greater than a second threshold.

[0022] Optionally, the electronic device may also increase the maximum power or maximum frequency of the GPU to improve the rendering performance of the GPU.

[0023] Optionally, the electronic device can also adjust the CPU time allocated to Draw Call to improve command processing performance

[0024] Optionally, for images with the same rendering status, the electronic device packages the instruction information corresponding to multiple images with the same rendering status into one instruction. Then the electronic device only needs to call Draw Call to execute one instruction.

[0025] Optionally, the electronic device may also choose other methods to increase the performance of the device to increase the speed of writing commands by the CPU and / or increase the rendering speed of the GPU, which is not limited in this application.

[0026] In combination with the first aspect, in a possible implementation, after determining that the first video has lost frames, the method further includes: stopping obtaining frame rate data of the first video; continuing to obtain frame rate data of the first video after a first time length, and determining whether the first video has lost frames based on the frame rate data of the first video obtained after the first time length.

[0027] Since it takes a certain amount of time for the electronic device to adjust the performance supply level, in an optional implementation, after the electronic device reports the result of determining that the video will lose frames to the system, the electronic device can suspend obtaining the frame rate data of the process that is playing the video, and no longer calculate the frame drawing capacity required for each video. After several seconds (e.g., five seconds), the electronic device can continue to re-acquire the frame rate data of the process that is playing the video, and continue to calculate the frame drawing capacity required for each video according to the aforementioned method.

[0028] Optionally, the electronic device may not update the target number of frames, target frame interval, maximum frame interval and minimum frame interval. The electronic device may set the target frame interval corresponding to the first frame rate data obtained after 5 seconds as the maximum frame interval, and when the frame interval between a subsequent frame rate data and the previous frame rate data is greater than the maximum frame interval, the electronic device begins to use the frame interval of the adjacent frame rate data to calculate the remaining number of frames and the target frame interval corresponding to each frame rate data, so as to determine the video frame loss. Alternatively, after the performance supply level is increased, the electronic device may reselect multiple frame data of the first video obtained after 5 seconds to update the target number of frames, target frame interval, maximum frame interval and minimum frame interval, and then the electronic device may continue to obtain the frame rate data of the first video and determine whether the first video continues to lose frames according to the above operation. In this way, even in an operating environment where the performance supply level is constantly changing, the electronic device can still promptly determine that frame loss will occur during the video playback process, and intervene in the frame loss phenomenon in a timely manner before the video process produces obvious freezes due to frame loss, thereby reducing the frame loss phenomenon of the video process and improving the user experience.

[0029] In combination with the first aspect, in a possible implementation manner, the display process of the multiple frames of images corresponding to the multiple pieces of frame rate data on the screen of the electronic device is not interrupted.

[0030] In addition, the acquisition process of the above-mentioned multiple frame rate data takes a certain amount of time. If the process of displaying the image in the first video on the display screen is terminated during the acquisition process, the frame interval between the last frame rate data before the interruption and the first frame rate data after the display is resumed may be particularly long, which may cause the target frame number, maximum frame interval and minimum frame interval calculated based on the multiple frame rate data to fail to truly reflect the frame rate status of the first video when it is normally played, which may further lead to the accuracy of the frame loss result judgment. Therefore, in this embodiment, if the display process of the image in the first video on the screen is interrupted during the acquisition of the above-mentioned multiple frame rate data, the electronic device needs to stop the acquisition of the frame rate data and discard the previously acquired frame rate data, and restart the acquisition of the multiple frame rate data of the first video after the image of the first video continues to be continuously displayed on the screen. It should be noted that "the display process of the image on the screen is interrupted" can include the electronic device stopping playing the above-mentioned first video, and the situation where the playback window of the above-mentioned first video on the screen is completely blocked by other windows. For example, when the amount of the multiple frame rate data is 60 pieces, the electronic device may need to spend 2-3 seconds to successfully collect 60 pieces of frame rate data corresponding to 60 frames of images in the first video. If the electronic device only collects the above 30 pieces of frame rate data for 30 frames of images in the first video, and the user pauses the first video, the electronic device can stop collecting the frame rate data of the first video, and re-collect the 60 pieces of frame rate data corresponding to the above 60 frames of images (excluding the above 30 pieces of frame rate data collected previously) after the user continues to play the first video.

[0031] In combination with the first aspect, in a possible implementation, the target number of frames is determined based on an average frame interval of the first video, the maximum frame interval and the minimum frame interval are determined based on a frame interval standard deviation, the average frame interval is the average value of the frame intervals between all adjacent frame rate data in the multiple frame rate data, and the frame interval standard deviation is the standard deviation of the frame intervals between all adjacent frame rate data in the multiple frame rate data.

[0032] Assuming that the amount of the plurality of frame rate data is N or more, the average frame interval IFS calculated based on the plurality of frame rate data is avg for:

[0033]

[0034] Among them, IFS a，b Represents the time interval between the frame generation timestamp of frame rate data a and the frame generation timestamp of frame rate data b, where frame rate data a and frame rate data b are two adjacent frame rate data.

[0035] The electronic device may calculate the frame rate when displaying the first video in the first process, that is, the number of images that the electronic device needs to draw in one second, which is also the target frame rate f tgt , and its calculation method can be expressed as:

[0036]

[0037] in, Indicates floor operation. Represents the largest integer less than X.

[0038] It is not difficult to understand that the frame rate of the first video when it is played normally is f tgt , the frame rate of the first video is not less than f tgt In this case, the picture observed by the user is basically smooth, which will not affect the user's viewing experience.

[0039] In addition, based on IFS avg The standard deviation IFS of the frame intervals of the plurality of frame rate data can be calculated std . Its calculation method can be expressed as:

[0040]

[0041] Furthermore, based on the above IFS std , the maximum frame interval IFS can be calculated max and minimum frame interval IFS min , and its calculation method can be expressed as:

[0042] IFS min =IFS avg -M×IFS std ;

[0043] IFS max =IFS avg +M×IFS std ;

[0044] Wherein, M can be any value between 1 and 10, and “×” represents a multiplication operation; specifically, the value of M can be 3.

[0045] It can be understood that the standard deviation is a quantitative indicator used to characterize the degree to which each data in the statistical data set deviates from the mean value. Mathematically, the M-fold standard deviation method to determine outliers is a commonly used data processing method, which involves two concepts: determining the "allowable range" of each sample based on the size of the overall sample standard deviation; sample values ​​that exceed the "allowable range" are outliers. The size of the overall sample standard deviation is related to the definition of outliers. Generally, three times the sample standard deviation is used as the sample allowable deviation range, that is, if a sample exceeds three times the standard deviation, the sample will be considered an outlier.

[0046] In a second aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method in the first aspect or any possible implementation of the first aspect.

[0047] In a third aspect, a chip system is provided, which is applied to an electronic device, and the chip system includes one or more processors, and the processors are used to call computer instructions so that the electronic device executes a method as in the first aspect or any possible implementation of the first aspect.

[0048] According to a fourth aspect, a computer program product comprising instructions, when the computer program product is run on an electronic device, enables the electronic device to execute the method according to the first aspect or any possible implementation of the first aspect.

[0049] In a fifth aspect, a computer-readable storage medium is provided, comprising instructions, which, when executed on an electronic device, enable the electronic device to execute a method in the first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A flowchart of a frame loss determination method provided in an embodiment of the present application;

[0051] Figure 2 A schematic diagram of a process for screening a video being played provided in an embodiment of the present application;

[0052] Figure 3 An architecture diagram of an ETW system provided in an embodiment of the present application;

[0053] Figure 4 A schematic diagram of a process for determining a frame rate parameter based on frame rate data provided in an embodiment of the present application;

[0054] Figure 5 A comparison diagram of image display time intervals in different playback states provided in an embodiment of the present application;

[0055] Figure 6 A schematic diagram of a calculation process for calculating the target frame rendering capability frame by frame provided in an embodiment of the present application;

[0056] Figure 7 A schematic diagram of a scenario in which a CPU and a GPU work together, provided in an embodiment of the present application;

[0057] Figure 8 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items.

[0059] To facilitate understanding, the relevant terms involved in the embodiments of the present application are first introduced below.

[0060] (1) Performance and power consumption of electronic equipment

[0061] Performance refers to the properties and functions of a substance. For electronic devices such as computers and tablets, the most important evaluation indicator is the computing speed of the electronic device, that is, the number of instructions that the electronic device can execute per second, generally described as "millions of instructions per second". Understandably, the computing speed of electronic devices will be further reflected in the dimensions of whether the electronic device runs smoothly and whether there is obvious lag; when the performance of the electronic device is higher, the frequency of lag when the electronic device is running is lower, and vice versa.

[0062] Power consumption refers to the amount of energy consumed by an electronic device per unit time. For electronic devices, power consumption specifically refers to the amount of electrical energy consumed by the electronic device per unit time, measured in W. Even in standby mode, an electronic device will still consume a certain amount of electrical energy (unless the power is turned off, it will not consume energy).

[0063] In order to balance the performance and power consumption of electronic devices, some electronic devices can currently adjust their performance upper limit adaptively according to the performance requirements of the current operating scenario, and reduce the power consumption of electronic devices by sacrificing some performance. When the operation of an electronic device is obviously stuck, it means that the current performance upper limit of the electronic device is not enough to meet the performance requirements of the current operating scenario, so the electronic device can adjust the performance upper limit to reduce the number of occurrences of the stuck phenomenon. Specifically, for electronic devices such as desktop computers and laptops, since the central processing unit (CPU) is the computing and control core of the system in these electronic devices, the maximum power that the CPU can supply and the maximum operating frequency of the CPU also determine the performance upper limit of the electronic device. Take the maximum power that the CPU can supply as an example to illustrate. Generally speaking, for microprocessors of the same series, the greater the maximum power that the CPU can supply, the better the performance upper limit of the electronic device.

[0064] Therefore, for the electronic device provided in the present application, when the electronic device determines that a freeze phenomenon occurs during operation (for example, playback freeze caused by video frame drops), the electronic device can appropriately increase the maximum CPU supply power and / or the maximum CPU operating frequency to improve the performance of the electronic device, so as to reduce the number of freeze phenomena and improve the user experience.

[0065] (2) Frame loss

[0066] Frame loss is also called frame skipping. Videos played by electronic devices are composed of many pictures displayed quickly, where each picture can be called a frame. During the video playback process, one or several frames are skipped and not displayed on the screen of the electronic device. This phenomenon is called frame loss.

[0067] Frame loss is common in video playback, such as online video, real-time video transmission, etc. Generally speaking, the image in the video stays in the human mind for about 0.1 to 0.4 seconds. At present, the frame rate of most videos is generally 18-24 frames, that is, 18-24 images need to be played in 1 second, so as to give people a visual effect of continuous images. When the frame loss phenomenon is not particularly serious, for example, only one frame of the 18 images that need to be played in one second is skipped and not displayed on the screen, the user may not notice it; but if the frame loss phenomenon is more serious, for example, two or three consecutive frames of the 18 images that need to be played in one second are skipped, the user will clearly feel that the picture on the screen is discontinuous.

[0068] (3) Frame rate data

[0069] For the processes that need to be processed by the electronic device, the producers in the system will generate a number of corresponding instruction information (events) for the processes, so that the consumers in the system can obtain and execute these instructions.

[0070] In the present application, for any process that is playing a video, there is a producer in the system (such as DirectX-Graphis-Infrastructure (DXGI) or Direct-3D (D3D)) that generates image processing instructions for the process. These image processing instructions can be obtained by the electronic device from its own cache or log file. In the present application, these image processing instructions can be referred to as "frame rate data".

[0071] Specifically, since video playback is actually an electronic device drawing and displaying images one frame at a time on a display screen, for the process of playing a video, each frame of the video will have a frame rate data, and each frame rate data contains the corresponding process number, frame number, frame generation timestamp, and image processing cache number. Among them, the frame rate data with the same process number and the same image processing cache number are the frame rate data corresponding to the image in the same video file. For the frame rate data corresponding to the image of the same video file, the image corresponding to the frame rate data with an earlier frame generation time will be displayed on the screen earlier by the electronic device.

[0072] In the present application, the timestamp contained in the frame rate data can be approximately regarded as the time when the image corresponding to the frame rate data is displayed on the screen.

[0073] (4) Frame interval

[0074] The multiple frames of images contained in the video file all contain corresponding timestamps. When the video is played, the order in which these images are displayed on the screen is determined by their corresponding timestamps - images with earlier timestamps will be displayed on the screen before images with later timestamps. Among them, the interval between the frame generation timestamps of the frame rate data corresponding to any two frames of images can be called the interframe space (IFS) of the two frame rate data. Correspondingly, the time difference between any two frames of images displayed on the screen can be approximately regarded as the frame interval between the frame rate data corresponding to the two frames of images.

[0075] In this application, among the multiple pieces of frame rate data corresponding to the multiple consecutive images in the video, the average value of the frame interval between each two adjacent frame rate data can be referred to as the average frame interval IFS of the multiple pieces of frame rate data. avg .

[0076] In addition, based on the average frame interval of the plurality of frame rate data, the frame interval standard deviation IFS of adjacent frame rate data in the plurality of frame rate data can be calculated.std Furthermore, based on the frame interval standard deviation IFS of adjacent frame rate data, the estimated maximum value IFS of the frame interval between subsequent adjacent frame rate data can be calculated. max and the estimated minimum value IFS min ,in:

[0077] IFS min =IFS avg -3×IFS std ;

[0078] IFS max =IFS avg +3×IFS std ;

[0079] It is understandable that in statistics, for a group of values ​​with linear fluctuations, the triple standard deviation method can be used to determine the values ​​in this group that have obvious differences (i.e., large fluctuations). In the present application, in adjacent displayed images, the frame intervals of the corresponding frame rate data of two groups of adjacent displayed images (such as the first frame image and the second image, and the second frame image and the third frame image) may also be different. Therefore, the frame intervals between the frame rate data corresponding to these images can also be regarded as a group of linearly fluctuating values. When calculating the estimated maximum value IFS of the frame intervals of the above multiple frame rate data max and the estimated minimum value IFS min If there are two adjacent images displayed later, the frame interval is greater than IFS. max or less than IFS min , it means that the frame interval of the frame rate data corresponding to the two images is obviously different from the frame interval of the previous multiple frame rate data, which also means that the video file may be abnormal during the subsequent playback process.

[0080] (5) Performance counters

[0081] Performance counter, also called performance monitor, is a system function provided by the operating system. It can collect and analyze the performance data related to applications, services, and drivers in the system in real time, so as to analyze the bottleneck of the system, monitor the performance of components, and finally help users to reasonably allocate the system. Performance counter can monitor multiple performance objects. The performance objects in a general system can include processors, processes, threads, network communications (such as TCP, UDP, ICMP, IP, etc.), system services (such as ACS / RSVP Service), etc.

[0082] Specifically, there is a graphics processing unit (GPU) engine counter in the performance counter, and there is a utilization percentage module in the GPU engine counter. For each process, the module can monitor the count values ​​of the video decoding module (videodecode) and the video processing module (videoprocessing) in the system in real time. When the count values ​​of these two modules are greater than 0, it means that the corresponding process is the process that is playing the video.

[0083] In daily use, users often use electronic devices such as desktop computers and laptops to play videos. However, during the video playback process, when the performance it can provide is not enough to meet the performance requirements of the current operation scenario of the electronic device, the video played by the electronic device is likely to drop frames, and the picture displayed by the electronic device will be stuck.

[0084] It is difficult for current electronic devices to uniformly monitor frame loss at the operating system level, and they are unable to identify video frame loss in a timely manner, so that in many cases timely intervention in the frame loss quadrant is not possible. In many cases, the system only responds after the user clearly feels that the video is stuck, which greatly damages the user experience.

[0085] In response to the above defects, the present application provides a method for determining frame loss, which analyzes the frame rate data of the played video to determine the interval range corresponding to the normal frame interval of the frame rate data; when the frame interval between a subsequent frame image and the previous frame image exceeds the interval range, the frame drawing capability required for the electronic device to maintain a normal frame rate is determined based on the frame interval between each frame rate data and the previous frame rate data, and when the frame drawing capability required by the electronic device exceeds the interval range corresponding to the normal frame interval, it is determined that the video process has a frame loss phenomenon. By implementing this method, the electronic device can promptly determine the frame loss phenomenon in the video process, and intervene in the frame loss phenomenon in time before the video process produces obvious freezes due to frame loss, thereby reducing the frame loss phenomenon of the video process and improving the user experience.

[0086] like Figure 1 As shown, the frame loss determination method provided in the present application may include the following steps:

[0087] S101: The electronic device determines a process number of a first process that is playing a video.

[0088] The electronic device may be a mobile phone, a vehicle-mounted device (such as an on-board unit (OBU)), a tablet computer (pad), a computer with data transceiver function (such as a laptop, a PDA, etc.), a mobile Internet device (MID), a terminal in a smart city, a terminal in a smart home, a terminal device in a 5G network, or a terminal device in a public land mobile network (PLMN) to be evolved in the future, etc. It is understandable that the present application does not limit the specific form of the electronic device.

[0089] Optionally, the electronic device may currently have multiple processes, which may include one or more processes that are playing videos and processes that are not playing videos. The first process may be any one of the one or more processes that are playing videos.

[0090] Specifically, the electronic device can monitor the performance data of the video decoding module and the video processing module through the utilization percentage module in the GPU engine counter in the performance counter of the system, and determine the process whose corresponding counts of these two modules are greater than 0 as the process that is playing the video. It can be understood that if the occupancy rate of the video decoding module and the video processing module of the process is greater than 0, it means that the process is using these two modules to encode, decode, transcode, color space and pixel conversion and other operations on the video, so the process is the process that is playing the video.

[0091] Figure 2 The figure shows the process of selecting the process that is playing the video from all the processes based on the utilization rate of the video decoding module and the video processing module of each process. Figure 2 As shown, the process currently to be processed by the electronic device may include Figure 2 The process shown is process p1, process p2, process p3, ... process pn, and at least one of these processes is playing a video. For any of the processes, the electronic device can query the performance counter for the occupancy rate of the video decoding module and the video processing module of the process, and will query the process with a video decoding module occupancy rate greater than 0 and a video processing module occupancy rate greater than 0, that is, Figure 2 The processes play1, play2, ..., playn in the video are the processes currently playing the video. The first process may be any one of the processes play1 to playn.

[0092] It should be noted that when the processes that are playing the video are screened out, the electronic device will simultaneously obtain and save the process numbers of these processes, so as to subsequently obtain the frame rate data of the corresponding processes based on the process numbers of these processes.

[0093] S102: The electronic device obtains N pieces of frame rate data of the first process, and determines a target number of frames, a target frame interval, a maximum frame interval, and a minimum frame interval based on the N pieces of frame rate data.

[0094] The N pieces of frame rate data are frame rate data corresponding to N consecutive frames of images in the first video played by the first process. Optionally, the specific value of N may be 60.

[0095] After the electronic device determines that the first process is the process that is playing the video from multiple processes to be processed, the electronic device can obtain instruction information corresponding to the first process (i.e., the event whose process number is the process number of the first process) from the event tracing for Windows (ETW), and the instruction information includes instruction information related to image processing.

[0096] It should be noted that ETW is a general event tracking system provided by the operating system of an electronic device to monitor the performance of a system with load. Figure 3 The ETW architecture diagram is shown. In the operating system 30, the entire ETW system 301 can be composed of three parts: producers (including producers 303, producers 304 and producers 305), consumers 306 and controllers 302, where:

[0097] The controller 302 is a controller. It has two main tasks: one is to control and manage the session control object. It is responsible for responding to the producer's registration request to create the session control object in the memory, and is also responsible for sending the instruction information (events) recorded in the session control object to the consumer. The second is to manage the producer, such as starting or stopping the producer. In order to avoid additional overhead, the producer will not work all the time, but only start working after being started.

[0098] The producer is the provider of command information (i.e., event), which can be a system component, driver, or developed application. Before generating command information, the producer needs to register a session control object with the system. The electronic device will allocate a corresponding cache area for each session control object. When the producer is started by the controller, the producer can start sending command information to the cache area in the corresponding session control object. Figure 3Taking producer 303 in as an example, session control object 3011 is registered by producer 303. When controller 302 starts producer 303, producer 303 will start to generate instruction information and write it into the cache of session control object 3011, waiting to be obtained and executed by consumer 306 later.

[0099] Consumer 306 is used to receive instruction information (events) provided by one or more session control objects. Specifically, consumer 306 can receive instruction information (events) transmitted to the log file by the session control object from the log file, or can receive instruction information (events) from the session control object in real time. When processing instruction information (events), consumer 306 can specify the start time and end time of receiving instruction information (events), and the producer can also only transmit instruction information events generated within the time range.

[0100] In this application, most of the instruction information (events) related to image processing in the first process are generated by the two modules DXGI or D3D. That is to say, in the ETW system, DXGI and D3D can act as producers to generate part of the instruction information (events) required to complete the first process. The electronic device can monitor the instruction information written into the ETW system by these two modules as producers, and obtain the frame rate data corresponding to the first process from the acquired instruction information based on the process number of the first process.

[0101] It should be understood that in the same process, there may be a situation where multiple videos are played at the same time, but the frame rate data corresponding to the images in different videos will be stored in different image processing caches. Therefore, after obtaining the frame rate data belonging to the above-mentioned first process, the electronic device also needs to determine the frame rate data with the same image cache number as the frame rate data corresponding to the image in the same video according to the image cache number contained in the frame rate data. Further, after obtaining multiple (≥N) frame rate data corresponding to multiple frames of images in the same video, the electronic device can arrange the frame rate data according to the order of the timestamps in the frame rate data, and take the continuous N frame rate data as the frame rate data corresponding to the N frames of the above-mentioned first process. That is to say, after determining all the processes for playing videos and obtaining the process numbers, the electronic device can obtain the corresponding frame rate data for any two-tuple (process number, image cache number), and from the multiple frame rate data obtained for the same two-tuple (process number, image cache number), select the continuous N frame rate data as the N frame rate data of the above-mentioned first process according to the order of the timestamps contained in each frame rate data. For ease of explanation, the following description is made by taking N pieces of frame rate data corresponding to an image in a first video played by a first process as an example.

[0102] Figure 4 The data information contained in each of the N frame rate data and the process of the electronic device determining the target number of frames, target frame interval, maximum frame interval and minimum frame interval according to the N frame rate data are exemplified.

[0103] like Figure 4 As shown in (A), any of the N frame rate data above contains a process ID (ProcessID), a frame ID (EventID), a frame generation timestamp (Timestamp), and an image processing cache ID (pSwapchain or pIDXGISwapChain). The frame rate data generated by DXGI is stored in a cache of the pSwapchain type, while the frame rate data generated by D3D is stored in a cache of the pIDXGISwapChain type.

[0104] Figure 4 (B) in FIG. 1 exemplarily shows the key information contained in the frame rate data with frame number 1 among the N pieces of frame rate data. Figure 4 As shown in (B), the frame rate data belongs to the process with process number "14216", which can be the first process mentioned above. The frame generation timestamp contained in the frame rate data is "133287886730547216"; it should be noted here that the frame generation timestamp in the frame rate data actually represents the generation time of the frame rate data, but it can be approximately regarded as the time consumed by the producer (that is, the time when the image corresponding to the frame rate data is displayed on the display screen). In some embodiments, in order to facilitate the comparison of the order of the times corresponding to the timestamps, Figure 4 The frame generation timestamp contained in the frame rate data (B) may also be in other formats, which is not limited in this application. The image processing cache number in which the frame rate data is stored is "0x173DDFACAC0". In addition to storing the frame rate data with frame number 1, the image processing cache may also store multiple other frame rate data, which correspond to multiple frames of images in the same video file.

[0105] It is assumed here that the frame rate data with a smaller frame number is generated earlier. Figure 4 (C) in the figure shows a table obtained by arranging multiple frame rate data according to the order of the timestamps in the frame rate data. It can be understood that the time corresponding to the timestamp TS1 is earlier than the time corresponding to the timestamp TS2, and the time corresponding to the timestamp TS2 is earlier than the time corresponding to the timestamp TS3, and so on. The timestamps contained in these frame rate data also determine the time when the corresponding image is finally displayed on the screen. The earlier the timestamp corresponds to, the earlier the corresponding image of the frame rate data will be displayed on the screen by the electronic device.

[0106] Optionally, in order to facilitate comparison of the order of times corresponding to the timestamps, the electronic device may also convert the frame generation timestamp included in the frame rate data into other formats. Figure 4 (D) in FIG. 1 shows a table obtained by arranging multiple frame rate data according to the order of the corresponding time of the converted timestamps after the electronic device converts the timestamps into other formats. Figure 4 Take the frame rate data of frame number 1 in the table shown in (D) as an example, and assume that the frame rate data is Figure 4 From the frame rate data with frame number 1 shown in (B), it can be seen that the original timestamp of the frame rate data is "133287886730547216", but after being converted by the electronic device, its timestamp is converted into another format, that is, Figure 4 "168814255.7563" as shown in (D) in the figure; it should be understood that the conversion of the timestamp format does not change the actual time corresponding to the timestamp, nor does it change the temporal relationship between the timestamp and other timestamps.

[0107] After arranging the multiple frame rate data obtained in chronological order, the electronic device can take N continuous frame rate data as the N frame rate data of the above-mentioned first process. It should be understood that the "continuous" mentioned here means that the image corresponding to the N frame rate data can constitute a video segment in the video file played by the above-mentioned first process, and this video segment has no frame loss. In order to avoid frame loss in the video segment corresponding to the N frame rate data, the electronic device can complete the acquisition operation of the above-mentioned N frame rate data under a high-performance supply working state. For example, the electronic device can perform the above-mentioned operation to obtain the above-mentioned N frame rate data when the maximum power supply of the CPU is large enough; it is understandable that when the maximum power supply of the CPU is large enough, the performance supply of the electronic device is sufficient, and the video playback is not prone to frame loss.

[0108] After obtaining the N pieces of frame rate data, the electronic device can calculate the frame drawing capacity required to complete the first process through the N pieces of frame rate data, that is, how many images the electronic device needs to draw (or display) on the screen per second on average without losing frames. Figure 4 As shown in (E), the electronic device can generate timestamps based on the frames included in the N pieces of frame rate data, and calculate the target number of frames, the average frame interval, the maximum frame interval, and the minimum frame interval, where:

[0109] The above average frame interval is the frame interval average value IFS of the above N frame rate data. avg , which is the average frame interval of the above N frame rate data, can be calculated as follows:

[0110]

[0111] Among them, IFS a，b Indicates the time interval between the frame generation timestamp of frame rate data a and the frame generation timestamp of frame rate data b, where frame rate data a and frame rate data b are two adjacent frame rate data, for example Figure 4 In (D), there are two pieces of frame rate data with frame number 1 and frame number 2.

[0112] It is not difficult to understand that based on the above average frame interval, the electronic device can calculate the frame rate when sending the first video in the first process for display, that is, the number of images that the electronic device needs to draw in one second, which is also the above target frame number f tgt , and its calculation method can be expressed as:

[0113]

[0114] in, Indicates floor operation. Represents the largest integer less than X.

[0115] The above target frame number f tgt It is calculated by the electronic device based on the N frame rate data obtained when no frame loss occurs in the electronic device. Therefore, it is not difficult to understand that the frame rate of the first video when it is played normally is f tgt , the frame rate of the first video is not less than f tgt In this case, the picture observed by the user is basically smooth, which will not affect the user's viewing experience.

[0116] In addition, based on the above IFS avg The standard deviation IFS of the frame interval of N frame rate data can be calculated std . Its calculation method can be expressed as:

[0117]

[0118] Furthermore, based on the above IFS std , the above maximum frame interval IFS can be calculated max and minimum frame interval IFS min , and its calculation method can be expressed as:

[0119] IFSS min =IFS avg -M×IFS std

[0120] IFS max =IFS avg +M×IFS std ;

[0121] Wherein, M can be any value between 1 and 10, and “×” represents a multiplication operation; specifically, the value of M can be 3.

[0122] It can be understood that the standard deviation is a quantitative indicator used to characterize the degree to which each data in the statistical data set deviates from the mean value. Mathematically, the M-fold standard deviation method to determine outliers is a commonly used data processing method, which involves two concepts: determining the "allowable range" of each sample based on the size of the overall sample standard deviation; sample values ​​that exceed the "allowable range" are outliers. The size of the overall sample standard deviation is related to the definition of outliers. Generally, three times the sample standard deviation is used as the sample allowable deviation range, that is, if a sample exceeds three times the standard deviation, the sample will be considered an outlier.

[0123] In the present application, the frame interval of the image corresponding to each two adjacent frame rate data in the above N pieces of frame rate data may not be a fixed value, but presents a set of linearly fluctuating values. Therefore, in the present application, after the electronic device calculates the above target frame number, average frame interval, maximum frame interval and minimum frame interval, the electronic device can set the above target frame number f tgt Determine the frame rate of the video played by the first process (that is, how many frames of images need to be displayed in one second when playing the video), and set the above target frame interval IFS avg Determine the ideal value (or standard value) for the display time interval between each subsequent frame image and the previous frame image, and set the above minimum frame interval IFS min and the maximum frame interval IFS max , which is used as a basis for determining whether the frame interval between subsequent adjacent displayed images is abnormal. Specifically, the electronic device may set the minimum frame interval IFS min And the above maximum frame interval IFS max The interval range composed of [IFS min , IFS max ] is used as the value range of the above-mentioned normal frame interval (that is, the "allowable range" in the above description). If the frame interval between two adjacent frames of images displayed subsequently does not fall within this range, it means that the frame interval between these two frames of images is obviously different from the frame interval between the previous adjacent images, which also means that an abnormality has occurred in the video playback process of the above-mentioned first process.

[0124] It is understandable that, when the video played by the first process is not interrupted, DXG or D3D, as a producer, will continuously generate corresponding instruction information (events). Assume that the N frame rate data are from the first video played in the first process, and the frame rate data generated by DXGI or D3D based on the first video are stored in the first cache. Then, when the number of frame rate data generated based on the first video in the first cache is less than the above N, the electronic device may not calculate the above target frame number f first. tgt , average frame interval IFS avg , Maximum frame interval IFS max And the minimum frame interval IFS min When the number of frame rate data generated based on the first video in the first buffer is greater than or equal to the above N, the electronic device selects N consecutive frame rate data from the first buffer for calculation.

[0125] In an optional implementation, the electronic device may determine N consecutive frame rate data selected from the first cache in a high-performance supply state as the N frame rate data, and determine the target frame number f based on the N frame rate data. tgt , average frame interval IFS avg , Maximum frame interval IFS max And the minimum frame interval IFS min Afterwards, if the frame rate data obtained in the first cache and the frame rate data before it fall within the interval [IFS min , IFS max ], the electronic device does not process it; if the frame rate data subsequently obtained from the first cache does not fall within the interval [IFS min , IFS max ], the electronic device can determine that an abnormality has occurred in the playback process of the above-mentioned first video, and the electronic device can execute the subsequent step S103.

[0126] In addition, since the electronic device may reduce the performance supply of the electronic device (for example, reducing the maximum power supply of the CPU) in order to reduce the power consumption of the device, this process may affect the playback of the video in the first process. Therefore, in another optional implementation, the frame interval between the latest frame rate data stored in the first cache and the previous frame rate data is a normal frame interval (i.e., falls within [IFS min , IFS max ]), the electronic device can continuously select the N pieces of frame rate data most recently stored in the first cache for calculation as the latest frame rate data in the first cache is written, so as to update the target frame number f tgt , average frame interval IFS avg , Maximum frame interval IFSmax And the minimum frame interval IFS min For example, assuming that the value of N is 60, when the electronic device is in a high-performance supply state, after DXGI or D3D stores the 60th frame rate data generated based on the first video into the first cache, the electronic device can use the 1st to 60th frame rate data in the first cache as the N frame rate data, and calculate the target frame number f based on the 1st to 60th frame rate data. tgt , average frame interval IFS avg , Maximum frame interval IFS max And the minimum frame interval IFS min Next, DXGI or D3D stores the 61st frame rate data generated based on the first video into the first cache (here, it is assumed that the time interval between the frame generation timestamps of the 61st frame rate data and the 60th frame rate data falls within the range [IFS min , IFS max ]), the electronic device may use the 2nd to 61st frame rate data in the first cache as the N frame rate data, and calculate and update the target frame number f based on the 2nd to 61st frame rate data. tgt , average frame interval IFS avg , Maximum frame interval IFS max And the minimum frame interval IFS min until the frame interval between the latest frame rate data written into the first cache and the previous frame rate data does not fall into the latest calculated interval [IFS min , IFS max ], the electronic device can determine that an abnormality has occurred in the playback process of the first video, and the electronic device can no longer use the N frame rate data most recently stored in the first cache to calculate and update the target frame number f tgt , average frame interval IFS avg , Maximum frame interval IFS max And the minimum frame interval IFS min value, but executes the subsequent step S103.

[0127] In addition, the acquisition process of the above-mentioned N frame rate data takes a certain amount of time. Therefore, in an optional embodiment, if the display process of the image in the above-mentioned first video on the screen is interrupted during the acquisition of the above-mentioned N frame rate data, the electronic device needs to stop the acquisition of the frame rate data and discard the previously acquired frame rate data, and restart the acquisition of the N frame rate data of the first video after the image of the subsequent first video continues to be continuously displayed on the screen. It should be noted that "the display process of the image on the screen is interrupted" can include the electronic device stopping playing the above-mentioned first video, and the playback window of the above-mentioned first video on the screen is completely blocked by other windows. For example, when the above-mentioned N is 60, the electronic device may take 2-3 seconds to successfully acquire 60 frame rate data corresponding to 60 frames of images in the first video. If the electronic device only acquires 30 frame rate data corresponding to 30 frames of images in the first video, the user pauses the first video, then the electronic device can stop acquiring the frame rate data of the above-mentioned first video, and after the user continues to play the first video, re-acquire the 60 frame rate data corresponding to the subsequent 60 frames of images (excluding the above-mentioned 30 frame rate data previously acquired).

[0128] S103. When the frame interval between the first frame rate data and the second frame rate data is greater than the above-mentioned maximum frame interval, the electronic device sets the corresponding remaining frame number for each frame rate data starting from the first frame rate data, and determines the target frame interval corresponding to each frame rate data based on the remaining frame number corresponding to each frame rate data and the frame interval between each frame rate and the previous frame rate data.

[0129] The first frame rate data is the frame rate data acquired after the N frame rate data, and the first frame rate data has the same process number and the same image processing cache number as the N frame rate data. The second frame rate data is the previous frame rate data adjacent to the second frame rate data. Specifically, the first frame rate data may be the first frame rate data acquired by the electronic device after the N frame rate data, and the frame interval between the first frame rate data and the adjacent frame rate data is greater than IFS. max frame rate data.

[0130] Starting from the first frame rate data, the number of remaining frames corresponding to the nth frame rate data obtained subsequently is f res_n The calculation method of is as follows:

[0131]

[0132] in, is the number of remaining frames corresponding to the previous frame rate data, f res_1 That is, the number of remaining frames corresponding to the first frame rate, and f res_1 =f tgt -1.

[0133] Starting from the first frame rate data, the target frame interval IFS corresponding to the nth frame rate data obtained subsequently tgt_n The calculation method of is as follows:

[0134]

[0135] Among them, IFS n IFS is the frame interval between the nth frame rate data and the previous frame rate data, starting from the first frame rate data. n1 This is the first frame rate data mentioned above.

[0136] Combined with the above description, it can be seen that when the frame interval between the subsequently acquired frame rate data and the previous frame rate data does not fall into the latest calculated interval [IFS min , IFS max ], the electronic device can determine that the video playback may be abnormal. Therefore, when the frame interval between the first frame rate data and the previous frame rate data is greater than the currently calculated IFS max When the first process plays a video, the electronic device may determine that frame loss may occur in the subsequent video.

[0137] It should be understood that it does not mean that as long as the frame interval between two subsequent frame rate data is greater than IFS max When the first video played by the first process is played, the electronic device can determine that frame loss will occur. In fact, the most important basis for the electronic device to determine whether the video playback is frame-dropped is whether the electronic device is capable of maintaining the frame rate of the video playback at f. tgt , that is, whether the electronic device is still able to draw f within 1s tgt For example, even if the frame interval between the first frame rate data and the second frame rate data is greater than IFS max , the frame rate data obtained later may also have frame intervals significantly smaller than IFS avg , or even less than IFS min Therefore, even if the frame interval between the first frame rate data and the second frame rate data is greater than IFS max In this case, the electronic device may still be able to maintain the frame rate at the target frame number f tgt , that is, the electronic device may still display f within 1 second tgt The electronic device cannot determine whether frame loss will occur during the playback of the first video.

[0138] Therefore, the frame interval between the first frame rate data and the second frame rate data is greater than the currently calculated IFS. maxThis is only the primary condition for the electronic device to determine whether the video has lost frames. In the subsequent process, it is also necessary to determine whether the electronic device is capable of maintaining the frame rate at the target frame rate f tgt , that is, whether the electronic device can continue to output f within 1 second tgt Frame image.

[0139] It should be noted that, for the frame rate data obtained after the above N pieces of frame rate data, if the corresponding target frame interval is smaller, the electronic device maintains the frame rate at the target frame number f. tgt The difficulty will also be greater.

[0140] by Figure 5 For convenience of explanation, it is assumed here that the average frame interval IFS calculated by the electronic device based on the above N pieces of frame rate data is avg is 0.041, and the target frame number f tgt It is 24 frames.

[0141] Figure 5 (A) shows the process of the electronic device displaying 24 frames of images during the period of 1 second from time t50 to time t51 (i.e., time length T0 is 1 second) during the process of playing the first video. Figure 5 In (A), the time when any frame of image is displayed on the screen can be approximately regarded as the frame generation timestamp of the frame rate data corresponding to the image; and the time interval between the time when any frame of image is displayed on the screen with the previous frame of image can be regarded as the frame interval between the frame rate data corresponding to the image and the frame rate data corresponding to the previous frame of image. For example, the frame rate data corresponding to image P1 is the same as that of image P0 (image P0 is the image displayed before image P1 and adjacent to image P1, Figure 5 The frame interval between the two corresponding frame rate data is Figure 5 Similarly, the frame interval between the two frame rate data corresponding to the image P2 and the frame rate data corresponding to the image P1 is Figure 5 The T2 duration shown in (A), and so on.

[0142] The electronic device can convert Figure 5 In (A), a total of 24 frames of images, images P1 to P24, are displayed on the screen of the electronic device in sequence. It should be noted that, ideally, the interval between the display moments of any two frames of images can be IFS. avg , that is, the values ​​of T1-T24 are all IFS avgHowever, in fact, when the first video is played normally, the time interval between the display moments of two different frames of images on the screen is likely to be different, but they are all within the interval [IFS min , IFS max ]. For example, the duration of T1 can be different from the duration of T2, and the duration of T3 can also be different from the duration of T4, and there is a period greater than IFS in T1-T24. avg There are also values ​​less than IFS avg In this case, the electronic device can display 24 frames of images, namely, image P1 to image P24, on the screen within 1 second.

[0143] However, when the interval between two adjacent frames of images displayed on the screen is obviously too large (greater than the maximum frame interval IFS), max ), then there is at least one frame of the two images, and the speed at which the electronic device draws the image is obviously slow, which means that if the electronic device needs to draw the image at a faster speed in the subsequent drawing process, it can ensure that it can display 24 frames of images on the screen in succession within 1 second.

[0144] like Figure 5 As shown in (B), Figure 5 The images P1'-P24' shown in (B) may be images in the first video, and the 24 frame rate data corresponding to the images P1'-P24' may be frame rate data acquired after acquiring the N frame rate data. It is assumed here that in the subsequent playback of the first video, the electronic device reduces the performance supply of the electronic device in order to reduce power consumption, resulting in the difference between image P1' and image P0' (image P0' is an image displayed before image P1' and adjacent to image P1', Figure 5 The time interval T1' between the frames is not shown in the figure, and T1' is greater than the above-mentioned maximum frame interval IFS. max .

[0145] Since the time interval between image P1' and image P0' is obviously too large, if the electronic device wants to display 24 frames of images on the screen in succession within the 1 second period from t50' to t51', the electronic device must display the subsequent 23 frames of images (i.e., image P2' to image P24') on the screen within the remaining period (1-T1'). In other words, if the electronic device can maintain the frame rate of the first video at 28 frames / second from t50' to t51', the average value of the display time interval between each two adjacent images in the subsequent 23 frames of images must be (1-T1') / 23, which is the target frame interval corresponding to the above-mentioned first frame rate data.

[0146] It is understandable that the frame rate data corresponding to the image P1' may be the first frame rate data mentioned above. Starting from the frame rate data corresponding to the image P1, the display process of each frame image will occupy part of the duration of t50'-t51', which will definitely affect the time used to display subsequent images on the screen. In other words, the frame interval between the frame rate data corresponding to each subsequent frame image and the frame rate data corresponding to the previous frame image will affect the possibility of the electronic device maintaining the frame rate at 24 frames / second. For example, assuming that the display time interval T2' between image P2' and image P1' is greater than (1-T1') / 23, it can be considered that the time spent by the electronic device to display image P2' on the screen exceeds the time originally planned. In order to keep the frame rate of the first video at 28 frames / second from time t50' to time t51', the electronic device needs to display the subsequent 22 frames of images (i.e., image P3'-image P24') on the screen within the remaining (1-T1'-T2') period of time. The average value of the display time interval between each two adjacent images in the subsequent 22 frames of images must be (1-T1'-T2) / 22, which is the target frame interval corresponding to the frame rate data corresponding to image P2'. Because T2' is greater than (1-T1') / 23. Simple mathematical reasoning shows that [(1-T1'-T2) / 22]<[(1-T1') / 23], which means that the electronic device needs to complete the display process of subsequent images at a faster speed, and the pressure on the performance of the electronic device is also greater.

[0147] S104: When the frame interval between the third frame rate data and the fourth frame rate data is greater than the target frame interval corresponding to the third frame rate data, and the target frame interval corresponding to the third frame rate data is less than the above minimum frame interval, the electronic device determines that the first video has a frame loss.

[0148] Combined with the above description, it can be known that for the above-mentioned first frame rate data and the above-mentioned frame rate data obtained after the above-mentioned first frame rate data, the smaller the target frame interval corresponding to the frame rate data, the more difficult it will be for the electronic device to maintain the frame rate at the target frame number. Therefore, in the embodiment of the present application, if there is a third frame rate data, and the previous frame rate data adjacent to the third frame rate data is the above-mentioned fourth frame rate data, if the frame interval between the third frame rate data and the fourth frame rate data is greater than the target frame interval corresponding to the fourth frame rate data, and the frame interval corresponding to the third frame rate data is less than the above-mentioned minimum frame interval IFS min When the electronic device determines that the current frame rendering capability (ie, the current performance supply level of the electronic device) cannot maintain the frame rate of the first video played by the first process at f tgt .

[0149] First, the target frame interval corresponding to the fourth frame rate data is the frame interval determined by the electronic device for the subsequent frame rate data based on the sum of the frame intervals of all previous frame rate data (from the first frame rate data to the fourth frame rate data) (that is, the time required to output the images corresponding to these frame rate data). When the frame interval between the third frame rate data and the fourth frame rate data (that is, the time taken to output the image corresponding to the third frame rate data) is greater than the target frame interval corresponding to the fourth frame rate data, it means that the electronic device needs to further speed up the display speed of subsequent images. Secondly, IFS min It is calculated by the electronic device based on the aforementioned N pieces of frame data when the performance of the electronic device is sufficiently high. This also means that even when the performance of the electronic device is sufficiently high, when playing the first video, the frame interval of the frame rate data corresponding to two consecutive frames of images may not be reduced to IFS. min When the target frame interval corresponding to the third frame rate data is too small, it means that the frame interval of the frame rate data before the frame rate data is too large. This is probably because the electronic device has reduced the performance supply level. For example, in order to reduce power consumption, the electronic device has reduced the maximum power that the CPU can supply. When the performance supply level is reduced, the electronic device will be even less able to reduce the frame interval between subsequent frame rate data to less than IFS. min Therefore, when determining that the target frame interval corresponding to the third frame rate data is less than IFS min After that, the electronic device may determine that the current frame drawing capability is insufficient (ie, the current electronic device cannot maintain the frame rate at f tgt ), so the video will drop frames during subsequent playback.

[0150] Figure 6 The specific process of calculating the remaining number of frames and the target frame interval corresponding to each frame rate data based on the subsequently collected frame rate data after the electronic device collects the first frame rate data is shown.

[0151] like Figure 6 As shown in (A), it is assumed here that the above N is 60, and the average frame interval IFS calculated by the electronic device based on the above N frame rate data avg is 0.041, and the target frame number f tgt The minimum frame interval is 24 frames. min 0.034, the maximum frame interval IFS max The calculation result can be that the electronic device is based on the frame rate data of frame number 1 to frame number 60 ( Figure 6 The frame rate data of the electronic device may be calculated based on the frame rate data of frame number 8 (some frame rate data are not shown in FIG. Figure 6 The frame rate data from (not shown) to frame number 67 is calculated.

[0152] exist Figure 6 In (B), the frame interval between the frame rate data with frame number 68 and the frame rate data with frame number 67 is 0.051. Since 0.051>IFS min =0.034, so the frame rate data with frame number 68 can be the above-mentioned first frame rate data, and the frame rate data with frame number 67 can be the above-mentioned second frame rate data. Combined with the above description, it can be known that the electronic device will use the frame generation timestamp of the frame rate data with frame number 67 as the start time of the segment with a duration of 1 second, and use the image corresponding to the above-mentioned first frame rate data as the first frame image output by the electronic device in the 1-second duration; in addition, the electronic device will also set the corresponding remaining number of frames for each frame rate data starting from the frame rate data with frame number 68, and determine the target frame interval corresponding to each frame rate data based on the remaining number of frames corresponding to each frame rate data and the frame interval between each frame rate and the previous frame rate data (the target frame interval corresponding to the frame rate data before the frame rate data with frame number 68 can all be set to 1FS max , that is, 0.048).

[0153] Taking the frame rate data with frame number 68 as an example, since the frame interval between it and the frame rate data with frame number 67 is 0.051, it means that it takes 0.051 seconds for the electronic device to display the image corresponding to the frame rate data with frame number 68 on the screen. If the electronic device wants to maintain a frame rate of 24 frames / second, the electronic device needs to complete the display of at least 23 subsequent frames of images within the subsequent (1-0.051)=0.949 seconds. It is not difficult to calculate that the corresponding average frame interval is 0.949÷23≈0.0413 seconds. Therefore, the remaining duration corresponding to the frame rate data with frame number 68 is 0.949 seconds, the remaining number of frames is 23 frames, and the corresponding target frame interval IFS tgt_68 The frame interval between the frame rate data with frame number 68 and the frame rate data with frame number 67 is greater than the target frame interval IFS corresponding to the frame rate data with frame number 67. max = 0.048 seconds, so in the column "Whether the current frame interval meets the target set by the previous frame", the label value corresponding to the frame rate data with frame number 68 is 0, indicating that the current frame interval is greater than the target frame interval set by the electronic device based on the previous frame data. In addition, since the target frame interval IFS corresponding to the frame rate data with frame number 68 is tgt_68 = 0.0413 seconds greater than the minimum frame interval IFS min =0.034 seconds, so in the column "Whether the next frame target interval meets the minimum frame interval value", the label value corresponding to the frame rate data with frame number 68 is 1.

[0154] Taking the frame rate data with frame number 76 as an example, since the frame interval between it and the frame rate data with frame number 75 is 0.051, it means that it takes 0.092 seconds for the electronic device to display the image corresponding to the frame rate data with frame number 76 on the screen. If the electronic device wants to maintain a frame rate of 24 frames per second, the electronic device needs to complete the display of at least 15 subsequent frames of images within the subsequent (0.643-0.092)=0.551 seconds. It is not difficult to calculate that the corresponding average frame interval is 0.551÷15≈0.0367 seconds. Therefore, the remaining duration corresponding to the frame rate data with frame number 76 is 0.551 seconds, the remaining number of frames is 15 frames, and the corresponding target frame interval IFS tgt_76 The frame interval between the frame rate data with frame number 76 and the frame rate data with frame number 75 is greater than the target frame interval IFS corresponding to the frame rate data with frame number 75. tgt_76 = 0.0402 seconds, so in the column "Whether the current frame interval meets the target set by the previous frame", the label value corresponding to the frame rate data with frame number 76 is 0, indicating that the current frame interval is greater than the target frame interval set by the electronic device based on the previous frame data. In addition, since the target frame interval IFS corresponding to the frame rate data with frame number 76 is tgt_76 = 0.0367 seconds greater than the minimum frame interval IFS min =0.034 seconds, so in the column of “Whether the next frame target interval meets the minimum frame interval value”, the label value corresponding to the frame rate data with frame number 76 is also 1.

[0155] In the case where the label values ​​of "Whether the current frame interval meets the target set for the previous frame" and "Whether the next frame target interval meets the minimum frame interval value" in the frame rate data are not both 0, the electronic device can continue to calculate the remaining number of frames and the corresponding target frame interval corresponding to the subsequent frame rate data, and examples are not given here one by one. Until the label values ​​of "Whether the current frame interval meets the target set for the previous frame" and "Whether the next frame target interval meets the minimum frame interval value" in a certain frame rate data are both 0, the electronic device can determine that the current frame drawing energy is insufficient.

[0156] Reference Figure 6 In (B), the frame rate data of frame number 81 and frame number 82, the frame rate data of frame number 82 can be the aforementioned third frame rate data, and the frame rate data of frame number 81 can be the aforementioned fourth frame rate data. Since the frame interval between the frame rate data of frame number 82 and the frame rate data of frame number 81 is 0.047, the frame interval is greater than the target frame interval IFS corresponding to the frame rate data of frame number 81. tgt_81= 0.0345 seconds, so in the column "Whether the current frame interval meets the target set for the previous frame", the label value corresponding to the frame rate data with frame number 82 is 0. In addition, since the target frame interval IFS corresponding to the frame rate data with frame number 82 is tgt_82 = 0.0331 seconds greater than the minimum frame interval IFS min = 0.034 seconds, so in the column "Whether the next frame target interval meets the minimum frame interval value", the label value corresponding to the frame rate data with frame number 82 is also 0. Combined with the above description, it can be seen that when the frame interval between the third frame rate data and the fourth frame rate data is greater than the target interval corresponding to the fourth frame rate data, and the frame interval corresponding to the third frame rate data is less than the above minimum frame interval IFS min When the electronic device determines that the current frame rendering capability (ie, the current performance supply level of the electronic device) cannot satisfy the requirement of maintaining the frame rate of the first video played by the first process at f tgt Therefore, when the electronic device determines that 0.047>IFS tgt_81 And IFS tgt_82 <IFS min Afterwards, the electronic device may determine that the current frame drawing capability is insufficient and frame loss may occur during video playback.

[0157] It is understandable that for all processes that are playing videos and all videos played by the processes, the electronic device can collect the frame rate data corresponding to the video, and determine whether the electronic device can maintain the frame rate as the frame rate of the video when it is played normally according to the above steps. When the electronic device cannot maintain the frame rate of any video when it is played normally, the electronic device will determine that the current frame drawing capability is insufficient, that is, the video played by the electronic device (at least one) will drop frames.

[0158] Optionally, the electronic device may report to the system the result of the determination that the device frame drawing capability is insufficient and the video playback will experience frame loss when N1 seconds before the end of the 1 second duration or when the remaining frame number is N2 frames, so that the electronic device can intervene in the frame loss phenomenon in time. Specifically, the value of N1 may be 0.3, and the value of N2 may be 5.

[0159] S105: When the first video loses frames, the electronic device increases the performance supply level.

[0160] Understandably, if the electronic device does not have sufficient frame drawing capabilities, and if the electronic device does not perform other interventions, in order to prevent the video image and sound from being out of sync during subsequent playback, the electronic device will skip the image that should have been displayed, that is, discard one or more frames of images, which is the frame loss phenomenon during video playback. Although this can keep the video image and sound synchronized, the user may also clearly feel that the image in the video will be incoherent. When the frame loss frequency is too high, the user may even feel that the video playback is obviously stuck.

[0161] Figure 7 The operation process of the CPU and GPU of the electronic device during video playback is shown.

[0162] like Figure 7 As shown in the figure, in the process of drawing the image in the video file to the screen of the electronic device, the CPU will collect the required mesh information (including vertex position information, normal direction, vertex color, etc.) and texture information from the hard disk, and set the rendering state (such as transparent or opaque, double-sided or single-sided, using fixed-point shader or fragment shading, etc.), and then call the DrawCall method to write all information (drawing instructions) into the buffer. After that, the GPU reads the relevant drawing instructions from it for rendering. When the CPU writes commands too slowly or the GPU renders too slowly, the frame interval between the frame rate data will become larger, resulting in frame loss during video playback.

[0163] Therefore, in an embodiment of the present application, when the frame drawing capability of an electronic device is insufficient, the electronic device may increase the performance supply level to reduce or avoid frame loss during subsequent video playback.

[0164] Optionally, the electronic device can increase the maximum power supply PL1 (Package Long-duration Power Limit) or the maximum operating frequency (Max Frequency) of the CPU; optionally, the electronic device can also adjust the maximum duration of the CPU power temporarily breaking through the power consumption wall (i.e., the maximum power supply of the CPU) to improve the command processing performance of the CPU.

[0165] Optionally, the electronic device may also increase the maximum power or maximum frequency of the GPU to improve the rendering performance of the GPU.

[0166] Optionally, the electronic device may also adjust the time rate that the CPU can allocate to Draw Call to improve command processing performance.

[0167] Optionally, for images with the same rendering status, the electronic device packages the instruction information corresponding to multiple images with the same rendering status into one instruction. Then the electronic device only needs to call Draw Call to execute one instruction.

[0168] Optionally, the electronic device may also choose other methods to increase the performance of the device to increase the speed of writing commands by the CPU and / or increase the rendering speed of the GPU, which is not limited in this application.

[0169] Since it takes a certain amount of time for the electronic device to adjust the performance supply level, in an optional implementation, after the electronic device reports the determination result that frame loss may occur to the system, the electronic device can suspend obtaining the frame rate data of the process that is playing the video, and no longer calculate the frame drawing capacity required for each video. After N3 seconds, the electronic device can continue to re-acquire the frame rate data of the process that is playing the video, and continue to calculate the frame drawing capacity required for each video according to the aforementioned method.

[0170] Optionally, the electronic device may not update the target number of frames, the target frame interval, the maximum frame interval, and the minimum frame interval. The electronic device may set the target frame interval corresponding to the first frame rate data (the frame rate data is the frame rate data of the first video) obtained after N3 seconds to the above-mentioned IFS max , and the frame interval between a subsequent frame rate data and the previous frame rate data is greater than IFS max In this case, the frame intervals of adjacent frame rate data are used to calculate the remaining number of frames and the target frame interval corresponding to each frame rate data, so as to determine the video frame loss. Alternatively, the electronic device can, after the performance supply level is increased, reselect the N frame data of the first video obtained after the above N3 seconds to update the above target number of frames, target frame interval, maximum frame interval and minimum frame interval. After that, the electronic device can continue to obtain the frame rate data of the first video and determine whether the first video has frame loss according to the operations described in the above steps S103-S104. In this way, even in an operating environment where the performance supply level is constantly changing, the electronic device can still promptly determine that frame loss will occur during the video playback process, and intervene in the frame loss phenomenon in time before the video process produces obvious freezes due to frame loss, thereby reducing the frame loss phenomenon of the video process and improving the user experience.

[0171] Next, the electronic device provided by this application is introduced.

[0172] The electronic device may be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA) or a dedicated camera (such as a SLR camera, a card camera), etc. The present application does not impose any restrictions on the specific type of the electronic device.

[0173] Figure 8 The structure of the electronic device is exemplarily shown.

[0174] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an audio module 170, a speaker 170A, a display screen 194, etc.

[0175] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0176] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0177] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0178] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0179] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0180] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the display screen 194, etc. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0181] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel.

[0182] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0183] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.

[0184] The electronic device 100 can implement audio functions, such as playing videos, through the audio module 170, the speaker 170A, the display screen 194 and the application processor.

[0185] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0186] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0187] The internal memory 121 and / or the memory in the processor 110 may be used to store instruction information generated by the producer in the process, such as frame rate data generated by the producer DXGI module and / or D3D module in the process of playing a video.

[0188] When the processes of the electronic device 100 include a process that is playing a video, the processor 110 can collect N frame rate data corresponding to the first video in the process that is playing the video when the performance supply of the electronic device is large enough, and calculate the target number of frames, target frame interval, maximum frame interval and minimum frame interval of the corresponding video based on the N frame rate data.

[0189] The processor 110 is also used to continue to obtain the frame rate data corresponding to the above-mentioned first video after obtaining the above-mentioned N frame rate data. When the frame interval between the first frame rate data and the previous frame rate data is greater than the above-mentioned maximum frame interval, starting from the first frame rate data, the corresponding remaining frame number is set for each frame rate data, and the target frame interval corresponding to each frame rate data is determined based on the remaining frame number corresponding to each frame rate data and the frame interval between each frame rate and the previous frame rate data. When the frame interval between the second frame rate data and the third frame rate data is greater than the target frame interval corresponding to the second frame rate data, and the target frame interval corresponding to the second frame rate data is less than the above-mentioned minimum frame interval, the processor 110 can determine that the current frame drawing capability is insufficient and that frame loss will occur during the playback of the first video. The processor 110 is also used to adjust the maximum power that can be supplied by the processor 110 when it is determined that frame loss will occur during the playback process of the video. For details, please refer to the aforementioned Figure 1 The relevant instructions will not be repeated here.

[0190] An embodiment of the present application also provides an electronic device, which includes: one or more processors and a memory; wherein the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method shown in the aforementioned embodiment.

[0191] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0192] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0193] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. A method for determining frame loss, characterized in that: include: Determine a target frame number, a maximum frame interval, and a minimum frame interval based on multiple pieces of frame rate data of the first video, wherein the target frame number is the average number of images displayed per second when the electronic device plays the first video, and the minimum frame interval and the maximum frame interval are used to determine whether the frame interval between two adjacent frame data is abnormal; In the case where the frame interval between the first frame rate data and the second frame rate data is greater than the maximum frame interval, respectively determining a target frame interval corresponding to each frame rate data after the second frame rate data, the target frame interval being determined based on the target number of frames and the frame interval between each frame rate data and the second frame rate data; the first frame rate data is the frame rate data acquired after the plurality of frame rate data are acquired, and the second frame rate data is the previous frame rate data adjacent to the first frame rate data; When the frame interval between the third frame rate data and the fourth frame rate data is greater than the target frame interval corresponding to the fourth frame rate data, and the target frame interval corresponding to the third frame rate data is less than the minimum frame interval, it is determined that the first video has lost a frame, and the fourth frame rate data is the frame rate data acquired after the multiple frame rate data are acquired, and the fourth frame rate data is the previous frame rate data adjacent to the third frame rate data.

2. The method according to claim 1, characterized in that Before determining the target number of frames, the maximum frame interval, and the minimum frame interval of the first video based on the plurality of frame rate data of the first video, the method includes: When the maximum power that can be supplied by the CPU of the electronic device is greater than a first threshold, a plurality of frame rate data of the first video are obtained.

3. The method according to claim 1 or 2, characterized in that: Before determining the target number of frames, the maximum frame interval, and the minimum frame interval of the first video based on the plurality of frame rate data of the first video, the method further includes: Determine at least one process for playing a video from a plurality of processes, and obtain a process number of each process in the at least one process for playing a video, wherein the at least one process for playing a video includes a first process, and the first process is a process for playing the first video; Acquire multiple pieces of instruction information of the first process based on the process number of the first process, wherein any piece of instruction information of the multiple pieces of instruction information includes a cache number storing the any piece of instruction information; A plurality of continuous instruction information in which the cache numbers in the instruction information are all the first cache numbers are used as a plurality of frame data of the first video.

4. The method according to any one of claims 1 to 3, characterized in that: After determining that the first video has lost a frame, the method further includes: Increase the maximum power that the CPU can supply or the maximum operating frequency of the CPU; And / or increase the GPU operating power or GPU operating frequency.

5. The method according to any one of claims 1 to 3, characterized in that: After determining that the first video has lost a frame, the method further includes: Stop acquiring the frame rate data of the first video; After the first time period, continue to obtain the frame rate data of the first video, Determine whether the first video has frame loss based on frame rate data of the first video acquired after the first duration.

6. The method according to any one of claims 1 to 5, characterized in that: The display process of the multiple frames of images corresponding to the multiple pieces of frame rate data on the screen of the electronic device is not interrupted.

7. The method according to any one of claims 1 to 6, characterized in that: The target number of frames is determined based on the average frame interval of the first video, and the maximum frame interval and the minimum frame interval are determined based on the frame interval standard deviation. The average frame interval is the average value of the frame intervals between all adjacent frame rate data in the multiple frame rate data, and the frame interval standard deviation is the standard deviation of the frame intervals between all adjacent frame rate data in the multiple frame rate data.

8. An electronic device, characterized in that: The electronic device comprises: one or more processors, a memory and a display screen; The memory is coupled to the one or more processors, and the memory is used to store computer program codes, wherein the computer program codes include computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method according to any one of claims 1 to 7.

9. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the processor is used to call computer instructions so that the electronic device executes the method as described in any one of claims 1-7.

10. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is caused to execute the method as claimed in any one of claims 1 to 7.

Images