Video lag detection method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a video lag detection method and device, electronic equipment and a storage medium. The method comprises the following steps: detecting the number of residual video frames in a target cache queue, wherein the target cache queue is used for caching received video frames to be played by a target video playing end; the duration of the cache queue empty state of the target cache queue is determined, the cache queue empty state is the state that the number of residual video frames in the target cache queue is smaller than the preset number of video frames, and the preset number of video frames is 1; and performing video lag detection on the target video playing end based on the duration of the empty state of the cache queue. According to the scheme, whether the video frame cached in the cache queue is empty or not is detected, and the video lag degree is finely judged according to the duration of the cache queue empty state in which the number of the cached video frames is empty, so that the problem of inaccurate judgment of video lag according to network parameters is solved; and more memory overhead does not need to be occupied.

Description

Technical Field

[0001] The present invention relates to the technical field of video data processing, and particularly relates to a video stuttering detection method, device, electronic device, and storage medium. Background Art

[0002] In a video detection system, a video acquisition end acquires video data and sends the video data to a video playback end through a network. The video playback end decodes and plays the video data and presents it to the user for viewing.

[0003] When the video is played on the video playback end, video stuttering may occur, that is, the video picture is not smooth. Detecting the stuttering phenomenon during the video playback can be used to improve the data monitoring of the overall system, and the video acquisition parameters can also be adjusted or the video link can be switched according to the stuttering situation. However, the related solutions only judge whether the video is stuttering by detecting whether the actual playback frame rate is consistent with the expected playback frame rate, and cannot distinguish between minor stuttering and severe stuttering, resulting in inaccurate judgment of the stuttering situation when the video stutters. Summary of the Invention

[0004] The present invention provides a video stuttering detection method, device, electronic device, and storage medium to solve the problem of inaccurate judgment of video stuttering based on network parameters, and detect different stuttering degrees according to the depletion situation of video frames in the video frame buffer queue in actual applications.

[0005] In a first aspect, an embodiment of the present invention provides a video stuttering detection method, the method including:

[0006] Detecting the remaining number of video frames in a target buffer queue, where the target buffer queue is used to buffer video frames to be played by a target video playback end;

[0007] Determining the duration of the depletion state of the buffer queue of the target buffer queue, where the depletion state of the buffer queue is a state where the remaining number of video frames in the target buffer queue is less than a preset number of video frames, and the preset number of video frames is 1;

[0008] Performing video stuttering detection on the target video playback end based on the duration of the depletion state of the buffer queue, where different durations of the depletion state of the buffer queue correspond to different video stuttering degrees.

[0009] In a second aspect, an embodiment of the present invention further provides a video stuttering detection device, the device including:

[0010] A frame number detection module, configured to detect the remaining number of video frames in a target buffer queue, where the target buffer queue is used to buffer video frames to be played by a target video playback end;

[0011] A determination module, configured to determine the duration of the empty state of the target cache queue, where the empty state of the cache queue is a state in which the remaining number of video frames in the target cache queue is less than a preset number of video frames, and the preset number of video frames is 1;

[0012] A stutter detection module, configured to perform video stutter detection on the target video playback end based on the duration of the empty state of the cache queue, and different durations of the empty state of the cache queue correspond to different degrees of video stutter.

[0013] In a third aspect, an electronic device is further provided in an embodiment of the present invention. The electronic device includes:

[0014] At least one processor; and

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor is enabled to execute the video stutter detection method described in any one of the above embodiments.

[0017] In a fourth aspect, a computer-readable medium is further provided in an embodiment of the present invention. The computer-readable medium stores computer instructions, and when the computer instructions are executed by a processor, the video stutter detection method described in any one of the above embodiments is implemented.

[0018] In the embodiment of the present invention, the remaining number of video frames in the target cache queue is detected. The target cache queue is used to cache the video frames to be played by the target video playback end; the duration of the empty state of the target cache queue is determined. The empty state of the cache queue is a state in which the remaining number of video frames in the target cache queue is less than a preset number of video frames, and the preset number of video frames is 1; video stutter detection is performed on the target video playback end based on the duration of the empty state of the cache queue, and different durations of the empty state of the cache queue correspond to different degrees of video stutter. This solution detects whether stuttering occurs based on whether the video frames cached in the cache queue are emptied, and makes a refined determination of the degree of video stuttering based on the duration of the empty state of the cache queue in which the cached video frames are emptied, solves the problem that it is inaccurate to judge video stuttering based on network parameters, and does not require a large amount of memory overhead.

[0019] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In combination with the accompanying drawings and with reference to the following specific embodiments, the above and other features, advantages and aspects of the embodiments of the present invention will become more apparent. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic and the original elements and elements are not necessarily drawn to scale.

[0021] Figure 1 is a schematic flowchart of a video stuttering detection method provided by an embodiment of the present invention;

[0022] Figure 2 is another schematic flowchart of a video stuttering detection method provided by an embodiment of the present invention;

[0023] Figure 3 is a schematic diagram of the principle of a video stuttering detection provided by an embodiment of the present invention;

[0024] Figure 4 is still another schematic flowchart of a video stuttering detection method provided by an embodiment of the present invention;

[0025] Figure 5 is a schematic diagram of the operating principle of a stuttering sample analysis queue in video stuttering detection provided by an embodiment of the present invention;

[0026] Figure 6 is a schematic diagram of the operating principle of a state machine in video stuttering detection provided by an embodiment of the present invention;

[0027] Figure 7 is a schematic diagram of the structure of a video stuttering detection device provided by an embodiment of the present invention;

[0028] Figure 8 is a schematic diagram of the structure of an electronic device for implementing a video stuttering detection method provided by an embodiment of the present invention. Specific Embodiments

[0029] The embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not used to limit the protection scope of the present invention.

[0030] It should be understood that the various steps recorded in the method embodiments of the present invention can be executed in different orders and / or executed in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this regard.

[0031] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0032] It should be noted that the concepts such as "first", "second", etc. mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0033] It should be noted that the modification of "one" and "a plurality of" mentioned in the present invention is illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly specified in the context, it should be understood as "one or more".

[0034] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0035] Figure 1 The figure is a schematic flowchart of a video stutter detection method provided by an embodiment of the present invention. The embodiments of the present invention are applicable to the situation of video stutter detection during video playback. This method can be executed by a video stutter detection device, which can be implemented in the form of software and / or hardware and is generally integrated in any electronic device with network communication functions. The electronic device can be a mobile terminal, a PC or a server, etc.

[0036] As Figure 1 shown, the video stutter detection method of the embodiment of the present invention may include the following processes:

[0037] S110. Detect the remaining number of video frames in the target cache queue, where the target cache queue is used to cache the video frames to be played by the target video player.

[0038] The video acquisition end can acquire video frame data and send the acquired video frame data to the target cache queue corresponding to the target video player for video frame caching through the network. Subsequently, the target video player can sequentially obtain video frame data from the target cache queue for video playback.

[0039] When the number of video frames for playback in the target cache queue is sufficient, the target video player can smoothly obtain video frames from the target cache queue for playback, and generally there will be no stuttering. However, when the number of video frames for playback in the target cache queue is insufficient, for example, when the video frames cached in the target cache queue are exhausted by the target video player (the number of cached video frames is zero), at this time, the target video player cannot continue to obtain video frames from the target cache queue. Once the video frames cannot reach the target video player in time, it will cause stuttering when the target video player plays the video.

[0040] Based on the above analysis, periodically detect the number of remaining video frames in the target cache queue, and determine whether the number of remaining video frames in the target cache queue is less than a preset number of video frames (for example, the preset number of video frames is 1), and the preset number of video frames is 1. If it is detected that the number of remaining video frames in the target cache queue begins to be less than the preset number of video frames, it is considered that the video frames to be used for playback in the target video player in the target cache queue have been exhausted by the target video player; if it is detected that the number of remaining video frames in the target cache queue has not begun to be less than the preset number of video frames, it is considered that the video frames to be used for playback in the target video player in the target cache queue have not been exhausted by the target video player.

[0041] S120. Determine the duration of the cache queue exhaustion state of the target cache queue. The cache queue exhaustion state is the state where the number of remaining video frames in the target cache queue is less than the preset number of video frames, and the preset number of video frames is 1.

[0042] Optionally, the preset number of video frames is 1 or a preset value, and the preset value can be set based on the value of the influence of the number of video frames in the target cache queue on video stuttering.

[0043] By detecting the number of remaining video frames in the target cache queue, if it is detected that the number of remaining video frames in the target cache queue begins to be less than the preset number of video frames, at this time, the remaining video frames in the target cache queue have been exhausted. At the same time, define the continuous exhaustion of video frames in the target cache queue as stuttering. The cache queue exhaustion state can be the state where the video frames in the target cache queue are exhausted. At this time, if video frames are not cached and replenished in the target cache queue in time, then the target video player will have video stuttering because it cannot continue to obtain video frames from the target cache queue.

[0044] Meanwhile, when the video frames in the target cache queue continue for a certain duration after being emptied, the video picture will inevitably freeze. Therefore, it is possible to determine whether video stuttering will occur during video playback on the target video player by analyzing whether the remaining number of video frames in the target cache queue has been emptied. Meanwhile, for the target cache queue of the target video player, when it is detected that the remaining number of video frames in the target cache queue starts to be less than the preset number of video frames, the duration during which the remaining number of video frames in the target cache queue is less than the preset number of video frames can be continuously counted to obtain the duration of the cache queue empty state persistence of the target cache queue.

[0045] S130. Perform video stuttering detection on the target video player based on the duration of the cache queue empty state persistence. Different durations of the cache queue empty state persistence correspond to different degrees of video stuttering.

[0046] If the duration of the cache queue empty state persistence in which the remaining number of video frames in the target cache queue is less than the preset number of video frames is longer, it means that the target cache queue cannot supply video frames to the target video player for a long time, and the severity of the video stuttering event generated by the target video player is higher; if the duration of the cache queue empty state persistence in which the remaining number of video frames in the target cache queue is less than the preset number of video frames is shorter, it means that the target cache queue can only not supply video frames to the target video player for a short time and quickly resumes supply, and the severity of the video stuttering event generated by the target video player is lower.

[0047] With the above solution, it is detected whether stuttering occurs according to whether the video frames cached in the cache queue are emptied, and the degree of video stuttering is finely determined and analyzed according to the duration of the cache queue empty state persistence of the emptied cached video frames, solving the problem of inaccurate judgment of video stuttering based on network parameters.

[0048] As an optional but non-limiting implementation, performing video stuttering detection on the target video player based on the duration of the cache queue empty state persistence includes steps A1 - A2:

[0049] Step A1. Detect whether the duration of the cache queue empty state persistence is greater than the first stuttering detection duration threshold, and the first stuttering detection duration threshold is used to measure whether the cache queue empty state of the target cache queue will cause video stuttering of the target video player.

[0050] The target cache queue caches the video frame data required for the video playback operation of the target video playback end. When it is detected that the remaining number of video frames in the target cache queue starts to be less than the preset number of video frames, it indicates that the video frames in the target cache queue have been exhausted. However, it is caused by the target video playback end obtaining video frames from the target cache queue, so at this time, the target video playback end can still play the video normally without video stuttering. Only when the video frames in the target cache queue remain exhausted for a certain period of time after being exhausted will the target video playback end experience video stuttering.

[0051] Therefore, a first stuttering detection duration threshold can be pre-configured. By comparing the duration of the cache queue exhaustion state with the first stuttering detection duration threshold, it can be determined whether the cache queue exhaustion state of the target cache queue will cause video stuttering of the target video playback end during video playback. In this way, it is possible to effectively distinguish the situation where the video frames in the target cache queue are exhausted for a short time but will be quickly replenished later, and will not cause video stuttering of the target video playback end, avoiding false detection of video stuttering.

[0052] Step A2: If the duration of the cache queue exhaustion state is greater than the first stuttering detection duration threshold, determine the video stuttering degree of the target video playback end based on the duration of the cache queue exhaustion state and the second stuttering detection duration threshold, so as to realize the detection of video stuttering of the target video playback end. The first stuttering detection duration threshold is less than the second stuttering detection duration threshold. The second stuttering detection duration threshold is used to distinguish different video stuttering degrees caused by the cache queue exhaustion state of the target cache queue in the target video playback end. Different durations of the cache queue exhaustion state correspond to different video stuttering degrees.

[0053] When it is detected that the duration of the cache queue exhaustion state is not greater than the first stuttering detection duration threshold, it indicates that although the video frames in the target cache queue have been exhausted, since the target video playback end has obtained a part of the video frames from the target cache queue, only quick replenishment is needed after being exhausted for a short time, and it will not cause video stuttering of the video playback end. Therefore, there is no need to continue analyzing the stuttering degree level.

[0054] When it is detected that the duration of the empty state of the cache queue is greater than the first card detection duration threshold, it indicates that the video frames in the target cache queue have been emptied for a short period of time. At this time, the video frames fetched by the target video player from the target cache have also been consumed. If video frames still cannot be obtained from the target cache, video stuttering of the video player will inevitably occur. At this time, a second card detection duration threshold can be obtained in advance, and by comparing the duration of the empty state of the cache queue with the second card detection duration threshold, different video stuttering degrees caused by the empty state of the cache queue of the target cache queue at the target video player can be judged.

[0055] By adopting the above optional method, it is possible to accurately detect the stuttering that significantly affects the video viewing quality, and to exclude the influence of minor stuttering that does not affect the overall viewing quality, so as to avoid misidentifying the empty situation that only causes the cache to be emptied but does not cause video stuttering as stuttering; at the same time, it can also distinguish different types of video stuttering degrees, which helps to make a correct judgment for dealing with video stuttering.

[0056] The technical solution of the embodiment of the present invention detects whether stuttering occurs according to whether the video frames cached in the cache queue are emptied, and makes a refined determination of the stuttering degree of the video according to the duration of the empty state of the cache queue where the cached video frames are emptied, solves the problem that it is inaccurate to judge video stuttering based on network parameters, and does not require a large amount of memory overhead. Moreover, this solution only needs to detect the number of video frames in the cache queue used to store the video frames required by the video player within a preset duration, define the situation where the number of video frames in the cache queue is continuously emptied as stuttering, and set the stuttering detection duration threshold in combination with the actual usage scenario, so as to flexibly and accurately detect video stuttering without consuming a large amount of memory overhead.

[0057] Figure 2 It is a schematic flowchart of another video stuttering detection method provided by the embodiment of the present invention. The technical solution of this embodiment further optimizes the process of determining the video stuttering degree of the target video player based on the duration of the empty state of the cache queue and the second stuttering detection duration threshold on the basis of the technical solution of the foregoing embodiment. This embodiment can be combined with each optional solution in one or more of the above embodiments.

[0058] As Figure 2 shown, the video stuttering detection method of the embodiment of the present invention may include the following process:

[0059] S210. Detect the remaining number of video frames in the target cache queue, where the target cache queue is used to cache the video frames to be played by the target video player.

[0060] S220: Determine the duration of the cache queue empty state of the target cache queue, where the cache queue empty state is a state where the number of remaining video frames in the target cache queue is less than a preset number of video frames, where the preset number of video frames is 1.

[0061] S230, detecting whether the duration of the cache queue empty state is greater than a first freeze detection duration threshold, where the first freeze detection duration threshold is used to measure whether the cache queue empty state of the target cache queue will cause video freeze at the target video playback end.

[0062] S240. If the duration of the cache queue empty state is greater than the first jamming detection duration threshold, detect whether the duration of the cache queue empty state is greater than the second jamming detection duration threshold. The first jamming detection duration threshold is less than the second jamming detection duration threshold. The second jamming detection duration threshold is used to distinguish different video jamming degrees caused by the cache queue empty state of the target cache queue at the target video playback end. Different cache queue empty state durations correspond to different video jamming degrees.

[0063] See also Figure 3 When it is detected that the duration of the empty state of the cache queue is longer than the first jam detection duration threshold, it indicates that the video frames in the target cache queue have been exhausted for a long time. At this time, the video frames taken from the target cache by the target video player have also been consumed. If the video frames still cannot be obtained from the target cache, it will inevitably cause video jams on the video player. At this time, it is possible to continue to detect whether the duration of the empty state of the cache queue is longer than the second jam detection duration threshold, so as to analyze the video jam level of the video jams on the target video player.

[0064] S250. If the duration of the empty state of the cache queue is greater than the second freeze detection duration threshold, it is determined that the target video playback end has generated a first level of video freeze. The first level of video freeze is a single freeze of the video screen during the video playback process of the target video playback end that lasts longer than a preset freeze duration.

[0065] See also Figure 3 If it is detected that the cache queue is empty for a duration greater than the second freeze detection duration threshold, it indicates that the target video player has a long single freeze duration, and the single video freeze has had a serious impact on the normal video playback of the target video player. At this time, it can be determined that the target video player has a first level of video freeze, that is, severe video freeze.

[0066] S260. If the duration of the cache queue depletion state is between the first video freeze detection duration threshold and the second video freeze detection duration threshold, combine the start time of the cache queue depletion state and the duration of the cache queue depletion state of the cache queue depletion state into a video freeze sample and write it into the freeze sample analysis queue.

[0067] See Figure 3 , if it is detected that the duration of the cache queue depletion state is not greater than the second video freeze detection duration threshold, but is between the first video freeze detection duration threshold and the second video freeze detection duration threshold, it indicates that the duration of a single video freeze of the target video player is not very long, and the impact of a single video freeze on the normal video playback of the target video player is not very serious. However, if this situation occurs multiple times, it will have a relatively serious impact on the normal video playback of the target video player.

[0068] Therefore, the start time T s (the start time of the cache queue depletion state when the video frames in the target cache queue are depleted) of the cache queue depletion state and the duration T d (the duration of the depletion of video frames in the target cache queue) of the cache queue depletion state can be determined, and the start time T s of the cache queue depletion state corresponding to the cache queue depletion state and the duration T d are combined into a video freeze sample (T s , T d ) and added to the freeze sample analysis queue to enable the aggregation and analysis of the start time T s of the cache queue depletion state corresponding to the cache queue depletion state statistically at different times and the duration T d of the cache queue depletion state.

[0069] S270. Based on the cumulative value of the durations of the cache queue depletion states corresponding to each freeze sample stored in the freeze sample analysis queue at the current moment, determine whether the target video player has a second-level video freeze. The second-level video freeze means that the frequency of video frame freezes during the video playback of the target video player is greater than the preset freeze frequency.

[0070] See Figure 3 , the freeze sample analysis queue can be used to store all freeze samples within the freeze detection period duration T set and calculate the cumulative freeze duration T set of all freeze samples within the freeze detection period T sum . For the n freeze samples in the freeze sample analysis queue, in the order of being stored in the freeze sample analysis queue, they are expressed as {T s0 , T d0}, {T s1 , T d1}, {T s2 , T d2}, ……, {T sn , T dn}, {T s0 , T d0}, {T sn , T dn} represents the earliest stored stuttering sample, and {T set , T dn} represents the latest stored stuttering sample. In this way, it is possible to determine whether a large number of stuttering samples frequently appear within the stuttering detection cycle duration T set by analyzing the stuttering sample analysis queue, so as to judge whether the target video playback end generates a second-level video stutter, that is, to judge whether the target video playback end generates frequent video stutters.

[0071] Optionally, the duration of the empty state of the cache queue of all stuttering samples stored in the stuttering sample analysis queue within the stuttering detection cycle duration T set is cumulatively calculated to obtain the cumulative value of the duration of the empty state of the cache queue corresponding to the stuttering samples within the stuttering detection cycle duration T set ; furthermore, based on the cumulative value of the duration of the empty state of the cache queue corresponding to the stuttering samples within the stuttering detection cycle duration T set , it is determined whether the target video playback end generates a second-level video stutter.

[0072] By adopting the above solution, stuttering detection and judgment are performed by caching video frames in the cache queue, historical stuttering sample data is recorded, and stuttering judgments of different degrees are made. Without sacrificing additional efficiency or memory space, relatively accurate stuttering detection and judgment are achieved; by detecting whether the duration of the empty state of the cache queue for a single stutter exceeds the preset stuttering detection duration threshold, it is judged whether a serious stutter occurs, and the stuttering detection duration threshold is set in combination with the actual usage scenario to make the detection result more in line with the actual situation; and by statistically analyzing the stuttering samples within a preset time range in the stuttering sample analysis queue, it is judged whether frequent stuttering occurs, and the stuttering detection duration threshold and the stuttering detection cycle duration are set in combination with the actual usage scenario to make the detection result more in line with the actual situation.

[0073] As an optional but non-limiting implementation manner, after performing video stuttering detection on the target video playback end based on the duration of the empty state of the cache queue, the following steps B1 - B2 are further included:

[0074] Step B1. If a first-level video freeze occurs at the target video playback end, select a target video transmission link from multiple candidate video transmission links corresponding to the target video playback end, and use the target video transmission link to write the video frames to be played by the target playback end into the target cache queue. The target video transmission link is the video transmission link with the highest video transmission performance among the multiple candidate video transmission links. The first-level video freeze means that the single freeze duration of the video picture during the video playback at the target video playback end is greater than the preset freeze duration.

[0075] Step B2. If a second-level video freeze occurs at the target video playback end, send encoding configuration information to the target video capture end corresponding to the target video playback end, so that the target video capture end performs video capture according to the new encoding configuration information and sends video frames to the target video playback end. The encoding configuration information is used to configure video resolution, video frame rate, video bit rate, and video encoding format. The second-level video freeze means that the freeze frequency of the video picture during the video playback at the target video playback end is greater than the preset freeze frequency.

[0076] Among them, when the second-level video freeze is triggered, a configuration with a lower comprehensive score in the multiple candidate video parameter configurations of the encoding configuration can be selected, and the target video capture end performs video capture according to the new encoding configuration information and sends video frames to the target video playback end.

[0077] In the case of the above-mentioned first-level video freeze (severe video freeze) or second-level video freeze (frequent video freeze), it is considered that the video playback quality at the target video playback end at the current moment can no longer meet the requirement of smooth viewing, and certain measures need to be taken to improve the current video playback quality. The improvement solutions for video playback quality include but are not limited to the following two:

[0078] When it is detected that a first-level video freeze occurs at the target video playback end, select a video transmission link with the highest comprehensive score of current transmission cost and transmission quality from each candidate video transmission link in the pre-established link pool as a new data transmission channel to write the video frames to be played by the target playback end into the target cache queue.

[0079] When it is detected that a second-level video freeze occurs at the target video playback end, notify the encoding module of the video capture end to select a level with a lower comprehensive score of resolution, frame rate, bit rate, and encoding format than the current encoding configuration information among the multiple pre-set encoding configuration information, so as to reduce the network resource occupancy during the network transmission of video frames.

[0080] By adopting the above optional solutions, it is possible to make a refined judgment on the video stuttering situation, and then adopt a processing method matching the stuttering result to ensure that the video playback quality on the target video playback end can meet the requirements of smooth viewing, and avoid the inability to perform matching video playback caused by inappropriate processing methods.

[0081] In the technical solution of the embodiment of the present invention, it is detected whether stuttering occurs according to whether the video frames cached in the cache queue are exhausted, and the stuttering degree of the video is refinedly determined according to the duration of the cache queue exhaustion state in which the cached video frames are exhausted, so as to solve the problem that it is inaccurate to judge video stuttering based on network parameters, and it does not require a large amount of memory overhead. Moreover, this solution only needs to detect the number of video frames in the cache queue for storing the video frames required by the video playback end within a preset duration, define the situation where the number of video frames in the cache queue is continuously exhausted as stuttering, and set a stuttering detection duration threshold in combination with the actual usage scenario, so as to flexibly and accurately detect video stuttering without consuming a large amount of memory overhead.

[0082] Figure 4 FIG. is a schematic flowchart of another video stuttering detection method provided by an embodiment of the present invention. The technical solution of this embodiment further optimizes the process of determining whether the target video playback end generates a second-level video stuttering based on the cumulative value of the cache queue exhaustion state duration corresponding to each stuttering sample stored in the stuttering sample analysis queue at the current moment on the basis of the technical solution of the foregoing embodiment. This embodiment can be combined with each optional solution in one or more of the above embodiments.

[0083] As Figure 4 shown, the video stuttering detection method of the embodiment of the present invention may include the following processes:

[0084] S410. Detect the remaining number of video frames in the target cache queue, where the target cache queue is used to cache the video frames to be played by the target video playback end.

[0085] S420. Determine the duration of the cache queue exhaustion state of the target cache queue, where the cache queue exhaustion state is a state where the remaining number of video frames in the target cache queue is less than a preset number of video frames, and the preset number of video frames is 1.

[0086] S430. Detect whether the duration of the cache queue exhaustion state is greater than a first stuttering detection duration threshold, where the first stuttering detection duration threshold is used to measure whether the cache queue exhaustion state of the target cache queue will cause video stuttering of the target video playback end.

[0087] S440. If the duration of the cache queue depletion state is greater than the first video freeze detection duration threshold, check whether the duration of the cache queue depletion state is greater than the second video freeze detection duration threshold. The first video freeze detection duration threshold is less than the second video freeze detection duration threshold. The second video freeze detection duration threshold is used to distinguish different degrees of video freeze caused by the cache queue depletion state of the target cache queue at the target video player. Different durations of the cache queue depletion state correspond to different degrees of video freeze.

[0088] S450. If the duration of the cache queue depletion state is between the first video freeze detection duration threshold and the second video freeze detection duration threshold, combine the start time of the cache queue depletion state and the duration of the cache queue depletion state into a video freeze sample and write it into the freeze sample analysis queue.

[0089] S460. Determine multiple reference freeze samples from the various freeze samples stored in the freeze sample analysis queue at the current moment. The freeze samples in the freeze sample analysis queue are written into storage in sequence according to the start time corresponding to the freeze sample. The multiple reference freeze samples are consecutive multiple freeze samples in the freeze sample analysis queue that can make the reference cumulative detection duration not greater than the preset freeze detection cycle duration and can reach the maximum sample quantity. The reference cumulative duration is the cumulative elapsed duration from the minimum start time corresponding to the multiple reference freeze samples to the current moment.

[0090] For the n freeze samples in the freeze sample analysis queue, in the order of being deposited into the freeze sample analysis queue, they are represented as {T s0 , T d0}, {T s1 , T d1}, {T s2 , T d2}, ……, {T sn , T dn}, {T s0 , T d0} represents the earliest deposited freeze sample, and {T sn , T dn} represents the latest deposited freeze sample. The freeze sample analysis queue will update the cumulative value T sum of the cache queue depletion state duration of the freeze sample according to the following formula:

[0091]

[0092] where T sum represents the cumulative sum of the cache queue depletion state durations of the freeze samples, and T di represents the cache queue depletion state duration corresponding to the i-th freeze sample in the freeze sample analysis queue.

[0093] Optionally, when the duration of the empty state of the target cache queue (the duration of the empty state of the cache queue when the video frame is empty) T d satisfies the following conditions

[0094] T d ∈(T h1 , T h2 )

[0095] then record the depletion of the video frames cached in the target cache queue this time as a freeze, and use T s and T d as freeze samples and add them to the freeze sample analysis queue. At this time, the values of the following variables need to be updated:

[0096] T sum = T sum + T d

[0097] Since new freeze samples will be continuously added to the freeze sample analysis queue, and the freeze detection and analysis are performed in units of the preset freeze detection cycle duration, it is usually necessary to select multiple reference freeze samples within the nearest preset freeze detection cycle duration from the current moment, and calculate the cumulative value of the duration of the empty state of the cache queue for the multiple reference freeze samples within the nearest preset freeze detection cycle duration from the current moment. That is, ensure that multiple reference freeze samples with the maximum sample quantity are selected from the freeze sample analysis queue, and the cumulative elapsed duration from the minimum start time corresponding to the multiple reference freeze samples with the maximum sample quantity to the current moment is not greater than the preset freeze detection cycle duration, then multiple reference freeze samples within the nearest preset freeze detection cycle duration from the current moment can be obtained.

[0098] As an optional but non-limiting implementation manner, determining multiple reference freeze samples from each freeze sample stored in the freeze sample analysis queue at the current moment includes the following steps C1 - C4:

[0099] Step C1: Determine the start time of the empty state of the cache queue corresponding to the target freeze sample stored in the freeze sample analysis queue, where the target freeze sample is the freeze sample with the earliest start time in the freeze sample analysis queue.

[0100] Step C2: Determine the target cumulative detection duration from the start time corresponding to the target freeze sample through multiple freeze samples to the current moment in the freeze sample analysis queue.

[0101] Step C3: If the target cumulative detection duration is greater than the preset card - lag detection cycle duration, delete the target card - lag sample in the card - lag sample analysis queue, and return to determine a new target card - lag sample from the card - lag sample analysis queue until the new target cumulative detection duration calculated using the start time corresponding to the new target card - lag sample is no longer greater than the preset card - lag detection cycle duration.

[0102] Step C4: Determine the remaining undeleted card - lag samples in the card - lag sample analysis queue as multiple reference card - lag samples.

[0103] For the n card - lag samples in the card - lag sample analysis queue, in the order of being stored in the card - lag sample analysis queue, they are expressed as {T s0 , T d0}, {T s1 , T d1}, {T s2 , T d2}, ……, {T sn , T dn}. Record a cumulative detection duration T cnt of the card - lag samples in the card - lag sample analysis queue, which is used to record the cumulative elapsed time from the start time T s0 of the first sample in the card - lag sample analysis queue to the current time T cur , that is:

[0104] T cnt = T cur - T s0

[0105] When T cnt > T set , it is necessary to delete the earliest - stored sample in the card - lag sample analysis queue. At this time, the values of the following variables need to be updated:

[0106] T sum = T sum - T d0

[0107] T cnt = T cur - T s1

[0108] If the card - lag detection queue becomes empty after deleting the sample, then

[0109] T sum = 0

[0110] T cnt = 0

[0111] When deleting the earliest - stored card - lag sample {T s0 , T d0}, the remaining n - 1 stutter samples will be updated according to the order in which they are stored in the stutter sample analysis queue. The updated stutter sample analysis queue is represented as {T s0 , T d0}, {T s1 , T d1}, {T s2 , T d2}, ……, {T sn-1 , T dn-1}.

[0112] As an optional but non - limiting implementation, after determining the target cumulative detection duration from the start time corresponding to the target stutter sample through multiple stutter samples to the current moment in the stutter sample analysis queue, the following steps are further included:

[0113] If the target cumulative detection duration is greater than the preset stutter detection cycle duration, then determine the remaining stutter samples in the stutter sample analysis queue as multiple reference stutter samples.

[0114] When T cnt > T set , it is necessary to delete the earliest - stored sample in the stutter sample analysis queue and directly determine all the remaining stutter samples in the stutter sample analysis queue as multiple reference stutter samples.

[0115] S470. Based on the cumulative value of the duration of the cache queue depletion state corresponding to multiple reference stutter samples, determine whether the target video player generates a second - level video stutter. The second - level video stutter means that the stutter frequency of the video picture during the video playback of the target video player is greater than the preset stutter frequency.

[0116] As an optional but non - limiting implementation, based on the cumulative value of the duration of the cache queue depletion state corresponding to multiple reference stutter samples, determining whether the target video player generates a second - level video stutter includes steps D1 - D2:

[0117] Step D1. If the cumulative value of the duration of the cache queue depletion state corresponding to multiple reference stutter samples is greater than the third stutter detection duration threshold, then determine that the target video player generates a second - level video stutter. The third stutter detection duration threshold is used to identify whether the cache queue depletion state of the target cache queue will cause a second - level video stutter, and the first stutter detection duration threshold is less than the third stutter detection duration threshold.

[0118] Step D2. If the cumulative value of the duration of the cache queue depletion state corresponding to multiple reference stutter samples is not greater than the third stutter detection duration threshold, determine that the target video player does not generate a first - level video stutter and a second - level video stutter.

[0119] See Figure 5, in addition to configuring a queue for analyzing stutter samples, the solution of this application also configures a state machine. The queue for analyzing stutter samples is used to record stutter samples that have occurred in the past and to detect whether frequent stuttering has occurred. The state machine is used to detect whether first-level video stuttering occurs currently and store stutter samples that do not meet the first-level video stuttering into the queue for analyzing stutter samples. The queue for analyzing stutter samples is used to store all stutter samples within the past T set time. Each stutter sample among them contains the start time and duration of the buffer queue depletion state corresponding to the stutter sample. At the same time, the queue for analyzing stutter samples will calculate the sum of the durations of the buffer queue depletion states of all stutter samples stored in the queue.

[0120] See Figure 5 . When the stutter samples in the queue for analyzing stutter samples are not empty, first judge whether the sum of the durations of the buffer queue depletion states of all stutter samples in the queue for analyzing stutter samples is greater than the third stutter detection duration threshold Th3. When the sum of the durations of the buffer queue depletion states is greater than the third stutter detection duration threshold Th3, it is judged that a frequent stuttering event has occurred, notify the processing module, and clear the queue for analyzing stutter samples. When the sum of the durations of the buffer queue depletion states does not exceed the third stutter detection duration threshold Th3, calculate the difference between the current time and the start time of the buffer queue depletion state of the earliest sample in the queue. If the difference is greater than T se t, it means that the earliest sample has exceeded the range of the most recent T set time, and delete the earliest sample. When the queue for analyzing stutter samples is empty, wait for new stutter samples to be sent into the queue for analyzing stutter samples. When the cumulative value of the durations of the buffer queue depletion states within the past T set time exceeds the third stutter detection duration threshold Th3, notify the next-level module that frequent stuttering has occurred, and clear the cumulative value of the durations of the buffer queue depletion states and the queue for analyzing stutter samples.

[0121] See Figure 5 and Figure 6 . Design a state machine, including an initial state, a buffer queue depletion state, and a non-buffer queue depletion state. Its operating principle is as follows Figure 6 shown. The detection module is default in the initial state. At this time, the stutter detection module will not collect the number of video frames of the target buffer queue until the module is officially started.

[0122] When the module starts, the state machine changes from the initial state to the non-buffer queue depletion state. At this time, the stutter detection module will continuously detect whether the number of video frames in the target buffer queue is 0. When the number of video frames in the target buffer queue is 0, the state machine changes from the non-buffer queue depletion state to the buffer queue depletion state; at the same time, the current time point is the start time point of this stutter.

[0123] When in the state of the cache queue being emptied, the stutter detection module continuously calculates the duration of the current state of the cache queue being emptied. When the accumulated duration of the cache queue being emptied exceeds the second stutter detection duration threshold Th2, it notifies the next-level module that a first-level video stutter (i.e., severe video stutter) has occurred, and clears the accumulated historical duration of the cache queue being emptied.

[0124] When in the state of the cache queue being emptied, if it is detected that the number of video frames in the target cache queue is greater than 0, the state machine changes from the state of the cache queue being emptied to the state of the cache queue not being emptied; meanwhile, if the accumulated duration of the cache queue being emptied exceeds the first stutter detection duration threshold Th1 at this time, the start time and duration of the cache queue being emptied of this cache queue being emptied state are sent as a stutter sample to the stutter sample analysis queue.

[0125] As an optional but non-limiting implementation manner, based on the accumulated value of the duration of the cache queue being emptied corresponding to multiple reference stutter samples, determining whether the target video playback end generates a second-level video stutter includes steps E1 - E3:

[0126] Step E1, determine the target stutter frequency of the target video playback end, where the target stutter frequency is the ratio of the accumulated value of the duration of the cache queue being emptied corresponding to multiple reference stutter samples to the preset stutter detection period duration.

[0127] Step E2, if the target stutter frequency is greater than the preset stutter frequency threshold, it is determined that the target video playback end generates a second-level video stutter. The preset stutter frequency threshold is the ratio of the third stutter detection duration threshold to the preset stutter detection period duration. The third stutter detection duration threshold is used to identify whether the state of the cache queue being emptied of the target cache queue will cause a second-level video stutter, and the first stutter detection duration threshold is less than the third stutter detection duration threshold.

[0128] Step E3, if the target stutter frequency is not greater than the preset stutter frequency threshold, it is determined that the target video playback end does not generate a first-level video stutter and a second-level video stutter.

[0129] If the frame rate of the video is f, the interval Tinterval between two video frames is:

[0130]

[0131] The calculation formula for the stutter frequency Fstutter within a period of time is:

[0132]

[0133] According to the above formula theory, the ratio of the cumulative duration of the cache queue empty state corresponding to multiple reference jam samples to the preset jam detection cycle duration can be determined as the target jam frequency. At the same time, the ratio of the third jam detection duration threshold to the preset jam detection cycle duration is determined as the preset jam frequency threshold.

[0134] On the basis of the technical solution of the above-mentioned embodiment, optionally, the freeze detection time threshold values ​​corresponding to the first freeze detection time threshold, the second freeze detection time threshold and the third freeze detection time threshold are respectively determined based on the flow size of the captured object in the collection area where the target video acquisition end is located corresponding to the target video playback end and the scene change size of the collection area. The freeze detection time threshold determines the freeze detection sensitivity in the video freeze detection process.

[0135] You can pre-define several usage scenarios, and preset the jam detection time thresholds Th1, Th2, Th3 and the jam detection cycle time T for each scenario. set In the actual application process, the preset parameters are used according to the scene. The following is a usage scenario: when the shooting object is a certain road, the degree of change of the scene varies according to the traffic volume in different time periods. When the traffic volume is large, the scene changes greatly, and a more sensitive jam detection duration threshold parameter set can be used; when the traffic volume is small or there are no moving vehicles, the scene changes less, and a less sensitive jam detection duration threshold parameter set can be used.

[0136] Based on the technical solution of the above embodiment, optionally, the first freeze detection duration threshold, the second freeze detection duration threshold and the third freeze detection duration threshold each correspond to an integer multiple of a reference time interval, and the reference time interval is the time interval between two adjacent video frames in the target cache queue.

[0137] The jam detection duration thresholds Th1, Th2, Th3 and the jam detection cycle duration Tset should be set in the following relationship with the frame rate:

[0138] The first freeze detection duration threshold Th1 is an integer multiple of Tinterval, assuming it is N times. The value of N can be dynamically set according to the actual usage scenario. The value of N is negatively correlated with the freeze detection sensitivity. In scenes where the video picture changes greatly, freeze detection requires a higher sensitivity, and N can be set to a smaller value; in scenes where the video picture changes slightly, freeze detection requires a lower sensitivity, and N can be set to a larger value.

[0139] The second freeze detection duration threshold Th2 is a judgment condition for severe freeze, and can be dynamically set according to the actual usage scenario. In scenes where the video picture changes greatly, Th2 needs to be set to a smaller value, but cannot be less than Th1; in scenes where the video picture changes slightly, Th2 can be set to a larger value.

[0140] The third freeze detection time threshold Th3 and T set is the judgment condition for frequent freezes, and the ratio of the two is the video freeze rate threshold. If it exceeds the threshold, it is judged as frequent freezes.

[0141] The jam rate threshold determines the sensitivity of detecting frequent jams. set The value of can be set dynamically according to the actual usage scenario. In the scenario where the video screen changes greatly, Th3 and T set The ratio of Th3 and T is a smaller value; in scenes with small video picture changes, Th3 and T set The ratio is a larger value, but Th3 must be greater than Th1, and T set The value cannot be too short.

[0142] The technical solution of the embodiment of the present invention detects whether a freeze occurs based on whether the cached video frames in the cache queue are exhausted, and makes a refined judgment on the degree of video freeze based on the duration of the cache queue being exhausted when the number of cached video frames is exhausted, thereby solving the problem of inaccurate judgment of video freeze based on network parameters, and not only that it takes up a lot of memory overhead. In addition, this solution only needs to detect the number of video frames in the cache queue used to store the video frames required by the video playback end within a preset duration, and defines the situation where the number of video frames in the cache queue is continuously exhausted as freeze, and sets the freeze detection duration threshold in combination with the actual usage scenario, so that video freeze can be detected flexibly and accurately without consuming a lot of memory overhead.

[0143] Figure 7 This is a structural schematic diagram of a video freeze detection device provided in an embodiment of the present invention. The embodiment of the present invention is applicable to the situation where video freeze detection is performed during video playback. The video freeze detection device can be implemented in the form of software and / or hardware, and is generally integrated on any electronic device with network communication function, which can be a mobile terminal, PC or server, etc.

[0144] like Figure 7 As shown, the video freeze detection device of the embodiment of the present invention may include: a frame number detection module 710, a determination module 720 and a freeze detection module 730. Among them:

[0145] A frame number detection module 710 is configured to detect the remaining video frame numbers in a target buffer queue, where the target buffer queue is used to buffer video frames to be played by a target video player;

[0146] A determination module 720 is configured to determine the duration of the empty state of the buffer queue of the target buffer queue. The empty state of the buffer queue is a state where the remaining video frame numbers in the target buffer queue are less than a preset number of video frames, and the preset number of video frames is 1;

[0147] A stuttering detection module 730 is configured to perform video stuttering detection on the target video player based on the duration of the empty state of the buffer queue. Different durations of the empty state of the buffer queue correspond to different degrees of video stuttering.

[0148] Based on the technical solution of the above embodiment, optionally, performing video stuttering detection on the target video player based on the duration of the empty state of the buffer queue includes:

[0149] Detecting whether the duration of the empty state of the buffer queue is greater than a first stuttering detection duration threshold, where the first stuttering detection duration threshold is used to measure whether the empty state of the buffer queue of the target buffer queue will cause video stuttering of the target video player;

[0150] If the duration of the empty state of the buffer queue is greater than the first stuttering detection duration threshold, then determine the degree of video stuttering of the target video player based on the duration of the empty state of the buffer queue and a second stuttering detection duration threshold. The first stuttering detection duration threshold is less than the second stuttering detection duration threshold, and the second stuttering detection duration threshold is used to distinguish different degrees of video stuttering caused by the empty state of the buffer queue of the target buffer queue in the target video player.

[0151] Based on the technical solution of the above embodiment, optionally, determining the degree of video stuttering of the target video player based on the duration of the empty state of the buffer queue and the second stuttering detection duration threshold includes:

[0152] If the duration of the empty state of the buffer queue is greater than the second stuttering detection duration threshold, then determine that the target video player generates a first-level video stuttering, where the first-level video stuttering is that the single stuttering duration of the video picture during the video playback of the target video player is greater than a preset stuttering duration;

[0153] If the duration of the empty state of the buffer queue is between the first stuttering detection duration threshold and the second stuttering detection duration threshold, then combine the start time of the empty state of the buffer queue and the duration of the empty state of the buffer queue into a video stuttering sample and write it into the stuttering sample analysis queue;

[0154] Based on the cumulative value of the duration of the empty state of the cache queue corresponding to each stuttering sample stored in the stuttering sample analysis queue at the current moment, determine whether the target video player generates a second-level video stutter, where the second-level video stutter means that the stuttering frequency of the video picture during the video playback of the target video player is greater than the preset stuttering frequency.

[0155] Based on the above technical solution of the embodiment, optionally, determining whether the target video player generates a second-level video stutter based on the cumulative value of the duration of the empty state of the cache queue corresponding to each stuttering sample stored in the stuttering sample analysis queue at the current moment includes:

[0156] Determine multiple reference stuttering samples from each stuttering sample stored in the stuttering sample analysis queue at the current moment. Each stuttering sample in the stuttering sample analysis queue is written and stored in order of the start time corresponding to the stuttering sample. The multiple reference stuttering samples are consecutive multiple stuttering samples in the stuttering sample analysis queue that can make the reference cumulative detection duration not greater than the preset stuttering detection cycle duration and can reach the maximum sample number. The reference cumulative duration is the cumulative elapsed duration from the minimum start time corresponding to the multiple reference stuttering samples to the current moment;

[0157] Based on the cumulative value of the duration of the empty state of the cache queue corresponding to the multiple reference stuttering samples, determine whether the target video player generates a second-level video stutter.

[0158] Based on the above technical solution of the embodiment, optionally, determining multiple reference stuttering samples from each stuttering sample stored in the stuttering sample analysis queue at the current moment includes:

[0159] Determine the start time of the empty state of the cache queue corresponding to the target stuttering sample stored in the stuttering sample analysis queue. The target stuttering sample is the stuttering sample with the earliest start time in the stuttering sample analysis queue;

[0160] Determine the target cumulative detection duration from the start time corresponding to the target stuttering sample through multiple stuttering samples to the current moment in the stuttering sample analysis queue;

[0161] If the target cumulative detection duration is greater than the preset stuttering detection cycle duration, then delete the target stuttering sample in the stuttering sample analysis queue, and return to determine a new target stuttering sample from the stuttering sample analysis queue until the new target cumulative detection duration calculated using the start time corresponding to the new target stuttering sample is no longer greater than the preset stuttering detection cycle duration;

[0162] Determine the remaining undeleted stuttering samples in the stuttering sample analysis queue as multiple reference stuttering samples.

[0163] Based on the technical solution of the above embodiment, optionally, after determining the target cumulative detection duration from the start time corresponding to the target stuttering sample through multiple stuttering samples to the current moment in the stuttering sample analysis queue, it further includes:

[0164] If the target cumulative detection duration is greater than the preset stuttering detection cycle duration, then determine the remaining stuttering samples in the stuttering sample analysis queue as multiple reference stuttering samples.

[0165] Based on the technical solution of the above embodiment, optionally, determining whether the target video player generates a second-level video stutter based on the cumulative value of the cache queue depletion state duration corresponding to multiple reference stuttering samples includes:

[0166] If the cumulative value of the cache queue depletion state duration corresponding to multiple reference stuttering samples is greater than the third stuttering detection duration threshold, it is determined that the target video player generates a second-level video stutter. The third stuttering detection duration threshold is used to identify whether the cache queue depletion state of the target cache queue will cause a second-level video stutter, and the first stuttering detection duration threshold is less than the third stuttering detection duration threshold;

[0167] If the cumulative value of the cache queue depletion state duration corresponding to multiple reference stuttering samples is not greater than the third stuttering detection duration threshold, it is determined that the target video player does not generate a first-level video stutter and a second-level video stutter.

[0168] Based on the technical solution of the above embodiment, optionally, determining whether the target video player generates a second-level video stutter based on the cumulative value of the cache queue depletion state duration corresponding to multiple reference stuttering samples includes:

[0169] Determine the target stuttering frequency of the target video player, where the target stuttering frequency is the ratio of the cumulative value of the cache queue depletion state duration corresponding to multiple reference stuttering samples to the preset stuttering detection cycle duration;

[0170] If the target stuttering frequency is greater than the preset stuttering frequency threshold, it is determined that the target video player generates a second-level video stutter. The preset stuttering frequency threshold is the ratio of the third stuttering detection duration threshold to the preset stuttering detection cycle duration. The third stuttering detection duration threshold is used to identify whether the cache queue depletion state of the target cache queue will cause a second-level video stutter, and the first stuttering detection duration threshold is less than the third stuttering detection duration threshold;

[0171] If the target stuttering frequency is not greater than the preset stuttering frequency threshold, it is determined that the target video player does not generate a first-level video stutter and a second-level video stutter.

[0172] Based on the technical solutions of the above embodiments, optionally, the sizes of the first video freeze detection duration threshold, the second video freeze detection duration threshold, and the third video freeze detection duration threshold corresponding to each of them are determined based on the traffic volume of the shooting object in the acquisition area where the target video acquisition end corresponding to the target video playback end is located and the degree of scene change in the acquisition area. The video freeze detection duration threshold determines the sensitivity of video freeze detection during the video freeze detection process.

[0173] Based on the technical solutions of the above embodiments, optionally, the sizes of the first video freeze detection duration threshold, the second video freeze detection duration threshold, and the third video freeze detection duration threshold corresponding to each of them are integer multiples of a reference time interval, and the reference time interval is the time interval between two adjacent video frames in the target buffer queue.

[0174] Based on the technical solutions of the above embodiments, optionally, after performing video freeze detection on the target video playback end based on the duration of the empty state of the buffer queue, it further includes:

[0175] If the target video playback end generates a first-level video freeze, select a target video transmission link from multiple candidate video transmission links corresponding to the target video playback end, and use the target video transmission link to write the video frames to be played by the target playback end into the target buffer queue. The target video transmission link is the video transmission link with the highest video transmission performance among the multiple candidate video transmission links. The first-level video freeze means that the single freeze duration of the video picture during the video playback of the target video playback end is greater than a preset freeze duration.

[0176] If the target video playback end generates a second-level video freeze, send encoding configuration information to the target video acquisition end corresponding to the target video playback end, so that the target video acquisition end performs video acquisition according to the new encoding configuration information and sends video frames to the target video playback end. The encoding configuration information is used to configure the video resolution, video frame rate, video bit rate, and video encoding format. The second-level video freeze means that the freeze frequency of the video picture during the video playback of the target video playback end is greater than a preset freeze frequency.

[0177] The technical solution provided by the embodiment of the present invention detects the remaining number of video frames in the target buffer queue, where the target buffer queue is used to buffer the video frames to be played by the target video player; determines the duration of the buffer queue depletion state of the target buffer queue, and the buffer queue depletion state is a state where the remaining number of video frames in the target buffer queue is less than the preset number of video frames, and the preset number of video frames is 1; based on the duration of the buffer queue depletion state, performs video stutter detection on the target video player, and different durations of the buffer queue depletion state correspond to different degrees of video stuttering. This solution detects whether stuttering occurs based on whether the video frames buffered in the buffer queue are depleted, and makes a refined determination of the degree of video stuttering according to the duration of the buffer queue depletion state in which the buffered video frames are depleted, solves the problem that it is inaccurate to judge video stuttering based on network parameters, and does not require a large amount of memory overhead.

[0178] The video stutter detection device provided by the embodiment of the present invention can execute the video stutter detection method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the video stutter detection method.

[0179] It should be noted that the various units and modules included in the above device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of mutual distinction and do not limit the protection scope of the embodiments of the present invention.

[0180] Figure 8 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. Next, refer to Figure 8 , which shows a schematic structural diagram of an electronic device 800 suitable for implementing the embodiment of the present invention (such as Figure 8 the terminal device or server in). The terminal device in the embodiment of the present invention may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), vehicle terminals (such as vehicle navigation terminals), etc. and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and usage scope of the embodiments of the present invention.

[0181] As Figure 8As shown, the electronic device 800 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 801, which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 are also stored. The processing device 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An editing / output (I / O) interface 805 is also connected to the bus 804.

[0182] Generally, the following devices may be connected to the I / O interface 805: an input device 806 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 808 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 8 the electronic device 800 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. More or fewer devices may be alternatively implemented or had.

[0183] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart may be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program codes for executing the video freeze detection method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device 809, or installed from the storage device 808, or installed from the ROM 802. When the computer program is executed by the processing device 801, the above functions defined in the video freeze detection method of the embodiment of the present invention are executed.

[0184] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0185] The electronic device provided by the embodiment of the present invention and the video freeze detection method provided by the above embodiment belong to the same inventive concept. Technical details not described in detail in this embodiment may be referred to in the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0186] An embodiment of the present invention provides a computer storage medium, on which a computer program is stored, and when the program is executed by a processor, the video stutter detection method provided by the above embodiment is implemented.

[0187] It should be noted that the computer-readable medium of the present invention may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which computer-readable program code is carried. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, and the computer-readable signal medium may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0188] In some embodiments, the client and the server may communicate using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with digital data communication in any form or medium (for example, a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (for example, the Internet), and end-to-end networks (for example, ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0189] The above computer-readable medium may be included in the above electronic device; or it may exist separately and not be assembled into the electronic device.

[0190] The above computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to:

[0191] The above computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: detect the remaining number of video frames in a target buffer queue, where the target buffer queue is used to buffer video frames to be played by a target video player; determine the duration of the buffer queue depletion state of the target buffer queue, where the buffer queue depletion state is a state in which the remaining number of video frames in the target buffer queue is less than a preset number of video frames; and perform video stutter detection on the target video player based on the duration of the buffer queue depletion state, where different durations of the buffer queue depletion state correspond to different degrees of video stutter.

[0192] Computer program code for performing the operations of the present invention may be written in one or more programming languages or combinations thereof. The programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0193] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0194] The units involved in the embodiments of the present invention can be implemented in software or in hardware. Among them, the name of the unit does not constitute a limitation on the unit itself in some cases. For example, the first acquisition unit can also be described as "the unit for acquiring at least two Internet protocol addresses".

[0195] The functions described above in this article can be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chip (SOCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0196] In the context of the present invention, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include electrical connections based on one or more wires, portable computer disks, hard disks, Random Access Memory (RAM), Read Only Memory (ROM), Erasable Programmable Read Only Memory (EPROM or flash memory), optical fibers, portable compact disc read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0197] The above description is only a preferred embodiment of the present invention and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in the present invention is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with (but not limited to) the technical features with similar functions disclosed in the present invention.

[0198] Moreover, although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the foregoing discussion, these should not be construed as limitations on the scope of the invention. Certain features that are described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0199] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A method for detecting video stuttering, characterized in that, The method includes: Detecting the remaining number of video frames in the target cache queue, where the target cache queue is used to cache the video frames to be played by the target video player; Determining the duration of the cache queue depletion state of the target cache queue, where the cache queue depletion state is a state in which the remaining number of video frames in the target cache queue is less than the preset number of video frames, and the preset number of video frames is 1; Performing video stutter detection on the target video player based on the duration of the cache queue depletion state, where different durations of the cache queue depletion state correspond to different degrees of video stutter.

2. The method according to claim 1, characterized in that, Performing video stutter detection on the target video player based on the duration of the cache queue depletion state, including: Detecting whether the duration of the cache queue depletion state is greater than the first stutter detection duration threshold, where the first stutter detection duration threshold is used to measure whether the cache queue depletion state of the target cache queue will cause video stutter in the target video player; If the duration of the cache queue depletion state is greater than the first stutter detection duration threshold, determining the degree of video stutter of the target video player based on the duration of the cache queue depletion state and the second stutter detection duration threshold, where the first stutter detection duration threshold is less than the second stutter detection duration threshold, and the second stutter detection duration threshold is used to distinguish different degrees of video stutter caused by the cache queue depletion state of the target cache queue in the target video player.

3. The method according to claim 2, characterized in that, Determining the degree of video stutter of the target video player based on the duration of the cache queue depletion state and the second stutter detection duration threshold, including: If the duration of the cache queue depletion state is greater than the second stutter detection duration threshold, determining that the target video player generates a first-level video stutter, where the first-level video stutter is that the single stutter duration of the video picture during the video playback of the target video player is greater than the preset stutter duration; If the duration of the cache queue depletion state is between the first stutter detection duration threshold and the second stutter detection duration threshold, combining the start time of the cache queue depletion state of the cache queue depletion state and the duration of the cache queue depletion state into a video stutter sample and writing it into the stutter sample analysis queue; Determining whether the target video player generates a second-level video stutter based on the cumulative value of the durations of the cache queue depletion states corresponding to the various stutter samples stored in the stutter sample analysis queue at the current moment, where the second-level video stutter is that the stutter frequency of the video picture during the video playback of the target video player is greater than the preset stutter frequency.

4. The method according to claim 3, characterized in that, Determining whether the target video player generates a second-level video stutter based on the cumulative value of the durations of the cache queue depletion states corresponding to the various stutter samples stored in the stutter sample analysis queue at the current moment, including: Determine multiple reference stuttering samples from each stuttering sample stored in the stuttering sample analysis queue at the current moment. Each stuttering sample in the stuttering sample analysis queue is written into storage in sequence according to the start time corresponding to the stuttering sample. The multiple reference stuttering samples are consecutive multiple stuttering samples in the stuttering sample analysis queue that can make the reference cumulative detection duration not greater than the preset stuttering detection cycle duration and can reach the maximum sample quantity. The reference cumulative duration is the cumulative elapsed duration from the minimum start time corresponding to the multiple reference stuttering samples to the current moment; Based on the cumulative value of the duration of the empty state of the cache queue corresponding to the multiple reference stuttering samples, determine whether the target video player generates video stuttering at the second level.

5. The method according to claim 4, characterized in that, Based on the cumulative value of the duration of the empty state of the cache queue corresponding to the multiple reference stuttering samples, determine whether the target video player generates video stuttering at the second level, including: If the cumulative value of the duration of the empty state of the cache queue corresponding to the multiple reference stuttering samples is greater than the third stuttering detection duration threshold, it is determined that the target video player generates video stuttering at the second level. The third stuttering detection duration threshold is used to identify whether the empty state of the cache queue of the target cache queue will cause video stuttering at the second level, and the first stuttering detection duration threshold is less than the third stuttering detection duration threshold; If the cumulative value of the duration of the empty state of the cache queue corresponding to the multiple reference stuttering samples is not greater than the third stuttering detection duration threshold, it is determined that the target video player does not generate video stuttering at the first level and the second level.

6. The method according to claim 4, characterized in that, Based on the cumulative value of the duration of the empty state of the cache queue corresponding to the multiple reference stuttering samples, determine whether the target video player generates video stuttering at the second level, including: Determine the target stuttering frequency of the target video player. The target stuttering frequency is the ratio of the cumulative value of the duration of the empty state of the cache queue corresponding to the multiple reference stuttering samples to the preset stuttering detection cycle duration; If the target stuttering frequency is greater than the preset stuttering frequency threshold, it is determined that the target video player generates video stuttering at the second level. The preset stuttering frequency threshold is the ratio of the third stuttering detection duration threshold to the preset stuttering detection cycle duration. The third stuttering detection duration threshold is used to identify whether the empty state of the cache queue of the target cache queue will cause video stuttering at the second level, and the first stuttering detection duration threshold is less than the third stuttering detection duration threshold; If the target stuttering frequency is not greater than the preset stuttering frequency threshold, it is determined that the target video player does not generate video stuttering at the first level and the second level.

7. The method according to claim 5 or 6, characterized in that, The magnitudes of the stuttering detection duration thresholds corresponding to the first stuttering detection duration threshold, the second stuttering detection duration threshold, and the third stuttering detection duration threshold are determined based on the traffic volume of the shooting object in the acquisition area where the target video acquisition end corresponding to the target video player is located and the degree of scene change in the acquisition area. The stuttering detection duration threshold determines the stuttering detection sensitivity in the video stuttering detection process.

8. A video stuttering detection device, characterized in that, The device includes: A frame number detection module for detecting the remaining video frame numbers in a target buffer queue, where the target buffer queue is used to buffer video frames to be played by a target video player; A determination module for determining the duration of the empty state of the buffer queue of the target buffer queue, where the empty state of the buffer queue is a state where the remaining video frame numbers in the target buffer queue are less than a preset video frame number, and the preset video frame number is 1; A video stutter detection module for performing video stutter detection on the target video player based on the duration of the empty state of the buffer queue, where different durations of the empty state of the buffer queue correspond to different degrees of video stutter.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; A storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the video stutter detection method according to any one of claims 1-7.

10. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions are used to execute the video stutter detection method according to any one of claims 1-7 when executed by a computer processor.

Images