Jitter buffer queue adjustment method, device, medium and computing equipment

Abstract

The present disclosure provides a method, apparatus, medium and computing device for adjusting a jitter buffer queue. The method includes: in response to receiving a current video frame of a target video, inputting it into a jitter buffer queue, and determining the difference between the receiving moment of the current video frame and its corresponding reference moment as the current jitter duration; when the current jitter duration meets the peak stretching condition, adjusting the buffering duration of the jitter buffer queue to the maximum jitter duration, so that the current video frame waits in the jitter buffer queue for the maximum jitter duration before being output to the display component, wherein the maximum jitter duration is not less than the current jitter duration; if a video frame meeting the peak stretching condition is not received within the maintenance time period after the size of the jitter buffer queue is adjusted, adjusting the size of the jitter buffer queue to the random jitter duration at the current moment. This method can timely achieve a de-jittering effect for the video in a burst-type weak network scenario with less storage space.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of video technology, and more specifically, to a method, apparatus, medium, and computing device for adjusting a jitter buffer queue. Background Art

[0002] This section is intended to provide a background or context for the embodiments of the present disclosure. No admission is made that the description herein is prior art by virtue of its inclusion in this section.

[0003] Videos played by playback devices typically consist of multiple frames. In network transmission scenarios, these frames must be transmitted one at a time. During the transmission of video frames, various factors, such as network instability and inconsistent frame sizes, can cause jitter, manifesting as an inconsistency in the time intervals between adjacent frames arriving at the receiver (usually the display device). The magnitude of this jitter can affect the video playback quality on the display device, especially in highly real-time video consumption scenarios like live broadcasts and video conferencing, where jitter can significantly impact video playback.

[0004] In order to enable the display device to display the smoothest possible video images and avoid video freezes, the receiver of the video frame can use a jitter buffer queue (Jitter Buffer, or jitter buffer) to temporarily store the video frames received within a certain period of time, and output and display each video frame at a relatively stable frame rate. In order to balance display performance and storage resources, a scalable jitter buffer queue (whose size can be reflected in the buffering time) can be used. To this end, accurately estimating the size of the jitter buffer queue becomes the key to ensuring that the jitter buffer queue performs well. In related technologies, the Kalman filter algorithm is often used to calculate the frame size transmission time difference and network random jitter, and the sum of the two is used as the buffering time of the jitter buffer queue. Summary of the Invention

[0005] However, the Kalman filter algorithm is difficult to effectively apply to the current bursty weak network scenarios. Specifically, the premise of the Kalman filter algorithm is that the network transmission jitter conforms to the characteristics of random distribution, and it can better adapt the size of the jitter buffer queue in a random jitter network. However, the current video transmission networks are often bursty weak networks. The Kalman filter algorithm often directly ignores a few occasional burst sample points, resulting in the jitter buffer queue not being stretched in time and easily causing jamming. In addition, a simple exponential filter is also used in the estimation of the transmission jitter difference caused by frame size. Especially in scenes with frequent switching between motion and stillness, the frame size will fluctuate dramatically. The exponential filter frame size method is difficult to truly reflect the frame size transmission jitter, resulting in poor anti-jitter effect of the jitter buffer queue.

[0006] Therefore, an improved jitter buffer queue adjustment solution is highly desirable to ensure timely stretching of the jitter buffer queue in bursty weak network scenarios, thereby improving video display quality in such network scenarios. In this context, embodiments of the present disclosure provide a jitter buffer queue adjustment method, apparatus, medium, and computing device.

[0007] In a first aspect of an embodiment of the present disclosure, a method for adjusting a jitter buffer queue is provided, the method comprising:

[0008] In response to receiving a current video frame of a target video, inputting the current video frame into a jitter buffer queue, and determining a difference between a reception time of the current video frame and a reference time corresponding to the current video frame as a current jitter duration of the current video frame;

[0009] When the current jitter duration satisfies the peak stretching condition, adjusting the buffering duration of the jitter buffer queue to the maximum jitter duration corresponding to the current video frame, so that the current video frame waits in the jitter buffer queue for the maximum jitter duration before being output to the display component, wherein the maximum jitter duration is not less than the current jitter duration;

[0010] If no video frame meeting the peak stretching condition is received within the maintenance time period after the buffering duration is adjusted, the buffering duration of the jitter buffer queue is adjusted from the maximum jitter duration to the random jitter duration at the current moment, and the random jitter duration is less than the maximum jitter duration.

[0011] Optionally, the receiving the current video frame of the target video includes:

[0012] When multiple image data packets containing the same sequence number are received, the multiple image data packets are reassembled into the current video frame corresponding to the sequence number.

[0013] Optionally, the current jitter duration satisfies a peak stretching condition, including: the current video frame satisfies at least one of the following, or the current video frame and at least one other video frame received within a discarded time period before the receiving moment respectively satisfy at least one of the following:

[0014] The current jitter duration is not less than a multiple threshold of the random jitter duration at the current moment;

[0015] The difference between the current jitter duration and the random jitter duration at the current moment is not less than the first duration threshold;

[0016] The current jitter duration is not less than a second duration threshold.

[0017] Optionally, determining a maximum jitter duration corresponding to the current video frame includes:

[0018] Determine the current jitter duration as the maximum jitter duration; or,

[0019] If at least one video frame meeting the peak stretching condition is received within the elimination time period before the receiving moment, the maximum value between the current jitter duration and the jitter duration of the at least one video frame is determined as the maximum jitter duration.

[0020] Optionally, also include:

[0021] The video frames in the jitter buffer queue are sequentially output to the display component according to corresponding playback intervals to play the target video.

[0022] Optionally, each video frame in the jitter buffer queue has a corresponding sequence number, and the method further includes:

[0023] If the arrangement order of the video frames in the jitter buffer queue is different from the sequence of the serial numbers of the video frames, the video frames are reordered to adjust the arrangement order of the video frames to be the same as the sequence of the serial numbers.

[0024] Optionally, determining the playback interval between two adjacent video frames includes:

[0025] Determine the preset playback step length as the playback interval between the two adjacent video frames; or,

[0026] The frame rate of the target video is determined, and the playback interval between the two adjacent video frames is calculated according to the frame rate.

[0027] Optionally, adjusting the buffering duration of the jitter buffer queue from the maximum jitter duration to a random jitter duration at a current moment includes:

[0028] Determining an adjustment rate within a convergence period according to the maximum jitter duration and the random jitter duration at a current moment;

[0029] In the convergence time period, the buffering duration is uniformly adjusted from the maximum jitter duration to the random jitter duration at the current moment according to the adjustment rate.

[0030] Optionally, the random jitter duration at the current moment is a preset fixed duration; or is obtained by calculating each received video frame of the target video through a Kalman filter algorithm.

[0031] Optionally, the maintenance duration of the maintenance time period is a preset duration, or an expected maintenance duration issued by a data platform.

[0032] Optionally, also include:

[0033] The freeze duration generated during the playback of the target video is uploaded to the data platform, so that the data platform can count the freeze durations uploaded by multiple video playback devices to obtain the expected maintenance duration under at least one statistical dimension. The expected maintenance duration is used by the data platform to be sent to the video playback devices that meet the corresponding statistical dimension.

[0034] In a second aspect of an embodiment of the present disclosure, a device for adjusting a jitter buffer queue is provided, the device comprising:

[0035] a video frame input module, configured to, in response to receiving a current video frame of a target video, input the current video frame into a jitter buffer queue, and determine a difference between a reception time of the current video frame and a reference time corresponding to the current video frame as a current jitter duration of the current video frame;

[0036] a first adjustment module, configured to adjust the buffering duration of the jitter buffer queue to the maximum jitter duration corresponding to the current video frame if the current jitter duration satisfies a peak stretching condition, so that the current video frame waits in the jitter buffer queue for the maximum jitter duration before being output to the display component, wherein the maximum jitter duration is not less than the current jitter duration;

[0037] The second adjustment module is used to adjust the buffering time from the maximum jitter time to the random jitter time at the current moment if no video frame meeting the peak stretching condition is received within the maintenance time period after the buffering time is adjusted, and the random jitter time is less than the maximum jitter time.

[0038] Optionally, the video frame input module is specifically used to:

[0039] When multiple image data packets containing the same sequence number are received, the multiple image data packets are reassembled into the current video frame corresponding to the sequence number.

[0040] Optionally, the current jitter duration satisfies a peak stretching condition, including: the current video frame satisfies at least one of the following, or the current video frame and at least one other video frame received within a discarded time period before the receiving moment respectively satisfy at least one of the following:

[0041] The current jitter duration is not less than a multiple threshold of the random jitter duration at the current moment;

[0042] The difference between the current jitter duration and the random jitter duration at the current moment is not less than the first duration threshold;

[0043] The current jitter duration is not less than a second duration threshold.

[0044] Optionally, a duration determination module is further included, configured to:

[0045] Determine the current jitter duration as the maximum jitter duration; or,

[0046] If at least one video frame meeting the peak stretching condition is received within the elimination time period before the receiving moment, the maximum value between the current jitter duration and the jitter duration of the at least one video frame is determined as the maximum jitter duration.

[0047] Optionally, a video frame output module is also included, which is used to:

[0048] The video frames in the jitter buffer queue are sequentially output to the display component according to corresponding playback intervals to play the target video.

[0049] Optionally, each video frame in the jitter buffer queue has a corresponding sequence number, and the device further includes a de-sequence reassembly module configured to:

[0050] If the arrangement order of the video frames in the jitter buffer queue is different from the sequence of the serial numbers of the video frames, the video frames are reordered to adjust the arrangement order of the video frames to be the same as the sequence of the serial numbers.

[0051] Optionally, the video frame output module is specifically used to:

[0052] Determine the preset playback step length as the playback interval between the two adjacent video frames; or,

[0053] The frame rate of the target video is determined, and the playback interval between the two adjacent video frames is calculated according to the frame rate.

[0054] Optionally, the second adjustment module is specifically configured to:

[0055] Determining an adjustment rate within a convergence period according to the maximum jitter duration and the random jitter duration at a current moment;

[0056] In the convergence time period, the buffering duration is uniformly adjusted from the maximum jitter duration to the random jitter duration at the current moment according to the adjustment rate.

[0057] Optionally, the random jitter duration at the current moment is a preset fixed duration; or is obtained by calculating each received video frame of the target video through a Kalman filter algorithm.

[0058] Optionally, the maintenance duration of the maintenance time period is a preset duration, or an expected maintenance duration issued by a data platform.

[0059] Optionally, a sample upload module is also included for:

[0060] The freeze duration generated during the playback of the target video is uploaded to the data platform, so that the data platform can count the freeze durations uploaded by multiple video playback devices to obtain the expected maintenance duration under at least one statistical dimension. The expected maintenance duration is used by the data platform to be sent to the video playback devices that meet the corresponding statistical dimension.

[0061] In a third aspect of the embodiments of the present disclosure, a medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for adjusting the jitter buffer queue described in any embodiment of the first aspect is implemented.

[0062] In a fourth aspect of an embodiment of the present disclosure, a computing device is provided, including:

[0063] a processor; a memory for storing instructions executable by the processor;

[0064] The processor implements the jitter buffer queue adjustment method described in any one of the embodiments of the first aspect by running the executable instructions.

[0065] According to the jitter buffer queue adjustment method of the embodiment of the present disclosure, the video playback device can input the current video frame into the jitter buffer queue in response to receiving the current video frame of the target video, and determine the difference between the receiving time of the current video frame and the reference time corresponding to the video frame as the current jitter duration of the video frame; then, when the current jitter duration meets the peak stretching condition, the buffering duration of the jitter buffer queue is adjusted to the maximum jitter duration corresponding to the current video frame, so that the current video frame is output to the display component after waiting for the maximum jitter duration in the jitter buffer queue, wherein the maximum jitter duration is not less than the current jitter duration; if the video frame meeting the peak stretching condition is not received within the maintenance time period after the size of the jitter buffer queue is adjusted, the size of the queue is adjusted from the maximum jitter duration to the random jitter duration at the current moment, and the random jitter duration is less than the maximum jitter duration.

[0066] In the above solution, the video playback device can adjust the buffering duration of the jitter buffer queue to the maximum jitter duration when the current jitter duration of the current video frame meets the peak stretching condition, ensuring that the current video frame waits in the jitter buffer queue for the maximum jitter duration before being output and displayed. It is understandable that the current video frame is any video frame in the target video, so the video playback device can determine whether each video frame of the target video meets the above peak stretching condition one by one, and adjust the cache duration of the jitter buffer queue accordingly based on the video frames that meet the condition.

[0067] Therefore, even in bursty, weak network scenarios, where the jitter duration of individual (sparse) video frames increases momentarily, the video playback device can still promptly adjust the jitter buffer queue's cache duration based on these frames. Furthermore, the output timing of each video frame in the adjusted jitter buffer queue is also adjusted accordingly, ensuring that the display timing of all video frames, including the current one, is as uniform as possible, avoiding noticeable lag and achieving the intended function of the jitter buffer queue. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings, in which several embodiments of the present disclosure are shown in an illustrative and non-limiting manner, wherein:

[0069] Figure 1 The following schematically shows a flow chart of a method for adjusting a jitter buffer queue according to an embodiment of the present disclosure;

[0070] Figure 2 A schematic diagram schematically illustrates a buffering duration adjustment process according to an embodiment of the present disclosure;

[0071] Figure 3 A schematic diagram schematically illustrates a sample uploading process according to an embodiment of the present disclosure;

[0072] Figure 4 A schematic diagram schematically illustrates a medium according to an embodiment of the present disclosure;

[0073] Figure 5 A block diagram schematically illustrates a device for adjusting a jitter buffer queue according to an embodiment of the present disclosure;

[0074] Figure 6 A schematic diagram of a computing device according to an embodiment of the present disclosure is schematically shown.

[0075] In the drawings, the same or corresponding reference numerals denote the same or corresponding parts. Specific embodiments

[0076] The principles and spirit of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided solely to enable those skilled in the art to better understand and implement the present disclosure, and are not intended to limit the scope of the present disclosure in any way. Rather, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0077] Those skilled in the art will appreciate that the embodiments of the present disclosure may be implemented as a system, apparatus, device, method, or computer program product. Therefore, the present disclosure may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software.

[0078] According to an embodiment of the present disclosure, a method, apparatus, medium, and computing device for adjusting a jitter buffer queue are proposed.

[0079] In this document, any number of elements in the drawings is for illustration and not for limitation, and any naming is for distinction only and does not have any limiting meaning.

[0080] The principles and spirit of the present disclosure are explained in detail below with reference to several representative embodiments of the present disclosure. SUMMARY OF THE INVENTION

[0082] The inventors have discovered that the Kalman filter algorithm is difficult to be effectively applied to the current burst-type weak network scenarios. Specifically, the premise of the Kalman filter algorithm is that the network transmission jitter conforms to the characteristics of random distribution, and the size of the jitter buffer queue can be well adapted in a random jitter network. However, the current video transmission networks are often burst-type weak networks, and the Kalman filter algorithm often directly ignores a few occasional burst sample points, resulting in the jitter buffer queue not being stretched in time and easily causing jamming. In addition, a simple exponential filtering process is also used in the estimation of the transmission jitter difference caused by the frame size. Especially in scenes with frequent switching between motion and stillness, the frame size will change dramatically. The exponential filtering frame size method is difficult to truly reflect the frame size transmission jitter, resulting in a poor anti-jitter effect of the jitter buffer queue.

[0083] In order to solve the above problems, the present disclosure provides a method, apparatus, medium and computing device for adjusting a jitter buffer queue, and an exclusive content brief introduction.

[0084] In this manner, the video playback device can adjust the jitter buffer queue's buffering duration to the maximum jitter duration if the current jitter duration of the current video frame meets the peak stretching condition, ensuring that the current video frame waits in the jitter buffer queue for the maximum jitter duration before being output and displayed. It will be understood that the current video frame is any video frame in the target video, so the video playback device can determine whether each video frame of the target video meets the peak stretching condition one by one, and adjust the jitter buffer queue's buffering duration accordingly based on the video frames that meet the condition.

[0085] Therefore, even in bursty, weak network scenarios, where the jitter duration of individual (sparse) video frames increases momentarily, the video playback device can still promptly adjust the jitter buffer queue's cache duration based on these frames. Furthermore, the output timing of each video frame in the adjusted jitter buffer queue is also adjusted accordingly, ensuring that the display timing of all video frames, including the current one, is as uniform as possible, avoiding noticeable lag and achieving the intended function of the jitter buffer queue.

[0086] After introducing the basic principles of the present disclosure, various non-limiting embodiments of the present disclosure are described in detail below.

[0087] Application Scenario Overview

[0088] It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in this respect. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario.

[0089] According to the jitter buffer queue adjustment method described in the embodiment of the present disclosure, the method can be applied to a video playback device, which can receive each video frame of the target video sent by other devices through the network, and put the received video frames into the jitter buffer queue maintained by itself, so that the video frames wait in the jitter buffer queue for a certain period of time before being output and displayed, thereby eliminating or reducing the possible lag in the target video playback process caused by the jitter in the video frame transmission process.

[0090] Among them, the video playback device described in the embodiment of the present disclosure is a Jitter Buffer, which is essentially a queue. The role of the queue in this solution is to force the video frame to wait for a certain period of time before being output, and this period of time is the buffering period of the jitter buffer queue described in this solution. Of course, no matter in any time period, each video frame in the jitter buffer queue can be output from the queue in sequence according to the order in which it enters the queue, that is, first-in-first-out, but the playback interval of each video frame output from the queue may change, that is, the frame rate of the target video may not be a fixed value. It should be noted that the buffering period of the jitter buffer queue described in this solution can be expressed by time, which can actually correspond to the queue length (or capacity) of the queue, that is, the buffering period can also be used to characterize the number of video frames that the jitter buffer queue can store at the same time, which will not be repeated.

[0091] The present disclosure does not limit the specific form of jitter duration. For example, the video playback device can be a mobile phone, a tablet device, a laptop computer, a PDA (Personal Digital Assistants), a wearable device (such as smart glasses, smart watches, etc.), etc., and one or more embodiments of this specification do not limit this. The terminal device can be equipped with a display component (screen, etc.) and a network component (such as a network card, etc.), wherein the network component is used to establish a network connection between the video playback device and other devices in a wired or wireless manner, so as to orderly receive the video frames of the target video sent by other devices; the display component is used to sequentially display each video frame output from the jitter buffer queue to present the playback effect of the target video to the user.

[0092] In addition, the adjustment scheme of the jitter buffer queue described in this solution can be implemented as an independent application, or can also be implemented as a functional plug-in such as SDK (Software Development Kit). Taking SDK as an example, an application with a video playback function can be run in a video display device, so that the SDK can run in conjunction with the application (such as being called by the application) to achieve the video de-jittering function expected by this solution. In addition, the application can be a client program, which can obtain the target video to be played from the corresponding server program. At this time, the server program (or the physical device such as the server where it is located) is the sender of the target video, and can therefore be regarded as the aforementioned other devices.

[0093] According to the jitter buffer queue adjustment method described in an embodiment of the present disclosure, the video playback device can input the current video frame into the jitter buffer queue in response to receiving the current video frame of the target video, and determine the difference between the receiving time of the current video frame and the reference time corresponding to the video frame as the current jitter duration of the video frame; then, when the current jitter duration meets the peak stretching condition, the buffering duration of the jitter buffer queue is adjusted to the maximum jitter duration corresponding to the current video frame, so that the current video frame is output to the display component after waiting for the maximum jitter duration in the jitter buffer queue, wherein the maximum jitter duration is not less than the current jitter duration; if the video frame that meets the peak stretching condition is not received within the maintenance time period after the size of the jitter buffer queue is adjusted, the size of the queue is adjusted from the maximum jitter duration to the random jitter duration at the current moment, and the random jitter duration is less than the maximum jitter duration.

[0094] Exemplary Methods

[0095] The following describes a method for adjusting a jitter buffer queue according to an exemplary embodiment of the present disclosure, with reference to the accompanying drawings. It should be noted that the aforementioned application scenarios are merely provided to facilitate understanding of the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in this respect. Rather, the embodiments of the present disclosure can be applied to any applicable scenario.

[0096] Figure 1 The flowchart of a method for adjusting a jitter buffer queue according to an embodiment of the present disclosure is schematically shown. Figure 1 As shown, the method may include the following steps S102-S106.

[0097] Step S102: In response to receiving the current video frame of the target video, the current video frame is input into the jitter buffer queue, and the difference between the receiving time of the current video frame and the reference time corresponding to the current video frame is determined as the current jitter duration of the current video frame.

[0098] The current video frame described in the embodiment of the present disclosure can be any video frame of the target video. In other words, for each video frame of the received target video, the video playback device can use the jitter buffer queue adjustment scheme described in the embodiment of the present disclosure to process it. This is hereby explained.

[0099] Because a video frame typically contains a large amount of data, it can be transmitted in packets. Specifically, the sender can split a video frame into multiple data packets and transmit them separately to the receiver. Each data packet generated from the same video frame has the same sequence number (the sequence number of each video frame represents its position within the timeline of the target video), indicating to the receiver that the data packets belong to the same video frame. Of course, each data packet can also contain position information to indicate to the receiver the position within the video frame to which the video packet corresponds. Conversely, in one embodiment, the receiving video playback device can have a video framing function. For example, upon receiving multiple image data packets containing the same sequence number, the video playback device can reassemble the multiple image data packets into the current video frame corresponding to that sequence number. For example, the data packets can be reassembled in sequence according to the position information recorded in each data packet. In this way, the video playback device can accurately receive the current video frame of the target video. Alternatively, the video framing function can be integrated into a jitter buffer queue, with this queue implementing the aforementioned video framing function. This will not be discussed further.

[0100] Of course, transmitting each video frame of the target video in separate packets is only one available transmission method. In order to reduce the workload of the video playback device for pre-processing after receiving the video frame, the sender can also transmit each video frame separately as a whole. Alternatively, considering that the type of video frame is usually related to its data volume, the sender can also use different methods to transmit different types of video frames. For example, for key frames (I frames) with generally large data volumes, the aforementioned sub-packet transmission method can be used to transmit them to the receiver; while for predicted frames (P frames) and bidirectional predicted frames (B frames) with generally small data volumes, they can be transmitted separately as a whole. In this way, the transmission efficiency of the video frames and the pre-processing workload of the video playback device can be taken into account as much as possible.

[0101] Regardless of the method used to receive the current video frame of the target video, after receiving the video frame, the video playback device can, on the one hand, enter the video frame into the jitter buffer queue; on the other hand, the video playback device can calculate the current jitter duration of the video frame. For example, the current jitter duration of the current video frame can be determined as the difference between the reception time of the current video frame and the reference time corresponding to the current video frame.

[0102] The receiving time of the current video frame is the time when the video playback device receives the current video frame. Specifically, in the aforementioned video framing scenario, this time can be the time when all data packets are received; it can also be the time when framing is completed. The specific definition of the receiving time can be set according to actual conditions. The reference time corresponding to the current video frame is the theoretical value (or expected value) of the receiving time of the current video frame. Specifically, the sum of the receiving time of the previous video frame of the current video frame and the random jitter duration of the current time can be used as the reference time corresponding to the current video frame.

[0103] In addition, the random jitter duration at the current moment described in the embodiment of the present disclosure can be a preset fixed duration. Alternatively, the random jitter duration can be obtained by calculating the received video frames of the target video through the Kalman filter algorithm. Among them, the received video frames of the target video can be understood as the historical video frames of the target video received by the video playback device before the current moment, or can also be understood as the current video frame and the historical video frames. The specific process of calculating the random jitter duration through the Kalman filter algorithm can be referred to the records in the relevant technology, and the present disclosure will not repeat it.

[0104] Step S104, when the current jitter duration meets the peak stretching condition, the buffering duration of the jitter buffer queue is adjusted to the maximum jitter duration corresponding to the current video frame, so that the current video frame waits in the jitter buffer queue for the maximum jitter duration before being output to the display component, wherein the maximum jitter duration is not less than the current jitter duration.

[0105] After determining the current jitter duration of the current video frame, the video playback device can determine whether the duration meets the peak stretching condition: if so, the buffering duration of the jitter buffer queue can be adjusted to the maximum jitter duration corresponding to the current video frame; otherwise, if not, there is no need to adjust the buffering duration, that is, the buffering duration continues to remain unchanged. It should be noted that the buffering duration of the jitter buffer queue at the current moment (that is, the buffering duration before the adjustment) may be the aforementioned random jitter duration, or it may be the (previous) maximum jitter duration after the previous adjustment - at this time, the current moment is still within the maintenance time period corresponding to the previous adjustment.

[0106] It is understood that the greater the current jitter duration relative to the random jitter duration, the greater the jitter of the current video frame relative to the normal jitter during video frame transmission, that is, the greater the delay of the current video frame relative to its reference time. In other words, the current video frame has burst jitter. Based on this, whether the peak stretching condition is met can be determined based on the current jitter duration. Specifically, the peak stretching condition can be determined to be met if the current video frame meets at least one of the following conditions:

[0107] For example, it can be determined that the current video frame meets the peak stretching condition when the current jitter duration is not less than the multiple threshold of the random jitter duration at the current moment. Assuming that the current jitter duration at the current moment is T and the random jitter duration is T0, then when T≥k*T0 (where k is not less than 1) determined by the aforementioned method, it can be determined that the current video frame meets the peak stretching condition. Among them, the multiple threshold can be set according to the actual situation such as the frame rate (Frame Rate) and resolution of the target video, and the embodiment of the present disclosure is not limited to this. For example, the k can be negatively correlated with the frame rate of the target video (that is, the higher the frame rate of the target video, the smaller k is set, and the lower the frame rate, the larger k is set), or positively correlated with the resolution of the target video (the higher the resolution, the larger the data volume of a single video frame, and the greater the jitter may be) (that is, the higher the resolution of the target video, the larger k is set, and the lower the resolution, the smaller k is set), etc. Exemplarily, the k can be equal to 3, 6, 4, etc.

[0108] For another example, when the difference between the current jitter duration and the random jitter duration at the current moment (i.e., T-T0) is not less than the first duration threshold, it can be determined that the current video frame meets the peak stretching condition. Exemplarily, the first duration threshold can be 200ms, 300ms, 280ms, etc. For another example, when the current jitter duration is not less than the second duration threshold, it can be determined that the current video frame meets the peak stretching condition. Among them, the above-mentioned first duration threshold and the second duration threshold can be pre-set according to actual conditions. Similar to the aforementioned multiple threshold, the first duration threshold and the second duration threshold can also be set according to parameters such as the frame rate and resolution of the target video, such as being negatively correlated with the frame rate of the target video, positively correlated with the resolution of the target video, etc., which will not be repeated.

[0109] It should also be noted that, when the current jitter duration satisfies at least one of the above conditions, it can be determined that the current video frame meets the peak stretching condition. Taking the aforementioned multiple threshold and the first duration threshold as an example, assuming that k=4 corresponding to the pre-set multiple threshold and the first duration threshold is 280ms, then when the random jitter duration at the current moment is T0=100ms (k*T0=400ms at this time), if the current jitter duration T=260ms determined by the aforementioned method, it can be determined that the current jitter duration does not meet the peak stretching condition (because T<280ms and T<400ms at this time); or, if the current jitter duration T=350ms determined by the aforementioned method, it can be determined that the current jitter duration meets the peak stretching condition (because T>280ms at this time); or, if the current jitter duration T=460ms determined by the aforementioned method, it can be determined that the current jitter duration meets the peak stretching condition (because T>280ms and T>400ms at this time).

[0110] The above method can be used to determine whether the current jitter duration meets the peak stretching condition. If it does, the video playback device can adjust the buffering duration of the jitter buffer queue to the maximum jitter duration corresponding to the current video frame. To this end, the video playback device needs to first determine the maximum jitter duration. As an exemplary embodiment, the current jitter duration can be determined as the maximum jitter duration. It can be seen from the above judgment conditions that the current jitter duration meets the peak stretching condition, which means that the current jitter duration is already large enough, so at this time, this duration can be directly used as the maximum jitter duration.

[0111] In fact, even in a network environment with a bursty weak network, the jitter duration may result in the reception of multiple video frames with a larger jitter duration in a short period of time. In this regard, as another exemplary embodiment, if the video playback device receives at least one video frame that meets the peak stretching condition within the elimination time period before the receiving moment, the maximum value between the current jitter duration and the jitter duration of each of the at least one video frame can be determined as the maximum jitter duration. In this way, the true maximum value can be determined among the jitter durations corresponding to the current jitter duration and the multiple video frames with larger jitter durations received within the elimination time period (including the current moment), and the maximum value can be used as the maximum jitter duration. In this way, it can be ensured that the current video frame and the multiple video frames with larger jitter durations can be output after waiting for the corresponding duration in the jitter buffer queue, thereby using as little storage space as possible in exchange for the smoothness of the video playback process in this network environment.

[0112] In addition, the maximum jitter duration at any moment can be a preset duration, or calculated by the Kalman algorithm. For details, please refer to the description of the above embodiment, which will not be repeated here.

[0113] Figure 2 The following schematically shows a process for adjusting the buffering duration according to an embodiment of the present disclosure. Figure 2 As shown, the horizontal axis represents time (corresponding to the various moments described in this solution), and the vertical axis represents duration (corresponding to the various durations described in this solution). The jitter duration of each video frame received by the video playback device is shown by a waveform diagram, and the peak heights of different peaks in the waveform diagram are used to characterize the jitter duration of each video frame received by the video playback device. However, the characterization is only schematic, and the actual duration corresponding to each peak on the vertical axis has no specific physical meaning, nor does it represent the specific value of the corresponding jitter duration. This is hereby explained.

[0114] Among them, three video frames are received in the time period [0, t1], and the jitter duration of each video frame is a random jitter duration. In this time period, the buffering duration of the jitter buffer queue is always the corresponding random jitter duration T0 (this value is only exemplary, and the buffering duration before adjustment can be any value). The video playback device receives the current video frame at time t1 (i.e., the current time) and detects that its current jitter duration meets the peak stretching condition. Then, starting from time t1, the buffering duration of the jitter buffer queue can be adjusted to the maximum jitter duration Tmax corresponding to the current video frame.

[0115] After adjusting the buffering time, the current video frame will wait in the jitter buffer queue for the maximum jitter time before being output to the display component for rendering and display. It should be noted that regardless of whether the jitter buffer queue's buffering time is adjusted or not, each video frame stored in the jitter buffer queue will be output to the display component in sequence according to the corresponding playback interval.

[0116] Step S106: If no video frame meeting the peak stretching condition is received within the maintenance time period after the buffering duration is adjusted, the buffering duration of the jitter buffer queue is adjusted from the maximum jitter duration to the random jitter duration at the current moment, and the random jitter duration is less than the maximum jitter duration.

[0117] After the adjustment of the buffering duration is completed, the size of the buffering duration is the maximum jitter duration, and the jitter buffer queue is in a peak stretching state. It can be understood that the longer the buffering duration is, the more storage space (such as more valuable memory space) the jitter buffer queue occupies. Considering that the jitter buffer queue is usually set in the memory of the video playback device, in order to save storage space as much as possible, the size of the buffering duration can be adjusted to a smaller size when the sudden jitter ends, so as to reclaim the storage space occupied by the jitter buffer queue and shorten the delay in the video playback process.

[0118] Among them, if the video playback device does not receive a video frame that meets the peak stretching condition within the maintenance time period after the buffering duration is adjusted, it can be determined that the burst jitter corresponding to the current video frame has ended. At this time, the buffering duration of the jitter buffer queue can be adjusted from the maximum jitter duration to the random jitter duration at the current moment, and the random jitter duration is smaller than the maximum jitter duration, so that the adjusted buffering duration (at this time the size is the random jitter duration) is smaller than the buffering duration before adjustment (at this time the size is the maximum jitter duration).

[0119] In addition, if (at least one) new video frame that meets the peak stretching condition is received within the maintenance time period after the buffering duration is adjusted, it can be determined that the burst jitter corresponding to the current video frame has not yet ended, and the video playback device can further adjust the buffering duration in response to the new video frame, such as adjusting the buffering duration from the maximum jitter duration to a new maximum jitter duration - the new maximum jitter duration may be greater than or less than the maximum jitter duration (no adjustment is required when the two are equal). Among them, the judgment of the new video frame and the corresponding skipping process for the buffering duration are no different in essence from the aforementioned embodiment, and reference can be made to the records of the aforementioned embodiment, which will not be repeated here.

[0120] The starting point of the maintenance time period is the current moment (ie, the moment when the buffering time period is adjusted to the current jitter time period), and the duration of the maintenance time period is recorded as the maintenance time period. Figure 2 As shown, the length of the maintenance window t2-t1 is the maintenance duration of the maintenance time period.

[0121] It is understandable that the size of the maintenance duration determines whether the jitter buffer queue is in a peak stretch state when a sudden jitter occurs later. Therefore, the size of the maintenance duration is very critical. If the maintenance duration is too short, the jitter buffer queue is not in a peak stretch state when a sudden jitter occurs, which is prone to jamming. If the maintenance window is too large, there may be no sudden jitter most of the time, but it will bring about a large delay side effect, which is not worth the loss. In this regard, the maintenance duration can be a preset duration, such as 4s, 10s, 5s, etc. Alternatively, the maintenance duration can also be the expected maintenance duration issued by the data platform.

[0122] Among them, the data platform can issue the expected maintenance duration according to a fixed period, so that the video playback device can use the new expected maintenance duration to replace the locally saved (last received) expected maintenance duration each time it receives it. Based on this, the video playback device can determine the locally saved expected maintenance duration as the maintenance duration of the maintenance time period at the current moment. In this way, the data platform can plan to issue the expected maintenance duration according to a longer fixed period. Alternatively, the video playback device can also send a duration acquisition request to the data platform at the current moment (or within a period of time after the moment), and receive the expected maintenance duration issued by the data platform in response to the request, and then determine the duration as the maintenance duration of the maintenance time period. In this way, the video playback device can directly obtain the latest version of the maintenance duration from the data platform each time it needs maintenance duration, which helps to achieve timely update of the maintenance time period.

[0123] The data platform can obtain sample data uploaded by multiple video playback devices, and based on the sample data, calculate the expected duration and send it to the corresponding video playback device. The video playback device can upload the duration of the freeze generated during the playback of the target video to the data platform. The freeze duration is then used by the video playback device as sample data generated by itself. Accordingly, the data platform can calculate the freeze duration uploaded by multiple video playback devices to obtain the expected duration under at least one statistical dimension. The expected duration can be used by the data platform to send it to the video playback device that meets the corresponding statistical dimension. During the playback of the target video, the video playback device can determine the corresponding freeze duration after each freeze. The duration can include the duration of the freeze phenomenon, the duration of the freeze period when the freeze occurs, and / or the jitter duration of the video frame corresponding to the freeze. The statistical dimensions can include the video frame rate, resolution (4K, 1080p, etc.), format (mp4, avi, etc.), etc., and can also include the network type (wired, wireless, etc.), application (i.e., APP), geographic location, operator, etc. of the video playback device. In the implementation of the solution, the above-mentioned freeze duration and statistical dimensions can be reasonably set according to actual conditions, and the embodiments of the present disclosure do not limit this.

[0124] The uploading of sample data can be completed by the aforementioned SDK. Figure 3 The following schematically shows a sample uploading process according to an embodiment of the present disclosure. Figure 3As shown, SDK1, SDK2...SDKn can be run in different video playback devices respectively, and each SDK collects the freeze duration generated by the video playback device in which it is located during the video playback process, and uploads it as sample data to the data platform. Among them, the SDK can have an RTC (Real Time Lock) function, and the video playback device can implement the aforementioned data interaction process with the data platform based on this function. In addition, the SDK can also upload the dimension information corresponding to the sample data at the same time to indicate to the data platform which statistical dimensions the sample data is applicable to. Accordingly, after obtaining the sample data uploaded by multiple SDKs, the data platform can perform statistics according to the aforementioned statistical dimensions to obtain the expected maintenance duration under at least one statistical dimension. For example, the intervals of freezes in each dimension can be counted by day, and all freeze intervals can be sorted according to the duration, and the TOP95 value (or median, average, etc.) can be taken as the typical freeze interval value.

[0125] Furthermore, upon receiving a duration acquisition request initiated by any video playback device under any statistical dimension, or upon the arrival of a fixed period, the data center may send the typical freeze interval value under any statistical dimension to any video playback device, so that the device can use it as its own expected maintenance duration. It should be noted that the expected maintenance duration under any statistical dimension can be sent to each video playback device under that dimension for use, and it should be avoided as much as possible to avoid sending it to video playback devices under other statistical dimensions to ensure the rationality of the expected maintenance duration used by each video playback device, thereby ensuring the de-jitter effect of the subsequent jitter buffer queue and the video playback performance of the video playback device.

[0126] Through this approach, video playback devices can continuously report data. The data platform processes and organizes the data to determine the appropriate typical pause interval value and sends it to the video playback device, thus forming a closed-loop chain of collection, decision-making, feedback, and execution. If the video transmission network changes, this closed-loop system interface dynamically detects the network changes and issues the appropriate typical pause interval value, thereby achieving the effect of adaptively adjusting the buffering time of the jitter buffer queue, achieving the most cost-effective solution to meet the requirements of smoothness and latency.

[0127] It is understandable that the buffering time is a parameter of the jitter buffer queue, so the process of adjusting the buffering time is actually the process of adjusting the parameter. In one embodiment, the video playback device can complete the adjustment instantly in the process of adjusting the buffering time from the maximum jitter time to the random jitter time at the current moment, that is, the buffering time is adjusted from the maximum jitter time to the random jitter time at the current moment at one time. However, since the jitter buffer queue may store video frames that have not yet been output (this part of the video frames may include the current video frame) before this adjustment, the instantaneous adjustment of the buffering time may cause this part of the video frames to be output to the display component in a short time (such as instantaneously), which may cause the video to appear to be played back at an accelerated (or fast-forwarded) rate, or even cause rendering and display errors of the display component.

[0128] In this regard, the video playback device can avoid the above-mentioned problem by adjusting the buffering duration at a uniform speed. For example, the video playback device can determine the adjustment rate within the convergence time period based on the maximum jitter duration and the random jitter duration at the current moment; and then, within the convergence time period, the buffering duration is uniformly adjusted from the maximum jitter duration to the random jitter duration at the current moment according to the adjustment rate. Considering that each video frame in the jitter buffer queue is output from the jitter buffer queue one by one in units of one video frame, during the uniform adjustment process, the adjustment can be made with the playback interval between two adjacent video frame output queues as the step size.

[0129] like Figure 2 As shown in the figure, the above adjustment is completed within the convergence window. The length of the convergence window t3-t2 is the time required for the adjustment. Among them, the random jitter duration at time t3 is the same as that at time t3, both are T0. Therefore, after the adjustment is completed, the buffering length at time t3 is adjusted (i.e., restored) to T0.

[0130] For example, assume that the local adjustment needs to adjust the buffering time from 600ms (i.e., the current jitter time) to 480ms (i.e., the random jitter time at the current moment), and there are three video frames in the jitter buffer queue before this adjustment (video frames 1, 2, and 3 in order of output), and the output playback intervals of adjacent video frames are 40ms (frame rate is 25fps). In this regard, the video playback device can adjust according to the following process: after video frame 1 is output, the buffering time is adjusted from 600ms to 560ms; after video frame 2 is output, the buffering time is adjusted from 560ms to 520ms; after video frame 3 is output, the buffering time is adjusted from 520ms to 480ms. It can be seen that the above adjustment process is completed at a uniform speed in three steps (corresponding to three video frames respectively). It should be noted that, assuming that the playback interval between video frame 1 and video frame 2 is 50ms, and the playback interval between video frame 2 and video frame 3 is 45ms, the video playback device can adjust according to the following process: after video frame 1 is output, the buffering time is adjusted from 600ms to 550ms; after video frame 2 is output, the buffering time is adjusted from 550ms to 505ms; after video frame 3 is output, the buffering time is adjusted from 505ms to 480ms. In this adjustment process, although the step size of each adjustment step is different, it should also be regarded as a uniform speed adjustment, and this embodiment also does not exceed the scope of protection of this embodiment.

[0131] In one embodiment, at any time before or after adjusting the buffering duration, the video playback device can sequentially output the video frames in the jitter buffer queue to the display component according to the corresponding playback interval to play the target video. It is understood that the input of video frames received by the video playback device into the jitter buffer queue and the output of video frames from the queue do not affect each other; that is, the input and output processes of video frames into and out of the jitter buffer queue are independent and asynchronous. Based on this, the target video can be played smoothly even if the current video frame is significantly delayed.

[0132] The order in which the video frames are received by the video playback device may not match the order in which they were sent by the sender (this order is reflected by the serial numbers of each video frame), meaning the video frames may be out of order. After the video playback device sequentially inputs the received video frames into the jitter buffer queue, it is clear that the video frames are still out of order. At this point, directly outputting the video frames in the order in which they were input may result in a chaotic image or even anomalies such as playback failure.

[0133] In this regard, based on the corresponding serial numbers of each video frame in the jitter buffer queue, the video playback device can determine whether the arrangement order of the video frames in the jitter buffer queue is consistent with the order of the serial numbers of the video frames. If the two orders are different, the video frames can be shuffled and reassembled to adjust the arrangement order of the video frames to be consistent with the order of the serial numbers, and then the video frames can be output in sequence according to the order of the serial numbers so that the display component can display them sequentially. Of course, if the two orders are the same, it indicates that the multiple video frames are not out of order, and there is no need to shuffle and reassemble them. The video frames only need to be output sequentially.

[0134] In the process of outputting each video frame, the video playback device can determine the playback interval between two adjacent video frames in a variety of ways. For example, the video playback device can determine the preset playback step size as the playback interval between two adjacent video frames. At this time, the playback interval between two adjacent video frames in the multiple video frames of the target video is the same, so the target video is played at a fixed frame rate. For another example, considering that the shooting frame rate of the target video may not be a fixed value (that is, the video adopts a variable frame rate during the shooting stage), the video playback device can also first determine the frame rate of the target video, and then calculate the playback interval between the two adjacent video frames based on the frame rate. In this way, it can be ensured that the playback frame rate of the target video during the actual playback process is consistent with the shooting frame rate during its shooting process, so that the best playback effect of the target video can be presented as much as possible.

[0135] At this point, the introduction to the adjustment method of the jitter buffer queue described in the present disclosure is completed. Through the aforementioned embodiment, the video playback device can adjust the buffering duration of the jitter buffer queue to the maximum jitter duration when the current jitter duration of the current video frame meets the peak stretching condition, ensuring that the current video frame waits in the jitter buffer queue for the maximum jitter duration before being output and displayed. It can be understood that the current video frame is any video frame in the target video, so the video playback device can determine one by one whether each video frame of the target video meets the above-mentioned peak stretching condition, and adjust the cache duration of the jitter buffer queue accordingly according to the video frames that meet the condition.

[0136] According to the jitter buffer queue adjustment method of the disclosed embodiment, even in a bursty weak network scenario, where the current jitter duration of individual (sparse) video frames increases momentarily, the video playback device can still promptly adjust the jitter buffer queue's cache duration based on the video frame. Furthermore, the output timing of each video frame in the adjusted jitter buffer queue is also adjusted accordingly, thereby ensuring that the display timing of each video frame, including the current video frame, is as uniform as possible, avoiding noticeable lag, and achieving the intended function of the jitter buffer queue.

[0137] Exemplary media

[0138] After introducing the method of the exemplary embodiment of the present disclosure, next, reference is made to Figure 4 A medium according to an exemplary embodiment of the present disclosure is described.

[0139] In this exemplary embodiment, the above method can be implemented by a program product, such as a portable compact disc read-only memory (CD-ROM) containing program code, which can be executed on a device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, the readable medium 40 can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0140] The program product can be any combination of one or more readable media. The readable medium 40 can be a readable signal medium or a readable medium. The readable medium 40 can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples of readable media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0141] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a computer-readable medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0142] The program code contained on the readable medium 40 may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RE, etc., or any suitable combination thereof.

[0143] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and the like, as well as conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user computing device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0144] Exemplary devices

[0145] After introducing the medium of the exemplary embodiment of the present disclosure, next, reference is made to Figure 5 The apparatus of the exemplary embodiment of the present disclosure is described. Regarding the apparatus described below, the specific manner in which each functional module performs operations and the specific functions implemented after performing the operations have been described in detail in the aforementioned embodiments of the jitter buffer queue adjustment method, and will not be elaborated on here.

[0146] Figure 5 A block diagram of a jitter buffer queue adjustment device according to an embodiment of the present disclosure is schematically shown. The jitter buffer queue adjustment device may include:

[0147] The video frame input module 501 is configured to, in response to receiving a current video frame of a target video, input the current video frame into a jitter buffer queue, and determine a difference between a reception time of the current video frame and a reference time corresponding to the current video frame as a current jitter duration of the current video frame;

[0148] A first adjustment module 502 is configured to adjust the buffering duration of the jitter buffer queue to the maximum jitter duration corresponding to the current video frame if the current jitter duration satisfies the peak stretching condition, so that the current video frame waits in the jitter buffer queue for the maximum jitter duration before being output to the display component, wherein the maximum jitter duration is not less than the current jitter duration;

[0149] The second adjustment module 503 is used to adjust the buffering time of the jitter buffer queue from the maximum jitter time to the random jitter time at the current moment if no video frame meeting the peak stretching condition is received within the maintenance time period after the buffering time is adjusted, and the random jitter time is less than the maximum jitter time.

[0150] Optionally, the video frame input module 501 is specifically configured to:

[0151] When multiple image data packets containing the same sequence number are received, the multiple image data packets are reassembled into the current video frame corresponding to the sequence number.

[0152] Optionally, the current jitter duration satisfies a peak stretching condition, including: the current video frame satisfies at least one of the following, or the current video frame and at least one other video frame received within a discarded time period before the receiving moment respectively satisfy at least one of the following:

[0153] The current jitter duration is not less than a multiple threshold of the random jitter duration at the current moment;

[0154] The difference between the current jitter duration and the random jitter duration at the current moment is not less than the first duration threshold;

[0155] The current jitter duration is not less than a second duration threshold.

[0156] Optionally, the method further includes a duration determination module 504, configured to:

[0157] Determine the current jitter duration as the maximum jitter duration; or,

[0158] If at least one video frame meeting the peak stretching condition is received within the elimination time period before the receiving moment, the maximum value between the current jitter duration and the jitter duration of the at least one video frame is determined as the maximum jitter duration.

[0159] Optionally, the system further includes a video frame output module 505, configured to:

[0160] The video frames in the jitter buffer queue are sequentially output to the display component according to corresponding playback intervals to play the target video.

[0161] Optionally, each video frame in the jitter buffer queue has a corresponding sequence number, and the apparatus further includes a de-sequence reassembly module 506 for:

[0162] If the arrangement order of the video frames in the jitter buffer queue is different from the sequence of the serial numbers of the video frames, the video frames are reordered to adjust the arrangement order of the video frames to be the same as the sequence of the serial numbers.

[0163] Optionally, the video frame output module 505 is specifically configured to:

[0164] Determine the preset playback step length as the playback interval between the two adjacent video frames; or,

[0165] The frame rate of the target video is determined, and the playback interval between the two adjacent video frames is calculated according to the frame rate.

[0166] Optionally, the second adjustment module 503 is specifically configured to:

[0167] Determining an adjustment rate within a convergence period according to the maximum jitter duration and the random jitter duration at a current moment;

[0168] In the convergence time period, the buffering duration is uniformly adjusted from the maximum jitter duration to the random jitter duration at the current moment according to the adjustment rate.

[0169] Optionally, the random jitter duration at the current moment is a preset fixed duration; or is obtained by calculating each received video frame of the target video through a Kalman filter algorithm.

[0170] Optionally, the maintenance duration of the maintenance time period is a preset duration, or an expected maintenance duration issued by a data platform.

[0171] Optionally, a sample uploading module 507 is further included, which is used to:

[0172] The freeze duration generated during the playback of the target video is uploaded to the data platform, so that the data platform can count the freeze durations uploaded by multiple video playback devices to obtain the expected maintenance duration under at least one statistical dimension. The expected maintenance duration is used by the data platform to be sent to the video playback devices that meet the corresponding statistical dimension.

[0173] Exemplary computing devices

[0174] After introducing the method, medium and apparatus of the exemplary embodiments of the present disclosure, the following is a reference to Figure 6 A computing device according to an exemplary embodiment of the present disclosure is described.

[0175] Figure 6 The computing device 60 shown is only an example and should not bring any limitation to the functionality and scope of use of the embodiments of the present disclosure.

[0176] like Figure 6 As shown, computing device 60 is a general-purpose computing device. Components of computing device 60 may include, but are not limited to, at least one processing unit 601, at least one storage unit 602, and a bus 603 connecting different system components (including processing unit 601 and storage unit 602).

[0177] The bus 603 includes a data bus, a control bus, and an address bus.

[0178] The storage unit 602 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 6021 and / or a cache memory 6022 , and may further include a readable medium in the form of a non-volatile memory, such as a read-only memory (ROM) 6023 .

[0179] The storage unit 602 may also include a program / utility 6025 having a set (at least one) of program modules 6024, such program modules 6024 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0180] The computing device 60 may also communicate with one or more external devices 604 (eg, a keyboard, pointing device, etc.).

[0181] Such communication may be performed via input / output (I / O) interface 605. Furthermore, computing device 60 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via network adapter 606. Figure 6 As shown, network adapter 606 communicates with other modules of computing device 60 via bus 603. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with computing device 60, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0182] It should be noted that although the above detailed description mentions several units / modules or sub-units / modules of the jitter buffer queue adjustment device, this division is merely exemplary and not mandatory. In practice, according to embodiments of the present disclosure, the features and functions of two or more units / modules described above may be embodied in a single unit / module. Conversely, the features and functions of a single unit / module described above may be further divided and embodied by multiple units / modules.

[0183] Furthermore, although the operations of the disclosed method are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all illustrated operations must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0184] Although the spirit and principles of the present disclosure have been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the specific embodiments disclosed, and the division into various aspects does not mean that the features of these aspects cannot be combined to benefit. Such division is merely for the convenience of expression. The present disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A method for adjusting a jitter buffer queue, comprising: In response to receiving a current video frame of a target video, inputting the current video frame into a jitter buffer queue, and determining a difference between a reception time of the current video frame and a reference time corresponding to the current video frame as a current jitter duration of the current video frame; When the current jitter duration satisfies the peak stretching condition, adjusting the buffering duration of the jitter buffer queue to the maximum jitter duration corresponding to the current video frame, so that the current video frame waits in the jitter buffer queue for the maximum jitter duration before being output to the display component, wherein the maximum jitter duration is not less than the current jitter duration; If no video frame meeting the peak stretching condition is received within the maintenance time period after the buffering duration is adjusted, the buffering duration of the jitter buffer queue is adjusted from the maximum jitter duration to the random jitter duration at the current moment, and the random jitter duration is less than the maximum jitter duration.

2. The method according to claim 1, wherein receiving the current video frame of the target video comprises: When multiple image data packets containing the same sequence number are received, the multiple image data packets are reassembled into the current video frame corresponding to the sequence number.

3. The method according to claim 1, wherein the current jitter duration satisfies a peak stretching condition, comprising: The current video frame satisfies at least one of the following conditions, or the current video frame and at least one other video frame received within a elimination period before the receiving moment respectively satisfy at least one of the following conditions: The current jitter duration is not less than a multiple threshold of the random jitter duration at the current moment; The difference between the current jitter duration and the random jitter duration at the current moment is not less than the first duration threshold; The current jitter duration is not less than a second duration threshold.

4. The method according to claim 1, determining the maximum jitter duration corresponding to the current video frame, comprising: Determining the current jitter duration as the maximum jitter duration; or, If at least one video frame meeting the peak stretching condition is received within the elimination time period before the receiving moment, the maximum value between the current jitter duration and the jitter duration of the at least one video frame is determined as the maximum jitter duration.

5. The method according to claim 1, further comprising: The video frames in the jitter buffer queue are sequentially output to the display component according to corresponding playback intervals to play the target video.

6. The method according to claim 5, wherein each video frame in the jitter buffer queue has a corresponding sequence number, and the method further comprises: If the arrangement order of the video frames in the jitter buffer queue is different from the sequence of the serial numbers of the video frames, the video frames are reordered to adjust the arrangement order of the video frames to be the same as the sequence of the serial numbers.

7. The method according to claim 5 or 6, wherein determining the playback interval between two adjacent video frames comprises: Determine a preset playback step length as the playback interval between two adjacent video frames; or, The frame rate of the target video is determined, and the playback interval between the two adjacent video frames is calculated according to the frame rate.

8. The method according to claim 1, wherein adjusting the buffering duration of the jitter buffer queue from the maximum jitter duration to the random jitter duration at the current moment comprises: Determining an adjustment rate within a convergence period according to the maximum jitter duration and the random jitter duration at a current moment; In the convergence time period, the buffering duration is uniformly adjusted from the maximum jitter duration to the random jitter duration at the current moment according to the adjustment rate.

9. According to the method described in any one of claims 1, 3 and 8, the random jitter duration at the current moment is a preset fixed duration; or is obtained by calculating each video frame of the received target video through a Kalman filter algorithm.

10. The method according to claim 1, wherein the maintenance duration of the maintenance time period is a preset duration, or an expected maintenance duration issued by a data platform.

11. The method according to claim 10, further comprising: The freeze duration generated during the playback of the target video is uploaded to the data platform as sample data, so that the data platform can count the freeze durations uploaded by multiple video playback devices to obtain the expected maintenance duration under at least one statistical dimension. The expected maintenance duration is used by the data platform to be sent to the video playback devices that meet the corresponding statistical dimension.

12. A jitter buffer queue adjustment device, comprising: a video frame input module, configured to, in response to receiving a current video frame of a target video, input the current video frame into a jitter buffer queue, and determine a difference between a reception time of the current video frame and a reference time corresponding to the current video frame as a current jitter duration of the current video frame; a first adjustment module, configured to adjust the buffering duration of the jitter buffer queue to the maximum jitter duration corresponding to the current video frame if the current jitter duration satisfies a peak stretching condition, so that the current video frame waits in the jitter buffer queue for the maximum jitter duration before being output to the display component, wherein the maximum jitter duration is not less than the current jitter duration; The second adjustment module is used to adjust the buffering time from the maximum jitter time to the random jitter time at the current moment if no video frame meeting the peak stretching condition is received within the maintenance time period after the buffering time is adjusted, and the random jitter time is less than the maximum jitter time.

13. A medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

14. A computing device comprising: processor; a memory for storing processor-executable instructions; The processor implements the method according to any one of claims 1 to 11 by running the executable instructions.

Images