Live video playing method and device, storage medium and computer program product

Abstract

The present disclosure relates to a live video playing method and device, a storage medium and a computer program product. The method comprises: obtaining a current download frame rate of a live video in a live process and an original frame rate of the live video; obtaining a download frame rate jitter time difference based on the download frame rate and the original frame rate; obtaining a transcoding jitter frame rate of the live video, and obtaining a transcoding output frame rate jitter time difference of the live video based on the transcoding jitter frame rate and the original frame rate; obtaining a jitter delay of the live video based on the download frame rate jitter time difference and the transcoding output frame rate jitter time difference, and determining a buffer time length of the live video based on the jitter delay. In this way, by comprehensively considering the influence of the download frame rate fluctuation and the transcoding jitter on the live delay, and by adjusting the buffer length of the video buffer, the video live stalling problem can be effectively solved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of live video playing, and particularly relates to a live video playing method and device, a storage medium and a computer program product. BACKGROUND

[0002] In a low-latency live scenario, in order to avoid as much as possible the phenomenon of lag during live video playing, the related art mainly adjusts the length of buffered video according to network status. However, the related art considers only a single factor when dealing with the problem of lag in a low-latency live scenario, and cannot effectively solve the problem of lag during live video playing. SUMMARY

[0003] The present disclosure provides a live video playing method, device, storage medium and computer program product.

[0004] According to a first aspect of the present disclosure, a live video playing method is provided, and the method comprises:

[0005] obtaining a current download frame rate of a live video during live playing and an original frame rate of the live video;

[0006] obtaining a download frame rate jitter time difference based on the download frame rate and the original frame rate;

[0007] obtaining a transcoding output frame rate jitter time difference of the live video based on a transcoding jitter frame rate of the live video and the original frame rate;

[0008] obtaining a jitter delay of the live video based on the download frame rate jitter time difference and the transcoding output frame rate jitter time difference, and determining a buffer duration of the live video based on the jitter delay.

[0009] According to a second aspect of the present disclosure, a live video playing device is provided, and the device comprises:

[0010] a live data obtaining module configured to obtain a current download frame rate of a live video during live playing and an original frame rate of the live video;

[0011] a download frame rate jitter time difference obtaining module configured to obtain a download frame rate jitter time difference based on the download frame rate and the original frame rate;

[0012] a transcoding output frame rate jitter time difference obtaining module configured to obtain a transcoding output frame rate jitter time difference of the live video based on a transcoding jitter frame rate of the live video and the original frame rate;

[0013] The buffer duration determination module is configured to obtain a jitter delay of the live video based on the download frame rate jitter time difference and the transcoding output frame rate jitter time difference, and determine a buffer duration of the live video based on the jitter delay.

[0014] According to a third aspect of the present disclosure, an electronic device is provided. The electronic device comprises a memory and a processor, the memory having stored thereon a computer program, the processor implementing the method as described above when executing the program.

[0015] According to a fourth aspect of the present disclosure, a computer readable storage medium is provided, having stored thereon a computer program, the program being executed by a processor to implement the method as described above.

[0016] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, the computer program being executed by a processor to implement the method as described above.

[0017] The live video playing method, device, storage medium and computer program product provided by the embodiments of the present disclosure can obtain the jitter delay of the live video by obtaining the current download frame rate of the live video in the live process and the original frame rate of the live video, and obtaining the download frame rate jitter time difference. And based on the obtained transcoding jitter frame rate and the original frame rate of the live video, the transcoding output frame rate jitter time difference of the live video is obtained. The jitter delay of the live video obtained by the download frame rate jitter time difference and the transcoding output frame rate jitter time difference can determine the buffer duration of the live video. In this way, by comprehensively considering the influence of the download frame rate fluctuation and the transcoding jitter on the live delay, the buffer length of the video buffer can be adjusted, and the problem of lag during live video can be effectively solved. BRIEF DESCRIPTION OF DRAWINGS

[0018] More details, features and advantages of the present disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0019] Figure 1 A flowchart of a live video playing method provided by an exemplary embodiment of the present disclosure is shown;

[0020] Figure 2 A flowchart of a live video playing method provided by another exemplary embodiment of the present disclosure is shown;

[0021] Figure 3 A flowchart of a live video playing method provided by yet another exemplary embodiment of the present disclosure is shown;

[0022] Figure 4 A functional module schematic block diagram of a live video playing device provided by an exemplary embodiment of the present disclosure is shown;

[0023] Figure 5A structural block diagram of an electronic device provided for an exemplary embodiment of the present disclosure;

[0024] Figure 6 A structural block diagram of a computer system provided for an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] Embodiments of the present disclosure will be described in more detail by referring to the drawings. Although certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein, but rather the embodiments are provided so that the present disclosure can be more thoroughly and completely understood. It is understood that the drawings and embodiments of the present disclosure are for exemplary purposes only and are not intended to limit the scope of protection of the present disclosure.

[0026] It is understood that each of the steps recited in the method embodiments of the present disclosure can be executed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the execution of the steps shown. The scope of the present disclosure is not limited in this respect.

[0027] The term "comprising" and variations thereof as used herein are open-ended, that is, "comprising but not limited to." The term "based on" is "based, at least in part, on." The term "one embodiment" means "at least one embodiment." The term "another embodiment" means "at least one additional embodiment." The term "some embodiments" means "at least some embodiments." Related definitions are given throughout the detailed description. It is noted that the "first", "second", etc. concepts mentioned in the present disclosure are merely used to distinguish different devices, modules or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules or units.

[0028] It is noted that the modification of "one", "multiple" mentioned in the present disclosure is illustrative and not restrictive, and those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more".

[0029] In the low-latency live streaming scenario, the related technology mainly adjusts the video playback buffer length by evaluating the terminal network quality to combat the playback stall, so as to achieve the purpose of reducing the stall rate by combating network fluctuations. This makes the related technology in the low-latency live streaming scenario have too single factor consideration in the analysis of the stall problem, and does not consider the volatility of the video transcoding output frame rate. After the network amplification effect, the frame rate fluctuation of the video will be intensified, thereby causing the user to encounter the stall phenomenon in the low-latency live streaming even in the case of sufficient and stable network bandwidth. Therefore, the related technology has certain limitations in adjusting the playback buffer according to only the network fluctuation factor, and cannot completely solve the stall problem.

[0030] Therefore, the embodiments of the present disclosure can comprehensively improve the fluency of low-latency live streaming by dynamically adjusting the data length of the play buffer and the play strategy of the frame chasing logic by fusing various factors such as network fluctuation and video transcoding output fluctuation, and can bring the best low-latency live streaming viewing experience to users.

[0031] In the embodiments, the video source station transcodes live video and forwards the transcoded live video stream through a CDN (Content Delivery Network) server, and the terminal receives and plays the live video stream forwarded by the CDN. Specifically, the live video can be downloaded by a play network module in the terminal and cached in a play buffer module, and the video data in the buffer can be analyzed and processed by a play analysis control module in the terminal.

[0032] The video source station is a server device located in a machine room and is responsible for producing and pushing low-latency live video stream resources to a CDN server.

[0033] The CDN server is a server device located in a machine room and is responsible for distributing and sending low-latency live video stream resources.

[0034] The play network module is located in a user's mobile phone video APP (application) and is used to download live video stream resources from the CDN server side and store them in a play buffer module.

[0035] The play buffer module is located in a user's terminal such as a mobile phone video APP and is used to store and retrieve play buffer data, i.e., intermediate video stream play data.

[0036] The play analysis control module is located in a user's terminal such as a mobile phone video APP and is used to receive video frame rate collection data stored (i.e., downloaded) by the play buffer module and analyze and process the original data.

[0037] The embodiments of the present disclosure can specifically include two stages: the first stage is data collection and processing, in which the play analysis control module collects video download frame rate data and dynamically calculates and adjusts video buffer time length parameters and play strategies in combination with transcoding frame rate data; and the second stage is dynamic compensation, in which the play delay parameters and play strategies are dynamically compensated according to the actual play process stall frequency and time length data.

[0038] In the first stage, data collection and processing can be performed first.

[0039] (1) In the case of low latency, the terminal obtains the download frame rate raw data reported by the player on the terminal during playing the live video, which can include vFPS and netFPS; wherein vFPS represents the original frame rate of the video resource, i.e., the original frame rate of the live video; and netFPS represents the download frame rate of the current video of the player.

[0040] (2) Calculate the jitter time difference corresponding to the download frame rate of the video relative to the original frame rate of the video.

[0041] In the embodiment, the download frame rate of the current video and the original frame rate of the video can be obtained by the following formula (1) to obtain the downward jitter frame rate difference DFPS:

[0042] DFPS=vFPS-netFPS (1)

[0043] In the embodiment, the jitter time difference dT corresponding to the download frame rate jitter difference DFPS can be calculated by the following formula (2):

[0044] dT=(1000 / vFPS)*DFPS (2)

[0045] Wherein, the original frame rate of the video is the number of video frames played and displayed within 1000ms, and the average playing time of each frame can be obtained by calculation, and the jitter time difference corresponding to the number of jitter frames within 1000ms can be calculated.

[0046] (3) Obtain the smooth average f(sT) of the download frame rate jitter time difference.

[0047] In the implementation, after obtaining the jitter time difference dT, the dT can be smoothed by the following formula (3).

[0048]

[0049] Wherein, sT represents the influence degree of the download frame rate on the live video playing delay based on the original frame rate playing the live video within a unit time; Vnet represents the download rate of the live video; Mbr represents the code rate of the live video; the initial value of sT is the first dT value when the network bandwidth resource is sufficient, i.e., Vnet is not less than Mbr; a is a smoothing coefficient (value interval 0-1), which can be set according to needs, and can be an empirical value; dT is the current jitter time difference.

[0050] It should be noted that when the network bandwidth is insufficient, i.e., VnetMbr, the current dT value is not included as a calculation factor of sT data, to avoid the interference of the network quality itself on the download frame rate jitter; when the network bandwidth is sufficient, the current download frame rate jitter time difference smoothing average sT is calculated by combining the smoothing coefficient, the current jitter time difference dT data and the sT data of the last period.

[0051] wherein the download frame rate jitter time difference smoothing average f(sT) represents the time length that the terminal theoretically should obtain per second under the environment compared to the actual obtained data playing time length, that is, in the low delay video live scene, the buffer data playing time length required to cope with the network jitter factor.

[0052] (4) Calculate the video source transcoding output frame rate jitter time difference voT.

[0053] In the embodiment, the video source transcoding output frame rate jitter time difference voT can be calculated by the following formula (4).

[0054] voT = voFPS * (1000 / vFPS) (4)

[0055] wherein voT represents the influence degree of the transcoding jitter frame rate on the live video playing delay when playing the live video based on the original frame rate per unit time; voFPS is the video source transcoding output average downward jitter frame rate, which is related to the server video transcoding performance, and the voFPS value of the same source station is generally a fixed value, which can be obtained by the video APP of the mobile terminal and the server for message interaction; the video source transcoding output frame rate jitter time difference voT can be the buffer data playing time length required to cope with the transcoding output frame rate jitter factor in the low delay live; vFPS represents the original frame rate of the video resource.

[0056] (5) Calculate the live video delay T.

[0057] In the embodiment, the live video delay T can be calculated by the following formula (5):

[0058] T = sT + voT (5)

[0059] wherein the live video delay T represents the minimum delay of the live video in the low delay live scene without appearing jitter when playing, that is, the player buffer playing time length. T represents the coping resistance value to avoid the playing jitter event caused by the network jitter factor and the transcoding output jitter factor. When the buffer video of the live video buffer contains at least T time length, the playing can avoid the jitter phenomenon caused by the network jitter and the transcoding jitter in the low delay scene.

[0060] In the second stage, the video buffered in the video buffer can be dynamically compensated.

[0061] In the embodiment, the historical jitter events reported by the player can be collected, and the jitter number N in the historical jitter time and the jitter time length Ti when each jitter occurs can be counted. And according to the jitter number N and the jitter time length Ti, it is judged whether the buffer data is too long to trigger the delay compensation;

[0062] Specifically, if the player does not appear to be stuck in the current cycle time, and the video buffer data playable video duration is not less than the threshold T during this period η , then the delay T of the player, that is, the buffer length corresponding to the playing duration, can be reduced by T γ . To avoid the risk of causing a stuck problem due to the reduction of the delay, the value range of T γ may be 0-T η ; T η may be an empirical value.

[0063] When the number of stuck times reaches the threshold M times, or the total stuck time ∑Ti exceeds a certain threshold, that is, the duration of the relatively obvious stuck phenomenon perceived by the human eye, then the delay compensation is triggered, and the delay T of the player, that is, the buffer length corresponding to the playing duration, can be increased by T γ . Wherein, the value range of T γ may be half of the total stuck time, that is, ∑Ti / 2, which can be set according to the needs, and the embodiments are not limited thereto.

[0064] In the embodiments, the player can also compensate for the playing of the live video.

[0065] For example, when the actual video buffer data playable duration durT is too large, the video can be played at fast-forward to reduce the delay of video playing; when the actual video buffer data playable duration durT is too small, the video can be played at slow-forward to avoid the playing stuck phenomenon caused by the small video data cached in the buffer.

[0066] Specifically, the difference between the video playing delay T and the actual video buffer data playable duration durT can be obtained, and it can be decided whether to use fast-forward or slow-forward mode for playing effect experience optimization, and the specific strategy is as follows:

[0067] When the difference between the buffer playable duration durT and the video playing delay T is greater than the threshold ΔT1, the player can be set to use a fast-forward mode with a speed greater than 1 times. In this way, the time difference between the playing picture and the live source picture is shortened by quickly consuming the buffer data, thereby achieving the purpose of low delay of live video; at the same time, the buffer playable duration durT is ensured to be not less than the video playing delay T, so as to avoid the stuck phenomenon of the live video during playing.

[0068] When the difference between the buffer playable duration durT and the video playing delay T is less than the threshold ΔT2, the player is set to use a slow-forward mode with a speed less than 1 times, so as to slow down the consumption of the buffer data to gain time for playing data download, thereby achieving the purpose of low stuck of live video.

[0069] In other cases, the player uses 1 times speed to play, that is, normal play of the live video. For example, when the difference between the buffer playable duration durT and the live video playing delay T is between AT2 and AT1, the live video is played normally. Wherein, AT1 and AT2 can be set according to needs, which can be experience values.

[0070] Based on the above embodiments, the embodiments of the present disclosure also provide a live video playing method, as shown in the formula (1), which can include the following steps: Figure 1 As shown in the formula (1), the method can include the following steps:

[0071] In step S110, the current download frame rate of the live video in the live process and the original frame rate of the live video are obtained.

[0072] In the embodiments, in the low-delay video live scene, when the user watches the live video through the terminal such as a mobile phone, the user may be affected by network fluctuations, and then the phenomenon of video playing lag occurs when the live video is played. Before the live video is played, the live data needs to be downloaded and a certain amount of video data needs to be buffered in the buffer of the player, so that the video can be played by reading the buffered video data in the buffer. When the video is played, the video is usually played at a fixed frame rate, such as 24 frames per second. Because the data amount contained in each frame of the video frame may be different, such as the amount of data contained in the key frame is usually greater than the amount of data contained in the non-key frame, when the video is played at a fixed frame number, the number of video frames downloaded (i.e. buffered) in a unit of time may be reduced due to the large amount of data contained in some frames, so that the video data buffered in the buffer is not enough to support the video to be played at a fixed frame number, and then the video lag occurs.

[0073] Therefore, in the embodiments, the current download frame rate of the video in the live process and the original frame rate of the live video need to be obtained, and then the influence degree of the current download frame rate of the live video on the live video playing delay is determined.

[0074] In step S120, the download frame rate jitter time difference is obtained based on the download frame rate and the original frame rate.

[0075] Wherein, the download jitter time difference represents the influence degree of the download frame rate on the live video playing delay when the live video is played based on the original frame rate in a unit of time.

[0076] In the embodiments, the frame rate difference between the original frame rate and the download frame rate can be obtained by the above formula (1), and based on the frame rate difference and the original frame rate of the live video, the download frame rate jitter time difference can be obtained, which can be obtained by the above formula (2).

[0077] In the embodiment, if the download frame rate is less than the original frame rate of the live video, the number of video frames downloaded per unit time is less than the number of video frames played per unit time, which causes the video to be played to be stuck. Therefore, when the download frame rate is less than the original frame rate of the live video, a certain amount of video data needs to be buffered in the buffer of the video in advance to make up for the case that the download frame rate is small. If only the download frame rate is considered, the buffer needs to at least buffer video data of a first target playing time in advance, and the video data of the first target playing time is the above-mentioned download frame rate jitter time difference, so as to ensure smooth playing of the live video and avoid the situation of being stuck.

[0078] In step S130, the transcoding jitter frame rate of the live video is obtained, and the transcoding output frame rate jitter time difference of the live video is obtained based on the transcoding jitter frame rate and the original frame rate.

[0079] The transcoding output frame rate jitter time difference represents the influence degree of the transcoding jitter frame rate on the playing delay of the live video when the live video is played based on the original frame rate per unit time.

[0080] In the embodiment, in the low-latency video live scene, the user may be affected by the transcoding jitter when watching the live video through a terminal such as a mobile phone, and then the phenomenon of video playing stuck occurs when the live video is played. Before the live video is played, the live data needs to be downloaded and a certain amount of video data needs to be buffered in the buffer of the player, so that the video can be played by reading the video data buffered in the buffer.

[0081] When the video is played, the video is usually played at a fixed frame rate of 24 frames per second. Since the transcoding rates of different transcoding servers may be different, and the transcoding may be downward jittered during the transcoding process, once the transcoding rate of the corresponding transcoding server is lower than the original frame rate of the live video, the number of video frames downloaded (i.e. buffered) per unit time will decrease, that is, the buffer cannot download enough video frames per unit time, so that the video data buffered in the buffer is not enough to support the video to be played at a fixed frame rate, and then the phenomenon of video stuck occurs.

[0082] In the embodiment, if the video transcoding output exists downward jitter frame rate, the number of video frames received by the terminal in a unit of time is affected, which can cause the number of video frames downloaded by the terminal in a unit of time to be less than the number of video frames played in a unit of time, and further cause the phenomenon of lag during video playing. Therefore, a certain amount of video data needs to be buffered in the buffer of the video in advance to compensate for the case where the download frame rate is small. If only the jitter rate of the transcoding of the live video is considered, the buffer needs to at least buffer video data of the second target playing time in advance, and the video data of the first target playing time is the jitter time difference of the transcoding output frame rate of the live video. In this way, smooth playing of the live video can be ensured, and the phenomenon of lag can be avoided.

[0083] It should be noted that the transcoding jitter frame rate of the live video can be the average downward jitter frame rate of the video source transcoding output. By determining the push stream server corresponding to the live video, the average downward jitter frame rate of the push stream server when transcoding the live video is obtained, and the average downward jitter frame rate is taken as the transcoding jitter frame rate of the live video. The average downward jitter frame rate of the video source transcoding output in a unit of time can be obtained, the jitter time difference of the transcoding output frame rate when the live video is played can be obtained, and when the playing time of the video data buffered in the video buffer is not less than the jitter time difference of the transcoding output frame rate, smooth playing of the live video can be ensured, and the phenomenon of lag can be avoided.

[0084] In step S140, the jitter delay of the live video is obtained based on the download frame rate jitter time difference and the transcoding output frame rate jitter time difference, and the buffering time of the live video is determined based on the jitter delay.

[0085] In the embodiment, the influence of network fluctuation and transcoding jitter on the live video during playing is considered respectively, so that the influences of the two can be combined, that is, based on the download frame rate jitter time difference and the transcoding output frame rate jitter time difference, the buffering time buffered in the video buffer is ensured to be at least not less than the sum of the download frame rate jitter time difference and the transcoding output frame rate jitter time difference, so that smooth playing of the live video can be ensured, and the phenomenon of lag can be avoided.

[0086] It should be noted that the buffering time of the live video in the embodiment is the playing time corresponding to the buffered data in the buffer.

[0087] The live video playing method provided in the embodiments of the present disclosure can obtain the jitter time difference of the download frame rate by obtaining the current download frame rate of the live video in the live process and the original frame rate of the live video. The jitter time difference of the transcoding output frame rate of the live video can be obtained based on the obtained jitter frame rate of the transcoding of the live video and the original frame rate of the live video. The jitter delay of the live video obtained through the jitter time difference of the download frame rate and the jitter time difference of the transcoding output frame rate can determine the buffering time length of the live video. In this way, by comprehensively considering the influence of the download frame rate fluctuation and the transcoding jitter on the live delay, the buffering length of the video in the video buffer can be adjusted, and the live video can be effectively solved.

[0088] Based on the above embodiments, in another embodiment provided in the present disclosure, in order to avoid the large fluctuation of the download frame rate from having a large impact on the video buffered in the video buffer, the embodiment can also perform smoothing processing on the jitter time difference of the download frame rate.

[0089] In the embodiment, specifically, the jitter time difference of the download frame rate in the current period can be discarded by combining the above formula (3) and the corresponding embodiment and by obtaining the download rate of the live video and the code rate of the live video, in the case that the download rate of the live video is less than the code rate of the live video. In this way, in the case that the network bandwidth is insufficient, that is, Vnet

[0090] In the case that the download rate of the live video is not less than the code rate of the live video, the jitter time difference of the download frame rate in the current period can be smoothed based on the jitter time difference of the download frame rate in the current period and the jitter time difference of the download frame rate in the previous period adjacent to the current period. In this way, in the case that the network bandwidth is sufficient, the smoothed average of the jitter time difference of the download frame rate in the current period can be calculated by combining the smoothing coefficient and the jitter time difference of the download frame rate in the current period and the jitter time difference of the download frame rate in the previous period, and the purpose of smoothing the jitter time difference of the download frame rate in the current period can be achieved.

[0091] Therefore, in the case that the download rate of the live video is not less than the code rate of the live video, the jitter delay of the live video can be obtained based on the smoothed jitter time difference of the download frame rate and the jitter time difference of the transcoding output frame rate, based on the jitter time difference of the download frame rate and the jitter time difference of the transcoding output frame rate. The jitter delay of the live video obtained in this way can be smoother, and the large fluctuation of the video buffered in the video buffer can be avoided, and the case that the live video has too large delay or is stuck can be avoided.

[0092] It should be noted that one period in the embodiment can be 1 second, and the embodiment is not limited thereto.

[0093] Based on the above embodiments, in another embodiment provided by the present disclosure, as shown in Figure 2 The method can further include the following steps:

[0094] In step S150, the stall information of the live video in the current period is obtained.

[0095] The stall information includes the stall times and the stall duration.

[0096] In step S160, when the live video does not appear to be stalled in the current period, and the playing duration of the buffered video in the video buffer is not less than the first preset duration, the playing duration of the buffered video in the video buffer is reduced.

[0097] In the embodiment, if the live video does not appear to be stalled in the current period, and the playing duration of the buffered video in the video buffer is not less than the first preset duration, it means that there is enough video data cached in the video buffer. Although this can avoid the stall phenomenon during playing, it will cause a large delay to the live video. Therefore, in this case, the video data in the video buffer can be appropriately reduced, for example, by fast-forward playing, or according to the video content, the unimportant content can be skipped to play, so as to achieve the purpose of reducing the buffered data in the buffer.

[0098] In step S170, when the stall times of the live video in the current period are greater than the first threshold, or the total stall duration in the current period is greater than the second preset duration, the playing duration of the buffered video in the video buffer is increased.

[0099] In the embodiment, when the stall times of the live video in the current period are greater than the first threshold, or the total stall duration in the current period is greater than the second preset duration, it means that the amount of data of the buffered video in the video buffer is not enough to support the stall-free playing of the live video. At this time, the playing duration of the buffered video in the video buffer needs to be increased. For example, by slow playing or repeatedly playing the buffered data, more buffering time is created for the video data in the buffer, so as to achieve the purpose of stall-free playing of the live video.

[0100] It should be noted that the size of the video data reduced or increased in the buffer in the embodiment can refer to the description of the above embodiments, which will not be repeated here.

[0101] Based on the above embodiments, in another embodiment provided by the present disclosure, as shown in Figure 3 The method can further include the following steps:

[0102] In step S191, the playing duration of the buffered video in the video buffer and the pre-set playing delay in the live video player are obtained.

[0103] In step S192, when the difference between the playing duration of the video buffered in the video buffer and the playing delay is greater than a second threshold, the live video is played at a fast-forward speed by the player.

[0104] In the embodiment, when the playing duration of the video data buffered in the video buffer is greater than the playing delay by the second threshold, it indicates that too much video is buffered in the video buffer, which will further increase the video delay. At this time, the length of the video in the buffer can be consumed by fast-forward playing to reduce the delay during live video playing. The second threshold can be the threshold value AT1 in the above embodiment.

[0105] In step S193, when the difference between the playing duration of the video buffered in the video buffer and the playing delay is less than a third threshold, the live video is played at a slow-forward speed by the player.

[0106] When the playing duration of the video data buffered in the video buffer is less than the playing delay by the third threshold, it indicates that too little video is buffered in the video buffer, which will cause the live video to appear a stuttering phenomenon during playing. At this time, the length of the video in the buffer can be increased by slow-forward playing to avoid the stuttering during live video playing. The third threshold can be the threshold value AT2 in the above embodiment.

[0107] In step S194, when the difference between the playing duration of the video buffered in the video buffer and the playing delay is between the third threshold and the second threshold, the live video is played at a normal speed by the player.

[0108] In the embodiment, when the difference between the playing duration durT of the video buffered in the buffer and the playing delay T of the live video is between AT2 and AT1, the live video can be played at a normal speed, which will neither cause too much delay of the live video nor cause a stuttering phenomenon.

[0109] In the case of dividing each functional module corresponding to each function, the embodiment of the present disclosure provides a live video playing device, which can be a server, a terminal or a chip applied to a server. Figure 4 The functional module schematic diagram of the live video playing device provided by an example embodiment of the present disclosure is shown in FIG. 1. As shown in the figure, the live video playing device includes: Figure 4

[0110] A live data obtaining module 10 is configured to obtain a current download frame rate of a live video in a live process and an original frame rate of the live video.

[0111] A download frame rate jitter time difference obtaining module 20 is configured to obtain a download frame rate jitter time difference based on the download frame rate and the original frame rate.

[0112] ​The transcoding output frame rate jitter time difference acquisition module 30 is configured to acquire a transcoding jitter frame rate of the live video, and acquire a transcoding output frame rate jitter time difference of the live video based on the transcoding jitter frame rate and the original frame rate.

[0113] The buffer duration determination module 40 is configured to acquire a jitter delay of the live video based on the download frame rate jitter time difference and the transcoding output frame rate jitter time difference, and determine a buffer duration of the live video based on the jitter delay.

[0114] In another embodiment provided in the present disclosure, the download frame rate jitter time difference acquisition module is specifically configured to:

[0115] acquire a frame rate difference between the original frame rate and the download frame rate;

[0116] acquire a download frame rate jitter time difference based on the frame rate difference and the original frame rate.

[0117] In another embodiment provided in the present disclosure, the device further comprises a smoothing processing module, which is specifically configured to:

[0118] acquire a download rate of the live video and a live video code rate;

[0119] in a case where the download rate of the live video is less than the live video code rate, discard the download frame rate jitter time difference in a current period;

[0120] in a case where the download rate of the live video is not less than the live video code rate, perform smoothing processing on the download frame rate jitter time difference in a current period based on the download frame rate jitter time difference in the current period and a download frame rate jitter time difference in a previous period adjacent to the current period.

[0121] In another embodiment provided in the present disclosure, the smoothing processing module is specifically configured to:

[0122] in a case where the download rate of the live video is not less than the live video code rate, acquire a jitter delay of the live video based on the download frame rate jitter time difference after the smoothing processing and the transcoding output frame rate jitter time difference.

[0123] In another embodiment provided in the present disclosure, the device further comprises a transcoding jitter frame rate determination module, which is specifically configured to:

[0124] determine a push stream server corresponding to the live video;

[0125] acquire an average downward jitter frame rate corresponding to the push stream server when the live video is transcoded, and take the average downward jitter frame rate as the transcoding jitter frame rate of the live video.

[0126] In another embodiment provided by the present disclosure, the device further comprises:

[0127] The stall information acquisition module is configured to acquire stall information of the live video in the current period, the stall information comprising a stall frequency and a stall duration;

[0128] The play duration reduction module is configured to reduce the play duration of the buffered video in the video buffer when the live video does not stall in the current period and the play duration of the buffered video in the video buffer is not less than the first preset duration.

[0129] The play duration increase module is configured to increase the play duration of the buffered video in the video buffer when the stall frequency of the live video in the current period is greater than a first threshold or the total stall duration in the current period is greater than a second preset duration.

[0130] In another embodiment provided by the present disclosure, the device further comprises a play control module, which is specifically configured to:

[0131] acquire the play duration of the buffered video in the video buffer and a play delay preset in the player corresponding to the live video;

[0132] when the difference between the play duration and the play delay is greater than a second threshold, fast-forward play the live video through the player;

[0133] when the difference between the play duration and the play delay is less than a third threshold, slow-forward play the live video through the player;

[0134] when the difference between the play duration and the play delay is between the third threshold and the second threshold, normally play the live video through the player.

[0135] The live video play device provided by the embodiments of the present disclosure can obtain the download frame rate jitter time difference by acquiring the current download frame rate of the live video in the live process and the original frame rate of the live video. And based on the acquired transcoding jitter frame rate and the original frame rate of the live video, the transcoding output frame rate jitter time difference of the live video is obtained. The jitter delay of the live video obtained through the download frame rate jitter time difference and the transcoding output frame rate jitter time difference can determine the buffering duration of the live video. In this way, by comprehensively considering the influence of the download frame rate fluctuation and the transcoding jitter on the live delay, the buffering length of the buffered video in the video buffer can be adjusted, so as to effectively solve the stall problem in the live video.

[0136] The electronic device provided by the embodiment of the present disclosure further includes at least one processor, and a memory for storing instructions executable by the at least one processor; wherein the at least one processor is configured to execute the instructions to implement the above-mentioned method disclosed by the embodiment of the present disclosure.

[0137] Figure 5 A structural schematic diagram of an electronic device provided by an exemplary embodiment of the present disclosure is shown in FIG. 18. Figure 5 As shown in FIG. 18, the electronic device 1800 includes at least one processor 1801 and a memory 1802 coupled to the processor 1801, and the processor 1801 can execute corresponding steps in the above-mentioned method disclosed by the embodiment of the present disclosure.

[0138] The processor 1801 can also be referred to as a central processing unit (CPU), which can be an integrated circuit chip having a signal processing capability. Each step in the above-mentioned method disclosed by the embodiment of the present disclosure can be completed by an integrated logic circuit of hardware or an instruction in the form of software in the processor 1801. The processor 1801 can be a general-purpose processor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), an FPGA (field-programmable gate array) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in conjunction with the embodiment of the present disclosure can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in the memory 1802, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, or other mature storage media in the art. The processor 1801 reads information in the memory 1802 and completes the steps of the above-mentioned method in conjunction with the hardware thereof.

[0139] In addition, when various operations / processes according to the present disclosure are implemented by software and / or firmware, the computer system with a dedicated hardware structure, such as the computer system 1900 shown in FIG. 19, can be installed with programs constituting the software, and the computer system, when installed with various programs, can perform various functions, including functions such as those described above. Figure 6 Figure 6 A structural block diagram of a computer system provided by an exemplary embodiment of the present disclosure is shown in FIG. 19. ​

[0140] The computer system 1900 is intended to represent various forms of digital electronic computer devices, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0141] As shown in Figure 6 The computer system 1900 includes a computing unit 1901 that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1902 or a computer program loaded from a storage unit 1908 into a random access memory (RAM) 1903. Various programs and data required for the operation of the computer system 1900 can also be stored in the RAM 1903. The computing unit 1901, the ROM 1902, and the RAM 1903 are connected to each other through a bus 1904. An input / output (I / O) interface 1905 is also connected to the bus 1904.

[0142] Various components in the computer system 1900 are connected to the I / O interface 1905, including an input unit 1906, an output unit 1907, the storage unit 1908, and a communication unit 1909. The input unit 1906 can be any type of device that can input information to the computer system 1900, and can receive inputted digital or character information, and generate key signal inputs related to user settings and / or function controls of the electronic device. The output unit 1907 can be any type of device that can present information, and can include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1908 can include, but is not limited to, a magnetic disk, an optical disk. The communication unit 1909 allows the computer system 1900 to exchange information / data with other devices through a network such as the Internet, and can include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0143] The computing unit 1901 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 1901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 1901 performs various methods and processes described above. For example, in some embodiments, the above-described methods disclosed by the embodiments of the present disclosure can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1908. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device via the ROM 1902 and / or the communication unit 1909. In some embodiments, the computing unit 1901 can be configured to perform the above-described methods disclosed by the embodiments of the present disclosure by any other appropriate means (e.g., by means of firmware).

[0144] The embodiments of the present disclosure also provide a computer-readable storage medium, wherein when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the above-described methods disclosed by the embodiments of the present disclosure.

[0145] The computer-readable storage medium in the embodiments of the present disclosure can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The above-described computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any appropriate combination thereof. More specifically, the above-described computer-readable storage medium can include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or a flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof.

[0146] The above-described computer-readable medium can be contained in the above-described electronic device; or can exist separately without being assembled into the electronic device.

[0147] The embodiments of the present disclosure also provide a computer program product, comprising a computer program, wherein the computer program is executed by a processor to implement the above-described methods disclosed by the embodiments of the present disclosure.

[0148] Computer program code for carrying out operations of the present disclosure can be written in any one or more of a variety of programming languages or combinations of languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0149] The flow diagrams and the block diagrams in the drawings are meant as possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams and the block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0150] The modules, components or units described in the embodiments of the present disclosure can be implemented by software or by hardware. In some cases, the name of the module, component or unit does not constitute a limitation on the module, component or unit itself.

[0151] The functions described above in the detailed description of embodiments of the present disclosure can be implemented in one or more hardware logic components or by computer instructions that are executed in a hardware logic component. For example, and without limitation, illustrative hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-Chip (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0152] The above description is merely exemplary of some embodiments of the present disclosure and of the principles thereof. It is to be understood that the disclosure is not limited in scope to the particular embodiments described herein, which are intended as examples only, and that the scope of the disclosure is, instead, defined by the appended claims, along with the full range of equivalents to which such claims are entitled. For example, the features of the various embodiments described above can be combined with each other, unless expressly prohibited by the above description.

[0153] While some specific embodiments of the present disclosure have been described in detail, those skilled in the art should understand that the above examples are merely exemplary and are not intended to limit the scope of the present disclosure. Those skilled in the art should understand that the above embodiments can be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A live video play method, characterized in that, The method comprises: obtaining a current download frame rate of a live video in a live broadcast process and an original frame rate of the live video; based on the download frame rate and the original frame rate, obtaining a download frame rate jitter time difference; obtaining a transcoding jitter frame rate of the live video, and based on the transcoding jitter frame rate and the original frame rate, obtaining a transcoding output frame rate jitter time difference of the live video; wherein the transcoding jitter frame rate comprises an average downward jitter frame rate; based on the download frame rate jitter time difference and the transcoding output frame rate jitter time difference, obtaining a jitter delay of the live video, and based on the jitter delay, determining a buffer duration of the live video.

2. The method of claim 1, wherein, The method further comprises: obtaining a frame rate difference between the original frame rate and the download frame rate; based on the frame rate difference and the original frame rate, obtaining a download frame rate jitter time difference.

3. The method of claim 1, wherein, The method further comprises: obtaining a download speed of the live video and a live video code rate; in a case where the download speed of the live video is less than the live video code rate, discarding the download frame rate jitter time difference in a current period; or, in a case where the download speed of the live video is not less than the live video code rate, based on the download frame rate jitter time difference of a current period and the download frame rate jitter time difference of a previous period adjacent to the current period, performing smoothing processing on the download frame rate jitter time difference of the current period.

4. The method of claim 3, wherein, The method further comprises: in a case where the download speed of the live video is not less than the live video code rate, based on the download frame rate jitter time difference after the smoothing processing and the transcoding output frame rate jitter time difference, obtaining a jitter delay of the live video.

5. The method of claim 1, wherein, The method further comprises: determining a push stream server corresponding to the live video; obtaining an average downward jitter frame rate corresponding to the live video when the push stream server transcodes the live video, and taking the average downward jitter frame rate as the transcoding jitter frame rate of the live video.

6. The method of claim 1, wherein, The method further comprises: obtaining stall information of the live video in a current period, the stall information comprising a stall number and a stall duration; in a case where the live video does not appear to stall in the current period, and a play duration of buffered video in a video buffer is not less than a first preset duration, reducing the play duration of the buffered video in the video buffer; or, in a case where the stall number of the live video in the current period is greater than a first threshold, or a total stall duration in the current period is greater than a second preset duration, increasing the play duration of the buffered video in the video buffer.

7. The method of claim 1, wherein, The method further comprises: obtaining a play duration of buffered video in a video buffer and a preset play delay in a player corresponding to the live video; in a case where a difference between the play duration and the play delay is greater than a second threshold, performing fast-forward play on the live video through the player; or, in a case where the difference between the play duration and the play delay is less than a third threshold, performing slow-forward play on the live video through the player; Or, when the difference between the playing duration and the playing delay is between the third threshold and the second threshold, normally playing the live video by the player.

8. An electronic device, comprising: Comprising: at least one processor; a memory for storing instructions executable by the at least one processor; wherein the at least one processor is configured to execute the instructions to implement the method of any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the method of any one of claims 1-7.

10. A computer program product, characterised in that, A computer program which, when executed by a processor, implements the method of any one of claims 1-7.

Images