A method for optimizing the smoothness of live video playback on a mobile terminal, a storage medium, and an electronic device

Abstract

The present invention specifically relates to the field of live video on a mobile terminal. The purpose of the present invention is to propose a method, storage medium and electronic device for optimizing the smoothness of live video playback on a mobile terminal. During the video playback process, video data to be played is cached in a video buffer, and the amount of video data received during a predetermined time period is calculated. According to the calculation result, the video playback frame rate is dynamically changed to avoid frequent freezes or delays in the video caused by network jitter. In the process of changing the frame rate, smoothing processing is used to avoid sudden changes in the frame rate causing a sense of jumpiness in the video content. The jitter pool technology is optimized, and the focus is on solving the problem of smooth video playback and reducing video delays in an unstable network environment.

Description

Technical Field

[0001] The present invention relates to the field of live video broadcasting, in particular to the field of live video broadcasting on a mobile terminal, and specifically to a method for optimizing the playback fluency of live video broadcasting on a mobile terminal. Background Art

[0002] The smoothness of live video playback is one of the most important factors affecting the quality of user experience. On the mobile side, due to the limitations of device performance and wireless network stability, video streaming transmission is prone to fluctuations, which affects the smoothness of video playback. To solve this problem, the current common solutions are:

[0003] 1. Add video jitter pool technology on the mobile terminal to alleviate the video playback jam. After the video stream data is transmitted to the mobile device, the mobile device opens a circular queue or a double queue video buffer to pre-store video data of a certain time or size by delayed playback. When the network environment is unstable and the video transmission rate jitters, the device obtains the pre-stored data through the jitter pool to achieve normal playback in a short time and optimize the user viewing experience; the use of jitter pool or video buffer technology is essentially to obtain smooth video playback in a short time by video delay means. In the case of long-term video playback, the video playback delay is prone to become more and more serious. At the same time, when the network environment returns to normal, the delayed data enters the jitter pool. At this time, the data must be processed by frame loss, which is prone to distortion such as video jitter.

[0004] 2. Add variable bitrate support. When the current network environment is unstable or the network bandwidth is poor, the video bitrate can be reduced to reduce the size of the video stream data to obtain a smooth video playback experience. Variable bitrate technology can achieve smooth video playback by obtaining low-bitrate video data when the network environment is poor, but low-bitrate video will affect the user's viewing experience, and the video continuity will be more or less interrupted when the bitrate is switched. Summary of the invention

[0005] In view of the shortcomings of the prior art, the purpose of the present invention is to propose a method, storage medium and electronic device for optimizing the smoothness of live video playback on a mobile terminal. During the video playback process, the video data to be played is cached in a video buffer, and the amount of video data received during this period is calculated according to a predetermined time period. According to the calculation result, the video playback frame rate is dynamically changed to avoid frequent video freezes or delays caused by network jitter. In the process of changing the frame rate, smoothing processing is used to avoid sudden changes in the frame rate causing a sense of jumpiness in the video content. The jitter pool technology is optimized, and the focus is on solving the problem of smooth video playback and reducing video delays in an unstable network environment.

[0006] In order to solve the above technical problems, the present invention provides a method for optimizing the smoothness of live video playback on a mobile terminal, which is characterized by comprising the following steps:

[0007] (1) When video data is received, create a video buffer;

[0008] (2) storing the data in the video buffer, determining the current network video download speed, and if the network download speed is lower than a preset speed, reducing the playback frame rate; if the download speed is higher than the preset speed, increasing the playback frame rate;

[0009] (3) After determining the frame rate change in step (2), the frame rate is calculated according to the number of video frames in the video buffer: when the number of video frames in the video buffer is lower than a predetermined value, the video playback rate is dynamically calculated by a monotonically increasing function to obtain the current frame rendering time on the screen; when the number of video frames in the video buffer is higher than a predetermined value, the current frame rendering time on the screen is dynamically calculated according to the number of video frames stored in the current video buffer, the preset number of video frames stored and the hardware basic drawing time; the video frame rate is smoothly changed to achieve the effect of smoothly changing the video playback rate.

[0010] Preferably, the video buffer is implemented with a dual queue data structure, and the dual queue data structure includes a first queue and a second queue, the first queue is used to store received data, and the second queue is used to store display data.

[0011] Preferably, in the step (2), the current network video download speed is judged, specifically including simultaneously recording the video frame reception time, calculating the average interval time between adjacent frames according to the video frame acquisition time, and judging the current network video download speed according to the calculation result.

[0012] Preferably, if the interval between adjacent frames of video in the video buffer is greater than a preset time value, it is determined that the video delay exceeds the maximum delay time, and the system is prompted to enter a reconnection mechanism.

[0013] Preferably, in step (3), the monotonically increasing function is a cosine function, and when the number of video frames buffered in the video buffer is lower than a predetermined value, the screen rendering time of the current frame is the cosine value of the ratio of the number of video frames stored in the current video buffer to the preset number of video frames stored.

[0014] Preferably, the cosine value calculation formula is: .

[0015] Preferably, in step (3), when the number of video frames in the video buffer is higher than a predetermined value, the screen rendering time of the current frame is the ratio of the number of video frames stored in the current video buffer to the preset number of video frames stored minus the difference between the hardware basic drawing time.

[0016] Preferably, the time difference calculation formula is:

[0017] The above method can be applied to video playback on a mobile terminal.

[0018] A non-volatile storage medium includes one or more computer instructions, and the one or more computer instructions, when executed, implement a method for optimizing the smoothness of live video playback on a mobile terminal.

[0019] An electronic device, characterized by comprising the non-volatile storage medium.

[0020] Compared with the prior art, the present invention has at least the following beneficial effects or advantages:

[0021] The method, storage medium and electronic device for optimizing the smoothness of live video playback on a mobile terminal provided by the present invention optimize the existing video jitter pool playback technology, dynamically calculate the video playback rate, and optimize the video playback smoothness; adopt dynamic frame rate calculation technology to make the video frame rate change smoother and the user experience smoother; reduce video frame loss processing and bit rate conversion processing, and reduce video distortion. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The present invention will be further described in detail below with reference to the accompanying drawings;

[0023] Figure 1 It is a workflow diagram of the method of the present invention.

[0024] Figure 2 It is a relationship diagram between rendering time and buffer frame number of the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] The present invention provides a method for optimizing the smoothness of live video playback on a mobile terminal, such as Figure 1 As shown, the following steps are included:

[0027] (1) When video data is received, create a video buffer;

[0028] (2) storing the data in the buffer, recording the time of receiving the video frame, and judging the current network video download speed. If the network download speed is lower than the preset speed, the playback frame rate is reduced; if the download speed is higher than the preset speed, the playback frame rate is increased; if the interval time between adjacent frames of the video in the video buffer is greater than the preset time value, executing step (4);

[0029] (3) After determining the frame rate change in step (2), the frame rate is calculated according to the number of video frames in the video buffer: when the number of video frames buffered in the video buffer is lower than a predetermined value, the video playback rate is dynamically calculated by a monotonically increasing function, such as taking a cosine function, and taking the cosine value of the ratio of the number of video frames stored in the current video buffer to the preset number of video frames stored As the current frame rendering time on the screen; when the number of video frames in the video buffer is higher than the preset value, take the ratio of the number of video frames stored in the current video buffer to the preset number of video frames stored minus the difference between the hardware basic drawing time As the current frame on the screen rendering time; Figure 2 As shown, the video frame rate is changed smoothly to achieve the effect of smoothly changing the video playback rate. When the number of buffered frames is lower than the preset frame, the video frame rate is smoothly reduced. When the number of buffered frames is higher than the preset frame, the video frame rate is smoothly increased.

[0030] (4) If the video delay exceeds the maximum delay time, the system will be prompted to enter the reconnection mechanism.

[0031] The video buffer is implemented with a dual-queue data structure, where queue 1 is used to store received data and queue 2 is used to store displayed data. By adopting the dual-queue data structure, mutual exclusion and competition operations existing in thread operations are reduced.

[0032] In the step (2), the average interval between adjacent frames is calculated based on the video frame acquisition time, and the current network video download speed is judged based on the calculation result.

[0033] A non-volatile storage medium includes one or more computer instructions, and the one or more computer instructions, when executed, implement a method for optimizing the smoothness of live video playback on a mobile terminal.

[0034] An electronic device, characterized by comprising the non-volatile storage medium.

[0035] In summary, the mobile live video playback smoothness optimization method, storage medium and electronic device of the present invention optimize the existing video jitter pool playback technology, dynamically calculate the video playback rate, and optimize the video playback smoothness; adopt dynamic frame rate calculation technology to make the video frame rate change smoother, so that the user experience is smoother; reduce video frame loss processing and bit rate conversion processing, and reduce video distortion. During the video playback process, the video data to be played is cached in the video buffer, and the amount of video data received during this period is calculated according to a predetermined time period. According to the calculation result, the video playback frame rate is dynamically changed to avoid frequent video freezes or delays caused by network jitter. In the process of changing the frame rate, excessive smoothing is used to avoid sudden changes in the frame rate causing a sense of jump in the video content. The jitter pool technology is optimized, and the focus is on solving the problem of smooth video playback and reducing video delays in unstable network environments.

[0036] The above examples are merely illustrative of the present invention and do not constitute a limitation on the protection scope of the present invention. All designs that are the same or similar to the present invention fall within the protection scope of the present invention.

Claims

1. A method for optimizing the smoothness of live video playback on a mobile terminal, characterized in that: The steps include: (1) When video data is received, create a video buffer; (2) storing the data in the video buffer, determining the current network video download speed, and if the network download speed is lower than a preset speed, reducing the playback frame rate; if the download speed is higher than the preset speed, increasing the playback frame rate; (3) After determining the frame rate change in step (2), the frame rate is calculated according to the number of video frames in the video buffer: when the number of video frames in the video buffer is lower than a predetermined value, the video playback rate is dynamically calculated by a monotonically increasing function to obtain the current frame rendering time on the screen; when the number of video frames in the video buffer is higher than a predetermined value, the current frame rendering time on the screen is dynamically calculated according to the current number of video frames in the video buffer, the preset number of video frame storages and the hardware basic drawing time; the video playback rate is smoothly changed by smoothly changing the video frame rate; In step (3), the monotonically increasing function is a cosine function, and when the number of video frames in the video buffer is lower than a predetermined value, the screen rendering time of the current frame is a cosine value of the ratio of the number of video frames in the current video buffer to the preset number of video frames stored; In step (3), when the number of video frames in the video buffer is higher than a predetermined value, the screen rendering time of the current frame is: ； The cosine value calculation formula is: .

2. The method according to claim 1, characterized in that: The video buffer is implemented with a dual-queue data structure, which includes a first queue and a second queue. The first queue is used to store received data, and the second queue is used to store display data.

3. The method according to claim 1, characterized in that: In the step (2), the current network video download speed is judged, specifically including simultaneously recording the video frame reception time, calculating the average interval time between adjacent frames according to the video frame acquisition time, and judging the current network video download speed according to the calculation result.

4. The method according to claim 3, characterized in that: If the interval between adjacent frames of video in the video buffer is greater than the preset time value, it is determined that the video delay exceeds the maximum delay time, and the system is prompted to enter the reconnection mechanism.

5. A non-volatile storage medium, characterized in that: The method comprises one or more computer instructions, which implement the method according to any one of claims 1 to 4 when executed.

6. An electronic device, characterized in that: Comprising the storage medium as claimed in claim 5.

Images