A method and system for optimizing adaptive bit rate video scheduling in continuous tunnel scenarios

Abstract

The present invention discloses a method for optimizing adaptive bit rate video scheduling in a continuous tunnel scenario, the method comprising: in response to a client needing to send a video segment request to a QUIC server, the client using a custom ABR algorithm to select a video segment with a suitable bit rate according to the current network speed and the buffer length corresponding to the current moment, and the client requesting the selected video segment with a suitable bit rate from the QUIC server; the custom ABR algorithm mainly calculates the bit rate based on the bandwidth and the buffer, while monitoring the network status to determine whether it is in a tunnel crossing scenario, and replacing the transmission protocol with the QUIC protocol that performs better in a weak network environment for transmission. Compared with the traditional TCP-based Dash video stream, the adaptive algorithm based on the QUIC protocol has a significant improvement in video transmission efficiency and reduces the playback freeze rate.

Description

Technical Field

[0001] The present invention belongs to the field of data communication technology, and relates to a video playback optimization method and system based on a hybrid control ABR (adaptive bit rate) algorithm, based on the QUIC protocol, under weak network conditions, especially in a continuous tunnel scenario to optimize the adaptive bit rate video scheduling method. Background Art

[0002] With the rapid development of mobile Internet and the widespread popularity of 4G and 5G around the world, online video services have prospered and become popular. Streaming platforms such as YouTube, TikTok, and BiliBili have grown rapidly around the world. As network traffic becomes increasingly large, traditional TCP-based HTTP video streaming has also encountered a development bottleneck. According to a Cisco report, from 2016 to 2021, Internet video traffic has increased fourfold, and IP video traffic will account for as much as 82% of the total Internet traffic. This also puts higher requirements on the transmission of video technology.

[0003] Since 2011, MPEG-DASH (Dynamic Adaptive Streaming over HTTP) technology has been released, and the Dash protocol has been continuously updated to improve its performance. In the Dash system, it is divided into Dash Server and Dash Client. Dash Server will slice the video into 2s to 5s long video segments, and generate the corresponding MediaPresentation Description (MPD) file to describe the video information, including the size and location of the video segments. The mpd file is transmitted to the Dash Client through the http protocol. The client parses the mpd file, initiates a segment request to the server, and determines the buffer file size and the logic of switching the bit rate based on the algorithm.

[0004] HTTPS is an HTTP protocol based on SSL / TLS. SSL2.0 originated in 1995 and has been in use for a long time, with many research results. HTTPS uses the TCP (Transmission Control Protocol) transmission protocol. Using HTTPS / TCP to encrypt video data is a more traditional method.

[0005] With the widespread popularity of mobile devices, the number of people using mobile devices has exceeded that of traditional computers. Mobile devices often switch between WLAN and cellular networks. The network switching will cause IP changes. After the IP changes, the TCP protocol needs to re-handshake. The time consumption caused by the handshake will affect the video playback experience. Summary of the invention

[0006] Purpose: In order to overcome the deficiencies in the prior art, the present invention provides a method and system for optimizing adaptive bitrate video scheduling in a continuous tunnel scenario, which optimizes video playback under weak network conditions based on the QUIC protocol, so that mobile devices can obtain a better user experience, make the playback experience smoother, and reduce the playback freeze rate.

[0007] The QUIC protocol is a low-latency Internet transmission protocol based on UDP developed by Google. In May 2021, IETF launched the standard version RFC9000, which means the official launch of the QUIC protocol. The main advantage of the QUIC protocol is that it establishes a connection quickly. Compared with https, which requires multiple handshakes, the QUIC protocol only needs 1RTT to establish a connection. Therefore, when switching networks, the QUIC protocol reconnects quickly. In addition, QUIC also implements reliable transmission. Each packet transmitted by QUIC has a packet number PKN. When the server receives the corresponding packet number, it needs to return confirmation SACK to inform the client which data packets it has received. The client will retransmit the lost packets based on SACK, but PKN is always monotonically increasing, and the QUIC packet uses offset to confirm the location of the lost packet to ensure a reliable connection.

[0008] The QUIC protocol supports multiplexing without head-of-line blocking, which can better ensure the smoothness of streaming in a weak network environment. In addition, the QUIC connection also interweaves version negotiation, encryption, and transmission handshake to reduce the connection establishment delay. This feature makes the first frame faster and the delay smaller for video streaming, so the QUIC protocol will theoretically perform better than TCP in transmitting video streams.

[0009] QUIC uses a public packet header and connection ID, and there is no need to handshake again when reconnecting during network switching. From indoor WLAN to outdoor cellular network, it is theoretically possible to switch connection modes without disconnecting the network. Therefore, when Dash is combined with the QUIC protocol, the theoretical performance will be even better.

[0010] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is:

[0011] In a first aspect, a method for optimizing adaptive bit rate video scheduling in a continuous tunnel scenario is provided, comprising:

[0012] Step (1), using an encoder to encode the video uploaded by the user, cutting it into several required bit rates according to requirements, and obtaining an encoded video;

[0013] Step (2), slicing the encoded video. After slicing, each video will generate a corresponding file directory structure, which contains video slices at different bit rates and generates a corresponding document description mpd file based on the slicing information, and stores it in the QUIC server;

[0014] Step (3), in response to the client needing to send a video segment request to the QUIC server, the client i The buffer length B corresponding to the current moment t , using a custom ABR algorithm to select video segments with appropriate bitrates, the client requests the selected video segments with appropriate bitrates from the QUIC server;

[0015] Step (4): The QUIC server sends video segments with the requested bit rate to the client.

[0016] In some embodiments, a custom ABR algorithm is used to select video segments with appropriate bitrates, including:

[0017] The current bit rate is R k The default playback duration of a single video segment is T = 2s, and the corresponding segment range is from 1....i....max; the buffer size is B1....B t ....B m , where B t is the buffer length corresponding to time t; the size of a single slice at the current bit rate is C k , the transmission time of each slice is T i ; The current network speed is S i ; The switching threshold of the lower bit rate corresponding to the current bit rate is S k , the thresholds corresponding to different bit rates are S1....S k ....S max , bitrate from low to high;

[0018] Buffer size B t Below buffer threshold B l , it will determine whether the bit rate switching is needed; the maximum length of the buffer is B m ;

[0019] The buffer size has an upper limit to prevent users from requesting data from the server when they are not watching the video, which would waste the server bandwidth. The buffer stores videos that have been downloaded but not yet played. t Indicates the buffer length corresponding to time t; the buffer size increased by each slice is L t The buffer size changes as the video chunks are downloaded. The time it takes to download a segment is T i , its model is: B(t+1) =B t +L t -T i ;

[0020] Record the duration of segment download to determine whether the network has entered the tunnel state:

[0021] When a single segment download does not exceed 10 seconds, the tunnel state is not entered;

[0022] Calculate the client's current network speed based on the speed of the latest j fragment downloads S k is the downward switching threshold corresponding to the current bit rate; S k+1 B is the switching threshold corresponding to the next higher bit rate; t is the buffer length corresponding to time t, B l is the lower limit threshold of the buffer, B m The maximum length of the buffer;

[0023] (1)S i <S k :

[0024] (1a) In response to S i <S k , and B l ≤B t , buffer size B t Not less than the buffer lower limit threshold B l , the bit rate remains at the current level and does not switch the bit rate;

[0025] (1b) In response to S i <S k , and B l >B t , buffer size B t Below the buffer lower limit threshold B l , you need to reselect the appropriate bit rate; through S i The value of S i The threshold interval is selected, and the corresponding video segment with a lower bit rate is selected, and then the cache is restarted;

[0026] (2) In response to S k <S i ≤S k+1 , B t <λB m ; λ is the buffer coefficient, ranging from (0, 1]; the default initial value of λ is 0.1, and the initial buffer length is 12s; the buffer length can be adjusted by adjusting λ;

[0027] (3) In response to S i >Sk+1 , meet the higher definition selection conditions, then select higher than S i The corresponding bit rate of level 1.

[0028] In some embodiments, when a single segment download exceeds 10 seconds, i.e., S i =0, enter the tunnel state:

[0029] The buffer capacity is expanded to B m , calculate S after the network is restored i , and select S i The corresponding low-level bit rate, and then use the maximum capacity to request video segments to fill the buffer; based on the tunnel traversal time often does not exceed 120s, in the tunnel case, the buffer maximum value B m Set to 120s;

[0030] When the system enters tunnel mode continuously, the algorithm determines whether the cache resource duration is less than λB m ,λ=0.5,not less than λB m Under the premise of continuing to cache at the current bit rate, less than λB m Then continue to reduce the bit rate;

[0031] When the buffer is full, the video resource cache is sufficient, and the network is continuously stable, exit the tunnel mode.

[0032] In some embodiments, the video format is in mp4 format.

[0033] In some embodiments, the video is encoded using FFMPEG.

[0034] In some embodiments, the code rates are cut into several required code rates according to requirements, including:

[0035] According to the information of the video uploaded by the user, the highest bit rate of the video is obtained, and the video bit rate is cut into multiple bit rate levels, namely 4K, 2K, 1080P, 720P, 480P, 360P, 144P. The upper limit of the bit rate is the bit rate of the source video.

[0036] In a second aspect, a system for optimizing adaptive bit rate video scheduling in a continuous tunnel scenario is provided, including:

[0037] Encoding and slicing server: It is configured to encode the videos uploaded by users and slice them into several required bit rates according to requirements to obtain the encoded videos;

[0038] Slice the encoded video. Each video will generate a corresponding file directory structure after slicing, which contains video slices at different bit rates and corresponding document description mpd files generated according to the slicing information and stored in the QUIC server.

[0039] Client: is configured to respond to the client's need to send a video segment request to the QUIC server, and the client responds to the current network speed S i The buffer length B corresponding to the current moment t , using a custom ABR algorithm to select video segments with appropriate bitrates, the client requests the selected video segments with appropriate bitrates from the QUIC server;

[0040] QUIC server: is configured to: store multiple multi-bitrate video segments; send video segments of the requested bitrate to the client.

[0041] In a third aspect, the present invention provides an apparatus for optimizing adaptive bit rate video scheduling in a continuous tunnel scenario, including a processor and a storage medium;

[0042] The storage medium is used to store instructions;

[0043] The processor is used to operate according to the instructions to execute the steps of the method according to the first aspect.

[0044] In a fourth aspect, the present invention provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in the first aspect.

[0045] Beneficial effect: The method and system for optimizing adaptive bit rate video scheduling in a continuous tunnel scenario under a narrowband environment provided by the present invention have the following advantages: the bit rate is calculated based on the bandwidth and buffer, and the network status is monitored at the same time to determine whether it is in a tunnel crossing scenario, and the transmission protocol is replaced with the QUIC protocol that performs better in a weak network environment for transmission. Compared with the traditional TCP-based Dash video stream, the adaptive algorithm based on the QUIC protocol has a significant improvement in video transmission efficiency and reduces the playback freeze rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A flow chart of communication between a Dash server and a client according to an embodiment of the present invention;

[0047] Figure 2 A slice file directory diagram according to an embodiment of the present invention;

[0048] Figure 3 A flow chart for building a system according to an embodiment of the present invention;

[0049] Figure 4 The internal interaction process of the Dash system according to an embodiment of the present invention;

[0050] Figure 5 A model for switching bitrates of a custom ABR algorithm according to an embodiment of the present invention;

[0051] Figure 6 4 is a flowchart of a tunnel mode workflow according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The present invention will be further described below in conjunction with the accompanying drawings and embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0053] In the description of the present invention, "several" means more than one, "many" means more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0054] In the description of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0055] Example 1

[0056] A method for optimizing adaptive bit rate video scheduling in a continuous tunnel scenario, comprising:

[0057] Step (1), using an encoder to encode the video uploaded by the user, cutting it into several required bit rates according to requirements, and obtaining an encoded video;

[0058] Step (2), slicing the encoded video. After slicing, each video will generate a corresponding file directory structure, which contains video slices at different bit rates and generates a corresponding document description mpd file based on the slicing information, and stores it in the QUIC server;

[0059] Step (3), in response to the client needing to send a video segment request to the QUIC server, the client i The buffer length B corresponding to the current moment t , using a custom ABR algorithm to select video segments with appropriate bitrates, the client requests the selected video segments with appropriate bitrates from the QUIC server;

[0060] Step (4): The QUIC server sends video segments with the requested bit rate to the client.

[0061] In some embodiments, a custom ABR algorithm is used to select video segments with appropriate bitrates, including:

[0062] The current bit rate is R k The default playback duration of a single video segment is T = 2s, and the corresponding segment range is from 1....i....max; the buffer size is B1....B t ....B m , where B t is the buffer length corresponding to time t; the size of a single slice at the current bit rate is C k , the transmission time of each slice is T i ; The current network speed is S i ; The switching threshold of the lower bit rate corresponding to the current bit rate is S k , the thresholds corresponding to different bit rates are S1....S k ....S max , bitrate from low to high;

[0063] Buffer size B t Below buffer threshold B l , it will determine whether the bit rate switching is needed; the maximum length of the buffer is B m ;

[0064] The buffer size has an upper limit to prevent users from requesting data from the server when they are not watching the video, which would waste the server bandwidth. The buffer stores videos that have been downloaded but not yet played. t Indicates the buffer length corresponding to time t; the buffer size increased by each slice is L t The buffer size changes as the video chunks are downloaded. The time it takes to download a segment is T i , its model is: B (t+1) =B t +L t -T i ;

[0065] Record the duration of segment download to determine whether the network has entered the tunnel state:

[0066] When a single segment download does not exceed 10 seconds, the tunnel state is not entered;

[0067] Calculate the client's current network speed based on the speed of the latest j fragment downloads S k is the downward switching threshold corresponding to the current bit rate; S k+1B is the switching threshold corresponding to the next higher bit rate; t is the buffer length corresponding to time t, B l is the lower limit threshold of the buffer, B m The maximum length of the buffer;

[0068] (1)S i <S k :

[0069] (1a) In response to S i <S k , and B l ≤B t , buffer size B t Not less than the buffer lower limit threshold B l , the bit rate remains at the current level and does not switch the bit rate;

[0070] (1b) In response to S i <S k , and B l >B t , buffer size B t Below the buffer lower limit threshold B l , you need to reselect the appropriate bit rate; through S i The value of S i The threshold interval is selected, and the corresponding video segment with a lower bit rate is selected, and then the cache is restarted;

[0071] (2) In response to S k <S i ≤S k+1 , B t <λB m ; λ is the buffer coefficient, ranging from (0, 1]; the default initial value of λ is 0.1, and the initial buffer length is 12s; the buffer length can be adjusted by adjusting λ;

[0072] (3) In response to S i >S k+1 , meet the higher definition selection conditions, then select higher than S i The corresponding bit rate of level 1.

[0073] According to the survey, the speed of high-speed rail passing through tunnels is generally around 250km / h, which is 69.4m / s. According to the definition of tunnels, there are short tunnels (railway: L≤500m), medium tunnels (railway: 500≤L≤3000m), and long tunnels (railway: 3000m≤L≤10000m). The time required for a railway to pass through a short tunnel does not exceed 7.2s, the time required to pass through a medium tunnel does not exceed 43s, and the time required to pass through a long tunnel does not exceed 142s. After the equipment leaves the tunnel, it connects to the operator's equipment and connects to the server. It also takes a certain amount of time, so the tunnel network judgment limit is designed to be 10s.

[0074] In some embodiments, when a single segment download exceeds 10 seconds, i.e., S i =0, enter the tunnel state:

[0075] The buffer capacity is expanded to B m , calculate S after the network is restored i , and select S i The corresponding low-level bit rate, and then use the maximum capacity to request video segments to fill the buffer; based on the tunnel traversal time often does not exceed 120s, in the tunnel case, the buffer maximum value B m Set to 120s;

[0076] When the system enters tunnel mode continuously, the algorithm determines whether the cache resource duration is less than λB m ,λ=0.5,not less than λB m Under the premise of continuing to cache at the current bit rate, less than λB m Then continue to reduce the bit rate;

[0077] When the buffer is full, the video resource cache is sufficient, and the network is stable, exit the tunnel mode. Figure 6 .

[0078] In some embodiments, the video format is in mp4 format.

[0079] In some embodiments, the video is encoded using FFMPEG.

[0080] In some embodiments, the code rates are cut into several required code rates according to requirements, including:

[0081] According to the information of the video uploaded by the user, the highest bit rate of the video is obtained, and the video bit rate is cut into multiple bit rate levels, namely 4K, 2K, 1080P, 720P, 480P, 360P, 144P. The upper limit of the bit rate is the bit rate of the source video.

[0082] like Figure 1 As shown, in a second aspect, a system for optimizing adaptive bit rate video scheduling in a continuous tunnel scenario is provided, including:

[0083] Encoding and slicing server: It is configured to encode the videos uploaded by users and slice them into several required bit rates according to requirements to obtain the encoded videos;

[0084] Slice the encoded video. Each video will generate a corresponding file directory structure after slicing, which contains video slices at different bit rates and corresponding document description mpd files generated according to the slicing information and stored in the QUIC server.

[0085] Client: is configured to respond to the client's need to send a video segment request to the QUIC server, and the client responds to the current network speed S i The buffer length B corresponding to the current moment t , using a custom ABR algorithm to select video segments with appropriate bitrates, the client requests the selected video segments with appropriate bitrates from the QUIC server;

[0086] QUIC server: is configured to: store multiple multi-bitrate video segments; send video segments of the requested bitrate to the client.

[0087] A DASH streaming media server based on the QUIC protocol, comprising:

[0088] DASH server based on QUIC protocol: provides multiple multi-bitrate video segments and encodes DASH video streams into multiple bitrate points: stores the video segments sliced ​​by the DASH server in the server; when the client requests a video segment, the QUIC server distributes the content to meet the client's request for videos of various bitrates;

[0089] Client: Sends video segment requests to the QUIC server. The client uses a custom ABR algorithm to select video segments to more efficiently utilize the current bandwidth and cache. The core of the custom ABR algorithm lies in the management of the buffer and the monitoring of network conditions. The most critical part is the monitoring of scenes where the signal disappears for a long time, such as when traveling through tunnels or underground parking lots, to improve the QoE of the video.

[0090] Encoder: Encode the video uploaded to the server and cut it into multiple bit rate levels, including 4K, 2K, 1080P, 720P, 480P, 360P, and 144P, to ensure the most suitable bit rate for different users under different network bandwidths, so as to obtain the best bit rate and smooth playback effect.

[0091] Video slicing: Videos with different bit rates encoded by the encoder are sliced. After slicing, each video will generate a file directory structure, which contains video slices and slicing information at different bit rates, and generate mpd files. This is the key to Dash's ability to dynamically select the bit rate.

[0092] Since the mainstream network devices now all access the Internet through wireless networks, and the characteristics of wireless networks are large network fluctuations and susceptibility to interference, and bandwidth is not easy to predict, it is difficult to accurately predict changes in network bandwidth, which brings challenges to ABR decision-making. On the other hand, QoE indicators conflict with each other. The higher the video quality, the larger the video block, and the larger the bandwidth required, so the higher the probability of jamming. If you conservatively choose a lower quality video block, although you can reduce the probability of jamming, you will also sacrifice video quality. Therefore, QoE cannot be improved. Therefore, it is unreliable to use network bandwidth as the only factor for bit rate switching. Therefore, the video buffer Buffer is introduced to select the appropriate bit rate based on the existing Buffer capacity and playback time as well as the current network bandwidth.

[0093] The steps for customizing the ABR algorithm include:

[0094] Step a, add custom ABR rules from dash.js source code.

[0095] Step b: The client requests and executes the script file.

[0096] Step c: The client monitors the buffer size and network bandwidth in real time, and requests the video with the best bit rate from the server according to the network status.

[0097] Step d: Due to the fluctuation of the network environment, when the network environment is good, the video stream can maintain high bitrate playback for a long time, while when the network is poor, maintaining high bitrate playback will cause frequent freezes. In order to ensure video QoE, the subsequent bitrate is selected based on the buffer size and network throughput.

[0098] Step e: The QoE form of current ABR academic research is to linearly add the main influencing factors after weighting them, as shown in the following formula:

[0099] / / Subscript of letters

[0100]

[0101] The right side of the equation represents the total video quality, total video quality switching, total freeze time, and total startup delay of the video stream. Different users have different preferences for these four parts of QoE.

[0102] Slice the video, set q(·):R→R+ as a non-decreasing function, and set the selected bit rate R k Mapped to user q(R k ) The perceived video quality. The average download speed experienced during the download process C k , B k is the buffer occupancy when starting to download chunk k. Where λ, μ, and μs are non-negative weighted parameters corresponding to video quality variation, rebuffering time, and startup delay, respectively. A relatively small λ indicates that the user is not particularly concerned about the variability of video quality; the larger λ is, the more effort is made to achieve smoother video quality variation. Relative to other parameters, a large μ s This means that users are very concerned about heavy buffering. In the case where users prefer low startup delay, we use a large startup delay, with the startup time being T s The QoE of a video stream is determined by the quality of playback, freezes, startup delay, and video quality switching. In other words, the better the video quality, the higher the QoE, and the fewer the other factors, the better.

[0103] Step g, the length of the buffer will change with the playback time. When the video starts to play, the appropriate bit rate and buffer are calculated according to the network bandwidth to ensure smooth initial playback of the video. The buffer size is gradually expanded as the video plays. The default buffer length is 30s, and the request is stopped.

[0104] In step h, the algorithm will calculate the time when the most recent segment transmission is completed in real time. When a segment starts to request transmission and still has not been completed after 10 seconds, the algorithm enters the 10-second tunnel judgment timing waiting stage. If no resource is requested within 10 seconds, it enters the tunnel mode. At this time, the system releases the buffer limit and lowers the primary bit rate. After the network is restored, it requests the buffer at its maximum capacity. Based on the fact that the tunnel crossing time does not exceed 120 seconds, the buffer size is set to 120 seconds. When the system enters the tunnel mode continuously, the algorithm determines whether the cache resource duration is less than 60 seconds. If it is not less than 60 seconds, it continues to cache at the current bit rate. If it is less than 60 seconds, it continues to lower the bit rate. When the cache exceeds 100 seconds and the network is not interrupted, the tunnel mode is released and the normal hybrid control algorithm is restored. The buffer is quickly filled when the network is restored. Prevent the network from being disconnected when entering the tunnel state again, resulting in the video not being able to play normally without a network.

[0105] Encoder: Encode the video resources of the server. The videos uploaded by users generally have only one bit rate, but the Dash server needs to switch between multiple bit rates, so the data source needs to be processed.

[0106] The default video format of the present invention is mp4 format. This server supports up to 4K video playback, so when the user uploads the video information, obtains its highest bit rate, and cuts the video bit rate into multiple bit rate levels, which are 4K, 2K, 1080P, 720P, 480P, 360P, 144P, and the upper limit of the bit rate is the bit rate of the source video.

[0107] The present invention uses FFMPEG to encode the video by default. If you want to encode other video formats, you can choose other encoders. It should be noted that the encoding uses the x264 encoding library to encode the video into H.264 / AVC format. It is also necessary to apply AAC format encoding to audio files.

[0108] Only appropriate encoding can ensure the most appropriate bit rate for different users under different network bandwidths, so as to obtain the best bit rate and smooth playback effect.

[0109] Video slicing: Dash transmits video slices. The above only encodes the video and cannot be transmitted using Dash. Simple encoding cannot make Dash work, so a slicing tool is needed to slice the videos with different bit rates.

[0110] Each video will generate a file directory structure after slicing. Figure 2 As shown, the file directory contains video slices and slice information at different bit rates, and generates a document description file so that Dash can switch video slices with different bit rates.

[0111] The stream.mpd file generated by the slicing contains the information of the video after slicing, init.mp4 is the initial playback video, and the seg-1.m4s file is the slice file generated after slicing.

[0112] QUIC server: The server supports the QUIC protocol and is used as a server to provide multiple multi-bitrate video segments to the client and encode the DASH video stream into multiple bitrate points.

[0113] The video segments sliced ​​by the DASH server are stored in the server. When the client requests the video segment, the QUIC server distributes the content to meet the client's request for videos of various bit rates.

[0114] Client: The client exchanges data with the server through the QUIC protocol, obtains the video slices stored in the server through the requested stream.mpd file, and then selects more appropriate video slices through the improved video bitrate switching method to achieve the network bandwidth that best fits the current video bitrate.

[0115] like Figure 3As shown: The whole system construction flow chart includes the above modules and the ABR algorithm defined in Dash to switch the bit rate.

[0116] The above system is a method for implementing a Dash server based on the QUIC protocol. In the above system, the client uses an improved bitrate switching method adapted to the QUIC protocol.

[0117] The ABR algorithm is the core of the DASH system. The bit rate it selects directly affects QoE factors such as video quality, freeze rate, switching, and latency. However, choosing an appropriate bit rate is difficult, and it mainly faces the following challenges:

[0118] The various sub-goals of QoE conflict with each other. Maximizing the bitrate requires the player to reduce the cache to improve the utilization of available bandwidth and increase the bitrate, while minimizing the jamming requires the player to have enough cache to absorb network fluctuations. In addition, achieving the best bitrate with high bitrate and low jamming requires the player to frequently switch the bitrate to match the dynamically changing network bandwidth in real time, which conflicts with the goal of less switching. For live broadcasts, low latency requires the playback cache to be as small as possible, while low jamming requires a large cache to absorb the impact of bandwidth jitter. Since videos with higher bitrates take longer to download, improving video quality will also increase the risk of jamming and latency.

[0119] The network bandwidth fluctuates violently. This has two effects. First, the throughput is difficult to accurately predict. When the throughput is overestimated, the bitrate selected by the algorithm may cause lag. Second, under unstable network conditions, the throughput and cache volume fluctuate violently, and the video bitrate is prone to switch back and forth.

[0120] Therefore, how to coordinate the buffer and system bitrate, as well as the switching bitrate evaluation, all need to be carefully considered. Therefore, a buffer-based bandwidth evaluation ABR algorithm is designed. The internal workflow of the Dash system is as follows: Figure 4 shown.

[0121] A custom ABR algorithm is needed to complete the Dash video streaming data request task on the client side. Since the fluctuation of mobile networks changes with the environment, it is difficult to predict the data fluctuation under various network conditions based on various algorithms. Therefore, a Cient-Side ABR algorithm based on buffer and network throughput is proposed.

[0122] When a single segment download is stalled for 10 seconds, it enters tunnel mode: the buffer capacity is expanded to B m , calculate S after the network is restored i , and select S iThe corresponding low-level bit rate, and then use the maximum capacity to request video segments to fill the buffer; based on the tunnel traversal time often does not exceed 120s, in the tunnel case, the buffer maximum value B m Set to 120s; when the system enters tunnel mode continuously, the algorithm determines whether the cache resource duration is less than λB m ,λ=0.5,not less than λB m Under the premise of continuing to cache at the current bit rate, less than λB m The bit rate will continue to be lowered; when the buffer is full, the video resource cache is sufficient, and the network is continuously stable, the tunnel mode will be exited.

[0123] In summary, the above algorithm dynamically adjusts the bitrate based on the network speed and buffer size, prioritizes QoE, and ensures that the buffer has an appropriate size. The bitrate can be switched as little as possible to reduce user perception.

[0124] According to the ABR algorithm, a reasonable bandwidth switching threshold is required.

[0125] The sizes of video clips with different bit rates are different, and the transmission speed required for each bit rate is also different. Therefore, it is necessary to reasonably calculate the threshold at which each bit rate segment can be played smoothly.

[0126] The slice playback duration is T, and the size of a slice is C. k , the current network speed is S i , downloading slices takes T i , when T i <T, it means that the downloading time is less than the playing time and there will be no lag. However, due to network fluctuations, extra bandwidth needs to be reserved.

[0127] So B l =5T*σ, (T=2s, σ>=1, the buffer threshold is controlled by adjusting σ)

[0128] Pre-calculate the thresholds required for videos with different bit rates for switching bit rates.

[0129] Compared with the ABR algorithm based on machine learning, the algorithm based on bandwidth and buffer has better universality and applicability. In an environment with good network, that is, sufficient bandwidth, the video can always be stably transmitted at the highest bit rate. However, when the network condition is poor, especially when taking high-speed rail, subway and other rail transportation, due to the poor network condition in the tunnel, limited network transmission and poor network quality, it is extremely difficult to have a good playback experience on the high-speed rail and subway, and there will be long-term freezes and poor playback. In this case, we need to adjust the buffer size and expand the buffer size so that when we pass through the tunnel, that is, when the network condition returns to a good state, we can buffer enough video to ensure smooth playback the next time the network condition deteriorates.

[0130] When a single DASH segment request continuously times out, that is, when the mobile device is continuously disconnected from the network, the tunnel mode is triggered and the buffer size is expanded. When the network is restored, the buffer starts to cache as much as possible to cope with subsequent unstable network conditions and ensure smooth video playback in poor network conditions such as tunnels. This patent mainly considers the optimization of video playback in tunnel conditions, and innovatively uses the QUIC protocol for transmission, taking advantage of its excellent weak network transmission performance.

[0131] Example 2

[0132] In a second aspect, this embodiment provides an apparatus for optimizing adaptive bit rate video scheduling in a continuous tunnel scenario, including a processor and a storage medium;

[0133] The storage medium is used to store instructions;

[0134] The processor is used to operate according to the instructions to execute the steps of the method according to embodiment 1.

[0135] Example 3

[0136] In a third aspect, this embodiment provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in Example 1 are implemented.

[0137] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0138] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0139] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0140] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0141] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for optimizing adaptive bit rate video scheduling in a continuous tunnel scenario, characterized in that: include: Step (1), using an encoder to encode the video uploaded by the user, cutting it into several required bit rates according to requirements, and obtaining an encoded video; Step (2), slicing the encoded video. After slicing, each video will generate a corresponding file directory structure, which contains video slices at different bit rates and generates a corresponding document description mpd file based on the slicing information, and stores it in the QUIC server; Step (3), in response to the client needing to send a video segment request to the QUIC server, the client i The buffer length B corresponding to the current time t , using a custom ABR algorithm to select video segments with appropriate bitrates, the client requests the selected video segments with appropriate bitrates from the QUIC server; Step (4), the QUIC server sends the video segments of the requested bit rate to the client; Among them, the custom ABR algorithm is used to select video segments with appropriate bit rates, including: The current bit rate is R k The default playback duration of a single video segment is T = 2s, and the corresponding segment range is from 1....i....max; the buffer size is B1....B t ....B m , where B t is the buffer length corresponding to time t; the size of a single slice at the current bit rate is C k , the transmission time of each slice is T i ; The current network speed is S i ; The switching threshold of the lower bit rate corresponding to the current bit rate is S k , the corresponding thresholds for different bit rates are S1....S k ....S max , bitrate from low to high; Buffer size B t Below buffer threshold B l , it will determine whether the bit rate switching is needed; the maximum length of the buffer is B m ; The buffer size has an upper limit to prevent users from requesting data from the server when they are not watching the video, which would waste the server bandwidth. The buffer stores videos that have been downloaded but not yet played. t Indicates the buffer length corresponding to time t; the buffer size increased by each slice is L t The buffer size changes as the video chunks are downloaded. The time it takes to download a segment is T i , its model is: B (t+1) =B t +L t -T i ; Record the duration of segment download to determine whether the network has entered the tunnel state: When a single segment download does not exceed 10 seconds, the tunnel state is not entered; Calculate the client's current network speed based on the speed of the latest j fragment downloads S k is the downward switching threshold corresponding to the current bit rate; S k+1 B is the switching threshold corresponding to the next higher bit rate; t is the buffer length corresponding to time t, B l is the lower limit threshold of the buffer, B m The maximum length of the buffer; (1)S i <S k : (1a) In response to S i k , and B l ≤B t , buffer size B t Not less than the buffer lower limit threshold B l , the bit rate remains at the current level and does not switch the bit rate;​ (1b) In response to S i k , and B l >B t , buffer size B t Below the buffer lower limit threshold B l , you need to reselect the appropriate bit rate; through S i The value of S i The threshold interval is selected, and the corresponding video segment with a lower bit rate is selected, and then the cache is restarted;​ (2) In response to S k i ≤S k+1 , B t <λB m ; λ is the buffer coefficient, ranging from (0, 1]; the default initial value of λ is 0.1, and the initial buffer length is 12s; the buffer length can be adjusted by adjusting λ;​ (3) In response to S i >S k+1 , if the conditions for selecting a higher level of clarity are met, the corresponding bit rate higher than Si level is selected.

2. The method for optimizing adaptive bit rate video scheduling in a continuous tunnel scenario according to claim 1, characterized in that: When a single segment download exceeds 10 seconds, S i =0, enter the tunnel state: The buffer capacity is expanded to B m , calculate S after the network is restored i , and select S i The corresponding low-level bit rate, and then use the maximum capacity to request video segments to fill the buffer; based on the tunnel traversal time often does not exceed 120s, in the tunnel case, the buffer maximum value B m Set to 120s; When the system enters tunnel mode continuously, the algorithm determines whether the cache resource duration is less than λB m ,λ=0.5,not less than λB m Under the premise of continuing to cache at the current bit rate, less than λB m Then continue to reduce the bit rate; When the buffer is full, the video resource cache is sufficient, and the network is continuously stable, exit the tunnel mode.

3. The method for optimizing adaptive bit rate video scheduling in a continuous tunnel scenario according to claim 1, characterized in that: The video format is mp4.

4. The method for optimizing adaptive bit rate video scheduling in a continuous tunnel scenario according to claim 1, characterized in that: Use FFMPEG to encode the video.

5. The method for optimizing adaptive bit rate video scheduling in a continuous tunnel scenario according to claim 1, characterized in that: Cut into several required bit rates according to requirements, including: According to the information of the video uploaded by the user, the highest bit rate of the video is obtained, and the video bit rate is cut into multiple bit rate levels, namely 4K, 2K, 1080P, 720P, 480P, 360P, 144P. The upper limit of the bit rate is the bit rate of the source video.

6. An optimized adaptive bit rate video scheduling system in a continuous tunnel scenario, characterized in that: include: Encoding and slicing server: It is configured to encode the videos uploaded by users and slice them into several required bit rates according to requirements to obtain the encoded videos; Slice the encoded video. Each video will generate a corresponding file directory structure after slicing, which contains video slices at different bit rates and corresponding document description mpd files generated according to the slicing information and stored in the QUIC server. Client: is configured to respond to the client's need to send a video segment request to the QUIC server, and the client responds to the current network speed S i The buffer length B corresponding to the current time t , using a custom ABR algorithm to select a video segment with a suitable bit rate, and the client requests the selected video segment with a suitable bit rate from the QUIC server; wherein, using a custom ABR algorithm to select a video segment with a suitable bit rate includes: The current bit rate is R k The default playback duration of a single video segment is T = 2s, and the corresponding segment range is from 1....i....max; the buffer size is B1....B t ....B m , where B t is the buffer length corresponding to time t; the size of a single slice at the current bit rate is C k , the transmission time of each slice is T i ; The current network speed is S i ; The switching threshold of the lower bit rate corresponding to the current bit rate is S k , the corresponding thresholds for different bit rates are S1....S k ....S max , bitrate from low to high; Buffer size B t Below buffer threshold B l , it will determine whether the bit rate switching is needed; the maximum length of the buffer is B m ; The buffer size has an upper limit to prevent users from requesting data from the server when they are not watching the video, which would waste the server bandwidth. The buffer stores videos that have been downloaded but not yet played. t Indicates the buffer length corresponding to time t; the buffer size increased by each slice is L t The buffer size changes as the video chunks are downloaded. The time it takes to download a segment is T i , its model is: B (t+1) =B t +L t -T i ; Record the duration of segment download to determine whether the network has entered the tunnel state: When a single segment download does not exceed 10 seconds, the tunnel state is not entered; Calculate the client's current network speed based on the speed of the latest j fragment downloads S k is the downward switching threshold corresponding to the current bit rate; S k+1 B is the switching threshold corresponding to the next higher bit rate; t is the buffer length corresponding to time t, B l is the lower limit threshold of the buffer, B m The maximum length of the buffer; (1)S i <S k : (1a) In response to S i k , and B l ≤B t , buffer size B t Not less than the buffer lower limit threshold B l , the bit rate remains at the current level and does not switch the bit rate;​ (1b) In response to S i k , and B l >B t , buffer size B t Below the buffer lower limit threshold B l , you need to reselect the appropriate bit rate; through S i The value of S i The threshold interval is selected, and the corresponding video segment with a lower bit rate is selected, and then the cache is restarted;​ (2) In response to S k i ≤S k+1 , B t <λB m ; λ is the buffer coefficient, ranging from (0, 1]; the default initial value of λ is 0.1, and the initial buffer length is 12s; the buffer length can be adjusted by adjusting λ;​ (3) In response to S i >S k+1 , meet the higher definition selection conditions, then select higher than S i The corresponding bit rate of the first level; QUIC server: is configured to: store multiple multi-bitrate video segments; send video segments of the requested bitrate to the client.

7. A device for optimizing adaptive bit rate video scheduling in a continuous tunnel scenario, characterized in that: including processor and storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 5.

8. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images