Re-buffering reduction for adaptive bit rate video streaming

Abstract

The invention relates to re-buffer reduction for adaptive bit rate video streaming. A method is provided that includes setting, by a controller, a first bitrate level of a next video segment and comparing a fill level of a play buffer to a first threshold. If a fill level of a play buffer satisfies a first threshold, a first bitrate level of the next video segment is replaced by setting a second bitrate level of the next video. Issuing a first request to a server for a next video segment encoded at a first bit rate level or encoded at the second bit rate level if a fill level of the playback buffer meets a first threshold, and downloading the requested next video segment and storing the requested video segment in the playback buffer. The decoder decodes the next video segment from the playback buffer for playback on the display device after the next video segment has been downloaded and stored in the playback buffer.

Description

[0001] Information about divisional applications

[0002] This case is a divisional application of the invention patent application with the application date of February 17, 2023, application number 202310141866.3, and invention name “Rebuffering reduction for adaptive bit rate video streaming”. Technical Field

[0003] The present description relates generally to video streaming, including, for example, adaptive bitrate video streaming. Background Art

[0004] Internet-based video technologies can rely on adaptive streaming over the Hypertext Transfer Protocol (HTTP). Such protocols have been standardized under the umbrella of Dynamic Adaptive Streaming over HTTP (DASH). In a DASH system, video content is divided into multiple segments or chunks, each corresponding to a playback time period. The video content is encoded at multiple discrete bit rates, and the segments or chunks from the different bit rate streams are aligned so that the video player can switch to different bit rates at chunk boundaries in response to changes in network bandwidth conditions. Summary of the Invention

[0005] On the one hand, the present disclosure relates to a method, which includes: setting a first bit rate level of a next video segment by a controller; comparing a fill level of a playback buffer with a first threshold; if the fill level of the playback buffer meets the first threshold, then replacing the first bit rate level of the next video segment set by the controller by setting a second bit rate level of the next video segment; issuing a first request to a server for the next video segment encoded at the first bit rate level or at the second bit rate level if the fill level of the playback buffer meets the first threshold; initiating downloading of the requested next video segment from the server and storing the next video segment in the playback buffer; and decoding the next video segment from the playback buffer by a decoder for playback on a display device after the next video segment has been downloaded and stored in the playback buffer.

[0006] On the other hand, the present disclosure relates to a non-temporary computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations including: issuing a first request to a server for a next video segment encoded at a first bit rate level; initiating downloading of the next video segment encoded at the first bit rate level from the server and storing the next video segment in a playback buffer; comparing a fill level of the playback buffer with a first threshold; changing a first playback speed set for the next video segment to a second playback speed if the fill level of the playback buffer meets the first threshold; and decoding the next video segment from the playback buffer for playback on a display device at the first playback speed after the next video segment has been downloaded and stored in the playback buffer, or at the second playback speed if the fill level of the playback buffer meets the first threshold.

[0007] On the other hand, the present disclosure relates to an electronic device, comprising: a memory including a playback buffer; and a processor coupled to the memory and configured to: set a first bit rate level for a next video segment by a controller; compare a fill level of the playback buffer with a first threshold; if the fill level of the playback buffer meets the first threshold, replace the first bit rate level set by the controller for the next video segment by setting a second bit rate level for the next video segment; and issue a request to a server for the next video segment to be encoded at the first bit rate level or, if the fill level of the playback buffer meets the first threshold, at the second bit rate level. A first request for a video segment; initiating downloading of the next video segment from the server and storing the next video segment in a playback buffer; comparing the fill level of the playback buffer with a second threshold; changing the first playback speed set for the next video segment to a second playback speed if the fill level of the playback buffer meets the second threshold; and decoding the next video segment from the playback buffer by a decoder for playback on a display device at the first playback speed after the next video segment has been downloaded and stored in the playback buffer, or at the second playback speed if the fill level of the playback buffer meets the second threshold. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Certain features of the technology are set forth in the following claims. However, for illustrative purposes, several embodiments of the technology are set forth in the drawings.

[0009] Figure 1 is a diagram illustrating adaptive bit rate streaming of video content in accordance with aspects of the present technology.

[0010] Figure 2 An example of a network environment 200 is illustrated in which an ABR video streaming system according to aspects of the present technique may be implemented.

[0011] Figure 3 is a block diagram illustrating components of an electronic device according to aspects of the present technology.

[0012] Figure 4 is a flow diagram illustrating an example adaptive bitrate streaming process in accordance with aspects of the present technique.

[0013] Figure 5 is a graph illustrating the occurrence of re-buffering during adaptive video player operation according to aspects of the present technology.

[0014] Figure 6 is a block diagram illustrating components of an electronic device according to aspects of the present technology.

[0015] Figure 7 is a flow chart illustrating an example process for setting the bit rate level of a video segment in accordance with aspects of the present technique.

[0016] Figure 8 is a block diagram illustrating components of an electronic device according to aspects of the present technology.

[0017] Figure 9 is a flow chart illustrating an example process for using playback speed control for a video clip in accordance with aspects of the present technology.

[0018] Figure 10 is a flow chart illustrating an example process for using playback speed control for a video clip in accordance with aspects of the present technology.

[0019] Figure 11 is a block diagram illustrating an electronic system in which aspects of the present technology may be implemented. DETAILED DESCRIPTION

[0020] The detailed description set forth below is intended as a description of various configurations of the present technology and is not intended to represent the only configuration in which the present technology may be practiced. The accompanying drawings are incorporated herein and form part of the detailed description. The detailed description includes specific details for providing a thorough understanding of the present technology. However, the present technology is not limited to the specific details set forth herein and may be practiced without one or more of the specific details. In some instances, structures and components are shown in block diagram form to avoid obscuring the concepts of the present technology.

[0021] Internet-based video technology can rely on adaptive streaming over the Hypertext Transfer Protocol (HTTP). Such protocols have been standardized under the umbrella of Dynamic Adaptive Streaming over HTTP (DASH). In a DASH system, video content is divided into multiple segments or chunks, each containing a portion of the video content data corresponding to a playback period (e.g., 2 seconds, 5 seconds, 10 seconds, etc.). The term "playback" refers to the presentation of decoded video content (which may include both visual and audio content) on a display device, such as a television, laptop, tablet, smartphone, etc. The video content is encoded at multiple discrete bitrate levels, and segments or chunks from different bitrate streams are aligned so that a video player can switch to different bitrate levels at chunk boundaries in response to changes in network bandwidth conditions. The bitrate level indicates the amount of data per unit time (e.g., megabits per second) at which the video content was encoded and should be accommodated by available network bandwidth for uninterrupted streaming of the video content. The increased data volume provided by higher bitrate levels can be used to improve the quality of the streaming video content, for example, by increasing resolution and / or increasing frame rate. The terms "segment" and "chunk" are used interchangeably herein.

[0022] For example, Figure 1 is a diagram illustrating adaptive bit rate streaming of video content according to aspects of the present technology. Figure 1 As depicted in FIG, M copies of a video content are stored on a server. Each copy of the video content is divided into K segments or chunks and transmitted at a rate from a set of available bitrate levels. The streamed K segments or chunks of video content encoded at different bit rate levels are encoded at different bit rate levels, resulting in M ​​streams of video content encoded at different bit rate levels being stored on the server. The solid lines drawn through the various streams of video content represent example paths illustrating the changes in bit rate levels made when streaming the video content. The streamed K segments or chunks of video content encoded at different bit rate levels are streamed along Figure 1 The timeline layout is illustrated at the bottom of the .

[0023] Quality of Experience (QoE) plays a key role in Internet video applications because it ultimately affects the revenue stream of content providers. Specifically, metrics such as rebuffering duration (e.g., the player's play buffer has no content to play), startup latency (i.e., the lag between a user click and when playback can begin), average playback bitrate, and the availability of the delivered bitrate have become key factors. Of all those factors, rebuffering time is the most important because excessively long or frequent rebuffering can cause users to abandon the current channel and switch to something else.

[0024] This technology addresses control algorithms designed to reduce both the amount of rebuffering time and the frequency of rebuffering events during adaptive bitrate streaming of video content. According to aspects of this technology, a control algorithm is introduced over existing ABR control algorithms that can replace the bitrate level decisions made by the ABR control algorithm based on the fill level of the playout buffer in order to reduce rebuffering time and frequency. Alternatively, or in addition, another control algorithm can be employed that controls the playback speed of video content based on the fill level of the playout buffer during streaming to reduce rebuffering time and frequency. These control algorithms and their associated improvements are discussed in more detail below.

[0025] Figure 2 An example of a network environment 200 is illustrated in which an ABR video streaming system according to aspects of the present technology may be implemented. However, not all of the depicted components may be required, and one or more implementations may include additional components not shown. Changes may be made to the arrangement and types of components without departing from the spirit or scope of the claims as set forth herein. Additional, different, or fewer components may be employed.

[0026] Example network environment 200 includes a content delivery network (CDN) 210 communicatively coupled to an electronic device 220, e.g., via a network 208. CDN 210 may include and / or be communicatively coupled to a content server 212, an antenna 216, and / or a satellite launch device 218. Content server 212 may encode and / or transmit an encoded data stream, such as an MPEG AVC (Advanced Video Coding) / ITU-T H.264 encoded video stream, an MPEG HEVC (High Efficiency Video Coding) / ITU-T H.265 encoded video stream, a VP9 encoded video stream, an AOM AV1 encoded video stream, and / or an MPEG VVC (Versatile Video Coding) / ITU-T H.266 encoded video stream, via network 208. Antenna 216 transmits the encoded data stream over the air, and satellite launch device 218 may transmit the encoded data stream to satellite 215.

[0027] The electronic device 220 may include and / or may be coupled to a satellite receiving device 222, such as a satellite dish, that receives the encoded data stream from the satellite 215. In one or more implementations, the electronic device 220 may further include an antenna for receiving the encoded data stream (e.g., encoded video stream) over the air from the antenna 216 of the CDN 210. The content server 212 and / or the electronic device 220 may be or may include the content server 212 described below. Figure 3 、 6 , 8 and / or 11 discuss one or more components of the electronic system.

[0028] The network 208 can be a public communication network (e.g., the Internet, a cellular data network, or a dial-up modem via a telephone network) or a private communication network (e.g., a private local area network (LAN) or leased lines). The network 208 can also include, but is not limited to, any one or more of the following network topologies, including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or hierarchical network, and the like. In one or more implementations, the network 208 can include a transmission line, such as a coaxial transmission line, a fiber optic transmission line, or substantially any transmission line, that communicatively couples the content server 212 and the electronic device 220.

[0029] Content server 212 may include or be coupled to one or more processing devices, a data store 214, and / or an encoder. The one or more processing devices execute computer instructions stored in data store 214, for example, to implement a content delivery network. Data store 214 may store the computer instructions on a non-transitory computer-readable medium. Data store 214 may further store multiple copies of video content delivered by CDN 210, encoded at different corresponding bitrate levels. The encoder may encode the video stream using a codec, such as the AVC / H.264 codec, the HEVC / H.265 codec, the VP9 codec, the AV1 codec, the VVC / H.266 codec, or any other suitable codec.

[0030] In one or more implementations, the content server 212 can be a single computing device, such as a computer server. Alternatively, the content server 212 can represent multiple computing devices working together to perform the actions of a server computer, such as a computer cloud and / or distributed system. The content server 212 can be coupled with various databases, storage services, or other computing devices, such as an adaptive bit rate (ABR) server, which can be collocated with the content server 212 or located separately from the content server 212.

[0031] The electronic device 220 may include or be coupled to one or more processing devices, memory, and / or decoders, such as hardware decoders. The electronic device 220 may be any device capable of decoding an encoded data stream, such as a VVC / H.266 encoded video stream.

[0032] In one or more implementations, the electronic device 220 may be or may include all or part of a laptop or desktop computer, a smart phone, a tablet device, a wearable electronic device (such as a pair of glasses or a watch having one or more processors coupled thereto and / or embedded therein), a set-top box, a television or other display having one or more processors coupled thereto and / or embedded therein, a video game console, or other electronic device that can be used to receive and decode an encoded data stream (such as an encoded video stream).

[0033] exist Figure 2 , the electronic device 220 is depicted as a set-top box, e.g., a device coupled to a display 224 and capable of displaying video content on the display 224, such as a television, a monitor, or any device capable of displaying video content. In one or more embodiments, the electronic device 220 may be integrated into the display 224, and / or the display 224 may be capable of outputting audio content in addition to video content. The electronic device 220 may receive a stream, e.g., an encoded data stream, from the CDN 210, which includes a stream of video content, such as a television program, a movie, or generally any content item. The electronic device 220 may receive the encoded data stream from the CDN 210 via an antenna 216, via the network 208, and / or via a satellite 215 and decode the encoded data stream, e.g., using a hardware decoder.

[0034] Figure 3 is an electronic device (e.g. Figure 2 2 is a block diagram of components of an electronic device 220 (shown in FIG. 2 ). However, not all of the depicted components may be required, and one or more implementations may include additional components not shown in the figures. Changes may be made in the arrangement and types of components without departing from the spirit or scope of the claims as set forth herein. Connections between depicted or described components are not limited to direct connections and may be implemented using one or more intervening components.

[0035] Figure 3 3. The electronic device depicted in FIG3 (e.g., an adaptive video player) includes an HTTP engine 302, a playout buffer 304, a throughput predictor 306, an ABR controller 308, and a decoder 310. According to aspects of the present technology, the HTTP engine 302 issues a request ("GET") for a segment of video content encoded at a selected bit rate level to a server 312 via a network 314 (e.g., the Internet). The HTTP engine 302 downloads the requested segment from the server 312 via the network 314 and stores the segment in the playout buffer 304. The HTTP engine 302 may also report the network throughput (i.e., bandwidth) experienced when downloading the segment from the server 312.

[0036] The ABR controller 308 selects a bitrate level for the next chunk to be downloaded by the HTTP engine 302 and notifies the HTTP engine 302 of the selection. The ABR controller 308 makes the bitrate level selection based on one or more inputs received from other components of the electronic device. For example, the throughput predictor 306 estimates the network bandwidth expected to be available for downloading the next chunk based on previous bandwidth measurements delivered to the throughput predictor 306 by the HTTP engine 302. The playout buffer 304 reports or can be used to query the playout buffer's fill level based on the amount of available playback time from the video content chunks buffered in the playout buffer. Other metrics, such as the number of chunks or video segments buffered in the playout buffer, can be used to measure the playout buffer's fill level. One or both of the estimated network bandwidth and the playout buffer's fill level can be used by the ABR controller 308 to make the bitrate level selection. The ABR controller 308 can also use other inputs in addition to or in place of the two inputs described above.

[0037] The decoder 310 consumes and decodes video content segments from the playout buffer 304 and provides the decoded video content to the display 316 for playing the video content to the viewer. The decoder 310 may also report a user-perceived quality of experience (QoE) score to assist in the decision-making logic in the ABR controller 308. Examples of the processes outlined above are explained in more detail in the description provided below.

[0038] Figure 3 Each of the components depicted in the , or one or more portions thereof, may be implemented in software (e.g., instructions, subroutines, code), in hardware (e.g., application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), controllers, state machines, gated logic, discrete hardware components, or any other suitable devices), and / or as a combination of both.

[0039] Figure 4 is a flow chart illustrating an example adaptive bit rate streaming process according to aspects of the present technology. For explanation purposes, Figure 4 The blocks of the processes described in the present invention are described herein as occurring serially or linearly. However, multiple blocks of the processes may occur in parallel. Additionally, the blocks of the processes need not be executed in the order shown and / or one or more blocks of the processes need not be executed and / or may be replaced by other operations.

[0040] For description Figure 4 For the purposes of the process described in

[15] , downloaded video content is modeled as a set of consecutive video segments or chunks V = {1, 2, . . . , K}, where L is the segment time (i.e., each segment contains L seconds of video). Multiple copies of the video content are stored on the server, where each copy is downloaded at a rate selected from a set of available bitrate levels. The different bit rates selected are used for encoding, where r1 <r2<…,<r M The adaptive video player can choose to play at the selected bit rate level. Download video segment k. B(t)∈[0,B max ] is the fill level at time t (i.e., the playback time of the video content remaining in the playback buffer), B max is the maximum amount of time that video content can be played back in the playback buffer. k =B(t k ) represents time t k Fill level of the playout buffer at time t k is the time when the adaptive video player starts downloading video segment k from the server.

[0041] exist Figure 4 At the start of the process in

[15] , the tracking parameters total_rebuffering_time (tr_time) and total_rebuffering_events (tr_events) are both reset to 0. Total_rebuffering_time keeps track of the total amount of time the adaptive video player spends rebuffering during playback of the video content, and therefore does not play the video in response to the decoder draining the playback buffer to add one or more video segments to the playback buffer. Total_rebuffering_events keeps track of the total number of instances during playback of the video content in which rebuffering occurs. Additionally, the current number of video segments is set to 1 (k=1), the bitrate level selected for the first video segment is set to the lowest available bitrate level (R1=r1), and the fill levels of the remaining playback buffers are initialized to 0 (B1=0). With these values, the HTTP engine issues a request for video segment 1 encoded at bitrate r1, and the fill level is updated to B2=B1+L. After the parameter initialization outlined above, playback by the decoder begins.

[0042] When the process has begun, the occupancy level of the play buffer is checked to determine whether there is room in the play buffer to append video segment k (block 402). If there is not enough room in the play buffer for video segment k, the occupancy level of the play buffer is adjusted using the Playback (B k+1 +LB max ) + ) to reduce, where Playback(t) is a function of consuming the video content of length t in the playback buffer, and (x) + = max(x, 0) ensures that the term can never be negative (block 404). Next, the fill level of the playout buffer is updated to Bk+1 =B k+1 -(B k+1 +LB max ) + .

[0043] Δt k Indicates the waiting time for the HTTP engine to start downloading chunk k+1 after the download of chunk k has been completed. In some ABR control algorithms, Δt k To improve the fairness of multi-player video streaming. k , then the fill level of the playback buffer is calculated using Playback(Δt k ) to decrease and the fill level is updated to B k+1 =B k+1 -Δt k (Box 406).

[0044] The bit rate level of the next video segment (R k+1 ) is determined by the ABR controller (block 408). For example, the bit rate level for the next video segment can be determined using the function To determine, is the bandwidth predicted for downloading the next video segment, and B k+1 is the fill level of the playback buffer when downloading video segment k+1 starts. The predicted throughput or bandwidth may depend on previous measurements of the network bandwidth. For example, C={C k-N+1 ,C k-N+2 ,…,C k-1 ,C k} may represent the measured network bandwidth experienced during the download of the last N video segments, and The predicted network bandwidth determined for the last N video segments can be expressed as follows: C and The function of For example, the predicted bandwidth It can be determined by taking the harmonic mean of the previously measured bandwidth as shown in the equation below.

[0045]

[0046] The present technology is not limited to algorithms that use both predicted bandwidth and fill level. For example, some algorithms may use only predicted bandwidth to select and set the bit rate level for the next video segment. Alternatively, other algorithms may use only fill level to select and set the bit rate level for the next video segment.

[0047] With respect to the bit rate level set for the next video segment, k is incremented (k=k+1) (block 410), and the HTTP engine issues a request to the server for the next video segment (block 412).

[0048] The ABR controller may periodically monitor the network bandwidth experienced during the video segment download and may make a decision to abandon downloading the video segment at the current bitrate level and, if certain conditions are met, to restart downloading the video segment at a new, lower bitrate level (block 414). The conditions may include whether the elapsed download time at the current bitrate level meets a first predefined threshold, whether the remaining download time at the current bitrate level meets another predefined threshold, whether the bitrate level estimated based on the network bandwidth experienced so far during the video segment download is less than the current bitrate level set for the video segment, and / or whether the estimated size of the video segment at the lower bitrate level is less than the size of the video segment remaining to be downloaded at the current bitrate level. The present technology is not limited to these conditions for evaluating whether to abandon downloading the video segment and begin downloading the video segment at the lower bitrate level.

[0049] If the download of the video segment at the current bitrate level is abandoned (block 414), the amount of rebuffering time and the number of rebuffering events during the period before the download was abandoned are determined and the occupancy level of the playout buffer is updated to reflect the amount of video content in the playout buffer consumed by the decoder (block 416). These updates may be made based on the following equations:

[0050]

[0051] where sd k (R k ) is the size of the portion of the video segment that is downloaded at the current bit rate level before the download is abandoned, SC k is the average network bandwidth (throughput) experienced during the download of a video segment at the current bitrate level before the download is abandoned, and

[0052] Regarding the current bit rate level R that is abandoned k Determine the new bit rate level for the video clip being downloaded And the R of the video clip k Set to (Block 418). For example, the new bit rate level (in ) can be the size of the video clip downloaded so far sd k (R k ), current video clip size d k (R k ), the network bandwidth SC experienced so fark and the current bitrate level R k function, i.e.:

[0053]

[0054] The present technique is not limited to any particular function for determining a new bitrate level In addition, the function may be based on fewer factors or more factors than those outlined above. Regarding the new bitrate level set for video segment k, the process returns to the HTTP engine to issue a request to the server for the video segment encoded at the new bitrate level (box 412).

[0055] If it is allowed to complete the download of video segment k rather than abandon it (box 414), then the amount of re-buffering time and the number of re-buffering events are determined during the period in which the download occurs and the occupancy level of the play buffer is updated to reflect the amount of video content consumed by the decoder during that period in the play buffer (box 420). These updates may be made based on the following equations:

[0056]

[0057] where d k (R k ) is the size of video segment k encoded at bitrate level R k and C k is the average network bandwidth (throughput) experienced during the download of video segment k.

[0058] The process continues by determining whether any video segments of the video content remain to be downloaded from the server (k < K) (box 422). If a video segment remains to be downloaded on the server, the process returns to check the occupancy level of the play buffer to determine whether there is space in the play buffer to append video segment k (box 402). If all K video segments of video content V have been downloaded from the server, the video segments held in the play buffer are decoded by the decoder for playback and the playback of the video content ends.

[0059] As described above, re-buffering is an important factor in evaluating QoE in Internet video applications. Figure 5 is a graph illustrating the occurrence of re-buffering during the operation of an adaptive video player according to aspects of the present technique. At time t k , the adaptive video player starts downloading chunk (video segment) k. The download time for this chunk will be d k (R k ) / C k , i.e., it depends on the size of the chunk selected at bitrate R k and the average download bandwidth (throughput) Ck Once chunk k is completely downloaded, the video player waits Δt k And at time t k+1 Start downloading the next chunk k+1 at t. The fill level B(t) evolves as chunks are downloaded and the video is played. Specifically, the fill level increases L seconds after chunk k is downloaded and decreases by d as the player consumes chunks from the play buffer. k (R k ) / C k sec. The buffer dynamics can be expressed as follows:

[0060]

[0061] like Figure 5 In the description, if Then the playback buffer becomes empty while the adaptive video player is still downloading chunk k, resulting in a rebuffering event. The total amount of rebuffering time experienced during the playback of video content V can be defined as:

[0062]

[0063] Similarly, the total number of rebuffering events experienced during playback of video content V can be defined as:

[0064]

[0065] An example of a QoE metric that can be used to evaluate the performance of an adaptive video system considers the user experience, including the average video quality across all downloaded segments and the loss caused by the average quality variation from one segment to another, rebuffering time, and startup delay. The QoE metric is defined as:

[0066]

[0067] Where q(.) is the selected bit rate level R k Mapped to the video quality q(R k ), and where α, β, and γ are non-negative weighting parameters corresponding to video quality variation, rebuffering time, and startup delay, respectively.

[0068] By assuming the mass mapping function q(R k )=R k and set α=1, and γ = 0, the QoE metric can be simplified to the effective bit rate (i.e., the average playback bit rate minus the loss of rebuffering time and streaming video quality variation). Then, the effective bit rate for streaming K chunks can be defined as:

[0069]

[0070] Here, the initial start-up delay is not included in the effective bit rate metric since it is typically a fixed amount of time regardless of which ABR control algorithm is used.

[0071] Figure 6 is a block diagram illustrating components of an electronic device according to aspects of the present technology. However, not all of the depicted components may be required, and one or more implementations may include additional components not shown in the figures. Changes may be made in the arrangement and types of components without departing from the spirit or scope of the claims as set forth herein. Connections between depicted or described components are not limited to direct connections and may be implemented using one or more intervening components.

[0072] Figure 6 The electronic device described in the embodiment is similar to the electronic device described above in conjunction with Figure 3 For example, both electronic devices include an HTTP engine 302, a playout buffer 304, a throughput predictor 306, an ABR controller 308, and a decoder 310, and their descriptions will not be repeated. Figure 6 The electronic device depicted in FIG. 1 includes an insufficient and sufficient buffer controller 602 that applies a control algorithm based on the control algorithm applied by the ABR controller 308 to set the bit rate level of video segments requested and downloaded from the server 312 via the network 314. The new control algorithm applied by the insufficient and sufficient buffer controller 602 reduces rebuffering times and events and improves QoE by using the fill level of the playout buffer 304 to identify conditions under which the bit rate level set by the ABR controller 308 can be replaced with a bit rate level that is more appropriate for the corresponding fill level. The process performed by the insufficient and sufficient buffer controller 602 is described below in conjunction with Figure 7 Describe in more detail.

[0073] Figure 6 Each of the components depicted in the , or one or more portions thereof, may be implemented in software (e.g., instructions, subroutines, code), in hardware (e.g., application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), controllers, state machines, gated logic, discrete hardware components, or any other suitable devices), and / or as a combination of both.

[0074] Figure 7 is a flow chart illustrating an example process for setting the bit rate level of a video segment according to aspects of the present technology. For explanation purposes, Figure 7The blocks of the processes described in the present invention are described herein as occurring serially or linearly. However, multiple blocks of the processes may occur in parallel. Additionally, the blocks of the processes need not be executed in the order shown and / or one or more blocks of the processes need not be executed and / or may be replaced by other operations.

[0075] Figure 7 The process described in replaces or supplements the above combined Figure 4 The operation described in block 408 is as follows. Figure 7 During the process, the ABR controller Figure 4 The bit rate level of the next video segment to be downloaded by the HTTP engine is determined and set in the manner described in block 408 of FIG. 7 (block 702). Specifically, the bit rate level of the next video segment can be determined using the function To determine, is the bandwidth predicted for downloading the next video segment, and B k+1 is the fill level of the playback buffer when video segment k+1 starts downloading.

[0076] Will fill level B k+1 and a first threshold value Th representing a relatively low fill level of the play buffer low Compare. If fill level B k+1 Less than the first threshold Th low (Block 704), the first bit rate level for the next video segment set by the ABR controller is replaced with a predetermined second bit rate level (Block 706). For example, the ABR controller may write the set first bit rate level into a control register accessible to the HTTP engine. When the fill level meets the first threshold, the second bit rate level may replace the first bit rate level in the control register. The predetermined second bit rate level is a relatively low bit rate level that will allow the next video segment to be downloaded into the playout buffer more quickly and hopefully avoid a rebuffering event. For example, the predetermined second bit rate level may be from The lowest bit rate level, making R k+1 =r1.

[0077] If fill level B k+1 Not less than the first threshold Th low (Block 704), then fill level B k+1 and a second threshold value Th representing a relatively high fill level of the play buffer high Compare. If fill level B k+1 Greater than the second threshold Th high(Block 708), then the first bit rate level set by the ABR controller for the next video segment is replaced with a predetermined third bit rate level (Block 710). The predetermined third bit rate level is a relatively high bit rate level that will utilize the play time buffer provided by the relatively high fill level to increase the bit rate level of the next video segment and thereby improve the quality of the next video segment during play. For example, the predetermined third bit rate level may be from The highest bit rate level, making R k+1 =r M .

[0078] If fill level B k+1 If the second threshold is not met (block 708), then the bit rate level for the next video segment set by the ABR controller remains unchanged. In summary, the insufficient and sufficient buffer controller modifies the adaptive video player so that the bit rate level for the next video segment R is not met. k+1 The bit rate level is set according to the following guidelines:

[0079]

[0080] The first and second thresholds Th low and Th high Can be configurable. According to aspects of the present technology, two thresholds can be set as follows:

[0081]

[0082] Figure 8 is a block diagram illustrating components of an electronic device according to aspects of the present technology. However, not all of the depicted components may be required, and one or more implementations may include additional components not shown in the figures. Changes may be made in the arrangement and types of components without departing from the spirit or scope of the claims as set forth herein. Connections between depicted or described components are not limited to direct connections and may be implemented using one or more intervening components.

[0083] Figure 8 The electronic device described in the embodiment is similar to the electronic device described above in conjunction with Figure 3 and 6 For example, all three electronic devices include an HTTP engine 302, a play buffer 304, a throughput predictor 306, an ABR controller 308, and a decoder 310, and their descriptions will not be repeated. Figure 8 The electronic device depicted in FIG. 1 includes insufficient and sufficient buffer controllers 602, and its description will not be repeated. However, the modified Figure 8 The electronic device depicted in FIG. 8 includes a playback speed controller 802. It should be noted that Figure 8The incorporation of the insufficient and sufficient buffer controller 602 in the electronic device is optional, and the playback speed controller 802 can be incorporated into the electronic device with or without the insufficient and sufficient buffer controller 602.

[0084] According to aspects of the present technology, the playback speed controller 802 implements a control algorithm that adjusts the playback speed of a video clip based on the fill level of the playback buffer. The control algorithm exploits the nature of the human visual system, in which viewers generally do not notice playback speed changes of video content within approximately 10% (either 10% higher or lower) of the real-time playback speed. For example, if the fill level is less than a predefined low threshold, i.e., Th low , or if the filling level is less than a predefined intermediate threshold, Th mid , and the previous video segment download is abandoned, then the decoder is set to a slow play mode of the video segment (e.g., 90% of the real-time play speed). Slowing down the decoder's play speed for the video segment provides more time to download the next video segment into the play buffer before the current video segment is finished playing and thus reduces the chance of a rebuffering event or potentially shortens the duration of a rebuffering event. If the fill level is above a predefined high threshold, i.e., Th high , and the playback lags behind the real-time playback speed due to the previous use of the slow playback mode, the decoder switches to the fast playback mode of the video segment (e.g., 110% of the real-time playback speed) to catch up with the playback speed. Otherwise, the decoder is set to the normal real-time playback mode. The operation of the playback speed controller 802 is described below in conjunction with Figure 9 and 10 Describe in more detail.

[0085] Figure 8 Each of the components depicted in the , or one or more portions thereof, may be implemented in software (e.g., instructions, subroutines, code), in hardware (e.g., application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), controllers, state machines, gated logic, discrete hardware components, or any other suitable devices), and / or as a combination of both.

[0086] Figure 9 is a flow chart illustrating an example process for using playback speed control for a video clip according to aspects of the present technology. For explanation purposes, Figure 9 The blocks of the processes described in the present invention are described herein as occurring serially or linearly. However, multiple blocks of the processes may occur in parallel. Additionally, the blocks of the processes need not be executed in the order shown and / or one or more blocks of the processes need not be executed and / or may be replaced by other operations.

[0087] Figure 9 The procedures described in replace or supplement the above procedures for Figure 4As discussed above, this stage of the process follows the decision to abandon the download of the video segment in order to restart the video segment using a lower bit rate level. Initially, the fill level of the play buffer is equal to the threshold value PTh. low A comparison is made (block 902). If the fill level is less than PTh low , then the play speed controller 802 changes the play speed set for the video clip by reducing the play speed by a specified amount (e.g., 10%, 5%, 3%, etc.), thereby changing the play speed from the first play speed to the second play speed (box 904).

[0088] If the fill level of the play buffer does not meet the threshold PTh low , then the fill level of the play buffer is greater than PTh low Threshold PTh mid The comparison is performed (block 906). Additionally, the process determines whether the download of the previous video segment should be abandoned and restarted at a lower bit rate level (block 906). This determination can be made by checking whether a flag in a specified memory location has been set. If it is determined that the fill level is less than a threshold value PTh mid If the download of the previous video segment is abandoned, the playback speed controller 802 changes the playback speed of the video segment by reducing the playback speed by a specified amount (e.g., 10%, 5%, 3%, etc.), thereby changing the first playback speed to a second playback speed (block 904). The amount by which the playback speed is reduced may be the same as for a video in which the fill level satisfies the threshold value PTh. low The conditions are the same, or they may be reduced by different amounts.

[0089] If the fill level of the play buffer does not meet the threshold PTh mid , then the fill level of the play buffer is greater than PTh mid Threshold PTh high A comparison is made (block 908). Additionally, the process makes a determination as to whether the amount of play lag time is greater than zero (block 908). The play lag time tracks the amount by which the temporal play of the current video content lags behind the real-time play of the video content due to the reduced play speed of one or more previous video segments. If the fill level is greater than a threshold value PTh high If the current playback lag time is greater than zero, the playback speed controller 802 changes the playback speed of the video clip by increasing the playback speed by a specified amount (e.g., 10%, 5%, 3%, etc.), thereby changing the playback speed of the first playback speed to a third playback speed (block 910). high , or if there is no playback lag time, the playback speed of the video clip will not be changed.

[0090] After reducing the playback speed of the video segment (block 904), increasing the playback speed of the video segment (block 910), or if the playback speed is not changed, the process continues by updating the amount of rebuffering time, the number of rebuffering events, the fill level, and the amount of playback lag time for video segment k during the period during which downloading began until it was abandoned (block 912). The manner in which these parameters are updated varies depending on whether the playback speed of the video segment was reduced, increased, or maintained unchanged.

[0091] If the playback speed of the video clip is reduced, the following equations are used to update the parameters:

[0092]

[0093]

[0094] Where δ is a small amount by which the playback speed is changed (e.g., 0.1, 0.05, 0.03, etc.), and pt_lag is the amount of playback lag time accumulated during the playback of the video content. The playback lag time is calculated at the beginning of the playback of the video content (e.g., Figure 4 δ is initialized to 0 at the beginning of the process represented in . δ can be a configurable value. If the playback speed of the video segment increases and the playback lag time is greater than 0, the following equation is used to update the parameter:

[0095]

[0096] If the playback speed of the video clip is not changed, the parameters are as described above. Figure 4 Next, the process sets the previous video segment discard flag to true to reflect the discard of downloading the video segment (block 914).

[0097] Figure 10 is a flow chart illustrating an example process for using playback speed control for a video clip according to aspects of the present technology. For explanation purposes, Figure 10 The blocks of the processes described in the present invention are described herein as occurring serially or linearly. However, multiple blocks of the processes may occur in parallel. Additionally, the blocks of the processes need not be executed in the order shown and / or one or more blocks of the processes need not be executed and / or may be replaced by other operations.

[0098] Figure 10 The procedures described in replace or supplement the above procedures for Figure 4 As discussed above, this stage of the process follows the decision not to abandon downloading video segment k. The portion of the process (blocks 1002, 1004, 1006, 1008, and 1010) that adjusts the playback speed of video segment k based on the fill level of the playback buffer follows the process described above with respect to Figure 9The same corresponding portion of the process described (blocks 902, 904, 906, 908, and 910) and the description of this portion of the process will not be repeated.

[0099] After reducing the playback speed of the video segment (block 1004), increasing the playback speed of the video segment (block 1010), or if the playback speed is not changed, the process continues by updating the amount of rebuffering time, the number of rebuffering events, the fill level, and the amount of playback lag time after video segment k has completed downloading (block 1012). Figure 9 As discussed, the way these parameters are updated differs depending on whether the playback speed of the video clip is slowed down, increased, or remains constant.

[0100] If the playback speed of the video clip is reduced, the following equations are used to update the parameters:

[0101]

[0102] If the playback speed of the video clip is increased and the playback lag is greater than 0, the following equations are used to update the parameters:

[0103]

[0104] If the playback speed of the video clip is not changed, the parameters are as described above. Figure 4 Next, the process sets the previous video segment discard flag to false to reflect that downloading of video segment k is complete (block 1014).

[0105] Threshold PTh low , PTh mid and PTh high Can be configurable. For example, these thresholds can be set as follows:

[0106]

[0107]

[0108] Although the PTh indicated above low and PTh high The value is combined with the above Figure 7 PTh of the described embodiments low and PTh high The value of is the same, but the present technology is not limited to using the same threshold for these different control algorithms and different control algorithms with different thresholds may be implemented.

[0109] Changing the playback speed used by the decoder to decode a video segment from the playback buffer can rely on trick modes to slow down or speed up the playback speed while matching the target display frame rate. For example, frame rate conversion (FRC) can be turned on in the decoder when the playback speed is reduced to insert additional frames between existing frames in the video segment. Similarly, the decoder can periodically drop one or more existing frames from the video segment during playback when the playback speed is increased. Additionally, audio / speech pitch correction can be applied to the audio portion of the video segment to match the expected sound pitch experienced during the constant playback speed.

[0110] Figure 11 An electronic system 1100 is conceptually described in which one or more embodiments of the present technology may be implemented. The electronic system 1100 may be, for example, a data device, a media converter, a desktop computer, a laptop computer, a tablet computer, a server, a phone, or substantially any electronic device capable of transmitting signals over a network and implementing an adaptive video player in the manner described above. This electronic system 1100 includes various types of computer-readable media and interfaces for various other types of computer-readable media. In one or more embodiments, the electronic system 1100 is or includes one or more of the server 112 and / or the electronic device 120. The electronic system 1100 includes a bus 1108, one or more processing units 1112, a system memory 1104, a read-only memory (ROM) 1110, a permanent storage device 1102, an input device interface 1114, an output device interface 1106, and a network interface 1116, or subsets and variations thereof.

[0111] The bus 1108 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of the electronic system 1100. In one or more embodiments, the bus 1108 communicatively connects one or more processing units 1112 to the ROM 1110, the system memory 1104, and the permanent storage device 1102. The one or more processing units 1112 retrieve instructions to execute and data to process from these various memory units in order to perform the processes of the present disclosure. In different embodiments, the one or more processing units 1112 can be a single processor or a multi-core processor.

[0112] ROM 1110 stores static data and instructions required by one or more processing units 1112 and other modules of the electronic system. Permanent storage 1102, on the other hand, is a read-write memory device. Permanent storage 1102 is a non-volatile memory unit that stores instructions and data even when the electronic system 1100 is turned off. One or more embodiments of the present disclosure utilize a mass storage device (e.g., a solid-state drive or a magnetic or optical disk and its corresponding magnetic disk drive) as permanent storage 1102.

[0113] Other embodiments use removable storage devices (e.g., a flash memory drive and its corresponding disk drive, an external magnetic hard drive, etc.) as permanent storage 1102. Like permanent storage 1102, system memory 1104 is a read-write memory device. However, unlike permanent storage 1102, system memory 1104 is a volatile read-write memory, such as random access memory. System memory 1104 stores any of the instructions and data required by one or more processing units 1112 during execution. In one or more embodiments, the processes of the present disclosure are stored in system memory 1104, permanent storage 1102, and / or ROM 1110. One or more processing units 1112 retrieve instructions to execute and data to process from these various memory units in order to perform the processes of one or more embodiments.

[0114] Bus 1108 is also connected to input device interface 1114 and output device interface 1106. Input device interface 1114 enables a user to communicate information and selection commands to the electronic system. Input devices used with input device interface 1114 include, for example, an alphanumeric keyboard and a pointing device (also referred to as a "cursor control device"). Output device interface 1106 enables, for example, the display of images generated by electronic system 1100. Output devices used with output device interface 1106 include, for example, a printer and a display device, such as a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a flexible display, a flat panel display, a solid-state display, a projector, or any other device for outputting information. One or more embodiments include a device that functions as both an input device and an output device, such as a touch screen. In these embodiments, feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, voice, or tactile input.

[0115] Finally, if Figure 11 As shown in FIG. 1 , bus 1108 also couples electronic system 1100 to one or more networks (not shown) via one or more network interfaces 1116. In this manner, the computer can be part of one or more computer networks (e.g., a local area network (LAN), a wide area network (WAN), or an intranet, or a network of networks, such as the Internet). Any or all components of electronic system 1100 may be used in conjunction with the present disclosure.

[0116] Implementations within the scope of the present disclosure may be implemented in part or in whole using a tangible computer-readable storage medium (or multiple tangible computer-readable storage media of one or more types) encoding one or more instructions.The tangible computer-readable storage medium may also be non-transitory in nature.

[0117] Computer-readable storage media can be any storage medium that can be read, written, or otherwise accessed by a general-purpose or special-purpose computing device, including any processing electronics and / or processing circuitry capable of executing instructions. By way of example, and not limitation, computer-readable media can include any volatile semiconductor memory, such as RAM, DRAM, SRAM, T-RAM, Z-RAM, and TTRAM. Computer-readable media can also include any non-volatile semiconductor memory, such as ROM, PROM, EPROM, EEPROM, NVRAM, Flash, nvSRAM, FeRAM, FeTRAM, MRAM, PRAM, CBRAM, SONOS, RRAM, NRAM, racetrack memory, FJG, and Millipede memory.

[0118] Furthermore, the computer-readable storage medium may include any non-semiconductor memory, such as optical disk storage, magnetic disk storage, magnetic tape, other magnetic storage, or any other medium capable of storing one or more instructions. In some embodiments, the tangible computer-readable storage medium may be directly coupled to the computing device, while in other embodiments, the tangible computer-readable storage medium may be indirectly coupled to the computing device, for example, via one or more wired connections, one or more wireless connections, or any combination thereof.

[0119] Instructions may be directly executable or may be used to develop executable instructions. For example, instructions may be implemented as executable or non-executable machine code or as instructions in a high-level language that can be compiled to generate executable or non-executable machine code. Instructions may also be implemented as data or may include data. Computer-executable instructions may also be organized in any format, including routines, subroutines, programs, data structures, objects, modules, applications, applets, functions, and the like. As will be appreciated by those skilled in the art, details including, but not limited to, the number, structure, sequence, and organization of instructions may vary significantly without changing the underlying logic, functionality, processing, and output.

[0120] While the above discussion primarily refers to microprocessors or multi-core processors executing software, one or more implementations are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In one or more implementations, such integrated circuits execute instructions stored on the circuits themselves.

[0121] According to aspects of the present technology, a method is provided that includes setting, by a controller, a first bitrate level for a next video segment and comparing a fill level of a playout buffer to a first threshold. If the fill level of the playout buffer meets the first threshold, overriding the first bitrate level set by the controller for the next video segment by setting a second bitrate level for the next video segment. A first request is issued to a server for the next video segment encoded at the first bitrate level or, if the fill level of the playout buffer meets the first threshold, at the second bitrate level, and initiating downloading of the requested next video segment from the server and storing the next video segment in the playout buffer. The next video segment from the playout buffer is decoded for playback on a display device after the next video segment has been downloaded and stored in the playout buffer.

[0122] The method may further include comparing the fill level of the playout buffer to a second threshold; and if the fill level of the playout buffer satisfies the second threshold, overriding the first bit rate level of the next video segment set by the controller by setting a third bit rate level for the next video segment. The first request is issued to the server for the next video segment to be encoded at the third bit rate level if the fill level of the playout buffer satisfies the second threshold. The second threshold may be greater than the first threshold, and the fill level may satisfy the second threshold by exceeding the second threshold and satisfy the first threshold by being less than the first threshold. The third bit rate level may be greater than the second bit rate level. The second bit rate level may be the lowest bit rate level from a set of bit rate levels at which the next video segment is encoded on the server, and the third bit rate level may be the highest bit rate level from the set of bit rate levels at which the next video segment is encoded on the server. The fill level of the playout buffer may be determined before storing the next video segment in the playout buffer.

[0123] The method may further include: stopping the download of the next video segment from the server before completion based on one or more network bandwidth conditions; comparing the fill level of the playback buffer to a third threshold; and changing the first playback speed set for the next video segment to a second playback speed if the fill level of the playback buffer meets the third threshold. The method may further include: setting a fourth bitrate level for the next video segment; and issuing a second request to the server for the next video segment encoded at the fourth bitrate level, wherein the next video segment is decoded by the decoder for playback on the display device at the first playback speed set for the next video segment or at the second playback speed if the fill level of the playback buffer meets the third threshold.

[0124] The method may further include: comparing the fill level of the playback buffer to a fourth threshold if the fill level of the playback buffer does not meet the third threshold; and changing the first playback speed of the next video segment to a third playback speed if the fill level of the playback buffer meets the fourth threshold, wherein the fourth threshold is greater than the third threshold and the third playback speed is greater than the second playback speed. The method may further include determining whether a current playback lag time is greater than 0, wherein if the fill level meets the fourth threshold and the current playback lag time is greater than 0, changing the playback speed set for the next video segment to the third playback speed.

[0125] The method may further include comparing the fill level of the playout buffer to a fifth threshold, wherein the fifth threshold is greater than the third threshold and less than the fourth threshold; determining whether downloading of a previous video segment has stopped; and if the fill level of the playout buffer meets the fifth threshold and downloading of the previous video segment has stopped, changing the first play speed set for the next video segment to the second play speed. The play speed may be reduced by the decoder inserting one or more frames into the next video segment using frame rate conversion, and the play speed may be increased by the decoder periodically dropping one or more frames from the next video segment. The method may further include applying pitch correction to the audio portion of the next video segment based on the change in the play speed set for the next video segment.

[0126] According to aspects of the present technology, a non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations is provided. The operations include: issuing a first request to a server for a next video segment encoded at a first bit rate level; initiating downloading of the next video segment encoded at the first bit rate level from the server and storing the next video segment in a playback buffer. The operations further include: comparing a fill level of the playback buffer with a first threshold; if the fill level of the playback buffer meets the first threshold, changing a first playback speed set for the next video segment to a second playback speed; and decoding the next video segment from the playback buffer for playback on a display device at the first playback speed after the next video segment has been downloaded and stored in the playback buffer, or at the second playback speed if the fill level of the playback buffer meets the first threshold.

[0127] The operation may further include: comparing the fill level of the playback buffer with a second threshold; determining whether a current playback lag time is greater than 0; and if the fill level of the playback buffer satisfies the second threshold and if the current playback lag time is greater than 0, changing the first playback speed set for the next video segment to a third playback speed, wherein the second threshold is greater than the first threshold and the third playback speed is greater than the second playback speed. The operation may further include: comparing the fill level of the playback buffer with a third threshold, wherein the third threshold is greater than the first threshold and less than the second threshold; determining whether downloading of a previous video segment has stopped; and if the fill level of the playback buffer satisfies the third threshold and downloading of the previous video segment has stopped, changing the first playback speed of the next video segment to the second playback speed.

[0128] The operations may further include: stopping the download of the next video segment from the server before completion based on one or more network bandwidth conditions; setting the bit rate level of the next video segment to a second bit rate level; issuing a second request to the server for the next video segment encoded at the second bit rate level; and initiating downloading the next video segment encoded at the second bit rate level and storing the next video segment in the playback buffer.

[0129] The operation may further include: setting, by a controller, the first bit rate level of the next video fragment; comparing the fill level of the playback buffer with a third threshold; and replacing the first bit rate level of the next video fragment set by the controller by setting a third bit rate level of the next video fragment if the fill level of the playback buffer satisfies the third threshold, wherein the first request is issued to the server for the next video fragment to be encoded at the third bit rate level if the fill level of the playback buffer satisfies the third threshold.

[0130] The operation may further include comparing the fill level of the playback buffer with a fourth threshold greater than the third threshold; and replacing the bit rate level of the next video fragment set by the adaptive bit rate controller by setting a fourth bit rate level for the next video fragment greater than the third bit rate level if the fill level of the playback buffer satisfies the fourth threshold, wherein the first request is issued to the server for the next video fragment encoded at the fourth bit rate level if the fill level of the playback buffer satisfies the fourth threshold, and wherein the fill level satisfies the fourth threshold by exceeding the fourth threshold and satisfies the third threshold by being less than the third threshold.

[0131] According to aspects of the present technology, an electronic device is provided, comprising: a memory including a playout buffer; and a processor coupled to the memory. The processor is configured to: set a first bitrate level for a next video segment by a controller; compare a fill level of the playout buffer with a first threshold; and, if the fill level of the playout buffer meets the first threshold, override the first bitrate level set by the controller for the next video segment by setting a second bitrate level for the next video segment. The processor is further configured to: issue a first request to a server for the next video segment encoded at the first bitrate level or, if the fill level of the playout buffer meets the first threshold, at the second bitrate level; initiate downloading the next video segment from the server and storing the next video segment in the playout buffer; and compare the fill level of the playout buffer with a second threshold. If the fill level of the playback buffer meets the second threshold, the first playback speed set for the next video segment is changed to a second playback speed, and the next video segment from the playback buffer is decoded by the decoder for playback on the display device at the first playback speed after the next video segment has been downloaded and stored in the playback buffer, or at the second playback speed if the fill level of the playback buffer meets the second threshold.

[0132] The processor may be further configured to: compare the fill level of the playback buffer with a third threshold greater than the first threshold; and if the fill level of the playback buffer satisfies the third threshold, then replace the first bit rate level of the next video segment set by the controller by setting a third bit rate level of the next video segment greater than the second bit rate level, wherein the fill level satisfies the third threshold by exceeding the third threshold and satisfies the second threshold by being less than the second threshold.

[0133] The processor may be further configured to: stop the download of the next video segment from the server before completion based on one or more network bandwidth conditions; set a third bit rate level for the next video segment; and issue a second request to the server for the next video segment encoded at the third bit rate level, wherein the next video segment encoded at the third bit rate level is downloaded and stored in the playback buffer and decoded from the playback buffer for playback on the display device.

[0134] The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Accordingly, the claims are not intended to be limited to the aspects shown herein, but will conform to the full scope consistent with the language of the claims, wherein reference to an element in the singular is not intended to mean "one and only one," but rather "one or more," unless expressly stated otherwise. Unless expressly stated otherwise, the term "some" refers to one or more. Pronouns in the masculine gender (e.g., he) include the feminine and neuter genders (e.g., she and it), and vice versa. Headings and subheadings, if any, are used for convenience only and do not limit the present disclosure.

[0135] The predicates "configured to," "operable to," and "programmed to" do not imply any specific tangible or intangible modification of the subject matter, but are intended to be used interchangeably. For example, a processor configured to monitor and control an operation or component may also mean that the processor is programmed to monitor and control the operation or that the processor is operable to monitor and control the operation. Similarly, a processor configured to execute code may be interpreted as a processor programmed to execute code or operable to execute code.

[0136] Phrases such as "an aspect" do not imply that such aspect is essential to the present technology or that such aspect applies to all configurations of the present technology. Disclosure relating to an aspect may apply to all configurations or to one or more configurations. Phrases such as an aspect may refer to one or more aspects, and vice versa. Phrases such as "a configuration" do not imply that such configuration is essential to the present technology or that such configuration applies to all configurations of the present technology. Disclosure relating to a configuration may apply to all configurations or to one or more configurations. Phrases such as a configuration may refer to one or more configurations, and vice versa.

[0137] The word “example” is used herein to mean “serving as an example or illustration.” Any aspect or design described herein as an “example” is not necessarily to be construed as preferred or advantageous over other aspects or designs.

[0138] All structural and functional equivalents of the elements of the various aspects described throughout this disclosure that are known or that will later become known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be covered by the claims. In addition, nothing disclosed herein is intended to be contributed to the public, regardless of whether this disclosure is explicitly cited in the claims. No claim element should be interpreted under the provisions of 35 U.S.C. § 112 (f) unless the element is explicitly cited using the phrase "means for..." or, in the case of a method claim, the element is cited using the phrase "step for..." In addition, to the extent that the term "including," "having," or the like is used in the description or claims, this term is intended to be inclusive in a manner similar to the term "comprising," as "comprising" is interpreted when used as a transition word in the claims.

[0139] It will be appreciated by those skilled in the art that the various illustrative blocks, modules, elements, components, methods, and algorithms described herein may be implemented as electronic hardware, computer software, or a combination of both. To illustrate this interchangeability of hardware and software, the various illustrative blocks, modules, elements, components, methods, and algorithms are generally described above in terms of their functionality. Whether this functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application. The various components and blocks may be arranged differently (e.g., arranged in a different order or partitioned in a different way) without departing from the scope of the present technology.

[0140] The predicates "configured to," "operable to," and "programmed to" do not imply any specific tangible or intangible modification of the subject, but are intended to be used interchangeably. For example, a processor configured to monitor and control an operation or component may also mean that the processor is programmed to monitor and control the operation or that the processor is operable to monitor and control the operation. Similarly, a processor configured to execute code may be interpreted as a processor programmed to execute code or operable to execute code.

Claims

1. An electronic device comprising: a memory including a play buffer; and a processor coupled to the memory and configured to: issuing a first request to a server for a first video segment encoded at a first bit rate level; based on one or more conditions related to downloading of the first video segment, stopping downloading of the first video segment encoded at the first bit rate level from the server before the first video segment has completed downloading; setting a second bit rate level for the first video segment that is different from the first bit rate level; issuing a second request to the server for the first video segment encoded at the second bit rate level; and The first video segment is decoded for playback on a display device in response to the first video segment encoded at the second bitrate level being downloaded and stored in the playback buffer. 2 . The electronic device of claim 1 , wherein the second bit rate level is lower than the first bit rate level.

3. An electronic device according to claim 1, wherein the second bit rate level is determined based on the size of the portion of the first video segment downloaded before the download was stopped, the size of the first video segment that has been completed encoded at the first bit rate level, the network bandwidth experienced during the download of the first video segment before the download was stopped, or one or more of the first bit rate levels.

4. An electronic device according to claim 1, wherein the one or more conditions related to the downloading of the first video fragment include the elapsed download time of the first video fragment encoded at the first bit rate level, the remaining download time of the first video fragment encoded at the first bit rate level, the network bandwidth experienced during the downloading of the first video fragment encoded at the first bit rate level, or the estimated size of the first video fragment encoded at the second bit rate level.

5. The electronic device of claim 1 , wherein the processor is further configured to: The first bitrate level is set based on predicted network bandwidth and a current fill level of the playout buffer.

6. The electronic device of claim 5, wherein the processor is further configured to: comparing the fill level of the playout buffer to a first buffer threshold and to a second buffer threshold; If the fill level of the playout buffer satisfies the first buffer threshold, replacing the first bit rate level with a highest bit rate level from a set of bit rate levels used to encode the first video segment on the server; and If the fill level of the playout buffer satisfies the second buffer threshold, the first bit rate level is replaced with a lowest bit rate level from the set of bit rate levels.

7. The electronic device of claim 1 , wherein the processor is further configured to: comparing a fill level of the playout buffer to a second buffer threshold and to a third buffer threshold; Determine whether the current playback lag time is greater than 0; If the fill level of the playback buffer satisfies the second buffer threshold, reducing a playback speed of the downloaded first video segment played on the display device; as well as If the fill level of the playback buffer satisfies the third buffer threshold and the playback lag is greater than 0, increasing the playback speed of the downloaded first video segment played on the display device, The second buffer threshold is smaller than the third buffer threshold.

8. The electronic device of claim 7, wherein the processor is further configured to: comparing the fill level of the playout buffer to a fourth buffer threshold; and If the fill level of the playback buffer satisfies the fourth buffer threshold, reducing the playback speed of the downloaded first video segment played on the display device, Wherein the fourth buffer threshold is less than the second buffer threshold.

9. The electronic device of claim 7, wherein the playback speed is reduced by inserting one or more frames into the first video segment during playback, and the playback speed is increased by dropping one or more frames from the first video segment during playback.

10. The electronic device of claim 7, wherein the processor is further configured to apply pitch correction to an audio portion of the first video segment during playback based on a change to the playback speed of the first video segment.

11. A method comprising: issuing a first request to a server for a first video segment encoded at a first bit rate level; stopping downloading of the first video segment encoded at the first bitrate level from the server before the first video segment has completed downloading and is stored in a playout buffer based on one or more conditions related to downloading of the first video segment; setting a second bit rate level for the first video segment that is different from the first bit rate level; issuing a second request to the server for the first video segment encoded at the second bit rate level; and After the first video segment encoded at the second bitrate level has been downloaded and stored in the playout buffer, the first video segment is decoded for playout on a display device.

12. The method of claim 11, wherein the second bit rate level is lower than the first bit rate level.

13. The method according to claim 11, further comprising: The first bitrate level is set based on predicted network bandwidth and a current fill level of the playout buffer.

14. The method according to claim 13, further comprising: comparing the fill level of the playout buffer to a first buffer threshold and to a second buffer threshold; If the fill level of the playout buffer satisfies the first buffer threshold, replacing the first bit rate level with a highest bit rate level from a set of bit rate levels used to encode the first video segment on the server; and If the fill level of the playout buffer satisfies the second buffer threshold, the first bit rate level is replaced with a lowest bit rate level from the set of bit rate levels.

15. The method of claim 11, further comprising: comparing a fill level of the playout buffer to a second buffer threshold and to a third buffer threshold; Determine whether the current playback lag time is greater than 0; If the fill level of the playback buffer satisfies the second buffer threshold, reducing a playback speed of the downloaded first video segment played on the display device; as well as If the fill level of the playback buffer satisfies the third buffer threshold and the playback lag is greater than 0, increasing the playback speed of the downloaded first video segment played on the display device, The second buffer threshold is smaller than the third buffer threshold.

16. The method according to claim 15, further comprising: comparing the fill level of the playout buffer to a fourth buffer threshold; as well as If the fill level of the playback buffer satisfies the fourth buffer threshold, reducing the playback speed of the downloaded first video segment played on the display device, Wherein the fourth buffer threshold is less than the second buffer threshold.

17. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: setting the first bit rate level based on the predicted network bandwidth and the current fill level of the playout buffer; issuing a first request to a server for a first video segment encoded at the first bit rate level; stopping downloading the first video segment encoded at the first bitrate level from the server before the first video segment has completed downloading and is stored in the playout buffer based on one or more conditions related to downloading of the first video segment; setting a second bit rate level different from the first bit rate level for the first video segment, wherein the second bit rate level is lower than the first bit rate level; issuing a second request to the server for the first video segment encoded at the second bit rate level; and After the first video segment encoded at the second bitrate level has been downloaded and stored in the playout buffer, the first video segment is decoded for playout on a display device.

18. The non-transitory computer-readable medium of claim 17, wherein the operations further comprise: comparing the fill level of the playout buffer to a first buffer threshold and to a second buffer threshold; If the fill level of the playout buffer satisfies the first buffer threshold, replacing the first bit rate level with a highest bit rate level from a set of bit rate levels used to encode the first video segment on the server; and If the fill level of the playout buffer satisfies the second buffer threshold, the first bit rate level is replaced with a lowest bit rate level from the set of bit rate levels.

19. The non-transitory computer-readable medium of claim 17, wherein the operations further comprise: comparing a fill level of the playout buffer to a second buffer threshold and to a third buffer threshold; Determine whether the current playback lag time is greater than 0; If the fill level of the playback buffer satisfies the second buffer threshold, reducing a playback speed of the downloaded first video segment played on the display device; as well as If the fill level of the playback buffer satisfies the third buffer threshold and the playback lag is greater than 0, increasing the playback speed of the downloaded first video segment played on the display device, The second buffer threshold is smaller than the third buffer threshold.

20. The non-transitory computer-readable medium of claim 19, wherein the operations further comprise: comparing the fill level of the playout buffer to a fourth buffer threshold; as well as If the fill level of the playback buffer satisfies the fourth buffer threshold, reducing the playback speed of the downloaded first video segment played on the display device, Wherein the fourth buffer threshold is less than the second buffer threshold.

Images