Adaptive bitrate streaming using video quality information

Abstract

The invention relates to adaptive bitrate streaming using video quality information. In some implementations, a method receives a quality metric value for a profile level of a plurality of profile levels for a piece of content. An available bandwidth and an associated bit rate for a profile level of the plurality of profile levels are evaluated to select a first profile level. The quality metric value for a profile level of the plurality of profile levels is evaluated to select a second profile level. The method selects a profile level of the plurality of profile levels based on the first profile level and the second profile level. The selected profile level is requested for the segment.

Description

[Background technology]

[0001] In adaptive bitrate streaming, content (such as video or audio) is divided into multiple segments. The segments are then encoded at multiple levels, which can be called profile levels. Each profile level can be associated with different characteristics, which can be based on bitrate or quality. Content providers can distribute segments to client devices using a content delivery network. An adaptive bitrate algorithm can monitor the available bandwidth and dynamically adjust the profile level requested by the content. The adaptive bitrate algorithm can ensure that the client device gets the best viewing experience because the client device can request a profile level that can be considered optimal for the available bandwidth.

[0002] The content delivery network may charge content providers based on the amount of data transferred. The amount charged may be based on the rate multiplied by the amount of data transferred during a certain period of time. In some embodiments, the content delivery network may charge more during certain times of the day, which may be referred to as peak times. For example, peak times may be considered to be times in the evening when a large number of client devices are streaming content through playback sessions. Off-peak times may be during the day when a large number of users are not streaming content, resulting in a lower number of active playback sessions. Therefore, transferring large amounts of data during peak times may result in higher costs for content providers.

[0003] The amount of data transferred may depend on the profile level selected. For example, a higher profile level may have a higher bitrate and quality, thereby using a higher bitrate to transfer data through the content delivery network. Conversely, using a lower profile level may use a lower bitrate. The content delivery network may measure the amount of data based on the number of bytes transferred. Therefore, if a lower quality profile level with a lower bitrate is used, the cost may be lower than if a higher quality profile level with a higher bitrate is used. However, the adaptive bitrate algorithm is configured to request a profile level based on the available bandwidth and may determine the profile level with the highest possible bitrate that is not expected to cause rebuffering.

Brief Description of the Drawings

[0004] The included drawings are for illustrative purposes and are only used to provide examples of possible structures and operations of the disclosed inventive systems, devices, methods, and computer program products. These drawings in no way limit any changes in form and details that may be made by those skilled in the art without departing from the spirit and scope of the disclosed implementations.

[0005] Figure 1 A simplified system for performing an adaptive bitrate algorithm using bitrate and quality according to some embodiments is depicted.

[0006] Figure 2A A graph is shown for selecting a profile level when quality differences are minimal according to some embodiments.

[0007] Figure 2B Depicted are examples of cost savings when using quality metrics to select profile levels according to some implementations.

[0008] Figure 3 Depicted is a simplified flow chart of a method for determining peak hours and quality metric values ​​according to some embodiments.

[0009] Figure 4 Examples of quality metrics according to some implementations are depicted.

[0010] Figure 5 An example is depicted of sending peak hours and quality metrics using the DASH protocol in accordance with some implementations.

[0011] Figure 6 An example of a playlist for sending quality metrics using the HLS protocol according to some implementations is depicted.

[0012] Figure 7 Depicted is a simplified flow chart of a profile level selection method using an adaptive bitrate algorithm according to some embodiments.

[0013] Figure 8 Depicted is a table of calculated scores according to some embodiments.

[0014] Fig. 9 Depicted is a simplified flow chart of a method for performing an adaptive bitrate algorithm based on peak hours in accordance with some embodiments.

[0015] Fig.10 A video streaming system that communicates with multiple client devices via one or more communication networks is depicted according to one embodiment.

[0016] Fig.11 A diagrammatic view of an apparatus for viewing video content and advertisements is depicted. [Specific implementation method]

[0017] This article describes technologies for content distribution systems. In the following description, for the purpose of explanation, many examples and specific details are set forth to provide a thorough understanding of some implementations. Some implementations defined by the claims may include some or all of the features in these examples alone or in combination with other features described below, and may also include variations and equivalents of the features and concepts described herein.

[0018] System Overview

[0019] The content distribution system may determine which profile level to select based on the available bitrate and quality. In contrast to using only bitrate, the adaptive bitrate algorithm may also consider quality when selecting a profile level during content distribution. For example, the adaptive bitrate algorithm may determine whether a marginal gain in quality from a first profile level having a lower bitrate to a second profile level having a higher bitrate may not require selecting the second profile level having a higher bitrate. Selecting the first profile level may save bandwidth usage, such as during peak periods, which may transmit less data and therefore result in less cost. In addition, using a lower profile level may use less bandwidth than the currently available bandwidth, allowing the buffer to buffer more data. Having a larger data buffer may improve the distribution of content because the more data in the buffer, the more the playback of the content can adapt to any future changes in bandwidth. For example, a network outage may not affect playback much if the data in the buffer is used during the outage resolution period. In addition, during peak periods, the network may be more congested, and using a lower profile level may reduce the use of bandwidth, thereby improving the distribution of content, but the quality may be similar. During peak periods, the cost of distributing content is also reduced.

[0020] system

[0021] Figure 1 A simplified system 100 is depicted that performs an adaptive bitrate algorithm using bitrate and quality according to some embodiments. The system 100 includes a server system 102 and a client device 104. Although a single instance of the server system 102 and the client device 104 is shown, multiple instances of the server system 102 and the client device 104 are contemplated. For example, multiple client devices 104 can request content from a single server system 102 or multiple server systems 102.

[0022] The server system 102 includes a content management system 106 that can facilitate the distribution of content to the client devices 104. For example, the content management system 106 can communicate with a plurality of content distribution networks 118 (also referred to as content distribution networks 118) to distribute content to the plurality of client devices 104. The content distribution network 118 includes servers that can distribute content to the client devices 104. The content can be video, audio, or other types of content. The video can be used for discussion purposes, but other types of content can be used instead of video. In some embodiments, the content distribution network 118 distributes video clips to the client devices 104. The clip can be a portion of a video, such as a 6-second video. The video can be encoded with multiple profile levels, which correspond to different levels, which can be different bit rates or quality levels (such as resolution). The client device 104 can request a video clip from one of the profile levels based on current network conditions. For example, the client device 104 can use an adaptive bitrate algorithm to select a profile level for the video based on an estimated current available bandwidth and other network conditions.

[0023] Client device 104 may include a mobile phone, smart phone, set-top box, television, living room device, tablet device or other computing device. Client device 104 may include a media player 110 displayed on interface 112. Media player 110 or client device 104 may request content from content distribution network 118.

[0024] A profile ladder of requestable segments may be provided to the client device 104. The profile ladder may list different profile levels for each segment. The adaptive bitrate algorithm 114 may determine which profile level to select for each segment. The client device 104 may send segment requests associated with the profile level. For example, the client device 104 may request the segment using the identification information of the profile level. The content delivery network 118 may then send the video segment of the requested profile level to the client device 104, and the client device 104 may display the segment in the media player 110 on the interface 112. The client device 104 may change the profile level requested for the segment based on current network conditions.

[0025] The peak period prediction system 108 can predict when a peak period will occur. For example, there may be a peak period that occurs during the day. In some embodiments, the peak period can be defined by the content distribution network 118 and can be based on the number of currently active playback sessions, such as the number of playback sessions that can meet a threshold (e.g., higher than a threshold). For example, when an average of 1 million playback sessions occur simultaneously during a period, the content distribution network 118 can designate the period as a peak period. Other criteria can be used to determine when a peak period occurs, such as the total count of playback sessions during a period. The content management system 106 may not know when a peak period can be designated by the content distribution network 118. Therefore, the peak period prediction system 108 can analyze historical data and determine when a peak period can be designated by the content distribution network 118. If there are multiple content distribution networks 118, the peak period prediction system 108 can perform the same process for each content distribution network 118.

[0026] The quality determination system 116 can determine a quality metric value at the profile level. The quality metric value can be a numerical value that measures a quality characteristic. The quality metric value can indicate the perceived content quality. In some embodiments, video multi-method assessment fusion (VMAF) can be used as a quality metric because it can have a high correlation with the perceived video quality. The perceived video quality can estimate the video quality perceived by a human user when watching the content. If the perceived quality difference is negligible, the human user may not be able to tell the negligible quality difference. Other metrics, such as peak signal-to-noise ratio (PSNR), can also be used. The metric can be different from the parameters of the profile level used to encode the segment, which can be bit rate and resolution. That is, VMAF is a metric different from resolution. The quality determination system 116 can analyze the segments at each profile level and determine the quality metric value of each profile level for each segment.

[0027] The content management system 106 can provide peak hour predictions and quality metrics to the client device 104, for example, via the content distribution network 118. In some embodiments, the peak hour predictions and quality metrics can be provided via communications, which are sent using a streaming protocol such as HTTP Dynamic Adaptive Streaming (DASH) or HTTP Live Streaming (HLS). Streaming protocols can refer to communications differently, such as manifests in DASH or playlists in HLS. The communication can send information required to request a segment of the content of the session to be played (e.g., links to segments in the content), control information (e.g., segment lengths, etc.), and other information. Other methods can also be used to provide peak hour predictions and quality metrics, such as in out-of-band communications in a control channel. The reason why DASH or HLS is used to send communications in-band is that the information of the peak hours and quality metrics is sent in the manifest or playlist, which is being sent to enable streaming of the content.

[0028] The adaptive bitrate algorithm 114 may use the peak period prediction and the quality metric to determine the profile level to request for the upcoming segment. As described above, the adaptive bitrate algorithm 114 may incorporate the quality metric in addition to the bitrate when selecting the profile level to request.

[0029] Figure 2A A graph 200 is shown for selecting a profile level when a quality difference is minimal according to some embodiments. The Y-axis is a quality metric and the X-axis is a bitrate. A first curve 202 is based on characteristics of a first instance of content and a second curve 204 is based on characteristics of a second instance of content. For curve 202, the quality difference between the first profile level at selection 206-1 and the second profile level at selection 208-1 is minimal and may not be perceived by a user. However, for curve 204, the quality change between the first profile level at selection 206-2 and the second profile level at selection 208-2 is not minimal and may be perceived by a user. Although the term minimal is used, a threshold value may be used to determine when the quality difference is minimal (e.g., difference < threshold value). Thus, when curve 202 is encountered, a profile level with a lower bitrate at 206-1 may be selected. However, when curve 204 is encountered, a profile level with a higher bitrate at 208-2 may be selected because its perceived quality is more likely to be noticed by a user.

[0030] Figure 2BAn example of cost savings when a quality metric is used to select a profile level according to some embodiments is depicted. The Y-axis may be bandwidth usage and the X-axis may be the hour of the day. During peak hours at 212, curve 214 indicates bandwidth usage when the profile level at 206-1 is selected. The curve at 214 shows bandwidth usage when the profile level at 208-1 is selected. Thus, cost savings occur when a profile level with a lower bitrate is selected. However, the perceived quality difference is minimal and may not be perceived by a human user.

[0031] Peak hours and quality metrics

[0032] The following describes the determination of peak hours and quality metrics according to some embodiments. Figure 3 A simplified flow chart 300 of a method for determining peak time periods and quality metrics according to some embodiments is depicted. At 302, the server system 102 receives historical data. The historical data can be from a period such as N days, where N is a number. In some embodiments, the historical data can indicate the number of play sessions that were active during a portion of a day. In some embodiments, each day can have a cumulative play session count for each hour of the day. One or more hours in which the number of session counts is above a threshold can then be represented as a peak hour. Other methods can also be used to determine peak time periods, such as analyzing the number of bytes transmitted in a day and selecting the time period with the maximum number of bytes transmitted.

[0033] At 304, the peak hour prediction system 108 may determine the peak hour based on historical data. When the peak hour is unstable, the peak hour prediction system 108 may use a prediction method to predict the peak hour based on the predicted traffic. 1 ------D N Sort by time and D N The peak hour prediction system 108 calculates D N With D i The "distance" between them, i = 1------N-1, where the distance can be Euclidean distance or other vector distance metrics. Variable v 1 -----v k is the number of days k closest to day N, sorted by proximity. The predicted flow is:

[0034]

[0035] and

[0036]

[0037] Peak hour prediction system 108 calculation The peak hour h, where [h-∈, h+∈] is the peak period, where h is the hour and ∈ is a parameter measured in hours and can be tuned.

[0038] In an embodiment, there are two hours that may be peak hours, so the traffic data for one day indicates the count of active play sessions for each hour, and the historical data covers a period of N=5 days, which means there will be five 2-D vectors, and each 2-D vector represents the traffic data for one day. Let k=3, the data is shown in Table I:

[0039]

[0040]

[0041] Table I

[0042] The first column lists the day, the second column lists the traffic data, the third column lists the distance between a day and the current day, and the fourth column lists the variable v which is the number of days k closest to day N. 1 -----v k , sorted by proximity, and the fifth column lists the variable α j The predicted traffic data for day N+1 is The weighted average of:

[0043]

[0044] Therefore, the value of [6, 7] is the predicted active play session count for each candidate in the peak hour. The second hour with more sessions can be considered as D 6 In fact, candidates for peak hours are usually all hours of the day, so the daily traffic data is 24-hour data, which means that D i is a 24-dimensional vector.

[0045] When the peak hours tend to stabilize each day, such as from 7:00 PM to 9:00 PM, the peak hours may be hard-coded and not predicted for a period of time. Thereafter, additional historical data may be reanalyzed and the peak hours re-determined.

[0046] At 306, server system 102 causes the peak time periods to be sent to client device 104, for example via content distribution network 118. As described above, the peak time periods may be inserted into communications sent during the play session or in out-of-band communications. An embodiment of sending the peak time periods is described below.

[0047] At 308, the quality determination system 116 calculates quality metric values ​​for the profile levels. The quality metric calculations may be performed in parallel with or in series with the peak hour calculations. The quality metric values ​​may be numerical values ​​that measure quality characteristics of the segments for each profile level of the content. As described above, VMAF may be used as a quality metric due to its high correlation with subjective ratings of perceived video quality that may be obtained through human evaluation. The quality determination system 116 may analyze the segments of the content instance and assign a quality metric value to each profile level of the segments.

[0048] At 310, server system 102 may cause the quality metric to be sent to client device 104, such as via content distribution network 118. As described above, the quality metric may be inserted into communications sent during the playback session or out-of-band. Embodiments of sending quality metrics are described below. Figure 4 An example of a quality metric value in accordance with some embodiments is depicted. Graph 400 shows at 402 the quality levels of three segments, segment 1, segment 2, and segment 3, although an instance of content may have multiple segments. The Y-axis shows quality and the X-axis shows time in the instance of content. Also, profile level 1, profile level 2, and profile level 3 are shown, although other profile levels are contemplated. The quality determination system 116 may analyze the content of the segments and assign a quality metric value to a combination of the segments and profile levels. For example, a combination of segment 1, profile level 1 has a quality value of q 1，1 = 22; segment 1, where 1, 1 are the segment and profile identifiers. Profile level 2 has a quality metric value q 1，2 = 43, and segment 1, profile level 3 has a quality metric value q 1、3 = 80. Segment 2 and Segment 3 also have quality metric values ​​assigned to corresponding profile levels.

[0049] At 404, content-level quality metric values ​​are shown. The content-level quality metric may be an average of the segments for one profile level of the content instance. In some implementations, the content-level quality metric value may be an average of the corresponding quality metric scores for all segments for a particular profile level. For example, the average for profile level 1 is [22, 21, 23] = 22. The average for profile level 2 is 44, and the average for profile level 3 is 83. In some cases, when providing segment-level quality metrics is unacceptable, e.g., due to greatly increasing the manifest size due to too many segments, content-level quality metrics may be sent as an alternative for one or more segment-level quality metric values.

[0050] As described above, the peak hours and quality metrics may be sent during a playback session using in-band communications or via out-of-band communications. Communications may be sent using different protocols for messaging during a playback session. The adaptive bitrate algorithm 114 may be configured to receive quality metrics when communications are received during the playback process. For DASH, a manifest file is sent that describes the segments that may be requested. For HLS, a multi-variant playlist and multiple media playlists may be sent to describe the profile levels and the segments that may be requested.

[0051] Figure 5 An example of sending peak hours and quality metrics using the DASH protocol according to some embodiments is depicted. In this example, the quality metrics are sent as attributes of the DASH protocol. At 502, the average quality of profile level 1 is sent using a presentation tag. The average value for profile level 1 is 23. Then, at 504, the quality metrics of each segment of profile level 1 are sent using segment URL attributes. For example, the quality metric value seg_1 of segment 1 is 22, the quality metric value seg_2 of segment 2 is 21, and the quality metric value seg_3 of segment 3 is 23.

[0052] The quality metric value for profile level 2 is shown at 506 as an average of 43. Then, for profile level 2, at 508, the quality metric values ​​for segment 1, segment 2, and segment 3 are 43, 45, and 42, respectively. Although these attributes are used to send the above quality metric values, other attributes may be used. In addition, the peak time period may be sent using another attribute or via a separate communication other than the communication of the streaming protocol.

[0053] Figure 6An embodiment of a playlist 600 for sending quality metrics using the HLS protocol according to some embodiments is depicted. The quality metrics may be sent using attributes of HLS. A multi-variant playlist tag EXT-X-Stream-INF may be used to specify content-level video quality with a new attribute of average quality. A media playlist tag that specifies a segment-level quality metric for each segment may be defined. The media playlist tag EXT-X-Quality may specify the segment-level quality metric for each segment. For example, at 602, the average quality of the first profile level is 23; at 604, the average quality of the second profile level is 43; and at 606, the average quality of the third profile level is 83. Then, the media playlist tag of EXT-X-Quality specifies the segment-level quality metric for each segment. For example, at 608, a media playlist for the first profile level is shown. At 610, the quality metric value of segment 1 may be 22, at 612, the quality metric value of segment 2 may be 21, and at 614, the quality metric value of segment 3 may be 22. Furthermore, the peak period may be transmitted using another attribute or a separate communication other than the communication via the streaming protocol.

[0054] Figure 5 and Figure 6 The communications sent in the include other information that the client device 104 uses to request the segments for playback during the playback session, such as using a link to the segment. The quality metric and peak time period can be transmitted to the client device 104 in these communications.

[0055] Profile Level Selection

[0056] The client device 104 may select a profile level for one or more upcoming segments using an adaptive bitrate algorithm 114. The client device 104 may send a request for a segment at the selected profile level, and the segment is then distributed to the client device 104 by the content delivery network 118. The adaptive bitrate algorithm 114 uses the available bandwidth, the bitrate of the segment, and the quality metric of the segment in its decision process to select the profile level.

[0057] Figure 7 A simplified flow chart 700 of a profile selection method using an adaptive bitrate algorithm 114 according to some embodiments is depicted. At 702, a client device 104 receives a communication having a quality metric value. As described above, the quality metric value may have been received in a communication sent using a streaming protocol.

[0058] At 704, the adaptive bitrate algorithm 114 determines an estimate of the rebuffering time. The estimate of the rebuffering time may estimate a potential rebuffering time in a future period for each profile level. Rebuffering may occur when there is not enough data in the buffer to support continuous playback of the content. The estimate of the rebuffering time may be used to assess the risk of rebuffering when different profile levels are selected. The rebuffering time may be an amount of time in the future when rebuffering may occur. For example, the rebuffering time may be three seconds. A zero rebuffering time may indicate that no rebuffering will occur. In some embodiments, the adaptive bitrate algorithm 114 estimates a potential rebuffering time in a future period for each profile level. This step is to assess the risk of rebuffering when different profile levels are selected.

[0059] In some embodiments, the adaptive bitrate algorithm 114 estimates the possible rebuffering time during the download of the next L segments (e.g., from segment i to segment i+L-1). The adaptive bitrate algorithm 114 may use a function formulated as This means that the rebuffering time during the download of segment i to segment i+L-1 with quality level j is calculated with the help of function F using the information of the current buffer length, buffer length offset Δ and estimated bandwidth B.

[0060] The adaptive bitrate algorithm 114 calculates the rebuffering time using the function F as follows. The adaptive bitrate algorithm 114 converts r ij = 0, buf′ = buf + Δ initialization. For k from segment i to segment i + L-1: download time of segment k, Where S kj is the size of segment k with quality level j. The estimated rebuffering time during downloading segment k is r = max(0, t-buf'). The adaptive bitrate algorithm 114 adds the rebuffering time to the result: r ij =r ij +r and update the buffer length: buf′=max(0, buf′-t)+d k , where d k is the duration of segment k. The adaptive bitrate algorithm 114 calculates the rebuffering time r for each quality level j∈[1,M] ij , where the rebuffering time is an M-dimensional vector R.

[0061] The estimation method can simulate buffer length variations. The adaptive bitrate algorithm 114 estimates the rebuffering time for future segments L=3, for example to predict how long rebuffering will last during downloading segments i, i+1, i+2. In the estimation, the estimated bandwidth B is constant for all segments at all quality levels.

[0062] In some embodiments, the current “adjusted” buffer length buf+Δ is 1 second, and each segment d i The duration of is also constant at 5 seconds. The following process is simulated in sequence. For the download of segment i (for example, from 0 seconds to 5 seconds), the estimated download time is is 5 seconds, which means that from 0 to 5 seconds, the media player is downloading segment i. In parallel with the download, the content is still playing. Since there is only 1 second of content in the buffer, the buffer will become empty after 1 second. Playback will stop and rebuffering will occur until new content is received (the newly downloaded segment), which has a rebuffering time of 4 seconds. The buffer will be filled with the new content, and then the buffer length will be 5 seconds, equal to the duration of segment i.

[0063] The adaptive bitrate algorithm 114 simulates the download of segment i+1 (e.g., from 5 seconds to 11 seconds). The estimated buffer length before downloading segment i+1 is 5 seconds. Similarly, the estimated download time is 6 seconds (because segment i+1 has a larger size). Rebuffering will occur at 10 seconds and last for 1 second until the segment is finally downloaded at 11 seconds. The buffer will fill with new content and then the buffer length will be 5 seconds, equal to the duration of segment i+1.

[0064] The adaptive bitrate algorithm 114 simulates the download of segment i+2 (from 11 seconds to 15 seconds). The estimated download time is 4 seconds. Since the buffer still has 5 seconds of content to play, no rebuffering occurs until segment i+2 is finally downloaded at 15 seconds. Therefore, the total estimated rebuffering time r for quality j at segment i is ij A total of 5 seconds (4 seconds for segment i and 1 second for segment i+1).

[0065] At 706, the adaptive bitrate algorithm 114 determines a bitrate decision. The adaptive bitrate algorithm 114 may use the profile-level bitrate and the estimated rebuffering time for the segment to determine the profile level to select. The adaptive bitrate algorithm 114 may use a similar function to determine the bitrate decision and the quality decision described below. The function may use the bitrate when calculating the bitrate decision or the quality metric when calculating the quality decision. The function estimates the rebuffering time for the profile level, outputs a score, and selects a profile level based on the score.

[0066] In some embodiments, the function is X={x j} is an M-dimensional vector representing the bit rate for profile level j. When using bit rate, the adaptive bit rate algorithm 114 sets x j =b j, i.e., the bit rate of profile level j. The adaptive bitrate algorithm 114 estimates the rebuffering time R at 704 = {r ij}. The coefficient μ is the coefficient to combine X and R. The coefficient μ is different when X uses a bitrate or quality metric value, or may not be used. This is because using the same value of μ will make the tradeoff between video bitrate and rebuffering time very different from the tradeoff between video quality and rebuffering time, because the magnitude of bitrate and quality is different compared to rebuffering time. In the following embodiments, when the bitrate unit is kbps, μ=200 means that 1 second of rebuffering offsets a 200kbps bitrate increase. Such an increase in video bitrate may be common and feasible. However, if this value μ is applied to a quality metric (such as VMAF, μ is typically in the range of 20-100), then this means that 1 second of rebuffering offsets a 200 VMAF increase. Such an increase in video quality is not feasible given the normal range mentioned above. The threshold σ is a threshold that indicates whether two profile levels of the same segment have similar perceptual quality, which means that when When the first profile level j 1 and the second profile level j 2 can be considered to have similar quality (eg, within a threshold). When analyzing bitrate and quality, the threshold σ can be different.

[0067] The adaptive bitrate algorithm 114 calculates the function G as follows. The adaptive bitrate algorithm 114 initializes the decision j'=1. For a profile level j from 1 to M: The adaptive bitrate algorithm 114 calculates the score s for that level j :s j =x j -μ×r ij If profile level j can be distinguished from profile j′ with respect to bitrate and has a higher score, e.g., |x j′ -x j |>σ and s j >s j′ , the adaptive bitrate algorithm 114 selects profile: j′=j. When using bitrate, the adaptive bitrate algorithm 114 determines the profile level decision j b =G({b j}, R, μ b , σ b ). To make the decision consistent with adaptive bitrate algorithms that do not use quality metrics, the threshold σ can be b Set to 0.

[0068] After determining the bitrate decision, the adaptive bitrate algorithm 114 may determine a quality decision.At 708, the adaptive bitrate algorithm 114 determines a profile-level quality metric value for the segment.

[0069] At 710, the adaptive bitrate algorithm 114 determines a quality decision. The quality decision may select a profile level. Using the above function, the adaptive bitrate algorithm 114 may calculate another decision based on the quality metric value: q =G({q ij}, R, μ q , σ q ). When using the quality metric, the adaptive bitrate algorithm 114 sets, x j =q ij , i.e., the quality metric value of segment i with profile level j. If profile level j can be distinguished from profile j′ with respect to the quality metric value and has a higher score, e.g., |x j′ -x j |>σ and s j >s j′ , then the adaptive bitrate algorithm 114 selects profile: j′=j. When using the quality metric, the adaptive bitrate algorithm 114 determines the profile level decision j q =G({q ij}, R, μ q , σ q ).

[0070] At 712, the adaptive bitrate algorithm 114 selects a profile level based on the bitrate decision and the quality decision. Different methods can be used to select the profile level. In some embodiments, the adaptive bitrate algorithm 114 can select a lower profile level from the profile levels selected from the bitrate decision and the quality decision. For example, the adaptive bitrate algorithm 114 uses min(j b , j q ) selects a lower profile level for segment i. In some embodiments, if profile level 8 and profile level 6 have been selected, the adaptive bitrate algorithm 114 selects profile level 6 as the selected profile level. It is expected that profile level 6 may have a lower bitrate than profile level 8.

[0071] An embodiment of using a bitrate decision and a quality decision to calculate a profile level is described below. Figure 8A table 800 is depicted that calculates scores according to some embodiments. Column 802 includes profile level, column 804 includes rebuffer time, column 806 includes bitrate (kbps), column 808 includes quality metric value (VMAF), column 810 includes score for bitrate decision, and column 812 includes score for quality decision. The scores for bitrate decision and quality decision may be affected by rebuffer time in profiles 6 and 7. For example, for profile 6, a rebuffer time of 2 seconds has a bitrate score of 2100 at a bitrate of 2500 and a quality score of 70.68 at a quality metric value of 90.68.

[0072] After scoring, the bitrate decision and quality decision depend on the threshold σ b and σ q As mentioned above, in order to make the decision consistent with adaptive bitrate algorithms that do not use quality metrics, the threshold σ can be b is set to 0. Therefore, the bitrate decision will be profile level 6 with the highest score of 2100 at 814.

[0073] For the quality decision, the adaptive bitrate algorithm 114 determines that the marginal increase in the quality metric value when upgrading the profile level from profile level 4 to profile level 8 in column 808 becomes very small (e.g., the quality metric values ​​are 89.66, 90.25, 90.68, and 90.99, respectively). q = 0, then at 816, the quality decision will be profile level 5 with the highest score of 90.25. q = 1, which is possible because it may be difficult for human users to perceive when the quality metric score changes within 1, then the quality decision will be profile level 4 because there is no quality level j′ with a higher score than profile level 4, and the quality metric |x j′ -x 4 |>1. Profile level 5 has a quality metric score of 90.25, which is different from the value of 89.66 which is lower than 1. Therefore, profile 4 is selected because profile 4 has a lower bitrate.

[0074] If the profile level of the quality decision is a lower level than the profile level determined by the bitrate decision, the adaptive bitrate algorithm 114 may update the buffer length offset used in the bitrate decision and the quality decision to add the saved download time to the buffer length offset resulting from using the lower profile level. That is, using a lower profile level than would normally be selected without regard to the quality metric value may increase the buffer length because more bandwidth may be available given the bitrate at the lower profile level compared to the bitrate at the higher profile level. By adding the saved buffer length to the buffer length offset, the adaptive bitrate algorithm 114 may not be aware that more buffer length is available, which avoids switching to a higher than desired profile level the next time the adaptive bitrate algorithm is evaluated.

[0075] By adding the buffer length to the buffer length offset, additional buffer length can be added to account for any potential network issues in the future. This can improve the playback process because the buffer can have an additional buffer length that is resilient to sudden changes in network conditions. The logic for adding additional buffer length can be as follows. The adaptive bitrate algorithm 114 calculates the saved download time: The adaptive bitrate algorithm 114 then adds the saved download time to the buffer length offset: Δ = Δ + d. Following the previous embodiment, when the threshold σ b = 0 and threshold σ q = 1, the bit rate decision is j b =6 and the quality decision is j q =4, so the final quality level selected will be min(j b , j q )=4, and the buffer length offset will be updated to

[0076] peak hours

[0077] Can be based on when to specify peak hours Figure 7 For example, the process may be more aggressive in reducing the bit rate during peak hours.

[0078] Fig. 9 A simplified flow chart 900 of a method for performing an adaptive bitrate algorithm based on peak hours according to some embodiments is depicted. At 902, the adaptive bitrate algorithm 114 receives information about peak hours. As described above, the information about peak hours can be received in a communication (e.g., via a manifest or playlist) or in an out-of-band communication.

[0079] At 904, the adaptive bitrate algorithm 114 determines whether a peak period is being experienced at the future time. For example, the peak period may indicate an hour of the day, a minute of the day, a part of the day, etc. When the peak period is approaching, at 906, the adaptive bitrate algorithm 114 adjusts the parameters of the peak period. For example, the adaptive bitrate algorithm 114 may adjust the parameters in the bitrate decision or the quality decision based on the peak period experienced. In some embodiments, when in the peak period, the adaptive bitrate algorithm 114 may adjust the parameters to relax the requirements for determining the similarity of the quality metric values ​​between the two profile levels to save more bandwidth. For example, the adaptive bitrate algorithm 114 may preset two thresholds σ 1 <σ 2 As the threshold σ q When in peak hours, the adaptive bit rate algorithm 114 sets the threshold σ q =σ 2 To relax the requirement for determining the similarity of video quality between two quality levels, which can then save more bandwidth usage. Otherwise, at 908, when in non-peak hours, the adaptive bitrate algorithm 114 sets the threshold σ q =σ 1 To obtain higher quality. That is, by setting the threshold σ q Set equal to the threshold σ 2 , the adaptive bitrate algorithm 114 can allow more quality differences between profile levels to save more bandwidth. Otherwise, if it is not during peak hours, the threshold σ q Equal to or less than the threshold σ 2 The threshold σ 1 . Lower values ​​may emphasize selecting profile levels that have higher quality video. For example, the threshold σ 1 A profile level with higher quality can be selected instead to save bandwidth.

[0080] At 910, the adaptive bitrate algorithm 114 performs a process of selecting a profile level using the bitrate decision and the quality decision. If it is not a peak period, the parameters of the non-peak period can be used for the bitrate decision and the quality decision. If it is a peak period, the parameters of the peak period can be used in the bitrate decision and the quality decision. Other processes can also be used. For example, when it is not a peak period, the quality decision may not be calculated. Instead, only the profile level determined by the bitrate decision can be used.

[0081] in conclusion

[0082] Thus, the adaptive bitrate algorithm 114 can reduce the average bitrate during peak hours while also maintaining similar perceived quality. Thus, bandwidth is saved during peak hours, which can reduce costs. However, the perceived quality may be similar and not perceptible to the user. Additionally, adjustments to the buffer offset may be used so that the buffer length can be increased. Increased buffer length can improve playback performance by reducing rebuffering that may occur.

[0083] system

[0084] The features and aspects disclosed herein can be combined as Fig.10 The video streaming system 1000 shown in the embodiment is implemented by communicating with multiple client devices via one or more communication networks. Various aspects of the video streaming system 1000 are described only to provide an embodiment of an application for implementing the distribution and distribution of content prepared according to the present disclosure. It should be understood that the present technology is not limited to streaming video applications and can be applied to other applications and distribution mechanisms.

[0085] In one embodiment, a media program provider may include a media program library. For example, media programs may be aggregated and provided through a site (e.g., a website), an application, or a browser. A user may access the site or application of a media program provider and request media programs. A user may be limited to requesting only media programs provided by a media program provider.

[0086] In system 1000, video data can be obtained from one or more sources, such as from video source 1010, to be used as input to video content server 1002. Input video data can include original or edited frame-based video data in any suitable digital format, such as Moving Picture Experts Group (MPEG)-1, MPEG-2, MPEG-4, VC-1, H.264 / Advanced Video Coding (AVC), High Efficiency Video Coding (HEVC), or other formats. In another option, the video can be provided in a non-digital format and converted to a digital format using a scanner or code converter. The input video data can include various types of video clips or programs, such as television series, movies, and other content produced as primary content of interest to consumers. The video data can also include audio or can use only audio.

[0087] The video streaming system 1000 may include one or more computer servers or modules 1002, 1004, and 1007 distributed on one or more computers. Each server 1002, 1004, 1007 may include or may be operably connected to one or more data stores 1009, such as a database, index, file, or other data structure. The video content server 1002 may access a data store (not shown) of various video clips. The video content server 1002 may provide video clips as directed by a user interface controller in communication with a client device. As used herein, a video clip refers to a distinct portion of frame-based video data, such as may be used in a streaming video session to watch a television series, a movie, a recorded live performance, or other video content.

[0088] In some implementations, the video ad server 1004 may access a data store of relatively short videos (e.g., 10, 30, or 60 second video ads) configured as ads for a particular advertiser or message. The ads may be provided to the advertiser in exchange for some payment, or may include a promotional message, a public service message, or some other information for the system 1000. The video ad server 1004 may provide the video ad clips as directed by a user interface controller (not shown).

[0089] The video streaming system 1000 may also include a server system 102 .

[0090] The video streaming system 1000 may also include an integration and streaming component 1007 that integrates video content and video advertisements into the streaming video clips. For example, the streaming component 1007 may be a content server or a streaming media server. A controller (not shown) may determine the selection or configuration of advertisements in the streaming video based on any suitable algorithm or process. The video streaming system 1000 may include Fig.10 Other modules or units not depicted, such as management server, business server, network infrastructure, advertisement selection engine, etc.

[0091] The video streaming system 1000 can be connected to a data communication network 1012. The data communication network 1012 can include a local area network (LAN), a wide area network (WAN) (e.g., the Internet), a telephone network, a wireless network 1014 (e.g., a wireless cellular telecommunication network (WCS)), or some combination of these or similar networks.

[0092] One or more client devices 1020 may communicate with the video streaming system 1000 via a data communications network 1012, a wireless network 1014, or another network. Such client devices may include, for example, one or more laptop computers 1020-1, desktop computers 1020-2, "smart" mobile phones 1020-3, tablet devices 1020-4, network-enabled televisions 1020-5, or a combination thereof (via a router 1018 of a LAN, via a base station 1017 of a wireless network 1014, or via some other connection). In operation, such client devices 1020 may send and receive data or instructions to the system 1000 in response to user input or other input received from a user input device. In response, the system 1000 may provide video clips and metadata from a data store 1009 to the client device 1020 in response to a selection of a media program. The client device 1020 may output video content from the streaming video clip in a media player using a display screen, projector, or other video output device, and receive user input for interacting with the video content.

[0093] The distribution of audio-video data can be implemented to remote client devices from streaming components 1007 using various methods (e.g., streaming) through a combination of a computer network, a telecommunication network, and such networks. In streaming, a content server continuously streams audio-video data to a media player component that operates at least partially on a client device, and these media player components can play audio-video data while receiving streaming data from the server. Although streaming has been discussed, other distribution methods can be used. The media player component can initiate the playback of video data immediately after receiving the initial portion of data from a content provider. Traditional streaming technology uses a single provider that distributes data streams to a group of end users. High bandwidth and processing power may be required to distribute a single stream to a large audience, and the required bandwidth of the provider may increase with the increase of the number of end users.

[0094] Streaming media can be distributed on demand or live. Streaming enables immediate playback at any point in the file. The end user can skip the media file to start playing or change the playback to any point in the media file. Therefore, the end user does not need to wait for the file to be downloaded step by step. Usually, streaming media is distributed from several special servers with high bandwidth capabilities via a dedicated device that accepts the request for the video file, and the information about the format, bandwidth and structure of these files is used to distribute only the amount of data required for playing the video at the rate required for playing the video. The streaming media server can also consider the transmission bandwidth and the ability of the media player on the destination client. Streaming components 1007 can use control messages and data messages to communicate with client devices 1020 to adjust to the changing network conditions when playing the video. These control messages can include commands for enabling control functions, such as fast forward, fast rewind, pause or seek a specific part of the file at the client.

[0095] Because streaming component 1007 transmits video data only as needed and at the desired rate, precise control over the number of streams served can be maintained. Viewers would not be able to watch high data rate video over a lower data rate transmission medium. However, streaming media servers (1) provide users with random access to video files, (2) allow monitoring of who is watching what video program and for how long, (3) use transmission bandwidth more efficiently because only the amount of data required to support the viewing experience is transmitted, and (4) video files are not stored on the viewer's computer but are discarded by the media player, allowing more control over the content.

[0096] Streaming component 1007 can use TCP-based protocols, such as hypertext transfer protocol (HTTP) and real-time messaging protocol (RTMP). Streaming component 1007 can also distribute real-time webcasts and can be multicast, which allows more than one client to tune into a single stream, thereby saving bandwidth. Streaming media player can provide random access to any point in the media program without relying on buffering the entire video. On the contrary, this is achieved using control messages transmitted from the media player to the streaming media server. Other protocols for streaming have HTTP live streaming (HLS) or dynamic adaptive streaming (DASH) on HTTP. HLS and DASH protocols distribute videos on HTTP via a playlist of small segments, which are usually available from one or more content distribution networks (CDNs) with various bit rates. This allows media players to switch bit rates and content sources segment by segment. This switching helps to compensate for network bandwidth changes and infrastructure failures that may occur during video playback.

[0097] The distribution of video content via streaming can be accomplished under a variety of models. In one model, a user pays for viewing a video program, for example, by paying for access to a library of media programs or a limited portion of media programs, or by using a pay-per-view service. In another model, which became widely adopted by broadcast television soon after its inception, sponsors pay for the presentation of a media program in exchange for the right to present advertisements during or near the presentation of the program. In some models, advertisements are inserted at predetermined times in the video program, which times may be referred to as "ad spots" or "ad breaks." For streaming video, a media player may be configured so that during a specified advertising period, a client device cannot play the video without also playing a predetermined advertisement.

[0098] refer to Fig.11 , shows a schematic diagram of an apparatus 1100 for viewing video content and advertisements. In a selected embodiment, the apparatus 1100 may include a processor (CPU) 1102 operably coupled to a processor memory 1104, which stores binary-coded functional modules for execution by the processor 1102. Such functional modules may include an operating system 1106 for handling system functions such as input / output and memory access, a browser 1108 for displaying web pages, and a media player 1110 for playing videos. The modules may also include an adaptive bit rate algorithm 114. The memory 1104 may store Fig.11 Additional modules not shown, such as modules for performing other operations described elsewhere in this document.

[0099] The bus 1114 or other communication components may support information communication within the device 1100. The processor 1102 may be a dedicated or specialized microprocessor that is configured or operable to perform specific tasks in accordance with the features and aspects disclosed herein by executing machine-readable software code that defines the specific tasks. A processor memory 1104 (e.g., a random access memory (RAM) or other dynamic storage device) may be connected to the bus 1114 or directly to the processor 1102 and store information and instructions to be executed by the processor 1102. The memory 1104 may also store temporary variables or other intermediate information during the execution of such instructions.

[0100] The computer-readable medium in the storage device 1124 can be connected to the bus 1114 and store static information and instructions for the processor 1102; for example, when the apparatus 1100 is powered off, the storage device (CRM) 1124 can store modules for the operating system 1106, the browser 1108, and the media player 1110, and when the apparatus 1100 is powered on, the modules can be loaded from the storage device 1124 into the processor memory 1104. The storage device 1124 may include a non-transitory computer-readable storage medium that stores information, instructions, or some combination thereof, such as instructions that, when executed by the processor 1102, cause the apparatus 1100 to be configured or operable to perform one or more operations of the methods described herein.

[0101] A network communication (comm.) interface 1116 may also be connected to the bus 1114. The network communication interface 1116 may optionally provide or support bidirectional data communication between the apparatus 1100 and one or more external devices (e.g., the streaming system 1000) via a router / modem 1126 and a wired or wireless connection 1125. Alternatively or additionally, the apparatus 1100 may include a transceiver 1118 connected to an antenna 1129, through which the apparatus 1100 may communicate wirelessly with a base station for a wireless communication system or with a router / modem 1126. In another alternative, the apparatus 1100 may communicate with the video streaming system 1000 via a local area network, a virtual private network, or other network. In another alternative, the apparatus 1100 may be incorporated as a module or component of the system 1000 and communicate with other components via the bus 1114 or by some other modality.

[0102] The device 1100 may be connected (e.g., via the bus 1114 and the graphics processing unit 1120) to a display unit 1128. The display 1128 may include any suitable configuration for displaying information to an operator of the device 1100. For example, the display 1128 may include or utilize a liquid crystal display (LCD), a touch screen LCD (e.g., a capacitive display), a light emitting diode (LED) display, a projector, or other display device to present information to a user of the device 1100 in a visual display.

[0103] One or more input devices 1130 (e.g., an alphanumeric keyboard, microphone, keypad, remote control, game controller, camera, or camera array) can be connected to the bus 1114 via the user input port 1122 to transmit information and commands to the device 1100. In selected embodiments, the input device 1130 can provide or support control of cursor positioning. Such a cursor control device (also known as a pointing device) can be configured as a mouse, trackball, trackpad, touch screen, cursor direction keys, or other device for receiving or tracking physical movement and converting the movement into electrical signals indicating cursor movement. The cursor control device can be incorporated into the display unit 1128, for example, using a touch-sensitive screen. The cursor control device can transmit direction information and command selections to the processor 1102 and control cursor movement on the display 1128. The cursor control device can have two or more degrees of freedom, such as allowing the device to specify a cursor position in a plane or three-dimensional space.

[0104] Some embodiments may be implemented in a non-transitory computer-readable storage medium for use with or in conjunction with an instruction execution system, apparatus, system, or machine. The computer-readable storage medium contains instructions for controlling a computer system to perform the methods described in some embodiments. The computer system may include one or more computing devices. The instructions may be configured or operable to perform the instructions described in some embodiments when executed by one or more computer processors.

[0105] As used in the description herein and throughout the appended claims, "a", "an", and "the" include plural references unless the context clearly dictates otherwise. Also, as used in the description herein and throughout the appended claims, the meaning of "in" includes "in" and "on", unless the context clearly dictates otherwise.

[0106] The above description illustrates various embodiments and examples of how aspects of some embodiments may be implemented. The above embodiments and implementations should not be considered to be the only implementations, and are presented to illustrate the flexibility and advantages of some embodiments defined by the appended claims. Based on the above disclosure and the appended claims, other arrangements, implementations, implementations, and equivalents may be employed without departing from the scope of the invention defined by the claims.

Claims

1. A method comprising: receiving a quality metric value for a profile level in a plurality of profile levels for a content segment; evaluating available bandwidths and associated bit rates of profile levels in the plurality of profile levels to select a first profile level; evaluating the quality metric of a profile level in the plurality of profile levels to select a second profile level; and A profile level among the plurality of profile levels is selected based on the first profile level and the second profile level, wherein the selected profile level is requested for the fragment.

2. The method according to claim 1, wherein: Receiving the quality metric value includes: The quality metric value is received in communication with information requesting the content segment.

3. The method according to claim 1, wherein: Receiving the quality metric value includes: The quality metric value is received in communication with information regarding the plurality of profile levels.

4. The method according to claim 1, wherein: Receiving the quality metric value includes: The quality metric value is received using attributes associated with a protocol for transmitting information regarding the plurality of profile levels.

5. The method according to claim 1, wherein: Evaluating the available bandwidths and the associated bit rates of profile levels in the plurality of profile levels to select the first profile level comprises: generating a score for a profile level in the plurality of profile levels based on a comparison of the available bandwidth and a corresponding bit rate of the profile level; and The first profile level is selected based on the respective scores of the profile levels in the plurality of profile levels.

6. The method of claim 5, wherein generating the score comprises: calculating a rebuffering time for the profile level in the plurality of profile levels based on the available bandwidth; as well as The score for the profile level is generated based on the re-buffering time and the corresponding bit rate for the profile level.

7. The method of claim 1 , wherein evaluating the quality metric value of a profile level in the plurality of profile levels to select the second profile level comprises: generating a score for a profile level in the plurality of profile levels based on the quality metric; as well as The first profile level is selected based on the respective scores of the profile levels in the plurality of profile levels.

8. The method of claim 7, wherein generating the score comprises: calculating a rebuffering time for the profile level in the plurality of profile levels based on the available bandwidth; as well as The score for the profile level is generated based on the re-buffering time and the corresponding quality metric score for the profile level.

9. The method according to claim 1, further comprising: calculating a rebuffering time for the profile level among the plurality of profile levels based on the available bandwidth and the corresponding bit rate for the profile level; as well as The first profile level and the second profile level are selected using the re-buffering times of the profile levels.

10. The method of claim 1, wherein the segment comprises a first segment, the method further comprising: defining a first parameter value for a first time period and a second parameter value for a second time period; selecting the second profile level for the first segment using the first parameter value in evaluating the quality metric value of a profile level in the plurality of profile levels; as well as The second parameter value is used in evaluating the quality metric value of a profile level in the plurality of profile levels to select a third profile level in the plurality of profile levels for a second segment.

11. The method according to claim 10, wherein: Evaluating available bandwidths and associated bit rates of profile levels in the plurality of profile levels to select the first profile level comprises: The first profile level is selected using a third parameter value.

12. The method of claim 1, wherein the segment comprises a first segment, the method further comprising: Determine the time period; selecting the profile level using the first profile level and the second profile level when selecting the profile level for the first segment within the time period; and When it is not within the time period, for the second segment, perform: evaluating available bandwidths and associated bit rates of profile levels in the plurality of profile levels to select a third profile level; and The third profile level of the plurality of profile levels is selected, wherein the third profile level is requested for the second fragment.

13. The method of claim 10, wherein: The first time period is predicted to be a time when a content distribution network charges a first price for distributing the content; and The second time period is predicted to be a time when the content distribution network charges a second price for distributing the content, wherein the second price is lower than the first price.

14. A non-transitory computer-readable storage medium having stored thereon computer-executable instructions which, when executed by a computing device, render the computing device operable to: receiving a quality metric value for a profile level in a plurality of profile levels for a content segment; evaluating available bandwidths and associated bit rates of profile levels in the plurality of profile levels to select a first profile level; evaluating the quality metric of a profile level in the plurality of profile levels to select a second profile level; and A profile level among the plurality of profile levels is selected based on the first profile level and the second profile level, wherein the selected profile level is requested for the fragment.

15. A method comprising: receiving data from a time when the content instance was distributed; Determine the time period based on data that meets the characteristics; calculating a quality metric value of a profile in a plurality of profiles for a segment of the content instance based on analyzing characteristics of the segment; as well as The time period and the quality metric value of a profile in the plurality of profiles for the segment are caused to be sent to a client device, wherein the client device selects a profile in the plurality of profiles for the segment using the time period and the quality metric value.

16. The method according to claim 15, wherein: The quality metric of a profile is based on a characteristic measuring a perceived quality of the segment and the corresponding profile.

17. The method according to claim 15, wherein: Causing sending of the quality metric value for the configuration file to include: The quality metric value of the configuration file is caused to be sent in communication with information for requesting the content segment.

18. The method according to claim 17, wherein: The quality metric value of the profile is sent using properties of a protocol in the communication.

19. The method according to claim 15, wherein: Such that sending the time period comprises: The time period is caused to be sent in communication with information for requesting the segment.

20. The method of claim 15, further comprising calculating an average of the profiles of the plurality of segments of the content instance using the quality metric values ​​of the profiles of the plurality of segments.

Images