Cross-period quality smoothing in adaptive bit rate algorithms

Abstract

In some embodiments, a method generates a reward function for a profile in a profile step based on a current segment of bandwidth and content for an adaptive bitrate function. The reward function indicates that a first profile should be selected for the current segment. The method performs a quality change analysis to select a second profile from the profiles. The quality change analysis analyzes quality changes based on a previously selected profile from a previous segment and a profile in a profile step. A second profile is requested for the current segment from the profile step.

Description

Background Art

[0001] In a video streaming system, a video provider may use a content delivery network (CDN) to store segments of content (e.g., video or audio) and deliver these segments to a client. In order to improve the quality of video delivery, content may be provided in multiple profiles with different quality and bitrate characteristics. In a simple example, a profile ladder may include high bitrate / high quality, medium bitrate / medium quality, and low bitrate / low quality profiles; however, the profile ladder may have more profiles. The client may analyze the network conditions to determine the profile to be selected from the profile ladder. For example, a client may use an adaptive bitrate (ABR) algorithm to select a suitable profile for a video segment to be downloaded from a content delivery network based on the current network conditions. The adaptive bitrate algorithm may consider the estimated available network bandwidth and the bitrate of the upcoming segment to determine the profile to be requested for one or more upcoming segments. For example, if the current network condition has a high available bandwidth, the client selects a high bitrate / high quality profile, and if the current network condition has a low available bandwidth, the client selects a low bitrate / low quality profile. This process works well when the profile ladder used is consistent throughout the life cycle of a session. However, in some cases, the video ladder may not be the same throughout a playback session. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] The included drawings are for illustrative purposes and are only used to provide examples of possible structures and operations of the disclosed inventive systems, devices, methods, and computer program products. These drawings are in no way intended to limit any changes in form and detail that may be made by those skilled in the art without departing from the spirit and scope of the disclosed implementations.

[0003] Figure 1 A simplified system for streaming segments of content is depicted in accordance with some embodiments.

[0004] Figure 2A Depicted is a first profile ladder that may be associated with a first time period in accordance with some embodiments.

[0005] Figure 2B A second profile ladder is depicted in accordance with some embodiments.

[0006] Figure 2C A third profile ladder is depicted in accordance with some embodiments.

[0007] Figure 3 Depicted is a simplified flow chart method for selecting a profile using an adaptive bitrate algorithm according to some embodiments.

[0008] Figure 4 Depicted is a simplified flow chart of mass change analysis according to some embodiments.

[0009] Figure 5 A diagram depicting a quality-based preference supplement according to some embodiments.

[0010] Fig. 6A The adaptive bit rate function f(x) according to some embodiments is depicted n )’s picture.

[0011] Figure 6B Depicted is an example of a graph applying quality variation to an adaptive bitrate function in accordance with some embodiments.

[0012] Figure 7 Depicted is a profile ladder including profiles in accordance with some embodiments.

[0013] Figure 8 A video streaming system that communicates with a plurality of client devices via one or more communication networks is depicted according to one embodiment.

[0014] Fig. 9 A schematic diagram of an apparatus for viewing video content and advertisements is depicted. DETAILED DESCRIPTION

[0015] Described herein are technologies for video streaming systems. In the following description, for the purpose of explanation, many examples and specific details are set forth to provide a thorough understanding of some embodiments. Some embodiments as defined by the claims may include some or all of the features in these examples alone or in combination with other features described below, and may also include modifications and equivalents of the features and concepts described herein.

[0016] System Overview

[0017] The video streaming system can deliver content during a streaming session that can include multiple time periods. In some embodiments, the streaming session can include a bumper period, a supplementary content period, and a main content period, but other periods can be used, such as multiple supplementary content periods, an ending period, etc. The bumper period can include an introduction, such as a logo of a video provider or production company, a preview of the main content, or other content. The supplementary content can include supplementary content added to the main content. The supplementary content can be an advertisement inserted during a break in the playback of the main content. The main content can be content requested by a user account for playback, such as a movie, a show, etc. The ending content can be content played after the main content.

[0018] Different time periods may have associated profile ladders that may be different. For example, the guide content, supplemental content, or primary content may be different because the content displayed in each time period may be unrelated. For example, the guide content may be a preview of a program, and the supplemental content may be an advertisement that is unrelated to the preview of the program. The guide time period may use a first profile ladder, and the supplemental content time period may use a second profile ladder. The first profile ladder and the second profile ladder may have profiles with different resolutions and bit rates. In addition, the primary content may have content that is unrelated to the supplemental content, such as characters from a show or movie that are not found in the supplemental content. The third profile ladder for the primary content may be different from the first profile ladder and the second profile ladder.

[0019] When a streaming session transitions between different profile steps (such as from one time period to another), the playback experience may become degraded. For example, the content played back during the guide period may have a higher resolution. However, when the supplementary content period begins, low-resolution content may be displayed due to the use of different profile steps. As will be described below, the first profile step may display content at a resolution of 2560x1440 at a bit rate of approximately 4,500,000 bits per second (bps), while the second profile step may display content at a resolution of 1280x720 at approximately the same bit rate. In this case, a segment with a resolution of 2560x1440 may be displayed, and then the resolution of the next segment may be 1280x720. The sudden change in resolution may be noticeable to the viewer and degrade the playback experience.

[0020] In some embodiments, the system improves the profile selection of the adaptive bitrate algorithm when the profile ladder may change, such as when there is a period change. The adaptive bitrate algorithm can consider network bandwidth and the bitrate of the video profile, but also consider changes in quality. Considering changes in quality when selecting a new profile can improve the playback experience by smoothing transitions when playing back multiple segments. For example, when the adaptive bitrate algorithm considers quality changes, the difference in quality when the streaming session transitions from a guide period to a supplemental content period may not be as drastic. As will be described in more detail below, the adaptive bitrate algorithm can consider changes in quality levels or changes in profile levels when making decisions about selecting a new profile. Although considering quality changes when period changes or profile ladder changes occur will be discussed below, the adaptive bitrate algorithm can consider quality changes during the entire playback session even if the same profile ladder is being used.

[0021] system

[0022] Figure 1A simplified system 100 for streaming segments of content is depicted in accordance with some embodiments. System 100 includes a content delivery network 102 and a client 104. Although a single instance of content delivery network 102 and client 104 is shown, it is understood that multiple instances of content delivery network 102 and client 104 may be requesting content from a single content delivery network 102 or multiple content delivery networks 102.

[0023] The content delivery network 102 includes servers that can deliver content to the client 104. For example, the content delivery network 102 may include one or more servers that can deliver content to the client 104. In some embodiments, the content delivery network 102 delivers segments of a video to the client 104. These segments can be a portion of a video, such as six seconds in a video. The video can be encoded in a plurality of profiles corresponding to different levels, which can be different levels of bit rate or quality. The client 104 can request a segment of a video from one of the profile levels based on the current network conditions. For example, the client 104 can use an adaptive bit rate algorithm to select a profile for the video based on the estimated current available bandwidth and other network conditions.

[0024] Client 104 may include a mobile phone, smart phone, set-top box, television, living room device, tablet device or other computing device. Client 104 may include a media player 110 displayed on interface 112. Media player 110 or client 104 may request content from content delivery network 102. In some embodiments, the content may be video, audio or other content. Media player 110 may use adaptive bit rate system 108 to select a profile when requesting a segment of content. In response, content delivery network 102 may deliver (e.g., stream) the segment in the requested profile to client 104 for playback using media player 110.

[0025] For the segments that can be requested, a profile ladder can be provided to the client 104. The profile ladder can list different profiles for each segment. The adaptive bitrate system 108 can use an adaptive bitrate algorithm to determine which profile to select for each segment. The client 104 can send a request for a segment associated with a profile. For example, the client 104 can request a segment using the identification information of the profile. The content delivery network 102 can then send a segment of the video for the requested profile to the client 104, which can then display the segment in the media player 110 on the interface 112. The client 104 can change the profile requested for the segment based on the current network conditions.

[0026] The use of an adaptive bitrate algorithm to improve the playback experience is described below.As discussed above, the decision about which profile to select may be based on quality changes.

[0027] Profile Ladder

[0028] FIG. 2A to FIG. 2C Different profile ladders according to some embodiments are depicted. The profile ladder includes profiles that can be used to request content. These profiles include different levels associated with one or more characteristics, such as a bit rate characteristic or a quality characteristic. The bit rate can be the number of bits per second associated with a segment, and the quality characteristic can indicate the quality of the video (e.g., resolution or other metric). The resolution of the video can be based on the number of pixels in the frame, and higher resolutions are associated with higher quality. Higher bit rates can also be associated with higher quality. The level of the profile in the profile ladder can be determined based on how the video is transcoded for that level, such as the video can be transcoded at 400000bps and a resolution of 1920x1080. The resolution of 1920x1080 can represent the number of pixels used in the video.

[0029] Figure 2A A first profile ladder 200 that can be associated with a first time period according to some embodiments is depicted. In some examples, the profile ladder 200 can be associated with a guide time period in which guide content is played. The profile ladder 200 includes a first column 201-1 listing profiles, a second column 202-1 showing resolutions of different profiles, and a third column 204-1 showing the associated bit rates of the profiles. In each row of the profile ladder 200, different profile levels are listed for profiles 0-4, where profile #0 is considered a lower profile and profile #4 is considered a higher profile. For example, at 208, profile #0 can have a resolution of 640x360 and a bit rate of 573801bps (bits per second). In addition, at 210, profile #4 can have a resolution of 2560x1440 and a bit rate of 4324470bps. Each profile can have an associated resolution and bit rate.

[0030] Figure 2BA second profile ladder 206 is depicted in accordance with some embodiments. In some examples, the second profile ladder 206 can be used in a second period that can be associated with a supplemental content period in which supplemental content is displayed. The second profile ladder 206 can include different resolutions or bit rates in columns 202-2 and 204-2 compared to the first profile ladder 200, and the second profile ladder 206 also includes six profiles #0 to #5 in column 201-2. In some embodiments, at least one resolution or bit rate can be different from the profile ladder 200. For example, at 212, profile #0 in the second profile ladder 206 includes a resolution of 640x360 and a bit rate of 710538bps. In this case, the resolution of profile #0 in the second profile ladder 206 is the same as the resolution found in profile #0 in the first profile ladder 200. However, in the second profile ladder 206, the bit rate of 710538 in profile #0 at 212 is different from the bit rate of 573801 in profile #0 in the first profile ladder 200 at 208. For profile #4 of the two profile ladders, the resolution and bit rate are different, as shown at 210 and 214. That is, the resolution of profile #4 in the first profile ladder 200 at 210 is 2560x1440, which is different from the resolution of 1920x1080 at 214 in the second profile ladder 206. Similarly, the bit rate of 4324470 at 210 in the first profile ladder 200 is different from the bit rate of 5287095 at 214 in the second profile ladder 206. In addition, there are other differences between the two profile ladders, such as profile #5 of the second profile ladder 206 is not included in the first profile ladder 200.

[0031] Figure 2C A third profile ladder 216 is depicted in accordance with some embodiments. In some examples, the third profile ladder 216 can be used in a third period that can be associated with a primary content period in which primary content is displayed. The third profile ladder 216 includes different resolutions or bit rates in columns 202-3 and 204-3 compared to the first profile ladder 200 and the second profile ladder 206, and the third profile ladder 216 also includes a different number of profiles #0 through #6 in column 201-3. Although three profile ladders are discussed, different numbers of two or more profile ladders are contemplated.

[0032] Problems may occur when the profile ladder changes between the first profile ladder 200, the second profile ladder 206, or the third profile ladder 216 during a streaming session. In this case, when the profile ladder changes for a segment, such as when the profile ladder changes for a continuous segment, the quality of the segment may change even when the bit rate remains similar. For example, after the time period changes, the high-resolution guide content may become a lower-resolution supplementary content. In some examples, if the available bandwidth is 4400000bps, the profile #4 at 210 in the first profile ladder 200 can be used by the adaptive bitrate algorithm to request video segments during the first time period. However, when the time period changes to use the second profile ladder 206, the adaptive bitrate algorithm can select the profile #3 at 218. Because the bit rate of profile #3 is 4334738bps, this is the closest bit rate to the available bandwidth of 4400000bps at this time. However, the quality of the clips in profile #3 of the second profile ladder 206 is a resolution of 1280x720, which is a much lower quality than the resolution of 2560x1440 in profile #4 of the first profile ladder 200 at 210. Therefore, when watching a video clip with a resolution of 2560x1440 and then watching a video clip with a resolution of 1280x720 thereafter, the quality difference may be noticeable and the playback experience is degraded. If a profile with a higher quality, such as profile #4 at 214, is used in the second profile ladder 206, the quality can be increased to a resolution of 1920x1080, but the bit rate required to download the clip is 5287095bps. If profile #4 is used, playback problems may result, such as rebuffering due to the bit rate of the video clip being higher than the available bandwidth of 4400000bps. Similar problems may result when transitioning from the second profile ladder 206 to the third profile ladder 216.

[0033] To overcome the above problems, the adaptive bitrate system 108 may take quality changes into account when determining changes in profiles.

[0034] Adaptive bit rate algorithm

[0035] Figure 3A simplified flow chart 300 of a method for selecting a profile using an adaptive bitrate algorithm according to some embodiments is depicted. At 302, the adaptive bitrate system 108 executes an adaptive bitrate algorithm to generate an adaptive bitrate function. The adaptive bitrate function can represent the profiles of the profile ladder with a reward value. For example, based on an analysis of the available bitrate of the estimate of the current network conditions and the bitrate of the associated profile, the adaptive bitrate algorithm can generate a reward value for each profile. The adaptive bitrate function can also take into account other factors, such as rebuffering possibility. The adaptive bitrate function can be used to determine the reward value. An example of an adaptive bitrate function will be described in FIG. 6 below.

[0036] The adaptive bitrate system 108 can select a profile using the available bandwidth and the bitrate information of the corresponding profile. The adaptive bitrate algorithm can select the highest profile with a bitrate that is less than the available bandwidth. The adaptive bitrate algorithm can also consider other factors, such as buffer length, to select the profile with the highest bitrate while avoiding rebuffering and frequent bitrate changes.

[0037] At 304, the adaptive bitrate system 108 performs a quality change analysis. The quality change analysis may involve analyzing changes in quality levels or profile levels. Examples of changes in quality levels and profile levels are described below in Figures 4 to 7 The quality variation analysis may adjust the reward value that may have been generated by the adaptive bitrate algorithm for the adaptive bitrate function.

[0038] At 306, the adaptive bitrate system 108 selects a profile based on the quality change analysis. For example, the previous profile may have been equivalent to Profile #5 in the profile ladder of the segment to be requested, which has a resolution of 1920x1080. The original adaptive bitrate function may have indicated that Profile #1 provides the highest reward. However, the quality change analysis indicates that the change in quality may be too large and may cause a degradation in the viewer experience. The quality change analysis may indicate that Profile #3 should be selected for the next segment requested. Profile #1 may have a resolution of 854x480, while Profile #3 may have a resolution of 1280x720. The resolution of 1280x720 may be closer to the resolution of 1920x1080 of the most recent segment. Using Profile #3 in the new profile ladder may provide an improved viewing experience, where consecutive video segments include more similar quality. This can make the viewer experience smooth, and the viewer may not notice the sudden change in quality. Additionally, the new profile selected may smooth bitrate variations across multiple segments, which may improve the playback experience.

[0039] Quality change analysis

[0040] Figure 4 A simplified flow chart of quality variation analysis according to some embodiments is depicted. At 402, the adaptive bitrate system 108 generates an adaptive bitrate function. The adaptive bitrate function can be based on an estimated available bitrate and a bitrate associated with a profile ladder. Additionally, at 404, the adaptive bitrate system 108 generates a quality-based preference supplement. The quality-based preference supplement can capture constraints on quality variation.

[0041] The video change analysis may analyze the change of the quality level and may also analyze the change of the profile level. The change of the quality level may be analyzed in 406 to 408 and the change of the profile level may be analyzed in 410 to 416.

[0042] At 406, the adaptive bitrate system 108 generates a reward function based on the adaptive bitrate function and the quality-based preference supplement. The reward function may represent an ABR function with an adjustment based on the quality-based preference supplement. In some embodiments, the adaptive bitrate algorithm may be based on the bandwidth and the bitrate of the corresponding profile. The adaptive bitrate function f(xn) may be calculated as follows:

[0043] f 基于带宽 (x n )=b n - bandwidth, with b n <Bandwidth conditions,

[0044] where b n Is the selected profile x n The bit rate is the bit rate of , and bandwidth is the estimated available bandwidth. f(x n ) is considered a better choice, which means that the adaptive bitrate algorithm selects a bitrate with less than the estimated available bandwidth (b n ). Another adaptive bitrate algorithm may take other characteristics into account, such as rebuffering. The f(x) function may be:

[0045] f MPC (x n )=b n -α*r n -β*|b n -b n-1 |,

[0046] where b n Is the selected profile x n The bit rate, b n-1 is the previously selected profile x n-1 The bit rate, and r n Yes Select Profile x nThe estimated rebuffering time at . α and β are hyperparameters used to balance preferences for different perspectives on rebuffering and bandwidth. The goal of this adaptive bitrate algorithm is to select a profile with the highest bitrate, but one that does not cause excessive rebuffering and avoids frequent bitrate changes.

[0047] The quality-based preference complement can be expressed as the following function r(x n ) in s(x n , x n-1 ):

[0048] r(x n )=s(x n , x n-1 )+f(x n ).

[0049] Quality-based preference supplements s(x n , x n-1 ) can be calculated as follows:

[0050] s(x n , x n-1 )=c*log(n*(delta(x n , x n-1 )+1)),

[0051] in

[0052] Here, the function q(x n ) is to select profile x n The quality measure when q(x n-1 ) is the previously selected profile x n-1 The term "c" is a hyperparameter that can be used to scale up or down the output of the log function. The quality metric analyzed can be different. In some embodiments, Video Multi-Method Assessment Fusion (VMAF) can be used as the quality metric. However, other metrics such as Peak Signal-to-Noise Ratio (PSNR) or other metrics can be used. Here, delta(x n , x n-1 ) means the rate of change of perceived quality when selecting the next video profile. n , x n-1 ) may be log(x+1). When the quality level is to be reduced, the quality-based preference supplementation may add a negative preference. The greater the quality reduction, the less preference for the change added by the quality-based preference supplementation. Thus, as the change from the most recently requested quality level increases, the quality-based preference supplementation may make it less likely that a lower quality level will be selected. An example of quality-based preference supplementation would be in Figure 5 In addition, the generation of the reward function will be described in the following Fig. 6A and Figure 6B are discussed in

[0053] At 408, the adaptive bitrate system 108 selects a profile based on a reward function. For example, the profile with the highest associated reward is selected. In some examples, Profile #3 may be selected. Then processing proceeds to the profile level change decision.

[0054] The profile level change decision can analyze a change in resolution or a change in profile level. Although the analysis is described continuously, the analysis can be determined in different ways, such as first performing a profile level span analysis and second performing a resolution span analysis. Additionally, the profile level span analysis or the resolution span analysis can be used independently. At 410, the adaptive bitrate system 108 determines whether the resolution span is less than a first threshold m (e.g., resolution span < m). Although the condition of less than is described, the condition can be any condition that is satisfied, such as the condition can be less than or equal to the threshold. The resolution span can be the change in resolution that may be caused by a previously selected profile when the current profile is selected. If the resolution span is less than the threshold m, the profile meets the resolution span requirement. The resolution span can be based on resolution levels. For example, there can be five levels of resolution, which can be 640x360, 854x480, 1280x720, 1920x1080, and 2560x1440. In this case, the resolution of 1280x720 is two resolution levels away from the resolution of 2560x1440, the resolution of 640x360 is four resolution levels away from the resolution of 2560x1440, and so on. Other methods can be used to determine the resolution span, such as using the difference in the number of pixels (e.g., 2560x1440 - 640x360).

[0055] At 412, if the resolution span meets the threshold m (e.g., greater than or equal to m), the adaptive bitrate system 108 changes the profile. For example, the adaptive bitrate system 108 can select a profile that meets the resolution span requirement. Then, after changing the profile, or if the resolution span meets the threshold m (e.g., less than m), at 414, the adaptive bitrate system 108 determines whether the profile step span is less than a threshold n. Although the condition of less than is described, the condition can be any condition that is satisfied, such as the condition can be less than or equal to the threshold. The profile level span can be determined based on the previous profile level and the selected profile level. The previous profile can be from another profile step different from the current profile step of the segment to be requested. The previous profile can be inserted into the current profile step based on the resolution and bitrate of the previous profile and the resolution and bitrate of the current profile step. Using Figure 2CAs an example, if the previously selected profile may have a resolution of 1280x720 and 3000 kbps, the adaptive bitrate system 108 may insert the previous profile into the current profile ladder 216 based on its resolution and bitrate. In some cases, the position may be between profile #1 (854x480 and 1499532 bps) and profile #2 (1280x720 and 3609421 bps) because the resolution is higher than profile #1, but the bitrate is lower than profile #2. Alternatively, the profiles in the current profile ladder may be selected based on the previous profile ladder, such as a profile that includes a resolution or bitrate that is most similar to the previous profile. The profile ladder may have profiles that may include the same resolution, such as there are two profile levels with a resolution of 1280x720 and a resolution of 1920x1080. The profile level span may not be based on resolution, but on profile level. If the previous profile was profile #6, and the selected profile is #2, then the profile level span is 4 (e.g., 6-2=4)

[0056] If the profile step span is not less than the threshold value n, then at 416, the adaptive bitrate system 108 may change the profile level. For example, as an example, the adaptive bitrate system 108 may change the profile level from profile level #3 to profile level #4. This makes the profile level span two profile levels. After changing the profile, the process proceeds to 418 to output a profile decision. If the profile step span is less than the threshold value n, then at 418, the adaptive bitrate system 108 outputs a profile decision. The profile decision may be profile #4, which may be Figure 4 The profile selected at 408, 412 or 416 in.

[0057] Quality level decision

[0058] The following will now discuss in more detail the Figure 4 The quality level decisions discussed at 406 and 408 in . Figure 5 A graph 500 is depicted of quality-based preference supplementation according to some embodiments. The values ​​on the X-axis are the change in profile level compared to the most recently selected profile. The Y-axis is the change in quality.

[0059] Line 502 may represent quality-based preference supplementation. When the video quality may be reduced, a negative preference may be added using quality-based preference supplementation. The more the quality is reduced, the smaller the preference is given. For example, at 504, when the quality level has not changed, the value of the quality-based preference supplementation is zero. If the profile quality level is increased by two levels, the value of the quality-based preference supplementation may be 0.5, as shown at 506. For example, if the previous profile is #3 and the selected profile is profile #5, the difference is two levels. The x-axis is the profile change in level, and a value of 2 is found on the positive side of the x-axis. The corresponding value of the quality-based preference supplementation is approximately 0.5. However, if the quality level is reduced by -1, the quality-based preference supplementation value may be -2, as shown at 508. For example, if the previous profile is #5 and the selected profile is profile #4, the difference is a negative one level (-1). A value of -1 is found on the negative side of the x-axis. The corresponding value of the quality-based preference supplementation is approximately -2. When the profile level change exceeds a -1 level change, the quality-based preference supplement becomes even more negative.

[0060] An example will be shown below to demonstrate the calculation of a reward function for selecting a profile using quality-based preference supplementation. Fig. 6A The adaptive bit rate function f(x) according to some embodiments is depicted n ). The X-axis of the graph 600 may be the profile level, and the Y-axis may represent the value of the reward assigned to the corresponding profile. The adaptive bitrate function f(x n ) may represent the reward associated with a profile without taking into account the quality-based preference supplement. Line 602 represents the adaptive bitrate function and assigns values ​​to the corresponding profile levels shown on the x-axis. In this profile ladder, there are a total of six profiles (e.g., profile #0 to profile #5). The most recently downloaded profile is represented as x at 606. n-1 Each point on line 602 is associated with a reward value calculated by the adaptive bitrate function for each profile level. According to the adaptive bitrate algorithm, a higher value represents a better choice of profile level. In this example, profile level #3 has the highest reward value, as shown at 604.

[0061] As described above, the original adaptive bitrate algorithm may not have considered quality variations and would have selected profile # 3. However, the advantages of considering quality variations will be shown below. Figure 6B An example of a graph 608 of applying quality changes to an adaptive bitrate function according to some embodiments is depicted. Line 602 shows the original adaptive bitrate function and the previously selected profile is shown at 606. Figure 5The quality-based preference supplement of line 502 (not shown) is used to adjust line 602, such as by subtracting line 502 (not shown) from line 602. Line 610 then represents a function r(x) based on the original adaptive bitrate function and the quality-based preference supplement. n ). At 612, profile #4 now has the highest reward value, rather than profile #3. There are other differences as well, such as profile #5 having a higher reward compared to the original adaptive bitrate function. In addition, the reward values ​​for profiles 0, 1, and 2 have been significantly reduced because the quality-based preference addition adds a negative preference because the profile level has reduced quality more than the previous profile.

[0062] The preference supplement based on quality has adjusted the reward value and profile #4 with the highest reward value is selected, which is closer in resolution to the most recently selected profile level. This can avoid drastic quality drops while also taking into account parameters of the original adaptive bitrate algorithm, such as network conditions or buffer information. Thus, an improved profile selection taking into account quality changes is performed.

[0063] In some examples, when downloading segment x using an adaptive bitrate function n The reward values ​​for selecting different profiles can be as follows:

[0064] <![CDATA[x n ]]> 0 1 2 3 4 5 <![CDATA[Reward(f(x n ))]]> 1.2 2.3 3.0 3.8 2.9 2.5

[0065] The profile selected by function f(x) is profile #3. The previously downloaded segment x n-1 It was downloaded using profile #4. Based on the quality change, s(x n , x n-1 ) function can have the following values:

[0066] <![CDATA[s(x n ,x n-1 )]]> 0 1 2 3 4 5 Preference -3.1 -2.2 -1.9 -1.2 0 0.2

[0067] Therefore, the final reward r(x n )=s(x n , x n-1 )+f(x n ) can be calculated as follows:

[0068] <![CDATA[r(x n )]]> 0 1 2 3 4 5 Final Reward -1.9 0.1 1.1 2.6 2.9 2.7

[0069] After considering the quality-based preference supplement, the final profile selection will be Profile #4 (with the highest reward). Although the same profile is selected in this example, the quality-based preference supplement can make different changes to the profile. For example, the quality-based preference supplement can change the profile from Profile #1 to Profile #3.

[0070] In another example, a table of profiles is used to illustrate resolution span decisions. Figure 7 A profile ladder 700 including profiles according to some embodiments is depicted. Column 701 shows the profile levels of profile #0 to profile #6, column 702 is the resolution, and column 704 is the bit rate. Figure 4 The resolution span is calculated based on the selected profile selected at 408 in FIG. 1 and the quality of the most recently selected previous profile for the most recent segment. The quality of the most recent segment's previous profile #5 is shown at 708 as a resolution of 1920x1080. Note that a different profile ladder could have been used to request the previous segment, such as Figure 2A You can select an equivalent profile level for a previous profile in the current profile ladder.

[0071] If the selected profile is 854x480, the resolution span is two resolution levels. That is, the profile ladder 700 includes five levels of resolution of 640x360, 854x480, 1280x720, 1920x1080, and 2560x1440. The resolution of 854x480 is two resolution levels away from the previously selected resolution of 1920x1080. If the resolution span needs to be less than a threshold, a value of m=2 may result in a resolution span that is not less than the threshold m. In this case, the adaptive bitrate system 108 changes the profile. For example, the selected profile is a profile level that is one level away from the resolution of 1920x1080, which is the resolution of 1280x720 of profile #2 at 706.

[0072] Profile Level Decisions

[0073] The number of profile levels for which the profile level decision analysis was changed. Figure 7 , each row of the profile ladder 700 can be a different profile level. As can be seen, some resolutions of the profiles can include the same resolution, such as profile levels #2 and #3 have a 1280x720 resolution, and profile levels #4 and #5 have the same 1920x1080 resolution. If the previous profile level is considered to be profile #5 at 708, and the selected profile is profile #2 at 706, then the profile level span is 3, (2961kbps->2371kbps->1781kbps->1446kbps).

[0074] If the threshold is less than three profile levels, the profile level span may not be satisfied because the profile level span is equal to 3. Therefore, the adaptive bitrate system 108 can change the profile level. For example, at 710, the adaptive bitrate system 108 can change the profile level to profile #3 with a resolution of 1280x720 and 1781kbps. This makes the profile level span two profile levels. Note that the quality remains the same, but the bit rate increases. The change in profile level attempts to ensure the smoothness of the change in bit rate when changing the profile. For example, a higher bit rate may mean higher quality. For example, a higher bit rate can allow other quality characteristics to increase, such as a higher bit rate profile at the same resolution can allow a change from 30 frames per second (fps) to 60fps, or a change from standard definition (SDR) to high definition (HDR), which results in a higher quality user experience.

[0075] The above is useful when changing the time period over which the profile steps may change can limit drastic quality changes. In addition, the use of quality changes can change the profile selected for at least one segment compared to the profile that may have been selected from the adaptive bitrate function. However, after making this change, the next segment may change the profile level of the next segment again. This may gradually reduce the quality of the video segments being requested in succession until the desired bitrate is achieved based on the current network conditions. By limiting the number of video segments that are changed based on quality changes, the opportunity for rebuffering can be limited. However, the viewer experience is greatly improved.

[0076] in conclusion

[0077] Therefore, the adaptive bitrate algorithm can be improved by taking into account quality changes. Quality changes can analyze quality level changes and profile level changes. Taking these two changes into account can improve the playback experience by providing a smoother playback experience that may not drastically change the quality level or profile level.

[0078] system

[0079] The features and aspects disclosed herein may be implemented in conjunction with a video streaming system 800 that communicates with a plurality of client devices via one or more communication networks, such as Figure 8 . Aspects of the video streaming system 800 are described merely to provide examples of applications for implementing distribution and delivery of content prepared in accordance with the present disclosure. It should be understood that the present technology is not limited to streaming video applications and may be applicable to other applications and delivery mechanisms.

[0080] In one embodiment, a media program provider may include a library of media programs. For example, media programs may be aggregated and provided through a site (e.g., a website), an application, or a browser. A user may access the site or application of a media program provider and request media programs. A user may be limited to requesting only media programs provided by a media program provider.

[0081] In system 800, video data may be obtained from one or more sources (e.g., from video source 810) for use as input to video content server 802. The input video data may include raw or edited frame-based video data in any suitable digital format (e.g., Moving Picture Experts Group (MPEG)-1, MPEG-2, MPEG-4, VC-1, H.264 / Advanced Video Coding (AVC), High Efficiency Video Coding (HEVC), or other formats). In an alternative, the video may be provided in a non-digital format and converted to a digital format using a scanner or transcoder. The input video data may include various types of video clips or programs, such as television series, movies, and other content produced as primary content of interest to consumers. The video data may also include audio or may use only audio.

[0082] The video streaming system 800 may include one or more computer servers or modules 802, 804, and 807 distributed on one or more computers. Each server 802, 804, 807 may include or may be operably coupled to one or more data repositories 809, such as databases, indexes, files, or other data structures. The video content server 802 may access a data repository (not shown) of various video clips. The video content server 802 may provide video clips as directed by a user interface controller communicating with a client device. As used herein, a video clip refers to a determined portion of frame-based video data, such as may be used in a streaming video session to watch a television series, a movie, a recorded live performance, or other video content.

[0083] In some embodiments, the video ad server 804 may access a data repository of relatively short videos (e.g., 10, 30, or 60 second video ads) that are configured as ads for a particular advertiser or message. The ads may be provided for the advertiser in exchange for some type of payment, or may include a promotional message, a public service message, or some other information for the system 800. The video ad server 804 may provide the video ad clips as directed by a user interface controller (not shown).

[0084] The video streaming system 800 may also include an integration and streaming component 807 that integrates video content and video advertisements into the streaming video clips. For example, the streaming component 807 may be a content server or a streaming media server. A controller (not shown) may determine the selection or configuration of advertisements in the streaming video based on any suitable algorithm or process. The video streaming system 800 may include Figure 8 Other modules or units not depicted in the diagram, such as management servers, commerce servers, network infrastructure, advertisement selection engine, etc.

[0085] The video streaming system 800 can be connected to a data communication network 812. The data communication network 812 can include a local area network (LAN), a wide area network (WAN) (e.g., the Internet), a telephone network, a wireless network 814 (e.g., a wireless cellular telecommunication network (WCS)), or some combination of these or similar networks.

[0086] One or more client devices 820 can communicate with the video streaming system 800 via a data communications network 812, a wireless network 814, or another network. Such client devices may include, for example, one or more laptop computers 820-1, desktop computers 820-2, "smart" mobile phones 820-3, tablet devices 820-4, network-enabled televisions 820-5, or a combination thereof, via a router 818 for a LAN, via a base station 817 for a wireless network 814, or via some other connection. In operation, such client devices 820 can send and receive data or instructions to the system 800 in response to user input or other input received from a user input device. In response, the system 800 can provide video clips and metadata from a data repository 809 to the client device 820 in response to the selection of a media program. The client device 820 can output video content from the streaming video clips in a media player using a display screen, projector, or other video output device, and receive user input for interacting with the video content.

[0087] Various methods (e.g., streaming) can be used to realize the distribution of audio-video data from streaming component 807 to remote client devices through a combination of a computer network, a telecommunication network, and such networks. In streaming, a content server continuously streams audio-video data to a media player component that operates at least partially on a client device, and the media player component can play audio-video data concurrently with receiving streaming data from the server. Although streaming has been discussed, other methods of delivery can be used. The media player component can initiate the playback of video data immediately after receiving the initial portion of data from a content provider. Traditional streaming technology uses a single provider that delivers data streams to a group of end users. High bandwidth and processing power may be required to deliver a single stream to a large number of viewers, and the required bandwidth of the provider may increase as the number of end users increases.

[0088] Streaming media can be delivered on demand or in real time. Streaming enables playback at any point in the file immediately. The end user can skip the media file to start playback or change playback to any point in the media file. Therefore, the end user does not need to wait for the file to be downloaded step by step. Usually, streaming media is delivered from several dedicated servers with high bandwidth capabilities via dedicated equipment, which accepts the request for the video file and has information about the format, bandwidth and structure of these files, and delivers the data volume just needed for playing the video at the rate required for playing the video. The streaming media server can also consider the transmission bandwidth and the ability of the media player on the destination client. Streaming component 807 can use control messages and data messages to communicate with client device 820 to adapt to the changing network conditions when playing the video. These control messages can include commands for enabling control functions, such as fast forward, fast rewind, pause or seek a specific part of the file at the client.

[0089] Because the streaming component 807 transmits video data only as needed and at the desired rate, precise control over the number of streams provided can be maintained. Viewers would not be able to watch high data rate video over a lower data rate transmission medium. However, the streaming media server (1) provides users with random access to video files, (2) allows monitoring of who is watching what video programs and how long they are being watched, (3) uses transmission bandwidth more efficiently because only the amount of data required to support the viewing experience is transmitted, and (4) the video files are not stored on the viewer's computer but are discarded by the media player, thereby allowing more control over the content.

[0090] Streaming component 807 can use TCP-based protocols, such as hypertext transfer protocol (HTTP) and real-time messaging protocol (RTMP). Streaming component 807 can also deliver live network broadcasts and can perform multicasting, which allows more than one client to tune into a single stream, thereby saving bandwidth. Streaming media player can provide random access to any point in the media program without relying on buffering the entire video. Instead, this is achieved using control messages transmitted from the media player to the streaming media server. Other protocols for streaming are HTTP live streaming (HLS) or dynamic adaptive streaming (DASH) by HTTP. HLS and DASH protocols deliver video by HTTP via a playlist of small segments, which are usually available from one or more content delivery networks (CDNs) with various bit rates. This allows media players to switch both bit rates and content sources segment by segment. Switching helps to compensate for network bandwidth changes and infrastructure failures that may occur during the playback of the video.

[0091] Delivery of video content by streaming can be implemented under various models. In one model, users pay for viewing of video programs, for example, paying for access to a library of media programs or a portion of a limited media program, or using a pay-per-view service. In another model, which was widely adopted soon after the founding of broadcast television, sponsors pay for the presentation of media programs in exchange for the right to present advertisements during or near the presentation of the program. In some models, advertisements are inserted at predetermined times in the video program, which may be referred to as "ad slots" or "ad breaks." For streaming video, a media player may be configured so that a client device cannot play a video without also playing a scheduled advertisement during a specified ad slot.

[0092] Reference Fig. 9 , illustrates a schematic diagram of an apparatus 900 for viewing video content and advertisements. In a selected embodiment, the apparatus 900 may include a processor (CPU) 902 operably coupled to a processor memory 904, which holds binary-coded functional modules for execution by the processor 902. Such functional modules may include an operating system 906 for handling system functions such as input / output and memory access, a browser 908 for displaying web pages, and a media player 910 for playing videos. These modules may also include an adaptive bit rate system 108. The memory 904 may hold Fig. 9 Additional modules not shown, such as modules for performing other operations described elsewhere in this document.

[0093] The bus 914 or other communication components may support the communication of information within the device 900. The processor 902 may be a specialized or dedicated microprocessor that is configured or operable to perform specific tasks by executing machine-readable software code that defines the specific tasks according to the features and aspects disclosed herein. A processor memory 904 (e.g., a random access memory (RAM) or other dynamic storage device) may be connected to the bus 914 or directly to the processor 902 and store instructions and information to be executed by the processor 902. The memory 904 may also store temporary variables or other intermediate information during the execution of such instructions.

[0094] The computer-readable medium in the storage device 924 can be connected to the bus 914 and store static information and instructions for the processor 902; for example, the storage device (CRM) 924 can store modules for the operating system 906, the browser 908, and the media player 910 when the apparatus 900 is powered off, and the modules can be loaded from the storage device 924 into the processor memory 904 when the apparatus 900 is powered on. The storage device 924 may include a non-transitory computer-readable storage medium that retains information, instructions, or some combination thereof (e.g., instructions that, when executed by the processor 902, cause the apparatus 900 to be configured or operable to perform one or more operations of the methods described herein).

[0095] A network communication (communication) interface 916 may also be connected to the bus 914. The network communication interface 916 may optionally provide or support bidirectional data communication between the apparatus 900 and one or more external devices (e.g., the streaming system 800) via a router / modem 926 and a wired or wireless connection 925. Alternatively or additionally, the apparatus 900 may include a transceiver 918 connected to an antenna 929, through which the apparatus 900 may communicate wirelessly with a base station for a wireless communication system or with a router / modem 926. In an alternative, the apparatus 900 may communicate with the video streaming system 800 via a local area network, a virtual private network, or other network. In another alternative, the apparatus 900 may be incorporated as a module or component of the system 800 and communicate with other components via the bus 914 or in some other form.

[0096] The device 900 may be connected (e.g., via the bus 914 and the graphics processing unit 920) to a display unit 928. The display 928 may include any suitable configuration for displaying information to an operator of the device 900. For example, the display 928 may include or utilize a liquid crystal display (LCD), a touch screen LCD (e.g., a capacitive display), a light emitting diode (LED) display, a projector, or other display device to present information to a user of the device 900 in a visual display.

[0097] One or more input devices 930 (e.g., an alphanumeric keyboard, a microphone, a keypad, a remote control, a game controller, a camera, or a camera array) can be connected to the bus 914 via the user input port 922 to convey information and commands to the device 900. In selected embodiments, the input device 930 can provide or support control of the positioning of the cursor. Such cursor control devices (also referred to as pointing devices) can be configured as a mouse, a trackball, a trackpad, a touch screen, cursor direction keys, or other devices for receiving or tracking physical movement and converting the movement into electrical signals indicating cursor movement. The cursor control device can be incorporated into the display unit 928, for example using a touch-sensitive screen. The cursor control device can convey direction information and command selections to the processor 902 and control the movement of the cursor on the display 928. The cursor control device can have two or more degrees of freedom, such as allowing the device to specify a cursor position in a plane or three-dimensional space.

[0098] Some embodiments may be implemented in a non-transitory computer-readable storage medium for use by or in conjunction with an instruction execution system, device, system, or machine. The computer-readable storage medium contains instructions for controlling a computer system to perform the methods described in some embodiments. The computer system may include one or more computing devices. When executed by one or more computer processors, the instructions may be configured or operable to perform the operations described in some embodiments.

[0099] As used in the description herein and throughout the claims that follow, "a," "an," and "the" include plural references unless the context clearly dictates otherwise. Further, as used in the description herein and throughout the claims that follow, the meaning of "in" includes "in" and "on," unless the context clearly dictates otherwise.

[0100] The above description illustrates various embodiments and examples of how aspects of some embodiments may be implemented. The above examples and embodiments should not be considered the only embodiments, and are presented to illustrate the flexibility and advantages of some embodiments as defined by the appended claims. Based on the above disclosure and the appended claims, other arrangements, embodiments, implementations and equivalents may be employed without departing from the scope of the present disclosure as defined by the claims.

Claims

1. A method comprising: generating, by a computing device, a reward function for profiles in a profile ladder for a current segment of the adaptive bitrate function based on bandwidth and content, wherein the reward function indicates that a first profile should be selected for the current segment; performing, by the computing device, a quality change analysis to select a second profile from the profile ladder, wherein the quality change analysis analyzes quality changes of profiles in the profile ladder and a previously selected profile from a previous segment; as well as A second profile is requested, by the computing device, from the profile ladder for the current segment.

2. The method according to claim 1, wherein: The profile ladder comprises a first profile ladder, and The previous segment is associated with the second profile ladder. 3 . The method of claim 2 , wherein the second profile ladder comprises at least one profile having a different bit rate or resolution than the bit rates or resolutions of the profiles in the first profile ladder. The method of claim 2 , wherein the second profile ladder comprises a different number of profiles than the first profile ladder.

5. The method according to claim 2, wherein: The third profile in the first profile ladder is associated with a bandwidth, and A second profile in the second profile ladder is associated with a bandwidth, and The third profile is associated with a different quality characteristic value than the second profile. The method of claim 5 , wherein the quality characteristic value is resolution.

7. The method of claim 1 , wherein the reward function comprises a first reward function, and wherein performing the quality analysis comprises: generating a quality supplement based on a quality change of a profile according to a previously selected profile and a profile ladder from a previous segment; as well as A second reward function based on the first reward function and the quality supplement is generated.

8. The method according to claim 7, wherein: A first reward function assigns a first value to a profile in the profile ladder, and The quality supplement adjusts one or more of the first values ​​of the first reward function to generate second values ​​of the second reward function. The method of claim 8 , wherein the second value is used to select the second profile.

10. The method according to claim 9, wherein: The second value of the highest value is used to select the second profile, and The second profile is different than the highest value profile using the first value.

11. The method according to claim 7, wherein: Quality supplementation applies negative preference to profiles where the quality is reduced compared to the previously selected profile, and The quality metric is used to measure the quality degradation between a profile and a previously selected profile.

12. The method of claim 1, wherein performing quality analysis comprises: comparing the first profile to a previously selected profile; as well as A second profile is selected based on the comparison. The method of claim 12 , wherein the second profile is selected when a difference in profile level from the first profile does not satisfy a threshold. The method of claim 12 , wherein the second profile is selected when a difference in quality characteristics from the first profile does not satisfy a threshold.

15. The method of claim 12, wherein: When the difference from the quality characteristic of the first profile does not satisfy a first threshold, selecting a second profile, and When the difference in profile level from the first profile does not satisfy a second threshold, the second profile is selected.

16. A non-transitory computer-readable storage medium having stored thereon computer-executable instructions which, when executed by a computing device, render the computing device operable to: generating a reward function for profiles in a profile ladder for a current segment based on bandwidth and content for an adaptive bitrate function, wherein the reward function indicates that a first profile should be selected for the current segment; performing a quality change analysis to select a second profile from the profile ladder, wherein the quality change analysis analyzes quality changes of profiles in the profile ladder and a previously selected profile from a previous segment; as well as A second profile is requested for the current segment from the profile ladder.

17. A method comprising: generating, by the computing device, a first reward function for a profile in a profile ladder for a current segment of the adaptive bitrate function based on bandwidth and content, wherein the first reward function indicates that a first profile should be selected for the current segment; generating, by a computing device, a quality supplement based on a quality change of a profile according to a previously selected profile and a profile ladder from a previous segment; generating, by the computing device, a second reward function based on the first reward function and the quality supplement, wherein the second reward function indicates that a second profile should be selected for the current segment; comparing, by a computing device, a first span associated with a quality characteristic based on a previously selected profile and a second profile; When the first span does not satisfy the first threshold, selecting, by the computing device, a third profile that satisfies the first threshold; comparing, by the computing device, a second span associated with a profile level in the profile ladder based on the previously selected profile and the second profile; as well as When the second span does not satisfy the second threshold, a fourth profile satisfying the first threshold is selected by the computing device.

18. The method according to claim 17, further comprising: One of a second profile, a third profile, and a fourth profile is requested by the computing device from the profile ladder for the current segment.

19. The method according to claim 17, further comprising: When the first span satisfies the first threshold, not selecting the third profile; as well as When the second span satisfies the second threshold, the fourth profile is not selected.

20. The method of claim 19, wherein: When the first span satisfies the first threshold and the second span satisfies the second threshold, the second profile is selected.

Images