Packager for Segmenter Liquidity

Abstract

To provide methods, computer program products and systems to improve media playback.SOLUTION: In a system for delivering client-specific playlists and processed variant streams to client devices, a method comprises: receiving a variant stream; identifying respective maximum segment durations for a plurality of different types of client devices that will play media content contained in the variant stream; generating, using the variant stream, a playlist for each of the different types of client devices, where the playlists each contain different maximum segment durations; and delivering the respective playlists to at least one of the different types of client devices via a distribution network.SELECTED DRAWING: Figure 1

Description

background

[0001] Adaptive bitrate streaming (ABR) is a computer network streaming technique that can provide continuous, uninterrupted media playback to a client device. ABR involves measuring the client device's network bandwidth and data throughput in real time and adjusting the streaming quality delivered to the client device accordingly.

[0002] Streaming quality is reflected in the size of the file containing the media content. A high-quality stream represents a high streaming bitrate and a relatively large file size. Therefore, a high-quality stream requires more network bandwidth and greater data throughput from the client device to ensure continuous, uninterrupted media playback on the client device. Similarly, a low-quality stream represents a low streaming bitrate and a relatively small file size. Therefore, a low-quality stream requires less network bandwidth and data throughput from the client device to ensure continuous, uninterrupted media playback on the client device. [Brief explanation of the drawings]

[0003] So that the above aspects may be achieved and understood in detail, a more particular description of the embodiments briefly outlined hereinabove may be had by reference to the accompanying drawings, in which it should be noted, however, that the accompanying drawings illustrate exemplary embodiments and, therefore, should not be construed as limiting, as other equally effective embodiments are possible. [Figure 1] 1 illustrates a system for delivering client-specific playlists and processed variant streams to client devices, according to one embodiment. [Figure 2]1 illustrates an encoded variant stream for generic media content according to one embodiment. [Figure 3] 1 shows a flowchart for operating a packager, according to one embodiment. [Figure 4] 10 illustrates a processed variant stream with a corresponding client-specific playlist according to one embodiment. [Figure 5] 10 illustrates segment durations and segment boundaries for different client-specific playlists for the same variant stream, according to one embodiment. [Figure 6] 10 illustrates changes in segment duration groupings at various times during media playback, according to one embodiment. [Figure 7] 1 illustrates media playback on a client device according to one embodiment. Detailed Description

[0004] To facilitate a detailed understanding of the present disclosure, embodiments of the present invention disclosed herein may refer to HTTP Live Streaming (HLS) as a computer network streaming protocol. Client device platforms or operating systems (OSs) may be referred to as OS-A, OS-B, OS-C, etc. Examples of OSs include Android (trademark) and iOS (trademark). Client device media players may be referred to as Media Player A, Media Player B, Media Player C, etc. An example of a client device media player is a JavaScript-based media player. However, the disclosed embodiments should not be construed as limited to any of the aforementioned network protocols, platforms or OSs, or media players.

[0005] HLS is a computer network streaming protocol that can deliver media content via a master playlist that references multiple variant streams, each representing a different streaming quality for the same media content. Each variant stream contains a media playlist that references a set of contiguous maximal duration segments of the media content that are played sequentially on a client device.

[0006] Segments are identified in a media playlist by one or more URIs (Uniform Resource Identifiers) and optional byte ranges. A URI is a string that identifies a resource on the Internet. Each segment in a media playlist begins and ends on a segment boundary that is typically synchronized across multiple variant streams. ABR is performed by adjusting the streaming quality at these aligned segment boundaries. These adjustments involve switching from a group of segments in one variant stream to a contiguous group of segments in another variant stream.

[0007] Optimizing media playback for HLS can be hindered by differences in client devices and client media players. When media playback begins, many client devices download a fixed number of segments, regardless of the duration of the segments. Some client devices buffer for a fixed period of time, regardless of the number of segments downloaded. Therefore, different client devices (e.g., clients with different types of OS or media players) may have different or conflicting performance optimizations for media playback.

[0008] Furthermore, many media players initiate media playback at medium streaming quality rather than high streaming quality to minimize skips in media playback due to buffering caused by an inability to maintain the initial high streaming quality. Starting media playback at medium streaming quality has a disadvantage when a higher streaming quality is supported by the client device's available bandwidth and data throughput. In such cases, media playback performance is not optimized because time is spent at medium streaming quality while the client is downloading a certain number of segments or a segment for a certain period of time. This contrasts with shifting media playback to a higher streaming quality as soon as the client's bandwidth and data throughput allow.

[0009] For example, assume that client device A is programmed to download five segments with a segment duration of eight seconds before switching to another streaming quality, and assume that client device A's available bandwidth and data throughput support high streaming quality at all points during media playback. If client device A starts streaming at medium streaming quality and wishes to transition to high streaming quality, client device A must stream for 40 seconds at medium streaming quality before transitioning to high streaming quality. Streaming for 40 seconds at medium streaming quality is not optimized media playback because client device A could have streamed for 40 seconds at high streaming quality.

[0010] Solutions to the aforementioned problems and further optimization of media playback can be achieved through the use of segment fluidity, which is a technique for grouping segments during media creation or processing so that segments of optimal duration for each particular client device platform can be presented to each client device as needed.

[0011] In embodiments herein, a packager generates a client-specific playlist that references one or more URIs with segment durations optimized for a particular type of client device. When a client device receives the playlist from a server, the server can identify the type of client device making the request and deliver the playlist optimized for the particular client device to the client device. The client device uses the optimized client-specific playlist to fetch or request variable groups of segments from the server. Variable grouping of segments allows for real-time optimization of segment durations, improving media playback performance. In this way, segment fluidity improves media playback performance on client devices implementing different operating systems and media players.

[0012] In one embodiment, variable grouping of segments is used to provide shorter segment durations at the start of media playback and at seek points, while providing longer segment durations during portions of media playback where the end user is less likely to initiate start or seek operations.

[0013] 1 illustrates a system for delivering client-specific playlists and processed variant streams to client devices according to one embodiment. In this embodiment, an audiovisual (AV) input 110 generates a multimedia stream that is delivered to a transcoder 130. The transcoder 130 converts the multimedia stream into one or more encoded variant streams, each containing the same media content but at different bit rates. The encoded variant streams include partition boundaries at segment boundaries aligned throughout the encoded variant stream. The encoded variant streams are delivered to a packager 140.

[0014] The packager 140 comprises a data unit 142, a playlist unit 144, and a delivery unit 146. In one embodiment, the data unit 142, the playlist unit 144, and the delivery unit 146 are software modules executing on hardware (e.g., a processor and memory). The data unit 142 includes data regarding the maximum segment time length. This data provides optimal media playback performance for the operating systems 158a and media players 158b of various client devices. The maximum segment time length is optimized for each client device. For example, a first type of client device may have optimal media playback using a shorter maximum segment time length, while a second type of client device provides optimal feedback using a longer maximum segment time length.

[0015] The playlist unit 144 identifies the maximum segment duration of each client device's operating system 158a or media player 158b and creates a respective media playlist (client-specific playlist) for each of these client device's operating systems 158a or media players 158b. The encoded variant streams are processed so that maximum segment durations, or groups of maximum segment durations, can be accessed using byte ranges. A client-specific playlist is then generated for each processed variant stream 148.

[0016] The delivery unit 146 delivers the client-specific playlists to the distribution network 150, where, in one embodiment, the client-specific playlists and processed variant streams 148 are first sent to one or more servers 152. The client-specific playlists are then sent to the client devices 156. The one or more servers handle any fetches / requests for segments of the media content from the client devices. That is, using the client-specific playlists, the client devices 156 can submit requests to the distribution network 150 for segments identified in those playlists in order to play the media content.

[0017] Alternatively, the client-specific playlist and processed variant stream 148 are first sent to one or more servers 152. The client-specific playlist is then sent to a content delivery network (CDN) 154, which distributes the client-specific playlist to client devices 156 and handles client device fetches / requests for segments of media content. In yet another embodiment, the client-specific playlist and processed variant stream 148 are first sent to a CDN 154, which distributes the client-specific playlist to client devices 156 and handles client device fetches / requests for segments of media content. Using a CDN to process the fetched / requested segments can improve media playback by localizing CDN edge servers to reduce delivery times for the fetched / requested segments.

[0018] 2 illustrates coded variant streams of a common (i.e., the same) media content, according to one embodiment. In this embodiment, variant stream 1 206 and variant stream 2 204 are coded variant streams of different streaming qualities (bitrate 1 and bitrate 2). Each variant stream includes partition boundaries of potential segments that may exist after the coded variant stream is processed. Each potential segment is designated by an equal segment duration (e.g., 204d1-204dn) or by adjacent segment boundaries (e.g., 204b1 and 204b2).

[0019] Segment boundaries are aligned across the variant stream. In this embodiment, ABR variant switching occurs at segment boundaries 204b1-n and 206b1-n. At each segment boundary, a different variant stream can be delivered to the client device based on the client device's network bandwidth and data throughput. That is, a client device implementing ABR can decide to switch to a lower or higher bitrate variant stream at a segment boundary based on the client device's current network bandwidth and data throughput.

[0020] For example, if bitrate 1 is 5 Mbps and bitrate 2 is 2 Mbps, media playback 202 may begin by streaming variant stream 2, 204, which has a lower streaming quality. If the client device's available network bandwidth and data throughput allows playback of variant stream 1, 206, which has a higher streaming quality, the client device retrieves the playlist for variant stream 1 and sends a request for a segment of variant stream 1, 206. The switch from variant stream 2, 204 to variant stream 1, 206 may occur at any segment boundary (e.g., 204b1 and 206b1) aligned throughout variant stream 1, 206 and variant stream 2, 204.

[0021] Thereafter, if the client device's available network bandwidth and data throughput can no longer sustain the higher streaming quality variant stream 1, 206, the client can request a segment of the lower streaming quality variant stream 2, 204 from the server or CDN processing the request. The switch from variant stream 1, 206 to variant stream 2, 204 occurs at a subsequent segment boundary (e.g., 204bn and 206bn) that is aligned across variant stream 1, 206 and variant stream 2, 204.

[0022] A chapter is a transition point in the media content. For example, a chapter may indicate the beginning or end of an advertisement break or a natural break in a conversation between people in the media content. In one embodiment, a chapter serves as a segment boundary. Regardless of the chapter's location relative to a non-chapter segment boundary, a chapter may occur at any suitable place within the media playback 202.

[0023] FIG. 3 illustrates a flow chart of the operation of a packager, according to one embodiment. Because different client devices have different optimal maximum segment durations for optimal media playback, FIG. 3 illustrates an approach for delivering client-optimized segment durations to each client device. Furthermore, because the optimal segment duration may change during media playback, the optimization approach of FIG. 3 allows for real-time modification of the segment duration span. The packager operates to address the aforementioned issues. While FIG. 3 is described with reference to other figures in the present application, such references are for clarity and should not be construed to limit the description of FIG. 3 to the embodiments of those figures.

[0024] In block 302, the packager receives encoded variant streams from the transcoder. Each encoded variant stream contains segment boundaries of potential segments of media content. Each encoded variant stream has the same media content but different streaming qualities / bit rates. The beginning and end of each potential segment of media content are segment boundaries (e.g., 204b1 and 204b2 in FIG. 2), and these segment boundaries are aligned across all encoded variant streams. Two adjacent segment boundaries (e.g., 204b1 and 204b2) specify a segment duration (e.g., 204dn).

[0025] In block 304, the playlist unit 144 identifies client devices for delivering the client-specific playlist and processed variant stream. In one embodiment, the data unit 142, the playlist unit 144, and the delivery unit 146 are software modules executing on hardware (e.g., a processor and memory). The playlist unit 144 can collect data regarding client characteristics from the data unit 142, which is populated with predetermined data about each client device. The predetermined data can be determined by testing media players and clients for optimal performance. For example, field tests may show that for a first type of client device, using a playlist with a maximum segment time length of 4 seconds reduces video startup time, speeds up variant switching, or reduces server load, and for a second type of client device, using a playlist with a maximum segment time length of 8 seconds reduces video startup time, speeds up variant switching, or reduces server load. Thus, in this example, a maximum segment time length of 4 seconds is the predetermined data characteristic of the first client, and a maximum segment time length of 8 seconds is the predetermined data characteristic of the second client.

[0026] At block 306, the playlist unit 144 identifies the maximum segment duration in the encoded variant stream for optimal playback for each client device. The playlist unit 144 may collect data regarding the maximum segment duration for each client device from the data unit 142, which includes data regarding the maximum segment duration that provides optimal media playback performance for the various client device operating systems 158a and media players 158b.

[0027] The maximum segment duration is optimized for each client device. For example, a first client device may use OS-B, while a second client device uses OS-A. Or, a first client device may use media player A, while a second client device uses media player B. The optimal maximum segment duration for the first client may be 8 seconds, while the optimal maximum segment duration for the second client device is 4 seconds. A playlist that delivers segments with a maximum segment duration optimized for the first client will negatively impact the playback experience of the second client device, at least because a longer maximum segment duration means that the second client device has fewer opportunities to switch variant streams when performing ABR.

[0028] The above problem is solved by providing each client device with a client-specific playlist. Thus, a first client device performs optimally when it receives a client-specific playlist with a maximum segment time length of exclusively 8 seconds. Similarly, a second client device performs optimally when it receives a client-specific playlist with a maximum segment time length of exclusively 4 seconds.

[0029] At block 308, the packager generates processed variant streams by segmenting the encoded variant streams at partition boundaries included in the encoded variant streams. Segmenting the encoded variant streams may include segmenting each encoded variant stream into multiple segment files that collectively constitute the corresponding processed variant stream, or may include preparing each corresponding processed variant stream for single-file byte-range access to each maximum-duration segment or group thereof.

[0030] Each encoded variant stream contains segment boundaries of potential segments of the media content, and therefore each processed variant stream contains (realizations of) segments of the media content. The start and end of each segment in the processed variant stream is a segment boundary, and this segment boundary is aligned across the processed variant stream for each client device. The difference between the timestamps of two adjacent segment boundaries indicates the segment duration.

[0031] In block 310, the packager generates a client-specific playlist for each processed variant stream for each client device. Once the packager segments the encoded variant streams, the playlist unit 144 generates a client-specific playlist for each client device. This is shown in FIG.

[0032] 4 illustrates a variant stream with a corresponding client-specific playlist according to one embodiment. In this embodiment, three variant streams correspond to the same media content, and each variant stream includes three client-specific playlists. In this example, the client-specific playlists are for client devices with three different operating systems or media players. As noted above, embodiments herein are not limited to generating specific playlists for clients with different operating systems or media players. Packagers can generate client-specific playlists for client devices with other types of software or hardware modules.

[0033] As shown, playlists 1H, 1M, and 1L are playlists for the same type of client device, i.e., client devices using media player A. Playlist 1H 412 is a client-specific playlist for high-quality variant stream 1 410. Playlist 1M 422 is a client-specific playlist for medium-quality variant stream 2 420. Playlist 1L 432 is a client-specific playlist for low-quality variant stream 3 430. Media player A in this embodiment provides optimal media playback performance with a segment time length of 2 seconds. Therefore, the maximum segment time length for playlists 1H, 1M, and 1L is 2 seconds.

[0034] Playlists 2H, 2M, and 2L are optimized for client devices implementing OS-A. Playlist 2H, 414, is a client-specific playlist for high-quality variant stream 1, 410. Playlist 2M, 424, is a client-specific playlist for medium-quality variant stream 2, 420. Playlist 2L, 434, is a client-specific playlist for low-quality variant stream 3, 430. In this embodiment, OS-A has optimal media playback performance with a segment duration of 4 seconds. Therefore, the maximum segment duration of playlists 2H, 2M, and 2L is 4 seconds.

[0035] Playlists 3H, 3M, and 3L are made available to clients implementing OS-B. Playlist 3H 416 is a client-specific playlist for high-quality variant stream 1 410. Playlist 3M 426 is a client-specific playlist for medium-quality variant stream 2 420. Playlist 3L 436 is a client-specific playlist for low-quality variant stream 3 430. In this embodiment, OS-B provides optimal media playback performance with a segment duration of 8 seconds. Therefore, the maximum segment duration for playlists 3H, 3M, and 3L is 8 seconds.

[0036] ABR is implemented by delivering appropriate segments from high-quality, medium-quality, or low-quality variant streams of a client-specific playlist for the same client device. For example, if currently playing using segments from playlist 3M, 426, the client will download segments for OS-A from medium-quality variant stream 2, 420.

[0037] If the available network bandwidth and data throughput allow for a switch to higher quality variant stream 1, 410, segments are delivered to the client device from playlist 3H, 416, where OS-A is the client device. Alternatively, if the available network bandwidth and data throughput are not sufficient to maintain uninterrupted playback of medium quality variant stream 2, 420, segments are delivered to the client device from playlist 3L, 436, where OS-A is the client device. In this way, client devices can request client-specific playlists with optimized segment durations.

[0038] In one embodiment, a client-specific playlist is a UTF-8 text file that contains a list of one or more URIs and description tags. The URIs identify segments of media content within the processed variant stream. The description tags specify information and parameters for the client-specific playlist.

[0039] In one embodiment, each URI in the list is immediately preceded by an EXTINF tag, which indicates the segment duration of the segment corresponding to the URI. Segment groups may be specified with an EXT-X-BYTERANGE tag, which may indicate the range of URI resources that correspond to the segment group. The client-specific playlist contains an EXT-X-ENDLIST tag, which indicates that there are no more segments in the client-specific playlist.

[0040] In one embodiment, the client-specific playlist includes a byte range that specifies a maximum segment duration optimized for each client device. For example, if a client device uses OS-A, the optimal maximum segment duration may be 4 seconds. This client device receives playlist 2H, 414, representing processed variant stream 1, 410, with high streaming quality, playlist 2M, 424, representing variant stream 2, 420, with medium streaming quality, and playlist 2L, 434, representing variant stream 3, 430, with low streaming quality.

[0041] Because the client-specific playlists have segments and segment boundaries aligned across the processed variant stream, a client device can switch streaming qualities by requesting a group of segments from one of the client-specific playlists that the client device is not currently using. The server processing the request then delivers successive segments, at the next segment boundary, from the client-specific playlist that corresponds to the desired streaming quality to the client device.

[0042] Returning to Figure 3, at block 312, the packager distributes the client-specific playlist to the client via the distribution network. In one embodiment, the distribution unit 146 distributes the client-specific playlist to the distribution network 150, where the client-specific playlist and processed variant stream 148 are first sent to one or more servers 152. When a client device requests media content, the server may determine the type of client device based on user agent data included in the header of the media content request. The one or more servers then distribute the appropriate client-specific playlist to the client device 156 and handle any fetches / requests from the client device for segments of the media content.

[0043] In another embodiment, the delivery unit 146 delivers one or more master playlists to the distribution network 150. In the distribution network 150, the master playlist is first sent to one or more servers 152. The one or more servers 152 may contain multiple master playlists from multiple sources. Each master playlist contains variant streams with equal segment time lengths for all variant streams in its respective master playlist. However, the segment time lengths of the variant streams vary from master playlist to master playlist. For example, master playlist A may contain variant streams A1, A2, and A3, each with a segment time length of 4 seconds. On the other hand, master playlist B may contain variant streams B1, B2, and B3, each with a segment time length of 8 seconds. When a client device requests media content, the server delivers the master playlist containing the segment time lengths optimal for the client device to the client device. The server then handles any fetches / requests from the client device for segments of the media content.

[0044] In another embodiment, the client-specific playlist and processed variant stream 148 are first sent to one or more servers 152. The client-specific playlist is then sent to a content delivery network (CDN) 154. When a client device requests media content, the CDN may determine the type of client device based on user agent data included in the header of the media content request. The CDN then delivers the appropriate client-specific playlist to the client device 156 and handles the client device's fetch / request for segments of the media content.

[0045] In yet another embodiment, the client-specific playlist and processed variant stream 148 are first sent to the CDN 154. When a client device requests media content, the CDN determines the type of client device based on user agent data included in the header of the media content request. The CDN then delivers the appropriate client-specific playlist to the client device 156 and processes the client device's fetch / request for segments of the media content. Using a CDN to process the fetched / requested segments can improve media playback by localizing CDN edge servers to reduce delivery times for the fetched / requested segments. This embodiment differs from the previous embodiment in that this embodiment distributes the playlist to client devices solely through the use of a CDN, whereas the previous embodiment can use a combination of a server and a CDN to distribute the playlist to client devices.

[0046] Figure 5 shows segment durations and segment boundaries for different client-specific playlists for a single variant stream (e.g., variant stream 1 in Figure 4). In one embodiment, the high-quality variant stream includes three client-specific playlists 504, 506, and 508. The client-specific playlists represent the operating systems and media players of three client devices: OS-A, OS-B, and media player A.

[0047] Playlist 1H contains segments optimized for media player A. Two adjacent segment boundaries (e.g., 504b1 and 504b2) specify segment durations (e.g., 504d1). ABR may operate to switch streaming quality at segment boundaries (e.g., 504b2).

[0048] Playlist 2H contains segments optimized for OS-A. Two adjacent segment boundaries (e.g., 506b1 and 506b2) specify segment durations (e.g., 506d1). ABR may operate to switch streaming quality at segment boundaries (e.g., 506b2).

[0049] Playlist 3H contains segments optimized for OS-B. Two adjacent segment boundaries (e.g., 508b1 and 508b2) specify segment durations (e.g., 508d1). ABR may operate to switch streaming quality at segment boundaries (e.g., 508b2).

[0050] The segment durations of playlists 1H, 2H, and 3H may be different from one another. For example, playlist 3H may have a segment duration of 8 seconds, playlist 2H may have a segment duration of 4 seconds, and playlist 1H may have a segment duration of 2 seconds. Thus, in one embodiment, playlists 1H, 2H, and 3H reference segments in the same variant stream that have different segment durations between the respective playlists.

[0051] An optimized maximum segment duration for each client device improves media playback compared to, for example, receiving a fixed segment duration for all client devices. For example, a first client device running OS-B might have an optimal maximum segment duration of 8 seconds, while a second client device running OS-A might have an optimal maximum segment duration of 4 seconds. A playlist that delivers segments with a maximum segment duration optimized for the first client might negatively impact the playback experience of the second client device, at least in part because a longer maximum segment duration means the second client device has fewer opportunities to switch variant streams when performing ABR. Furthermore, a maximum segment duration that is too short unnecessarily increases the number of segment fetches / requests performed by the client device. This wastes processing power on the client device and increases the load on the server processing the fetches / requests.

[0052] Furthermore, if a client device's media player is programmed to download a certain number of segments before performing ABR, a maximum segment time length longer than the client device's optimized maximum segment time length may result in longer times spent with undesirable streaming quality. This is because segments with longer maximum segment time lengths require more time to download. For example, if a segment with an 8-second segment time length is delivered to a client device optimized for a 2-second segment time length and the client device's media player is programmed to download two segments before switching to a different streaming quality, the 8-second segment time length forces the media player to wait 16 seconds before it can perform ABR, compared to 4 seconds for a 2-second segment time length. Therefore, if a media player needs to switch to a higher-quality streaming quality after a start point or seek point, when using a segment with an 8-second segment time length, the media player will deliver 12 seconds or more of inferior streaming quality to the end user before switching to the higher-quality stream compared to when the media player uses a 2-second segment time length.

[0053] Furthermore, if the media player of the client device is programmed to download segments for a certain amount of time before performing ABR, a maximum segment time length greater than the client device's optimized maximum segment time length will result in a longer time spent with less desirable streaming quality, because the media player will necessarily exceed the required time before performing ABR.

[0054] For example, if a segment with an 8-second segment length is delivered to a client device optimized for a 2-second segment length, and the client device's media player is programmed to download a segment for 10 seconds before switching to a different streaming quality, the 8-second segment will exceed the required download time by 6 seconds because two 8-second segments are downloaded, since ABR cannot be performed between segments and downloading one 8-second segment does not satisfy the required 10-second segment download. Therefore, if the media player needs to switch to a higher quality streaming quality after a start point or seek point, when using a segment with an 8-second segment length, the media player will deliver more than 6 seconds of inferior streaming quality to the end user before switching to the higher quality stream compared to if the media player were using a 2-second segment length.

[0055] 6 illustrates the changing of segment duration groupings at various points in media playback, according to one embodiment. Variable segment duration groupings exploit segment fluidity to combine short-length segments into long-length segments in real time. Specifically, variable segment duration groupings are an optimization made possible by using client-specific playlists containing byte ranges to identify segments in the processed variant stream.

[0056] For example, Playlist 1H, 412 in Figure 4 indicates a client device with a maximum segment duration of 2 seconds. However, all segments specified in Playlist 1H, 412 are static. Media playback using Playlist 1H streams a series of 2-second segments. Variable segment duration grouping, on the other hand, allows for the construction of larger segments based on maximum duration segments to optimize during changing playback conditions.

[0057] In one embodiment, segment durations are modified by grouping one or more maximum segment durations. Adjacent segment boundaries (e.g., 608b1 and 608b2) define segments and segment durations. ABR may operate to switch streaming quality at segment boundaries (e.g., 608b3). Each segment may be identified using a byte range in a client-specific playlist. Multiple segments may be grouped together using byte ranges.

[0058] In this example, the client device achieves optimal media playback performance with a maximum segment time length of 2 seconds. The client device begins media playback 602 at medium streaming quality. The beginning of media playback, start 604, occurs at segment boundary 608b1.

[0059] At start point 604, the client device downloads multiple two-second segments, providing multiple opportunities to switch between different streaming qualities (i.e., multiple segment boundaries 608b2, 608b3). Having multiple opportunities to switch streaming qualities at start point 604 reduces the time required to achieve optimal playback performance because the streaming quality can be switched at an earlier segment boundary 608b2. Similarly, the time required to achieve optimal playback performance is also reduced for a client device media player that must download a certain number of segments before switching to a different streaming quality because the condition for switching can be satisfied earlier during media playback 602.

[0060] After downloading the initial set of segments, the client device uses byte ranges to request a group of three consecutive segments, each containing a combined 6-second segment designated by adjacent segment boundaries 608b3 and 608b4. Enabling dynamic grouping of segments balances the aforementioned optimization of media playback performance with reducing the load on the server delivering the segments, because a longer segment is easier for the server to process than multiple shorter segments.

[0061] A seek point is a specific location within the media playback 602 selected by an end user of a media player. The seek 606 begins at segment boundary 608b4. At the seek 606, the client device downloads multiple segments that are 2 seconds in length. If the client device does not have access to a cache containing the segments needed for immediate media playback, the seek acts like a starting point. Thus, the seek downloads an initial set of smaller segments and dynamically groups the smaller segments into larger segments.

[0062] 7 illustrates media playback on a client device according to one embodiment, where the client device requests multiple segments via a client-specific playlist and outputs media content to an end user, where the client device is enabled to dynamically group segments for optimal media playback.

[0063] At block 702, the client device receives one or more client-specific playlists. Each client-specific playlist references one or more segments using byte range tags. Each client-specific playlist corresponds to a different processed variant stream, which represents the same media content at different bit rates. Each client-specific playlist includes an Endlist tag, which indicates the end of the one or more client-specific playlists.

[0064] In one embodiment, to receive a client-specific playlist, the client device sends identification information to a packager, which uses that information to determine the type of client device and, therefore, the playlist that is best suited for the client device. The packager then sends the best suited playlist to the client device (more specifically, the media player on the client device). Once the playlist is identified, the client device can request the playlist from the distribution network and then begin downloading segments of media content using the playlist, as described below.

[0065] In another embodiment, the client device identifies itself to a CDN (rather than a packager) that contains playlists and variant streams. The CDN identifies a playlist with a maximum segment duration that is optimal for media playback on the client device, the client device requests the identified playlist, and the CDN delivers this playlist to the client device.

[0066] At block 704, the client device sends a fetch / request for a segment of the processed variant stream referenced by one or more client-specific playlists. When the client device requests a segment of the media content for the first time after the start point or seek point, the fetch / request is for the segment with the maximum segment time length specified in the client-specific playlist.

[0067] A client device can perform ABR by changing the streaming quality by targeting segments referenced in another client-specific playlist (i.e., one of the received client-specific playlists representing another variant stream). The client device decides to fetch / request segments from another client-specific playlist based on the client device's available network bandwidth and current data throughput.

[0068] The client device receives the fetched / requested segment of the processed variant stream at block 706. At block 708, the client device plays the media content referenced by the fetched / requested segment.

[0069] At block 710, the client device may use byte ranges to group multiple segments into a single segment group to be fetched / requested from one or more processed variant streams. The client device may use byte ranges to group together multiple contiguous segments of a maximum segment time length specified in a client-specific playlist. Grouping multiple segments into a single segment may optimize media playback by reducing the amount of processing power used by the client device compared to requesting, receiving, and processing multiple segments. Additionally, grouping multiple segments into a single segment may reduce the load on a server that is processing client device fetches / requests for media content for similar reasons.

[0070] If the last segment has been reached at block 712, then the playback of the media content referenced by the one or more client-specific playlists is terminated as per block 714. If the Endlist tag has not been reached, the client device continues streaming operations as per block 704.

[0071] In this disclosure, reference is made to various embodiments. However, it should be understood that the disclosure is not limited to the specific embodiments described. Instead, any combination of the following configurations and elements, whether related to various embodiments or not, is contemplated for implementing and practicing the teachings provided herein. Furthermore, when an element of an embodiment is described in the form of "at least one of A and B," it can be understood that an embodiment including only element A, an embodiment including only element B, or an embodiment including both elements A and B is contemplated, respectively. Furthermore, while some embodiments may achieve advantages over other possible solutions or the prior art, whether or not a particular advantage is achieved by a given embodiment does not limit the disclosure. Accordingly, the aspects, configurations, embodiments, and advantages disclosed herein are merely exemplary and should not be considered elements or limitations of the appended claims unless expressly recited in the claims. Similarly, references to the "present invention" should not be construed as generalizing the inventive subject matter disclosed herein, nor should they be considered elements or limitations of the appended claims unless expressly recited in the claims.

[0072] As will be appreciated by those skilled in the art, the embodiments described herein may be embodied as a system, a method, or a computer program product. Accordingly, the embodiments may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, all of which are collectively referred to herein as a "circuit," "module," or "system." Furthermore, the embodiments described herein may take the form of a computer program product embodied in one or more computer-readable medium(s) having computer-readable program code embodied therein.

[0073] The program code embodied on the computer readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, fiber optic cable, RF, etc., or any suitable combination thereof.

[0074] Computer program code for carrying out operations of embodiments of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and traditional procedural programming languages ​​such as the "C" programming language or similar programming languages. The program code may run entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0075] Aspects of the present disclosure are described herein with reference to flowchart diagrams or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It can be understood that each block of the flowchart diagrams or block diagrams, and combinations of blocks in the flowchart diagrams or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus forming a machine, such that the instructions, executed by the processor of the computer or other programmable data processing apparatus, create means for performing the functions / acts specified in the blocks of the flowchart diagrams or block diagrams.

[0076] These computer program instructions may also be stored on a computer-readable medium that can direct a computer, other programmable data processing apparatus, or other device to function in a particular manner, whereby the instructions stored on the computer-readable medium form an article of manufacture including instructions that implement the functions / acts specified in the flowchart or block diagram blocks.

[0077] Computer program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device and caused to execute a series of operational steps on the computer, other programmable data processing apparatus, or other device to create a computer-implemented process, whereby the instructions executing on the computer, other programmable data processing apparatus, or other device provide a process for performing the functions / operations specified in the flowchart or block diagram blocks.

[0078] The flowchart diagrams and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart diagrams or block diagrams represents a module, segment, or portion of code, which may comprise one or more executable instructions for performing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or may be executed in the reverse order or out of order, depending on the functionality involved. It should also be noted that each block in the block diagrams or flowchart diagrams, and combinations of blocks in the block diagrams or flowchart diagrams, may be implemented by a special-purpose hardware-based system that performs particular functions or operations, or by a combination of special-purpose hardware and computer instructions.

[0079] While the foregoing is directed to embodiments of the present disclosure, other and further embodiments of the present disclosure may be made without departing from the basic scope thereof, which scope is defined by the following claims.

Claims

1. identifying a maximum segment duration for each of a plurality of different types of client devices that play the media content included in the first variant stream; generating a respective playlist for each of a plurality of different types of client devices based on the first variant stream, each playlist including a different maximum segment time length; generating respective playlists for each of a plurality of client devices based on a second variant stream corresponding to the same media content as the first variant stream but having a different bit rate, each playlist including a different maximum segment time length, wherein each of the respective playlists generated from the first variant stream and the second variant stream is aligned at segment boundaries with segments of a time length specified for the platform of each particular client device; and delivering at least one of the respective playlists to at least one client device of a plurality of different types of client devices via a distribution network.

2. 10. The method of claim 1, wherein identifying a maximum segment time length for each of a plurality of different types of client devices is based on at least one of an operating system and a media player of the plurality of client devices.

3. The method of claim 1 , wherein each respective playlist references a single uniform resource indicator.

4. The method of claim 1 , further comprising inserting byte range tags into each playlist to demarcate each segment boundary and chapter in the first variant stream.

5. 10. The method of claim 1, further comprising receiving a request for grouped segments from one of a plurality of different types of client devices, the grouped segments being identified by a byte range that includes a plurality of segments of a maximum segment time length.

6. A non-transitory computer readable medium containing computer program code, the computer program code performing operations when executed by operation of one or more computer processors, including: identifying a maximum segment duration for each of a plurality of different types of client devices that play the media content included in the first variant stream; generating a respective playlist for each of a plurality of different types of client devices based on the first variant stream, each playlist including a different maximum segment time length; generating respective playlists for each of a plurality of client devices based on a second variant stream corresponding to the same media content as the first variant stream but having a different bit rate, each playlist including a different maximum segment time length, wherein each of the respective playlists generated from the first variant stream and the second variant stream is aligned at segment boundaries with segments of a time length specified for the platform of each particular client device; and delivering the respective playlist to at least one of a plurality of different types of client devices via a distribution network.

7. The computer readable program code further comprises:

7. The non-transitory computer-readable medium of claim 6, wherein identifying a maximum segment time length for each of a plurality of different types of client devices is configured to be based on at least one of an operating system and a media player of the plurality of client devices.

8. The non-transitory computer-readable medium of claim 6 , wherein each respective playlist references a single uniform resource indicator.

9. The computer readable program code further comprises:

7. The non-transitory computer-readable medium of claim 6, configured to include inserting byte range tags into respective playlists to demarcate each segment boundary and chapter in each of the first variant streams.

10. The computer readable program code further comprises:

7. The non-transitory computer-readable medium of claim 6, configured to include receiving a request for grouped segments from one of a plurality of different types of client devices, the grouped segments being identified by a byte range that includes a plurality of segments of a maximum segment time length.

11. a data unit including data indicating maximum segment time lengths for a plurality of different types of client devices; A playlist unit, identifying a maximum segment duration for each of a plurality of different types of client devices that play the media content included in the first variant stream; generating a respective playlist for each of a plurality of different types of client devices based on the first variant stream, each playlist including a different maximum segment time length; a playlist unit configured to generate, for each of a plurality of client devices, a respective playlist each including a different maximum segment time length based on a second variant stream corresponding to the same media content as the first variant stream but having a different bit rate, wherein each of the respective playlists generated from the first variant stream and the second variant stream is aligned at segment boundaries with segments of a time length specified for the platform of each particular client device; a distribution unit configured to distribute the respective playlists to at least one of a plurality of different types of client devices over a distribution network.

12. 12. The system of claim 11, wherein identifying a maximum segment time length for each of a plurality of different types of client devices is based on at least one of an operating system and a media player of the plurality of client devices.

13. 12. The system of claim 11, wherein each respective playlist references a single uniform resource indicator.

14. 12. The system of claim 11, wherein the playlist unit is configured to include inserting byte range tags into each playlist to demarcate each segment boundary and chapter in the first variant stream.

15. 12. The system of claim 11, wherein the distribution unit is configured to receive a request for grouped segments from one of a plurality of different types of client devices, the grouped segments being identified by a byte range that includes a plurality of segments of a maximum segment time length.

16. 1. A computing system comprising: a processor; a memory, the operations performed by a program the memory stores when executed by a processor, receiving a request for a first playlist from a first type of client device, the first playlist including a first maximum segment duration customized for the first type of client device; receiving a request for a second playlist from a second type of client device, the second playlist including a second maximum segment time length customized for the second type of client device, the first type of client device and the second type of client device being different, and the first and second maximum segment time lengths being different; transmitting the first and second playlists to the first and second types of client devices, respectively; and a memory in which each of the playlists is aligned on segment boundaries with segments of a specified duration for each particular client device platform.

17. The operation further comprises:

17. The computing system of claim 16, further comprising receiving a request for grouped segments from at least one of the first and second types of client devices, the grouped segments being identified by byte ranges including a plurality of segments of respective maximum segment time lengths.

Images