Method, system, and medium for delivering manifest-less streaming media content

Abstract

To provide methods, systems, and media for delivering manifestless streaming media content.SOLUTION: A method for delivering live streaming content comprises the steps of: transmitting, by a client device, a request for a plurality of segments of a content stream to a content delivery network; receiving a recent segment file from the content delivery network; detecting a mismatch between an audio segment file and a video segment file of the recent segment file; transmitting a request for the next segment; receiving the next segment file corresponding to the next segment; and loading segment files in response to detecting a mismatch between the audio segment file and the video segment file of the next segment file.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Patent Application No. 62 / 732,567, filed September 17, 2018, which is incorporated herein by reference in its entirety.

[0002] The disclosed subject matter relates to methods, systems, and mediums for delivering manifest-less streaming media content. [Background technology]

[0003] The development of digital media content sharing and Internet networking has enabled many features to enhance the user experience. For example, on the Internet, content hosting websites and content sharing services allow users to upload, view, and share digital media content, which may include audio / music clips, video clips, and combinations of audio and video content. Viewers can view media content that is streamed from a content sharing server to a viewing device for playback. To prevent buffering that interrupts the playback of media content, the content may be streamed using an adaptive bitrate protocol, such as HTTP Live Streaming (HLS).

[0004] HLS is a network protocol for streaming audio and video over HTTP from a web server for playback on various computing devices, and the quality of the streamed content can vary depending on the network connection of the viewing device. However, existing HLS-based approaches for streaming media face several challenges. One challenge is high latency. For example, existing HLS-based approaches require a client device to make three round trips to a content sharing server before the client device can play back any video content. The three round trips include one trip to fetch the master playlist, another trip for the media playlist, and yet another trip for the initial media segments. Such a requirement significantly increases the latency of content playback, especially in situations with high round-trip times (RTTs), such as over wireless networks.

[0005] Existing approaches for streaming media playback, such as HLS or Dynamic Adaptive Streaming over HTTP (DASH), also lack client control over stream selection when the media player used by the client does not support stream selection. For example, a client using such a media player to stream video using the HLS protocol cannot control which media segments to fetch and when to fetch them. When a streaming video is available at multiple bitrates or resolutions, the client is required to provide a master playlist URL to the media player, where the master playlist contains all available media playlists, each of which corresponds to a representation of the streaming video at a particular bitrate or resolution. During playback, the media player decides which representation of the streaming video to select for playback.

[0006] Furthermore, existing approaches for streaming video content playback are inefficient at supporting live video streaming. During live playback of streaming media, clients of existing HLS-based streaming video approaches are required to re-fetch the live playlist every few seconds for new media segments. In particular, these protocols involve a manifest file that clients must periodically download to discover new media segments of the streaming media. The periodic manifest download and segment size are examples of end-to-end latency limitations in these protocols. Summary of the Invention [Problem to be solved by the invention]

[0007] Therefore, it would be desirable to provide new methods, systems, and vehicles for delivering manifest-less streaming media content. [Means for solving the problem]

[0008] A method, system, and medium are provided for delivering manifest-less streaming media content. ("Manifest-less" streaming media content means that delivery of the streaming media content is achieved without requiring a client to download a manifest file.)

[0009] According to some embodiments of the disclosed subject matter, a method for delivering live streaming content is provided, the method including the steps of: receiving a request from a user device for a URL corresponding to a format of a live stream provided in a plurality of formats, the live stream including a plurality of segments for each of the plurality of formats, each segment including a respective sequence number; resolving the request for a specific segment of the live stream based on the requested URL, the resolving including: identifying a format of the live stream associated with the request from the plurality of formats based on the requested URL; identifying a sequence number associated with the request based on the requested URL; and identifying a segment from the plurality of segments that corresponds to the identified format and has a sequence number that matches the identified sequence number; loading a segment file corresponding to the specific segment; and returning the segment file to the user device in response to the request.

[0010] In some embodiments, the URL includes a base URL, and the format of the live stream associated with the request is identified from multiple formats based on the base URL.

[0011] In some embodiments, the segment file includes metadata indicating the identified sequence number.

[0012] In some embodiments, the sequence number is identified by determining a current head sequence number for the most recent segment available for playback of one or more segments, the identified sequence number being the current head sequence number, and the identified segment being the most recent segment available for playback.

[0013] In some embodiments, the sequence number is identified by identifying a sequence number associated with the request that is greater than the sequence number for the most recently complete segment available to load based on the URL.

[0014] In some embodiments, the method further includes iteratively loading one or more portions of the segment file as they are being generated, and iteratively returning the generated one or more portions to the user device in a single response.

[0015] According to some embodiments of the disclosed subject matter, a system for delivering live streaming content is provided, the system comprising: a memory; and a hardware processor coupled to the memory, the hardware processor being configured to: receive from a user device a request for a URL corresponding to a format of a live stream provided in a plurality of formats, the live stream including a plurality of segments for each of the plurality of formats, each segment including a respective sequence number; resolve the request to a specific segment of the live stream based on the URL, the resolving including identifying a format of the live stream associated with the request from the plurality of formats based on the URL; identifying a sequence number associated with the request based on the URL; and identifying a segment from the plurality of segments that corresponds to the identified format and has a sequence number that matches the identified sequence number; load a segment file corresponding to the specific segment; and return the segment file to the user device in response to the request.

[0016] According to some embodiments of the disclosed subject matter, a non-transitory computer-readable medium is provided that includes computer-executable instructions that, when executed by a processor, cause the processor to implement a method for delivering live streaming content, the method including: receiving, from a user device, a request for a URL corresponding to a format of a live stream provided in a plurality of formats, the live stream including a plurality of segments for each of the plurality of formats, each segment of the plurality of segments including a respective sequence number; resolving the request to a specific segment of the live stream based on the requested URL, the resolving including: identifying a format of the live stream associated with the request from the plurality of formats based on the requested URL; identifying a sequence number associated with the request based on the requested URL; and identifying a segment of the plurality of segments that corresponds to the identified format and has a sequence number that matches the identified sequence number; loading a segment file corresponding to the specific segment; and returning the segment file to the user device in response to the request. According to further embodiments of the disclosed subject matter, a computer-readable medium is provided that may be a transient computer-readable medium or a non-transitory computer-readable medium. The computer-readable medium includes computer-executable instructions that, when executed by a processor, cause the processor to perform a method according to any aspect or embodiment for delivering live streaming content.

[0017] According to some embodiments of the disclosed subject matter, a system for delivering live streaming content is provided, the system comprising: means for receiving, from a user device, a request for a URL corresponding to a format of a live stream provided in a plurality of formats, the live stream including a plurality of segments for each of the plurality of formats, each segment including a respective sequence number; means for resolving the request to a specific segment of the live stream based on the requested URL, the resolving means comprising: means for identifying a format of the live stream associated with the request from the plurality of formats based on the requested URL; means for identifying a sequence number associated with the request based on the requested URL; and means for identifying a segment from the plurality of segments that corresponds to the identified format and has a sequence number that matches the identified sequence number; means for loading a segment file corresponding to the specific segment; and means for returning the segment file to the user device in response to the request.

[0018] Various objects, features, and advantages of the disclosed subject matter may be more fully appreciated by reference to the following detailed description of the disclosed subject matter when considered in conjunction with the following drawings, in which like reference numerals identify like elements and in which: [Brief explanation of the drawings]

[0019] [Figure 1] FIG. 1 illustrates an example of an information flow diagram for delivering manifest-less streaming media content in accordance with some embodiments of the disclosed subject matter. [Figure 2] FIG. 2 illustrates an example process for resolving a URL request to a segment of a content stream, according to some embodiments of the disclosed subject matter. [Figure 3]FIG. 1 illustrates an example of an information flow diagram for delivering matching segments of a content stream, according to some embodiments of the disclosed subject matter. [Figure 4] FIG. 1 illustrates an example of an information flow diagram for delivering a segment for a specified point in the history of a content stream, according to some embodiments of the disclosed subject matter. [Figure 5] FIG. 1 illustrates an example of an information flow diagram for incremental streaming, in accordance with some embodiments of the disclosed subject matter. [Figure 6] FIG. 1 is a schematic diagram of an exemplary system suitable for implementing mechanisms described herein for delivering manifest-less streaming media content, in accordance with some embodiments of the disclosed subject matter. [Figure 7] 7 illustrates a detailed example of hardware that may be used in the server and / or user device of FIG. 6 in accordance with some embodiments of the disclosed subject matter. DETAILED DESCRIPTION OF THE INVENTION

[0020] According to various embodiments, mechanisms (which may include methods, systems, and media) for delivering streaming content are provided. In some embodiments, the mechanisms described herein may allow any application-level metadata to be supported without requiring support at the content delivery network level for caching metadata separate from segment files. Additionally or alternatively, in some embodiments, the mechanisms described herein may enable delivery of streaming content without requiring periodic delivery of a separate manifest file containing metadata and / or information for discovering new segments of the content stream, thereby reducing end-to-end latency. Additionally or alternatively, in some embodiments, the mechanisms described herein may enable user devices to access non-head segments of the stream, which allows control over which media segments to fetch.

[0021] In some embodiments, the mechanisms described herein may be used to deliver streaming content. For example, in some embodiments, the mechanisms described herein may be used to request a uniform resource locator (URL) for playback of a content stream, as described below in connection with FIG. 1. In a more specific example, the URL may include a base URL corresponding to the format of the content stream. Continuing with this example, in some embodiments, the URL may also include one or more URL parameters to indicate a sequence number associated with the request.

[0022] In some embodiments, a content stream may be provided for playback in one or more formats, such as a codec format, a resolution format, an audio format, a video format, and / or any other suitable format. For example, in some embodiments, a content stream being uploaded by a broadcaster or content creator at 1280x720 pixel resolution may have 720p H.264 video (1280x720 pixels), 480p H.264 video (854x480 pixels), 360p H.264 video (640x360 pixels), 240p H.264 video (426x240 pixels), 144p H.264 video (256x144 pixels), a format for AAC audio, and / or any other suitable format. In some embodiments, the content stream may be segmented into one or more segments of approximately constant duration for each video and / or audio format available for playback. In some embodiments, each segment may be several seconds in duration or any other suitable duration. In some embodiments, the content stream may be a live stream.

[0023] In some embodiments, the web server may provide one base URL for each format of the content stream available for playback (e.g., codec format, resolution format, audio format, video format, and / or any other suitable format). Continuing with the example above, in some embodiments, the web server may provide one base URL for each of 720p H.264 video, 480p H.264 video, 360p H.264 video, 240p H.264 video, 144p H.264 video, AAC audio, and / or any other suitable format. In some embodiments, the base URLs may be for Hypertext Transfer Protocol (HTTP), Hypertext Transfer Protocol Secure (HTTPS) protocol, Quick UDP Internet Connection (QUIC) protocol, and / or any other suitable protocol.

[0024] In some embodiments, a base URL for each format can locate a dynamically determined head segment file for that format. In some embodiments, the head segment can be the most recent segment of a live stream available to be loaded. In some embodiments, a request for a base URL can be resolved to a particular segment identified by a sequence number, which can be a dense integer index in the segment timeline (a "dense" index is the index at which an index entry appears relative to an index value). For example, in some embodiments, the first segment of a stream can have a sequence number of 0, and each subsequent segment can have its respective segment number incremented by 1 (or any other suitable incremental value).

[0025] In some embodiments, a user device can access non-head segments of a stream by adding one or more URL parameters to a base URL in a request. For example, in some embodiments, adding a “sq” parameter that identifies a particular content stream to a base URL can locate a particular segment of that stream by sequence number. Continuing this example, in some embodiments, adding “sq=0” to a base URL can locate the first segment of the stream identified by the base URL, adding “sq=1” can locate the second segment of the stream, and so on. As another example, in some embodiments, adding a “headm” parameter to a base URL can locate a segment of the stream that is a specified number of segments after the head segment. Continuing this example, in some embodiments, adding “headm=1” to a base URL can locate the segment that immediately precedes the head segment, adding “headm=2” to a base URL can locate the segment that is two segments before the head segment, and so on. In some embodiments, adding “headm=0” to a base URL can be equivalent to not providing any “headm” parameters.

[0026] In some embodiments, the mechanisms described herein may be used to resolve requests for segments of a content stream, as described below in connection with FIG. 1 and FIG. 2. For example, in some embodiments, the mechanisms described herein may be used to receive a request for a URL, identify a format of a content stream associated with the request, identify a sequence number associated with the request, and / or identify a segment of a content stream associated with the request, as described below in connection with FIG. 2. In some embodiments, the mechanisms described herein may be used to load and / or return a segment file associated with an identified segment, as described below in connection with FIG. 1. Note that in some embodiments, the mechanisms described herein may be used to embed metadata into the container format of a segment file. Thus, in some embodiments, the mechanisms described herein may enable any application-level metadata to be supported without requiring support at the content delivery network level for caching the metadata separately from the segment files. Additionally, in some embodiments, the mechanisms described herein may enable the delivery of streaming content without requiring the periodic delivery of a separate manifest file containing metadata and / or information for discovering new segments of a content stream, thereby reducing end-to-end latency.

[0027] In some embodiments, the mechanisms described herein may be used to deliver matching segments of a content stream. For example, in some embodiments, the mechanisms described herein may be used to detect inconsistencies between segments of a content stream received from a content delivery network, e.g., to detect inconsistencies between a sequence number for a segment of one file of the stream (e.g., an audio segment file) and a sequence number for a segment of another file of the stream (e.g., a video segment file), as described below in connection with FIG. 3. In some embodiments, the mechanisms described herein may be used to request one or more URLs for the next segment of the content stream and / or return one or more corresponding segment files, as described below in connection with FIG. 3. For example, in some embodiments, the mechanisms described herein may be used to wait a predetermined amount of time before responding to a request for a segment that has not yet been generated, as described below in connection with FIG. 3. Thus, in some embodiments, the mechanisms described herein may enable a player application to issue a request for the next segment before the next segment exists, thereby reducing and / or eliminating the contribution of network request latency, which may reduce end-to-end latency.

[0028] In some embodiments, the mechanisms described herein may be used to deliver segments for specified points in the history of a content stream. For example, in some embodiments, the mechanisms described herein may be used to detect inputs for seeking points in the history of a content stream, such as user inputs associated with a timeline representation of the content stream and / or any other suitable inputs, as described below in connection with FIG. 3. In some embodiments, the mechanisms described herein may be used to request a URL corresponding to a point in the history of the content stream, resolve the request to a segment corresponding to a point in the history of the content stream, and / or return a segment file associated with the segment, as described below in connection with FIG. 3. In some embodiments, the mechanisms described herein may be used to play back the returned content and / or to issue requests iteratively to approach a desired point in history. Thus, in some embodiments, the mechanisms described herein may enable a user to view historical portions of a live stream, such as when a user joins a live stream after the live stream has already begun.

[0029] In some embodiments, the mechanisms described herein may be used to stream content incrementally. For example, in some embodiments, the mechanisms described herein may be used to resolve URL requests for segments that are still being generated and / or to iteratively return portions of the segment files for that segment. As a more specific example, in some embodiments, the mechanisms described herein may be used to first return a portion of the segment that is currently being generated, and then incrementally return the remainder of the segment's data in the same response as the data is being generated. Thus, in some embodiments, the mechanisms described herein may enable a live stream to be played with low latency while still having the efficiency benefits of large segment sizes.

[0030] In some embodiments, the mechanisms described herein can be used to improve computer network content streaming. For example, in some embodiments, the mechanisms described herein can eliminate the need for periodic delivery of manifest files by initializing playback with a set of base URLs for each format of a content stream, each of which locates a dynamically determined head segment file for that format, and by embedding metadata that would normally be included in the manifest file in the segment files, thereby reducing end-to-end latency. As another example, the mechanisms described herein can support adaptive resolution playback of live streams and can enable playback of historical portions of a live stream. As yet another example, the mechanisms described herein can support playing streams with an end-to-end latency of less than one segment to enable low-latency playback with the efficiencies of large segment sizes.

[0031] These and other features for delivering manifest-less streaming media content are further described in connection with FIGS.

[0032] 1, an illustrative example of an information flow diagram 100 for delivering streaming content is shown, in accordance with some embodiments of the disclosed subject matter. As shown, the blocks of information flow diagram 100 may be performed by one or more suitable computing devices, such as one or more servers, one or more user devices, and / or any other suitable computing devices, such as those shown in and described below in connection with FIGS. 6 and 7.

[0033] At 102, a user device may request a URL for playback. For example, in some embodiments, a player application running on the user device may request a URL for playback of a content stream. In some embodiments, the URL is provided by a web server and may include a base URL corresponding to the format of the stream and one or more of: an "sq" parameter corresponding to a sequence number, a "headm" parameter corresponding to the number of segments after the head segment, and / or any other suitable URL parameters or information. As described below, in some embodiments, the user device may, for example, append one or more parameters to the requested base URL.

[0034] It should be noted that in some embodiments, the content stream may be a live stream and may be provided in one or more formats (e.g., codec formats, resolution formats, audio formats, video formats, and / or any other suitable formats). In some embodiments, the live stream may include one or more segments per format, and each segment may have a respective sequence number.

[0035] In some embodiments, the user device may request a URL for playback in any suitable manner. In some embodiments, the user device may request a URL based on one or more factors, such as an optimal playback resolution, a specified point in the history of the content stream, and / or any other suitable factor. For example, in some embodiments, the user device may automatically request a base URL corresponding to an appropriate format using intelligent playback resolution selection based on one or more of: currently available network bandwidth, the screen size of the user device, currently available hardware resources of the user device, and / or any other suitable criteria. Additionally or alternatively, in some embodiments, the user device may append one or more parameters, such as an “sq” parameter, a “headm” parameter, and / or any other suitable URL parameter, to the requested base URL to facilitate locating the specified point in history.

[0036] At 104, the content delivery network can resolve a URL request from the user device to a particular segment of the stream. In some embodiments, the content delivery network can resolve the request to a particular segment identified by a sequence number using any suitable information, such as a base URL including any additional parameters in the URL. In some embodiments, the content delivery network can resolve a URL request to a particular segment in any suitable manner.

[0037] For example, referring to Figure 2, an illustrative example of a process 200 for resolving a request for a URL to a segment of a content stream is shown, in accordance with some embodiments of the disclosed subject matter. As shown, the blocks of process 200 may be performed by one or more suitable computing devices, such as one or more servers, one or more user devices, and / or any other suitable computing device, as shown in and described below in connection with Figures 6 and 7. In some embodiments, the blocks of process 200 may be implemented in connection with block 104 of Figure 1.

[0038] At 202, the content delivery network may receive a request for a URL from a user device in any suitable manner.

[0039] At 204, the content delivery network can identify a format of the content stream associated with the request based on the URL. For example, in some embodiments, the content delivery network can determine the requested format from one or more available formats of the content stream by identifying a format associated with a base URL from the requested URL.

[0040] At 206, the content delivery network can identify a sequence number associated with the request based on the URL. For example, in some embodiments, the content delivery network can resolve the base URL to the head sequence number (or the sequence number for the head segment), such as by querying a cache of head sequence numbers to determine the current head sequence number. Additionally or alternatively, in some embodiments, the content delivery network can send a query to the origin server of the content stream for the head sequence number, and the origin server can return the current head sequence number in response, such as at 106. As another example, in some embodiments, the content delivery network can identify a sequence number associated with the request based on a "headm" parameter and / or a "sq" parameter included in the URL.

[0041] At 208, the content delivery network may identify a segment of the content stream associated with the request. In some embodiments, the content delivery network may identify a segment having a format that matches the identified format associated with the base URL and having a sequence number that matches the identified sequence number associated with the request.

[0042] Referring back to FIG. 1 , at 108, the content delivery network may load the segment file associated with the identified segment referenced by the sequence number in the request. In some embodiments, the content delivery network may load the segment file in any suitable manner. For example, in some embodiments, the content delivery network may load the segment file from one or more caches and / or any other suitable location. As another example, in some embodiments, the content delivery network may send a query to an origin server of the content stream for the segment file, and the origin server may return the requested segment file in response, such as at 110. In some embodiments, the content delivery network may resolve the head sequence number and load the content of the head segment with a single query to the origin server.

[0043] In some embodiments, segment files may include audio content, video content, and / or any other suitable content and / or data. In some embodiments, segment files may include application-level metadata embedded within an audio or video container format. In some embodiments, the metadata may be embedded within the container format using a mechanism supported by the container format for embedding information that is not interpreted by a media decoder. For example, in some embodiments, when the container format is FMP4, the metadata may be embedded in an "emsg" atom. As another example, in some embodiments, when the container format is WebM, the metadata may be embedded in a tag element. Note that in some embodiments, the metadata may be embedded using any suitable mechanism for any suitable container format.

[0044] It should also be noted that the metadata can include any suitable information. For example, in some embodiments, the metadata can include ASCII key-value pairs. As a more specific example, in some embodiments, the key-value pairs can be in the form "key:value" and can be separated by newline characters. As an even more specific example, in some embodiments, the metadata can include a key-value pair with a key "Sequence-Number" and a value equal to the sequence number, where the segment file can be located, such as by using the "sq" URL parameter. In some embodiments, the key-value pairs can be encoded according to the same rules as HTTP response headers.

[0045] At 112, the content delivery network may return the requested segment files to the user device in any suitable manner. For example, in some embodiments, the content delivery network may return the segment files as an HTTP response and / or any other suitable type of response.

[0046] At 114, the user device may receive the segment file from the content delivery network and load the segment file. For example, in some embodiments, a player application running on the user device may extract video and / or audio content from the segment file for playback. Additionally or alternatively, the player application may extract the value of the Sequence-Number key from the metadata of the segment file to determine which segment file was returned by the content delivery network. Note that in some embodiments, the player application may determine which segment file was returned when the player application loads a segment file with an indeterminate sequence number, such as when loading a segment file in response to a request for a base URL or using the "headm" parameter.

[0047] 3, an illustrative example of an information flow diagram for delivering matching segments of a content stream is shown, in accordance with some embodiments of the disclosed subject matter. As shown, the blocks of information flow diagram 300 may be performed by one or more suitable computing devices, such as one or more servers, one or more user devices, and / or any other suitable computing device, as shown in and described below in connection with FIGS. 6 and 7.

[0048] At 302, the user device may detect a mismatch between segments of a content stream received from a content delivery network. For example, in some embodiments, a player application running on the user device may detect a mismatch between sequence numbers for audio and video segment files received from the content delivery network, such as by extracting the value of a Sequence-Number key from the metadata of each segment file. It should be noted that the user device may detect a mismatch between segments in any suitable manner.

[0049] At 304, the user device may request one or more URLs for the next segment of the content stream. For example, in some embodiments, a player application running on the user device may request a base URL including an "sq" parameter set to the current value of the Sequence-Number key plus one for each of the next audio segment and / or the next video segment. Note that in some embodiments, the player application may determine a higher sequence number between the sequence number for the audio segment file and the sequence number for the video segment file, and the player application may set the "sq" parameter to the higher sequence number plus one. Note that the user device may request the URLs in any suitable manner.

[0050] At 306, the content delivery network may receive one or more requests for one or more URLs from the user device and may resolve the request to the next segment of the content stream. In some embodiments, the content delivery network may resolve the request for the URL from the user device in any suitable manner, such as that described above in connection with FIG. 2. In some embodiments, the content delivery network may determine that the request includes an “sq” parameter set to a value greater than the sequence number for the latest segment available to load. In some embodiments, the content delivery network may wait a predetermined duration (e.g., several seconds and / or any suitable duration) for the next segment to be generated before querying the origin server for the next segment and / or before responding to the request from the user device.

[0051] At 308, the content delivery network may load the segment file associated with the segment referenced by the sequence number in the request. In some embodiments, the content delivery network may send a query to the origin server of the content stream for the segment file, and the origin server may respond by returning the requested segment file, such as at 310. In some embodiments, the origin server may wait a predetermined duration (e.g., a few seconds and / or any suitable duration) for the next segment to be generated before responding to the request from the content delivery network.

[0052] At 312, the content delivery network may return the requested segment files to the user device in any suitable manner. For example, in some embodiments, the content delivery network may return the segment files as an HTTP response.

[0053] At 314, the user device can receive the segment files from the content delivery network in any suitable manner and can load the segment files with matching sequence numbers. For example, in some embodiments, a player application running on the user device can extract video and / or audio content from the segment files for playback.

[0054] 4, an illustrative example of an information flow diagram for delivering segments for specified points in the history of a content stream is shown, in accordance with some embodiments of the disclosed subject matter. As shown, the blocks of information flow diagram 400 may be performed by one or more suitable computing devices, such as one or more servers, one or more user devices, and / or any other suitable computing devices, such as those shown in and described below in connection with FIGS. 6 and 7.

[0055] At 402, the user device may detect an input to seek to a point in the history of the content stream in any suitable manner. For example, in some embodiments, a player application may detect that a user has selected a portion of a visual representation corresponding to a timeline of the content stream to specify a point in history to seek to and / or any other suitable input.

[0056] Note that in some embodiments, the origin server may use a predetermined, approximately constant segment duration for each content stream, and thus the relative difference in elapsed stream time between two segments may be approximately equal to the difference in their sequence number values multiplied by the predetermined segment duration of the content stream. For example, in some embodiments, given a predetermined segment duration of 3 seconds, the elapsed stream time between the first segment of a content stream and the tenth segment of a content stream may be approximately equal to (10 x 3 seconds) - (0 x 3 seconds) = 30 seconds.

[0057] At 404, the user device may request a URL for a segment corresponding to a specified point in the history of the content stream in any suitable manner. For example, in some embodiments, the player application may request a URL for a segment whose elapsed stream time is approximately equal to the specified point in the history of the content stream, such as by using an "sq" parameter and / or any other suitable URL parameter. As a more specific example, in some embodiments, given a predetermined segment duration of 3 seconds and a user request to find a point in the content stream that corresponds to 30 seconds, the player application may request a URL for the 10th segment by using an "sq" parameter set to a value of 10.

[0058] At 406, the content delivery network may, in some embodiments, resolve the URL request from the user device to a particular segment of the stream, such as one identified by a sequence number using the "sq" parameter in the request. In some embodiments, the content delivery network may resolve the URL request from the user device in any suitable manner, such as that described above in connection with FIG.

[0059] At 408, the content delivery network may load the segment files associated with the segments referenced by the sequence numbers in the request. For example, in some embodiments, the content delivery network may load the segment files from one or more caches, the origin server of the content stream, and / or any other suitable location.

[0060] At 410, the content delivery network may return the requested segment files to the user device in any suitable manner. For example, in some embodiments, the content delivery network may return the segment files as an HTTP response and / or any other suitable type of response. Additionally, in some embodiments, the content delivery network may return an HTTP response header that identifies the current head sequence number, the exact current elapsed stream time, and / or any other suitable information.

[0061] The user device may receive the segment files from the content delivery network and load the segment files at 412. For example, in some embodiments, the player application may determine which segment files were returned by the content delivery network and / or the elapsed stream time of the sequence files, such as by extracting the value of the Sequence-Number key from the metadata of the segment files or in any other suitable manner.

[0062] At 414, the user device may determine whether to play the returned segment file based on the elapsed stream time of the returned segment file. For example, in some embodiments, the player application may compare the elapsed stream time of the returned segment to a specified point in the history of the content stream. If the player application determines that the elapsed stream time of the returned segment matches the specified point in the history of the content stream, in some embodiments, the player application may play the content of the returned segment file at 416. If the player application determines that the elapsed stream time of the returned segment does not match the specified point in the history of the content stream, in some embodiments, the player application may continue playing the content from the elapsed stream time of the returned segment file at 416, or the player application may return to 404 by repeatedly issuing requests to approach the desired elapsed stream time.

[0063] Note that in some embodiments, the user device can use information from the response headers returned at 410 when playing the content at 416. For example, in some embodiments, the player application can use the current head sequence number and / or the exact current elapsed stream time to extend the visual representation of the timeline to include newly generated segments, even while the player application is playing historical portions of the content stream.

[0064] 5, an illustrative example of an information flow diagram for incremental streaming is shown, in accordance with some embodiments of the disclosed subject matter. As shown, the blocks of information flow diagram 500 may be performed by one or more suitable computing devices, such as one or more servers, one or more user devices, and / or any other suitable computing device, as shown in and described below in connection with FIGS. 6 and 7.

[0065] At 502, a user device may request a URL for playback. For example, in some embodiments, a player application running on the user device may request a URL that includes a base URL, a "headm" parameter set to a value of 0, and / or a "sq" parameter set to a sequence number for a segment that is still being generated.

[0066] At 504, the content delivery network may resolve the URL request from the user device for the segment that is still being generated in any suitable manner. In some embodiments, the content delivery network may resolve the URL request from the user device in any suitable manner, such as that described above in connection with FIG.

[0067] At 506, the content delivery network may load the portions of the segment files for the requested segment in any suitable manner. For example, in some embodiments, the content delivery network may query the origin server of the content stream for one or more portions of the requested segment files.

[0068] The origin server can receive a request for a segment that is still being generated, at 508, and can incrementally return portions of the segment as they are being generated. For example, in some embodiments, the origin server can first return a portion of the currently generated segment to the content delivery network, and then incrementally return the remainder of the data for that segment, at 510, in the same response as the data continues to be generated.

[0069] Note that in some embodiments, the origin server and / or content delivery network may return the generated portions of the segment using a response encoding that allows for returning a response having an indeterminate byte length. For example, in some embodiments, the origin server and / or content delivery network may respond using chunked transfer encoding, such as that identified by the HTTP response header "transfer-encoding: chunked" when using the HTTP protocol. Additionally, in some embodiments, the response may use a container format that supports incremental streaming. For example, in some embodiments, the container format may include one "moof" atom per frame of audio and / or video.

[0070] At 512, the content delivery network can return one or more portions of the segment to the user device as they are received from the origin server in a single response.

[0071] At 514, the user device can receive and load the portion of the segment from the content delivery network.

[0072] 6, an example of hardware 600 for delivering streaming content that may be used in accordance with some embodiments of the disclosed subject matter is shown. As shown, hardware 600 may include an origin server 602, a content delivery network 604 including one or more servers, such as server 606 and server 608, a communication network 610, one or more user devices 612, such as user device 614 and user device 616, and / or a web server 618.

[0073] Origin server 602 may be any suitable server for storing information, data, programs, and / or any other suitable type of content for delivering streaming content. In some embodiments, origin server 602 may store one or more segment files for each format of a content stream. In some embodiments, origin server 602 may store a current head sequence number for a head segment of a live stream. In some embodiments, origin server 602 may store any other suitable content or information. In some embodiments, origin server 602 may perform any suitable function, such as returning a head sequence number, returning one or more segment files, and / or gradually returning one or more portions of one or more segments as they are generated, as described above in connection with FIGS. 1 and 3-5.

[0074] Content delivery network 604 may be any suitable network for storing information, data, programs, and / or any other suitable type of content for delivering streaming content. Note that in some embodiments, a content delivery network as described herein may be implemented by one or more suitable computing devices, such as server 606 and / or server 608, which may be any suitable server for storing information, data, programs, and / or any other suitable type of content. For example, in some embodiments, server 606 and / or server 608 may store one or more segment files for one or more formats of content streams. Additionally, in some embodiments, server 606 and / or server 608 may store one or more head sequence numbers for live streams, such as in one or more caches. In some embodiments, origin server 602 may perform any suitable function, such as receiving a request for a URL, resolving the request to a segment of a content stream, identifying a format, identifying a sequence number, identifying a segment, loading a segment file, returning one or more segment files, and / or incrementally returning one or more portions of one or more segments as they are generated, as described above in connection with FIGS. 1-5.

[0075] The communications network 610, in some embodiments, may be any suitable combination of one or more wired and / or wireless networks. For example, the communications network 610 may include any one or more of the Internet, an intranet, a wide area network (WAN), a local area network (LAN), a wireless network, a digital subscriber line (DSL) network, a frame relay network, an asynchronous transfer mode (ATM) network, a virtual private network (VPN), and / or any other suitable communications network. In some embodiments, the origin server 602 may be connected to the communications network 610 via the content delivery network 604 by one or more communications links. In some embodiments, the communications network 610 may be linked to the content delivery network 604, user devices 612, and / or web servers 618 via one or more communications links. The communications link may be any communications link suitable for communicating data between the user device 612, the origin server 602, the content delivery network 604, and the web server 618, such as a network link, a dial-up link, a wireless link, a hardwired link, any other suitable communications link, or any suitable combination of such links.

[0076] Web server 618 may be any suitable server for storing information, data, programs, and / or any other suitable type of content. In some embodiments, web server 618 may perform any suitable functionality, such as providing one or more base URLs for one or more formats of content streams, as described above in connection with FIGS. 1-5.

[0077] The user device 612 may include any one or more user devices for presenting streaming content. For example, in some embodiments, the user device 612 may include a television, a speaker, a mobile phone, a tablet computer, a desktop computer, a laptop computer, a vehicle entertainment system, a wearable computer, and / or any other suitable type of user device. In some embodiments, the user device 612 may include a player application stored in memory and executed by the user device 612 to present the streaming content. In some embodiments, the user device 612 may perform any suitable functions, such as requesting a URL for playback, detecting unmatched segments of a stream, detecting inputs to seek to a point in the history of a stream, loading received segment files, and / or playing content, as described above in connection with FIGS. 1-5 .

[0078] Although origin server 602, server 606, server 608, and web server 618 are each illustrated as one device, the functions performed by origin server 602, server 606, server 608, and web server 618 may, in some embodiments, be performed using any suitable number of devices. For example, in some embodiments, multiple devices may be used to implement the functions performed by each of origin server 602, server 606, server 608, and web server 618.

[0079] Although two user devices 614 and 616 are shown in FIG. 6 to avoid overcomplicating the diagram, in some embodiments, any suitable number and / or type of user devices may be used.

[0080] Origin server 602, server 606, server 608, user device 612, and web server 618 may, in some embodiments, be implemented using any suitable hardware. For example, in some embodiments, devices 602, 606, 608, 612, and 618 may be implemented using any suitable general-purpose or special-purpose computer. For example, a mobile phone may be implemented using a special-purpose computer. Any such general-purpose or special-purpose computer may include any suitable hardware. For example, as illustrated in the example hardware 700 of FIG. 7 , such hardware may include a hardware processor 702, memory and / or storage 704, an input device controller 706, input devices 708, a display / audio driver 710, a display and audio output circuit 712, a communication interface 714, an antenna 716, and a bus 718.

[0081] Hardware processor 702, in some embodiments, may include any suitable hardware processor, such as a microprocessor, a microcontroller, a digital signal processor, special purpose logic, and / or any other suitable circuitry for controlling the functions of a general-purpose or special-purpose computer. In some embodiments, hardware processor 702 may be controlled by a server program stored in memory and / or storage of a server, such as server 606 and / or server 608. For example, in some embodiments, the server program may cause hardware processor 702 to receive a request for a URL, resolve the request to a segment of a content stream, identify a format, identify a sequence number, identify a segment, return a head sequence number, load a segment file, return one or more segment files, incrementally return one or more portions of one or more segments as they are generated, provide one or more base URLs for one or more formats of the content stream, and / or perform any other suitable function. In some embodiments, hardware processor 702 may be controlled by a program stored in memory and / or storage of a user device, such as user device 612 and / or user device 616. For example, in some embodiments, the program may cause the hardware processor 702 to request a URL for playback, detect unmatched segments of the stream, detect input to seek to a point in the history of the stream, load received segment files, play the content, and / or perform any other suitable function.

[0082] Memory and / or storage 704, in some embodiments, may be any suitable memory and / or storage for storing programs, data, and / or any other suitable information. For example, memory and / or storage 704 may include random access memory, read-only memory, flash memory, hard disk storage, optical media, and / or any other suitable memory.

[0083] Input device controller 706, in some embodiments, may be any suitable circuitry for controlling and receiving input from one or more input devices 708. For example, input device controller 706 may be circuitry for receiving input from a touchscreen, from a keyboard, from one or more buttons, from a voice recognition circuit, from a microphone, from a camera, from a light sensor, from an accelerometer, from a temperature sensor, from a near-field sensor, from a pressure sensor, from an encoder, and / or from any other type of input device.

[0084] Display / audio driver 710, in some embodiments, may be any suitable circuitry for controlling and driving output to one or more display / audio output devices 712. For example, display / audio driver 710 may be circuitry for driving a touch screen, a flat panel display, a cathode ray tube display, a projector, one or more speakers, and / or any other suitable display and / or presentation device.

[0085] Communications interface 714 may be any suitable circuitry for interfacing with one or more communications networks (e.g., communications network 610). For example, interface 714 may include network interface card circuitry, wireless communications circuitry, and / or any other suitable type of communications network circuitry.

[0086] Antenna 716, in some embodiments, may be any suitable antenna or antennas for wirelessly communicating with a communication network (e.g., communication network 610). In some embodiments, antenna 716 may be omitted.

[0087] Bus 718 may be any suitable mechanism for communicating between two or more components 702, 704, 706, 710, and 714, in some embodiments.

[0088] Any other suitable components may be included in hardware 700 according to some embodiments.

[0089] In some embodiments, at least some of the above-described blocks of the processes of Figures 1-5 may be executed or performed in any order or sequence, including but not limited to the order and sequence shown in and described with respect to the figures. Also, some of the above-described blocks of Figures 1-5 may be executed or performed substantially simultaneously or in parallel, where appropriate, to reduce latency and processing time. Additionally or alternatively, some of the above-described blocks of the processes of Figures 1-5 may be omitted.

[0090] In some embodiments, any suitable computer-readable medium may be used to store instructions for implementing the functions and / or processes described herein. For example, in some embodiments, the computer-readable medium may be transitory or non-transitory. For example, a non-transitory computer-readable medium may include a medium such as a magnetic medium in non-transitory form (such as a hard disk, a floppy disk, and / or any other suitable magnetic medium), an optical medium in non-transitory form (such as a compact disc, a digital video disc, a Blu-ray disc, and / or any other suitable optical medium), a semiconductor medium in non-transitory form (such as a flash memory, an electrically programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), and / or any other suitable semiconductor medium), any suitable medium that is not transitory or lacks some appearance of permanence during transmission, and / or any suitable tangible medium. As another example, a transitory computer-readable medium may include a wire, a conductor, an optical fiber, a circuit, any suitable medium that is transitory and lacks some appearance of permanence during transmission, and / or a signal in any suitable non-tangible medium.

[0091] Accordingly, a method, system, and medium for delivering manifest-less streaming media content are provided.

[0092] While the invention has been described and illustrated in the above exemplary embodiments, it should be understood that the disclosure has been made by way of example only, and that numerous changes in the details of the implementation of the invention may be made without departing from the spirit and scope of the invention, which is limited only by the claims that follow. The features of the disclosed embodiments may be combined and rearranged in various ways. [Explanation of symbols]

[0093] 100 Information Flow Diagram 200 processes 300 Information Flow Diagram 400 Information Flow Diagram 500 Information Flow Diagram 602 Origin Server, Device 604 Content Delivery Network 606 Servers, Devices 608 Servers, Devices 610 Communication Network 612 User Device, Device 614 User Devices 616 User Devices 618 Web Server, Device 700 Hardware 702 Hardware Processors, Components 704 Memory and / or Storage, Components 706 Input device controller, component 708 Input Devices 710 Display / Audio Driver, Components 712 Display and audio output circuits, display / audio output devices 714 Communication Interfaces, Interfaces, and Components 716 Antenna 718 Bus

Claims

1. transmitting, by a user device, a request for a plurality of segments of a content stream, the request including an identifier associated with the content stream; receiving, from a content delivery network (CDN), recent segment files corresponding to recent segments of the plurality of segments of the content stream, the recent segment files including a first audio segment file and a first video segment file; detecting, by the user device, a mismatch between the first audio segment file and the first video segment file by detecting a mismatch between a sequence number of the first audio segment file and a sequence number of the first video segment file; transmitting, by the user device, a request for a next segment of the plurality of segments of the content stream; receiving, from the CDN, a next segment file corresponding to the next segment of the plurality of segments of the content stream, the next segment file including a second audio segment file and a second video segment file; in response to detecting a match between the second audio segment file and the second video segment file by detecting a match between a sequence number of the second audio segment file and a sequence number of the second video segment file, loading the second audio segment file and the second video segment file for playback on the user device. method.

2. 2. The method of claim 1, wherein detecting a mismatch between the first audio segment file and the first video segment file includes detecting a mismatch between a sequence number of the first audio segment file and a sequence number of the first video segment file.

3. 3. The method of claim 2, wherein the sequence number of the first audio segment file is extracted from metadata of the first audio segment file, and the sequence number of the first video segment file is extracted from metadata of the first video segment file.

4. transmitting the request for the next segment of the plurality of segments of the content stream, determining the largest sequence number between the sequence number of the first audio segment file and the sequence number of the first video segment file; incrementing said highest sequence number; including the incremented sequence number in the request for the next segment of the plurality of segments of the content stream. The method of claim 2.

5. The method of claim 1 , wherein the content stream is a live stream and the request for the next segment is sent by a client device before any of the next segments are available on the CDN.

6. The method of claim 5 , wherein the next segment file corresponding to the next segment is received from the CDN after a predetermined duration.

7. The method comprises: receiving a third video segment file; determining that the third video segment file matches the first audio segment file by detecting a match between a sequence number of the first audio segment file and a sequence number of the third video segment file; loading the first audio segment file and the third video segment file for playback on the user device. The method of claim 1.

8. The method of claim 1 , wherein the user device sends a request for a segment of the content stream without having a manifest file for the content stream.

9. The method of claim 1 , wherein the identifier associated with the content stream is a uniform resource locator (URL).

10. 10. The method of claim 9, wherein the URL includes a base URL corresponding to a first format of multiple formats of the content stream and one or more URL parameters for indicating a sequence number associated with the request for the multiple segments of the content stream.

11. 11. The method of claim 10, wherein the base URL corresponding to the first format is used to locate a head segment file for the first format, the head segment file corresponding to the most recent segment of the content stream that is available on the CDN.

12. Memory and a processing device coupled to the memory, the processing device comprising: transmitting, by a user device, a request for a plurality of segments of a content stream, the request including an identifier associated with the content stream; receiving, from a content delivery network (CDN), recent segment files corresponding to recent segments of the plurality of segments of the content stream, the recent segment files including a first audio segment file and a first video segment file; detecting, by the user device, a mismatch between the first audio segment file and the first video segment file by detecting a mismatch between a sequence number of the first audio segment file and a sequence number of the first video segment file; transmitting, by the user device, a request for a next segment of the plurality of segments of the content stream; receiving, from the CDN, a next segment file corresponding to the next segment of the plurality of segments of the content stream, the next segment file including a second audio segment file and a second video segment file; loading the second audio segment file and the second video segment file for playback on the user device in response to detecting a match between the second audio segment file and the second video segment file by detecting a match between a sequence number of the second audio segment file and a sequence number of the second video segment file; performing an action including system.

13. 13. The system of claim 12, wherein detecting a mismatch between the first audio segment file and the first video segment file includes detecting a mismatch between a sequence number of the first audio segment file and a sequence number of the first video segment file.

14. 14. The system of claim 13, wherein the sequence number of the first audio segment file is extracted from metadata of the first audio segment file, and the sequence number of the first video segment file is extracted from metadata of the first video segment file.

15. transmitting the request for the next segment of the plurality of segments of the content stream; determining the highest sequence number between the sequence number of the first audio segment file and the sequence number of the first video segment file; incrementing said highest sequence number; and including the incremented sequence number in the request for the next segment of the plurality of segments of the content stream. The system of claim 14.

16. The operation is receiving a third video segment file; determining that the third video segment file matches the first audio segment file by detecting a match between a sequence number of the first audio segment file and a sequence number of the third video segment file; loading the first audio segment file and the third video segment file for playback on the user device. The system of claim 12.

17. The system of claim 12 , wherein the user device sends a request for a segment of the content stream without having a manifest file for the content stream.

18. 1. A computer-readable storage medium having stored thereon instructions that, when executed by a processing device, cause the processing device to: transmitting, by a user device, a request for a plurality of segments of a content stream, the request including an identifier associated with the content stream; receiving, from a content delivery network (CDN), recent segment files corresponding to recent segments of the plurality of segments of the content stream, the recent segment files including a first audio segment file and a first video segment file; detecting, by the user device, a mismatch between the first audio segment file and the first video segment file by detecting a mismatch between a sequence number of the first audio segment file and a sequence number of the first video segment file; transmitting, by the user device, a request for a next segment of the plurality of segments of the content stream; receiving, from the CDN, a next segment file corresponding to the next segment of the plurality of segments of the content stream, the next segment file including a second audio segment file and a second video segment file; loading the second audio segment file and the second video segment file for playback on the user device in response to detecting a match between the second audio segment file and the second video segment file by detecting a match between a sequence number of the second audio segment file and a sequence number of the second video segment file; performing an action including A computer-readable recording medium.

19. 20. The computer-readable medium of claim 18, wherein detecting a mismatch between the first audio segment file and the first video segment file comprises detecting a mismatch between a sequence number of the first audio segment file and a sequence number of the first video segment file.

20. 20. The computer-readable medium of claim 19, wherein the sequence number of the first audio segment file is extracted from metadata of the first audio segment file, and the sequence number of the first video segment file is extracted from metadata of the first video segment file.

Images