Bandwidth switching method by CMAF and DASH clients using Addressable Resource Index tracks and Addressable Resource Index events

Abstract

A method, apparatus and readable storage medium for processing a media stream, the media stream may be according to the DASH standard or the CMAF standard, the method may include receiving media stream data comprising a plurality of media chunks including a first media chunk and a second media chunk and Addressable Resource Index (ARI) information associated with the first media chunk, determining track change information based on the ARI information, determining based on the track change information that a switch to a different media track in the second media chunk is required, and receiving the first media chunk and the second media chunk via respective media tracks, each of the first media chunk and the second media chunk being delivered to a streaming client device with a delivery delay of no more than one chunk.

Description

[Technical Field]

[0001] Incorporation by Reference This application claims the benefit of priority to U.S. Provisional Application No. 63 / 388,577, filed July 12, 2022, which claims the benefit of priority to this provisional application, and to U.S. Non-provisional Application No. 18 / 342,230, filed June 27, 2023, which is incorporated by reference in its entirety into this disclosure.

[0002] The present disclosure relates generally to media streaming technologies, including Dynamic Adaptive Streaming over Hypertext transfer protocol (DASH) and Common Media Application Format (CMAF). In particular, the disclosed technologies relate to methods and apparatus for switching bandwidth (i.e., media tracks) based on information provided in Addressable Resource Index (ARI) tracks and / or ARI events. [Background technology]

[0003] The Background section of this specification is intended to provide a general background to the disclosure. To the extent described in this Background section, the work of the currently named inventors and other aspects of the description that may not have been prior art at the time of filing this application are not admitted, expressly or implicitly, as prior art to the present disclosure.

[0004] The Moving Picture Expert Group (MPEG) dynamic adaptive streaming over hypertext transfer protocol (DASH) provides a standard for streaming multimedia content over IP networks. In the DASH standard, a media presentation description (MPD) is used to provide DASH clients with information for adaptively streaming media content by downloading media segments from a DASH server. The DASH standard enables streaming of multi-rate content. One aspect of the DASH standard includes the transport of MPD events and in-band events, and the client processing model used to handle the events.

[0005] The Common Media Application Format (CMAF) is a standard used to package and deliver various forms of Hypertext Transfer Protocol (HTTP)-based media. The standard works with, for example, HTTP Live Streaming (HLS) and DASH protocols to simplify delivery of media to playback devices by packaging data in a standardized transport container file. The standard also uses chunked encoding and chunked transfer encoding to reduce latency, which reduces storage requirements and therefore costs. Summary of the Invention [Means for solving the problem]

[0006] In aspects of the present disclosure, methods and apparatuses for media stream processing are provided, and in particular, methods and apparatuses for switching bandwidth (i.e., media tracks) based on information provided in ARI tracks and / or ARI events are provided. In some implementations, a method for processing a media stream is disclosed. The media stream may include at least two media tracks, and may conform to the Dynamic Adaptive Streaming over HTTP (DASH) standard or the Common Media Application Format (CMAF). The method may be performed by, for example, a streaming client device and may include receiving media stream data comprising a plurality of media chunks, including a first media chunk and a second media chunk, and Addressable Resource Index (ARI) information associated with the first media chunk; determining track change information based on the ARI information; determining, based on the track change information, that switching to a different media track in the second media chunk is necessary; and receiving the first media chunk and the second media chunk via their respective media tracks. In some implementations, each of the first media chunk and the second media chunk is delivered to the streaming client device with a delivery delay of one chunk or less.

[0007] In some implementations, another method for processing a media stream is disclosed. A media stream having at least two media tracks and conforming to the Dynamic Adaptive Streaming over HTTP (DASH) standard or the Common Media Application Format (CMAF) is executed by a streaming client, the method comprising the steps of receiving one of an Addressable Resource Index (ARI) sample obtained from an ARI track associated with a first media slice in a first media track of the media stream or an ARI event associated with the media stream embedded in the first media slice in the first media track of the media stream, wherein the ARI event provides characteristic information of at least one of a first media slice in the first media track and a first parallel media slice in another media track of the media stream that is aligned in position with the first media slice, or a second media slice in the first media track that follows the first media slice and a second parallel media slice in another media track of the media stream that is aligned in time position with the second media slice, and determining, based on the characteristic information, that a switch to one of the other media tracks is necessary.

[0008] Aspects of the present disclosure also provide a media stream processing device or apparatus that includes circuitry configured to perform any of the above method implementations.

[0009] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions that, when executed by a computer, cause the computer to perform a method for media stream processing for decoding and / or encoding video.

[0010] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings described below. [Brief explanation of the drawings]

[0011] [Figure 1] 1 illustrates a system according to an embodiment of the present disclosure. [Figure 2] 1 illustrates a Dynamic Adaptive Streaming over HTTP (DASH) system according to an embodiment of the present disclosure. [Figure 3] 1 illustrates a DASH client architecture according to an embodiment of the present disclosure. [Figure 4] 1 illustrates an example of a DASH data model according to an embodiment of the present disclosure. [Figure 5] 1 illustrates an example of a CMAF data model according to an embodiment of the present disclosure. [Figure 6] 1 illustrates an example of switching media tracks at a segment / chunk level according to an embodiment of the present disclosure. [Figure 7] 1 illustrates exemplary extrapolated switching and interpolated switching based on ARI tracks / ARI samples carrying switching auxiliary information. [Figure 8] Shows a typical extrapolated switch based on ARI events carrying switch auxiliary information [Figure 9] 10 shows exemplary extrapolated and interpolated switching based on ARI events carrying switching assistance information with lag. [Figure 10] 1 shows a flowchart of a method according to an embodiment of the present disclosure. [Figure 11] 1 shows a schematic diagram of a computer system according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0012] Dynamic Adaptive Streaming Over Hypertext Transfer Protocol (DASH) and Media Presentation Description (MPD) One well-known media streaming format is Dynamic Adaptive Streaming over Hypertext Transfer Protocol (DASH), defined in ISO / IEC 23009-1. DASH is an adaptive bitrate streaming method that enables streaming of media content using hypertext transfer protocol (HTTP) infrastructures such as web servers, content delivery networks (CDNs), and various proxies and caches. DASH supports both on-demand and live streaming from DASH servers to DASH clients and allows DASH clients to control streaming sessions, eliminating the need for DASH servers to handle the increased load of adaptive streaming management in large-scale deployments. DASH also enables DASH clients to select streams from various DASH servers, thereby achieving further network load balancing and benefiting DASH clients. DASH allows, for example, dynamic switching between different media tracks by changing the bitrate to adapt to network conditions.

[0013] In DASH, a media presentation description (MPD) file provides DASH clients with information to adaptively stream media content by downloading media segments from a DASH server. The MPD may be in the form of an Extensible Markup Language (XML) document. The MPD file can be fragmented and delivered in parts to mitigate session start delays. The MPD file can also be updated during a streaming session. In some examples, the MPD file supports the representation of content accessibility features, ratings, and camera views. DASH also supports the delivery of content using multi-view scalable coding.

[0014] An MPD file can contain a series of one or more periods. Each of the one or more periods can be defined, for example, by a period element in the MPD file. An MPD file can contain the availableStartTime attribute of the MPD and the start attribute of each period. For media presentations of dynamic type (e.g., types used for live services), the sum of the start attribute of the period and the MPD attribute availableStartTime, together with the duration of the media segments, can indicate the validity time of the period in Coordinated Universal Time (UTC), and in particular, the first media segment of each representation of the corresponding period. For media presentations of static type (e.g., types used for on-demand services), the start attribute of the first period can be 0. For any other periods, the start attribute can specify the offset time of the corresponding period relative to the start time of the first period. Each period can extend until the beginning of the next period, or, in the case of the last period, until the end of the media presentation. Period start times can be highly accurate and can represent the actual timing that results from playing media from all previous periods. Implementations provide an MPD such that the next period is contiguous with the content of the previous period, or in some cases, with the content of the immediately following period, or with the content of a later period (e.g., the period after an advertising period has been inserted).

[0015] Each period can contain one or more adaptation sets, each of which can contain one or more representations of the same media content. A representation can be one of several alternative encodings of audio or video data. Representations can vary by encoding type, for example, by bitrate, resolution, and / or codec used for video data and bitrate and / or codec used for audio data. The term representation can be used to refer to a section of encoded audio or video data that corresponds to a particular period of multimedia content and is encoded in a particular way.

[0016] Adaptation sets for a particular period can be assigned to a group indicated by the group attribute in the MPD file. Adaptation sets within the same group are generally considered to be alternatives to each other. For example, each adaptation set for video data for a particular period can be assigned to the same group so that any adaptation set can be selected for decoding to display the video data of the multimedia content of the corresponding period. The media content of a period can be represented by one adaptation set from group 0 (if group 0 exists). If group 0 does not exist, the media content of a period can be represented by a combination of at most one adaptation set from each non-zero group (several examples exist). The timing data of each representation of a period can be expressed relative to the start time of the period.

[0017] A representation can contain one or more segments. Each representation can contain an initialization segment, or each segment of a representation can be self-initializing. If present, the initialization segment can contain initialization information for accessing the representation. In some cases, the initialization segment does not contain media data. Segments can be unambiguously referenced by an identifier such as a uniform resource locator (URL), uniform resource name (URN), or uniform resource identifier (URI).

[0018] In the implementation, URLs are written according to IETF RFC 3986. <absolute-uri>as, for example, using a fixed convention such as "http" or "https", possibly constrained by a byte range (if a range attribute is provided with the URL). A byte range may be expressed as a byte-range-spec, for example as specified in IETF RFC 2616, which may be constrained to a single expression specifying a contiguous range of bytes. In one embodiment, the segment may be included in the MPD along with the data URL, for example as specified in IETF RFC 2397.

[0019] An MPD file may provide an identifier for each segment, and in some cases may also provide a byte range in the form of a range attribute, which may correspond to data for a segment within a file that is accessible by a URL, URN, or URI.

[0020] A sub-representation can be embedded in a canonical representation (i.e., it contains a sub-representation) and described by a sub-representation element (e.g., SubRepresentation). The properties of one or several media content components embedded in a representation can be described by a sub-representation element. For example, the properties of an embedded audio component (e.g., codec, sampling rate, etc.) or the properties of embedded subtitles (e.g., codec) can be described by a sub-representation element, or several embedded lower-quality video layers (e.g., several low frame rates, etc.) can be described by a sub-representation element. A sub-representation and a representation element can share some common attributes and elements.

[0021] Each representation may also contain one or more media components, each of which may correspond to an encoded instance of one of the particular media types such as audio, video or timed text (e.g., timed text used for closed captioning). Media components may be contiguous in time across boundaries between consecutive media segments within a single representation.

[0022] In some implementations, a DASH client can access and download an MPD file from a DASH server; that is, the DASH client can obtain the MPD file for use when starting a live session. Based on the MPD file, the DASH client can make several decisions for each selected representation, including determining what the latest segment is available on the server, determining the segment valid start time of the next segment and possibly future segments, determining when to start playing the segment and from which timeline within the segment, and determining when to acquire / fetch a new MPD file. After the service is playing, the client can keep track of any drift between the live service and its own playback, which needs to be detected and compensated for.

[0023] Common Media Application Format (CMAF) The Segmented Media Common Media Application Format (CMAF) is an extensible standard used to encode and package segmented media objects for delivery and decoding to end-user devices in adaptive multimedia presentations. The CMAF specification defines several logical media objects, which are described below.

[0024] A CMAF track may contain encoded media samples, including audio, video, and subtitles. The media samples are stored in a CMAF-specified container derived from the ISO Base Media File Format (ISO_BMFF). The media samples may optionally be protected by MPEG Common Encryption. A track may contain a CMAF Header and one or more CMAF Fragments.

[0025] A CMAF switching set may contain alternative tracks that can be switched between and spliced ​​together at CMAF fragment boundaries to adaptively stream the same content at different bit rates and resolutions. An Aligned CMAF Switching Set is two or more CMAF Switching Sets that are encoded from the same source using different encoding techniques (e.g., different codecs) and are temporally aligned.

[0026] A CMAF selection set is a group of switching sets of the same media type that can contain alternative content (e.g., different languages) or alternative encoding schemes (e.g., different codecs).

[0027] A CMAF presentation may contain one or more presentation time synchronization selection sets.

[0028] To enable media content to be distributed across different platforms, CMAF supports Addressable Objects. A CMAF Addressable Object may contain: · CMAF Header: The Header contains information to initialize the track. · CMAF Segment: A series of one or more contiguous fragments from the same track. · CMAF Chunk: A chunk contains a contiguous subset of samples from a fragment. · CMAF Track File: A complete track in one ISO_BMFF file.

[0029] DASH and CMAF events In DASH and CMAF, events provide a means of signaling additional information to a DASH / CMAF client and one or more associated applications. In an implementation, events are timed, so that they have a start time and a duration. Event information may include metadata describing the content of the media presentation. Additionally or alternatively, event information may include control messages for the media player that are related to a particular time during playback of the media presentation, such as ad insertion cues. Events may be implemented, for example, as MPD events or in-band events. They can be part of a manifest file (e.g., MPD) or embedded in an ISOBMFF-based media file, such as in an event message (emsg) box.

[0030] A media presentation description (MPD) event is an event that can be signaled in the MPD. At the period level, an MPD can have a set of events assigned to a media presentation time. Homogeneous events can be specified by event stream elements (e.g., EventStream) within a period element. An event ends at the end of the period even if its start time falls after the period boundary or its duration extends beyond the period boundary. An event stream element contains a message scheme identification (e.g., @schemeIdUri) and an optional additional value (e.g., @value) for the event stream element. Furthermore, if the event stream contains timed events, it may have a time scale attribute (e.g., @timescale) to assign the event to a specific media presentation time within the period. The timed events themselves can be described by event elements contained in the event stream element.

[0031] Inband event streams can be multiplexed with a representation by including event messages as part of a media segment. An event stream may be present in selected representations, only in one or a few selected adaptation sets, or in all representations. For example, one possible configuration is for only the audio adaptation set to contain inband events, or only the video adaptation set to contain inband events. Inband event streams present in a representation can be indicated by inband event stream elements (e.g., InbandEventStream) at various levels, such as the adaptation set level or the representation level. Furthermore, a representation can contain multiple inband event streams, each indicated by a separate inband event stream element.

[0032] 1 shows a system 100 according to an embodiment of the present disclosure. The system 100 includes a content server 110 and an information processing device 120. The content server 110 can provide a content stream including primary content (e.g., a main program) and one or more time-specific metadata tracks.

[0033] The information processing device 120 can be connected to the content server 110 via an interface. For example, the information processing device 120 can play content received from the content server 110. The content can be played based on a manifest file (e.g., an MPD) received by the information processing device 120 (e.g., from the content server 110). The manifest file can further include signaling of one or more time-specific metadata tracks.

[0034] A typical DASH / CMAF system is shown in Figure 2. The DASH system (200) may include a content server (210), an advertisement server (220), and an information processing device (230), which are connected to a network (250). The DASH system (200) may also include one or more supplemental content servers.

[0035] The content server (210) can provide the information processing device (230) with primary content (e.g., a main program) and a manifest file (e.g., an MPD). For example, the manifest file can be generated by an MPD generator (214). In other embodiments, the primary content and the manifest file can be provided by different servers.

[0036] The information processing device (230) can receive the MPD and obtain primary content from the HTTP server (212) of the content server (210) based on the MPD. The MPD can be processed by a DASH client (232) running on the information processing device (230). The DASH client (232) can also obtain advertising content from the advertising server (220) and other content (e.g., interactive content) from one or more supplemental content servers. The primary content and advertising content can be processed by the DASH client (232) and output for display on a display device (236). The display device (236) can be integrated into the information processing device (230) or external to the information processing device (230). The DASH client (232) can also extract event information from one or more timed metadata tracks and send the extracted event information to an application (234) for further processing. For example, the application (234) can be configured to display supplemental content based on the event information.

[0037] The advertisement server (220) can store advertisement content in an advertisement storage (for example, a memory). The stored advertisement content can be requested by the information processing device (230) based on the event information.

[0038] 3 illustrates an example DASH / CMAF client architecture for processing DASH and CMAF events, according to an embodiment of the present disclosure. The DASH / CMAF client (i.e., DASH / CMAF player) communicates with an application (390) and can be configured to process various types of events, including (i) MPD events, (ii) in-band events, and (iii) timed metadata events.

[0039] The manifest parser (305) parses a manifest (e.g., an MPD). The manifest is provided, for example, by a content server (110, 210). The manifest parser (305) extracts event information about MPD events, in-band events, and timed metadata events embedded in timed metadata tracks. The extracted event information can be provided to the DASH logic (310) (e.g., DASH player control logic, selection logic, and heuristic logic). Based on the event information, the DASH logic (310) can notify the application (390) of the event scheme signaled in the manifest.

[0040] The event information can include event scheme information for distinguishing between different event streams. The event scheme information can be used by an application (390) to periodically retrieve event schemes of interest. Furthermore, the application (390) can notify the desired dispatch mode for each of the periodically retrieved schemes through one or more subscription APIs. For example, the application (390) can send a subscription request to a DASH client that identifies one or more event schemes of interest and any desired corresponding dispatch modes.

[0041] If the application (390) periodically obtains one or more event schemes delivered as part of one or more timed metadata tracks, the inband event 'moof' parser (325) can stream the one or more timed metadata tracks to the timed metadata track parser (330). For example, the inband event 'moof' parser (325) parses the movie fragment box ("moof") and then parses the timed metadata tracks based on control information obtained from the DASH logic (310).

[0042] The timed metadata track parser 330 can extract event messages embedded in the timed metadata tracks. The extracted event messages can be stored in the event timed metadata buffer 335. A synchronizer / dispatcher module 340 (e.g., an event and timed metadata synchronizer and dispatcher) can dispatch the periodically captured events to the application 390 (i.e., the synchronizer / dispatcher module 340 sends the periodically captured events to the application 390).

[0043] The MPD events described in the MPD can be parsed by the manifest parser (305) and stored in the buffer (335). For example, the manifest parser (305) parses each event stream element of the MPD and parses each event described in each event stream element. For each event signaled in the MPD, event information such as presentation time and event duration can be associated with the event and stored in the buffer (335).

[0044] The media segments can be parsed to extract inband event messages by the inband event 'moof' parser (325). The identified inband events and their associated presentation times and durations can be stored in a buffer (335).

[0045] Thus, the buffer (335) can store MPD events, in-band events, and / or timed metadata events. The buffer (335) can be, for example, a first-in-first-out (FIFO) buffer. The buffer (335) can be managed in association with the media buffer (350). For example, as long as a media segment is present in the media buffer (350), any events or timed metadata corresponding to that media segment can be stored in the buffer (335).

[0046] The DASH Access Application Programming Interface (API) 315 can manage the fetching and receiving of content streams (i.e., data flows) containing media content and various metadata through the HTTP protocol stack 320. The DASH Access API 315 can separate the received content streams into different data flows. The data flows provided to the inband event moof parser can include media segments, one or more timed metadata tracks, and inband event signaling contained in the media segments. In one embodiment, the data flows provided to the manifest parser 305 can include an MPD.

[0047] The DASH Access API (315) can forward the manifest to the manifest parser (305). In addition to describing events, the manifest can also provide information about media segments to the DASH logic (310), which can communicate with the application (390) and the inband event moof parser (325). The application (390) can be associated with media content processed by the DASH client. Control / sync signals exchanged between the application (390), DASH logic (310), manifest parser (305), and DASH Access API (315) can be used to control the fetching of media segments from the HTTP stack (320) based on the information about the media segments provided in the manifest.

[0048] The inband event moof parser (325) parses the media data flow to obtain media segments containing media content, timed metadata in the timed metadata track, and any inband events signaled in the media segments. The media segments containing media content can be parsed by the file format parser (345) and stored in the media buffer (350).

[0049] The events stored in the buffer (335) allow the synchronizer / dispatcher (340) to communicate available events relevant to the application (i.e., events of interest) to the application through an event / metadata API. The application can be configured to process available events (e.g., MPD events, in-band events, or timed metadata events) and periodically retrieve specific events or timed metadata by notifying the synchronizer / dispatcher (340). Any events stored in the buffer (335) that are not relevant to the application, but instead pertain to the DASH client itself, can be forwarded by the synchronizer / dispatcher (340) to the DASH logic (310) for further processing.

[0050] In response to an application (390) periodically retrieving a particular event, the synchronizer / dispatcher (340) can communicate application event instances (i.e., timed metadata samples) corresponding to the event schemes periodically retrieved by the application. The event instances can be communicated according to a dispatch mode indicated by a subscription request (e.g., a subscription request for a particular event scheme) or a default dispatch mode. For example, in a dispatch-on-receive mode, event instances can be sent to the application (390) as they are received in the buffer (335). Meanwhile, in a dispatch-on-start mode, event instances can be sent to the application (390) at their associated presentation times, e.g., synchronized with a timing signal from the media decoder (355).

[0051] DASH / CMAF media track switching DASH and CMAF support multiple switchable media tracks, which may be provided for the same media content, but with different bit rates to support different resolutions and different transmission bandwidth conditions. In a media streaming session, a client (e.g., a DASH client or a CMAF client) may choose to switch from one track to another, for example, to adapt to specific bandwidth conditions, the client's allocated bandwidth resources, etc.

[0052] In some implementations, a media track may also be described as a media representation.

[0053] Figure 4 shows an example of a DASH data model. As shown in Figure 4, adaptation set 3 contains four representations, each representing a different track with a different bitrate. The bitrate of representation 2 is 2 Mbps (megabits per second), and representation 2 is formed by one or more media segments. Typically, in the DASH data model shown in Figure 4, the smallest media slice unit is a "segment."

[0054] Figure 5 shows an example of a CMAF data model. As shown in Figure 5, switching set 3 includes four CMAF tracks, each representing a different bitrate. CMAF track 2 has a bitrate of 2 Mbps and is formed by one or more chunks. Typically, in the CMAF data model shown in Figure 5, the smallest media slice unit is a "chunk."

[0055] DASH / CMAF Addressable Resource Index In some implementations, it is desirable for an adaptive streaming client (e.g., a DASH client or a CMAF client) to have precise knowledge of Addressable Resource Index (ARI) information that describes all the details of addressable resources and subsets of a CMAF Switching Set, e.g., as defined in ISO / IEC 23000-19, in a metadata track. The ARI information may also describe all the detailed subsets of a DASH adaptation set. The ARI information may include the offset, size, duration, and quality of timed and ordered segments or chunks that reside in the same adaptation set / switching set. Such ARI information, along with relative information about, e.g., the nearest chunk or segment, may be used by a DASH / CMAF client to facilitate client heuristics. Addressable resources may include track files, segments, or chunks in a CMAF context. In the case of an on-demand service, a precise map of such information may be provided by a segment index. Note that similar concepts and implementations may also be applied to DASH contexts.

[0056] In some implementations, the ARI information may be carried in ARI samples in an ARI track or in ARI events.

[0057] In some implementations, the Addressable Resource Index (ARI) may be defined as follows: Sample entry type: 'cari' Container: Sample Description Box(stsd) Required: Not required Quantity: 0 or 1

[0058] The metadata describes all details of addressable resources and, for example, a subset of the CMAF Switching Set defined in ISO / IEC 23000-19 in one index track.

[0059] Table 1 below shows a typical sample entry in CMAF Addressable Resource Index metadata.

[0060] [Table 1]

[0061] Table 2 below shows the typical syntax of an ARI sample.

[0062] [Table 2]

[0063] Typically, the semantics of the above syntax are as described below. · switching_set_identifier specifies the unique identifier of the switching set in the application context. ·num_tracks indicates the track number assigned in the ARI track. · track_ID selects and sequences among samples of a track using the track_ID. ·num_quality_indicators specifies the number of quality indicators used to identify the quality of the chunk. quality_identifier specifies the identifier that indicates how the quality values ​​in the sample are to be interpreted. This is a 4CC code that can be registered. · segment_start_flag indicates whether the chunk is the start of a segment. ·marker specifies whether this chunk contains at least one styp box. ·SAP_type specifies the SAP type of the chunk. emsg_flag indicates whether this chunk provides at least one emsg box. ·prft_flag indicates whether this chunk contains at least one prft box. offset specifies the offset of the chunk from the beginning of the segment. ·size gives the size of the chunk in octets. quality gives the quality of the chunk according to a given quality scheme identifier. The data type of the quality value (integer or floating point) is determined by the quality scheme. If the quality scheme identifier is an empty string, the quality is an unsigned integer, interpreted proportionally with increasing values ​​indicating increasing quality. ·loss indicates that the chunk's media data was lost. num_prediction_pairs gives the number of pairs of expected predictions to provide. prediction_min_windows provides the minbuffer time value that is the same as the MPD value. predicted_max_bitrate provides a bandwidth value identical to the MPD semantic that is maintained for the duration of the prediction_min_windows value.

[0064] Transporting ARI using events In an implementation example in the case of DASH / CMAF, a dedicated metadata track, i.e., an ARI track, is created to carry ARI-related information such as offset, size, quality, etc. of timed and ordered segments or chunks that are in the same adaptation set / switching set, so that the client has relative information about the nearest chunks or segments to facilitate client heuristics, which can be used by the client, for example, when dynamically switching media tracks or representations.

[0065] Note that one of the disadvantages of using metadata tracks to carry ARI information (e.g., ARI samples) is the excessive signaling overhead: for example, an extra HTTP GET Request is required by the client for each segment that requires ARI information.

[0066] Embodiments of the present disclosure include a method for carrying ARI (i.e., ARI information, ARI samples) without using an ARI metadata track. That is, rather than using a metadata track to carry ARI (which would incur an extra HTTP GET request (as the ARI samples are sent separately with the media segments / chunks)), the present disclosure provides that ARI samples may be sent via events, such as in-band events or MPD events. This approach for carrying ARI samples is considered to be "ARI transmission associated with a media segment / chunk" because the ARI samples are sent together with the media segments / chunks. An event carrying ARI is referred to as an ARI event. Using an ARI event can provide at least the following advantages:

[0067] 1. No extra metadata track is required, resulting in one less HTTP GET request by the CMAF / DASH client for each segment / chunk for which additional ARI information is needed. For example, a CMAF / DASH client may need additional ARI information to facilitate processing a segment / chunk. In this case, the ARI information can be obtained directly from the ARI event carried with the segment / chunk.

[0068] 2. The event processing model allows for the processing of event messages and dispatching them to DASH / CMAF clients. The processing model allows for the timing of ARI samples to be carried as part of the event timing model.

[0069] 3. Flexibility: ARI information may be carried by events in one, some, or all representations in a DASH adaptation set or a CMAF switching set, depending on the in-band events required.

[0070] 4. Adaptability and portability: ARI events can be parsed by a packager (eg, obtained from in-band events or ARI tracks received from an encoder) and added to the MPD as MPD events.

[0071] In some implementations, the ARI information for a chunk / segment can be included in the same chunk / segment.

[0072] In some implementations, the ARI information for a chunk / segment may be included in subsequent chunks / segments arranged in time.

[0073] In some implementations, rather than using in-band events to carry ARI information, MPD events may be used to carry ARI information, which may be particularly suitable for on-demand content.

[0074] In some implementations, ARI information may be carried in emsg boxes, each of which may belong to an event scheme defined by or associated with a scheme URN identifier.

[0075] Table 3 below shows example parameters for an ARI event in an MPD.

[0076] [Table 3]

[0077] As shown in Table 3, two elements, EventStream and InbandEventStream, may be used to describe an ARI event. Both streams may contain a value attribute. The value attribute may carry the CmafAriMetaDataSampleEntry fields described in Table 1. For example, a CmafAriMetaDataSampleEntry field may contain the following fields: switching_set_identifier num_tracks num_quality_indicators · A numbered list of track_ids List of quality_identifiers

[0078] In some implementations, an event element may include a presentationTime attribute (eg, Event@presentationTime), which indicates the chunk offset from the beginning of the period to which the event's ARI information applies.

[0079] In some implementations, the event element may include a duration attribute (e.g., Event@duration), which indicates the duration for which the ARI information should be used. For example, this may include the duration of a chunk or the duration of a segment.

[0080] In some implementations, an event may include an event body, which may share the same structure as a CmafAriFormatStruct, as defined in Table 2.

[0081] Table 4 below shows an example of the emsg parameter for an inband ARI event.

[0082] [Table 4] Note that the event body in an MPD event and the message_data in an inband event share the same CMAF ARI sample structure, CmafAriFormatStruct. Therefore, the parsing and processing of the ARI sample after receiving the event from the event dispatcher is the same. In other words, the same parsing and processing logic is shared for MPD events and inband events.

[0083] In some embodiments, ARI events may be processed and dispatched, for example, according to section A.13 of ISO / IEC 23009-1, for example, in a typical DASH / CMAF client architecture as shown in FIG.

[0084] In some embodiments, after an ARI event is dispatched, it is post-processed, which may rely on the parameters shown in Table 5.

[0085] [Table 5] This disclosure describes various embodiments for enhancing media track switching by using ARI events or ARI track samples. For example, an ARI event (or ARI track sample) may carry size, quality, and offset information for some or all aligned chunks (parallel chunks) of some or all tracks in the same switching set / adaptation set. CMAF / DASH clients may use the information carried in the ARI track sample or ARI event to switch to another track / representation at the associated chunk boundary.

[0086] Figure 6 shows an example of switching media tracks at the segment / chunk level. As shown in Figure 6, there are three tracks / representations, which represent, for example, different bit rates. The temporal positions of chunks in each track are aligned with the temporal positions of corresponding chunks in other tracks. For example, the temporal positions of chunks C1 in each track are aligned. Such temporally aligned chunks in different tracks may be called parallel chunks. For example, all C1 chunks are parallel chunks, and all C2 chunks are parallel chunks. In Figure 6, a typical chunk-level switching is performed in the following manner:

[0087] After C1 610, it switches to C2 612, after C3 614 it switches to C4 616, after C4 616 it switches to C5 618.

[0088] Note that in Figure 6, track switching occurs at the level of the smallest media data unit supported by DASH or CMAF. For example, the unit may be a CMAF chunk or a DASH segment. Switching at other levels, such as the DASH representation level or the CMAF track level, may also be supported.

[0089] When switching tracks / representations, there are two important points in time with respect to the decision to switch. The first is when the client makes the decision to switch, and the second is when the switch occurs (e.g., from which chunk). For example, the decision to switch may be made at the beginning of a chunk (such as the C1 chunk in Figure 6), at the end of a chunk, or in the middle of a chunk. For example, the decision may be to switch to track 2 starting from the C2 chunk (and therefore the C2 chunk is the switch point), or to switch to track 3 starting from the C1 chunk (and therefore the C1 chunk is the switch point), or to switch to track 2 starting from the C3 chunk (and therefore the C3 chunk is the switch point).

[0090] In some embodiments, a decision to switch made at chunk i is a decision to switch at chunk i+n, where i and n are non-negative integers.

[0091] In some embodiments, switching auxiliary information may be carried in ARI events or ARI track samples, and the DASH / CMAF client may use the most recent available auxiliary information to make a decision about switching to a different track / representation.

[0092] In some embodiments, the auxiliary information for each track / representation may include: The offset and size of the current chunk (or next chunk). The quality of the current chunk (or next chunk). Quality may include media resolution.

[0093] In some embodiments, the auxiliary information may be implicit and may be used by a DASH / CMAF client to derive the switch location.

[0094] In some embodiments, the auxiliary information may be explicit information, for example, it may carry explicit notification of switch locations (e.g., chunks and their corresponding tracks), and the DASH / CMAF client can switch simply by following the auxiliary information.

[0095] In this disclosure, a DASH / CMAF client may use several different approaches for switching tracks / representations during a streaming session. In one example, the client may determine that a switch is needed at the next chunk / segment (e.g., the current chunk is C1 and switch at C2), or the client may determine that a switch is needed at the next n-th chunk / segment (e.g., the current chunk is C1 and switch at C4). In another example, the client may determine that an immediate switch is needed for the current chunk. An early switch decision may be advantageous in that the client can begin requesting / receiving / buffering media data early. An immediate switch decision may be desirable, however, when there is a need to quickly adapt to current bandwidth conditions. The client may use the appropriate approach as needed.

[0096] In some embodiments, if a client is streaming a representation with chunk transitions, the client is not only getting chunks / segments as they are streaming, but also getting ARI tracks or ARI events. For example, a DASH / CMAF client may receive ARI samples via a track separate from the media tracks, or a DASH / CMAF client may receive ARI events multiplexed with or embedded in the media chunks.

[0097] 7 to 9 show examples of timing when a DASH / CMAF client receives ARI information. The ARI information may carry only switching assistance information, or may carry other information together with the switching assistance information. For ease of explanation and illustration, these figures show only the timing between the encoder and the client. transfer It is assumed that there is zero latency. Under this assumption, the packager output (i.e., chunks / segments, their ARI samples, and chunks / segments with events embedded therein) is available to the client as soon as it is prepared by the packager. Note that the same basic principles will apply even when transition latency is taken into account. In the following embodiments, the media unit "chunk" is used for illustrative purposes. The embodiments also apply to other media units such as segments.

[0098] In Figures 7-9, vertical dotted lines at each time point, T1, T2, etc., indicate that information is available.

[0099] In some embodiments, the ARI information supporting bandwidth switching is carried in an ARI track, e.g., via ARI samples, as shown in Figure 7. Note that an ARI track is a different track from the media tracks that contain media chunks or media segments.

[0100] As shown in FIG. 7, the ARI information (e.g., location, size, and quality) of chunk C1 carried by ARI sample 1 is available at T1 (received by the DASH / CMAF client). The client can then use the ARI information of C1 (or the ARI information associated with C1) to make a decision to switch to C2. That is, at T1, based on the C1 ARI information, the client can decide to switch before receiving the C2 chunk (i.e., the chunk following C1). Because the ARI information is the ARI information for the received chunk C1 or the ARI information associated with the received chunk C1, and the switching decision is a decision regarding the next chunk, this method is referred to as extrapolated switching. That is, the switching decision is based on the ARI information of past chunks for switching in the next chunk, and therefore the switching decision is based on estimation / prediction. The ARI information is used to determine switching in future chunks, and the characteristics of the future chunks are estimated or predicted. For example, the ARI information of C1 is used to estimate / predict the characteristics of chunk C2, and a switching decision is made based on the ARI information, causing the client to switch to another track for chunk C2.

[0101] In some implementations, once the decision to switch is made, the client may begin receiving and / or buffering the desired chunk, the C2 chunk.

[0102] In some implementations, ARI sample 1 in Figure 7 may carry switching assistance information (location, size, and quality) for some or all of the parallel chunks (e.g., the C1 chunks of all tracks shown in Figure 6). For example, the ARI information may carry switching assistance information for all parallel C1 chunks.

[0103] In addition to the above-described extrapolation switching solution, an embodiment of the present disclosure further provides an interpolation switching solution when an ARI track is used. As shown in FIG. 7, the ARI information carried by ARI sample 2 for the C2 chunk becomes available to the client at T2. That is, the client needs to wait until T2 to obtain the ARI information for C2. Based on the C2 ARI information, the client may then make a decision to switch during C2 (i.e., the switch point is the C2 chunk, i.e., switch to a parallel C2 chunk in another track). In this case, the ARI information in the C2 chunk itself is used by the client to make a decision on whether to switch during the C2 chunk, and the ARI information is accurate. In this case, the switching decision is based on more accurate information than in the extrapolation solution, where the switching decision is based on estimation from one or more previous chunks.

[0104] Typically, the ARI information carried by ARI Sample 2 may be for one or more parallel C2 chunks, or may be for all parallel C2 chunks.

[0105] By comparison, in both the extrapolation switching method and the interpolation switching method described above, switching is selected in the C2 chunk. In the extrapolation switching method, the switching decision is made in T1 based on estimation / prediction from the ARI information in the C1 chunk. The estimation / prediction may be made by further referring to at least one of the following: Current network conditions, such as the bandwidth available to the DASH / CMAF client, Current playback requirements such as media quality, media size and media offset. Using inference / prediction from the ARI information of the C1 chunk, the DASH / CMAF client can make a decision as to whether a switch is necessary and, if so, which C2 chunk to select when switching.

[0106] On the other hand, in interpolative switching, the switching decision is made at T2 based on the ARI information in the C2 chunk, which is accurate to use for the switching decision. Note that the switching decision may also be made based on the ARI information with reference to the current network conditions and / or the current playback requirements.

[0107] Thus, extrapolated switching can achieve an earlier switch decision (i.e., decision at T1 before T2), but the decision is based on estimation / prediction. Interpolated switching can achieve a more accurate switch decision, but the decision is later, for example, by one chunk (i.e., decision at T2 one chunk later than T1). Once the switch decision is made, the client may start requesting / receiving / buffering the selected chunk.

[0108] In some implementations, according to an interpolation switching method, the client may first receive an ARI sample (eg, the ARI sample of chunk C2) and then receive the corresponding chunk one chunk later.

[0109] In some embodiments, bandwidth switching / track switching is performed based on ARI events (no lag). As shown in Figure 8, ARI information supporting bandwidth switching (i.e., switching auxiliary information) is carried in one or more ARI events that are multiplexed with or embedded in one or more chunks.

[0110] As shown in FIG. 8, switching auxiliary information (e.g., location, size, and quality) for chunk C1 carried by ARI event 1 (shown at 810) is available to (received by) the client at T1. Note that because the event is "inband" with chunk C1, both event 1 and chunk C1 are received by the DASH / CMAF client at T1. In this case, the extrapolated switching method described above may be used. That is, based on the ARI information of chunk C1, the client may make a switching decision using prediction / estimation, further referring to the current network conditions and / or current playback requirements. If switching to C2 is desired, the client may also select a specific C2 chunk from among the parallel C2 chunks.

[0111] Note that the interpolative switching method is not suitable in this scenario. In the case of interpolative switching, an ARI event in the current chunk is supposed to be used to indicate (directly or indirectly) a switch in the current chunk, but since the chunk has already been received, it is too late to perform the switch. For example, when ARI event 1 is received, chunk C1 is also received, so the earliest time to perform the switch is T2.

[0112] In some embodiments, bandwidth switching / track switching is performed based on ARI events with a lag. As shown in Figure 9, ARI information supporting bandwidth switching (i.e., switching auxiliary information) is carried in one or more ARI events delayed by one chunk. In this case, chunk i has an ARI event for chunk i+1, where i is an integer. An example will be described below with reference to Figure 9.

[0113] In Figure 9, ARI event 2 (denoted by 910) carries the switching auxiliary information for the C2 chunk, and ARI event 2 can be compiled only after the C2 chunk is ready from the encoder side. Therefore, there is a delay of one chunk on the packager side. That is, instead of packaging chunk C1 in parallel (i.e., synchronously) with the encoder output, the packager must wait one chunk duration until the C2 chunk is generated by the encoder and ARI event 2 is compiled based on the C2 chunk, after which ARI event 2 is multiplexed with the C1 chunk. Note that in this case, the current chunk, such as C1, carries the ARI event of the subsequent chunk (e.g., chunk i carries the ARI event of chunk i+n, where i and n are integers).

[0114] Figure 9 illustrates the delay pattern described above. As shown in Figure 9, for the same chunk, the packager's timeline has shifted by one chunk to the right.

[0115] In FIG. 9, in addition to the C1 chunk, switching auxiliary information in the C2 chunk can be used at T2.

[0116] In some implementations, an extrapolated switching method may be used to make a switching decision. For example, at T2, based on the switching auxiliary information in the C2 chunk carried in the C1 chunk, the client may use prediction / estimation to make a switching decision on whether to switch in the C3 chunk, further referring to the current network conditions and / or current playback requirements. If switching is desired, the client may also select a specific C3 chunk from among parallel C3 chunks. When using the extrapolated switching method, the switching decision is based on past chunks to switch in the next chunk (i.e., a future chunk), e.g., based on the C1 chunk to switch in the C3 chunk.

[0117] In some implementations, an interpolation switching method may be used to make the switching decision. For example, at T2, a switching decision as to whether to switch at the C2 chunk may be made based on the switching auxiliary information of the C2 chunk carried in the C1 chunk, with further reference to the current network conditions and / or current playback requirements. If switching is desired, the client may also select a specific C2 chunk from among the parallel C2 chunks. When an interpolation switching method is used, the switching decision is based on accurate switching auxiliary information for the chunk to switch to. For example, if the switching auxiliary information carried in the C1 chunk is information for the C2 chunk, the switching auxiliary information is accurate for making the switching decision about switching at the C2 chunk. Note that if the switching decision is made at T2, then the C2 chunk has not yet been fetched.

[0118] 10 illustrates an exemplary method 1000 for processing media streams. The media streams may include, for example, 4G media streams (for media streams delivered over a 4G network) and 5G media streams (for media streams delivered over a 5G network). The method may be performed, for example, by a computer system (described below), a client device (which may be part of or integrated into an encoder and / or decoder), or the like. The media streams may comply with the DASH standard or the CMAF standard. The method 1000 may include some or all of the following steps: receiving 1010 media stream data comprising a plurality of media chunks including a first media chunk and a second media chunk and Addressable Resource Index (ARI) information associated with the first media chunk; determining 1020 track change information based on the ARI information; determining 1030 that a switch to a different media track in the second media chunk is required based on the track change information; and receiving 1040 the first media chunk and the second media chunk via their respective media tracks.

[0119] In some implementations, in step 1040, upon receiving the first media chunk and the second media chunk via their respective media tracks, each of the first media chunk and the second media chunk is delivered to the streaming client device with a delivery delay of one chunk or less.

[0120] In some implementations, the ARI information may include or be conveyed via one or more ARI samples or one or more ARI events. Method 1000 may further include the ARI information comprising at least one of an ARI sample obtained from an ARI track associated with a first media slice in a first media track of the media stream, or an ARI event associated with a media stream embedded in a first media slice in the first media track of the media stream.

[0121] In some implementations, the method 1000 may further include receiving one of an Addressable Resource Index (ARI) sample obtained from an ARI track associated with a first media slice in a first media track of the media stream or an ARI event associated with the media stream embedded in the first media slice in the first media track of the media stream, wherein the ARI event provides characteristic information of at least one of the first media slice in the first media track and a first parallel media slice in another media track of the media stream that is aligned in position with the first media slice, or a second media slice in the first media track that follows the first media slice and a second parallel media slice in another media track of the media stream that is aligned in time position with the second media slice.

[0122] The embodiments of the present disclosure not only apply to both DASH and CMAF, but also to other media streaming technologies by applying similar basic principles.

[0123] The embodiments of the present disclosure may be used individually or combined in any order. A method of the present disclosure, such as method 1000 above, may include all or only some of the enumerated steps. Furthermore, each of the methods (i.e., embodiments), the DASH client, and the CMAF client may be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored on a non-transitory computer-readable medium. The embodiments of the present disclosure may be applied to the DASH and / or CMAF technologies / standards. Typically, each of the methods (or embodiments) may be performed by a DASH / CMAF client, which may be executed on a computing device having processing circuitry. For example, the client may be executed on an encoder and / or decoder.

[0124] The techniques described above can be implemented using computer-readable instructions and as computer software physically stored on one or more computer-readable media. For example, Figure 11 illustrates a computer system (1800) suitable for implementing certain embodiments of the disclosed subject matter.

[0125] Computer software can be coded using any suitable machine or computer language, which may be processed by assembly, compilation, linking, or similar mechanisms to generate code comprising instructions that may be executed directly, interpreted, executed by microcode, or the like, by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc.

[0126] The instructions may be executed by various types of computers or components thereof including, for example, personal computers, tablet computers, servers, smartphones, gaming consoles, Internet of Things devices, and the like.

[0127] 11 of the computer system (1800) are exemplary of course and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing the embodiments of the present disclosure. Furthermore, the arrangement of components should not be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary embodiment of the computer system (1800).

[0128] The computer system (1800) may include several human interface input devices. Such human interface input devices may respond to input by one or more human users through, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, clapping), visual input (e.g., gestures), and olfactory input (not shown). The human interface devices may also be used to capture media not necessarily directly associated with conscious human input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images captured from a still image camera), and video (e.g., two-dimensional video, three-dimensional video, including stereoscopic video).

[0129] The input human interface devices may include one or more (only one of each is shown) of a keyboard (1801), a mouse (1802), a trackpad (1803), a touch screen (1810), a data glove (not shown), a joystick (1805), a microphone (1806), a scanner (1807), and a camera (1808).

[0130] The computer system 1800 may also include several human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses through, for example, tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touchscreen (1810), data gloves (not shown), or joystick (1805), although haptic feedback devices that are not used as input devices may also be present), audio output devices (e.g., speakers (1809), headphones (not shown)), visual output devices (e.g., screens (1810), including CRT screens, LCD screens, plasma screens, and OLED screens, each with or without touchscreen input capability and with or without haptic feedback capability. Some of the above screens may be capable of outputting two-dimensional visual output, or three-dimensional or higher output through means such as stereographic output. Virtual reality glasses (not shown), holographic displays, smoke generator tanks (not shown), and printers (not shown) may also be included.

[0131] The computer system (1800) may also include storage devices and associated media that can be directly operated by humans, such as optical media (1821) including CD / DVD ROM / RW (1820) using CD / DVD or similar media, thumb drives (1822), removable hard drives or solid state drives (1823), legacy magnetic media such as tape or floppy disks (not shown), and specialized ROM / ASIC / PLD-based devices (not shown) such as security dongles.

[0132] Those skilled in the art will also appreciate that the term "computer-readable medium" as used in connection with the subject matter disclosed herein does not include transmission media, carrier waves, or other transitory signals.

[0133] The computer system 1800 may also include an interface 1854 to one or more communications networks 1855. The networks may be, for example, wireless, wired, or optical. Furthermore, the networks may be local, wide-area, metropolitan, vehicular, and industrial, real-time, delay-tolerant, etc. Examples of networks include local area networks such as Ethernet; cellular networks including WLAN, GSM, 3G, 4G, 5G, LTE, etc.; television wired or wireless wide-area digital networks including cable, satellite, and terrestrial television; vehicular and industrial networks including CAN bus; etc. Some networks require external network interface adapters that attach in a common manner to some general-purpose data port or peripheral bus 1849 (e.g., a USB port on the computer system 1800), while other networks are commonly integrated into the core of the computer system 1800 by attaching to the system bus, as described below (e.g., an Ethernet interface for a PC computer system or a cellular network interface for a smartphone computer system). Using any of these networks, the computer system 1800 can communicate with others. Such communication can be one-way, receive-only (e.g., television broadcasts), one-way transmit-only (e.g., a CANbus to a specific CANbus device), or bidirectional, e.g., bidirectional communication with other computer systems using local or wide-area digital networks. Several protocols and protocol stacks can be used for each of these networks and network interfaces described above.

[0134] The above-mentioned human interface devices, storage devices that can be directly operated by humans, and network interfaces can be attached to the core (1840) of the computer system (1800).

[0135] The core (1840) may include one or more central processing units (CPUs) (1841), graphics processing units (GPUs) (1842), specialized programmable processing units in the form of field programmable gate arrays (FPGAs) (1843), task-specific hardware accelerators (1844), graphics adapters (1850), etc. Such devices may be connected through a system bus (1848) along with read-only memory (ROM) (1845), random access memory (1846), and internal mass storage (1847) such as a hard drive or SSD that is not directly user-accessible. In some computer systems, the system bus (1848) may be directly accessible in the form of one or more physical plugs that allow expansion with additional CPUs, GPUs, etc. Peripherals may be attached directly to the core's system bus (1848) or via a peripheral bus (1849). In one example, a screen (1810) may be connected to the graphics adapter (1850). Peripheral bus architectures include PCI, USB, etc.

[0136] The CPU (1841), GPU (1842), FPGA (1843), and accelerator (1844) can execute several instructions that can be combined to make up the above-mentioned computer code. The computer code can be stored in ROM (1845) or RAM (1846). Transient data can also be stored in RAM (1846), while persistent data can be stored, for example, in internal mass storage (1847). Cache memory, which can be closely associated with one or more of the CPU (1841), GPU (1842), mass storage (1847), ROM (1845), RAM (1846), etc., can be used to enable fast storage and retrieval from any of the memory devices.

[0137] The computer-readable medium may carry computer code for performing various computer-implemented operations, and the medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those having skill in the computer software arts.

[0138] As a non-limiting example, a computer system having architecture (1800), and in particular core (1840), can provide functionality as a result of one or more processors (including CPUs, GPUs, FPGAs, accelerators, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be media associated with mass storage directly accessible by a user, as described above, or can even be specific storage of the core (1840), such as the core's internal mass storage (1847) or non-transitory storage, such as ROM (1845). Software implementing various embodiments of the present disclosure can be stored on such devices and executed by the core (1840). The computer-readable media can include one or more memory devices or chips, depending on particular requirements. Software enables the cores (1840), and in particular the processors (including CPUs, GPUs, FPGAs, etc.) in the cores (1840), to execute certain processes or certain portions of certain processes described herein, including defining data structures stored in RAM (1846) and modifying such data structures in response to the processes defined by the software. Additionally or alternatively, logic may be hardwired into circuitry (e.g., accelerators (1844)) that can act in place of or cooperate with software to execute certain processes or certain portions of certain processes described herein, or the computer system may provide functionality as a result of logic being otherwise embodied. Where appropriate, references to "software" may encompass logic, and vice versa. Where appropriate, references to "computer-readable medium" may encompass circuitry (e.g., integrated circuits (ICs)) that stores software to be executed, circuitry that implements the logic to be executed, or both. The present disclosure includes any appropriate combination of hardware and software.

[0139] While this disclosure has described several exemplary embodiments, there are alterations, permutations, and various substitute equivalents that are within the scope of this disclosure. Thus, it will be appreciated that those skilled in the art will be able to devise numerous systems and methods that embody the principles of the present disclosure and are therefore within its spirit and scope, even if not explicitly shown or described in this application. [Explanation of symbols]

[0140] 100 system, 110 content server, 120 information processing device, 200 DASH system, 210 content server, 212 HTTP server, 214 MPD generator, 220 advertising server, 230 information processing device, 232 DASH client, 234 application, 236 display device, 250 network, 305 manifest parser, 310 DASH logic, 315 DASH access API, 320 HTTP protocol stack, 325 inband event 'moof' parser, 330 timed metadata track parser, 335 event-timed metadata buffer, 340 dispatcher module, 345 file format parser, 350 media buffer, 355 media decoder, 390 application, 610 C1 chunk, 612 C2 chunk, 614 C3 chunk, 616 C4 chunk, 618 C5 chunk, 810 switching auxiliary information, 910 ARI event 2, 1000 Method, 1800 Computer system, 1801 Keyboard, 1802 Mouse, 1803 Trackpad, 1805 Joystick, 1806 Microphone, 1807 Scanner, 1808 Camera, 1809 Audio output device / speaker, 1810 Touch screen, 1820 CD / DVD ROM / RW, 1821 Optical medium, 1822 Thumb-drive, 1823 Solid-state drive, 1840 Core, 1841 CPU, 1842 GPU, 1843 FPGA, 1844 Hardware accelerator, 1845 ROM, 1846 RAM, 1847 Mass storage, 1848 System bus, 1849 Peripheral bus, 1850 Graphics adapter, 1854 Interface, 1855 Communication network

Claims

1. 1. A method for processing a media stream, performed by a streaming client device, the media stream comprising at least two media tracks including a first media track and a second media track, and conforming to the Dynamic Adaptive Streaming over HTTP (DASH) standard or the Common Media Application Format (CMAF), the method comprising: receiving media stream data comprising a plurality of media chunks including a first media chunk and a second media chunk, and Addressable Resource Index (ARI) information associated with the first media chunk; determining track switch information based on the ARI information; determining, based on the track switch information, that a switch from the first media track to the second media track is required in the second media chunk; receiving the first media chunk and the second media chunk via the first media track and the second media track, respectively, wherein each of the first media chunk and the second media chunk is delivered to the streaming client device with a delivery delay of 0 or 1 chunk due to switching between media tracks on a media chunk-by-media chunk basis.

2. The ARI information is an ARI sample obtained from an ARI track associated with a first media slice in a first media track of said media stream; or 10. The method of claim 1, comprising at least one ARI event associated with the media stream embedded in the first media slice in the first media track of the media stream.

3. the media stream conforms to the DASH standard and the first media slice is a segment in the media stream; or the media stream conforms to the CMAF format, and the first media slice is a chunk in the media stream; The method of claim 2.

4. the first media slice in the first media track and a first parallel media slice in another media track of the media stream, the first media slice being aligned with the first media slice; or a second media slice in the first media track that follows the first media slice, and a second parallel media slice in the other media track of the media stream that is aligned in time with the second media slice; The method according to claim 2 or 3, wherein the ARI information provides at least one characteristic information of the above.

5. The characteristic information is the offset of the first media slice of the first media track; the size of the first media slice of the first media track; an offset of each of the first parallel media slices; a size of each of the first parallel media slices; the quality of the first media slice of the first media track; or The method of claim 4 , comprising at least one of the qualities of each of the first parallel media slices.

6. determining, based on the track change information, that the switch to the different media track in the second media chunk is necessary, 5. The method of claim 4, further comprising determining that the switching to one of the other media tracks is necessary based on the characteristic information and bandwidth available to the streaming client device.

7. receiving the ARI event, wherein the ARI event provides characteristic information about the first media slice in the first media track and a first parallel media slice in another media track of the media stream that is temporally aligned with the first media slice; The step of determining whether the switching to one of the other media tracks is necessary comprises: selecting one of the second parallel media slices based on an estimation using the characteristic information; and and switching to one of the second parallel media slices after the first media slice. The method of claim 4.

8. 8. The method of claim 7, wherein the media stream is encoded and transmitted in real time, and the ARI event is constructed without regard to the second media slice and the second parallel media slice.

9. receiving the ARI event, wherein the ARI event provides characteristic information about the second media slice in the first media track that follows the first media slice and a second parallel media slice in the other media track of the media stream that is aligned in time with the second media slice; The step of determining whether the switching to one of the other media tracks is necessary comprises: selecting one of the second concurrent media slices based on the characteristic information; and switching to one of the second parallel media slices after the first media slice. The method of claim 4.

10. receiving the ARI event, wherein the ARI event provides characteristic information about the second media slice in the first media track that follows the first media slice and a second parallel media slice in the other media track of the media stream that is aligned in time with the second media slice; The step of determining whether the switching to one of the other media tracks is necessary comprises: selecting a target media slice next to one of the second parallel media slices based on estimation using the characteristic information; and switching to the target media slice after the second media slice. The method of claim 4.

11. The method of claim 10 , wherein the media stream is encoded and transmitted in real time, and the ARI event is constructed taking into account the second media slice and the second parallel media slice.

12. receiving an ARI sample, the ARI sample providing characteristic information about the first media slice in the first media track and the first parallel media slice in another media track of the media stream that is aligned in time with the first media slice; The step of determining whether the switching to one of the other media tracks is necessary comprises: selecting one of the second concurrent media slices based on estimation using the characteristic information; and switching to one of the second parallel media slices after the first media slice. The method of claim 4.

13. receiving the ARI sample, the ARI sample providing characteristic information about the first media slice in the first media track and the first parallel media slice in another media track of the media stream that is aligned in time with the first media slice; The step of determining whether the switching to one of the other media tracks is necessary comprises: selecting one of the first concurrent media slices based on the characteristic information; and switching to one of the first concurrent media slices. The method of claim 4.

14. 10. A device comprising a memory storing computer instructions and a processor in communication with said memory, said processor, when executing said computer instructions, causing said device to perform the method of claim 1.

15. A computer program comprising computer readable instructions which, when executed by a processor, cause the processor to perform the method of claim 1.

Images