Novel DASH client processing model for supporting handling of DASH event duplication

Abstract

A method and apparatus, the apparatus comprising computer code configured to cause one or more processors to perform: obtaining identifier values for a plurality of events of a media file in a DASH; determining whether a plurality of identifier values of the identifier values of the plurality of events have the same value; scheduling a first DASH event of the plurality of events from the DASH client to a DASH application, the scheduling of the first DASH event further comprising: scheduling the first DASH event based on determining that a second DASH event of the plurality of events is planned as an event to be scheduled and the second DASH event has the same value as the first DASH event, scheduling the first DASH event and the second DASH event together; and, based on at least scheduling the first DASH event and the second DASH event, controlling the DASH application.

Description

[0001] Reference Merge

[0002] This application claims priority to U.S. Provisional Application No. 63 / 526,157, filed on July 11, 2023, and Application No. 18 / 765,907, filed on July 8, 2024, the entire contents of which are expressly incorporated herein by reference. Technical Field

[0003] The present invention provides a processing model for a DASH client to handle repeated DASH events. Background Art

[0004] ISO Base Media File Format (ISOBMF) ISOBMFF is a widely used file format for media content. The Common Media Application Format (CMAF) standard defines common media format tracks that can be grouped into switch sets. CMAF switch sets are used to deliver media with alternate tracks that represent the same content but have different properties, such as bitrate, resolution, frame rate, and other possible characteristics.

[0005] The ISO / IEC 23009-1 DASH standard allows streaming of multi-bitrate content. The standard specification includes the transmission of MPD and in-band events, and provides a client processing model for these events.

[0006] MPEG DASH provides a standard for streaming multimedia content over IP networks. Recently, MPEG DASH has considered adding a mechanism to signal updated events. However, repeated events have not yet been defined, nor are there corresponding processing models.

[0007] Therefore, for any of the reasons mentioned above, there is a need for technical solutions to these problems arising in computer audio technology. Summary of the invention

[0008] In order to solve these technical problems, a method and an apparatus are proposed, wherein the apparatus includes a memory configured to store computer program code and one or more processors configured to access the computer program code and run according to the instructions of the computer program code. The computer program is configured to enable one or more processors to implement acquisition code, determination code, scheduling code and control code for a dynamic adaptive streaming media (DASH) application based on a hypertext transfer protocol (HTTP) implemented by one or more processors; the acquisition code is configured to enable at least one processor to acquire identifier values ​​of multiple events in a media file in DASH; the determination code is configured to enable at least one processor to determine whether multiple identifier values ​​in the identifier values ​​of multiple events have the same value; the scheduling code is configured to enable at least one processor to schedule a first DASH event among multiple events from a DASH client to a DASH application, wherein scheduling the first DASH event also includes: scheduling the first DASH event and the second DASH event together based on determining that a second DASH event among the multiple events is planned as an event to be scheduled and the second DASH event has the same value as the first DASH event; and the control code is configured to enable at least one processor to control the DASH application based on at least scheduling the first DASH event and the second DASH event.

[0009] The same value may include at least one of a scheme and an identifier in DASH.

[0010] Scheduling the first DASH event and the second DASH event together may include scheduling the second DASH event using the syntax “@status=repeat”.

[0011] The syntax "@status=repeat" may be provided by a DASH client as an indication of a Media Presentation Description (MPD) repeat event.

[0012] Scheduling the first DASH event together with the second DASH event includes: scheduling the second DASH event using an emsg box with a flag syntax “&2=1”.

[0013] Scheduling the first DASH event together with the second DASH event may include scheduling the second DASH event using an emsg box with a flag syntax “&2=1” provided by the DASH client as an indication of an in-band repeating event.

[0014] The same value may be at least of the syntax "scheme_id_uri". BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Further features, properties and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:

[0016] Figure 1 is a simplified schematic diagram of a computer environment according to an embodiment.

[0017] Figure 2 is a simplified schematic diagram of media processing according to an embodiment.

[0018] Figure 3 is a simplified block diagram of a decoder according to an embodiment.

[0019] Figure 4 is a simplified block diagram of an encoder according to an embodiment.

[0020] Figure 5 is a simplified block diagram of a CMAF feature according to an embodiment.

[0021] Figure 6 is a simplified block diagram of a DASH environment over HTTP according to an embodiment.

[0022] Figure 7 is a simplified diagram of event message features according to an embodiment.

[0023] Figure 8 is a simplified flow chart according to an embodiment.

[0024] Fig. 9 is a simplified flow chart according to an embodiment.

[0025] Fig.10 is a simplified flow chart according to an embodiment.

[0026] Fig.11 is a simplified flow chart according to an embodiment.

[0027] Fig.12 is a simplified block diagram of a computer environment according to an embodiment. DETAILED DESCRIPTION

[0028] The features presented in the following description may be used alone or in any order. In addition, the present application embodiments may be implemented by processing circuits (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-volatile computer-readable medium.

[0029] Figure 1A simplified block diagram of a communication system 100 according to an embodiment of the present disclosure is shown. The communication system 100 includes at least two terminals 102 and 103 interconnected via a network 105. For unidirectional data transmission, a third terminal 103 can encode video data at a local location for transmission to another terminal 102 via the network 105. The second terminal 102 can receive the encoded video data of the other terminal from the network 105, decode the encoded data and display the recovered video data. Unidirectional data transmission is common in media service applications and the like.

[0030] Figure 1 A first terminal 101 and a fourth terminal 104 are shown provided to support bidirectional transmission of encoded video, which may occur, for example, during a video conference. For bidirectional data transmission, each of the first terminal 101 and the fourth terminal 104 may encode video data captured at a local location for transmission to the other of the first terminal 101 and the fourth terminal 104 via a network 105. Each of the first terminal 101 and the fourth terminal 104 may also receive encoded video data transmitted by the other of the first terminal 101 and the fourth terminal 104, may decode the encoded data, and may display the restored video data on a local display device.

[0031] exist Figure 1 In, the first terminal 101, the second terminal 102, the third terminal 103 and the fourth terminal 104 can be shown as a server, a personal computer and a smart phone, but the principle of the present application is not limited thereto. The embodiments of the present application are applied to laptop computers, tablet computers, media players and / or dedicated video conferencing equipment. Network 105 represents any number of networks that transmit encoded video data between the first terminal 101, the second terminal 102, the third terminal 103 and the fourth terminal 104, including, for example, wired (wired) and / or wireless communication networks. Communication network 105 can exchange data in circuit switching and / or packet switching channels. Representative networks include telecommunication networks, local area networks (LAN), wide area networks and / or the Internet. For the purpose of this discussion, unless explained below, the architecture and topology of network 105 may be insignificant for the operation of the present application.

[0032] As examples of applications of the disclosed subject matter, Figure 2 The placement of the video encoder and the video decoder in a streaming environment is shown. The subject matter disclosed in the present application is equally applicable to other video-supported applications, including, for example, video conferencing, digital TV, storing compressed video on digital media including compact discs (CDs), digital video discs (DVDs), memory sticks, etc.

[0033] The streaming system may include an acquisition subsystem 203, which includes a video source 201, such as a digital camera, which creates, for example, an uncompressed video sample stream 213. The sample stream 213 is emphasized as a high-data-volume sample stream compared to an encoded video bitstream and may be processed by an encoder 202 coupled to the camera 201. The encoder 202 includes hardware, software, or a combination of hardware and software to implement or implement various aspects of the disclosed subject matter as described in more detail below. The encoded video bitstream 204 is emphasized as a lower-data-volume encoded video bitstream compared to a sample stream, which may be stored on a streaming server 205 for future use. One or more streaming clients 212 and 207 may access the streaming server 205 to retrieve a copy 208 and a copy 206 of the encoded video bitstream 204. Client 212 may include a video decoder 211 that decodes an incoming copy of the encoded video bitstream 208 and produces an output video sample stream 210 that may be displayed on a display 209 or other display device (not depicted). In some streaming systems, video bitstream 204, video bitstream 206, and video bitstream 208 may be encoded according to certain video codec / compression standards. Examples of these standards are mentioned above and will be further described herein.

[0034] Figure 3 may be a functional block diagram of a video decoder 300 according to an embodiment of the present invention.

[0035] Receiver 302 may receive one or more coded video sequences to be decoded by decoder 300. In the same or another embodiment, one coded video sequence is received at a time, wherein the decoding of each coded video sequence is independent of the other coded video sequences. The coded video sequence may be received from channel 301, which may be hardware / software leading to a storage device storing coded video data. Receiver 302 may receive coded video data as well as other data, such as coded audio data and / or auxiliary data streams that may be forwarded to their respective consuming entities (not shown). Receiver 302 may separate the coded video sequence from the other data. To prevent network jitter, buffer memory 303 may be coupled between receiver 302 and entropy decoder / parser 304 (hereinafter referred to as "parser"). Buffer memory 303 may also not be needed, or may be made smaller, when receiver 302 receives data from a storage / forward device with sufficient bandwidth and controllability or from an isochronous network. For use on traffic packet networks such as the Internet, a buffer memory 303 may also be required, which may be relatively large and may advantageously have an adaptive size.

[0036] The video decoder 300 may include a parser 304 to reconstruct symbols 313 from an entropy coded video sequence. The categories of these symbols include information for managing the operation of the decoder 300, and potentially information for controlling a display device such as a display screen 312, which is not part of the decoder but may be coupled to the decoder. The control information for one or more display devices may be in the form of a Supplementary Enhancement Information (SEI) or a Video Usability Information (VUI) parameter set fragment (not shown). The parser 304 may parse / entropy decode the received coded video sequence. The encoding of the coded video sequence may be performed according to a video coding technique or standard, and may follow various criteria known to those skilled in the art, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser 304 may extract a subgroup parameter set for at least one of the pixel subgroups in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. Subgroups may include Groups of Pictures (GOP), pictures, tiles, slices, macroblocks, Coding Units (CU), blocks, Transform Units (TU), Prediction Units (PU), etc. The entropy decoder / parser may also extract information such as transform coefficients, quantizer parameter values, motion vectors (MV), etc. from the encoded video sequence.

[0037] The parser 304 may perform an entropy decoding / parsing operation on the video sequence received from the buffer memory 303, thereby creating the symbol 313. The parser 304 may receive the encoded data and selectively decode a specific symbol 313. In addition, the parser 304 may determine whether the specific symbol 313 is to be provided to the motion compensation prediction unit 306, the scaler / inverse transform unit 305, the intra prediction unit 307, or the loop filter 311.

[0038] Depending on the type of the coded video picture or a portion of the coded video picture (e.g., inter- and intra-pictures, inter- and intra-blocks), and other factors, the reconstruction of the symbol 313 may involve a plurality of different units. Which units are involved and how they are involved may be controlled by subgroup control information parsed from the coded video sequence by the parser 304. For the sake of brevity, such subgroup control information flow between the parser 304 and the plurality of units below is not described.

[0039] In addition to the functional blocks already mentioned, decoder 300 may be conceptually subdivided into several functional units as described below. In a practical implementation operating under commercial constraints, many of these units interact closely with each other and may be at least partially integrated with each other. However, for the purpose of describing the disclosed subject matter, it is appropriate to conceptually subdivide into the functional units below.

[0040] The first unit is a sealer / inverse transform unit 305. The sealer / inverse transform unit 305 receives quantized transform coefficients as one or more symbols 313 from the parser 304, as well as control information, including which transform method to use, block size, quantization factor, quantization scaling matrix, etc. The sealer / inverse transform unit may output a block including sample values, which may be input into the aggregator 310.

[0041] In some cases, the output samples of the scaler / inverse transform unit 305 may belong to an intra-coded block, i.e., a block that does not use predictive information from a previously reconstructed picture, but may use predictive information from a previously reconstructed portion of the current picture. Such predictive information may be provided by the intra-picture prediction unit 307. In some cases, the intra-picture prediction unit 307 generates a block of the same size and shape as the block being reconstructed using surrounding reconstructed information extracted from the current (partially reconstructed) picture 309. In some cases, the aggregator 310 adds the prediction information generated by the intra-prediction unit 307 to the output sample information provided by the scaler / inverse transform unit 305 on a per-sample basis.

[0042] In other cases, the output samples of the sealer / inverse transform unit 305 may belong to an inter-frame coded and potentially motion compensated block. In this case, the motion compensated prediction unit 306 may access the reference picture memory 308 to extract samples for prediction. After the extracted samples are motion compensated according to the symbol 313 belonging to the block, these samples may be added to the output of the sealer / inverse transform unit (in this case referred to as residual samples or residual signal) by the aggregator 310, thereby generating output sample information. The motion compensation unit's retrieval of predicted samples from an address in the reference picture memory may be controlled by a motion vector, and the motion vector is provided to the motion compensation unit in the form of the symbol 313, which includes, for example, X, Y and reference picture components. Motion compensation may also include interpolation of sample values ​​extracted from the reference picture memory when using sub-sample accurate motion vectors, motion vector prediction mechanisms, and the like.

[0043] The output samples of aggregator 310 may be employed by various loop filtering techniques in loop filter unit 311. The video compression techniques may include in-loop filter techniques that are controlled by parameters included in the coded video bitstream and available to loop filter unit 311 as symbols 313 from parser 304. The video compression techniques may also be responsive to meta-information obtained during decoding of a coded picture or a previous (in decoding order) portion of a coded video sequence, and to previously reconstructed and loop filtered sample values.

[0044] The output of the loop filter unit 311 may be a sample stream that may be output to the display device 312 and stored in the reference picture memory 557 for subsequent inter-picture prediction.

[0045] Once fully reconstructed, certain coded pictures may be used as reference pictures for future prediction. Once a coded picture is fully reconstructed, and the coded picture is identified as a reference picture (e.g., by parser 304), current reference picture 309 may become part of reference picture buffer 308, and new current picture memory may be reallocated before starting reconstruction of a subsequent coded picture.

[0046] The video decoder 300 may perform decoding operations according to a predetermined video compression technique in a standard such as ITU-T Rec. H.265. In the sense that the encoded video sequence follows the syntax of the video compression technique or standard specified in the video compression technology document or standard (especially in its profile), the encoded video sequence may conform to the syntax specified by the video compression technology or standard used. For compliance, the complexity of the encoded video sequence is also required to be within the range defined by the hierarchy of the video compression technology or standard. In some cases, the hierarchy limits the maximum picture size, the maximum frame rate, the maximum reconstruction sampling rate (measured in, for example, mega samples per second), the maximum reference picture size, etc. In some cases, the limits set by the hierarchy may be further defined by the Hypothetical Reference Decoder (HRD) specification and the metadata of the HRD buffer management signaled in the encoded video sequence.

[0047] In an embodiment, the receiver 302 may receive additional (redundant) data along with the encoded video. The additional data may be part of one or more encoded video sequences. The additional data may be used by the video decoder 300 to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0048] Figure 4 It may be a functional block diagram of a video encoder 400 according to an embodiment of the present disclosure.

[0049] Encoder 400 may receive video samples from a video source 401 (not part of the encoder), which may capture one or more video pictures to be encoded by encoder 400 .

[0050] The video source 401 may provide a source video sequence in the form of a digital video sample stream to be encoded by the encoder 303, and the digital video sample stream may have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit ...), any color space (e.g., BT.601 Y CrCB, RGB ...), and any suitable sampling structure (e.g., Y CrCb4:2:0, Y CrCb 4:4:4). In a media service system, the video source 401 may be a storage device that stores previously prepared videos. In a video conferencing system, the video source 401 may be a camera that collects local image information as a video sequence. The video data may be provided as a plurality of separate pictures that are given motion when viewed in sequence. These pictures themselves may be constructed as a spatial array of pixels, wherein each pixel may include one or more samples depending on the sampling structure, color space, etc. used. The relationship between pixels and samples may be easily understood by those skilled in the art. The following focuses on describing samples.

[0051] According to an embodiment, the encoder 400 may encode and compress the pictures of the source video sequence into an encoded video sequence 410 in real time or under any other time constraints required by the application. It is a function of the controller 402 to implement the appropriate encoding speed. The controller controls other functional units as described below and is functionally coupled to these units. For the sake of brevity, the coupling is not indicated in the figure. The parameters set by the controller may include rate control related parameters (picture skipping, quantizer, lambda value of rate distortion optimization technology, etc.), picture size, GOP layout, maximum MV search range, etc. Those skilled in the art can easily identify other functions of the controller 402 because they may be related to the video encoder 400 optimized for a specific system design.

[0052] Some video encoders operate in what is readily recognizable to those skilled in the art as a "coding loop". As a very simplified description, the coding loop may include an encoding portion of an encoder (hereinafter referred to as a source encoder) (responsible for creating symbols based on an input picture to be encoded and one or more reference pictures) and a (local) decoder 406 embedded in the encoder 400. The decoder 406 reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data (because in the video compression techniques contemplated by the disclosed subject matter, any compression between the symbols and the encoded video bitstream is lossless). The reconstructed sample stream is input to a reference picture memory 405. Since the decoding of the symbol stream produces bit-accurate results that are independent of the decoder location (local or remote), the contents of the reference picture buffer also correspond bit-accurately between the local encoder and the remote encoder. In other words, the reference picture samples "seen" by the prediction portion of the encoder are exactly the same sample values ​​that the decoder will "see" when using prediction during decoding. This basic principle of reference picture synchronization (and the drift that results when synchronization cannot be maintained, for example due to channel errors) is also known to those skilled in the art.

[0053] The operation of the "local" decoder 406 may be combined with, for example, Figure 3 The same is true of the "remote" decoder 300 described in detail. However, brief reference is made to Figure 4 , when symbols are available and the entropy encoder 408 and the parser 304 are capable of losslessly encoding / decoding the symbols into an encoded video sequence, the entropy decoding portion of the decoder 300, including the channel 301, the receiver 302, the buffer 303 and the parser 304, may not be fully implemented in the local decoder 406.

[0054] At this point it can be observed that any decoder techniques other than parsing / entropy decoding present in the decoder need to be present in the corresponding encoder in substantially identical functional form. The description of encoder techniques can be simplified because these encoder techniques are reciprocal to the decoder techniques described comprehensively. A more detailed description is only required in certain areas and is provided below.

[0055] As part of its operation, source encoder 403 may perform motion compensated predictive coding. Motion compensated predictive coding predictively encodes an input frame with reference to one or more previously encoded frames from a video sequence designated as "reference frames." In this manner, encoding engine 407 encodes the differences between pixel blocks of an input frame and pixel blocks of one or more reference frames that may be selected as one or more prediction references for the input frame.

[0056] The local video decoder 406 can decode the encoded video data of those frames that can be designated as reference frames based on the symbols created by the source encoder 403. The operation of the encoding engine 407 can advantageously be a lossy process. When the encoded video data is available at the video decoder ( Figure 4 When decoded at a remote video decoder (not shown), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder 406 replicates the decoding process that may be performed by the video decoder on the reference frame and may cause the reconstructed reference frame to be stored in the reference picture cache 405. In this way, the encoder 400 may locally store a copy of the reconstructed reference frame that has common content (absent transmission errors) with the reconstructed reference frame to be obtained by the remote video decoder.

[0057] The predictor 404 may perform a prediction search for the encoding engine 407. That is, for a new frame to be encoded, the predictor 404 may search the reference picture memory 405 for sample data (as a candidate reference pixel block) or certain metadata, such as a reference picture motion vector, block shape, etc., that may serve as a suitable prediction reference for the new picture. The predictor 404 may operate pixel-by-pixel based on the sample block to find a suitable prediction reference. In some cases, based on the search results obtained by the predictor 404, it may be determined that the input picture may have a prediction reference taken from a plurality of reference pictures stored in the reference picture memory 405.

[0058] The controller 402 may manage encoding operations of the video encoder 403 including, for example, setting parameters and subgroup parameters for encoding video data.

[0059] The outputs of all the above functional units may be entropy encoded in the entropy encoder 408. The entropy encoder 408 performs lossless compression on the symbols generated by the various functional units according to techniques known to those skilled in the art, such as Huffman coding, variable length coding, arithmetic coding, etc., thereby converting the symbols into a coded video sequence.

[0060] Transmitter 409 may buffer one or more encoded video sequences created by entropy encoder 408 in preparation for transmission via communication channel 411, which may be hardware / software access to a storage device storing the encoded video data. Transmitter 409 may combine the encoded video data from video encoder 403 with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0061] The controller 402 may manage the operation of the encoder 400. During encoding, the controller 405 may assign a specific coded picture type to each coded picture, but this may affect the coding techniques that may be applied to the corresponding picture. For example, a picture may generally be assigned to any of the following picture types.

[0062] An intra picture (I picture) may be a picture that can be encoded and decoded without using any other frame in the sequence as a prediction source. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh pictures. Those skilled in the art will appreciate the variations of I pictures and their corresponding applications and features.

[0063] A predictive picture (P picture) may be a picture that can be encoded and decoded using intra prediction or inter prediction, which uses at most one motion vector and a reference index to predict sample values ​​for each block.

[0064] Bidirectional predictive pictures (B pictures), which can be pictures that can be encoded and decoded using intra prediction or inter prediction, which uses up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for reconstructing a single block.

[0065] The source picture may typically be spatially subdivided into blocks of samples (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples), and coded block by block. These blocks may be predictively coded with reference to other (already coded) blocks, which are determined according to the coding allocation applied to the block's corresponding picture. For example, a block of an I picture may be non-predictively coded, or the block may be predictively coded (spatial prediction or intra prediction) with reference to an already coded block of the same picture. A block of a P picture may be non-predictively coded by spatial prediction with reference to one previously coded reference picture or by temporal prediction. A block of a B picture may be non-predictively coded by spatial prediction with reference to one or two previously coded reference pictures or by temporal prediction.

[0066] The video encoder 400 may perform encoding operations according to a predetermined video encoding technique or standard, such as ITU-T Rec. H.265. In operation, the video encoder 400 may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in an input video sequence. Thus, the encoded video data may conform to the syntax specified by the video encoding technique or standard used.

[0067] In an embodiment, the transmitter 409 may transmit additional data when transmitting the encoded video. The source encoder 403 may include such data as part of the encoded video sequence. The additional data may include other forms of redundant data such as time / space / SNR enhancement layers, redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0068] Figure 5 An example 500 of CMAF tracks and CMAF switch sets defined by CMAF using a standard such as ISO / IEC JTC 1 / SC 29 / WG03N00654 is shown according to an exemplary embodiment. A CMAF switch is a set of CMAF tracks with some common constraints. The main purpose of a CMAF switch set is to provide alternative representations of the same content in multiple tracks so that during delivery or playback, a player can switch between tracks to adapt to changes in network bandwidth and other changing properties. For track format, the CMAF standard will use ISOBMFF, but it does not provide a standard to signal the presence of a CMAF switch set in an ISOBMFF file.

[0069] The embodiments herein introduce a new version of the ISOBMGG track selection box with unique properties. The new box contains several parameters for signaling CMAF switching sets and their properties.

[0070] According to some embodiments, there is a version 1 switching group that can (i) use a switching group id to identify the switching group, (ii) replace the group id (for a selection list), (iii) have a switchable group id, (iv) track the group id for pre-selected association relationships, and (v) indicate CMAF parameters.

[0071] The embodiments herein provide the following definitions:

[0072] Box type: 'tsel'

[0073] Container: UserDataBox of the corresponding TrackBox

[0074] Mandatory: No

[0075] Quantity: zero or one

[0076] Such a track selection box is contained in the user data box of the track it modifies.

[0077] Figure 6 An example 600 of a metadata track sample for an adaptation set segment index for any given adaptation set is shown. For example, for each adaptation set (AS) that is expected to signal an instantaneous segment bandwidth, a separate adaptation set may also be included in the file, such as Figure 6 shown.

[0078] like Figure 6 As shown, for AS i having k media representations (their segments are time-aligned), a new adaptation set AS index is added to the file containing a single representation. The single representation is a timed metadata track whose segments are also time-aligned with the segments of the AS i representation.

[0079] Figure 7 An example DASH client processing model 700 is shown, such as an example DASH client processing model for an example client architecture for processing DASH and CMAF events, in which a client request for a media segment can be based on an address described in a file that also describes a metadata track from which the client can access segments of the metadata track, parse the segments and send them to an application. In addition, according to an exemplary embodiment of the addresses of the media segments described below, the DASH file can provide addresses for index segments. Each index segment can provide information about the duration and size of a segment, and the representation index can provide index information for all segments of a given representation.

[0080] as Figure 7 , the client requests the media segment based on the address described in the file, and as shown in the lower part 701 of the example model 700, the MSE buffer contains the pipeline of the file format parser, media buffer and media decoder. In addition, the exemplary embodiment uses the following attributes and flags to apply MPD and in-band update events, as shown in Table 1:

[0081] Table 1 - Event semantics

[0082]

[0083]

[0084]

[0085] According to an example embodiment, an event with @status = 'update' is an updated instance of an earlier event with the same @schemeIdUri, @value, and @id attributes, which may have been previously processed by the DASH client. If the previous event has not been scheduled, the DASH client may replace the previous event with the updated instance. An event with @status = 'update' may differ from the previous event except for the following attributes: @schemeIdUri, @value, and @id. See Figure 8Example 800 in. The flags field is specified as follows: (flags&1) is equal to 1, indicating that the esmg is an update of another esmg with the same values ​​of the scheme_id_uri field, value field, and id field.

[0086] An example embodiment provides a repeated event as an "MPD repeated event", which is an event (Event) with @status = 'repeat', which is a repeated instance of an earlier event with the same values ​​of @schemeIdUri, @value and @id attributes, which may have been previously processed by the DASH client. The DASH client should schedule repeated events even if the DASH client has already scheduled an earlier event. An "Inband repeated event" is an emsg box with "flags&2 = 1", which is a repeated instance of an emsg box with the same scheme_id_uri field, value field and id field, which may have been previously processed by the DASH client. The DASH client should schedule repeated events even if the DASH client has already scheduled an earlier event.

[0087] The above statements define a precise definition of a repeating event that can be checked by a DASH client so that repeating events can be detected unambiguously.

[0088] Table 2 below provides an example of signaling the repeated event. The following syntax is used to signal the repeated event.

[0089] Table 2 - Event semantics with repeating event additions

[0090]

[0091]

[0092]

[0093]

[0094] According to an embodiment, an event with @status = 'repeat' is a repeated instance of an earlier event with the same @schemeIdUri, @value, and @id attributes, which may have been previously processed by the DASH client. The DASH client is expected to schedule this event instance even if the earlier event has a newer instance and even if the previous event has been scheduled. Such features represent the syntax and semantics for repeating events according to an embodiment.

[0095] According to an embodiment, for in-band repetition events, the flag field in the emsg box is specified as follows: (flags&2) is equal to 1, indicating that the esmg is a repetition of another esmg with the same values ​​of the scheme_id_uri field, value field and id field.

[0096] An emsg box with flags&2=1 is a repeated instance of an emsg box with the same scheme_id_uri field, value field, and id field, which may have been previously processed by the DASH client. The DASH client expects to schedule this event even if a previous event has been scheduled.

[0097] In addition, for the processing model for repeated events, the embodiment extends the DASH client event processing model to accurately handle repeated events. The improved processing model is described as follows.

[0098] Assuming an application subscribes to a specific event stream identified by a (scheme / value) pair with a specific dispatch_mode as described in DASH subclause A.13.7 or similar, i.e., on-start or on-receive, the processing model varies depending on the value of dispatch_mode.

[0099] Fig. 9 Example 900 shows a general process according to an embodiment of the present application, wherein the DASH client implements the following process. At S901, for a dispatch_mode (dispatch_mode) of on start, the DASH client creates a pending event table (PET). At S902, in the case of dispatch_mode = on_start, for each subscribed scheme_uri / (value), the PET maintains a single list of event ids waiting to be scheduled. At S903, for each subscribed scheme_uri / (value), the DASH client also creates a dispatched event table (Dispatched Event Table, DET). According to an embodiment, the DET maintains a single list of scheduled "emsg" ids.

[0100] At S904, parsing the "emsg" / timing metadata sample and retrieving the scheme_uri / (value) is achieved. At S905, if the application has not subscribed to the scheme_uri / (value) pair, the processing of the "emsg" ends. At S906, the ST of the event instance / metadata sample is also exported. At S907, the end time ET = ST + DU is also exported.

[0101] Event presentation / start time (ST), which is the moment when the event becomes active on the media timeline. ST is the moment when the event becomes active on the media timeline. This value can be calculated using the parameters included in the Dash event message box (DashEventMessageBox). Event duration (DU): The duration for which the event remains active. DU is signaled using a specific value in the event message box. According to an embodiment, ET is the event time.

[0102] Fig.10 Example 1000 of the on-receive processing according to an exemplary embodiment is shown, where, at S1001, when the scheduling_mode = on_receive, the DASH client implements the following process. At S1002, if the current presentation time value is greater than ET, the processing ends at S1004; otherwise, at S1003, in the case of an event, if the event is a repetition of another event, refer to S1006; otherwise, at S1005, the id of the event is compared with the entries of the DET having the same scheme_uri / (value) pair, and if there is an entry with the same id value, the processing ends at S1004. Otherwise, at S1006, according to the description in DASH sub-clause A.13.6, the event / timing metadata (including ST, id, DU, time scale, and message_data) is scheduled and the event is added to the DET.

[0103] In addition, Fig.11An example 11100 of on-start processing according to an exemplary embodiment is shown, wherein, at S1101, when scheduling_mode=on_start, the DASH client implements the following process. At S1102, if the event is an update of a previous event (which is signaled by the @status or emsg flag), then at S1103, any existing event (if any) with the same scheme_uri / (value) and id is deleted from the PET, which is not a repeated event. At S1104, the ST of the event instance / metadata sample is derived. At S1105, if the current media presentation time value is less than ST, go to S1108. At S1107, the end time ET=ST+DU is derived. At S1108, if the current presentation time value is greater than ET, the processing is terminated at S1106. However, at S1109, in the case of an event, if the event is a repeated event, go to S1111. Otherwise, at S1110, the id of the event is compared with an entry of the PET of the same scheme_uri / (value) pair, and if an entry with the same id value exists, the process ends at S1106. However, if not, at S1111, the id of 'emsg' is added to the corresponding PET. At S1112, the event / metadata message_data is scheduled at time ST, or, if the current presentation time is greater than ST, the event is immediately deleted from the PET (if present) as described in DASH subclause A.13.6, and the event is added to the DET.

[0104] Therefore, a method is provided for defining a plurality of repeated DASH events, each DASH repeated event comprising an event in response to the same corresponding identifier value, each DASH repeated event providing the same information as a previous DASH event, the DASH client processing model scheduling the repeated DASH event after the DASH client processing model has previously scheduled the previous DASH event. And, in response to the DASH client not yet scheduling but planning to schedule the previous DASH event, for each of the planned previous DASH event and the repeated DASH event, the DASH client processing model will schedule the previous DASH event and the repeated DASH event at a corresponding time. The identifier value may include at least one of the following: a scheme, a value, and an identifier.

[0105] Therefore, the embodiments provided in the present application provide a precise definition for repeated events and extend the DASH client event processing model to monitor repeated events and perform appropriate scheduling for repeated events.

[0106] The above techniques may be implemented as computer software that uses computer-readable instructions and is physically stored in one or more computer-readable media or implemented by one or more specially configured hardware processors. Fig.12 A computer system 1200 suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0107] The computer software may be encoded in any suitable machine code or computer language, and may be assembled, compiled, linked, or the like to create a code comprising instructions, which may be directly executed by a computer central processing unit (CPU), graphics processing unit (GPU), or the like, or executed by decoding, microcode, or the like.

[0108] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablets, servers, smartphones, gaming devices, IoT devices, and the like.

[0109] Fig.12 The components shown for computer system 1200 are exemplary in nature and are not intended to limit the scope of use or functionality of computer software implementing embodiments of the present application. The configuration of the components should not be interpreted as having any dependency or requirement on any component or combination of components shown in the exemplary embodiment of computer system 1200.

[0110] Computer system 1200 may include certain human-computer interface input devices. Such human-computer interface input devices may respond to input from one or more human users through tactile input (e.g., keyboard input, sliding, data glove movement), audio input (e.g., sound, applause), visual input (e.g., gestures), and olfactory input (not shown). The human-computer interface device may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., speech, music, ambient sound), images (e.g., scanned images, photographic images obtained from a still image camera), and videos (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0111] The human-machine interface input device may include one or more of the following (only one of which is depicted): keyboard 1201 , mouse 1202 , touch pad 1203 , touch screen 1210 , joystick 1205 , microphone 1206 , scanner 1208 , camera 1207 .

[0112] The computer system 1200 may also include certain human-computer interface output devices. Such human-computer interface output devices may stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human-computer interface output devices may include tactile output devices (e.g., tactile feedback through a touch screen 1210 or a joystick 1205, but there may also be tactile feedback devices that are not used as input devices), audio output devices (e.g., speakers 1209, headphones (not shown)), visual output devices (e.g., screens 1210 including cathode ray tube (CRT) screens, liquid crystal screens, plasma screens, organic light emitting diode screens, each of which has or does not have a touch screen input function, each of which has or does not have a tactile feedback function - some of which may output two-dimensional visual outputs or outputs of more than three dimensions by means such as stereoscopic picture output; virtual reality glasses (not shown), holographic displays, and smoke boxes (not shown)) and printers (not shown).

[0113] The computer system 1210 may also include human-accessible storage devices and their associated media, such as optical media including high-density read-only / rewritable optical disks (CD / DVD ROM / RW) 1220 or similar media with CD / DVD 1211, a thumb drive 1222, a removable hard disk drive or solid state drive 1223, traditional magnetic media such as tapes and floppy disks (not shown), special-purpose devices based on ROM / Application-Specific Integrated Circuit (ASIC) / Programmable Logic Device (PLD) such as a security software protector (not shown), and the like.

[0114] Those skilled in the art should also understand that the term "computer-readable media" used in connection with the disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0115] The computer system 1200 may also include an interface 1299 to one or more communication networks 1298. For example, the network 1298 may be wireless, wired, or optical. The network 1298 may also be a LAN, a wide area network, a metropolitan area network, an in-vehicle network, an industrial network, a real-time network, a delay-tolerant network, and the like. The network 1298 also includes LANs such as Ethernet, wireless LANs, cellular networks (Global System for Mobile communications (GSM), 3G, 4G, 5G, Long-Term Evolution (LTE), etc.), television wired or wireless wide-area digital networks (including cable television, satellite television, and terrestrial broadcast television), in-vehicle networks, and industrial networks (including controller area network buses (CANBus)), etc. Some networks 1298 typically require external network interface adapters for connecting to certain universal data ports or peripheral buses (1250 and 1251) (e.g., the Universal Serial Bus (USB) port of the computer system 1200). Other systems are typically integrated into the core of the computer system 1200 by connecting to a system bus such as the one described below (e.g., an Ethernet interface integrated into a PC computer system or a cellular network interface integrated into a smartphone computer system). Using any of these networks 1298, the computer system 1200 can communicate with other entities. The communication can be one-way, for reception only (e.g., wireless television), one-way, for transmission only (e.g., a CAN bus to certain CAN bus devices), or two-way (e.g., to other computer systems via a local or wide area digital network). Each of the above networks and network interfaces can use certain protocols and protocol stacks.

[0116] The above-mentioned human-machine interface devices, human-accessible storage devices, and network interfaces can be connected to the core 1240 of the computer system 1200.

[0117] The core 1240 may include one or more CPUs 1241, GPUs 1242, graphics adapters 1217, dedicated programmable processing units in the form of Field Programmable Gate Areas (FGPAs) 1243, hardware accelerators 1244 for specific tasks, etc. These devices, as well as read-only memory (ROM) 1245, random access memory 1246, internal mass storage (e.g., internal non-user accessible hard disk drive, solid state drive, etc.) 1247, etc., may be connected via a system bus 1248. In some computer systems, the system bus 1248 may be accessed in the form of one or more physical plugs so that it may be expanded by additional CPUs, GPUs, etc. Peripheral devices may be directly attached to the core's system bus 1248, or connected via a peripheral bus 1251. The architecture of the peripheral bus includes peripheral component interconnect (PCI), USB, etc.

[0118] The CPU 1241, GPU 1242, FPGA 1243, and accelerator 1244 may execute certain instructions, which in combination may constitute the above-mentioned computer code. The computer code may be stored in ROM 1245 or RAM 1246. Transitional data may also be stored in RAM 1246, while permanent data may be stored in, for example, internal mass storage 1247. Fast storage and retrieval of any memory device may be achieved by using a cache memory, which may be closely associated with one or more CPUs 1241, GPU 1242, mass storage 1247, ROM 1245, RAM 1246, and the like.

[0119] The computer readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be specially designed and constructed for the purpose of the present application, or may be the medium and code well known and available to those skilled in the art of computer software.

[0120] As an example and not a limitation, a computer system having architecture 1200, in particular core 1240, can be provided as a processor (including CPUs, GPUs, FPGAs, accelerators, etc.) to execute software contained in one or more tangible computer-readable media. Such computer-readable media can be media associated with the above-mentioned user-accessible mass storage, as well as a specific memory of the core 1240 having non-volatility, such as core internal mass storage 1247 or ROM 1245. Software implementing various embodiments of the present application can be stored in such a device and executed by the core 1240. According to specific needs, the computer-readable medium may include one or more storage devices or chips. The software can enable the core 1240, in particular the processor therein (including CPU, GPU, FPGA, etc.) to perform a specific process or a specific part of a specific process described herein, including defining a data structure stored in a random access memory (Random Access Memory, RAM) 1246 and modifying such a data structure according to a software-defined process. Additionally or alternatively, the computer system may provide logic hardwired or otherwise contained in circuitry (e.g., functionality of accelerator 1244) that may operate in place of or in conjunction with software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may include logic and vice versa. Where appropriate, references to computer-readable media may include circuits (e.g., ICs) storing executing software, circuits containing executing logic, or both. The present application includes any suitable combination of hardware and software.

[0121] Although the present application has described a number of exemplary embodiments, various changes, arrangements and various equivalent substitutions of the embodiments are within the scope of the present application. Therefore, it should be understood that those skilled in the art can design a variety of systems and methods, which, although not explicitly shown or described herein, embody the principles of the present application and are therefore within the spirit and scope of the present application.

Claims

1. A method, performed by at least one processor of a Dynamic Adaptive Streaming over Hypertext Transfer Protocol (HTTP) (DASH) client, the method comprising: Get the identifier values ​​of multiple events of a media file in DASH; determining whether a plurality of identifier values ​​of the plurality of events have the same value; Scheduling a first DASH event of the plurality of events from the DASH client to a DASH application, wherein the scheduling of the first DASH event further comprises: scheduling the first DASH event and the second DASH event together based on determining that a second DASH event of the plurality of events is planned as a to-be-scheduled event and the second DASH event has the same value as the first DASH event; and The DASH application is controlled based on scheduling at least the first DASH event and the second DASH event.

2. The method according to claim 1, wherein: The same value includes at least one of a scheme and an identifier in DASH.

3. The method according to claim 1, wherein: The scheduling the first DASH event and the second DASH event together includes: scheduling the second DASH event using the syntax "@status=repeat".

4. The method according to claim 3, wherein: The DASH client provides the syntax "@status=repeat" as an indication of a media presentation description (MPD) repeat event.

5. The method according to claim 1, wherein: The scheduling of the first DASH event and the second DASH event together includes: scheduling the second DASH event using an emsg frame with a flag syntax "&2=1".

6. The method according to claim 5, wherein: The scheduling of the first DASH event and the second DASH event together includes: scheduling the second DASH event using an emsg frame with a flag syntax "&2=1", wherein the flag syntax "&2=1" is provided by the DASH client and serves as an indication of an in-band repeated event.

7. The method according to claim 6, wherein: The same value is at least the syntax "scheme_id_uri".

8. An apparatus for a Dynamic Adaptive Streaming over Hypertext Transfer Protocol (HTTP) (DASH) client, the apparatus comprising: at least one memory configured to store computer program code; At least one processor is configured to access the computer program code and execute according to the instructions of the computer program code, wherein the computer program code comprises: Acquisition code configured to cause the at least one processor to acquire identifier values ​​of a plurality of events of a media file in DASH; determining code configured to cause the at least one processor to determine whether a plurality of identifier values ​​of the plurality of events have the same value; Scheduling code configured to cause the at least one processor to schedule a first DASH event among the multiple events from the DASH client to a DASH application, wherein the scheduling of the first DASH event further comprises: scheduling the first DASH event and the second DASH event together based on determining that a second DASH event among the multiple events is planned as a to-be-scheduled event and the second DASH event has the same value as the first DASH event; and Control code configured to cause the at least one processor to control the DASH application based on at least scheduling the first DASH event and the second DASH event.

9. The device according to claim 8, wherein: The same value includes at least one of a scheme and an identifier in DASH.

10. The device according to claim 8, wherein: The scheduling the first DASH event and the second DASH event together includes: scheduling the second DASH event using the syntax "@status=repeat".

11. The device according to claim 10, wherein: The DASH client provides the syntax "@status=repeat" as an indication of a media presentation description (MPD) repeat event.

12. The device according to claim 9, wherein: The scheduling of the first DASH event and the second DASH event together includes: scheduling the second DASH event using an emsg frame with a flag syntax "&2=1".

13. The device according to claim 12, wherein: The scheduling of the first DASH event and the second DASH event together includes: scheduling the second DASH event using an emsg frame with a flag syntax "&2=1", wherein the flag syntax "&2=1" is provided by the DASH client and serves as an indication of an in-band repeated event.

14. The device according to claim 13, wherein: The same value is at least the syntax "scheme_id_uri".

15. A non-transitory computer-readable storage medium storing a program for a Dynamic Adaptive Streaming over Hypertext Transfer Protocol (HTTP) (DASH) client, the program causing a computer to execute a process, the process comprising: Get the identifier values ​​of multiple events of a media file in DASH; determining whether a plurality of identifier values ​​of the plurality of events have the same value; Scheduling a first DASH event of the plurality of events from the DASH client to a DASH application, wherein the scheduling of the first DASH event further comprises: scheduling the first DASH event and the second DASH event together based on determining that a second DASH event of the plurality of events is planned as a to-be-scheduled event and the second DASH event has the same value as the first DASH event; and The DASH application is controlled based on scheduling at least the first DASH event and the second DASH event.

16. The non-transitory computer-readable storage medium of claim 15, wherein: The same value includes at least one of a scheme and an identifier in DASH.

17. The non-transitory computer-readable storage medium of claim 16, wherein: The scheduling the first DASH event and the second DASH event together includes: scheduling the second DASH event using the syntax "@status=repeat".

18. The non-transitory computer-readable storage medium of claim 17, wherein: The DASH client provides the syntax "@status=repeat" as an indication of a media presentation description (MPD) repeat event.

19. The non-transitory computer-readable storage medium of claim 18, wherein: The scheduling of the first DASH event and the second DASH event together includes: scheduling the second DASH event using an emsg frame with a flag syntax "&2=1".

20. The non-transitory computer-readable storage medium of claim 19, wherein: The scheduling of the first DASH event and the second DASH event together includes: scheduling the second DASH event using an emsg frame with a flag syntax "&2=1", wherein the flag syntax "&2=1" is provided by the DASH client and serves as an indication of an in-band repeat event; Wherein, the same value is at least the syntax "scheme_id_uri".

Images