Method, apparatus, and computer program for supporting prerolls and midrolls during media streaming and playback

Abstract

A method and apparatus may be provided for signaling an auxiliary media stream in a main media stream using an MSE buffer, comprising appending one or more main media segments to the MSE source buffer based on one or more respective main timestamp offsets associated with each of the one or more main media segments, and appending one or more auxiliary media segments to the MSE source buffer based on one or more auxiliary duration attributes associated with the one or more auxiliary media segments. The auxiliary duration attribute may be a period type element including a start time attribute indicating a first time offset at which a main media segment in the one or more main media segments may be stopped during a pre-roll or mid-roll insertion to play the auxiliary media segment in the one or more auxiliary media segments. The main media segments and the auxiliary media segments may be dispatched.

Description

Technical Field

[0001] [Cross - Reference to Related Applications] This application claims priority to U.S. Provisional Application No. 63 / 253,046, filed Oct. 6, 2021, and U.S. Application No. 17 / 949,684, filed Sep. 21, 2022, and incorporates these in their entirety by reference herein.

[0002] [Technical Field] Embodiments of the present disclosure are directed to streaming media content, and more specifically, to streaming media, advertisements, and live content according to MPEG (Moving Picture Experts Group)'s DASH (dynamic adaptive streaming over hypertext transfer protocol).

Background Art

[0003] MPEG DASH provides a standard for streaming media content over IP networks. In MPEG DASH, media presentation description (MPD) and events are used to deliver events related to the media timeline to the client. The ISO / IEC 23009 - 1 DASH standard enables streaming of multi - rate content. The DASH standard provides a single linear - type timeline where the periods are consecutive in a single timeline. ISO / IEC 23009 - 1 also provides tools for MPD chaining, i.e., signaling the URL of the next MPD to play in an MPD that can be used for preroll advertisement insertion.

[0004] MPEG DASH provides a standard for streaming multimedia content over IP networks. The standard addresses linear playback of media content, but it cannot handle non-linear operation scenarios where, for example, media segments are associated with different timelines that are independent of each other. MPD chaining and preroll ad insertion may be used to overcome the above drawbacks. However, when a DASH player uses W3C Media Source Extensions, it is very difficult to handle such non-linear playback with a single MSE source buffer, so even MPD chaining and preroll ad insertion fail. Further, when W3C Encrypted Media Extensions are used for playback of protected live content, MPD chaining and preroll ad insertion cannot be used when it is assumed that an ad will be played before using an MPD chaining or preroll element.

[0005] Therefore, a method for combining auxiliary content or independent content different from the main media content is needed. Specifically, a method and apparatus for combining auxiliary content and main media content as preroll playback or midroll playback are needed. Further, a method for handling W3C Encrypted Media Extensions for playing back protected live content using MPD chaining is also needed. Summary of the Invention

[0006] This disclosure addresses one or more technical problems. This disclosure includes a method, process, apparatus, and non-transitory computer-readable medium for implementing a new concept, an auxiliary period (AuxPeriod). The auxiliary period (also referred to as an auxiliary period attribute) enables presenting an auxiliary media or independent media presentation from a main media presentation. Further, embodiments of this disclosure also provide support for inserting preroll and midroll content during the main presentation.

[0007] Embodiments of this disclosure may provide a method for signaling an auxiliary media stream in a main DASH (Dynamic Adaptive Streaming over HTTP) media stream using a media source extension (MSE) buffer. The method may be executed by at least one processor and includes appending one or more main media segments to the MSE source buffer based on one or more respective main time stamp offsets associated with each of the one or more main media segments, and appending one or more auxiliary media segments to the MSE source buffer based on one or more auxiliary period attributes associated with the one or more auxiliary media segments, wherein the auxiliary period attribute of the one or more auxiliary period attributes may be a period type element that may include a start time attribute, and the start time attribute indicates a first time offset at which a main media segment in the one or more main media segments may be paused during preroll or midroll insertion for playing an auxiliary media segment in the one or more auxiliary media segments, and dispatching one or more main media segments and one or more auxiliary media segments within the MSE source buffer based on the one or more respective main time stamp offsets and the one or more auxiliary period attributes.

[0008] Embodiments of the present disclosure may provide an apparatus for signaling an auxiliary media stream in a main DASH (Dynamic Adaptive Streaming over HTTP) media stream using a media source extension (MSE) buffer. The apparatus may include at least one memory configured to store program code and at least one processor configured to access the program code and operate as instructed by the program code. The program code causes the at least one processor to append one or more main media segments to the MSE source buffer based on one or more respective main time stamp offsets associated with each of the one or more main media segments; a second append code configured to cause the at least one processor to append one or more auxiliary media segments to the MSE source buffer based on one or more auxiliary period attributes associated with the one or more auxiliary media segments, wherein the auxiliary period attribute of the one or more auxiliary period attributes is a period type element that may include a start time attribute, and the start time attribute indicates a first time offset at which a main media segment in the one or more main media segments is stopped during preroll or midroll insertion for playing an auxiliary media segment in the one or more auxiliary media segments; and dispatch code configured to cause the at least one processor to dispatch one or more main media segments and one or more auxiliary media segments in the MSE source buffer based on the one or more respective main time stamp offsets and the one or more auxiliary period attributes.

[0009] Embodiments of the present disclosure may provide a non-transitory computer-readable medium storing instructions. When the instructions are executed by one or more processors of a device for signaling an auxiliary media stream during a main DASH (Dynamic Adaptive Streaming over HTTP) media stream using a media source extension (MSE) buffer, the one or more processors are caused to append one or more main media segments to the MSE source buffer based on one or more respective main time stamp offsets associated with each of the one or more main media segments, cause at least one processor to append one or more auxiliary media segments to the MSE source buffer based on one or more auxiliary period attributes associated with the one or more auxiliary media segments, wherein the auxiliary period attribute of the one or more auxiliary period attributes is a period type element that may include a start time attribute, the start time attribute indicating a first time offset at which a main media segment in one or more main media segments is stopped during preroll or midroll insertion for playing an auxiliary media segment in the one or more auxiliary media segments, and may include one or more instructions for dispatching one or more main media segments and one or more auxiliary media segments within the MSE source buffer based on the one or more respective main time stamp offsets and the one or more auxiliary period attributes.

Brief Description of Drawings

[0010] Further features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

[0011] The proposed features discussed below may be used separately or in combination in any order. Further, embodiments may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-transitory computer-readable medium.

[0012] FIG. 1 shows a simplified block diagram of a communication system 100 according to an embodiment of the present disclosure. The communication system 100 includes at least two terminals 102 and 103 interconnected via a network 105. For one-way data transmission, the first terminal 103 may encode local video data for transmission to the other terminal 102 via the network 105. The second terminal 102 may receive the encoded video data of the other terminal from the network 105, decode the encoded data, and display the restored video data. One-way data transmission may be common in media delivery applications and the like.

[0013] FIG. 1 shows a second pair of terminals 101 and 104 provided to support bidirectional transmission of encoded video that may occur during a video conference, for example. For bidirectional transmission of data, each of terminals 101 and 104 may encode video data captured at a local location for transmission to other terminals via network 105. Also, each of terminals 101 and 104 may receive encoded video data transmitted by other terminals, may decode the encoded data, and may display the restored video data on a local display device.

[0014] In FIG. 1, terminals 101, 102, 103, and 104 may be shown as servers, personal computers, and smartphones, but the principles of the present disclosure are not limited thereto. Embodiments of the present disclosure are applicable to laptop computers, tablet computers, media players, and / or dedicated video conferencing devices. Network 105 represents any number of networks that transfer encoded video data between terminals 101, 102, 103, and 104, including, for example, wired and / or wireless communication networks. Network 105 may exchange data in a circuit-switched channel and / or a packet-switched channel. Representative networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For the purposes of this description, the architecture and topology of network 105 are not important to the operation of the present disclosure, unless otherwise described herein below.

[0015] FIG. 2 shows the arrangement of video encoders and decoders in a streaming environment. Embodiments are applicable to other video-enabled applications, including, for example, video conferencing, digital TV, storage of compressed video on digital media (including CDs, DVDs, memory sticks, etc.).

[0016] The streaming system may include a capture subsystem 203. For example, the capture subsystem 203 can include, for example, a digital camera that generates a stream 213 of uncompressed video pictures. The sample stream 213 may be emphasized as having a high data volume when compared to an encoded video bitstream and can be processed by an encoder 202 coupled to the video source 201. The encoder 202 can include hardware, software, or a combination thereof to enable or implement aspects of the embodiments, as will be described in more detail below. The encoded video data 204 may be emphasized as having a lower data volume when compared to the sample stream and can be stored in the streaming server 205 for future use. One or more streaming clients 212 and 207 can access the streaming server 205 to obtain the encoded video bitstreams 208 and 206, which may be copies 208 and 206 of the encoded video bitstream 204. The client 212 can include a video decoder 211, which decodes an input copy of the encoded video bitstream 208 and generates an output video sample stream 210 that can be rendered on a display 209 or other rendering device. In some streaming systems, the encoded video bitstreams 204, 206, and 208 can be encoded according to a particular video encoding / compression standard. Examples of these standards have been described above and will not be further elaborated here.

[0017] Figure 3 shows a sample DASH processing model 300, such as a sample client architecture for processing DASH and CMAF events. In the DASH processing model 300, requests for client media segments (e.g., advertisement media segments and live media segments) may be based on the addresses described in the manifest 303. The manifest 303 also describes metadata tracks that the client can access segments of, parse, and send to the application 301.

[0018] The manifest 303 includes MPD events or events, and the in-band event and "moof" parser 306 may parse MPD event segments or event segments and append the event segments to the event and metadata buffer 330. The in-band event and "moof" parser 306 may also obtain media segments and append them to the media buffer 340. The event and metadata buffer 330 may send event and metadata information to the event and metadata synchronizer and dispatcher 335. The event and metadata synchronizer and dispatcher 335 may dispatch specific events to the control, selection, and heuristic logic 302 of the DASH player and dispatch application-related events and metadata tracks to the application 301.

[0019] According to some embodiments, the MSE may include a pipeline that includes a file format parser 350, a media buffer 340, and a media decoder 345. The MSE 320 is a logical buffer for media segments, and the media segments may be tracked and ordered based on the presentation time of the media segments. The media segments may include, but are not limited to, advertisement media segments related to an advertisement MPD and live media segments related to a live MPD. Each media segment is added or appended to the media buffer 340 based on the time stamp offset of the media segment, and the time stamp offset may be used to order the media segments within the media buffer 340.

[0020] Embodiments of the present application may be directed to constructing a media source extension (MSE) buffer from two or more non-linear media sources using MPD chaining, and since the non-linear media sources may be an advertisement MPD and a live MPD, the file format parser 350 may be used to process different media and / or codecs used by the live media segments included in the live MPD. In some embodiments, the file format parser may issue a change type based on the codec, profile, and / or level of the live media segments.

[0021] As long as there is a media segment in the media buffer 340, the event and metadata buffer 330 maintains the corresponding event segment and metadata. The sample DASH processing model 300 may include a time-stamped metadata track parser 325 for tracking metadata related to in-band and MPD events. According to FIG. 3, the MSE 320 includes only a file format parser 350, a media buffer 340, and a media decoder 345. The event and metadata buffer 330 and the event and metadata synchronizer and dispatcher 335 are not native to the MSE 320 and prevent the MSE 320 from natively processing events and sending them to the application.

[0022] Auxiliary presentation Embodiments of the present disclosure define an auxiliary media presentation as a media presentation independent of the main media presentation of the MPD. As an example, an advertisement media segment or a live media segment independent of the main media segment may be an auxiliary presentation. Updates to the auxiliary media presentation or auxiliary media segment do not affect the main media segment. Similarly, updates to the main media segment do not affect the auxiliary media segment. Thus, the auxiliary media segment (also referred to as the auxiliary media presentation or auxiliary presentation) may be completely independent of the main media segment (also known in the present disclosure as the main media presentation and the media presentation).

[0023] Auxiliary period (also referred to as the auxiliary period attribute) FIG. 4 is an exemplary script 400 showing xml code that enables an auxiliary period for playing preroll and midroll auxiliary media content according to an embodiment.

[0024] As shown in FIG. 4, the AuxPeriod described by the EssentialProperty descriptor with @id = "e0" may be a preroll media segment, while the AuxPeriod described by the EssentialProperty descriptor with @id = "e1" and @id = "e2" may be a midroll media segment. In some embodiments, the auxiliary period may be disclosed before one or more periods in the auxiliary MPD.

[0025] Implementation of Auxiliary Period by MSE Source Buffer W3C MSE is a sequential buffer. As long as the time stamp offset (TSO) of the MSE source buffer is properly set so that the content that is expected to be played sequentially is appended at the correct position, the MSE source buffer plays the content correctly. According to one embodiment, at each time point when the main presentation is switched to the auxiliary presentation or the auxiliary presentation is switched back to the main presentation, the MSE time stamp offset (TSO) may be adjusted to place the main media segment and the auxiliary media segment at the correct position in the MSE buffer.

[0026] FIG. 5 is a diagram 500 of an MSE source buffer described using the xml code in FIG. 4.

[0027] As shown in FIG. 500, the auxiliary MPD aux e0 may be a preroll MPD, and aux e1 and aux e2 may be midroll MPDs. The live content p0 may be a main MPD that includes one or more main media segments.

[0028] Since AuxPeriod e0 is the preroll assist period, one or more assist media segments from aux e0 are acquired and may be placed at the head of the MSE source buffer before any main media segment. When acquiring and appending the preroll assist media segment in aux e0, MSE TSO is 0. T0 = e0@PTO... Equation (1) In some embodiments, following the acquisition and appending of each preroll assist media segment, MSE TSO may be updated based on the start time of live media p0, the presentation time of live media p0, and the depth of the time shift buffer. As an example, in FIG. 5, T1 may be based on the depth of the time shift buffer, the duration of the preroll assist media segment, and the presentation time offset related to the live media. T1 = Max(timeShiftBuffeerDepth, Live edge - P0Start)+e0@duration + p0@PTO - e0@PTO... Equation (2) Regarding midroll assist MPD aux e1 and aux e2, it is as follows. T2 = T1+e1@start + e1@PTO - p0@PTO... Equation (3) T3 = T2+e1@returnTime - p0@PTO - e1@PTO... Equation (4) T4 = T1+e2@start + e2@PTO - p0@PTO... Equation (5) T5 = T4 + p0@PTO - e2@PTO... Equation (6) Here, e0@PTO, e1@PTO, and e2@PTO are the presentation time offsets of the first period within the assist period linked to the essential descriptor in this order, and ei@starttime and ei@returnTime are the start time value and return time value in the corresponding descriptor with id = ei.

[0029] According to an embodiment, the update of MSE TSO may be based on the following.

[0030] The media segments of the preroll assist period may start from the start (0) of the MSE append window. Since the media segments of the preroll assist period are first acquired and appended, the MSE TSO is set equal to the PTO of the assist MPD (see, for example, Equation (1)).

[0031] In some embodiments, after appending the last segment of the preroll period assist media segment, a live edge related to the live media or the main media segment may be calculated. Using the size of the DVR window, the live edge, the start and duration of the preroll assist period or media segment, and the PTO of the live content, the MSE TSO may be adjusted such that there is a threshold time range between the first media segment appended to the MSE buffer and the last segment of the previously added preroll assist period. As an example, see Equation (2). In some embodiments, the threshold may be smaller than the DVR window, equal to the DVR window, or larger than the DVR window. The DVR window may include the ability for the client to scrub back or start playback at an earlier point in the main media stream.

[0032] According to embodiments, for each appended assist period or media segment, its offset from the live edge is calculated. The MSE TSO is incremented by this offset and adjusted based on the PTO of the live period and the PTO of the assist period / media segment. In other or the same embodiments, at the end of each assist period / media segment, the MSE TSO is adjusted based on the returnTime of the assist period / media segment, the PTO of the live period, and the PTO of the assist period / media segment.

[0033] In some embodiments, during preroll assist period / media segment playback, the client may decide to remove the time range of the preroll assist period / media segment from the MSE, and thus the preroll may be played only once. In some embodiments, when accessing the DVR, the preroll may not be played. The client may also remove the time range from the MSE append window buffer according to the value of timeShiftBufferDepth after participating in the live session.

[0034] The advantages of the assist period as defined herein include that the MSE / EME is initialized using the period during which a single MPD is correctly used, ensuring that the main content is played by the MSE / EME. This single MSE / EME initialization enables seamless playback of the content during the transition between the main presentation and the assist presentation.

[0035] Since all AuxPeriods are children of the MPD, the AuxPeriods remain unchanged (useful for preroll assist content) even if some Periods are removed in the MPD update. Further, since the assist period can be a remote element, the ability of this remote element enables late binding for the client's midroll in server-assisted advertisement insertion.

[0036] FIG. 6 is an exemplary flowchart of a process 600 for signaling assist media including preroll media content and midroll media content within a main media stream.

[0037] In operation 610, one or more main media segments may be appended to the MSE source buffer based on one or more respective main timestamp offsets associated with each of the one or more main media segments.

[0038] In operation 615, one or more auxiliary media segments may be appended to the MSE source buffer based on one or more auxiliary periods associated with the one or more auxiliary media segments. The auxiliary period may be a period type element including a start time attribute. The start time attribute may indicate a first time offset at which a main media segment in one or more main media segments may be stopped in order to play an auxiliary media segment in one or more auxiliary media segments. In some embodiments, the auxiliary period may include a return time attribute. The return time attribute may indicate a second time offset at which a main media segment in one or more main media segments may be restarted after a main media segment in one or more main media segments has been stopped in order to play an auxiliary media segment in one or more auxiliary media segments. In some embodiments, one or more auxiliary periods may be placed before one or more periods in an auxiliary media presentation description (MPD).

[0039] In operation 620, one or more main media segments and one or more auxiliary media segments within the MSE source buffer may be dispatched based on one or more respective main time stamp offsets and one or more auxiliary periods. In some embodiments, an auxiliary media segment in one or more auxiliary media segments may be a preroll media segment that is played before one or more main media segments based on the start time attribute being 0. In the same or different embodiments, the time stamp offset of the MSE source buffer associated with the auxiliary media segment may be set or initialized to be equal to the presentation time offset associated with the auxiliary media segment.

[0040] In some embodiments, the timestamp offset of the MSE source buffer may be updated based on the main media segment being stopped or resumed. In operation 625, the timestamp offset of the MSE source buffer may be updated based on the depth of the time shift buffer, the start time attribute, the total duration of all preroll media segments, and the presentation time offset associated with the first main media segment of one or more main media segments. In some embodiments, the above update may be based on the last preroll auxiliary media segment being appended to the MSE source buffer. The updated timestamp offset of the MSE source buffer may include a threshold time range between the last preroll auxiliary media segment and the first main media segment.

[0041] In some embodiments, the timestamp offset of the MSE source buffer may be updated based on the return time attribute and the presentation time offset associated with one of one or more auxiliary media segments within the MSE source buffer when one of the one or more auxiliary media segments is dispatched.

[0042] In the same or other embodiments, the timestamp offset of the MSE source buffer associated with the auxiliary media segment may be equal to the presentation time offset associated with the auxiliary media segment, and the one based on the auxiliary media segment among the one or more auxiliary media segments is the preroll auxiliary media segment.

[0043] FIG. 6 shows exemplary blocks of process 600, but in embodiments, process 600 may include more blocks, fewer blocks, different blocks, or differently arranged blocks than those shown in FIG. 6. In embodiments, any blocks of process 600 may be combined as needed, or arranged in any amount or order. In embodiments, two or more blocks of process 600 may be executed in parallel.

[0044] The above technology can be implemented as computer software using computer-readable instructions and can be physically stored on one or more computer-readable media, or can be implemented by one or more specially configured hardware processors. For example, FIG. 7 shows a computer system 700 suitable for implementing various embodiments.

[0045] The computer software can be encoded using any suitable machine code or computer language, and the machine code or computer language may be subject to assembly, compilation, linking, or similar mechanisms to generate code containing instructions, and the instructions can be executed directly or through an interpreter, microcode execution, etc. by a computer central processing unit (CPU), a graphics processing unit (GPU), etc.

[0046] The instructions can be executed on various types of computers or their components, including, for example, personal computers, tablet computers, servers, smartphones, game devices, Internet of Things devices, etc.

[0047] The components shown in FIG. 7 for the computer system 700 are exemplary in nature and are not intended to imply any limitation on the scope or functionality of the computer software used to implement the embodiments of the present disclosure. Also, the configuration of the components should not be construed as having any dependency or requirement on any one or combination of the components shown in the exemplary embodiments of the computer system 700.

[0048] The computer system 700 may include a specific human interface input device. Such a human interface input device may respond to input by one or more human users, for example, through tactile input (keystrokes, swipes, movement of a data glove, etc.), audio input (voice, clapping, etc.), visual input (gestures, etc.), and olfactory input. Also, the human interface device may be used to capture specific media that is not necessarily directly related to conscious input by humans, such as audio (e.g., conversation, music, ambient sound), images (scanned images, photographic images obtained from a still camera, etc.), and video (2D video, 3D video including stereoscopic pictures, etc.).

[0049] The input human interface device may include one or more of a keyboard 701, a mouse 702, a trackpad 703, a touch screen 710, a joystick 705, a microphone 706, a scanner 708, and a camera 707 (only one of each is shown in the drawing).

[0050] The computer system 700 may also include a specific human interface output device. Such a human interface output device may stimulate the senses of one or more human users, for example, through tactile output, sound, light, and smell / taste. Such a human interface output device includes a tactile output device (e.g., tactile feedback by the touch screen 710 or the joystick 705, although there may be a tactile feedback device that does not function as an input device), an audio output device (a speaker 709, headphones, etc.), a visual output device (each may or may not have a touch screen input function, each may or may not have a tactile feedback function, and some may be capable of outputting more than three-dimensional output, such as 2D visual output or stereoscopic output, including a CRT screen, an LCD screen, a plasma screen, an OLED screen of the screen 710, virtual reality glasses, a holographic display, and a smoke tank), and a printer.

[0051] Further, the computer system 700 may include a human-accessible storage device such as an optical medium including a CD / DVD ROM / RW 720 having a CD / DVD 711 or similar medium and associated media, a thumb drive 722, a removable hard drive or solid state drive 723, legacy magnetic media such as tapes and floppy disks, devices based on special ROM / ASIC / PLD such as security dongles, etc.

[0052] Also, those skilled in the art should understand that the term "computer-readable medium" as used in connection with the subject matter disclosed herein does not include a transmission medium, a carrier wave, or other non-transitory signals.

[0053] In addition, the computer system 700 can include an interface 799 to one or more communication networks 798. The network 798 can be, for example, wireless, wired, or optical. The network 798 can be local, wide area, metropolitan, vehicle and industrial, real-time, delay tolerant, etc. Examples of the network 798 include Ethernet, wireless LAN, cellular networks (including GSM, 3G, 4G, 5G, LTE, etc.), TV wired or wireless wide area digital networks (including cable TV, satellite TV, and terrestrial broadcast TV), vehicle and industrial (including CANBus), etc. A specific network 798 generally requires an external network interface adapter (such as a USB port of the computer system 700, etc.) attached to a specific general-purpose data port or peripheral bus (750 and 751), and other network interface adapters are generally integrated into the core of the computer system 700 by being attached to a system bus (such as an Ethernet interface to a PC computer system or a cellular network to a smartphone computer system) described below. Using any of these networks 798, the computer system 700 can communicate with other entities. Such communication can be only one-way reception (such as broadcast TV), only one-way transmission (such as CANBus to a specific CANbus device), or two-way to other computer systems using, for example, a local or wide area digital network. Specific protocols and protocol stacks can be used in each of the networks and network interfaces as described above.

[0054] The above human interface device, human-accessible storage device, and network interface can be attached to the core 740 of the computer system 700.

[0055] The core 740 can include one or more central processing units (CPUs) 741, a graphics processing unit (GPU) 742, a graphics adapter 717, a special programmable processing unit in the form of a field programmable gate array (FPGA), a hardware accelerator 744 for specific tasks, and the like. These devices may be connected through a system bus 748 together with a read-only memory (ROM) 745, a random access memory 746, and an internal mass storage device (an internal hard drive, SSD, etc. that is not accessible to the user). In some computer systems, the system bus 748 can be made accessible in the form of one or more physical plugs to enable expansion by additional CPUs, GPUs, etc. Peripheral devices can be directly attached to the core's system bus 748 or can be attached through a peripheral bus 751. The architecture of the peripheral bus includes PCI, USB, and the like.

[0056] The CPU 741, GPU 742, FPGA 743, and accelerator 744 can execute specific instructions, and the specific instructions can constitute the above computer code by combination. The computer code can be stored in the ROM 745 or the RAM 746. Also, temporary data can be stored in the RAM 746, while persistent data can be stored, for example, in the internal mass storage device 747. By using a cache memory that can be closely associated with one or more CPUs 741, GPUs 742, mass storage devices 747, ROM 745, RAM 746, etc., fast storage and retrieval to any of the memory devices are made possible.

[0057] A computer-readable medium can have computer code for executing operations implemented on various computers. The medium and the computer code can be specifically designed and constructed for the purposes of this disclosure or can be of the kind well-known and available to those skilled in the field of computer software.

[0058] By way of example and not limitation, the illustrated architecture, specifically, computer system 700 having core 740, can provide functionality as a result of software executed by a processor (including CPU, GPU, FPGA, accelerator, etc.) embodied on one or more tangible computer-readable media. Such computer-readable media can be media associated with a mass storage device accessible to the user as described above, as well as specific storage devices of core 740 of a non-transitory nature such as mass storage device 747 or ROM 745 inside the core. The software implementing various embodiments of the present disclosure can be stored in such devices and executed by core 740. The computer-readable media can include one or more memory devices or chips according to specific needs. The software can cause core 740, specifically, the processor (including CPU, GPU, FPGA, etc.) therein, to define data structures stored in RAM 746 and modify such data structures according to the processes defined by the software, thereby executing specific processes or specific parts of specific processes described herein. Further or alternatively, the computer system can provide functionality as a result of logic wired in a circuit (e.g., accelerator 744) or logic embodied in other ways, and the circuit can operate instead of or together with software to execute specific processes or specific parts of specific processes described herein. References to software include logic and, if necessary, vice versa. References to computer-readable media can, if necessary, include circuits (such as integrated circuits (ICs), etc.) storing software for execution, circuits embodying logic for execution, or both. The present disclosure includes any suitable combination of hardware and software.

[0059] Although this disclosure describes some exemplary embodiments, there are changes, substitutions, and various equivalent alternatives that fall within the scope of this disclosure. Accordingly, those skilled in the art will recognize that, although not explicitly illustrated or described herein, many systems and methods can be devised that embody the principles of this disclosure and are thus within the spirit and scope of this disclosure.

Claims

1. A method for signaling an auxiliary media stream during a main DASH (Dynamic Adaptive Streaming over HTTP) media stream using a Media Source Extension (MSE) buffer, comprising: appending the one or more main media segments to an MSE source buffer based on one or more respective main time stamp offsets associated with each of the one or more main media segments; appending the one or more auxiliary media segments to the MSE source buffer based on one or more auxiliary period attributes associated with the one or more auxiliary media segments, wherein the auxiliary period attribute of the one or more auxiliary period attributes is a period type element including a start time attribute, and the start time attribute indicates a first time offset at which a main media segment in the one or more main media segments is stopped during preroll or midroll insertion for playing an auxiliary media segment in the one or more auxiliary media segments; updating a time stamp offset of the MSE source buffer based on a time shift buffer depth, the start time attribute, a total duration of all preroll media segments, and a presentation time offset associated with a first main media segment of the one or more main media segments, based on the last preroll auxiliary media segment being appended to the MSE source buffer; dispatching the one or more main media segments and the one or more auxiliary media segments within the MSE source buffer based on the one or more respective main time stamp offsets and the one or more auxiliary period attributes; and the time stamp offset of the MSE source buffer is updated to include a threshold time range between the last preroll auxiliary media segment and the first main media segment.

2. The auxiliary period attribute further includes a return time attribute, and the return time attribute indicates a second time offset at which the main media segment in the one or more main media segments is resumed after the main media segment in the one or more main media segments is stopped for playing the auxiliary media segment in the one or more auxiliary media segments. The method according to claim 1.

3. The time stamp offset of the MSE source buffer is updated based on the main media segment being stopped or resumed. The method according to claim 2.

4. The one or more auxiliary period attributes are arranged before one or more periods in an auxiliary media presentation description (MPD). The method according to claim 1.

5. Based on the start time attribute being 0, the auxiliary media segment in the one or more auxiliary media segments is a preroll media segment played before the one or more main media segments. The method according to claim 1.

6. Based on the auxiliary media segment in the one or more auxiliary media segments being a preroll auxiliary media segment, the time stamp offset of the MSE source buffer related to the auxiliary media segment is equal to the presentation time offset related to the auxiliary media segment. The method according to claim 5.

7. When one of the one or more auxiliary media segments in the MSE source buffer is dispatched, the method further includes updating the time stamp offset of the MSE source buffer based on the return time attribute and the presentation time offset related to the one of the one or more auxiliary media segments. The method according to claim 6.

8. An apparatus for signaling an auxiliary media stream in a main DASH (Dynamic Adaptive Streaming over HTTP) media stream using a media source extension (MSE) buffer, At least one memory configured to store program code, At least one processor configured to access the program code and operate as commanded by the program code comprising wherein the program code causes the at least one processor to execute the method according to any one of claims 1 to 7, an apparatus. **Claim 9** A computer program that causes one or more processors to execute the method according to any one of claims 1 to 7.

Images