Method and apparatus for constructing a linear mse buffer using mpd links, computer readable medium

Abstract

A method and apparatus, computer readable medium for constructing a linear media source extension (MSE) buffer using MPD chaining can be provided. The method can include parsing at least one acquired media presentation segment, acquiring a live MPD based on at least one URL, the live MPD including one or more live media segments. The method can include appending the one or more advertisement media segments to a first MSE source buffer based on a first timing model, chaining the one or more live media segments to the one or more advertisement media segments by appending the one or more live media segments to the first MSE source buffer based on a second timing model. The one or more advertisement media segments and the one or more live media segments are scheduled based on the first timing model and the second timing model.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Provisional Application No. 63 / 246,233, filed September 20, 2021, and U.S. Application No. 17 / 947,801, filed September 19, 2022, the contents of which are expressly incorporated herein by reference in their entirety. Technical Field

[0003] Embodiments of this disclosure relate to streaming media content, and more specifically, to streaming media, advertising, and live content based on Dynamic Adaptive Streaming (DASH) of the Hypertext Transfer Protocol according to the Moving Picture Experts Group (MPEG). Background Technology

[0004] MPEG DASH provides a standard for streaming content over IP networks. In MPEG DASH, Media Presentation Descriptions (MPDs) and events are used to deliver events related to the media timeline to the client. The ISO / IEC 23009-1 DASH standard allows for the streaming of multi-rate content. The DASH standard provides a single linear timeline, where segments are continuations of each other on a single timeline. ISO / IEC 23009-1 also provides tools for MPD linking, namely signaling the URL of the next MPD to be played in an MPD that can be used for pre-roll ad insertion.

[0005] MPEG DASH provides a standard for streaming multimedia content over IP networks. While this standard solves the problem of linear playback of media content, it fails to address non-linear operations, such as media segments associated with different, independent timelines. These shortcomings can be overcome using MPD links and pre-roll ad insertions. However, even MPD links and pre-roll ad insertions fail when the DASH player uses W3C Media Source Extensions because handling such non-linear playback with a single MSE source buffer is very challenging. Furthermore, MPD links and pre-roll ad insertions cannot be used when it is expected to play advertisements before using MPD links or pre-roll elements, or when W3C Encrypted Media Extensions are used to play protected live content.

[0006] Therefore, a method is needed to construct a linear media source extension (MSE) buffer timeline from two or more non-linear sources using MPD linking. Furthermore, a method is needed to use MPD linking to process W3C encrypted media extensions for playing protected live content. Summary of the Invention

[0007] This disclosure addresses one or more technical problems. It provides a method, process, apparatus, and non-volatile computer-readable medium for constructing a Linear Media Source Extension (MSE) buffer from two or more non-linear media sources using MPD links. Furthermore, this disclosure provides a method, process, apparatus, and non-volatile system for implementing non-linear playback using a single MSE source buffer. Additionally, this disclosure enables the playback of advertisements and encrypted live content using a single Encrypted Media Extension (EME) interface.

[0008] Embodiments of this disclosure may provide a method for constructing a Linear Media Source Extension (MSE) buffer from two or more non-linear media sources using MPD linking. The method may be executed by at least one processor and may include: parsing at least one acquired media presentation segment comprising one or more ad media segments, the one or more ad media segments including at least one Uniform Resource Locator (URL); acquiring a live MPD based on the at least one URL, the live MPD comprising one or more live media segments; attaching the one or more ad media segments to a first MSE source buffer based on a first timing model; linking the one or more live media segments to the one or more ad media segments by attaching the one or more live media segments to the first MSE source buffer based on a second timing model, wherein the second timing model supports time-shifted buffer depth for at least the one or more live media segments; and scheduling the one or more ad media segments and the one or more live media segments based on the first timing model and the second timing model.

[0009] Embodiments of this disclosure may provide an apparatus for constructing a linear media source extension (MSE) buffer from two or more nonlinear media sources using MPD links. The apparatus may include: at least one memory configured to store computer program code; at least one processor configured to access the computer program code and operate according to instructions of the computer program code, the computer program code including: parsing code configured to cause the at least one processor to parse at least one acquired media presentation segment comprising one or more advertising media segments, the one or more advertising media segments comprising at least one Uniform Resource Locator (URL); first acquisition code configured to cause the at least one processor to acquire a live MPD based on the at least one URL, the live MPD comprising one or more live media segments; first attachment code configured to cause the at least one processor to attach the one or more advertising media segments to a first MSE source buffer based on a first timing model; second attachment code configured to cause the at least one processor to attach the one or more live media segments to the first MSE source buffer and link the one or more live media segments to the one or more advertising media segments by attaching the one or more live media segments to the first MSE source buffer based on a second timing model, wherein the second timing model supports time-shifted buffer depth for at least the one or more live media segments; and scheduling code configured to cause the at least one processor to schedule the one or more advertising media segments and the one or more live media segments based on the first timing model and the second timing model.

[0010] Embodiments of this disclosure may also provide an apparatus for constructing an MSE buffer using MPD links, the apparatus comprising: a parsing module for parsing at least one acquired media presentation segment including one or more advertising media segments, the one or more advertising media segments including at least one Uniform Resource Locator (URL); a first acquisition module for acquiring a live MPD based on the at least one URL, the live MPD including one or more live media segments; a first attachment module for attaching the one or more advertising media segments to a first media source extension MSE source buffer based on a first timing model; a second attachment module for attaching the one or more live media segments to the first MSE source buffer and linking the one or more live media segments to the one or more advertising media segments by attaching the one or more live media segments to the first MSE source buffer based on a second timing model, wherein the second timing model supports time-shifted buffer depth for at least the one or more live media segments; and a scheduling module for scheduling the one or more advertising media segments and the one or more live media segments based on the first timing model and the second timing model.

[0011] Embodiments of this disclosure may provide a non-volatile computer-readable medium for storing instructions that may include one or more instructions that, when executed by one or more processors of a device constructing a Linear Media Source Extension (MSE) buffer from two or more non-linear media sources using a Media Presentation Description (MPD) link, cause the one or more processors to: parse at least one acquired media presentation segment comprising one or more ad media segments, the one or more ad media segments comprising at least one Uniform Resource Locator (URL); acquire a live MPD based on the at least one URL, the live MPD comprising one or more live media segments; attach the one or more ad media segments to a first MSE source buffer based on a first timing model; link the one or more live media segments to the one or more ad media segments by attaching the one or more live media segments to the first MSE source buffer based on a second timing model, wherein the second timing model supports time-shifted buffer depth for at least the one or more live media segments; and schedule the one or more ad media segments and the one or more live media segments based on the first timing model and the second timing model. Attached Figure Description

[0012] Other features, properties, and various advantages of the disclosed subject matter will become further apparent from the following detailed description and accompanying drawings, wherein:

[0013] Figure 1 This is a simplified schematic diagram of a communication system according to an embodiment.

[0014] Figure 2 This is an exemplary schematic diagram illustrating the placement of components in a streaming environment according to an embodiment.

[0015] Figure 3 This is a simplified block diagram of the DASH processing model according to an embodiment.

[0016] Figure 4 This is a simplified diagram illustrating the Media Presentation Description (MPD) link between the advertising MPD and the live MPD according to an embodiment.

[0017] Figure 5A -B is a simplified schematic diagram of the Media Source Extension (MSE) buffer according to an embodiment.

[0018] Figure 6 This is an exemplary flowchart illustrating the construction of a linear MSE source buffer from two or more nonlinear sources using MPD linking, according to an embodiment.

[0019] Figure 7 This is a simplified schematic diagram of a computer system according to an embodiment. Detailed Implementation

[0020] The features discussed below can be used individually or in any combination in any order. Furthermore, embodiments can be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored on a non-volatile computer-readable medium.

[0021] Figure 1 A simplified block diagram of a communication system 100 according to an embodiment of the present disclosure is shown. The communication system 100 may include at least two terminal devices 102 and 103 interconnected via a network 105. For unidirectional data transmission, a first terminal device 103 may encode video data locally for transmission to the other terminal device 102 via the network 105. A second terminal device 102 may receive the encoded video data from the other terminal device via the network 105, decode the encoded video data, and display the recovered video data. Unidirectional data transmission is common in applications such as media services.

[0022] Figure 1 A second pair of terminal devices 101 and 104 is shown, supporting bidirectional transmission of encoded video, which may occur, for example, during a video conference. For bidirectional data transmission, each terminal device 101 and 104 can encode video data acquired at a local location for transmission to the other terminal device via network 105. Each terminal device 101 and 104 can also receive encoded video data transmitted by the other terminal device, decode the encoded video data, and display the recovered video data on a local display device.

[0023] exist Figure 1 In this disclosure, terminal devices 101, 102, 103, and 104 may be servers, personal computers, and smartphones, but the principles of this disclosure are not limited thereto. Embodiments of this disclosure are applicable to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network 105 refers to any number of networks, including, for example, wired and / or wireless communication networks, that transmit encoded video data between terminal devices 101, 102, 103, and 104. Communication network 105 may exchange data in circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet. For the purposes of this discussion, unless explained below, the architecture and topology of network 105 may be irrelevant to the operation of this disclosure.

[0024] As an example, Figure 2The illustration shows the placement of a video encoder and decoder in a streaming environment. This embodiment can be applied to other video-enabled applications, including, for example, video conferencing, digital TV, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0025] The streaming system may include an acquisition subsystem 203, which may include a video source 201, such as a digital camera, that creates, for example, an uncompressed video sample stream 213. The sample stream 213 may be emphasized as having a high data volume compared to an encoded video bitstream and may be processed by an encoder 202 coupled to the video source 201. The encoder 202 may include hardware, software, or a combination of hardware and software to implement or carry out aspects of the embodiments described in more detail below. The encoded video bitstream 204 may be emphasized as having a lower data volume compared to the sample stream 213 and may be stored on a streaming server 205 for future use. One or more streaming clients 212 and 207 may access the streaming server 205 to retrieve encoded video bitstreams 208 and 206, which may be copies of the encoded video bitstream 204. Client 212 may include a video decoder 211. Video decoder 211 decodes the incoming copy 208 of the encoded video bitstream and produces an output video sample stream 210 that can be displayed on display 209 or another presentation device. In some streaming systems, the video bitstreams 204, 206, and 208 can be encoded according to certain video codec / compression standards. Examples of these standards have been mentioned above and will be described further herein.

[0026] Figure 3 A sample DASH processing model 300 is illustrated, such as a sample client architecture for handling DASH and Common Media Application Format (CMAF) events. In DASH processing model 300, client requests for media segments (e.g., advertising media segments and live media segments) can be based on the addresses described in Listing 303. Listing 303 also describes a metadata track from which the client can access segments, parse these segments, and send them to application 301.

[0027] Listing 303 includes MPD events or events, and the in-band event and "moof" parser 306 can parse MPD event fragments or event segments and append the event segments to the event and metadata buffer 330. The in-band event and "moof" parser 306 can also acquire media segments and append them to the media buffer 340. The event and metadata buffer 330 can send event and metadata information to the event and metadata synchronizer and scheduler 335. The event and metadata synchronizer and scheduler 335 can schedule specific events to the DASH player's control, selection, and probing logic 302, and schedule application-related event and metadata tracks to the application 301.

[0028] According to some embodiments, the MSE may include a pipeline comprising a file format parser 350, a media buffer 340, and a media decoder 345. The MSE 320 is one or more logical buffers for media segments, wherein media segments can be tracked and ordered based on their presentation time. Media segments may include, but are not limited to, advertising media segments associated with advertising MPDs and live media segments associated with live MPDs. Each media segment can be added to or appended to the media buffer 340 based on its timestamp offset, and the media segments in the media buffer 340 can be ordered using the timestamp offset.

[0029] Since embodiments of this application may involve constructing a Linear Media Source Extension (MSE) buffer from two or more non-linear media sources using MPD links, and the non-linear media sources may be advertising MPDs and live MPDs, the file format parser 350 can be used to process the different media and / or codecs used by the live media segments included in the live MPD. In some embodiments, the file format parser may publish a change type based on the codec, profile, and / or level of the live media segment.

[0030] As long as media segments exist in media buffer 340, event and metadata buffer 330 maintains the corresponding event segments and metadata. Sample DASH processing model 300 may include a timed metadata track parser 325 to keep track of metadata associated with in-band and MPD events. Figure 3 The MSE 320 includes only a file format parser 350, a media buffer 340, and a media decoder 345. The event and metadata buffer 330, as well as the event and metadata synchronizer and scheduler 335, are not local to the MSE 320, thus preventing the MSE 320 from processing events locally and sending them to the application.

[0031] Figure 4 This is a simplified diagram 400 showing the MPD link between advertising MPD and live MPD.

[0032] like Figure 4 As shown, an advertising MPD 410 may include one or more Uniform Resource Locators (URLs) containing addresses to at least one live MPD 420. According to one aspect of the embodiment, the URL link between the advertising MPD 410 and the live MPD 420 may be an MPD link. Each advertising MPD 410 may contain one or more time slots Pa1, Pa2, etc. Similarly, each line MPD 420 may include one or more time slots P1, P2, etc. According to one aspect of this disclosure, a media player may play the advertising MPD 410 before the live MPD 420.

[0033] According to one aspect of this disclosure, a DASH player can be provided with one or more timing models for setting a single Media Source Extension (MSE) buffer from two or more non-linear media sources for media playback using MPD links. The MSE buffer can be configured to support a desired time-shift buffer depth. Reinitialized segments can be added to the MSE buffer to play ad media and live media with different codes, profiles, and / or levels. In some embodiments, ad media segments can be maintained in the time-shift buffer to enable the DASH player to retrieve ads, or to replace ad media segments with live media segments to enable the DASH player to locate the actual live time-shift buffer.

[0034] One or more MPDs can be obtained from video data. One or more MPDs can be parsed to generate and / or retrieve one or more ad media segments. In some embodiments, one or more ad media segments may include at least one Uniform Resource Locator (URL) associated with one or more live media segments. Ad media segments (also referred to as “on-demand content”) are available before playback. Ad media segments may not have a known duration, but may at least have a maximum duration of -D. MAX The presentation time offset associated with one or more advertising media segments can be referred to as having a time scale T. P PTO P .

[0035] Live content may include one or more live MPDs. One or more live MPDs may include one or more live media clips. The start time of the live content and the start time of the current time segment of the live content may be known. The presentation time offset associated with one or more live media clips may be referred to as having a time scale T. L PTO L Used to estimate the earliest presentation time (EEPT) of a fragment. LThe address of the segment that begins the live content may be unknown. The maximum segment duration associated with one or more live media segments may be referred to as the MSD, and the required time-shift buffer depth for the live content may be referred to as the TSB. L .

[0036] According to embodiments of this disclosure, since the advertising MPD may not have any information about the live MPD, the DASH client may need to download the live MPD before setting up the MSE buffer.

[0037] According to the above embodiments, a DASH player (e.g., 300) can download and parse one or more ad MPDs. A file format parser can find and / or retrieve the required codecs in the ad MPD. Using a URL from an ad media segment of the ad MPD, a live MPD can be downloaded, and the live edges and corresponding time periods of the live media MPD can be found. The file format parser can find / retrieve the required codecs in the live MPD. The MSE buffer can be set based on codec information in the ad MPD and live MPD (or ad media segments and live media segments). A timing model (e.g., Figures 5A-5B The timing model in the MSE buffer downloads ad media fragments from the ad MPD and appends them to the MSE buffer. After appending the ad media fragments, the timing model (e.g., ...) can be used. Figures 5A-5B Download live media clips (using the timing model in the MSE buffer) or attach them to the MSE buffer.

[0038] Figures 5A-5B Exemplary media source buffers 500 and 550 are shown using MPD links to process advertising media clips and live media clips.

[0039] like Figure 5A As shown, an MSE supplementary window can be defined as having a duration equal to or greater than the maximum ad duration (e.g., D). MAX (or larger). When attaching ad clips, the MSE attach window timestamp offset can be set to TSO. P =-(PTO) P / T P ).

[0040] like Figure 5B As shown, considering the DASH client's live MPD time-shift buffer (TSB) L The size of the MSE attachment window can be adjusted to D. 实况 In some embodiments, D 实况 =Max(TSB) L D MAX(or larger.) The MSE additional window timestamp offset can be reset to TSO. L Before attaching the first live clip, TSO L It can be equal to Max(TSB) L D MAX )+EEPT l +MSD-(PTO L / T L Based on the appended and timestamp offset, the search can be performed on the MSE buffer between the ad segment time and the live media segment time. The start of the live media segment can be found based on the last frame of the ad segment playing before the live media segment. If only one ad segment is expected to play, the ad media segment time range can be cleared. In embodiments where the player searches for a time-shifted live media segment before playing an ad media segment, a third time range can be created for the time-shifted live media segment.

[0041] Ad played once and removed

[0042] MSE Settings: Check if MSE supports advertising video codecs and profiles; create an MSE buffer; and set the MSE buffer append window duration to D. MAX .

[0043] Replay: The official MSE timestamp can be based on TSO = -(PTO) P / T P Settings; can retrieve the first ad media segment and append it to the MSE buffer; can append each ad media segment in the ad MPD to the MSE buffer. After each media segment is downloaded, the MSE append window can be extended to duration D. 实况 And the timestamp offset can be TSO L =Max(TSB) L D MAX )+EPT+MSD-(PTO L / T L The first live media clip can be attached to the MSE buffer, and each live media clip from the live MPD can also be attached. The start time of the live media clip is found based on when the end of the ad media clip is reached.

[0044] Time-shift buffer: The range from the beginning to the end of an ad media segment can be removed.

[0045] The ad plays once but remains in the mid-roll.

[0046] MSE Settings: Check if MSE supports advertising video codecs and profiles; create an MSE buffer; and set the MSE buffer append window duration to TSB. Max =D MAX +TSBD L The estimated value. The start of the additional window can be set to zero, and the end of the additional window can be set to TSB. Max +D 实况 In some embodiments, if the duration of the live program is unknown, the end of the additional window can be a large value or infinity.

[0047] Playback: The MSE timestamp offset can be set to TSO = TSB. Max -D MAX -(PTO P / T P It can retrieve the first ad media segment and append it to the MSE buffer; it can append each ad media segment in the ad MPD to the MSE buffer. It can publish changes to the codec, profile, and / or level of the live media segment. It can set the timestamp offset to TSO. L =Max(TSB) L D MAX )+EPT+MSD-(PTO L / T L The first live media segment can be attached to the MSE buffer, and each live media segment from the live MPD can also be attached. The start time of the live media segment is found based on the end of the ad segment. The first ad media segment can then be retrieved again and attached to the MSE.

[0048] Time-shift buffer management: The timestamp offset used for the time-shift buffer can be set to TSO = TSB. Max -D Max -(PTO P / T P )-(PTO L / T L Based on the replay of an ad segment, one or more ad segments can be retrieved and appended to the time-shift buffer. Each ad segment can be appended to the time-shift buffer.

[0049] W3C Encrypted Media Extension (EME) Interface

[0050] The W3C EME interface can be used to decrypt encrypted data during media playback. Advertisement content may generally be unprotected. However, live content can be protected. When live content is protected, the player may need to download the advertisement MPD and use the URL in the advertisement MPD to download the live MPD. Using the information in the live MPD, the player can set up EME and MSE pipelines. EME pipeline settings may include obtaining a license key from a license server. The player can then use the advertisement MPD to begin downloading advertisement clips and append them to the media pipeline.

[0051] In some embodiments, since the ad MPD may not contain information about the live MPD, the MSE and EME can be set sequentially. The player can download the ad MPD. The player can download and parse the live MPD using the URL in the ad MPD. Given the parameters of the live MPD, the player can set the MSE and EME buffers. The player can then download subsequent ad MPDs and begin downloading the corresponding segments and appending them to the media pipeline.

[0052] In some embodiments, the advertising MPD may contain information about the live MPD, thus allowing the MSE and EME to be set in parallel. The player can download the advertising MPD and then, based on parsing the advertising MPD, store its segment in its internal memory. The player can also download and parse the live MPD using a URL from the advertising MPD and store its segment in its internal memory. Given parameters for both the advertising and live MPDs, the player can set the MSE and EME buffers. Then, ad segments from the advertising segments can be downloaded and appended to the media pipeline.

[0053] Figure 6 An exemplary flowchart of a process 600 for constructing a linear media source extension (MSE) buffer from two or more non-linear media sources using MPD links is shown.

[0054] At operation 610, at least one acquired media presentation segment (MPD) comprising one or more advertising media segments can be parsed. The one or more advertising media segments may include at least one Uniform Resource Locator (URL). As an example, manifest parser 303, in-band event, and moof parser 306 can parse the acquired MPD. This operation may include retrieving one or more codes associated with at least one acquired MPD and a live MPD.

[0055] At operation 615, a live MPD, comprising one or more live media segments, can be obtained based on the at least one URL. As an example, live MPD 420 can be obtained based on parsing a URL in ad MPD 410.

[0056] At operation 620, the one or more advertising media segments can be appended to the first buffer based on a first timing model. In some embodiments, the first buffer can be extended to a first maximum duration based on at least one acquired MPD. As an example, the MSE buffer can be extended to D. MAX The timestamp offset of the first buffer can also be set based on the first presentation time offset associated with one or more ad media segments. As an example, the first presentation time offset could include TSO = TSB. Max -D MAX -(PTO P / T P In some embodiments, the first presentation time offset may include TSO = -(PTO). P / T P ).

[0057] At operation 625, the one or more live media segments can be appended to the first buffer based on a second timing model, linking the one or more live media segments to the one or more ad media segments. In some embodiments, live media segments can be appended only after each ad media segment has been added. Appending based on the second timing model can include extending the first buffer to a second maximum duration based on one or more live media segments. The timestamp offset of the first buffer can be reset based on a second presentation time offset associated with one or more live media segments. As an example, the MSE can be extended to TSO. L =Max(TSB) L D MAX )+EPT+MSD-(PTO L / T L ).

[0058] A first live media segment can be attached to a first buffer within one or more live media segments. According to an embodiment, a second presentation time offset is based on an estimated earliest presentation time associated with one or more live media segments and a maximum segment duration associated with one or more live media segments.

[0059] The player can look for the start of one or more live media segments attached to the first buffer, based on the last frame of one or more advertising media segments being played.

[0060] At operation 630, the one or more advertising media segments and the one or more live media segments can be scheduled based on the first timing model and the second timing model. In some embodiments, based on the first advertising media segment being configured to play once, the first advertising media segment can be cleared from one or more advertising media segments from the first buffer. In other embodiments, based on the first advertising media segment being configured to play more than once, the first advertising media segment from one or more advertising media segments can be appended to a third buffer (e.g., a timestamp buffer), and based on the first advertising media segment being a mid-roll ad during the first live media segment, the first live media segment from one or more live media segments can be appended before the first advertising media segment.

[0061] In some embodiments, based on the encryption of one or more live media segments, one or more license keys can be retrieved from a license server, and one or more license keys can be stored and / or appended to a second buffer.

[0062] although Figure 6 An example block of process 600 is shown, but in embodiments, process 600 may include more than Figure 6 The examples show more boxes, fewer boxes, different boxes, or different arrangements of boxes. In the embodiments, any box of process 600 can be combined or arranged in any number or order as needed. In the embodiments, two or more boxes of process 600 can be executed in parallel.

[0063] The above-described technologies can be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media, or implemented using one or more specially configured hardware processors. For example, Figure 7 A computer system 700 suitable for implementing various embodiments is shown.

[0064] The computer software can be encoded using any suitable machine code or computer language, and code including instructions can be created through mechanisms such as assembly, compilation, and linking. These instructions can be executed directly by the computer's central processing unit (CPU), graphics processing unit (GPU), or through decoding, microcode, or other means.

[0065] The instructions can be executed on various types of computers or their components, including, for example, personal computers, tablets, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0066] Figure 7The components shown for computer system 700 are exemplary in nature and are not intended to limit the scope or functionality of computer software implementing embodiments of this disclosure. Nor should the configuration of the components be construed as having any dependency or requirement on any component or combination thereof shown in the exemplary embodiments of computer system 700.

[0067] Computer system 700 may include certain human-machine interface (HMI) input devices. These HMI input devices may respond to input from one or more human users through tactile input (e.g., keyboard input, swiping, data glove movement), audio input (e.g., sound, applause), visual input (e.g., gestures), and olfactory input. The HMI device may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., speech, music, ambient sound), images (e.g., scanned images, photographic images acquired from still cameras), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0068] The human-machine interface input device may include one or more of the following (only one is shown in the figure): keyboard 701, mouse 702, touchpad 703, touch screen 710, joystick 705, microphone 706, scanner 708, and camera 707.

[0069] The computer system 700 may also include certain human-machine interface (HMI) output devices. Such HMI output devices can stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. These HMI output devices may include tactile output devices (e.g., tactile feedback via a touchscreen 710 or joystick 705, but may also include tactile feedback devices that are not used as input devices), audio output devices (e.g., speakers 709, headphones), visual output devices (e.g., screen 710 including cathode ray tube screens, liquid crystal screens, plasma screens, organic light-emitting diode screens, each with or without touchscreen input functionality, each with or without tactile feedback functionality—some of which may output two-dimensional or higher-dimensional visual outputs through means such as stereoscopic image output; virtual reality glasses, holographic displays, and smoke boxes), and printers.

[0070] The computer system 700 may also include human-accessible storage devices and related media, such as optical media including high-density read-only / rewritable optical discs (CD / DVD ROM / RW) 720 or similar media including CD / DVD 711, thumb drives 722, removable hard disk drives or solid-state drives 723, conventional magnetic media such as magnetic tapes and floppy disks, dedicated devices based on ROM / ASIC / PLD such as security software protectors, and so on.

[0071] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0072] Computer system 700 may also include an interface 799 providing access to one or more communication networks 798. For example, network 798 may be wireless, wired, or optical. Network 798 may also be a local area network (LAN), wide area network (WAN), metropolitan area network (MAN), vehicular and industrial network, real-time network, latency-tolerant network, etc. Examples of network 798 include Ethernet, wireless LAN, cellular networks (GSM, 3G, 4G, 5G, LTE, etc.), cable or wireless wide area digital networks (including cable TV, satellite TV, and terrestrial broadcast TV), vehicular and industrial networks (including CANBus), etc. Some networks 798 typically require external network interface adapters for connection to certain general-purpose data ports or peripheral buses (750 and 751) (e.g., the USB port of computer system 700); other systems are typically integrated into the core of computer system 700 via connection to system buses as described below (e.g., an Ethernet interface integrated into a PC computer system or a cellular network interface integrated into a smartphone computer system). By using any of these networks 798, computer system 700 can communicate with other entities. The communication can be unidirectional, used only for receiving (e.g., wireless television), unidirectional, used only for sending (e.g., CAN bus to certain CAN bus devices), or bidirectional, such as through a local area or wide area digital network to other computer systems. Each of the above networks and network interfaces can use certain protocols and protocol stacks.

[0073] The aforementioned human-machine interface device, human-accessible storage device, and network interface can be connected to the core 740 of the computer system 700.

[0074] The core 740 may include one or more central processing units (CPUs) 741, graphics processing units (GPUs) 742, graphics adapters 717, dedicated programmable processing units in the form of field-programmable gate arrays (FPGAs) 743, task-specific hardware accelerators 744, etc. These devices, along with read-only memory (ROM) 745, random access memory 746, internal mass storage (e.g., internal non-user-accessible hard disk drives, solid-state drives, etc.) 747, can be connected via a system bus 748. In some computer systems, the system bus 748 can be accessed via one or more physical connectors to allow for expansion with additional CPUs, GPUs, etc. Peripheral devices may be directly attached to the core's system bus 748 or connected via a peripheral bus 751. Peripheral bus architectures include peripheral component interconnect (PCI), universal serial bus (USB), etc.

[0075] The CPU 741, GPU 742, FPGA 743, and accelerator 744 can execute certain instructions, which, when combined, constitute the aforementioned computer code. This computer code can be stored in ROM 745 or RAM 746. Transient data can also be stored in RAM 746, while permanent data can be stored, for example, in internal mass storage 747. Fast storage and retrieval of any memory device can be achieved through the use of a cache memory, which can be closely associated with one or more CPUs 741, GPUs 742, mass storage 747, ROM 745, RAM 746, etc.

[0076] The computer-readable medium may contain computer code for performing various computer-implemented operations. The medium and computer code may be specially designed and constructed for the purposes of this disclosure, or they may be media and code well-known and usable by those skilled in the art of computer software.

[0077] By way of example and not limitation, a computer system 700 having the illustrated architecture, particularly core 740, can provide functionality as a processor (including CPU, GPU, FPGA, accelerator, etc.) to execute software contained in one or more tangible computer-readable media. Such computer-readable media can be media associated with the aforementioned user-accessible mass storage, as well as specific memory of the non-volatile core 740, such as core-internal mass storage 747 or ROM 745. Software implementing various embodiments of this disclosure can be stored in such a device and executed by core 740. Depending on specific needs, the computer-readable medium may include one or more storage devices or chips. The software can cause core 740, particularly the processor therein (including CPU, GPU, FPGA, etc.), to execute a specific process or a specific portion of a specific process described herein, including defining data structures stored in RAM 746 and modifying such data structures according to a software-defined process. Alternatively or as an alternative, the computer system can provide functionality that is logically hardwired or otherwise included in circuitry (e.g., accelerator 744), which can replace or operate with the software to execute the specific process or a specific portion of a specific process described herein. Where appropriate, references to software may include logic, and vice versa. Where appropriate, references to computer-readable media may include circuitry storing the software (such as an integrated circuit (IC)), circuitry containing the logic to perform the execution, or both. This disclosure includes any suitable combination of hardware and software.

[0078] While this disclosure has described several exemplary embodiments, various modifications, arrangements, and equivalent substitutions of the embodiments are within the scope of this disclosure. Therefore, it should be understood that those skilled in the art can design various systems and methods that, while not explicitly shown or described herein, embody the principles of this disclosure and are thus within its spirit and scope.

Claims

1. A method of constructing a linear media source extension (MSE) buffer using media presentation description (MPD) linking, characterized in that, The method includes: The parsing includes at least one obtained media presentation segment comprising one or more advertising media segments, wherein the one or more advertising media segments include at least one Uniform Resource Locator URL; Based on the at least one URL, obtain the live MPD, which includes one or more live media clips; Based on the first timing model, the one or more advertising media segments are appended to the first media source extended MSE source buffer; By attaching the one or more live media segments to the first MSE source buffer based on a second timing model, the one or more live media segments are linked to the one or more advertising media segments, wherein the second timing model supports time-shifted buffer depth for at least the one or more live media segments; and Based on the first timing model and the second timing model, schedule the one or more advertising media segments and the one or more live media segments; The step of attaching the one or more advertising media segments to the first MSE source buffer based on the first timing model includes: Based on the at least one acquired MPD, the first MSE source buffer is extended to a first maximum duration; and The timestamp offset of the first MSE source buffer is set based on the first presentation time offset associated with the one or more advertising media segments.

2. The method of claim 1, wherein, The step of appending the one or more live media clips to the first MSE source buffer based on the second timing model includes: Based on the one or more live media clips, the first MSE source buffer is extended to a second maximum duration; Based on the second presentation time offset associated with the one or more live media segments, reset the timestamp offset of the first MSE source buffer; and The first live media segment from the one or more live media segments is appended to the first MSE source buffer.

3. The method of claim 2, wherein, The second presentation time offset is based on the estimated earliest presentation time associated with the one or more live media segments and the maximum segment duration associated with the one or more live media segments.

4. The method of claim 1, wherein, Further includes: Retrieve one or more codes associated with the at least one acquired MPD and the live MPD.

5. The method of claim 1, wherein, Further includes: Based on the last frame of the one or more advertising media segments being played, the start of the one or more live media segments appended to the first MSE source buffer is located.

6. The method of claim 1, wherein, The method further includes: Based on the encryption of the one or more live media segments, retrieve one or more license keys from the license server; and The one or more license keys are appended to the second buffer.

7. The method of claim 1, wherein, Further includes: Based on the first advertising media segment being configured to play once, the first advertising media segment is cleared from the first MSE source buffer among the one or more advertising media segments.

8. The method of claim 1, wherein, Further includes: Based on the fact that the first advertising media segment is configured to play more than once, the first advertising media segment in the one or more advertising media segments is appended to the third buffer; as well as Based on the fact that the first advertising media segment is an in-roll ad during the first live media segment, the first live media segment is appended to one or more live media segments before the first advertising media segment.

9. An apparatus for constructing a linear media source extension MSE buffer using Media Presentation Description (MPD) links, characterized in that, The device includes: At least one memory is configured to store computer program code; At least one processor is configured to access the computer program code and operate according to the instructions of the computer program code to perform the method of any one of claims 1-8.

10. An apparatus for constructing a linear media source extension MSE buffer using Media Presentation Description (MPD) links, characterized in that, The device includes: A parsing module is used to parse at least one acquired media presentation segment, which includes one or more advertising media segments, said one or more advertising media segments including at least one Uniform Resource Locator URL; The first acquisition module is used to acquire a live MPD based on the at least one URL, wherein the live MPD includes one or more live media segments; The first additional module is used to attach the one or more advertising media segments to the first media source extension MSE source buffer based on the first timing model. A second additional module is configured to attach the one or more live media segments to the first MSE source buffer based on a second timing model, and link the one or more live media segments to the one or more advertising media segments, wherein the second timing model supports time-shifted buffer depths for at least the one or more live media segments; and The scheduling module is used to schedule the one or more advertising media segments and the one or more live media segments based on the first timing model and the second timing model; The first additional module includes: A first extension module is configured to extend the first MSE source buffer to a first maximum duration based on the at least one acquired MPD; and The first setting module is used to set the timestamp offset of the first MSE source buffer based on the first presentation time offset associated with the one or more advertising media segments.

11. The apparatus according to claim 10, characterized in that, The second additional module includes: The second extension module is used to extend the first MSE source buffer to a second maximum duration based on the one or more live media segments; The reset module is configured to reset the timestamp offset of the first MSE source buffer based on a second presentation time offset associated with the one or more live media segments; and A third additional module is used to attach a first live media segment from the one or more live media segments to the first MSE source buffer.

12. The apparatus according to claim 11, characterized in that, The second presentation time offset is based on the estimated earliest presentation time associated with the one or more live media segments and the maximum segment duration associated with the one or more live media segments.

13. The apparatus according to claim 10, characterized in that, Further includes: The first search module is used to search for the start of the one or more live media segments appended to the first MSE source buffer based on the last frame of the one or more advertising media segments being played.

14. The apparatus according to claim 10, characterized in that, Further includes: The retrieval module is used to retrieve one or more license keys from the license server based on the fact that the one or more live media segments are encrypted; as well as A fourth additional module is used to attach the one or more license keys to the second buffer.

15. The apparatus according to claim 10, characterized in that, Further includes: A first clearing module is configured to clear the first advertising media segment from the first MSE source buffer based on the first advertising media segment being configured to play once.

16. The apparatus according to claim 10, characterized in that, Further includes: The fifth additional module is used to attach the first advertising media segment from the one or more advertising media segments to the third buffer based on the first advertising media segment being configured to play more than once; as well as A sixth additional module is configured to append the first live media segment from one or more live media segments before the first advertising media segment, based on the fact that the first advertising media segment is an in-between advertisement during the first live media segment.

17. A non-volatile computer-readable medium, characterized in that, For storing instructions, the instructions including one or more instructions, when executed by one or more processors of a device that constructs a linear media source extension MSE buffer using a media rendering description MPD link, causing the one or more processors to perform the method of any one of claims 1-8.

Images