Latency management for content delivery

Abstract

This document discloses a method for playing back media content via a delivery channel. A delivery channel may generally refer to a channel through which an audio or video program is delivered (sent) to a user (receiver). Media content may generally include continuous media programs. In particular, for a specific media program within the media content, a corresponding content type for the specific media program is also provided. The method may include receiving an indication of the sensitivity of the media program to playback delay. The method may further include receiving at least a portion of the media program. The method may further include adjusting the playback of the media program based on the indication of the sensitivity of the media program to playback delay.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the following priority applications: U.S. Provisional Application No. 62 / 875,063 filed on July 17, 2019 (reference number: D18140USP1) and European Application No. 19186801.7 filed on July 17, 2019 (reference number: D18140EP), which are incorporated herein by reference. Technical Field

[0003] The present document relates to latency management for content delivery. In particular, the present document relates to traditional "24 / 7" broadcast channels (or any channel with continuous programming) and their delivery over IP networks. Background Art

[0004] Generally speaking, traditional "24 / 7" broadcast channels usually consist of continuous programs such as news, movies, advertisements, dramas, (live) game shows, live sports, etc. Usually, in such broadcast channels, the right program is automatically sent at the right time through a specific (program) scheduler.

[0005] One of the many challenges of delivering (multiple) A / V content over IP networks is latency. Typically, consumers value low-latency delivery of live broadcasts (e.g., sports, live game shows, etc.). However, achieving low latency over IP networks is not always possible and usually requires (bitrate) costs to the operator. In addition, it is well known that low-latency streaming is more likely to experience buffer underruns (e.g., caused by jitter in the network connection) than high-latency streaming, because low-latency streaming usually means using smaller buffers in the playback device.

[0006] Today, latency is usually associated with a stream and is considered to be constant over the duration of a given stream (at least for a period of time). However, only a few programs on a broadcast channel are latency sensitive (such as live sports), while others are not (such as movies).

[0007] Therefore, low-latency streaming is mainly needed for programs that require low latency (such as live sports), and when the program is not sensitive to latency (such as movies), it can switch to a higher latency. Summary of the invention

[0008] In view of some or all of the above problems (and use cases), the present disclosure proposes a method and device for playing back media content via a delivery channel and a system for delivering media content via a delivery channel, the method, device and system having the features of the corresponding independent claims.

[0009] As a broad aspect, the present disclosure provides a method for playing back media content via a delivery channel. A delivery channel may generally refer to a channel through which an audio or video program is delivered (sent) to a user (receiver), such as in a broadcast, multicast or unicast manner. For example, such a delivery channel may be a (traditional) "24 / 7" broadcast channel. Media content may generally include continuous media programs, such as news, movies, advertisements, dramas, (live) game performances, live sports, etc. In particular, for a specific media program within the media content, the corresponding content type of the specific media program is also provided together with the media content. As understood and recognized by the technician, the content type may be represented and delivered in any suitable manner. For example, in a header of a bitstream of a media program or in metadata accompanying a program, a specific or additional field may be used to represent or indicate a content type. As another example, a lookup table may be used to indicate a content type. Similarly, a content type may be delivered (sent) in real time with the delivery of a media program, or may be shared before the delivery of a media program. The media content may be carried in a media stream and delivered from a content delivery server to a user device via a communication network (eg, an Internet Protocol based network).

[0010] In particular, the method may include (e.g., at a user device, a receiver, or a playback device) receiving an indication of (the step of) sensitivity of a media program to playback delay (also referred to as a target delay for the media program). More particularly, the sensitivity of the media program to playback delay is associated with a content type of the media program. In other words, the sensitivity of the media program to playback delay is set based on the content type of the media program. However, there may be other factors that affect the desired playback delay. For example, while live sports broadcasts may generally tend to have low latency, a service operator may want to distinguish between sports programs where low latency is very important (finals of major tournaments) and matches of less important matches where low latency is relatively less important. Therefore, adjusting the playback of a media program based on an indication of the sensitivity of the media program to playback delay should not exclude other factors that are also considered for determining a target playback delay.

[0011] The method may further include receiving at least a portion of a media program. As will be understood and appreciated by the skilled person, a portion of a media program generally refers to a certain number of frames (pictures, data blocks, etc.) of the media program, depending on, for example, the representation and / or delivery method / mechanism of the media program. The method may further include adjusting playback of the media program based on an indication of the sensitivity of the media program to playback delay.

[0012] In some examples, another indication of the sensitivity of a media program to playback delay is received, and the playback of the same media program is gradually adjusted based on the received indication. This can be repeated multiple times so as to sequentially provide multiple target delays to the playback device. For example, a method using multiple sensitivity indications can be used to gradually manage playback characteristics before transitioning to a subsequent or another program, because otherwise the transition may cause loss or have other adverse effects on the user experience due to the jump of the target delay between programs. Therefore, in addition to a single jump of the target delay based on a single indication, it is also recommended to adjust the playback of the media program within a specific time period. This allows the target delay on the server side to be gradually changed (in multiple steps), thereby remotely controlling the adjustment speed on the receiver (decoder) side through the sequence of received playback delay indications.

[0013] In some examples, adjusting the playback of a media program may include, for example, adjusting the buffering of a media program at a receiver or playback device. In particular, the step of adjusting the playback of a media program may include adjusting the amount of media program data buffered (at the playback device) before the playback media program data. As recognized by the technician, the buffering mechanism (at the playback device) may be implemented at the application level, at the system level, or in any other suitable manner. For example, multiple incremental indications of changes in the sensitivity of a media program to playback delay may be applied to progressively manage the receiver buffering degree before transitioning to a subsequent or another program, because otherwise the transition may cause loss or have other adverse effects on the user experience due to the jump of the target delay between programs.

[0014] In some examples, the indication of the sensitivity of the media program to playback delay may include a qualitative indication or a quantitative indication of the delay that can be accepted before playing back the media program data. By way of example, the qualitative indication of delay can relate to the sensitivity of the media program being played back to the delay for the user, for example, whether a longer delay will be noticed by (multiple) program viewers, and / or whether a short delay will cause (multiple) noticeable artifacts (e.g., jitter, frame drops, etc.). In some cases, such a qualitative indication can be simply expressed as "high", "medium" or "low". On the other hand, the quantitative indication of delay can be, for example, a maximum delay or target delay (e.g., 5 seconds) that can be accepted.

[0015] In some examples, adjusting the playback of a media program may include adjusting the playback speed of the media program. In some cases, the playback speed of a media program may be expressed as, for example, frames per second (fps). For example, if the normal playback speed of a program is 20 fps, doubling the playback speed means that the program is played back at 40 fps.

[0016] In some examples, the playback of a media program may include modifying the content of the received media program. In particular, if the indication of the sensitivity of the media program to playback delay changes relative to the previous or subsequent media program of the delivery channel, the content of the received media program may be modified. Generally speaking, in some cases, this means that the playback device (e.g., the application of the playback device) can skip some or complete segments (frames) of the media program, such as to catch up. In some other cases, the playback device (e.g., the application of the playback device) may need to insert, for example, some "silent" or repeated segments or frames, in order to slow down the playback. Such a change is usually noticeable by the viewer, and therefore may be considered to be less desirable in some situations / scenes.

[0017] In some examples, additional content may be inserted into the current media program (e.g., by frame or segment). In particular, if the current media program allows for greater latency relative to the previous media program, or the subsequent media program allows for greater latency relative to the current media program, the insertion of (multiple) additional content may be performed. In some cases, this may mean that the buffer size at the playback device must be increased, and the buffer may be filled appropriately.

[0018] In some other examples, content may be skipped in the current media program (e.g., by frame or segment). In particular, if the current media program requires less latency relative to a previous media program, or a subsequent media program requires less latency relative to the current media program, skipping (multiple) contents may be performed. In some cases, this may mean that the buffer size at the playback device must be reduced, and the buffer may be flushed appropriately.

[0019] In some examples, it can be considered that a media program includes multiple consecutive segments. The segmentation can depend on, for example, the transmission mode or transmission format applied for sending the media program to the receiver. That is, in this case, the internal processing of the media player can be organized by segment. Correspondingly, adjusting the playback of the media program can then include adjusting the number of segments buffered before playing back the media program data. Optionally, as described above, at least one segment of the media program can be skipped or repeated if necessary.

[0020] In some examples, adjusting the playback of a media program can include switching a playback mode of the media program (e.g., a media player at a playback device). As understood and appreciated by the skilled person, playback modes can, for example, relate to different ways in which a media player outputs a media program. For example, in one playback mode, a media player waits to process or output a media program until the entire (transmission format) segment is received, while in another playback mode, the media player plays out as soon as it receives media data sufficient to decode. Therefore, when an indication of a low target latency is received, the playback mode can be switched to a mode corresponding to a low latency requirement, for example, outputting media data as quickly as possible without waiting for the receipt of a complete segment.

[0021] In some examples, an indication of the sensitivity of a media program to playback delay can be carried in out-of-band metadata associated with the media program. For example, such metadata can be carried in a media program description (MPD) according to ISO IEC 23009-1 or in any other suitable metadata, as recognized by the technician. Metadata can be provided to a playback device in many different ways, such as a configuration file read when a playback device is started, when switching to a delivery channel, when initially accessing a media stream, and / or when a new media program in a delivery channel begins. For this purpose, a playback device can be provided with a configuration file or a link to a configuration file that the playback device can access via a communication network.

[0022] In some examples, an indication of the sensitivity of a media program to playback delay may be generated by a (program) scheduler for a continuous media program delivered via a delivery channel. The (program) scheduler may reside on the network side of the content delivery system (e.g., on the broadcaster or operator side, or as an independent entity). Optionally, the scheduler may also be responsible for adding / inserting the sensitivity indication into the metadata of the media program via any suitable means. However, other suitable entities (functional blocks) may be used to add the sensitivity indication to the metadata of the media program, e.g., a network entity located further upstream. In any case, subsequent steps or entities (particularly multiplexing and segmentation steps) should also (explicitly) support the metadata.

[0023] In some examples, the method may further include adjusting the encoding mode of the media program at the encoder side. As the skilled person will understand and appreciate, adjusting the encoding mode may, for example, mean encoding the content with a reduced frame size for latency-critical content, while switching back to a longer frame duration for non-latency-sensitive content. In particular, the encoding mode may be adjusted based on the sensitivity of the media program to playback latency.

[0024] In some examples, an indication of a change in sensitivity of a media program to playback delay may be signaled via an inband event. For example, such an inband event may be defined in accordance with ISO IEC 23009-1. Receipt of an inband event that signals a change in sensitivity of a media program to playback delay may then cause the playback device to reread the metadata (e.g., MPD) of the media program and reconfigure the internal processing of the media stream associated with the media program, as proposed in the present disclosure.

[0025] In some examples, the sensitivity of a particular media program to playback delay may be signaled before (before) the start of the media program. In this case, the playback device (e.g., the player of the playback device) may have started preparations in advance (e.g., preparing a buffer, adjusting a playback mode, etc.) so that a seamless viewing experience of the media program (with different delays) can be enjoyed.

[0026] In some examples, adjusting the playback of a media program may be asynchronous with the start of the media program. For example, a player application of a playback device may switch to a lower latency before or after a latency-sensitive program starts, i.e., in an asynchronous (or only loosely synchronized) manner that is independent of the start of the media program.

[0027] Optionally, adjusting the playback of the media program may also include adjusting the playback of a previous media program.

[0028] As another broad aspect, a device for playing back media content via a delivery channel is provided. A delivery channel may generally refer to a channel through which an audio or video program is delivered (sent) to a user (receiver) in a broadcast or multicast manner. For example, such a delivery channel may be a (traditional) "24 / 7" broadcast channel. Media content may generally include continuous media programs such as news, movies, advertisements, dramas, (live) game performances, live sports, etc. In particular, for a specific media program within the media content, the corresponding content type of the specific media program is also provided. As the technician understands and recognizes, the content type may be represented and delivered in any suitable manner. For example, a specific or additional field in the header of the media program may be used to represent or indicate the content type. As another example, a lookup table may be used to indicate the content type. Similarly, the content type may be delivered (sent) in real time with the delivery of the media program, or may be shared before the delivery of the media program. Typically, the device may be configured to perform any of the previous method steps.

[0029] In particular, the device may include a receiver for receiving an indication of the sensitivity of a media program to playback delay. More particularly, the sensitivity of a media program to playback delay may be associated with the content type of the media program. The device may further include a receiver for receiving at least a portion of a media program via a media stream. A portion of a media program generally refers to a certain number of frames (pictures, data blocks, etc.) of a media program, depending on, for example, the representation and / or delivery of the media program. As recognized by the technician, a receiver for receiving an indication of the sensitivity of a media program and a receiver for receiving a media program may be implemented as a single physical receiver capable of receiving and distinguishing these two types of information (i.e., sensitivity indication and media program) packaged in, for example, a single bitstream. In other words, a receiver for receiving a sensitivity indication of a media program and a receiver for receiving a media program may be logically separated, but not necessarily physically separated. Both types of information may be received at a single physical receiver. For example, only one physical receiver may be used to receive both the indication of the sensitivity of a media program and the media program. In this case, the device may include a suitable mechanism for separating different input signals that may be received at the same physical receiver. In some examples, the receiver is arranged to receive information via a communication network based on the Internet Protocol (IP). Different physical layers (wireless or wired, copper, fiber) and application protocols (UDP, TCP, http...) can be used to transmit media streams and / or metadata carrying the sensitivity of the media program to playback delay. The communication network is sometimes also referred to as a content delivery network.

[0030] The device may further include a playback unit for playing the media content. The playback unit may, for example, call some suitable applications to play back the media content.

[0031] The device may further include a controller for adjusting playback of the media program based on the received indication of sensitivity of the media program to playback delay.

[0032] In some examples, the playback unit of the device (e.g., a media player) may include a buffer for received media program segments. On the other hand, in some cases, the buffer may also be provided, for example, by an operating system, and the playback unit may then be configured to access and use the buffer in any suitable manner. The buffer may, for example, be implemented as a memory (storage) block or in any other suitable manner, as recognized by the skilled person. In addition, the controller may be configured to adjust the amount of program data buffered before playing back the program data based on an indication of sensitivity to playback delay.

[0033] In some examples, the device may further include a media content modifier, which is configured to adjust the content of the received media program to insert additional media content or remove media content. In some examples, the media content modifier may be configured to insert / add additional content (e.g., by frame or segment) in the current media program. For example, if the current media program allows a larger delay relative to a previous media program, or a subsequent media program allows a larger delay relative to the current media program, then the insertion of (multiple) additional content may be performed. In some other examples, the media content modifier may be configured to skip or remove specific content (or a portion thereof). For example, this may be performed when the current media program requires a smaller delay relative to a previous media program, or a subsequent media program requires a smaller delay relative to the current media program.

[0034] In some examples, a receiver for receiving an indication of the sensitivity of a media program to playback delay may be configured to decode metadata of the media program or to receive an in-band event, e.g., depending on how and where the indication of the sensitivity to playback delay (or such a change in sensitivity) is delivered to a playback device as described above. An example of metadata may be an MPD according to ISO IEC 23009-1, and an example of an in-band event may be an in-band event defined according to ISO IEC 23009-1.

[0035] As another broad aspect, a system for delivering media content via a delivery channel is provided. A delivery channel may generally refer to a channel through which an audio or video program is delivered (sent) to a user (receiver) in a broadcast or multicast manner. For example, such a delivery channel may be a (traditional) "24 / 7" broadcast channel. Media content may generally include continuous media programs such as news, movies, advertisements, dramas, (live) game shows, live sports, etc. In particular, for a specific media program within the media content, the corresponding content type of the specific media program is also provided. As the technicians understand and recognize, the content type may be represented and delivered in any suitable manner. For example, a specific or additional field in the header of a media program may be used to represent or indicate the content type. As another example, a lookup table may be used to indicate the content type. Similarly, the content type may be delivered (sent) in real time with the delivery of the media program, or may be shared before the delivery of the media program. The system may include a content delivery server and other components such as a content provider server.

[0036] In particular, the system may include a scheduler that is used to schedule media programs to be continuously delivered through a delivery channel. In general, the scheduling process may include the appropriate time for scheduling (allocating) delivery channels for media programs. As recognized by the technical staff, the scheduling of media programs may be performed according to different situations and / or requirements such as program agendas, the importance of programs and / or the urgency. The system may also include at least one packager (or encapsulator) that is configured to package (encapsulate) media programs in a continuous media stream in any suitable manner. In particular, the packager may be configured to add corresponding indications of the sensitivity of media programs to playback delays to the media stream. As described above, the sensitivity of media programs to playback delays is associated with the content type of media programs.

[0037] In some examples, the packager can be configured to further adjust the encoding mode used for the media program based on the media program's sensitivity to playback latency, for example, encoding the media program with reduced segment durations for latency-critical content and switching back to longer segment durations for non-latency-sensitive content.

[0038] In some examples, the packager may be configured to further insert an indication of the sensitivity of the media program to playback delay into metadata of the media program or into an in-band event, e.g., depending on how and where the indication of the sensitivity to playback delay is delivered to the playback device as described above. An example of metadata may be an MPD according to ISO IEC 23009-1, and an example of an in-band event may be an in-band event defined according to ISO IEC 23009-1.

[0039] Another aspect of the present disclosure relates to computer software, a computer program product or any medium or data embodying computer software instructions for execution on a programmable computer or dedicated hardware comprising at least one processor, which causes the at least one processor to perform the method steps disclosed in the present document, in particular the disclosed method proposed in the present document.

[0040] Implementations of the disclosed apparatus may include the use of, but are not limited to, one or more processors, one or more application specific integrated circuits (ASICs), and / or one or more field programmable gate arrays (FPGAs). Implementations of the apparatus may also include the use of other conventional and / or custom hardware such as software programmable processors.

[0041] It will be appreciated that device / apparatus features may be interchangeable in a variety of ways. In particular, as will be appreciated by the skilled person, details of the disclosed device may be implemented as a method, and vice versa. In addition, even if not explicitly disclosed, the above aspects may be combined in a variety of ways. The skilled person will appreciate that these combinations of aspects and features / steps are possible unless a conflict that is explicitly excluded arises.

[0042] Other and further embodiments of the present invention will become apparent during the course of the following discussion with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Embodiments of the present disclosure are explained below in an exemplary manner with reference to the accompanying drawings, in which:

[0044] Figure 1 Examples of media content are schematically illustrated;

[0045] Figure 2 An example of a system for delivering media content is schematically illustrated;

[0046] Figure 3 An example of a device for playing back media content according to an embodiment of the present invention is schematically illustrated; and

[0047] Figure 4 An example of a method for playing back media content according to an embodiment of the present invention is schematically illustrated. DETAILED DESCRIPTION

[0048] Figure 1 Schematically illustrates an example of a portion of a media content stream 100 according to an embodiment of the present invention. The media content stream 100 may generally include consecutive (different) media programs 101-106. Figure 1 In the example shown, the media content stream 100 may include a live sports program 101 (e.g., a football game) during a time period t1 to t2; an advertisement program 102 during a time period t2 to t3; a news program 103 during a time period t3 to t4; a movie program 104 during a time period t4 to t5; another news program 105 during a time period t5 to t6; and a live game show 106 during a time period t6 to t7. It should be noted that Figure 1 Only the portion of the media content delivery channel that may operate continuously in a "24 / 7" manner is shown.

[0049] As those skilled in the art will appreciate, the media content stream 100 may include other media programs at different time periods and may include other media programs such as Figure 1 The media content stream 100 shown is only an example for illustration purposes. It is worth noting that the media content stream 100 can be transmitted via a content delivery network ( Figure 1 The delivery channel (not shown) is delivered from the content distributor. The delivery channel can generally refer to a channel (e.g., a wireless or wired channel) through which the audio or video program is delivered (sent) to the end user (receiver / viewer) in a broadcast, multicast or unicast manner, or even in a point-to-point (e.g., video on demand, VOD) manner. For example, such a delivery channel can be a (traditional) "24 / 7" broadcast channel or any other suitable (multiple) channel, as understood and recognized by the skilled person. The viewer (playback device) can then play back the received (multiple) media program in an appropriate manner (e.g., using an appropriate player application, calling an appropriate codec, etc.).

[0050] In addition, in order to facilitate the delivery of the media content stream 100, the content type of the specific media program in the media content stream 100 may be provided. Figure 1 ). Typically, the content type of a media program provides a broad classification of media programs in a variety of content categories such as news, sports, performances, movies, etc. The content type can be set by the content provider that produces the media program, or it can be set by the operator of the content delivery network. As the technician understands and recognizes, the content type can be represented and delivered in any suitable manner. For example, a specific or additional field in the header of the media program can be used to represent or indicate the content type. As another example, a lookup table can be used to indicate the content type. Similarly, the content type can be delivered (sent) in real time with the delivery of the media program, or it can be shared before the delivery of the media program.

[0051] In addition, an indication of the sensitivity of a particular media program to playback delay (also referred to as target delay) may also be provided. In particular, the sensitivity of a media program to playback delay may be a qualitative indication or a quantitative indication of a delay that can be accepted before playing back the media program data, and the qualitative indication or the quantitative indication of the delay is associated with the content type of the media program.

[0052] The key element in determining the target delay of a media program is the content type. In other words, the content type influences / dominates the specification of the target delay. However, there may be specific steps to get an indication of the target delay from the content type, because the mapping is not always clear. For example, a content distributor may select different target delays for different football matches based on the importance of the football match. There are also other content types (such as non-live content), for which the content distributor can select a fixed default value for the target delay without further adjustment / fine-tuning. But in other cases, the person who creates the program schedule may choose to deviate from such a default value (for example, the football final in the World Championship will achieve the lowest possible delay, while other football matches will not target such an extremely low number). Therefore, the target delay of a media program is determined based on the content type, but is not set exclusively according to the content type.

[0053] On the playback device side, at least a portion of the media program and the target delay of the media program are received. A portion of a media program generally refers to a certain number of frames (pictures, data blocks, etc.) of the media program, depending on, for example, the representation and / or delivery method / mechanism of the media program. Next, the playback of the media program is adjusted based on the target delay of the media program. The target delay can be changed (e.g., from low to high) in a single step, and the media player in the playback device can be adjusted accordingly. In addition, it is proposed to adjust the media player in the playback device within a specific time period, for example, in order to catch up with or slow down the buffering degree and / or playback speed. The broadcast adjustment algorithm on the receiving side can have a given characteristic of interpolating between a given target delay and an update in the target delay (if the jump of the target delay is too drastic, the update in the target delay may be performed in an undesirable manner). Therefore, the indicated target delay on the server side can be changed step by step (in multiple steps), so as to remotely control the adjustment speed of the receiver side and manage the buffering degree on the receiver side before transitioning to another program with a different target delay. Therefore, in addition to a single jump in the target delay, it is also proposed to adjust the playback of the media program by multiple steps arranged in a specific time period. In the latter case, the server gradually changes the signaled target delay with multiple sequential indications for the playback delay.

[0054] Figure 2 Schematically illustrates an example of a system 200 for delivering media content. The media content may include, for example, Figure 1 The continuous media program shown in 101-106 is shown. Figure 2 In the example of FIG. 2 , the system 200 may include, for example, a service (content) provider 210 for preparing and providing different media programs (e.g., Figure 1101-106 shown in the figure). By way of example and not limitation, content provider 210 may be an operator, a video program platform, or any other suitable provider as recognized by a skilled person. In particular, in some cases, the program may be a live (real-time) program 211 such as sports, live game performances, etc. In some other cases, the program may be a non-real-time program 212 such as an advertisement, a movie, etc. Such a non-real-time program 212 may be prepared (produced) before the program starts.

[0055] Available programs 211 and 212 may be delivered / sent to a scheduler 220 of the system 200 in real time or in advance. As will be appreciated and appreciated by the skilled person, the scheduler 220 may be implemented, for example, as a standalone application, a functional block of a controller, or in any other suitable manner. The scheduler 220 may reside in a computer or server that is connectable to the content provider 210 (in a suitable wireless, wired, etc. manner). The scheduler may be part of a content distribution system.

[0056] Scheduler 220 can be configured to schedule media programs to be continuously delivered through delivery channels. As recognized by the technician, the scheduling of media programs can be performed according to different situations and / or requirements such as program agendas, program priorities, etc. Generally speaking, the scheduling process can include, for example, scheduling (allocating) appropriate time resources for delivery channels for media programs based on the characteristics of delivery channels. Then, the scheduled media programs can be delivered / transmitted to, for example, encoders, packagers, segmenters, encryptors and / or manifest generators (these functions can appear in slightly different orders and can be combined into fewer processing steps) 230 via suitable interface 221, so that the media programs can be packaged / encoded in continuous media streams in any suitable manner. Then, the packaged media programs (i.e., media content) can be delivered (e.g., broadcast) to the playback device 240 of the terminal user via suitable network connection 231 (e.g., IP connection). Coding / decoding can refer to the corresponding processing of any media content, such as audio and / or video coding / decoding, etc.

[0057] In addition to appropriately scheduling and delivering the media program, the scheduler 220 may also be responsible for indicating the sensitivity of the media program to playback delay to (multiple) downstream devices (such as at least one packager 230) via an appropriate interface 222. Similarly, the at least one packager 230 may correspondingly propagate the indication of the sensitivity of the media program to playback delay to the end user's playback device 240 via an appropriate network connection 232 in an appropriate manner.

[0058] It is worth noting that in some cases, the packager 230 can be configured to package (multiplex) the indication of the sensitivity of the media program to playback delay in the bitstream together with the media program itself. For example, the packager 230 can package (encapsulate) the indication of the sensitivity of the media program to playback delay in the metadata of the media program. An example of metadata can be an MPD according to ISO IEC 23009-1. In addition, changes in metadata such as indications of sensitivity to playback delay can be signaled via in-band events. Configured like this, the two logical (network) connections 231 and 232 used to deliver media programs and sensitivity indications, respectively, do not need to be separated, so only one suitable connection can be used to deliver both media programs and sensitivity indications to the playback device 240.

[0059] In some cases, the packager 230 can be configured to further adjust the encoding mode used for the media program based on the media program's respective sensitivity to playback delay, for example, encoding the media program with reduced segment durations for delay-critical content and switching back to longer segment durations for non-delay-sensitive content.

[0060] As the technician understands and recognizes, the playback device 240 typically includes one or more suitable components, such as (multiple) receivers, (multiple) buffers, processors, displays / displays, speakers, etc., which will be described in more detail later. In particular, in order to correctly play back the received media content, the playback device 240 typically needs to establish its playback chain (e.g., application, codec, buffer, etc.), especially in order to support the expected delay of the media program. In particular, according to different situations, the setting can include reducing (or increasing) the number of fragments (or frames) buffered in the playback device to reduce (or increase) the delay. In addition, according to different situations (such as the transmission format used), the setting can also include switching to a specific (e.g., low-delay) playback mode. An example of a low-delay mode can be a low-delay presentation specified in ETSI TS 103 285V1.3.1 based on the Common Media Application Format (CMAF) specified in ISO / IEC 23000-19.

[0061] Figure 3 An example of a device 300 for playing back media content according to an embodiment of the present invention is schematically illustrated. The device 300 may correspond to Figure 2 The media content provided by the delivery channel may correspond to or be similar to Figure 1 The media content 100 includes a continuous media program and has a corresponding content type associated with the media program.

[0062] In particular, the device 300 may include a receiver 301 for receiving an indication of the sensitivity of a media program to playback delay. The sensitivity of a media program to playback delay is associated with the content type of the media program. As shown above, before being received by the device 300, the indication of sensitivity may be Figure 2 The scheduler 220 schedules and is Figure 2 The receiver 301 may be packaged by at least one packager 230 of the packetizer. Depending on how the indication of sensitivity is packaged (encoded) on the packetizer side, the receiver 301 may also be configured to appropriately unpack (extract or decode) such indication. For example, if the indication of sensitivity is located in metadata (such as an MPD according to ISO IEC 23009-1), the receiver 301 may be configured to extract the indication from such metadata. In some examples, the metadata is provided as a configuration file that is read by the receiver 301 at startup, when switching to a delivery channel, when initially accessing a media stream, and / or when a new media program within a delivery channel starts.

[0063] The device 300 may further include a receiver 302 for receiving at least a portion of a media program. A portion of a media program generally refers to a certain number of frames (pictures, data blocks, etc.) of the media program, depending on, for example, the representation and / or delivery of the media program.

[0064] As recognized by the skilled person, the receiver 301 for receiving the sensitivity indication (i.e., target delay) of the media program and the receiver 302 for receiving the media program can be implemented as a single receiver capable of receiving and distinguishing the two inputs (i.e., sensitivity indication and media program). In other words, the receiver 301 for receiving the sensitivity indication of the media program and the receiver 302 for receiving the media program can be logically separated, but not necessarily physically separated. Both inputs can be received at a single physical receiver. For example, only one physical receiver can be used to receive both the sensitivity indication of the media program and the media program. In this case, the device 300 may include a suitable mechanism for separating different input signals that may be received at the same physical receiver.

[0065] The device 300 may further include a playback unit 310 for playing a media program of the media content. The playback unit 310 may, for example, call a certain suitable application (and optionally suitable codec) to playback the media content, as recognized by the technician. In some examples, the playback unit of the device may include a buffer 315 for storing and accessing the received fragments of the media program. On the other hand, in some cases, the buffer may also be provided, for example, by an operating system, and then the playback unit may be configured to access and use the buffer in any suitable manner. The buffer 315 may, for example, be implemented as a memory (storage) block or implemented in any other suitable manner, as recognized by the technician.

[0066] The device 300 may further include a controller 320 for adjusting the playback of the media program based on an indication of the sensitivity of the received media program to the playback delay. In some cases, the controller 320 may be configured to adjust the amount of program data buffered before the playback program data based on an indication of the sensitivity to the playback delay. In particular, the change on the playback device 300 can be completed asynchronously. That is, (e.g., on the player application of the device 300) the playback can be switched to a lower delay before or after the delay-sensitive program starts (e.g., by the controller 320). In either case, the playback is required to catch up with the lower delay playback from the higher delay playback, or increase its delay. The controller 320 may include a broadcast adjustment algorithm for interpolating between the current target delay and the newly received target delay to avoid the unexpected drastic jump in the target delay.

[0067] In some cases, the device 300 may further include a media content modifier 330, which is configured to adjust the content of the received media program to insert additional media content or remove media content. In particular, the media content modifier 330 may be configured to insert / add additional content (e.g., by frame or segment) in the current media program. Additional content may be, for example, a "silent" (empty) segment or a repeated segment. For example, if the current media program allows a larger delay relative to a previous media program, or a subsequent media program allows a larger delay relative to the current media program, then the insertion of (multiple) additional content may be performed. In some other examples, the media content modifier 330 may be configured to skip or remove specific content (or a portion thereof). For example, this may be performed when the current media program requires a smaller delay relative to a previous media program, or a subsequent media program requires a smaller delay relative to the current media program. However, in some cases, this change of adding or removing content may be noticed by viewers, and therefore may be considered to be less desirable.

[0068] Alternatively or additionally, the controller 320 (or some other component of the device 300) can be configured to control the playback speed to be faster or slower than real time to catch up with progress or increase delay. For example, the playback unit 310 may include an algorithm that can change the playback speed, which is usually combined with the pitch correction of the audio. Generally, the playback speed should be changed without changing the pitch of the audio signal. Generally speaking, a playback speed difference of 10%-20% is generally considered to be unnoticed / unpleasant to the viewer. Therefore, the transition period from a higher delay (e.g., 25 seconds) to a lower delay (e.g., 2 seconds) may require playback to be accelerated (e.g., 10%) for about 230 seconds-nearly 4 minutes. In this case, this also means that programs (e.g., advertisements) in a short period of time are unlikely to change the delay because the transition period is longer than the actual content.

[0069] Furthermore, to prevent re-buffering events (e.g., in case the player needs to interrupt playback due to loss of data for decoding), particularly when switching from small segments to large segments (from low latency to high latency), the device 300 should be notified of this change in advance. For example, the in-band events defined in ISO / IEC 23009-1 can be used for such signaling. As indicated above, the same or different in-band events can also be used to notify the player of changes in sensitivity to latency. For example, in-band events can also be used to signal changes in the MPD. Upon receipt of such an in-band event, the player can access the MPD again to determine changes in the program metadata (such as changes in sensitivity to playback latency).

[0070] It should be noted that for the sake of brevity, Figure 3 The interconnections between the various units / components within the device 300 are omitted. However, as the skilled person will appreciate and understand, if necessary, the device 300 may include and / or provide such interconnections for exchanging (internal) messages between the units / components within the device 300. It should also be noted that other units / components may also be present in the device 300 for the successful operation of the device 300, although they may not be present in the device 300. Figure 3 These units / components may include, for example, but are not limited to, a display / screen for displaying the replayed program, one or more speakers for outputting audio, and the like.

[0071] Figure 4 An example of a method 400 for playing back media content according to an embodiment of the present invention is schematically illustrated. The method 400 may be, for example, Figure 3 The media content may correspond to or be similar to Figure 1 The media content 100 includes a continuous media program and has a corresponding content type associated with the media program.

[0072] In particular, method 300 may include a step S410 of receiving an indication of the sensitivity of the media program to playback delay. In particular, the sensitivity of the media program to playback delay is associated with the content type of the media program. Method 400 may further include a step S420 of receiving at least a portion of the media program. Finally, method 400 may also include a step S430 of adjusting the playback of the media program based on the indication of the sensitivity of the media program to playback delay.

[0073] It should be noted that the above-mentioned device (equipment) features correspond to corresponding method features, however, for reasons of brevity, these method features may not be explicitly described. The disclosure of this document is considered to also extend to such method features. In particular, the present disclosure is understood to relate to methods for operating the above-mentioned devices, and / or to methods for providing and / or arranging the corresponding elements of these devices.

[0074] It should also be noted that the disclosed embodiments may be implemented in a variety of ways using hardware and / or software configurations. For example, the disclosed embodiments may be implemented using dedicated hardware and / or hardware associated with software that can be executed thereon. The components and / or elements in the accompanying drawings are merely examples and do not limit the scope of use or functionality of any hardware, software in combination with hardware, firmware, embedded logic components, or a combination of two or more such components that implement a particular embodiment of the present disclosure.

[0075] It should also be noted that the description and drawings illustrate only the principles of the present disclosure. Those skilled in the art will be able to implement various arrangements that embody the principles of the present invention and are included in its spirit and scope, even if these arrangements are not explicitly described or shown herein. In addition, all examples and embodiments summarized in this disclosure are primarily and explicitly intended to be used only for explanatory purposes, in order to help the reader understand the principles of the proposed method. In addition, all statements herein providing the principles, aspects and embodiments of the present invention and their specific examples are intended to cover their equivalents.

[0076] Various aspects of the present invention may be understood from the following enumerated example embodiments (EEE):

[0077] 1. A method for playing back media content via a delivery channel, the media content comprising a continuous media program having a corresponding content type, the method comprising:

[0078] - receiving an indication of a sensitivity of a media program to playback delay, the sensitivity of the media program to playback delay being associated with a content type of the media program;

[0079] - receiving at least a portion of the media program;

[0080] -Adjusting playback of the media program based on the indication of sensitivity of the media program to playback latency.

[0081] 2. The method of claim 1, wherein adjusting the playback of the media program comprises adjusting the buffering of the media program, in particular adjusting the amount of the media program data buffered before playing back the media program data.

[0082] 3. The method of claim 1 or 2, wherein the indication of the media program's sensitivity to playback delay comprises a qualitative indication of an acceptable delay before playing back the media program data.

[0083] 4. A method as described in any preceding EEE, wherein adjusting the playback of the media program includes adjusting the playback speed of the media program.

[0084] 5. A method as described in any of the preceding EEEs, wherein adjusting the playback of the media program includes modifying the content of the media program if the indication of the received media program's sensitivity to playback latency changes relative to a previous or subsequent media program on the delivery channel.

[0085] 6. The method of EEE 5, wherein if the current media program allows a greater delay relative to the previous media program, or the subsequent media program allows a greater delay relative to the current media program, additional content is inserted into the current media program.

[0086] 7. The method of EEE 5, wherein if the current media program requires a smaller delay relative to the previous media program, or the subsequent media program requires a smaller delay relative to the current media program, content is skipped in the current media program.

[0087] 8. A method as described in EEE 6 or 7, wherein the media program includes multiple consecutive segments, and wherein adjusting the playback of the media program includes adjusting the number of buffered segments before playing back the media program data, wherein at least one segment of the media program is optionally skipped or repeated.

[0088] 9. A method as described in any preceding EEE, wherein adjusting the playback of the media program includes switching a playback mode for the media program in a media player of a playback device.

[0089] 10. A method as described in any preceding EEE, wherein another indication of sensitivity of the media program to playback delay is received, and playback of the media program is gradually adjusted based on the received indication.

[0090] 11. A method as described in any preceding EEE, wherein the indication of the sensitivity of the media program to playback delay is generated by a scheduler for the continuous media program delivered via the delivery channel, and the indication is optionally added to the metadata of the media program.

[0091] 12. The method as described in any preceding EEE further comprises adjusting an encoding mode for the media program according to the sensitivity of the media program to playback delay.

[0092] 13. A method as in any preceding EEE, wherein the indication of a change in the sensitivity of the media program to playback delay is signaled via an in-band event, for example in accordance with ISO IEC 23009-1.

[0093] 14. The method of EEE 13, wherein the sensitivity of the media program to playback delay is signaled before the media program begins.

[0094] 15. The method of EEE 14, wherein adjusting playback of the media program is asynchronous with the start of the media program and optionally includes adjusting playback of a previous media program.

[0095] 16. An apparatus for playing back media content via a delivery channel, the media content comprising a continuous media program having a corresponding content type, the apparatus comprising:

[0096] - a receiver for receiving an indication of a sensitivity of a media program to playback delay, said sensitivity of a media program to playback delay being associated with a content type of said media program;

[0097] - a receiver for receiving at least a portion of the media program;

[0098] - a playback unit, configured to play the media content; and

[0099] - A controller for adjusting playback of the media program based on the received indication of sensitivity of the media program to playback latency.

[0100] 17. The apparatus of EEE 16, wherein the playback unit includes a buffer for received segments of the media program, and the controller adjusts an amount of the program data buffered prior to playing back the program data based on the indication of sensitivity to playback latency.

[0101] 18. The apparatus according to EEE 16 or 17, further comprising a media content modifier for adjusting the content of the received media program to insert additional media content or remove media content.

[0102] 19. The apparatus of EEEs 16 to 18, wherein the receiver for receiving an indication of sensitivity of a media program to playback latency is configured to perform at least one of: decoding metadata of the media program, and receiving an in-band event.

[0103] 20. A system for delivering media content via a delivery channel, the media content comprising continuous media programs having corresponding content types, the system comprising:

[0104] - a scheduler for scheduling the media program for continuous delivery via the delivery channel;

[0105] -At least one packager for packaging the media program in a continuous media stream, wherein the packager is configured to add an indication of the media program's sensitivity to playback delay to the media stream, the media program's sensitivity to playback delay being associated with a content type of the media program.

[0106] 21. The system of EEE 20, wherein the packager is configured to adjust an encoding mode for the media program according to the sensitivity of the media program to playback delay.

[0107] 22. A system as described in EEE 20 or 21, wherein the packager is configured to perform at least one of the following: packaging the indication of the media program's sensitivity to playback delay in metadata of the media program, and signaling changes in playback delay via in-band events.

Claims

1. A method for playing back media content via a delivery channel, the media content comprising a continuous media program having a corresponding content type, the method comprising: receiving a plurality of indications of a sensitivity of a media program to playback delay over a period of time, the sensitivity of the media program to playback delay being associated with the content type of the media program; receiving at least a portion of the media program; Before transitioning to playback of the media program, adjusting playback of the current media program in response to the plurality of indications until a target latency for the media program is reached; as well as The media program is played back according to the target delay.

2. The method of claim 1, wherein: The indication of the sensitivity of a media program to playback delay comprises a qualitative indication of a delay that is acceptable prior to playback of the media program data.

3. The method according to claim 1 or 2, wherein: The adjusting playback of the media program includes modifying content of the media program if the indication of sensitivity to playback delay for the media program changes relative to a previous or subsequent media program of the delivery channel.

4. The method of claim 3, wherein: If the media program allows a greater delay relative to the previous media program, or the subsequent media program allows a greater delay relative to the media program, inserting additional content into the media program, or If the media program requires a smaller delay than the previous media program, or the subsequent media program requires a smaller delay than the media program, content is skipped in the media program.

5. The method of claim 4, wherein: The media program includes a plurality of consecutive segments, and wherein said adjusting playback of the media program includes adjusting a number of buffered segments prior to playing back the media program data.

6. The method of claim 5, wherein: The adjusting the number of buffered segments before playing back the media program data includes skipping or repeating at least one segment of the media program.

7. The method according to claim 1 or 2, wherein: The adjusting the playback of the media program includes switching a playback mode for the media program in a media player of a playback device.

8. The method according to claim 1 or 2, wherein: The plurality of indications of the sensitivity of a media program to playback latency are generated by a scheduler for the continuous media program delivered over the delivery channel.

9. The method of claim 8, wherein: The plurality of indications are added to metadata of the media program.

10. The method according to claim 1 or 2, wherein: An indication of the change in the sensitivity of the media program to playback delay is signaled via an in-band event.

11. The method of claim 10, wherein: The in-band events are defined according to ISO IEC 23009-1.

12. The method of claim 10, wherein: The sensitivity to playback delay for a media program is signaled before the media program begins.

13. The method of claim 12, wherein: The adjusting the playback of the media program is asynchronous with the start of the media program.

14. The method of claim 13, wherein: The adjusting the playback of the media program includes adjusting the playback of a previous media program.

15. An apparatus for playing back media content via a delivery channel, the media content comprising a continuous media program having a corresponding content type, the apparatus comprising: a receiver for receiving a plurality of indications of sensitivity of a media program to playback delay, the sensitivity of the media program to playback delay being associated with the content type of the media program; a receiver for receiving at least a portion of the media program; a controller configured to: adjust playback of a current media program in response to the plurality of indications before transitioning to playback of the media program until a target latency of the media program is reached; as well as A playback unit is used to play back the media program according to the target delay.

16. The apparatus of claim 15, further comprising a media content modifier for adjusting the content of the received media program to insert additional media content or remove media content.

17. The apparatus of claim 15 or 16, wherein: The receiver for receiving the plurality of indications of sensitivity of the media program to playback latency is configured to at least one of decode metadata of the media program and receive an in-band event.

18. A system for delivering media content via a delivery channel, the media content comprising continuous media programs having corresponding content types, the system comprising: - a scheduler for scheduling the media program for continuous delivery via the delivery channel; as well as -At least one packager for packaging the media program in a continuous media stream, wherein the packager is configured to add multiple indications of sensitivity of at least one media program to playback delay to the media stream, wherein the sensitivity of the at least one media program to playback delay is associated with the content type of the at least one media program, and wherein the multiple indications of the sensitivity of the at least one media program to playback delay cause playback of a current media program on a playback device to be adjusted until a target delay of the at least one media program is reached.

19. The system of claim 18, further comprising adjusting an encoding mode for the media program based on the sensitivity of the media program to playback delay.

20. The system of claim 18 or 19, wherein: The packager is configured to at least one of package the plurality of indications of the sensitivity of the media program to playback latency in metadata of the media program and signal a change in playback latency via an in-band event.

21. A computer program product comprising computer executable instructions for executing the method according to any one of claims 1 to 14 by one or more processors.

Images