METHOD FOR PROCESSING VIDEO STREAMS AND EQUIPMENT FOR IMPLEMENTING THE METHOD

The method inserts traceability data into signaling messages to measure latency in video streams across the distribution chain, addressing the limitations of existing methods by ensuring accurate and flexible latency assessment without degrading the stream.

FR3149455A1Active Publication Date: 2024-12-06ATEME
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
FR2023005553
Authority / Receiving Office
FR · FR
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-06-02
Publication Date
2024-12-06
Estimated Expiration
2043-06-02

AI Technical Summary

Technical Problem

Existing methods for measuring latency in video stream distribution chains modify the video stream, degrading it and limiting the ability to perform other measurements, and are limited to decoded streams, introducing bias due to uncontrolled decoding times.

Method used

A method that inserts traceability data into signaling messages within the video stream, allowing latency measurement without degrading the stream, and can be performed at various points in the distribution chain using signaling standards like SCTE-35, enabling accurate latency determination.

Benefits of technology

Enables accurate latency measurement across the distribution chain without degrading the video stream, allowing flexibility in measurement points and avoiding the need for video playback devices, thus providing reliable and flexible latency assessment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

A method for processing a multimedia stream comprising a sequence of images is proposed, which includes, in a multimedia stream management unit: generating traceability metadata for an image in the image sequence; inserting the image traceability metadata into a supplementary data stream obtained by the multimedia stream management unit; and multiplexing the multimedia stream and the supplementary data stream. Abstract figure: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

Title of the invention: METHOD FOR PROCESSING VIDEO STREAMS AND EQUIPMENT FOR IMPLEMENTING THE METHOD Technical field

[0001] The present disclosure relates to a method for processing video streams and a device for implementing this method. It applies in particular to the measurement of latency in a distribution chain of a video stream. Prior art

[0002] Video data is generally subject to source coding aimed at compressing it in order to limit the resources required for its transmission and / or storage. There are numerous coding standards, such as H.264 / AVC, H.265 / HEVC and MPEG-2, which can be used for this purpose. A video stream comprising a set of images is considered. In conventional coding schemes, the images of the video stream to be encoded are typically considered according to an encoding sequence, and each is divided into sets of pixels which are also processed sequentially.

[0003] The distribution of multimedia content, such as video streams, typically uses a video distribution network head comprising a video encoding unit which receives content to be encoded as input and delivers encoded streams called “elementary streams” to a formatting unit (sometimes called a “packager”). Content received as input is encoded according to a plurality of encoding profiles (a profile being for example defined with codec, resolution and bit rate parameters).

[0004] The data produced by the packager on the basis of the elementary streams are typically provided by the packager to a content server, then stored on content distribution networks, called "CDN" (from the English "Content Delivery Network"), providing cache capabilities which make it possible to improve the quality of the services, and to minimize the access times and latency for the viewing of content by one or more reading terminals (also commonly called "video players").

[0005] To measure latency in an audiovisual broadcasting chain in which a video stream is distributed, one possibility is to write (the English term "burn" is sometimes used to designate this writing or marking) the current date in an indicator of the video stream before encoding, then to compare this time indicator with the date at the time of display by a video player. The date indicator can take the form of a quick-read code (in English, "QR code"), in order to automate the measurement of latency at the time of display. However, this method requires modifying the images of the video stream by embedding an additional element (for example by embedding date information to perform a latency measurement), so that at least some of the images of the video stream on which the measurement is performed are irreparably modified.

[0006] This method of writing useful information for the desired measurement into the video stream has several drawbacks: on the one hand, the video stream is degraded, which prevents any measurement from being carried out in a production context and limits other measurements that would otherwise be possible on the stream, such as, for example, a quality comparison measurement between the source video stream and the video stream output from the broadcast chain. On the other hand, it is only possible to mark an uncompressed image, which excludes any measurement in an element of the distribution chain that does not process the decoded stream, such as, for example, the packager, and thus reduces the number of elements in the chain from which a latency measurement can be carried out. Indeed, since the latency measurement is carried out on decoded images, it is necessary to use a video player placed at each measurement point.However, the decoding time of this player is not controlled, and introduces a bias in the latency measurement.

[0007] There is thus a need for a method for processing video streams, and for equipment for implementing the method, making it possible to avoid or mitigate the aforementioned drawbacks.

[0008] There is also a need for a video stream processing method making it possible to measure the latency during the distribution of the stream in a distribution chain (or video stream distribution), and for equipment for implementing the method, making it possible to avoid or mitigate the aforementioned drawbacks. Summary

[0009] The present disclosure improves the situation.

[0010] According to a first aspect, there is provided a method for processing a multimedia stream comprising a sequence of images, the method being able to be implemented by a multimedia stream management unit, for example of a multimedia distribution system, and comprising: generating signaling message traceability data; modifying a signaling message comprising a marker of an image of the sequence of images, the signaling message being obtained by the multimedia stream management unit, the signaling message being modified by inserting the signaling message traceability data into the signaling message; and inserting the modified signaling message into the multimedia stream.

[0011] According to a variant of the first aspect, there is provided a method of treating a multimedia stream comprising a sequence of images, the method being able to be implemented by a multimedia stream management unit, for example of a system of dis multimedia delivery, and comprising: generating signaling message traceability data; modifying a signaling message of an event associated with an image of the image sequence, the signaling message being obtained by the multimedia stream management unit, the signaling message being modified by inserting the signaling message traceability data into the signaling message; and inserting the modified signaling message into the multimedia stream.

[0012] The method proposed according to the first aspect may advantageously be implemented within any device, such as for example a multimedia stream management unit (such as, for example, a video stream management unit), configured to be implemented within an audiovisual broadcasting chain, for example according to a video stream transport scheme of the MPEG2-TS type (from the English “Moving Picture Expert Group 2 - Transport Stream”), “HLS” (from the English “HTTP Live Streaming”, the acronym “HTTP” designating the “HyperText Transfer Protocol”), “MSS” (from the English “Microsoft Smooth Streaming”), “HDS” (from the English “HTTP Dynamic Streaming”), “MPEG-DASH” (from the English “MPEG Dynamic Adaptive Streaming over HTTP”), CMAF-ingest (from the English “Common Media Application Format”), CMAF or “HAS” (from the English “HTTP Adaptive Streaming”),or by using a technology for broadcasting multimedia content on a television broadcast network, for example according to a “DVB” (from the English “Digital Video Broadcast”) type scheme, or an “ATSC” (from the English “Advanced Television Systems Committee”) type scheme, such as, but not limited to, a video encoder, for example conforming to the current AV 1 specification or its evolutions, a packaging unit, any computer, server, audiovisual broadcasting network head-end equipment, audiovisual broadcasting network equipment, etc.,

[0013] In one or more embodiments, the message traceability data is inserted into a field of the signaling message.

[0014] In one or more embodiments, the signaling message is included in the media stream, and the proposed method further comprises: extracting the signaling message from the media stream.

[0015] According to a second aspect, there is proposed a method for processing a multimedia stream distributed by a multimedia distribution chain in a multimedia distribution system, the multimedia stream comprising a sequence of images, the method being implemented by a processing unit of the multimedia distribution system and comprising: upon detection, in the multimedia stream, at a first point of the multimedia distribution chain, of a signaling message comprising a marker of an image of the sequence of images, extracting from the signaling message signaling message traceability data (for example traceability during the transporting this message in the multimedia distribution chain), upon extracting the signaling message from the signaling message traceability data, generating first detection timestamp data of the signaling message at the first point of the multimedia distribution chain; upon detecting, in the multimedia stream, the signaling message at a second point of the multimedia distribution chain, upon determining that the signaling message includes the signaling message traceability data, generating second detection timestamp data of the signaling message at the second point of the multimedia distribution chain; and determining a latency measure associated with the image based on the first and second timestamp data.

[0016] The proposed method advantageously makes it possible to determine a latency measurement without degrading the video stream used for its implementation. Indeed, the proposed method makes it possible to perform a latency measurement on a multimedia stream distributed by the devices of a multimedia distribution chain by tracing at several points of the distribution chain a signaling message of the stream (for example integrated into the multimedia stream upstream of the distribution chain) which references an image of the multimedia stream. Detection timestamps of the signaling message can be noted respectively at several latency measurement points on the distribution chain, so as to determine a time difference between two timestamps.The latency measurement can then be determined for the image based on the time difference determined between the timestamps of the signaling message passing, and the time differences determined between the signaling message and the image at the different measurement points. For example, the time difference determined between the timestamps of the signaling message passing can be combined with the time differences determined between the signaling message and the image at the different measurement points, to obtain a latency measurement for the image. The use of a signaling message has the advantage of making it possible to temporally trace the passage of an image to which it refers at different measurement points in the distribution chain.Signaling standards such as the SCTE-35 standard can advantageously be used for the implementation of the proposed method within a distribution chain, for example of the OTT type, given that the messages of this standard are commonly used, in particular to signal breaks in flow to insert advertisements, and that they are therefore not filtered by the nodes of the distribution chain.

[0017] Furthermore, the proposed method allows the implementation of an automated latency measurement, for example by using a flow management unit configured to implement an embodiment of the proposed method of latency measurement at several measurement points of a distribution chain.

[0018] Furthermore, the proposed method can be implemented in any element of a distribution chain, including an element of the chain processing compressed streams, so that it offers increased flexibility by allowing latency measurements at different levels of the distribution chain.

[0019] Furthermore, unlike methods which use a modification of an image of the multimedia stream by embedding time stamp information therein or which use a subtitling data system, the proposed method advantageously makes it possible to obtain a latency measurement of an image of a multimedia stream distributed to video playback devices by a multimedia distribution chain, without necessarily having to use a video playback device (for example to access time stamp information embedded on an image of the stream or inserted in subtitling data), while allowing a latency measurement from different points of a video stream distribution chain.

[0020] The proposed method, according to its various aspects, advantageously uses a signaling message of the multimedia stream which comprises a marker of the image for which the latency in transport within the distribution chain is to be measured. The proposed method uses traceability data of the signaling message in order to be able to detect the passage of the signaling message at at least two distinct points in the distribution chain of the multimedia stream. Depending on different embodiments, the signaling message used to determine a latency measurement in the distribution chain may already be present in the multimedia stream, for example in cases where it has been inserted into this stream upstream of the proposed multimedia stream management unit, or be generated by the proposed multimedia stream management unit.

[0021] The proposed method is particularly well suited, although not exclusively, for measuring latency and / or for detecting and measuring latency drift in an audiovisual broadcasting chain, between a multimedia stream management unit of the chain and any processing and / or distribution device of the distribution chain placed downstream of the multimedia stream management unit.

[0022] The proposed method is not limited to implementations within a multimedia stream management unit, and may be implemented in any multimedia stream processing unit configured for the implementation of a method proposed according to an embodiment of the present disclosure. The multimedia stream processing unit may be implemented in software form, in which case it will take the form of a program executable by a processor, in hardware form, such as an application-specific integrated circuit (ASIC), a system-on-chip (SOC), or in the form of a combination of hardware and software elements, such as a system-on-chip (SOC). computer system comprising a processor, in which case it may comprise an input interface, a memory operatively coupled with at least one processor configured to implement the proposed method, or as a component of programmable logic arrays (FPGA, from the English "Field Programmable Gate Arrays"). Depending on the chosen embodiment, the video stream processing unit may be implemented, in whole or in part, within a video encoder, a packager, a video decoder, a video sensor and / or any computer or FPGA component configured to implement the multimedia stream processing unit.

[0023] The features set out in the following paragraphs may optionally be implemented. They may be implemented independently of one another or in combination with one another.

[0024] In one or more embodiments, the method proposed according to the second aspect further comprises: determining a first time difference between the first detection timestamp data of the signaling message at the first point and a time of making the image available at the first point, and determining a second time difference between the second detection timestamp data of the signaling message at the second point and a time of making the image available at the second point, and the latency measurement is determined based on a combination of the first timestamp data, the second timestamp data and the first and second time differences.

[0025] In one or more embodiments of the method proposed according to the second aspect, the multimedia distribution chain comprises a shaping unit, the first point of the multimedia distribution chain is upstream of the shaping unit and the second point of the multimedia distribution chain is downstream of the shaping unit, and the proposed method further comprising: extracting from the multimedia stream at the second point of the multimedia distribution chain a metadata of the image indicating a positioning of the image within a set of encoded images.

[0026] In one or more embodiments of the method proposed according to the first and / or second aspect, the signaling message traceability data comprises identification data usable for identifying the signaling message.

[0027] In one or more embodiments of the method proposed according to the first and / or second aspect, the marker of the image comprises information relating to an event associated with the image.

[0028] In one or more embodiments of the method proposed according to the first and / or second aspect, the signaling message comprises a time of occurrence of the event.

[0029] In one or more embodiments of the method proposed according to the first and / or second aspect, the time of occurrence of the event corresponds to the time of presentation of the image.

[0030] In one or more embodiments of the method proposed according to the first and / or second aspect, the signaling message is of type SCTE-35. For example, the signaling message may be a command of type “time_signal”.

[0031] In one or more embodiments of the method proposed according to the first and / or second aspect, the traceability data are inserted into a descriptor of the SCTE-35 type message.

[0032] In one or more embodiments of the method proposed according to the first aspect, the proposed method further comprises: generating a plurality of signaling messages respectively associated with images of the image sequence; and, for each signaling message of the plurality of signaling messages: generating traceability data of the signaling message; inserting into the signaling message the traceability data of the signaling message; inserting the signaling message into the multimedia stream, and the frequency of insertion of the signaling messages into the multimedia stream corresponds to the frequency of insertion of reference images by a video encoder when encoding the image sequence.

[0033] According to another aspect, a video stream processing device is provided, which comprises: a processor and a memory operatively coupled to the processor, wherein the processor is configured to implement an embodiment of a method provided in the present disclosure.

[0034] According to another aspect, a computer program, loadable into a memory associated with a processor, and comprising portions of code for implementing one or more embodiments of a proposed method during the execution of said program by the processor, is provided.

[0035] Another aspect relates to a data set representing, for example by compression or encoding, a computer program as proposed.

[0036] According to another aspect, a non-transitory storage medium for a computer-executable program is provided, which comprises a data set representing one or more programs, said one or more programs comprising instructions for, upon execution of said one or more programs by a computer comprising a processing unit operatively coupled to memory means and to an input / output interface module, causing the computer to implement one or more embodiments of a proposed method.

[0037] According to another aspect, there is provided a computer program comprising instructions for implementing all or part of a method as proposed in the present when this program is executed by a processor.

[0038] According to another aspect, there is provided a non-transitory, computer-readable recording medium on which such a program is recorded. Brief description of the drawings

[0039] Other characteristics, details and advantages will appear on reading the following description of non-limiting exemplary embodiments, with reference to the appended drawings, in which: Fig.l

[0040] [Fig.l] shows a diagram of a multimedia content distribution system for implementing the proposed method according to one or more embodiments. Fig. 2

[0041] [Fig.2] shows a diagram illustrating an example of implementation of a method proposed according to one or more embodiments. Fig. 3a

[0042] [Fig.3a] shows an example of a multimedia stream according to one or more embodiments. Fig. 3b

[0043] [Fig.3b] shows an example of distribution of a multimedia stream according to one or more embodiments. Fig. 4a

[0044] [Fig.4a] shows a non-limiting example of implementation of a proposed method of latency measurement of an image of a multimedia stream according to one or more embodiments. Fig. 4b

[0045] [Fig.4b] shows an example of latency measurement architecture in a chain of distribution of multimedia content according to one or more embodiments of the proposed method. Fig. 5a

[0046] [Fig.5a] shows a diagram illustrating an example of architecture of a device for implementing a method proposed according to one or more embodiments. Fig. 5b

[0047] [Fig.5b] shows a diagram illustrating an example of architecture of a device for implementing a method proposed according to one or more embodiments. Description of the embodiments

[0048] In the following detailed description of embodiments of the invention, numerous specific details are presented to provide a more complete understanding. Nevertheless, those skilled in the art may realize that embodiments lization can be put into practice without these specific details. In other cases, well-known features are not described in detail to avoid unnecessarily complicating the description.

[0049] The present description refers to systems, subsystems, functions, engines, units, modules, platforms, and diagrammatic illustrations of the methods and devices according to one or more embodiments. Each of the described systems, subsystems, functions, engines, modules, platforms, units and diagrams may be implemented in hardware, software (including in the form of embedded software ("firmware"), or "middleware"), microcode, or any combination thereof.In the case of a software implementation, the systems, subsystems, functions, engines, units, modules, platforms and / or diagram illustrations may be implemented by computer program instructions or software code, which may be stored or transmitted on a computer-readable medium, including a non-transitory medium, or a medium loaded into the memory of a generic, specific computer, or any other programmable data processing apparatus or device to produce a machine, such that the computer program instructions or software code executed on the computer or the programmable data processing apparatus or device, constitute means of implementing these systems, subsystems, functions, engines, units, modules, platforms and / or diagram illustrations.

[0050] Embodiments of a computer-readable medium include, but are not limited to, computer storage media and communication media, including any medium that facilitates the transfer of a computer program from one location to another. By "computer storage medium(s)" is meant any physical medium that can be accessed by a computer.Examples of computer storage media include, but are not limited to, flash memory disks or components or any other flash memory devices (e.g., USB keys, memory keys, memory sticks, key disks), CD-ROMs or other optical data storage devices, DVDs, magnetic disk data storage devices or other magnetic data storage devices, data memory components, RAM, ROM, EEPROM, memory cards ("smart cards"), SSD ("Solid State Drive") type memories, and any other form of media usable for carrying or storing or memorizing data or data structures that can be read by a computer processor.

[0051] Furthermore, various forms of computer-readable media may transmit or carry instructions to a computer, such as a router, gateway, server, or any data transmission equipment, whether wired (via coaxial cable, fiber optics, telephone wires, DSL cable, or cable Ethernet), wireless (infrared, radio, cellular, microwave), or virtualized transmission equipment (virtual router, virtual gateway, virtual tunnel endpoint, virtual firewall). The instructions may, depending on the embodiments, include code of any computer programming language or computer program element, such as, without limitation, assembly languages, C, C++, Visual Basic, HyperText Markup Language (HTML), Extensible Markup Language (XML), HyperText Transfer Protocol (HTTP), Hypertext Preprocessor (PHP), SQL, MySQL, Java, JavaScript, JavaScript Object Notation (JSON), Python, and bash scripting.

[0052] Furthermore, the terms "notably", "for example", "example", "typically" are used in the present description to designate examples or illustrations of non-limiting embodiments, which do not necessarily correspond to preferred or advantageous embodiments compared to other aspects or possible embodiments.

[0053] By "server" or "platform" is meant in the present description any service point (virtualized or not) or device operating data processing, one or more databases, and / or data communication functions. For example, and in a non-limiting manner, the term "server" or the term "platform" may refer to a physical processor operatively coupled with associated communication, database and data storage functions, or refer to a network, group, set or complex of processors and associated data storage and networking equipment, as well as an operating system and one or more database system(s) and application software supporting the services and functions provided by the server.A computing device may be configured to send and receive signals, via wireless and / or wired transmission network(s), or may be configured for processing and / or storing data or signals, and may therefore operate as a server. Thus, equipment configured to operate as a server may include, by way of non-limiting examples, dedicated rack-mounted servers, desktop computers, laptop computers, service gateways (sometimes referred to as "boxes" or "residential gateways"), multimedia decoders (sometimes referred to as "set-top boxes"), integrated equipment combining various functionalities, such as two or more of the functionalities mentioned above. Servers may vary greatly in their configuration or capabilities, but a server will generally include one or more central processing units and memory.A server may also include one or more mass storage devices, one or more power supplies, one or more wireless and / or wired network interfaces, one or more input / output interfaces, one or more . multiple operating system(s), such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, or equivalent.

[0054] By “OTT distribution” (from the English “Over-The-Top”), we mean any distribution or broadcasting of multimedia content through a network not dedicated to the distribution or broadcasting of multimedia content (for example through a data communication network such as the Internet network).

[0055] In the present description, the terms “fragment” and “segment” are used interchangeably to designate a set of one or more images or, as the case may be, one or more audio samples, which constitutes the elementary building block referenced in OTT formats (such as for example HLS, DASH, or CMAF).

[0056] In this description, the term “multimedia content” means any content of the audio and / or video, audiovisual, subtitle, music, sound, image and interactive graphical interface type, and any combination of these types of content.

[0057] In the present description, the terms "real-time" distribution, "linear mode" distribution, "linear TV mode" distribution, "dynamic mode" distribution and "live" or "live mode" distribution are used interchangeably to designate the dynamic mode or type distribution (in English "live" or "dynamic" mode) of multimedia content in a content distribution system to terminals, including in particular the distribution of the content as it is generated, as opposed to the distribution of previously generated content, upon request for access from a user (distribution upon request for access or "static" mode or type distribution - in English, "static"), such as, for example, content recorded on a server, and made available to users by a video on demand (VOD) service.

[0058] In this description, the terms “live content” refer to content, for example of a multimedia type, distributed, for example according to an OTT distribution mode, in dynamic mode (as opposed to the static distribution mode). Live content will typically be generated by a television channel, or by any type of television media, and may also be broadcast on a multimedia content distribution network, in addition to being made available on content servers in an OTT distribution system.

[0059] In the present description, the terms “terminal”, “user equipment”, “player”, “playback device”, “playback terminal” and “video player” are used interchangeably to designate any type of device, implemented by one or more software programs, one or more hardware programs, or a combination of one or more software programs and one or more hardware programs, configured to use multimedia content distributed according to a multi-screen distribution protocol, in particular by loading and playing the content. The terms “client” and “video playback client” are also used interchangeably to refer to any type of device, software and / or hardware, or any function or set of functions, implemented by software and / or hardware within a device, configured to use multimedia content distributed according to a multi-screen distribution protocol, including loading the content from a server and playing the content.

[0060] The terms "network" and "communication network" as used in the present description refer to one or more data links that can couple or connect equipment, possibly virtualized, so as to allow the transport of electronic data between computer systems and / or modules and / or other electronic devices or equipment, such as between a server and a client device or other types of devices, including between wireless devices coupled or connected by a wireless network, for example. A network may also include a mass memory for storing data, such as a NAS (network attached storage), a SAN (storage area network), or any other form of computer-readable or machine-readable medium, for example.A network may include, in whole or in part, the Internet, one or more local area networks (LANs), one or more wide area networks (WANs), wired connections, wireless connections, cellular connections, or any combination of these. Similarly, subnetworks may use different architectures or be compliant or compatible with different protocols, and interoperate with larger networks. Different types of equipment may be used to make different architectures or protocols interoperable. For example, a router may be used to provide a communications link or data link between two LANs that would otherwise be separate and independent.

[0061] The terms "operably coupled", "coupled", "mounted", "connected" and their various variations and forms used in this description refer to couplings, connections, assemblies, which may be direct or indirect, and include in particular connections between electronic equipment or between portions of such equipment which enable operations and functions as described in this description. In addition, the terms "connected" and "coupled" are not limited to physical or mechanical connections or couplings. For example, an operatively coupled may include one or more wired connections and / or one or more wireless connections between two or more equipment which enable simplex and / or duplex communication links between the equipment or portions of the equipment.In another example, an operational coupling or connection may include a wired and / or wireless coupling to enable data communications between a . server of the proposed system and other system equipment.

[0062] The world of digital television broadcasting has undergone real changes in recent years. In addition to traditional distribution carried out dynamically via pay-TV operators (e.g. digital bouquets and / or Internet Service Providers (ISPs)) for so-called linear television (ADSL (Asymmetric Digital Subscriber Line), Satellite, Cable, IPTV, Fiber, Digital Terrestrial Television (DTT), etc.), new audiovisual services have been added, such as video on demand (VOD) or catch-up TV (Catch-Up TV or Replay).

[0063] The design of new terminals that are ever more powerful and more mobile, the growing demand for the consumption of content independent of the consumer's location (in English "TV everywhere"), as well as the development of network infrastructures offering ever more bandwidth, have led to the emergence of a new market that does not require ownership of the access network to users, and called over-the-top (OTT) services. The OTT market operates on so-called unmanaged networks (i.e. with bandwidth and quality of service (QoS) not guaranteed). These multi-screen OTT services are provided by distribution / redistribution companies that are television content broadcasters and telecom operators.

[0064] The distribution infrastructures are now doubled, one for traditional broadcasting and one (at least) for OTT distribution.

[0065] In order to address multi-screen content distribution services, various protocols have been developed, such as Apple HLS (HTTP Live Streaming), Microsoft Smooth Streaming (MSS), Adobe HTTP Dynamic Streaming (HDS), MPEG Dynamic Adaptive Streaming over HTTP (MPEG-DASH) and MPEG Common Media Application Format (MPEG-CMAF). All these protocols are based on the concept of adaptive streaming using the HTTP (HyperText Transfer Protocol) (HAS, for "HTTP Adaptive Streaming") protocol.

[0066] The distribution of multi-screen content typically uses a video distribution network head comprising a video encoding unit which receives content to be encoded as input and delivers encoded streams called “elementary streams” to a formatting unit (sometimes called a “packager”). Content received as input is encoded according to a plurality of encoding profiles (a profile being for example defined with codec, resolution and bit rate parameters).

[0067] The encoded streams (elementary streams) delivered by the video encoding unit are cut within the formatting unit into at least two sequences of successive segments, respectively audio and video, the segments generally being of fixed duration (typically a few seconds), the format of which depends on the chosen protocol (for example MPEG2-TS, MP4, etc.). For each content, a set of metadata relating to multimedia content, sometimes called a media presentation container (from the English "Media Presentation Description", or MPD) or "manifest" (from the English "manifest"), is also generated by the formatting unit, for example in the form of a file, for example in XML format, indicating the characteristics of each profile and the available segments corresponding to the content.

[0068] These manifests and the associated segments are provided by the formatting unit to a content server, then stored on content distribution networks, called "CDN" (from the English "Content Delivery Network"), providing cache capabilities which make it possible to improve the quality of the services, and to minimize access times and latency for the viewing of content by reading terminals.

[0069] A terminal for playing video content distributed in dynamic mode (for example configured within user equipment such as a smartphone, a tablet or any other type of computer), in other words dynamic or “live” video content (as opposed to static content, such as VOD content for example), will typically be configured to play this content using the information contained in the manifest associated with the content.

[0070] [Fig.l] shows an example of a multimedia content distribution system architecture comprising a packaging subsystem and a video encoder, these two devices providing two non-limiting examples of a multimedia stream processing device according to one or more embodiments.

[0071] [Fig.l] shows a system (100) comprising a subsystem (101) for distributing multimedia content (for example video) configured to distribute multimedia content to one or more user terminals (for example video players (102) as illustrated in the figure) located at the end of the video distribution chain, for example at the request of these video players (102).In one or more embodiments, the subsystem (101) may typically comprise a transcoding subsystem (103) (in English "Live Transcoder") configured to receive as input "Live" multimedia data streams (103a) (in English "Live Channels") and to generate, on the basis of the data of these Live multimedia data streams (103a), encoded source multimedia streams (104a) (sometimes called "elementary streams") having respective quality levels, which are provided as output from the transcoding subsystem (103) to a packaging subsystem (in English "Packager") (104).

[0072] In one or more embodiments, the shaping subsystem (104) may be configured to receive as input the source media streams (104a) and generate OTT (Over The Top) content based on requests received from a video player (102) via a data communication network (105), for example of the content distribution network (CDN) type or of the content delivery network (CDN). The network (105) may be used as an intermediate data transmission network between the video players (102) and the shaping subsystem (104) configured to serve video content requests initiated by the video players (102).

[0073] The transcoding subsystem (103) may be configured for encoding or compressing data received on multimedia data streams in “Live” mode, which typically will not be compressed or will be very weakly compressed, as the data from these multimedia data streams are received, and for generating at output multimedia data streams encoded in “Live” mode, i.e. as the data is encoded. Content received at input will typically be encoded according to a plurality of encoding profiles (a profile being for example defined with codec, resolution and bit rate parameters). The use of a plurality of encoding profiles has at least two advantages: First of all, a plurality of profiles makes it possible to address reading terminals with different characteristics (supported codecs, screen resolution, decoding power).In addition, it offers the video players on these terminals the ability to adapt according to the available network bandwidth. In fact, a client's (terminal's) player can be configured to seek to achieve the encoding profile corresponding to the highest bit rate, based on the bandwidth measurements it constantly performs.

[0074] Depending on the embodiment, the transcoding subsystem (103) may be configured to encode or compress video content according to any video compression standard, such as for example the HEVC (High Efficiency Video Coding), AVI or VVC (Versatile Video Coding) standards or their evolutions, and to encode or compress audio content according to any audio compression standard, such as for example the AAC (Advanced Audio Coding) standards or its evolutions (including AAC+, HE-AAC, eAAC+, HD-AAC), or AC-4 or its evolutions.

[0075] The transcoding subsystem (103) and / or the shaping subsystem (104) may be advantageously configured to implement the proposed method, in order to perform latency measurements in the subsystem (101) between the transcoding subsystem (103) and any subsystem of the multimedia content distribution subsystem (101) located downstream of the transcoding subsystem (103). or the video player (102), or between the formatting subsystem (104) and any media content distribution subsystem (101) located downstream of the formatting subsystem (104), or the video player, respectively.

[0076] As illustrated in [Fig.l], in one or more embodiments, the transcoding subsystem (103) comprises one or more transcoding servers, each server possibly comprising one or more encoding and / or transcoding units. The transcoding servers will typically be configured for encoding content (such as, for example, video content and / or audio content) corresponding to one of the Live multimedia data streams received by the transcoding subsystem (103). In some embodiments, the transcoding servers may be configured for real-time encoding of different types of content (for example, different types of video content) at corresponding transcoding points along a transcoding chain.

[0077] In certain embodiments, the formatting subsystem (104) may be configured to format the encoded streams (elementary streams) delivered by the transcoding subsystem (103) by dividing these streams into two sequences of successive segments, respectively audio and video. In certain embodiments, the segments may be of fixed duration, and their format depend on the chosen multimedia data transport protocol (for example MPEG2-TS, MP4, etc.).

[0078] In certain embodiments, the formatting subsystem may further be configured to generate a set of metadata relating to multimedia content carried by the encoded streams (elementary streams) received from the transcoding subsystem (103).

[0079] In the embodiments using the MPEG-DASH streaming distribution protocol, this metadata will be generated in a file called a manifest or MPD file (Media Presentation Description), for example in XML format, indicating the characteristics of each profile and the available segments corresponding to each multimedia content.

[0080] In embodiments using the HLS streaming distribution protocol, this metadata will be generated in a file called a playlist, also indicating the characteristics of each profile and the available segments corresponding to each multimedia content.

[0081] The formatting subsystem (104) will thus typically be configured to format the received elementary streams by generating sequences of segments and metadata intended to allow a multimedia player having received, for example following one or more content requests, one or more segments, to reconstitute the elementary streams in order to decode them from the received segments.

[0082] The shaping subsystem (104) may thus typically be configured to generate files in a format corresponding to the media content transport protocols used to transmit the content to the video player so that the video player can play it. In an OTT distribution system, the content generated by the formatting subsystem (104) may be generated in a format allowing its distribution to video players according to OTT distribution schemes, such as the HLS protocol, the MSS protocol, the HDS protocol and the MPEG DASH protocol.

[0083] Data communications between the transcoding subsystem (103) and the shaping subsystem (104) may in some embodiments use media stream data transport schemes, such as MPEG-TS or DASH-IF Live Media Ingest Protocol data transmission schemes. A detailed description of the DASH-IF Live Media Ingest Protocol is available at the URL https: / / dashif-documents.azurewebsites.net / Ingest / master / DASH-IF-Ingest.html.

[0084] Data communications between the shaping subsystem subsystem (104) and the network (105) may in some embodiments use low latency data communications schemes, such as CMAF, DASH, or LL HLS (Low Latency HTTP Live Streaming) data communications schemes.

[0085] Thus, the network (105) may in certain embodiments be configured to receive output data from the formatting subsystem (104) according to the format specified for the CMAF or LL HLS scheme.

[0086] [Fig. 2] illustrates a non-limiting example of implementation (10) of a method proposed by the present disclosure according to one or more embodiments. The method of [Fig. 2] describes operations performed by a multimedia stream management unit to obtain a multimedia stream which comprises a signaling message which can be used to determine a latency measurement relating to an image of the multimedia stream (called "current image") during its distribution by a multimedia distribution chain. Thus, in the present disclosure, "current image" refers to an image of a sequence of images of a multimedia stream distributed by a distribution chain to one or more playback devices, for which a latency measurement is performed according to an embodiment of a method proposed by the present disclosure.

[0087] In one or more embodiments, the latency measurement for the current image may be determined based on measurements of the detection times of the signaling message passing respectively at two measurement points chosen on the distribution chain. In one or more embodiments, the signaling message used to perform the latency measurement includes traceability data. of the message which make it possible to determine the progress of the signaling message between the two measurement points by detecting the passage of the message at each measurement point.

[0088] With reference to [Fig.2], we consider a multimedia stream (for example video) comprising a sequence of images among which is an image called a “current image”. The multimedia stream can for example be a stream at the input of a video compressor, an encoder or a video codec, or at the output of a video compressor, an encoder or a video codec.

[0089] In one or more embodiments, signaling message traceability data is generated (11). This traceability data can advantageously be used to track the progress of the signaling message in the distribution chain, by detecting this signaling message at several points in the distribution chain. Depending on the embodiment, this traceability data may include message identification data (such as for example a message identifier), or more generally any data that can be used to detect the signaling message at different points in the distribution chain.

[0090] In one or more embodiments, when the signaling message traceability data has been generated, it is inserted (12) into a signaling message which is obtained by the multimedia flow management unit. In one or more embodiments, the insertion of the traceability data into the signaling message advantageously allows the detection of the passage of the signaling message at different points in the distribution chain.

[0091] In one or more embodiments, a correspondence between the signaling message and the current image is used to perform the latency measurement relative to the current image. This correspondence makes it possible to follow the progress of the current image along the distribution chain by means of tracking along the chain of the signaling message using the message traceability data which have been inserted into the signaling message. Depending on the embodiment, this correspondence can take the form of correspondence information carried by the signaling message, or be implicitly determined from the signaling message. In one or more embodiments, the correspondence link existing between the signaling message and the current image in the multimedia stream can evolve during the distribution of the message and the image in the distribution chain.Different embodiments of the proposed latency measurement method advantageously use the fact that this correspondence is maintained at different points in the distribution chain even though it can take different forms and be determined in different ways depending on the point in the chain considered.

[0092] For example, in one or more embodiments, when inserted into the multimedia stream being processed at certain points in the distribution chain (typically points upstream of a formatting unit), the signaling message may include reference information to the current image of the multimedia stream. The correspondence between the signaling message and the current image can thus be determined on the basis of the reference information.

[0093] In one or more embodiments, the signaling message may comprise reference data to the current image. For example, depending on the embodiment, the signaling message may comprise at least one of pointing data on the current image (such as for example a pointer on the image or a marker which directly or indirectly references the image), identification data of the current image, and / or any data carried by the signaling message which can be used to determine the correspondence between the signaling message and the current image.

[0094] In one or more embodiments, pointing data on the current image comprises data relating to an event associated with the current image, such as for example an end of program and / or a start of advertising. In these embodiments, the signaling message may comprise time stamp information corresponding to a time of occurrence of the event.

[0095] For example, in one or more embodiments, the signaling message may comprise time position information in the image sequence of the current image. Depending on the embodiment, the time position information may be carried, for example, by one or more of timestamp data of the current image, presentation time data of the current image, and position data of the current image in the image sequence (such as, for example, a sequence number).

[0096] The signaling message comprising the traceability data can then be inserted (13) into the multimedia stream in order to obtain a multimedia stream comprising a signaling message whose progress in the distribution chain which distributes the multimedia stream can be followed and for which a correspondence with the current image exists, possibly of different types (implicit or explicit) and in different forms depending on the point of the distribution chain considered. This signaling message can be advantageously used to determine a latency measurement in the distribution chain for this current image according to an embodiment of a proposed latency measurement method.

[0097] In one or more embodiments, the signaling message may be obtained by the media stream management unit by generating this message, in any format suitable for insertion into the media stream and transport of the generated signaling message in the distribution chain via the media stream. including the signaling message.

[0098] In one or more embodiments, the traceability data may be inserted into the signaling message when generating this message. Thus, in one or more embodiments, the multimedia stream management unit may be configured to generate signaling message traceability data and generate a signaling message in which the traceability data will have been inserted.

[0099] In other embodiments corresponding to cases in which the multimedia stream management unit operates on a multimedia stream which already comprises at least one signaling message which corresponds to an image of the stream, the signaling message can be obtained by the multimedia stream management unit by extracting the multimedia stream. In one or more embodiments the signaling message can correspond to an image of the stream in that it comprises for example information on the temporal position in the sequence of images of the image.

[0100] In one or more embodiments, once extracted from the media stream, the signaling message may be modified by inserting the signaling message traceability data therein. Thus, in one or more embodiments, the media stream management unit may be configured to generate signaling message traceability data, extract a signaling message from the media stream on which the unit is operating, and insert the traceability data into the signaling message extracted from the media stream.

[0101] In one or more embodiments, the signaling message is already included in the multimedia stream processed by the multimedia stream management unit, and the proposed method may further comprise: extracting the signaling message from the multimedia stream.

[0102] In one or more embodiments, the media stream management unit may be configured to determine that a signaling message included in the media stream on which the unit operates corresponds to an image of the stream and already includes data that can be used to trace that signaling message at multiple points in the distribution chain during its transport through the distribution chain.

[0103] In one or more embodiments, the media stream management unit may be configured to determine that a signaling message included in the media stream on which the unit is operating corresponds to an image of the stream. In some embodiments, the stream management unit may be configured to, without processing the extracted signaling message to determine whether it includes data that can be used to trace that signaling message at multiple points in the distribution chain during its transport in the distribution chain distribution, insert into this message traceability data which will have been obtained (for example generated) beforehand.

[0104] In one or more embodiments, the message traceability data may be inserted into a field of the signaling message, for example a field available for inserting proprietary information.

[0105] In one or more embodiments, the signaling message traceability data comprises identification data usable to identify the signaling message. For example, the signaling message traceability data comprises an identifier of the signaling message, or an identifier of a frame of the media stream usable to identify the signaling message.

[0106] In one or more embodiments, the signaling message traceability data may be inserted or already be included in a signaling message of any type and format suitable for transporting, within the various elements of the distribution chain, the signaling message in connection with the distribution of the multimedia stream in the distribution chain.

[0107] Depending on the embodiment, the signaling message can thus be modified or generated by inserting the message traceability data therein in order to be able to follow the progress of the signaling message in the distribution chain within the multimedia stream distributed in the distribution chain.

[0108] For example, in one or more embodiments, the signaling message may be a message of the SCTE (Society of Cable Telecommunications Engineers)-35 type, i.e. conforming to the specifications of the SCTE-35 standard, described in the document "Digital Program Insertion Cueing Message for Cable", in a past version (such as for example the "rl" version dating from 2019), current (version "2022a" dating from March 2022), or future.

[0109] The SCTE 35 standard defines signaling mechanisms relating to a multimedia stream to signal broadcast events, such as programs, chapters, pauses or advertisements in frames. In particular, it defines different types of messages (called "commands"), including the following command types: "splice_null", "splice_schedule", "splice_insert", "time_signal", "bandwidth_reservation", and "private_command". Broadcast events such as programs, chapters, pauses or advertisements in frames are defined in the "time_signal" type commands using one or more "segmentation_descriptors".

[0110] Depending on the scenario, once inserted into a multimedia stream upstream of a formatting unit, the SCTE-35 signaling messages can be transported in the multimedia stream by being inserted by the formatting unit into a playlist according to the HLS format (in English, “media playlist HLS”) or into a set of data called a DASH manifest as defined in the HLS and DASH standards, respectively. While SCTE-35 signaling messages are not different depending on whether they are inserted into a DASH manifest or an HLS playlist, the way to insert them differs depending on whether they are inserted into an HLS playlist or a DASH manifest: XML-based for DASH, and in tag form for HLS. At the output of the formatting unit, signaling data (such as SCTE-35 signaling messages) can also (besides their presence in a playlist or a manifest) be present in segments or fragments, for example by being transported in event signaling messages called "box emsg" for the mp4 format or in the MPEG-TS transport stream for the HLS TS streaming format.

[0111] SCTE-35 signaling data enables, among other things, the dynamic insertion of advertisements into a distributed (e.g., live) media stream, using data (in messages that are sometimes called "markers") that carry metadata. In this description, the terms "message," "signaling marker," and "marker" are used interchangeably to refer to any set of signaling data, such as, for example, any set of SCTE-35 type signaling data.

[0112] Different modes of insertion of SCTE-35 markers carrying signaling data are defined depending on the multimedia stream distribution scheme used. For example, the mode of insertion of an SCTE-35 marker carrying signaling data may vary depending on whether the multimedia stream distribution scheme used is a DASH (or MPEG-DASH) scheme or an HLS scheme. For example, an SCTE-35 marker may be inserted into the stream in the form of specific tags for the HLS scheme (for example the tag "#EXT-X-DATERANGE"). Another example, an SCTE-35 marker could be inserted into the stream using tags in XML format and encoded in base64 for the DASH schema, as described in the document DASH-IF IOP-5 V5.0.0 (2021-11) "DASH-IF Interoperability Points; Part 5: Ad Insertion in DASH", available at the URL "https: / / dashif.Org / docs / IGP-Guidelines / DASH-IF-IOP-Part5-v5.0.0.pdf", and as illustrated in particular by the example in Table 3 on page 24 of this document. Another example, an SCTE-35 marker could be inserted into the stream as an "emsg box" in the media segments, as described for example in the DASH specification, for example in section G.9 of the DASH version - ISO / IEC 230009-1 2022 ed. 5.

[0113] Thus, for example, in one or more embodiments, the multimedia stream management unit may use a pre-existing SCTE-35 type marker for inserting advertising into the distributed multimedia stream, and insert data into this marker marker traceability, possibly only after determining that the ad insertion marker does not already contain data that can be used to trace the marker at different points in the distribution chain. Such an ad insertion marker may then advantageously be used to determine a latency measurement relative to an image in the stream to which it corresponds (for example, an image located in the stream after or before the start of ad insertion).

[0114] The use of a SCTE-35 type marker is advantageous in that these markers are easily detectable among the data relating to a multimedia stream to which one or more processes (encoding, formatting, storage, etc.) are applied with a view to its distribution, according to one or more formats, to one or more terminals.

[0115] Furthermore, the data carried by a SCTE-35 type marker does not vary during its distribution in the distribution chain in connection with the distribution of the multimedia stream in the distribution chain, unlike subtitling data which can vary from Teletext format to TTML format for example, or video data which can be transcoded by an element of the distribution chain, to go from H.264 format to H.265 format for example. A SCTE-35 type marker can thus be easily detected at different points in a distribution chain during its distribution in the chain in connection with a multimedia stream.The proposed method takes advantage of this ease of detection to trace an image of the stream and follow this image during its multimedia distribution by means of a signaling message corresponding to this image (such as for example an SCTE-35 marker), which is traced using the traceability data that it contains, so as to carry out one or more latency measurements using the marker (and therefore the image which corresponds to it) at one or more points of the distribution chain.

[0116] In the following, exemplary embodiments of the present disclosure in which SCTE-35 type markers are inserted into a multimedia stream upstream of a shaping unit of a distribution chain before being inserted by the shaping unit into a playlist (for example of a type specified in the HLS standard) or into a manifest file (for example of a type specified in the DASH standard), are described. Those skilled in the art will understand that the use of SCTE-35 type markers is only a non-limiting example, and that other signaling messages, type of signaling messages, could be used in other embodiments.Additionally, the use of SCTE-35 type signaling inserted into playlists or manifests is only a non-limiting example, and other embodiments could use SCTE-35 type signaling messages inserted into media data units, such as fragments, segments, and / or chunks of media data. Likewise, other network or distribution chain architectures could be used in other embodiments of the present disclosure.

[0117] SCTE-35 type markers are an example of markers that can be used to carry particular information, such as information relating to an image of a multimedia stream (for example by means of an occurrence event linked to the position of an image in the stream) and traceability data, so that they can advantageously provide a transport vector for this information. The implementation of the proposed method can thus be advantageously carried out without any modification of the SCTE-35 specification, while taking advantage of the processing of SCTE-35 markers as specified in this specification.

[0118] SCTE-35 signaling data may be inserted into media streams, as illustrated by [Fig.3a] which shows an example of a media stream (20) comprising video data (images (or video frames)) (“V”) (21a, 21b, 21c, 21d, 21e, 21f, 21g), audio data (“A”) (22a, 22b), and SCTE-35 signaling data (“SCTE-35”) (23). The media stream (20) comprises a succession of data packets, each comprising video data, audio data and / or signaling data (which may be considered as metadata of the video and / or audio data).

[0119] Depending on the embodiment, the example of multimedia stream (20) to be processed illustrated by [Fig. 3a] may for example be in MPEG2-TS format or in MPEG-DASH format. As indicated in [Fig. 3a], the multimedia stream may comprise metadata comprising, for each video element (image “V”) and each audio element (audio data “A”), a presentation time parameter (for example noted PTS, from the English “Presentation Time Stamp”) relative to a reference clock and a display duration. In certain embodiments, the presentation time of the image (PTS) may be determined from a reference clock of the stream (PCR parameter) during demultiplexing. For example, image (21f) is associated with PTS presentation time "250", while image (21g) is associated with PTS presentation time "260", which indicates (e.g. to a media player) that image (21g) is to be displayed 10 time units after image (21f).

[0120] The multimedia stream (20) of [Fig.3a] further comprises SCTE-35 signaling data (SCTE-35 type marker (23)), inserted in the illustrated example between two images (21b and 21c). In the illustrated example, the SCTE-35 type marker (23) is an advertisement start marker which points to the image of the stream (21f) associated with the PTS presentation time “250” via an “ad_pts_time” parameter to which the PTS presentation time value “252” is associated. The parameter "ad_pts_time" allows to indicate a timestamp information, in the form of a time point expressed in terms of presentation time (PTS), which corresponds to the image (21f) of corresponding PTS. The signaling of the SCTE-35 type marker (23) is positioned in the stream before the image (21f) of the stream to which it relates to indicate to the elements of the distribution chain that the corresponding image (21f) is linked to an action or an event signaled by the SCTE-35 type marker (23). Thus, it can be determined that the SCTE-35 type marker corresponds to the image (21f) via the parameter "ad_pts_time" which designates a PTS time "252" which corresponds to that of the image (21f). This correspondence also allows to determine a time difference between the SCTE-35 type marker (23) and the corresponding image (21f).

[0121] [Fig.3b] shows an example of distribution of a multimedia stream in an OTT multimedia stream distribution system such as, for example, the subsystem 101 for distributing multimedia content (for example video) illustrated by [Fig.l]. [Fig.3b] illustrates by an example the mechanisms for processing SCTE-35 type signaling data when distributing a multimedia stream in a multimedia data distribution system.

[0122] The multimedia data distribution system illustrated by [Fig.3b] comprises a chain for distributing multimedia streams from a source (103a) via a multimedia encoder (103), a formatting unit (104), and a multimedia content distribution network (105), for distributing multimedia data to one or more terminals (102). The equipment (103a), (103), (104), (105) and (102) may for example correspond to those of [Fig.l] and described in connection with this figure.

[0123] Also shown in [Fig.3b] is the multimedia stream (20) of [Fig.3a] and described in connection with this figure. The multimedia stream may for example be generated by a multimedia data source (103a), implemented for example by a headend unit. This multimedia stream comprises a marker (23), in the illustrated example of the SCTE-35 type, as well as data packets carrying video and / or audio data, as described in connection with [Fig.3a].

[0124] As illustrated in [Fig.3b], the multimedia stream (20) may, as part of its provision for distribution to terminals (102), be provided as input to a multimedia encoder (103) configured to generate, from this input data, an encoded multimedia stream (20a). The processing time within the multimedia encoder induces a latency which leads to modifying the presentation time (PTS) to add, for each element of the input stream (20) associated with a presentation time, to the initial presentation time of the input stream (20) a processing time, for example equal to 20 time units, as illustrated in [Fig.3b].

[0125] The marker (23) is also modified to take into account the modification of the presentation time of the image (21f) to which it points (from “PTS 250” to “PTS 450”): the parameter “ad_pts_time”, initially configured at the value “252”, is modified to have the value “452” in the stream (20a) at the output of the video encoder (104).

[0126] As illustrated in [Fig. 3b], the encoded stream (20a) may, in one or more embodiments, be provided as input to a formatting unit (105), such as that described in connection with [Fig. 1]. In one or more embodiments, the formatting unit (105) may be configured to generate, from the data of the encoded stream (20b), one or more video segments (in the illustrated example, the “segment N” and the “segment N+1”), as well as one or more metadata files describing the contents of the received streams (in the illustrated example, the file (24a) which may for example be a manifest file (for example in XML format), for example of the DASH type or be a playlist file, for example of the HLS type.

[0127] In one or more embodiments, the formatting unit (105) may be configured to detect the marker (23) in the encoded stream (20b), and insert the metadata of the marker (23) into a metadata file (24a) generated to describe the contents of the encoded stream (20b). The insertion of the marker (23) into a metadata file (24a) may conform to the format rules of the metadata file (24a), such that, for example, this insertion may be different when the marker (23) is inserted into a DASH manifest and when the marker (23) is inserted into an HLS playlist.

[0128] In the embodiments in which the SCTE-35 type marker is integrated by the formatting unit (104) into an HLS type playlist, the correspondence between the data of the marker inserted into the playlist and the image (21f_b) is implicit: in this case, this image (2l_f) is the first image of the first segment which follows the marker in the playlist (in the figure, the “segment N+l”). A flow management unit which processes the playlist to detect the marker can thus know that the image to which the marker relates is the first image of the segment listed immediately after the marker in the playlist. It can also be considered in this case that the time difference between the detection of the marker in the list and the image corresponding to the marker is zero or almost zero.

[0129] These video segments ("segment N" and "segment N+1") and one or more metadata files (24a) are then provided to a content delivery network node (105) (e.g., a data server) configured to store them for distribution upon request to one or more terminals (102).

[0130] The SCTE-35 2022b specification (and corresponding standards) defines mechanisms for splicing encoded media streams that use a splice event to indicate an opportunity to splice one or more streams. Splice event identifiers ("splice_event_id") are also defined to uniquely identify a splice event. The SCTE-35 2022b specification further defines signaling messages to signal a signaling event to each device processing the media stream. For example, control messages of the type "time_signal" can be used to signal a splice event. For more details, please refer to the SCTE-35 2022b specification, in particular to Section 9 of this specification.

[0131] Section 10 of the SCTE-35 2022b specification defines parameters "splice_info_section" messages of type "time_signal" that include splice descriptors, including the specific splice descriptors "avail_descriptor()", "DTMF_descriptor()", "segmentation_descriptor()", "time_descriptor()", and "audio_descriptor()".

[0132] In particular, the syntax defined for the segmentation descriptor "segmentation_descriptor()" provides, among others, the following data fields: "segmentation_upid_type", "segmentation_upid_length", "segmentation_upid()" and "segmentation_type_id". For more details on connection descriptors, please refer to the SCTE-35 2022b specification, in particular Section 10 of this specification.

[0133] In one or more embodiments, the signaling message traceability data may be carried by a signaling marker, for example of SCTE-35 type, for example of type "thne_signal" (as specified in section 9.7.4 of the SCTE-35 2022b specification), as illustrated by Figs. 3a and 3b, associated with a "splice_descriptor()" parameter, corresponding to an image of the multimedia stream (such as for example via image presentation time data carried by the marker).

[0134] As defined by the SCTE-35 2022b specification, the "time_signal()" parameter allows synchronization information (for example of type "pts_time") to be associated with a descriptor of type "splice_info_section". The XML schema of the "splice_signal()" parameter is described in Figure 9 of the SCTE-35 2022b specification. The SCTE-35 type signaling message may be of type specified under the name "splice_info_section", and contain a command, which may for example be of type "time_signal". The "time_signal" type command may typically be used to carry timestamp information. Thus, an SCTE-35 signaling message may be configured to signal a single occurrence event in time, and contain a "thne_signal" type command to indicate, among other things, a timestamp of the event. For example, an event signaled by means of of an SCTE-35 message can be the end of a program and the start of an advertisement. One or more descriptors can be carried by the "time_signal" command to describe a timestamp of this type of event. For example, one or more descriptors can be used to describe (respectively, in the case of multiple descriptors) an end time point of the current program and / or an advertisement start time point.

[0135] Thus, in one or more embodiments, a command of type “time_signal” in a marker of type SCTE-35, associated with one or more descriptors, for example of type “segmentation_descriptor”, at least one of these descriptors being configured with time stamp information of an image of the multimedia stream, can be used as a signaling message.

[0136] The “time_signal” command type as described in the SCTE-35 2022b specification is particularly well suited for use in an embodiment of a method proposed in the present disclosure. Indeed, the “time_signal” type commands as specified carry timestamp information, which makes it possible to identify an image, which can be chosen as the current image to perform a latency measurement. In addition, the “time_signal” type commands as specified are associated with a list of generic “splice_descriptor” type descriptors. If the descriptors typically used in connection with a “time_signal” type command are “segmentation_descriptor” type descriptors to signal for example chapters, programs (respective beginnings and ends of programs), breaks, etc., each descriptor of type "segmentation_descriptor" contains a field called "segmentation_upid()" which can be used to carry additional information, such as information useful for a latency measurement.

[0137] In addition, “thne_signal” type commands are widely used in deployed systems to signal advertising to be inserted into a multimedia stream, such that the elements of the deployed distribution chains are generally configured to pass them along the chain. Thus, the methods proposed in the present disclosure are particularly well suited to implementations using SCTE-35 type signaling messages comprising “thne_signal” type commands. In one or more embodiments, the signaling message is of the SCTE-35 type.SCTE-35 type signaling messages may be advantageously used in certain embodiments because, like SCTE-35 type signaling messages used to signal advertisement insertions, SCTE-35 type signaling messages carrying data useful for a latency measurement relating to a multimedia stream distributed by a distribution chain, will typically not be filtered by an element of the distribution chain.

[0138] In one or more embodiments wherein the signaling message is of type SCTE-35, an SCTE-35 segmentation descriptor (of type "segmentation_descriptor") as specified can be used to insert into the message data useful for latency measurement, such as message traceability data.

[0139] Alternatively, an existing descriptor can be advantageously reused by adding data useful for latency measurement, such as message traceability data, in the “segmentation_upid” type field.

[0140] In one or more embodiments, a SCTE-35 segmentation descriptor ("segmentation_descriptor") may be used to transport data useful for a latency measurement relating to an image of a multimedia stream distributed by a multimedia distribution chain, such as for example traceability data of the SCTE-35 signaling message.

[0141] For example, the SCTE-35 segmentation descriptor data may include the following data fields:

[0142] A “segmentation_upid_type” field with one of the 8-bit values ​​defined in Table 22 of the SCTE-35 2022b specification. In one or more embodiments, the hexadecimal value 0x01 may advantageously be used to indicate that the “segmentation_upid_type” field is used to carry user data.

[0143] A “segmentation_upid()” field, the type and size (in bytes) of which are specified by means of the “segmentation_upid_type” field. For example, for a “segmentation_upid_type” field having the value 0x01, the type is “User Defined”, and the size in bytes is variable. In one or more embodiments, the seg-mentation_upid() field can advantageously carry traceability data of an image of the multimedia stream comprising an identifier (for example in the form of a universally unique identifier “UUID” (from the English “Universally Unique Identifier”) indicated by a “segmentation_upid_type” field of value “0x10”) and timestamp information (“timecode”)

[0144] A “segmentation_type_id” field with one of the 8-bit values ​​defined in Table 23 of the SCTE-35 2022b specification. Depending on the embodiment, the hexadecimal value 0x01 may advantageously be used to indicate that the segmentation descriptor (message) describes a content identifier “Content Identification”, or the hexadecimal value 0x22 may advantageously be used to indicate that the segmentation descriptor (message) describes a start of interruption (signaling for example a start of advertising) “Break Start”.

[0145] The signaling message traceability data can thus be carried, in one or more embodiments, by a marker of type SCTE-35 2022b which corresponds to an image of the flow, for example of type “segmentation_upid()”. Advantageously, these markers can be captured at different points in a multimedia content distribution chain, such as that illustrated by [Fig.l] or by [Fig.3b].

[0146] In one or more embodiments, a SCTE-35 command of type "private_command()" or "splice_insert()" can be used to transport data useful for a latency measurement relating to an image of a multimedia stream distributed by a multimedia distribution chain, such as for example traceability data of the SCTE-35 signaling message. The use of a command of type "splice_insert()" has the advantage of being widely used in the multimedia distribution systems currently deployed. However, unlike SCTE-35 type commands, in the current state of the SCTE-35 specification, a command of type "splice_insert()" does not include a field in which data for a specific use can be inserted. A command of type "private_command()" may in certain systems not be transmitted by certain equipment.

[0147] [Fig.4a] illustrates a non-limiting example of implementation (1000) of a proposed method for measuring the latency of an image of a multimedia stream according to one or more embodiments.

[0148] We again consider a multimedia stream distributed by a multimedia distribution chain in a multimedia distribution system, the multimedia stream comprising a sequence of images. The proposed method of measuring latency can for example be implemented by a processing unit of a multimedia distribution system.

[0149] We also define two points in the distribution chain that we wish to use to perform a latency measurement relative to an image of the flow between these two points.

[0150] In one or more embodiments, the passage of a signaling message corresponding to an image of the sequence of images and comprising traceability data in the multimedia distribution chain, at a first point of the multimedia distribution chain, is detected. Upon detection of the message at the first point, first timestamp data for detecting the signaling message at the first point of the multimedia distribution chain are generated (1001) and recorded. The first timestamp data make it possible to record a passage time at a first point of the distribution chain of a traceable signaling message in the distribution chain and corresponding to an image of the flow.

[0151] In one or more embodiments, passing the signaling message corresponding to the image of the image sequence and comprising the data of traceability in the multimedia distribution chain, at a second point in the multimedia distribution chain, is detected. Upon detection of the message at the second point, following the determination that the signaling message includes the traceability data, second detection timestamp data of the signaling message at the second point in the multimedia distribution chain are generated (1002) and recorded.

[0152] A latency measurement associated with the image may then be determined (1003) based on the first and second timestamp data, for example by determining a time difference between the second timestamp data and the first timestamp data.

[0153] In one or more embodiments, the latency measurement may further be determined by taking into account at least one of a first time difference between the first detection timestamp data of the signaling message at the first point and a time of making the image available at the first point, and a second time difference between the second detection timestamp data of the signaling message at the second point and a time of making the image available at the second point.

[0154] Thus, in one or more embodiments, one or more of a first time difference between the first detection timestamp data of the signaling message at the first point and a time of making the image available at the first point, and a second time difference between the second detection timestamp data of the signaling message at the second point and a time of making the image available at the second point, can be determined.

[0155] In one or more embodiments, the latency measurement may then be determined based on a combination of the first timestamp data, the second timestamp data, and the first and second time deviations.

[0156] In one or more embodiments, one or more of the first and second time differences may be determined based on respective times of provision of the image to which the signaling message points at the corresponding measurement point along the distribution chain, such that in certain embodiments, the measurement obtained may be considered as being a measurement of latency between two times of provision of an image of a multimedia stream for distribution to terminals along a distribution chain.

[0157] [Fig.4b] illustrates an example of latency measurement architecture in a multimedia content distribution chain according to one or more embodiments of the proposed method.

[0158] [Fig.4b] shows a distribution chain (110) of multimedia streams of the type illustrated by [Fig.l] and 3b, and described in connection with these figures. The distribution chain A multimedia stream delivery system (110) may typically comprise a multimedia encoder (e.g., comprising a video encoder) (103), a packaging unit (104), one or more CDN network nodes (105), for distributing multimedia streams (103a) generated by a source to one or more user devices (102).

[0159] In the example architecture illustrated by [Fig.4b], the processing method proposed according to the first aspect is implemented by a multimedia stream management unit (106) placed upstream of the distribution chain (101). In one or more embodiments, the management unit (106) is configured to generate signaling message traceability data, to obtain (for example generate) a signaling message comprising a marker of an image of the sequence of images of the multimedia stream and modify this signaling message by inserting the signaling message traceability data into the signaling message, then insert the modified signaling message into the multimedia stream.

[0160] In the embodiments in which the signaling message comprising a marker of an image of the sequence of images is of the SCTE-35 type, the management unit (106) can be configured to generate signaling message traceability data, and modify the SCTE-35 message by inserting the generated traceability data therein, then by inserting the modified SCTE-35 message into the multimedia stream presented at the input of the distribution chain (110).

[0161] In one or more embodiments applicable in cases in which the multimedia stream received by the management unit already comprises a signaling message (which may be referred to by the term "marker") comprising a marker of an image of the sequence of images (for example of the SCTE-35 type), the management unit (106) may be configured to extract this signaling message from the multimedia stream, and insert signaling message traceability data therein.

[0162] In one or more embodiments applicable in cases in which the multimedia stream received by the management unit does not already include a signaling message comprising a marker of an image of the sequence of images (for example of SCTE-35 type) in which signaling message traceability data advantageously usable for tracing the signaling message during its progression in the distribution chain (110) can be inserted, the management unit (106) can be configured to generate a new signaling message comprising a marker of an image of the sequence of images (for example of SCTE-35 type), and insert therein signaling message traceability data (such as for example a unique message identifier or a pointer to an image of the stream) advantageously usable for tracing the signaling message during its progression in the distribution chain (110). signaling during its progression through the distribution chain (110).

[0163] As illustrated in [Fig.4b], the multimedia stream (103b) traveling through the different nodes of the distribution chain (110), being placed downstream of the flow management unit (106), contains a signaling message which carries signaling message traceability data. This signaling message traceability data can be read at different measurement points (M1, M2, M3, M4) along the distribution chain (110) following detection of the signaling message which carries it using this signaling message traceability data. In the example illustrated by [Fig.4b], the signaling message is for example of the SCTE-35 type.

[0164] Those skilled in the art will understand that the architecture illustrated by [Fig.4b] is only a non-limiting example, and that in addition the management unit (106) could in other architectures be positioned at other points of the distribution chain (110), in particular to take into account the architecture of the distribution chain, of which Figs. 1, 3b, and 4b provide only a non-limiting example. Other nodes, types of nodes, and network architectures may be envisaged within the framework of embodiments of the present disclosure. For example, the management unit (106) may in certain embodiments be positioned downstream of the encoder (103) (for example between the encoder and the shaping unit (104)), or downstream of the shaping unit (104) (for example between the shaping unit (104) and the CDN network (105)).

[0165] In the example architecture illustrated by [Fig.4b], the processing method proposed according to the second aspect is implemented by a multimedia flow management unit (107) configured to trace SCTE-35 messages based on signaling message traceability data included in these messages at different points in the distribution chain (110).The management unit (107) may for example be configured to, upon detection, at a first point (M1, M2 or M3) of the multimedia distribution chain (110), of an SCTE-35 signaling message corresponding to an image of the sequence of images and comprising traceability data, generate first detection timestamp data of the signaling message at the first point (M1, M2 or M3) of the multimedia distribution chain (110), and following detection, in the multimedia stream, of the signaling message at a second point (M2, M3 or M4) of the multimedia distribution chain (110) by determining that the signaling message comprises the traceability data, generate second detection timestamp data of the signaling message at the second point (M2, M3 or M4) of the multimedia distribution chain, then determine a latency measurement associated with the image on the basis of the first and second timestamp data.

[0166] In one or more embodiments, the unit (107) may be configured to further: determine a first time difference between the first detection timestamp data of the signaling message at the first point (M1, M2 or M3) and a time of making the image available at the first point; and determine a second time difference between the second detection timestamp data of the signaling message at the second point (M2, M3 or M4) and a time of making the image available at the second point, the latency measurement then being determined on the basis of a combination of the first timestamp data, the second timestamp data and the first and second time differences.

[0167] As illustrated in [Fig.4b], the processing unit (107) can be configured to detect a signaling message of type SCTE-35 at the measurement points M1, M2, M3 and / or M4, and obtain from the data carried by this message, possibly in its version or in its format modified after processing by the encoder (103), the formatting unit (104), and the CDN network (105).

[0168] In one or more embodiments, the signaling message will be detected in different forms or formats depending on the measurement point considered along the distribution chain, which may be taken into account to calculate the time differences between the detection timestamp data of the signaling message at a point and the times for making the image available at the corresponding point.

[0169] For example, if the measurement point is located upstream of the shaping unit (104) (which is the case for example of points M1 and M2), the signaling message (in the example of [Fig.4b] of SCTE-35 type) will be integrated into the multimedia stream (as illustrated in Figures 3a and 3b). The time difference between the detection time of the signaling message and the passage of the image at the detection point may in this case for example be determined as a function of the respective positions in the multimedia stream of the signaling message and of the image to which the message points, for example on the basis of presentation time data carried by the message and by metadata of the image.

[0170] If the measurement point is located downstream of the shaping unit (104) (which is the case for example of points M3 and M4), the signaling message (in the example of [Fig.4b] of SCTE-35 type) will be integrated into a set of metadata (for example, depending on the distribution protocol used, of DASH manifest type or HLS playlist). The time difference between the detection time of the signaling message (in the set of metadata) and the passage of the image at the detection point may in this case be considered negligible. Thus, at a point located downstream of the shaping unit (104), it may be considered that the signaling message is presented (i.e. made available for distribution to a video player (102)) (almost) at the same time as the image at which it is reports. In certain embodiments, this time difference may be considered to be zero.

[0171] In one or more embodiments, the processing unit (107) may be configured to detect, at a first measurement point in the distribution chain (110), a signaling message (for example of the SCTE-35 type), then extract signaling message traceability data from the message, for example by reading the data carried by the message. As illustrated by FIGS. 3a and 3b, the signaling marker may for example point to an image to carry out the marking (for example by means of an image presentation time).Upon reading the image marking data, the processing unit (107) can determine a first time difference between the first detection timestamp data of the signaling message at the first point and a time of making the image available at the first point, and store in memory first image timestamp data (comprising the first detection timestamp data of the signaling message and this first time difference) to record a first image timestamp corresponding to the first detection point of the signaling message.These operations can be repeated for a second measurement point, so as to determine a second time difference between second timestamp data for detecting the signaling message at the second point and a time for making the image available at the second point, and store in memory second timestamp data of the image (comprising the second timestamp data for detecting the signaling message and this second time difference) to record a second timestamp of the image corresponding to the second detection point of the signaling message, to then determine a latency measurement corresponding to the first and second measurement points using the first and second timestamp data of the image (for example by comparing the first and second timestamps obtained for the image).

[0172] In one or more embodiments, it will be possible to provide time synchronization of one or more of the elements of the distribution chain (110) downstream of which it is desired to position a measurement point, with the processing unit (107). This time synchronization may for example be carried out using any suitable network synchronization protocol, such as for example the NTP protocol (from the English "Network Time Protocol") described in the 1TETF RFC-1305 specification for version 3 (or in one of the previous versions), or depending on the embodiment the PTP protocol (from the English "Precision Time Protocol") described in the IEEE IEEE-1588 specification for version 3 (or in one of the previous versions).

[0173] Those skilled in the art will understand that the architecture illustrated by [Fig.4b] is not that a non-limiting example, and that the processing unit (107) could in other architectures be implemented on one or more computing devices configured to operate with a common time synchronization base.

[0174] In one or more embodiments in which the multimedia stream already contains a signaling message comprising a marker of an image of the stream, the proposed method may provide for detecting this signaling message, and determining traceability data, such as for example an identifier, in this message. For example, in the case of an SCTE-35 message, the proposed method may provide for detecting an SCTE-35 message present in the multimedia stream and pointing to an image of the stream, and determining in a descriptor of this message an identifier likely to be used to trace this message during its transport in the distribution chain.

[0175] In one or more embodiments in which the multimedia stream already contains a signaling message comprising a marker of an image of the stream, the proposed method may provide for detecting this signaling message, and determining whether traceability data, such as for example an identifier, are present in this message. For example, in the case of an SCTE-35 message, the proposed method may provide for detecting an SCTE-35 message present in the multimedia stream and pointing to an image of the stream, and determining whether an identifier likely to be used to trace this message during its transport in the distribution chain is present in a descriptor of this message.

[0176] In cases where it is determined that the detected signaling message contains traceability data that can be used to trace this message during its transport in the distribution chain, the proposed method may provide for storing this traceability data in memory in order to use it to detect the passage of the signaling message at one or more measurement points along the distribution chain.

[0177] In cases where it is determined that the detected signaling message does not contain traceability data that can be used to trace this message during its transport in the distribution chain, the proposed method may provide for generating traceability data that can be used to trace this message during its transport in the distribution chain, and inserting them into the signaling message and then reinserting the modified message into the flow, having stored the traceability data in memory in order to use them to detect the passage of the signaling message at one or more measurement points along the distribution chain.

[0178] For example, in the case of a SCTE-35 type signaling message, it may be provided to add an identifier in a descriptor of the message (for example one of the pre-existing descriptor of the message, for example by filling in a special field "segmentation_upid" of the descriptor) in order to use this identifier to trace the passages of the signaling message at different points in the distribution chain.

[0179] Alternatively, it may be provided to add an additional descriptor comprising an identifier in the signaling message (for example a descriptor of type “content_identifier” which contains a field with an identifier), in order to use this identifier to trace the passages of the signaling message at different points in the distribution chain.

[0180] Thus, in one or more embodiments, depending on the scenario encountered, it may be provided to generate a signaling message comprising a marker of an image of the sequence of images and comprising signaling message traceability data, to modify a signaling message comprising a marker of an image of the sequence of images by inserting signaling message traceability data (for example an identifier) ​​into an existing field, or to modify a signaling message comprising a marker of an image of the sequence of images by inserting a new field comprising signaling message traceability data (for example an identifier).

[0181] For example, in one or more embodiments, the multimedia stream received by the management unit may already include a signaling message, for example a SCTE-35 signaling message of type "time_signal()". This scenario illustrates the advantage that the presence of each signaling message previously inserted in the multimedia stream received by the management unit (106) may provide the opportunity to insert signaling message traceability data therein, in order for example to carry out one or more latency measurements.

[0182] This scenario also presents several advantages for the implementation of the proposed method: on the one hand, the insertion of signaling message traceability data into a pre-existing signaling message in the multimedia stream has no influence on the encoding and the formatting, because the conditioning of the stream to insert traceability data is already constrained by the existing message in the received stream. On the other hand, the latency measurement at points M3 and M4 of [Fig.4b] is easy in the cases in which the image corresponding to the splice point is the first of the media segment generated by the formatting unit. Finally, the proposed method can advantageously be used in production.

[0183] In one or more embodiments, the multimedia stream received by the management unit may not include a signaling message, for example a SCTE-35 signaling message (or marker) of the “time_signal()” type. In this case, the management unit (106) may be configured to generate a signaling message (for example of the SCTE-35 type) and insert it into the received multimedia stream. depending on the embodiment, different types of message may be used (for example, in the SCTE-35 framework, a “content identification” type message or a “break start” type message), depending for example on constraints relating to the conditioning of the flow carried out by the encoder.

[0184] In one or more embodiments, the encoder can perform encoded stream conditioning, for example by inserting into this stream a reference image (IDR) used for encoding the image to which the signaling message points, so that, for example, in the case where the message signals an advertising break, the advertising break that is the subject of the message starts on the first image of a fragment (or, depending on the embodiments, of a media segment (the division into fragments (or, depending on the embodiments, into segments) of media being carried out downstream of the decoder by a shaping unit), and / or so that the break (for example advertising, or during a blackout) can be carried out without artifact by the video player or decoder.

[0185] In one or more embodiments in which the encoder performs conditioning of the encoded stream, the message type of the signaling marker may correspond to an advertising marker (for example of the “break start” type in the SCTE-35 case), in which case the encoder may be configured to place a reference image in relation to the image referenced by the signaling marker. The image referenced by the signaling marker may then advantageously be an IDR which will be located at the start of the media segment generated by a shaping unit located downstream of the encoder.

[0186] In these embodiments, the management unit may be configured to insert signaling markers into the multimedia stream at a rate corresponding to the frequency of insertion of the reference images into the stream encoded by the encoder.

[0187] These embodiments have several advantages for the implementation of the proposed method: on the one hand, the implementation is possible on any input multimedia stream. On the other hand, the latency measurement at points M3 and M4 of [Fig.4b] is easy in cases in which the image corresponding to the splice point is the first of the media segment generated by the shaping unit. Finally, the proposed method allows partial control of the frequency of the latency measurements.

[0188] In one or more embodiments in which the encoder performs conditioning of the encoded stream, the message type of the signaling marker may not correspond to an advertising marker, and for example be of the “content identification” type in the SCTE-35 case, in which case the encoder may not place reference images in connection with the image referenced by the signaling marker. The division into segments (or, depending on the embodiments, into fragments) carried out by a formatting unit located downstream of the encoder is therefore not modified by the reference images inserted into the stream by the encoder.

[0189] These embodiments have several advantages for the implementation of the proposed method: on the one hand, the proposed method allows total control of the frequency of latency measurements. On the other hand, the implementation of the proposed method has no influence on the video encoding.

[0190] In one or more embodiments, the proposed method may use an event signaling message (for example, for the DASH protocol or for the HLS-fmp4 protocol and therefore for the CMAF protocol, a message of type "emsg" (for "event message")) which carries an event signaling, as specified by the ISO-IEC 23000 specification (and more particularly the ISO / IEC 23009-1:2022(E) specification).

[0191] The detection of a latency drift advantageously makes it possible to implement corrective mechanisms in order to avoid disturbances in the reading of the video stream by the video player which are particularly unpleasant for the user. For example, in systems using the MPEG-DASH transport format, the video player will stop the reading of the video stream upon detection of a drift between the time which elapses and the reference time of the video stream greater than the duration of an MPEG-DASH segment. In many systems, a latency drift greater than a few seconds (for example greater than two or three seconds) causes a disturbance, or even an interruption of the reading of the video stream.

[0192] In one or more embodiments, an average of several latency measurements carried out according to the proposed method on several images of the video stream may be calculated, in order to obtain an average latency value for the stream. Different latency measurements may then be compared to the average latency value in order to determine the latency drift using the average value.

[0193] For example, in one or more embodiments, a latency measurement may be performed for all the images or every two images of the video stream during a predetermined measurement interval (for example lasting a few seconds) of display of the images of the video stream. In another embodiment, a predefined number of latency measurements may be performed (for example ten or twenty measurements) on images spaced apart by a predefined interval (for example 100 ms, in order to make ten or twenty measurements on images spaced 100 ms apart). An average of the latency measurements performed during the measurement interval may then be performed to obtain an average latency value. In embodiments where the distribution system uses multimedia data transport units, for example of the “chunk” or “segment” type, latency measurements may be performed according to the method proposed on multiple frames, or even on each frame of the unit, and determine an average latency value from the latency measurements which will represent an average latency value for the unit (the “chunk” or segment).

[0194] This method may be repeated, for example at regular intervals, in order to calculate several average latency values. For example, a first average latency value may be obtained, then a second average value may be determined after an interval of several minutes (for example 10 minutes or 30 minutes). A latency drift measurement may then be determined on the basis of the plurality of average latency values ​​determined.

[0195] In one or more embodiments, a drift detection threshold may be predefined, and one or more latency drift measurements or, depending on the embodiment, an average latency drift value, may be compared to this threshold in order to determine whether the measured latency drift exceeds a threshold. The proposed method may further comprise detecting the presence of a latency drift, triggering a latency drift alert and / or implementing corrective actions based on the comparison of the measured latency drift (e.g. an average latency drift value) with the predefined threshold.

[0196] Depending on the embodiment chosen, the timestamp data may comprise date and time data expressed with different levels of precision (for example a millisecond precision level) and in different formats (for example the UTC format, from the English “Coordinated Universal Time”).

[0197] [Fig.5a] illustrates an example of architecture of a device for implementing the proposed method of processing a multimedia stream, according to one or more embodiments.

[0198] With reference to [Fig.5a], the device 200a comprises a controller 201a, operatively coupled to an input interface 202a, an output interface 203a and to a memory 204a, which controls a multimedia stream management unit 205a.

[0199] The input interface 202a is configured to receive data corresponding to a video stream comprising a sequence of images.

[0200] The output interface 203a is configured to provide data generated by the multimedia stream management unit 205a in the form of an enriched video stream resulting from the multiplexing of a received multimedia stream with an additional data stream according to one or more embodiments of a proposed method.

[0201] The controller 201a is configured to control the multimedia stream management unit 205a for the implementation of one or more embodiments of a proposed method.

[0202] In one or more embodiments, the media stream management unit 205a may be configured to demultiplex the video stream received on the input interface 202a to generate traceability metadata of an image of the image sequence; insert the traceability metadata of the image into an additional data stream obtained by the multimedia stream management unit; and multiplex the multimedia stream and the additional data stream.

[0203] The device 200a may be a computer, a computer network, an electronic component, or other apparatus (such as, for example, a computer server) comprising a processor operably coupled to a memory, as well as, depending on the embodiment selected, a data storage unit, and other associated hardware such as a network interface and a media reader for reading and writing to a removable storage medium (not shown in the figure). The removable storage medium may be, for example, a compact disc (CD), a digital video / versatile disc (DVD), a flash drive, a USB flash drive, etc.Depending on the embodiment, the memory, the data storage unit or the removable storage medium contains instructions which, when executed by the controller 201a, cause this controller 201a to perform or control the input interface 202a, output interface 203a, memory 204a, and media stream management unit 205a portions for implementing one or more embodiments of a proposed method. The controller 201a may be a component implementing one or more processors or a computing unit for processing video streams according to the proposed method and controlling the units 202a, 203a, 204a and 205a of the device 200a.

[0204] Furthermore, the device 200a can be implemented in software form, in which case it takes the form of a program executable by a processor, or in hardware form, such as an application-specific integrated circuit (ASIC), a system on chip (SOC), or in the form of a combination of hardware and software elements, such as for example a software program intended to be loaded and executed on a component of the FPGA (Field Programmable Gate Array) type.

[0205] [Fig.5b] illustrates another example of architecture of a device for implementing the proposed method for performing a latency measurement, according to one or more embodiments.

[0206] Referring to [Fig.5b], the device 200b comprises a controller 201b, operatively coupled to an input interface 202b and a memory 203b, which drives a video processing unit 204b.

[0207] The input interface 202b is configured to receive data captured by the device 200b.

[0208] The controller 201b is configured to drive the video processing unit 204b for implementing one or more embodiments of a proposed method.

[0209] In one or more embodiments, the video processing unit 204b may be configured to perform the following operations: upon detection, at a first point in the multimedia distribution chain, of a signaling message corresponding to an image of the sequence of images and comprising traceability data (in particular in the multimedia distribution chain), generating first detection timestamp data of the signaling message at the first point in the multimedia distribution chain; upon detection, in the multimedia stream, of the signaling message at a second point in the multimedia distribution chain, upon determination that the signaling message comprises the traceability data, generating second detection timestamp data of the signaling message at the second point in the multimedia distribution chain; and determining a latency measurement associated with the image on the basis of the first and second timestamp data.

[0210] The device 200b may be a computer, a computer network, an electronic component, or other apparatus (such as, for example, a computer server) comprising a processor operably coupled to a memory, as well as, depending on the embodiment selected, a data storage unit, and other associated hardware such as a network interface and a media reader for reading and writing to a removable storage medium (not shown in the figure). The removable storage medium may be, for example, a compact disc (CD), a digital video / versatile disc (DVD), a flash drive, a USB flash drive, etc.Depending on the embodiment, the memory, data storage unit or removable storage medium contains instructions which, when executed by the controller 201b, cause this controller 201b to perform or control the input interface 202b, memory 203b and video processing unit 204b portions for implementing one or more embodiments of a proposed method. The controller 201b may be a component implementing one or more processors or a computing unit for processing video streams according to the proposed method and controlling the units 202b, 203b and 204b of the device 200b.

[0211] Furthermore, the device 200b can be implemented in software form, in which case it takes the form of a program executable by a processor, or in hardware form, such as an application-specific integrated circuit (ASIC), a system on chip (SOC), or in the form of a combination of hardware and software elements, such as for example a software program intended to be loaded and executed on a component of the FPGA (Field Programmable Gate Array) type. Industrial application

[0212] Depending on the chosen embodiment, certain acts, actions, events or functions of each of the methods described in this document may be performed or occur in a different order than described, or may be added, merged, or not performed or occur, as the case may be. Furthermore, in some embodiments, certain acts, actions, or events are performed or occur concurrently and not successively.

[0213] Although described through a number of detailed exemplary embodiments, the proposed control method and the device for implementing an embodiment of the method include various variations, modifications and improvements which will be apparent to those skilled in the art, it being understood that these various variations, modifications and improvements are within the scope of the present disclosure. In addition, different aspects and features described above may be implemented together, or separately, or substituted for each other, and all of the various combinations and sub-combinations of the aspects and features are within the scope of the present disclosure. Furthermore, some systems and equipment described above may not incorporate all of the modules and functions described for the preferred embodiments.

Claims

Claims

1. A method of processing a multimedia stream comprising a sequence of images, the method being implemented by a multimedia stream management unit of a multimedia distribution system, and comprising: • generating signaling message traceability data; • modifying a signaling message comprising a marker of an image of the sequence of images, the signaling message being obtained by the multimedia stream management unit, the signaling message being modified by inserting the signaling message traceability data into the signaling message; and • inserting the modified signaling message into the multimedia stream.

2. The method of claim 1, wherein the message traceability data is inserted into a field of the signaling message.

3. A method according to any preceding claim, wherein the signaling message is included in the media stream, the method further comprising: extracting the signaling message from the media stream.

4. A method for processing a multimedia stream distributed by a multimedia distribution chain in a multimedia distribution system, the multimedia stream comprising a sequence of images, the method being implemented by a processing unit of the multimedia distribution system and comprising: • upon detection, at a first point of the multimedia distribution chain, of a signaling message corresponding to an image of the sequence of images and comprising traceability data, generating first detection timestamp data of the signaling message at the first point of the multimedia distribution chain; • upon detection, in the multimedia stream, of the signaling message at a second point of the multimedia distribution chain, upon determination that the signaling message comprises the traceability data, generating second detection timestamp data of the signaling message lization at the second point in the multimedia distribution chain; • determine a latency measure associated with the image based on the first and second timestamp data.

5. The method of claim 4, further comprising: • determining a first time difference between the first detection timestamp data of the signaling message at the first point and a time of making the image available at the first point; and • determining a second time difference between the second detection timestamp data of the signaling message at the second point and a time of making the image available at the second point; • wherein the latency measurement is determined based on a combination of the first timestamp data, the second timestamp data, and the first and second time differences.

6. A method according to any preceding claim, wherein the signaling message traceability data comprises identification data usable for identifying the signaling message.

7. Method according to any one of claims 1 to 3, in which the marker of the image comprises information relating to an event associated with the image.

8. A method according to any preceding claim, wherein the signaling message is of type SCTE-35.

9. A media stream processing device, comprising a processor and a memory operatively coupled to the processor, wherein the processor is configured to perform a method according to any one of claims 1 to 8.

10. Computer program, loadable into a memory associated with a processor, and comprising portions of code for implementing the steps of a method according to any one of claims 1 to 8 during the execution of said program by the processor.

Citation Information

Patent Citations

  • Methods and systems configured to manage video transcoder latencies

    EP4181514A1