Metadata for signaling the timing information of the original image

Abstract

FIELD: image encoding and decoding.SUBSTANCE: method includes syntax for signalling source image timing metadata as supplemental enhancement information (SEI) messages for single-layer and multi-layer video sequences.EFFECT: providing signalling of timing information of source images during video encoding and decoding in order to increase the efficiency of video encoding and decoding.21 cl, 8 dwg

Description

CROSS-REFERENCES TO RELATED APPLICATIONS

[0001] This patent application claims priority to Provisional Patent Application (U.S.) Serial No. 63 / 511,150, filed June 29, 2023, and Provisional Patent Application (U.S.) Serial No. 63 / 587,233, filed October 2, 2023, each of which is incorporated herein by this reference in its entirety. TECHNICAL FIELD TO WHICH THE INVENTION RELATES

[0002] This document generally relates to image and video encoding and decoding. More specifically, embodiments of the present invention relate to metadata for signaling source image timing information.BACKGROUND

[0003] In 2020, the MPEG group of the International Organization for Standardization (ISO), together with the International Telecommunication Union (ITU), released the first version of the Universal Video Coding (VVC) standard, also known as H.266 (reference document [1]). Recently, the same group has been working on developing a next-generation encoding / decoding standard that provides improved encoding / decoding performance compared to existing video encoding / decoding technologies. As part of this research, new encoding / decoding technologies are also being analyzed.

[0004] In many applications, given a sequence of decoded images, it is of interest to determine the actual temporal distance between the corresponding source images before encoding. For example, for camera-captured content, the temporal distance between source images is the difference between the time the image sensor was triggered to acquire the source image associated with the current decoded image and the time the image sensor was triggered to acquire the source image associated with the previous decoded image in output order.

[0005] The inventors of the present invention realized that improved technologies for signaling such source picture timing information (SPTI) are needed and present them herein.

[0006] The approaches described in this section represent approaches that could be implemented, but are not necessarily approaches that have previously been conceived or implemented. Therefore, unless otherwise indicated, any approach described in this section should not be assumed to be prior art simply by virtue of its inclusion in this section. Similarly, problems identified with respect to one or more approaches should not be assumed on the basis of this section to be prior art, unless otherwise indicated. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] An embodiment of the present invention is illustrated by way of example and not by way of limitation in the accompanying drawings, in which like reference numerals refer to like elements, and in which:

[0008] Figs. 1A-1E illustrate example scenarios in which the output timing of decoded images differs from the timing with which the original images were captured or otherwise created, which requires original picture timing information (SPTI); and

[0009] Fig. 2A-2C illustrate exemplary encoding and decoding processes using SPTI messaging according to embodiments of this invention. DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

[00010] This document describes exemplary embodiments of the invention that relate to signaling timing information of source images in video encoding / decoding. In the following description, for purposes of explanation, many specific details are set forth to provide a thorough understanding of various embodiments of the present invention. However, it is obvious that various embodiments of the present invention can be practiced without these specific details. In other cases, well-known structures and devices are not described in exhaustive detail in order to avoid obscuring, obscuring, or complicating the embodiments of the present invention.SUMMARY OF THE INVENTIONExemplary embodiments of the invention described in this document relate to signaling source picture timing information in image and video encoding / decoding, which is captured by the encoder and signaled as metadata to the decoder to assist in decoding. The proposed methods include an example syntax for signaling source picture timing metadata as supplementary enhancement information (SEI) messages for both single-layer and multi-layer video sequences. ORIGINAL PICTURE TIMING INFORMATION (SPTI)Introduction

[00011] Reference documents [2-3] present earlier proposals for providing some kind of picture timing information through the exchange of supplementary enhancement information (SEI) messages.In both publications, the proposed SEI message was intended to indicate the actual motion rate of the content during capture for the case in which the video bitstream contains slow motion scenes. Both publications do not disclose indicating or signaling a timing scaling factor between the actual capture timing and the output timing. The proposed message exchange may also lead to conformance issues in the Hypothetical Reference Decoder (HRD).

[00012] The embodiments of the SPTI signaling proposed here are intended to cover a similar, but broader, range of use cases than in the reference documents [2-3], while avoiding all HRD conformance issues. The exemplary embodiments of the SPTI SEI message exchange convey information only regarding when the source images were captured or otherwise created.The proposed message exchange does not affect the timing of the decoded output pictures and therefore does not affect HRD compliance.

[00013] Although embodiments are presented herein using an example SEI message encoding / decoding syntax used in MPEG video encoding / decoding (e.g., AVC, HEVC, VVC, etc.), the same information may be conveyed using alternative metadata structures used by other encoding / decoding standards, such as AV1 and AVS3 and / or future versions of video encoding / decoding standards. Example Scenarios in Which Timing of Source and Decoded Output Pictures Differ

[00014] FIGS. 1A-1E illustrate scenarios in which the timing of the output of decoded pictures differs from the timing with which the source pictures were captured or otherwise created.The receiver or post-decoding process cannot extract the timing of the original source images from the timing of the output images; however, such information can be carried in the same bitstream as the encoded video via the proposed SPTI SEI message.

[00015] Fig. 1A illustrates a typical slow-motion playback scenario. The temporal distance between the decoded output images corresponding to the original source images (gray bars) is increased relative to the temporal distance between the original source images. In slow-motion processing, additional images (white bars) are typically synthesized (often using frame interpolation techniques) and inserted before encoding to achieve smoother motion.When the decoded output images correspond to both the original source images and the synthesized images, it may be advantageous to have a means of indicating which images correspond to the original source images and which correspond to the synthesized images, preferably in addition to providing a means of indicating the timing of the source images.

[00016] Fig. 1B illustrates an example of frame rate conversion in which the temporal distance between decoded output images corresponding to the original source images (gray bars) is similar to the temporal distance between the original source images. Additional images are synthesized (white bars) before encoding to increase the frequency of the output images.For example, reference papers [4-5] describe examples of frame rate conversion using generative neural networks; however, traditional techniques known in the art, such as motion-based frame interpolation, can also be applied.

[0006] ). Fig. 1C illustrates encoding high-speed source content such that the timing of the decoded output images corresponds to standard frame rates to facilitate human inspection, while the information contained in the SPTI SEI message can be used to facilitate scientific and machine analysis.

[00018] Fig. 1D illustrates encoding time-lapse source content such that the timing of the decoded output images corresponds to standard frame rates to facilitate human viewing and inspection, while the information contained in the SPTI SEI message can be used to facilitate scientific and machine analysis.

[00019] Fig. 1E illustrates a "rewind" use case in which the decoded images are output in reverse order relative to the corresponding source images.For example, time reversal of source video is used in multimedia applications for artistic effect and storytelling. Source image timing information contained in an SPTI SEI message can be used to facilitate causal playback and to facilitate scientific and machine learning.

[00020] Fig. 2A illustrates an example encoding and decoding process using SPTI messaging. As illustrated in Fig. 2A, given a source input generated by a video source (105), in an encoder, source image timing information (109) is captured and encoded by an SPTI message encoder (115) as SPTI messages (117) (e.g., SEI messages). The source content (107) may be content captured by a camera, content captured from a screen, content generated using artificial intelligence (AI) technologies, or content generated by any other means.After compression (110), such messages (117) are multiplexed with the encoded bitstream and transmitted in the downstream direction. Such multiplexing can be performed as part of the header information in the bitstream or as part of additional metadata (e.g., SEI messages, video usability information (VUI), etc.). In the decoder, the received bitstream is split into an SPTI signal (124) and decoded video frames (122) generated by the video decoder (120), which is agreed upon with the video encoder (110). Then, the playback devices (130, 135) can generate output video with the HRD-defined timing (137) and / or with the original timing (132). For playback using the original timing, in an embodiment of the invention, the encoded images may be stored in a buffer (140) connected to a playback system (130) having timing controlled by the timing information of the original images, which is extracted from the SPTI message.

[00021] The system illustrated in Fig. 2A can be used to provide playback of high-speed encoded video of fast motion, slow motion, slow motion encoded, and reverse motion encoded video for human viewing on commonly available consumer and professional displays.

[00022] Fig. 2B illustrates an example process of encoding and decoding using SPTI messaging, but intended for machine analysis. Machine analysis includes traditional machine analysis and AI-based machine analysis. The processes on the encoding side are similar to those described in the context of Fig. 2A; however, on the decoding side, decoded output images and timing information of the original images extracted from the SPTI message are input to the machine analysis system (140).Examples of machine analysis systems include vehicle speed determination; heart rate determination in medical imaging; athlete and ball tracking in sports video; in-scene physics modeling to enable game integration; video-to-text applications; forensic analysis to detect missing or deleted original images in an encoded video bitstream; and distinguishing original original images from synthesized images in an encoded bitstream.

[00023] Fig. 2C illustrates the use of data carried in an SPTI SEI message as an auxiliary input to post-decoding processes (145) of decoded output images (122). Post-decoding processes include traditional and AI-based processes, including processes using neural networks, signaled by post-filtering metadata messages.The processes on the encoding side are similar to those described in the context of Fig. 2A. On the decoding side, the decoded output images and the timing information of the source images extracted from the SPTI message decoder (125) are input to the post-decoding processor (145). Examples of post-decoding processes include frame rate conversion; video synthesis using generative AI technologies; and motion-aware spatial scaling. Exemplary embodiments of the SPTISEI message with timing information of the source images Aspect 1

[00024] The timing information of the source images can be transmitted as a temporal distance, denoted as an interval between source images, between source images that correspond to successive decoded output images.The interval between source pictures may be determined from the number of ticks (time units) that elapse in one second, specified by a time scale variable, say, spti_time_scale, and the number of ticks corresponding to the interval between source pictures, specified by spti_num_units_in_source_picture_interval. The syntax elements spti_time_scale and spti_num_units_in_source_picture_interval may be transmitted alone or in combination with other syntax elements and conditions, as described elsewhere in this document. An example of such an SEI message is illustrated in Table 1. In some applications, it may be appropriate to constrain the values ​​of spti_time_scale and spti_num_units_in_source_picture_interval so that they have fixed values ​​throughout the entire encoded / decoded sequence. Table 1. Example 1 SPTI SEI message. source_picture_timing_info( payloadSize ) { Дескриптор ... spti_time_scale u(32) spti_num_units_in_source_picture_interval u(32) ... }

[00025] The Source Picture Timing Information (SPTI) SEI message indicates the temporal distance between source pictures associated with the corresponding decoded output pictures before encoding. For example, for camera-captured content, the temporal distance between source pictures is the difference between the time at which the image sensor was triggered to acquire the source picture associated with the current decoded picture and the time at which the image sensor was triggered to acquire the source picture associated with the previous decoded picture in output order. - spti_time_scale specifies the number of clock cycles that occur in one second. The spti_time_scale value must not be equal to 0. For example, a time coordinate system that measures time using a 27 MHz clock has an spti_time_scale equal to 27000000.-spti_num_units_in_source_picture_interval specifies the number of clock cycles, running at spti_time_scale Hz, that corresponds to the specified interval between source pictures of corresponding sequential pictures in output order in the encoded / decoded layered video sequence (CLVS). A value of 0 can be used to indicate that the source picture corresponding to the current decoded output picture is identical to the source picture corresponding to the previous decoded output picture. The specified interval between source pictures, denoted by the SourcePictureInterval variable, in units of seconds, is equal to spti_num_units_in_source_picture_interval divided by spti_time_scale. For example, to represent an interval between source images of 0.04 seconds, spti_time_scale might be 27000000 and spti_num_units_in_source_picture_interval might be 1080000.When picture n is an output picture and is not the first picture in the bitstream to be output, the value of SourcePictureTime[ n ] is retrieved as follows: SourcePictureTime[ n ] = SourcePictureTime[ previousPicInOutputOrder ] + SourcePictureInterval, (xxx1) - where previousPicInOutputOrder is the last picture to be output that precedes picture n in the output order (if any). When the value of SourcePictureTime[ 0 ] is not provided by an external means not specified in this document, then the value of SourcePictureTime[ 0 ] is output as 0. All SEI messages with source picture timing information that apply to the same CLVS must have the same values ​​for the spti_time_scale and spti_num_units_in_source_picture_interval syntax elements.Note: When used in this document, the terms "equation (xxx1)" or "table xxx" simply indicate an unknown table or equation number that may be used in the encoding / decoding specification document describing this signaling, and which may include an unknown number of other equations or tables preceding this / this one. Aspect 2

[00026] Alternatively, syntax elements indicating the elemental source picture interval, spti_num_units_in_elemental_source_picture_interval, and the source picture interval scale, spti_source_picture_interval_scale_factor, may be signaled instead of spti_num_units_in_source_picture_interval. This syntax allows multiplication to be used instead of division when calculating the SourcePictureInterval value. Differences from aspect 1 are highlighted in italics. Table 2. Example 2 SPTI SEI messages. source_picture_timing_info( payloadSize ) { Дескриптор ... spti_time_scale u(32) spti_num_units_in_ elemental _source_picture_interval u(32) spti_source_picture_interval_scale_factor se(v) ... } Updated semantics - spti_num_units_in_elemental_source_picture_interval specifies the number of clock cycles, running at spti_time_scale Hz, that corresponds to the specified element interval between source images of successive images in CLVS output order. The specified element interval between source images, which can also be denoted by the variable ElementalSourcePictureInterval, in units of seconds, is equal to spti_num_units_in_elemental_source_picture_interval divided by spti_time_scale. For example, to represent an element interval between source images of 0.04 seconds, spti_time_scale might equal 27000000, and spti_num_units_in_elemental_source_picture_interval might equal 1080000.-spti_source_picture_interval_scale_factor specifies the scaling factor used to determine the interval between source pictures of corresponding consecutive pictures in CLVS output order. A value of 0 can be used to indicate that the source picture corresponding to the current decoded output picture is identical to the source picture corresponding to the previous decoded output picture. When the current picture is the first picture in CLVS to which the SPTI SEI message applies, the value of spti_source_picture_interval_scale_factor must be 0. When the current picture is the first picture in CLVS to which the SPTI SEI message applies and the value of spti_source_picture_interval_scale_factor is greater than 0, it must not have a huge absolute value (to prevent having an se(v) code that is too long).The specified interval between source pictures, denoted by the variable SourcePictureInterval, in units of seconds, is equal to the product of ElementalSourcePictureInterval and spti_source_picture_interval_scale_factor. When picture n is the picture being output and is not the first picture in the bitstream to be output, the value of the variable SourcePictureTime[ n ] is retrieved as follows: SourcePictureTime[ n ] = SourcePictureTime[ previousPicInOutputOrder ] + SourcePictureInterval, (xxx1) - where previousPicInOutputOrder is the last picture to be output that precedes picture n in the output order (if any). When the value of SourcePictureTime[ 0 ] is not provided by an external means not specified in this document, the value of SourcePictureTime[ 0 ] is inferred to be 0.Note: In some applications, it may be appropriate to signal spti_source_picture_interval_scale_factor as ue(v) instead of se(v) and send a separate flag, spti_source_picture_interval_scale_sign_flag, to indicate that the SourcePictureInterval value is less than zero, as follows: - spti_source_picture_interval_scale_factor specifies the absolute value of the scaling factor used to determine the interval between source pictures of corresponding successive pictures in CLVS output order. A value of 0 can be used to indicate that the source picture corresponding to the current decoded output picture is identical to the source picture corresponding to the previous decoded output picture.The specified interval between source pictures, denoted by the SourcePictureInterval variable, in units of seconds, is equal to the product of ElementalSourcePictureInterval and spti_source_picture_interval_scale_factor. spti_source_picture_interval_sign_flag, equal to 0, specifies the sign of the scaling factor used to determine whether the interval between source pictures of corresponding successive pictures in CLVS output order is greater than or equal to zero. spti_source_picture_interval_sign_flag, equal to 1, specifies the sign of the scaling factor used to determine whether the interval between source pictures of corresponding successive pictures in CLVS output order is less than zero.When picture n is the picture being output and is not the first picture in the bitstream to be output, the value of SourcePictureTime[ n ] is retrieved as follows: If spti_source_picture_interval_sign_flag is 0, SourcePictureTime[ n ] = SourcePictureTime[ previousPicInOutputOrder ] + SourcePictureInterval Otherwise, spti_source_picture_interval_sign_flag is 1, SourcePictureTime[ n ] = SourcePictureTime[ previousPicInOutputOrder ] - SourcePictureInterval where previousPicInOutputOrder is the last picture to be output that precedes picture n in the output order (if any). When SourcePictureTime[ 0 ] is not provided by an external means not specified in this document, SourcePictureTime[ 0 ] is inferred to be 0.Aspect 3

[00027] Source picture timing information may be specified for all decoded output pictures for temporal sublayers having a TemporalId less than or equal to the specified maximum TemporalID, using the spti_sublayer_max_tid syntax element. Multiple SPTI SEI messages may be present for CLVS to specify source picture timing information for different values ​​of the maximum TemporalID. The TemporalId variable may be specified as in VVC. Differences from aspect 2 are shown in italics. Table 3. Example 3 SPTI SEI messages. source_picture_timing_info( payloadSize ) { Дескриптор ... spti_sublayer_max_tid u(3) ... spti_time_scale u(32) spti_num_units_in_elemental_source_picture_interval u(32) spti_source_picture_interval_scale_factor ue(v) ... } Updated semantics...-spti_sublayer_max_tid specifies the maximum temporal sublayers for which the SPTI SEI message applies....-spti_source_picture_interval_scale_factor specifies the scaling factor used to determine the interval between source pictures of corresponding sequential pictures in CLVS output order that have a TemporalID less than or equal to spti_sublayer_max_tid. A value of 0 can be used to indicate that the source picture corresponding to the current decoded output picture is identical to the source picture corresponding to the previous decoded output picture. The specified interval between source pictures, denoted by the SourcePictureInterval variable, in units of seconds, is equal to the product of ElementalSourcePictureInterval and spti_source_picture_interval_scale_factor.When picture n is a picture to be output and is not the first picture of the bitstream to be output, the value of the variable SourcePictureTime[ n ] is retrieved as follows:SourcePictureTime[ n ] = SourcePictureTime[ previousPicInOutputOrder ] + SourcePictureInterval(xxx1) - where previousPicInOutputOrder is the last picture to be output that precedes picture n in the output order (if any). When the value of SourcePictureTime[ 0 ] is not provided by an external means not specified in this document, the value of SourcePictureTime[ 0 ] is inferred to be 0. Aspect 4

[00028] As an alternative to aspect 3, timing information of source pictures may be specified for decoded output pictures of specific temporal sublayers via the spti_sublayer_tid syntax element.The temporal distance between the source image corresponding to the first decoded output image of a temporal base layer (TemporalId equal to 0) and the source image corresponding to the first decoded output image of another temporal sublayer (TemporalId greater than 0) may be specified by the spti_sublayer_delay_factor syntax element.

[00029] Multiple SPTI SEI messages may be present for CLVS to specify timing information of source images for different TemporalID values. Differences from aspect 2 are shown in italics. Table 4. Example 4 SPTI SEI messages. source_picture_timing_info( payloadSize ) { Дескриптор ... spti_sublayer_tid u(3) ... spti_time_scale u(32) spti_num_units_in_elemental_source_picture_interval u(32) spti_source_picture_interval_scale_factor ue(v) spti_sublayer_source_picture_delay_factor ue(v) ... } Updated semantics...-spti_sublayer_tid specifies the temporal sublayer for which the SPTI SEI message applies....-spti_source_picture_interval_scale_factor specifies the scaling factor used to determine the interval between source pictures of corresponding sequential pictures in CLVS output order that have a TemporalID equal to spti_sublayer_tid. A value of 0 can be used to indicate that the source picture corresponding to the current decoded output picture is identical to the source picture corresponding to the previous decoded output picture. The specified interval between source pictures, denoted by the SourcePictureInterval variable, in units of seconds, is equal to the product of ElementalSourcePictureInterval and spti_source_picture_interval_scale_factor.-spti_sublayer_source_picture_delay_factor specifies the scaling factor used to determine the temporal distance between the source picture corresponding to the first decoded output picture of the temporal sublayer with a TemporalId of 0 and the source picture corresponding to the first decoded output picture of the temporal sublayer with a TemporalId of spti_sublayer_tid. When spti_sublayer_tid is 0, spti_sublayer_source_picture_delay_factor must be 0. The specified delay of the source pictures in the sublayer, denoted by the SublayerSourcePictureDelay variable, in units of seconds, is equal to the product of ElementalSourcePictureInterval and spti_sublayer_source_picture_delay_factor.When picture n is an output picture corresponding to the temporal sublayer specified by spti_sublayer_tid and is not the first picture in the bitstream to be output, the value of SourcePictureTime[ n ] is retrieved as follows:SourcePictureTime[ n ] = SourcePictureTime[ previousPicInOutputOrder ] + SourcePictureInterval + SublayerSourcePictureDelay(xxx1) - where previousPicInOutputOrder is the last picture to be output that precedes picture n in the output order (if any). When the value of SourcePictureTime[ 0 ] is not provided by an external means not specified in this document, the value of SourcePictureTime[ 0 ] is inferred to be 0....Aspect 5

[00030] Source picture timing information may be specified for multiple temporal sublayers by looping around the value of the spti_max_sublayers_minus1 syntax element.Differences from aspect 3 are highlighted in italics. Table 5. Example 5 SPTI SEI messages. source_picture_timing_info( payloadSize ) { Дескриптор ... spti_time_scale u(32) spti_num_units_in_elemental_source_picture_interval u(32) ... spti_max_sublayers_minus_1 u(3) for( i = 0; i <= spti_max_sublayers_minus1; i++) { ... spti_ sublayer _source_picture_interval_scale_factor[ i ] ue(v) ... } ... } Updated semantics...- spti_max_sublayers_minus_1plus 1 specifies the maximum number of temporal sublayers that may be present in a CLVS....- spti_sublayer_source_picture_interval_scale_factor[ i ] specifies the scaling factor used when determining the interval between source pictures of corresponding consecutive pictures in output order in a CLVS that have a TemporalID less than or equal to i. A value of 0 can be used to indicate that the source picture corresponding to the current decoded output picture is identical to the source picture corresponding to the previous decoded output picture. The specified interval between source pictures associated with output pictures having a TemporalID less than or equal to i, denoted by the variable SourcePictureInterval[ i ], in units of seconds, is equal to the product of ElementalSourcePictureInterval and spti_source_picture_interval_scale_factor[ i ].When image n is the image being output that has a TemporalId less than or equal to i and does not represent the first image of the bitstream to be output, the value of the variable SourcePictureTime[ n ] is retrieved as follows: SourcePictureTime[ n ] = SourcePictureTime[ previousPicInOutputOrder ] + SourcePictureInterval[ i ](xxx1) - where previousPicInOutputOrder represents the last image to be output that precedes image n in the output order (if any). When the value of SourcePictureTime[ 0 ] is not provided by an external means not specified in this document, the value of SourcePictureTime[ 0 ] is inferred to be 0....Aspect 6

[00031] The timing information of source pictures may be indicated as being inferred for a dyadic encoding / decoding structure based on temporal sublayers by means of the value of the spti_sublayer_dyadic_flag syntax element. Differences from aspect 5 are shown in italics.Table 6. Example 6 SPTI SEI messages. source_picture_timing_info( payloadSize ) { Дескриптор ... spti_time_scale u(32) spti_num_units_in_elemental_source_picture_interval u(32) ... spti_max_sublayers_minus_1 u(3) spti_sublayer_dyadic_flag u(1) if( !spti_sublayer_dyadic_flag) { for( i = 0; i <= spti_max_sublayers_minus1; i++) { ... spti_sublayer_source_picture_interval_scale_factor[ i ] ue(v) ... } ... } ... } Updated semantics... - spti_sublayer_dyadic_flag equal to 1 indicates that temporal sublayers are encoded / decoded in a dyadic relationship, and that the syntactic element spti_source_picture_interval_scale_factor[ i ] is not present in the SPTI SEI message. (Ed. note: may also indicate that other syntactic elements are also not present). spti_sublayer_dyadic_flag equal to 0 indicates that temporal sublayers may not be encoded / decoded in a dyadic relationship, and that the syntactic element spti_source_picture_interval_scale_factor[ i ] is present in the SPTI SEI message. (Ed. note: may also indicate that other syntactic elements are also present)....-spti_sublayer_source_picture_interval_scale_factor[ i ], when present, specifies the scaling factor used in determining the interval between source pictures of corresponding consecutive pictures in CLVS output order that have a TemporalID less than or equal to i. A value of 0 can be used to indicate that the source picture corresponding to the current decoded output picture is identical to the source picture corresponding to the previous decoded output picture. The specified scaling factor, denoted by the variable SublayerScaleFactor[ i ], is determined as follows: - If spti_sublayer_dydadic_flag is 0, SublayerScaleFactor[ i ] is equal to spti_sublayer_source_picture_interval_scale_factor[ i ].- Otherwise, spti_sublayer_dydadic_flag is 1, SublayerScaleFactor[ i ] is 2 (spti_max_sublayers_minus_1 - i)The specified interval between source images associated with output images that have a TemporalID less than or equal to i, denoted by the variable SourcePictureInterval[ i ], in units of seconds, is equal to the product of ElementalSourcePictureInterval and SublayerScaleFactor[ i ].When image n is an image to be output that has a TemporalId less than or equal to i and does not represent the first image of the bitstream to be output, the value of the variable SourcePictureTime[ n ] is retrieved as follows:SourcePictureTime[ n ] = SourcePictureTime[ previousPicInOutputOrder ] + SourcePictureInterval[ i ] (xxx1)- where previousPicInOutputOrder represents the last image to be is output, which precedes image n in output order (if any).When the SourcePictureTime[ 0 ] value is not provided by an external means not specified in this document, the SourcePictureTime[ 0 ] value is inferred to be 0....Aspect 7

[00032] The timing information of source pictures may be indicated as inferred for the specified temporal sublayer-based encoding / decoding structure by means of the spti_sublayer_implicit_timing_flag value and the spti_sublayer_implicit_timing_type value to indicate the type of temporal sublayer-based encoding / decoding structure. Differences from aspect 5 are shown in italics. Table 7. Example 7 SPTI SEI message. source_picture_timing_info( payloadSize ) { Descriptor ... spti_time_scale u(32) spti_num_units_in_elemental_source_picture_interval u(32) ... spti_max_sublayers_minus_1 u(3) spti_sublayer_implicit_timing_flag u(1) if( spti_sublayer_implicit_timing_flag ) { spti_sublayer_implicit_timing_type ue(v) else { for( i = 0; i <= spti_max_sublayers_minus1; i++) { ... spti_sublayer_source_picture_interval_scale_factor[ i ] ue(v) ... } ... } ... } Updated semantics: spti_sublayer_implicit_timing_flag equal to 1 indicates that the spti_sublayer_implicit_timing_type syntax element is present in the SPTI SEI message. spti_sublayer_implicit_timing_flag equal to 0 indicates that the spti_sublayer_implicit_timing_type syntax element is not present in the SPTI SEI message. spti_sublayer_implicit_timing_type, when present, indicates the encoding / decoding structure based on temporal sublayers, as described in Table 7axx. If not present, the value of spti_sublayer_implicit_timing_type is inferred to be 0. The value of spti_sublayer_implicit_timing_type must be in the range 0 to 2, inclusive, in bitstreams conforming to the current edition of this document. Values ​​3-7, inclusive, for spti_sublayer_implicit_timing_type are reserved for future use by ITU-T|ISO / IEC and shall not appear in bitstreams conforming to the current edition of this document.Decoders conforming to the current edition of this document shall ignore SPTI SEI messages with spti_sublayer_implicit_timing_type in the range 3 to 7, inclusive. Values ​​of spti_sublayer_implicit_timing_type greater than 7 shall not be present in bitstreams conforming to the current edition of this document and are not reserved for future use. When spti_sublayer_implicit_timing_type is 2, the field-frame informational SEI message shall be present for the current picture. Table 7axx. Sample informative description of spti_sublayer_implicit_timing_type. spti_sublayer_implicit_timing_type Description 0 Unknown, not specified, or specified by an external means not specified in this document 1 Double 2 Derived from the frame-field SEI information message 3..7 Reserved ...-spti_sublayer_source_picture_interval_scale_factor[ i ], when present, specifies the scaling factor used in determining the interval between source pictures of corresponding consecutive pictures in CLVS output order that have a TemporalID less than or equal to i. A value of 0 may be used to indicate that the source picture corresponding to the current decoded output picture is identical to the source picture corresponding to the previous decoded output picture. The specified scaling factor, denoted by the variable SublayerScaleFactor[ i ], is determined as follows:- If spti_sublayer_implicit_timing_flag is 0, SublayerScaleFactor[ i ] is equal to spti_sublayer_source_picture_interval_scale_factor[ i ].- Otherwise, if spti_sublayer_implicit_timing_flag is 1 and spti_sublayer_implicit_timing_type is 0, SublayerScaleFactor[ i ] is 2 (spti_max_sublayers_minus_1 - i)- Otherwise, spti_sublayer_implicit_timing_flag is 1 and spti_sublayer_implicit_timing_type is 1, SublayerScaleFactor[ i ] is (ed. note: TBD, and can also be specified by an external tool)The specified interval between source pictures associated with output pictures having TemporalID less than or equal to i, denoted by the variable SourcePictureInterval[ i ], in units of seconds, is equal to the product of ElementalSourcePictureInterval and SublayerScaleFactor[ i ].When picture n is an output picture with a TemporalId less than or equal to i and is not the first picture in the bitstream to be output, the value of SourcePictureTime[ n ] is retrieved as follows: SourcePictureTime[ n ] = SourcePictureTime[ previousPicInOutputOrder ] + SourcePictureInterval[ i ] (xxx1) - where previousPicInOutputOrder is the last picture to be output that precedes picture n in the output order (if any). When SourcePictureTime[ 0 ] is not provided by an external means not specified in this document, SourcePictureTime[ 0 ] is inferred to be 0.Aspect 8

[00033] The decoded output pictures associated with a temporal sublayer can be specified as corresponding to either the original source pictures or synthesized pictures (e.g., as in frame rate conversion applications) via the spti_synthesized_picture_flag value. Differences from aspect 5 are shown in italics. Table 8. Example 8 SPTI SEI messages. source_picture_timing_info( payloadSize ) { Descriptor ... spti_time_scale u(32) spti_num_units_in_elemental_source_picture_interval u(32) ... spti_max_sublayers_minus_1 u(3) for( i = 0; i <= spti_max_sublayers_minus1; i++) { ... spti_sublayer_source_picture_interval_scale_factor[ i ] ue(v) ... spti_sublayer_synthesized_picture_flag[ i ] u(1) ... } ... } Updated semantics... - spti_sublayer_synthesized_picture_flag equal to 1 indicates that the decoded output images belonging to the i-th temporal sublayer are synthesized and do not correspond to the unmodified original source images. spti_sublayer_synthesized_picture_flag equal to 0 provides no such indication. Aspect 9

[00034] The type of relationship between the timing of the source images and the timing of the decoded output images can be specified using spti_source_picture_timing_type. Differences from aspect 5 are highlighted in italics. Table 9. Example 9 SPTI SEI messages source_picture_timing_info( payloadSize ) { Descriptor ... spti_time_scale u(32) spti_num_units_in_elemental_source_picture_interval u(32) ... spti_source_picture_timing_type u(8) ... spti_max_sublayers_minus_1 u(3) for(i = 0; i <= spti_max_sublayers_minus1; i++) { ... spti_sublayer_source_picture_interval_scale_factor[ i ] ue(v) ... } ... } Updated semantics...- spti_source_picture_timing_type specifies the temporal relationship between source pictures and the corresponding decoded output pictures, as specified in table 9axx, where ( spti_source_picture_timing_type & bitMask ) not equal to 0 indicates that the temporal relationship has the interpretation associated with the bitMask value in table 9axx. When spti_source_picture_timing_type is greater than 0 and ( spti_source_picture_timing_type & bitMask ) equals 0, the interpretation associated with the bitMask value is not applicable to SPTI. When spti_source_picture_timing_type is 0, the timing relationship can be specified by the application. The value of spti_source_picture_timing_type must be in the range 0 to 127, inclusive, in bitstreams conforming to the current edition of this document.Values ​​128-255, inclusive, for spti_source_picture_timing_type are reserved for future use by ITU-T|ISO / IEC and shall not appear in bitstreams conforming to the current edition of this document. Decoders conforming to the current edition of this document shall ignore SPTI SEI messages with spti_source_picture_timing_type in the range 128 to 255, inclusive. Table 9axx. Example definition of spti_source_picture_timing_type. bitMask Interpretation 0×01 Slow motion 0×02 Frame rate conversion 0×04 High-speed imaging 0×08 Formation of time-lapse images 0×10 Time reversal 0×20 Still image / freeze frame 0×40 Sporadic or event-driven ...When picture n is the picture being output and is not the first picture in the bitstream to be output, the value of SourcePictureTime[ n ] is retrieved as follows: If (spti_source_picture_timing_type & bitMask) is 0, SourcePictureTime[ n ] = SourcePictureTime[ previousPicInOutputOrder ] + SourcePictureInterval Otherwise, (spti_source_picture_timing_type & bitMask) is 1, SourcePictureTime[ n ] = SourcePictureTime[ previousPicInOutputOrder ] - SourcePictureInterval where previousPicInOutputOrder is the last picture to be output that precedes picture n in the output order (if any). When SourcePictureTime[ 0 ] is not provided by an external means not specified in this document, SourcePictureTime[ 0 ] is inferred to be 0.Aspect 10

[00035] The type of relationship between the timing of the source images and the timing of the decoded output images can be explicitly specified as such by spti_source_timing_equals_output_timing_flag. Differences from aspect 5 are shown in italics. Table 10. Example 10 SPTI SEI messages. source_picture_timing_info( payloadSize ) { Descriptor ... spti_source_timing_equals_output_timing_flag u(1) if( !spti_source_timing_equals_output_timing_flag ) { ... spti_time_scale u(32) spti_num_units_in_elemental_source_picture_interval u(32) ... spti_max_sublayers_minus_1 u(3) for(i = 0; i <= spti_max_sublayers_minus1; i++) { ... spti_sublayer_source_picture_interval_scale_factor[ i ] ue(v) ... } ... } ... } Updated semantics... - spti_source_timing_equals_output_timing_flag equal to 1 indicates that the timing of the source images is equal to the timing of the corresponding decoded output images. spti_source_timing_equals_output_timing_flag equal to 0 indicates that the timing of the source pictures may not be the same as the timing of the corresponding decoded output pictures. When spti_source_timing_equals_output_timing_flag is 1 and a Picture Timing SEI message is present for the current picture, the timing of the source pictures can be determined from the information carried in the Picture Timing SEI message....Aspect 11

[00036] Discontinuities in the timing of the source pictures, such as scene cuts and splices, can be indicated by spti_source_timing_discontinuity_flag or, alternatively, by spti_source_timing_discontinuity_type. Differences from aspect 5 are shown in italics. Table 11. Example 11 SPTI SEI messages source_picture_timing_info( payloadSize ) { Descriptor ... spti_source_timing_discontinuity_flag u(1) ... spti_time_scale u(32) spti_num_units_in_elemental_source_picture_interval u(32) ... spti_max_sublayers_minus_1 u(3) for( i = 0; i <= spti_max_sublayers_minus1; i++) { ... spti_sublayer_source_picture_interval_scale_factor[ i ] ue(v) ... } ... } ... } Updated semantics... - spti_source_timing_discontinuity_flag equal to 1 indicates that the timing of the source images corresponding to the decoded output images is discontinuous. spti_source_timing_equals_output_timing_flag equal to 0 provides no such indication. Note: The timing of source images may be discontinuous due to scene cuts, scene splices, and event-triggered security video, as examples. Aspect 12

[00037] Discontinuities in the timing of source images, such as scene cuts and scene splices, may be indicated by spti_source_timing_discontinuity_type and further specified by the syntax elements spti_source_timing_discontinuity_type and spti_source_transition_type. Differences from aspect 11 are shown in italics. Table 12. Example 12 SPTI SEI messages source_picture_timing_info( payloadSize ) { Descriptor ... spti_source_timing_discontinuity_flag u(1) if( spti_source_timing_discontinuity_flag ) { spti_source_timing_discontinuity_type u(3) if( spti_source_timing_discontinuity_type == 1 ) { spti_source_transition_type u(8) } } ... spti_time_scale u(32) spti_num_units_in_elemental_source_picture_interval u(32) ... spti_max_sublayers_minus_1 u(3) for(i = 0; i <= spti_max_sublayers_minus1; i++) { ... spti_sublayer_source_picture_interval_scale_factor[ i ] ue(v) ... } ... } ... } Updated semantics... - spti_source_timing_discontinuity_type, when present and equal to 1, indicates that the timing of the source images corresponding to the decoded output images is discontinuous, as specified in Table 12axx. If not present, the value of spti_source_timing_discontinuity_type is inferred to be 0. Table 12axx. Sample definition of spti_source_timing_discontinuity_type Meaning Description 0 Unknown, not specified, or determined by an external means not specified in this document 1 Scene transition 2 Splice 3 Event detection 4..7 Reserved -spti_source_transition_type, when present, specifies the scene transition type that applies to source pictures corresponding to decoded output pictures, as specified in table 12bxx, where ( spti_source_transition_type & bitMask ) not equal to 0 indicates that the scene transition relationship has the interpretation associated with the bitMask value in table 12bxx. When spti_source_transition_type is greater than 0 and ( spti_source_transition_type & bitMask ) is 0, the interpretation associated with the bitMask value is not applicable to SPTI. When spti_source_transition_type is 0, the scene transition can be specified by the application. If not present, spti_source_transition_type is inferred to be 0. The value of spti_source_transition_type must be in the range 0 to 127, inclusive, in bitstreams conforming to the current edition of this document.Values ​​128-255, inclusive, for spti_source_transition_type are reserved for future use by ITU-T|ISO / IEC and shall not appear in bitstreams conforming to the current edition of this document. Decoders conforming to the current edition of this document shall ignore SPTI SEI messages with spti_source_transition_type in the range 128 to 255, inclusive. Table 12bxx. Example definition of spti_source_transition_type. bitMask Interpretation 0×01 Blackout 0×02 Disappearance from black 0×04 Mixing 0×08 Approximation 0×10 Distancing 0×20 Dolly shooting 0×40 Displacement Considerations for the scope of the SEI messageOption 1: The SEI message is valid for the entire CLVS. SEI message Scope Timing information of source images CLVS containing SEI message Semantics: When a Source Picture Timing Information SEI message is present for any CLVS image of a particular layer, the Source Picture Timing Information SEI message must be present for the first image of the CLVS. The Source Picture Timing Information SEI message is valid for the current layer in decoding order, from the current image to the end of the CLVS. All Source Picture Timing Information SEI messages that apply to the same CLVS must have similar content. Option 2: The scope of an SEI message is specified in the SEI message syntax. SEI message Scope Timing information of source images Specified by the SEI message syntax source_picture_timing_info( payloadSize ) { Descriptor spti_cancel_flag u(1) if( !spti_cancel_flag ) { spti_persistence_flag u(1) ... } } The semantics of ...-spti_cancel_flag equal to 1 indicates that the SPTI SEI message cancels the effect of any previous SPTI SEI message in output order that applies to the current layer. spti_cancel_flag equal to 0 specifies that the SPTI follows. - spti_persistence_flag specifies the effect of the SPTI SEI message for the current layer. - spti_persistence_flag equal to 0 specifies that the SPTI SEI message applies only to the current decoded picture. - spti_persistence_flag equal to 1 specifies that the SPTI SEI message applies to the current decoded picture and remains in effect (persists) for all subsequent pictures in the current layer in output order until one or more of the following conditions are met: - A new CLVS of the current layer begins. - The bitstream ends. - The picture in the current layer is output to the AU associated with the SPTI SEI message that follows the current picture in output order.

[00038] In another embodiment of the invention, Table 13 illustrates another example of the proposed SPTI message.Table 13. Example 13 SPTI SEI message. source_picture_timing_info( payloadSize ) { Descriptor spti_cancel_flag u(1) if(! spti_cancel_flag) { spti_persistence_flag u(1) spti_source_type_present_flag u(1) if( spti_source_type_present_flag ) spti_source_type u(16) spti_source_timing_info_present_flag u(1) if( spti_source_timing_info_present_flag ) { spti_time_scale u(32) spti_num_units_in_elemental_interval u(18) spti_max_sublayers_minus_1 u(3) for(i = 0; i <= spti_max_sublayers_minus1; i++) { spti_sublayer_interval_scale_factor[ i ] ue(v) spti_sublayer_synthesized_picture_flag[ i ] u(1) } } } }

[00039] Compared with the previous embodiments of the invention, table 13 includes the following changes. - spti_num_units_in_elemental_interval is now unsigned 18-bit instead of 32-bit to save bits. Table 13axx below is an improved version of table 9axx described above, adding clarification to various descriptions such as "slow motion" and the like. In addition, new semantic constraints are added to prevent mutually exclusive temporal relationships between source images and the corresponding decoded output images. As an example, the semantics prevents the combination of "high-speed imaging" and "timelapse imaging". - spti_source_timing_equals_output_timing_flag (defined in aspect 10) is now replaced by spti_source_timing_info_present_flag.- spti_source_timing_info_present_flag (defined in aspect 9) now represents spti_source_type. The spti_source_type_present_flag flag has been added and is set as follows: - spti_source_type_present_flag equal to 1 indicates that the spti_source_type syntax element is present in the SEI message. spti_source_type_present_flag equal to 0 indicates that the spti_source_type syntax element is not present in the SEI message. For completeness, the semantics of the renamed flags are as follows: -spti_source_type, when present, indicates the temporal relationship between the source images and the corresponding decoded output images, as specified in the 13axx table below, where ( spti_source_type & bitMask ) not equal to 0 indicates that the temporal relationship has the interpretation associated with the bitMask value in the corresponding 13axx table row.When spti_source_type is greater than 0 and ( spti_source_type & bitMask ) is 0, the interpretation associated with the bitMask value is not applicable to the SPTI SEI message. If not present, spti_source_picture_type is inferred to be 0. When spti_source_type is 0, the timing relationship may be specified by the application. The value of spti_source_type must be in the range 0 to 127, inclusive, in bitstreams conforming to the current edition of this document. Decoders conforming to the current edition of this document must ignore SPTI SEI messages with spti_source_picture_timing_type in the range 128 to 65535, inclusive. Table 13axx. Interpretation of spti_source_type. bitMask Interpretation 0×01 Slow motion: The absolute value of the temporal distance between successive source images should probably be smaller than the temporal distance between the corresponding decoded output images. 0×02 Fast motion: The absolute value of the temporal distance between successive source images is likely to be larger than the temporal distance between the corresponding decoded output images. 0×04 High-speed imaging: The absolute value of the temporal distance between successive source images should probably be less than 1 / 120 second. 0×08 Formation of time-lapse images: The time distance between the original images should probably be more than 1.001 / 24 seconds. 0×10 Time reversal: The absolute value of the temporal distance between successive source images is specified as negative (i.e., decoded images are output in reverse temporal order relative to the timing of the corresponding source images). 0×20 Still image / freeze frame: The temporal distance between the source images is likely to be 0 (i.e., two or more decoded images are likely to represent the same source image). 0×40 Sporadic or event-driven: The temporal distance between source images is likely to be variable. The value of ( spti_source_type & 0×04 ) & ( spti_source_type & 0×08 ) must be zero (i.e., spti_source_type must not specify both high-speed and time-lapse imaging). The temporalReversalFlag variable is ( spti_source_type & 0×10 )? 1 : 0. Alternatively, the temporalReversalFactor variable can be set to ( spti_source_type & 0×10 )? -1 : 1, with the appropriate modification of equation (8-X), as noted below. -spti_source_timing_info_present_flag equal to 1 indicates that the syntax elements spti_time_scale, spti_num_units_in_elemental_interval, spti_max_sublayers_minus_1, spti_sublayer_interval_scale_factor[ i ], and spti_sublayer_synthesized_picture_flag[ i ] are present in the SEI message. spti_source_timing_info_present_flag equal to 0 indicates that these syntax elements are not present in the SEI message.(Note: spti_num_units_in_elemental_interval is a shorthand for spti_num_units_in_elemental_source_picture_interval specified in 2 above), spti_sublayer_interval_scale_factor[ i ], when present, specifies the scaling factor used to determine the interval between source pictures of corresponding consecutive pictures in CLVS output order that have a TemporalID less than or equal to i. A value of 0 may be used to indicate that the source picture corresponding to the current decoded output picture is identical to the source picture corresponding to the previous decoded output picture. When ( spti_source_type & 0×04 ) is 1, the value of spti_sublayer_interval_scale_factor[ i ] shall be 0.The specified interval between source pictures associated with output pictures having a TemporalID less than or equal to i, denoted by the variable SourcePictureInterval[ i ], in units of seconds, is retrieved as follows:SourcePictureInterval[ i ] = ElementalSourcePictureInterval * spti_sublayer_interval_scale_factor[ i ] * ( 1 – 2 * temporalReversalFlag ) (8-X)(Note: spti_sublayer_interval_scale_factor[ i ] is a simplified name for spti_sublayer_source_picture_interval_scale_factor[ i ], defined earlier).When using the variable temporalReversalFactor, equation (8-X) is modified as follows:SourcePictureInterval[ i ] = ElementalSourcePictureInterval * spti_sublayer_interval_scale_factor[ i ] * temporalReversalFactor (8-X)List of References MaterialsEach of the reference materials listed in this document is incorporated into this document in its entirety by this reference.The term "JVET" refers to the Joint Video Experts Group of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29.

[0001] "Versatile Video Coding", Rec. ITU-T H.266, August 2020.

[0002] H.B. Teo et al., "AHG9: Alternative Picture Timing SEI", JVET-AC0141, 29th Meeting, by teleconference, January 11-20, 2023.

[0003] H.B. Teo et al., "AHG9: Alternative Output Timing Hint SEI", JVET-AD0161, 30th Meeting, Antalya, Turkey, April 21-28, 2023.

[0004] B. Chen et al., "AHG9: Generative Face Video SEI Message", JVET-AC0088, by teleconference, January 11-20, 2023.

[0005] B. Chen et al., "AHG9: Common SEI Message of Generative Face Video", JVET-AD0051, 30th Meeting, Antalya, Turkey, April 21-28, 2023.

[0006] Phantom / AMETEK T4040 camera main Web Page, accessed June 24, 2023, https: / / www.phantomhighspeed.com / products / cameras / tseries / t4040.Exemplary Implementation of a Computer System

[00040] Embodiments of the present invention may be implemented using a computer system, systems configured in electronic circuits and components, an integrated circuit device (IC) such as a microcontroller, a field programmable gate array (FPGA) or other configurable or programmable logic device (PLD), a discrete time processor or digital signal processor (DSP), an application-specific integrated circuit (ASIC), and / or a device (apparatus) that includes one or more of such systems, devices (devices) or components. The computer and / or IC may execute, control, or execute instructions related to signaling timing information of source images in encoding / decoding images and video, such as the instructions described herein. The computer and / or IC may calculate any of a variety of parameters or values,which relate to signaling timing information of source images in the encoding / decoding of images and video described herein. Image and video-related embodiments may be implemented in hardware, software, firmware, and various combinations thereof.

[00041] Certain implementations of the invention comprise computer processors that execute software instructions that instruct the processors to perform the method of the invention. For example, one or more processors in a display, encoder, set-top box, transcoder, and the like may implement methods related to signaling timing information of source images in the encoding / decoding of video and images, as described above, by executing software instructions in a program memory,accessible to processors. Embodiments of the invention may also be provided in the form of a software product. The software product may comprise any non-volatile and tangible medium that carries a set of computer-readable signals containing instructions that, when executed by a data processor, instruct the data processor to perform the method of the invention. Software products according to the invention may have any of a wide range of non-volatile and tangible forms. The software product may comprise, for example, physical media such as magnetic storage media including floppy disks, hard disks, optical storage media including CD-ROM, DVD, electronic storage media including ROM, flash RAM, and the like. Machine-readable signals in the software product may optionally be compressed or encrypted.

[00042] When a component (e.g., a software module, a processor, a node, a device,circuit, etc.) is mentioned above, unless otherwise indicated, reference to that component (including reference to "means") shall be interpreted as including, as equivalents of that component, any component that performs the function of the described component (e.g., that is functionally equivalent), including components that are not structurally equivalent to the disclosed structure that performs the function in the illustrated exemplary embodiments of the invention. Equivalents, additions, alternatives, and other information

[00043] Thus, exemplary embodiments of the invention have been described that relate to signaling timing information of source images in image and video encoding / decoding. In the foregoing detailed description, embodiments of the present invention have been described with reference to a variety of specific details that may vary depending on the implementation. Thus,The sole and exclusive indication of what the invention is and what the applicants intend by the invention is the set of claims that follow from this application, in the particular form in which such claims follow, including all subsequent amendments. All definitions expressly set forth herein for terms contained in these claims shall dictate the meaning of those terms when used in the claims. Therefore, limitations, elements, properties, features, advantages, or attributes that are not expressly set forth in the claims shall not limit the scope of these claims in any way. Therefore, the detailed description and drawings are to be considered illustrative,and not in a limiting sense.

[00044] Various aspects of the present disclosure can be understood from the following exemplary embodiments (EEE): EEE 1. A method for decoding a video bitstream, the method comprising: - receiving an encoded video bitstream comprising an encoded picture section comprising encoding a sequence of video pictures, and a signaling section including source picture timing parameters, wherein the source picture timing parameters comprise: - a source picture timeline parameter indicating the number of ticks that occur in one second; and - a source picture inter-picture interval number of units parameter indicating the number of ticks of a clock generator operating at the frequency of the source picture timeline parameter; and - decoding the sequence of video pictures based on the source picture timing parameters. EEE 2. The method of EEE 1,further comprising calculating the value of the interval between source pictures (SourcePictureInterval) as the quotient of the number of units associated with the source picture in the interval between source pictures by the time scale of the source pictures.EEE 3. A method for decoding a video bitstream, wherein the method comprises: - receiving an encoded video bitstream comprising an encoded picture section including the coding of a sequence of video pictures, and a signaling section including source picture timing parameters (SPT), wherein the source picture timing parameters comprise: - a source picture time scale parameter indicating the number of ticks that pass in one second; - a parameter of the number of units associated with the source picture in the elementary interval between source pictures indicating the number of ticks of a clock generator operating at the frequency of the source picture time scale parameter,which corresponds to a specified elemental interval between source images of successive output images; and- a parameter of the scaling factor of the interval between source images, specifying the scaling factor; and- decoding the sequence of video images based on the timing parameters of the source images.EEE 4. The method according to EEE 3, further comprising: calculating the value of the interval between source images (SourcePictureInterval) as the result of the product of the elemental interval between source images and the scaling factor of the interval between source images, wherein the elemental interval between source images is calculated as the quotient of dividing the parameter of the number of units associated with the source image in the elemental interval between source images by the time scale parameter of the source images.EEE 5. The method according to EEE 2 or EEE 4,further comprising a calculation of the source picture time for picture n as follows:SourcePictureTime[ n ] = SourcePictureTime[ previousPicInOutputOrder ] + SourcePictureInterval,- wherein picture n, n>0, denotes the index of an output picture that is not the first output picture in the video bitstream, and the variable previousPicInOutputOrder denotes the last picture to be output that precedes picture n in the output order (if any).EEE 6. The method of EEE 3, wherein the timing parameters of the source pictures further comprise a maximum temporal identifier of a sublayer of the source pictures identifying the maximum sublayer for which the SPT parameters apply.EEE 7. The method of EEE 3, wherein the timing parameters of the source pictures further comprise:- an identifier of a temporal sublayer of the source pictures,for which the SPT parameters are applied; and - a delay factor for the source picture sublayer that specifies a scaling factor used in determining the temporal distance between the source picture corresponding to the first decoded output picture of the temporal sublayer having a TemporalId value of 0 and the source picture corresponding to the first decoded output picture of the temporal sublayer having a TemporalId value equal to the identifier of the temporal sublayer of the source pictures. EEE 8. The method of EEE 7, further comprising calculating the source picture time for picture n as follows: SourcePictureTime[ n ] = SourcePictureTime[ previousPicInOutputOrder ] + SourcePictureInterval + SublayerSourcePictureDelay, - where picture n, n>0, denotes the index of an output picture that is not the first output picture in the video bitstream,and the variable previousPicInOutputOrder designates the last picture to be output that precedes picture n in the output order (if any), and SublayerSourcePictureDelay specifies a delay factor for the source picture sublayer.EEE 9. The method of EEE 3, wherein the source picture interval scaling factor parameter is specified for each of the N sublayers, wherein N > 0.EEE 10. The method of EEE 9, wherein the source picture timing parameters further comprise a source picture sublayer dyadicity flag that, if set to 1, indicates that the temporal sublayers are coded in a dyadic relationship, and that the source picture interval scaling factor parameter specifying the scaling factor syntax element is not present.EEE 11. The method of EEE 9,EEE 12. The method of EEE 9, wherein the timing parameters of the source images further include an implicit timing flag for a source image sublayer, which, when set to 1, indicates that the implicit timing type information for the sublayer is present for each of the sublayers. EEE 13. The method of EEE 9, wherein the timing parameters of the source images further include a flag of synthesized images in a source image sublayer for each of the N sublayers, which, when set to 1, indicates that the decoded output images belonging to the i-th temporal sublayer are synthesized and do not correspond to the unmodified original source images. EEE 14. The method of EEE 9, wherein the timing parameters of the source images further include a parameter of the timing type of the source images,which indicates a temporal relationship between the source images and the corresponding decoded output images according to a look-up table.EEE 14. The method of EEE 9, wherein the timing parameters of the source images further include a source image timing equals to output timing flag, which, when set to 1, indicates that the timing of the source images is the same as the timing of the corresponding decoded output images.EEE 15. The method of EEE 3, wherein the scaling factor parameter of the interval between the source images is specified using the absolute value of the scaling factor and the sign flag of the scaling factor.EEE 16. The method of EEE 9, wherein the timing parameters of the source images further include a source image timing discontinuity flag, which, when set to 1, indicates that the timing of the source images,corresponding to the decoded output images is discontinuous.EEE 17. The method of EEE 16, wherein the timing parameters of the source images further include a source image timing discontinuity type parameter and a source image transition type parameter, wherein the source image timing discontinuity type parameter indicates a discontinuity in the source images according to the first table, and the source image transition type parameter indicates a transition type in the source images according to the second table.EEE 18. The method of EEE 4, wherein the timing parameters of the source images further include an source type parameter, wherein the source type parameter indicates a temporal relationship between the source images and the corresponding decoded output images.EEE 19. The method of EEE 18, wherein the source type parameter may indicate one or more of the following: slow motion, fast motion,high-speed imaging, time-lapse imaging, time-reversal imaging, still imaging, or sporadic imaging.EEE 20. The method of EEE 19, wherein the source type parameter may not indicate both high-speed imaging and time-lapse imaging.EEE 21. The method of EEE 18, wherein if the source type parameter is present, the value of the interval between source pictures (SourcePictureInterval) is further adjusted by a function of the source type value.EEE 22. The method of any one of EEE 1-21, wherein the signaling section comprises a supplemental enhancement information (SEI) message section or a video usability information (VUI) message section.EEE 23. A tangible computer-readable storage medium storing computer-executable instructions for execution by one or more processors of the method according to any of the methods,set out in EEE 1-21.EEE 24. A device comprising a processor and configured to perform any of the methods set out in EEE 1-21.,

Claims

1. A method for decoding a video bitstream, wherein the method comprises the steps of: - receive an encoded video bit stream containing an encoded image section including the coding of a sequence of video images, and a signaling section including source image timing parameters (SPT), wherein the source image timing parameters contain: - a parameter of the time scale of the original images, indicating the number of ticks that pass in one second; - a parameter of the number of units associated with the source image in the elemental interval between source images, indicating the number of clock cycles of the clock generator operating at the frequency of the parameter of the time scale of the source images, which corresponds to the indicated elemental interval between source images of successive images in the output order, wherein the value of the indicated elemental interval between source images (ElementalSourcePictureInterval) in units of seconds is equal to the quotient of dividing the parameter of the number of units associated with the source image in the elemental interval between source images by the parameter of the time scale of the source images; and - a parameter of the scaling factor of the interval between the source images, specifying the scaling factor, wherein the value of the interval between the source images (SourcePictureInterval) in units of seconds is equal to the product of the mentioned elemental interval between the source images and the scaling factor of the interval between the source images; and - decode a sequence of video images based on the timing parameters of the original images.

2. The method according to claim 1, further comprising the step of calculating the time of the original images for image n as follows: SourcePictureTime[ n ] = SourcePictureTime[ previousPicInOutputOrder ] + SourcePictureInterval, - where image n, n>0, denotes the index of an output image that is not the first output image in the video bitstream, and the variable previousPicInOutputOrder denotes the last image to be output that precedes image n in the output order (if any).

3. The method according to claim 1, wherein the timing parameters of the source images further comprise a maximum temporal identifier of a sublayer of the source images, identifying the maximum sublayer for which the SPT parameters are applied.

4. The method according to paragraph 1, wherein the timing parameters of the source images additionally comprise: - the identifier of the temporary sublayer of the source images for which the SPT parameters are applied; and - a delay factor for the source image sublayer that specifies a scaling factor used in determining the temporal distance between the source image corresponding to the first decoded output image of the temporal sublayer having a TemporalId value equal to 0 and the source image corresponding to the first decoded output image of the temporal sublayer having a TemporalId value equal to the identifier of the temporal sublayer of the source images.

5. The method according to claim 4, further comprising the step of calculating the time of the original images for image n as follows: SourcePictureTime[ n ] = SourcePictureTime[ previousPicInOutputOrder ] + SourcePictureInterval + SublayerSourcePictureDelay, - where image n, n > 0, denotes the index of an output image that is not the first output image in the video bitstream, and the variable previousPicInOutputOrder denotes the last image to be output that precedes image n in the output order (if any), and SublayerSourcePictureDelay specifies a delay factor for the source image sublayer.

6. The method according to claim 1, wherein the parameter of the scaling factor of the interval between the original images is specified for each of the N sublayers, wherein N > 0.

7. The method according to claim 6, in which the timing parameters of the source images further comprise a 2-way flag of the source image sublayer, which, if set to 1, indicates that the temporal sublayers are encoded in a 2-way relationship, and that the scaling factor parameter of the interval between source images, specifying the scaling factor syntax element, is not present.

8. The method according to claim 6, in which the timing parameters of the source images further include an implicit timing flag for a sublayer of the source images, which, when set equal to 1, indicates that the implicit timing type information for the sublayer is present for each of the sublayers.

9. The method according to claim 6, wherein the timing parameters of the source images further include a flag of the synthesized images in the source image sublayer for each of the N sublayers, which, when set equal to 1: - indicates that the decoded output images belonging to the i-th temporal sublayer are synthesized and do not correspond to the unmodified original source images.

10. The method according to claim 6, wherein the source image timing parameters further include a source image timing type parameter that indicates a temporal relationship between the source images and the corresponding decoded output images according to a look-up table.

11. The method according to claim 6, in which the timing parameters of the source images further include a source image timing equals output timing flag, which, when set to 1, indicates that the timing of the source images is similar to the timing of the corresponding decoded output images.

12. The method according to claim 1, wherein the parameter of the scaling factor of the interval between the source images is specified using the value of the absolute value of the scaling factor and the sign flag of the scaling factor.

13. The method according to claim 6, in which the timing parameters of the source images further include a source image timing discontinuity flag, which, when set equal to 1, indicates that the timing of the source images corresponding to the decoded output images is discontinuous.

14. The method according to claim 13, in which the timing parameters of the source images further include a parameter of the type of discontinuity of the timing of the source images and a parameter of the type of transition of the source images, wherein the parameter of the type of discontinuity of the timing of the source images indicates discontinuity in the source images according to the first table, and the parameter of the type of transition of the source images indicates the type of transition in the source images according to the second table.

15. The method according to claim 1, wherein the timing parameters of the source images further include a source type parameter, wherein the source type parameter indicates a temporal relationship between the source images and the corresponding decoded output images.

16. The method of claim 15, wherein the source type parameter may indicate one or more of the following: slow motion, fast motion, high speed imaging, time-lapse imaging, time reversal, still imaging, or sporadic imaging.

17. The method of claim 16, wherein the initial type parameter may not indicate either high-speed imaging or high-speed imaging.

18. The method according to claim 15, wherein if the source type parameter is present, then the value of the interval between source images (SourcePictureInterval) is additionally adjusted by a function of the value of the source type.

19. The method according to any one of claims 1-18, wherein the signaling section comprises a supplemental enhancement information (SEI) message section or a video usability information (VUI) message section.

20. A tangible computer-readable storage medium storing computer-executable instructions for carrying out, by one or more processors, a method in accordance with any of the methods of paragraphs 1-19.

21. A device containing a processor and configured to implement any of the methods according to paragraphs 1-19.

Images