Metadata for transmitting source picture timing information

Abstract

This paper describes a method, system, and bitstream syntax for encoding and decoding video using source picture timing information captured by an encoder and transmitted to a decoder as metadata to aid in decoding. The proposed method includes syntax examples for transmitting source picture timing metadata as supplemental enhancement information (SEI) messaging for both single-layer and multi-layer video sequences.

Description

Technical Field

[0001] Cross - Reference to Related Applications This patent application claims the benefit of U.S. Provisional Patent Application No. 63 / 511,150, filed Jun. 29, 2023, and U.S. Provisional Patent Application No. 63 / 587,233, filed Oct. 2, 2023. The entire contents of each application are incorporated herein by reference.

[0002] This document generally relates to the encoding and decoding of images and videos. More specifically, embodiments of the present invention relate to metadata for signaling source picture timing information.

Background Art

[0003] In 2020, the MPEG group of the International Organization for Standardization (ISO), jointly with the International Telecommunication Union (ITU), released the first version of the Versatile Video Coding Standard (VVC), also known as H.266 (Reference [1]). More recently, the group has been working on the development of a next - generation coding standard that provides better coding performance than existing video coding technologies. As part of this research, new coding techniques are also being considered.

Summary of the Invention

Problems to be Solved by the Invention

[0004] In many applications, it is important to determine the actual temporal distance between corresponding source pictures before encoding when a sequence of decoded pictures is given. For example, in the case of camera - captured content, the temporal distance between source pictures is the difference between the time when the image sensor was exposed to generate the source picture related to the current decoded picture and the time when the image sensor was exposed to generate the source picture related to the previous decoded picture in the output order.

[0005] As the inventors acknowledge in this application, there is a need for improved techniques for transmitting such source picture timing information (SPTI) as a signal, and such techniques are presented herein.

[0006] The approaches described in this section are feasible approaches, but not necessarily previously conceived or pursued approaches. Therefore, unless otherwise specified, it should not be assumed that any of the approaches described in this section constitutes prior art simply because it is included in this section. Similarly, unless otherwise specified, it should not be assumed that any problems identified with respect to one or more approaches were recognized in the prior art based on this section. [Brief explanation of the drawing]

[0007] Embodiments of the present invention are shown illustratively and non-limitingly in the accompanying drawings. Similar reference numerals indicate similar elements.

[0008] [Figure 1A] This figure illustrates an exemplary scenario where source picture timing information (SPTI) is required because the output timing of the decoded picture differs from the timing when the source picture was taken or otherwise created. [Figure 1B] This figure illustrates an exemplary scenario where source picture timing information is required because the output timing of the decoded picture differs from the timing when the source picture was taken or otherwise created. [Figure 1C] This figure illustrates an exemplary scenario where source picture timing information is required because the output timing of the decoded picture differs from the timing when the source picture was taken or otherwise created. [Figure 1D] This figure illustrates an exemplary scenario where source picture timing information is required because the output timing of the decoded picture differs from the timing when the source picture was taken or otherwise created. [Figure 1E]This figure illustrates an exemplary scenario where source picture timing information is required because the output timing of the decoded picture differs from the timing when the source picture was taken or otherwise created.

[0009] [Figure 2A] This figure shows an example of encoding and decoding processing using SPTI messaging according to an embodiment of the present invention. [Figure 2B] This figure shows an example of encoding and decoding processing using SPTI messaging according to an embodiment of the present invention. [Figure 2C] This figure shows an example of encoding and decoding processing using SPTI messaging according to an embodiment of the present invention. [Modes for carrying out the invention]

[0010] Description of Exemplary Embodiments This specification describes examples of embodiments related to the signal transmission of source picture timing information in video coding. For illustrative purposes, many specific details are provided in the following description to allow for a thorough understanding of the various embodiments of the present invention. However, it is clear that various embodiments of the present invention can be implemented without these specific details. Furthermore, in other examples, detailed descriptions of well-known structures and devices are omitted to avoid unnecessarily complicating, obscuring, or making the embodiments of the present invention difficult to understand.

[0011] Outline The embodiments described herein relate to the transmission of source picture timing information in image and video encoding, which is acquired by an encoder and transmitted to a decoder as metadata to assist in decoding. The proposed method includes exemplary syntax for transmitting source picture timing metadata as supplemental enhancement information (SEI) messaging for both single-layer and multi-layer video sequences.

[0012] Source Picture Timing Information (SPTI) Introduction References [2-3] describe prior proposals for providing some picture timing information via supplemental enhancement messaging. In each proposal, the proposed SEI message was intended to indicate the actual motion speed of the content at the time of capture if the video bitstream included slow-motion scenes. Neither proposal indicates or transmits a timing scale coefficient between the actual capture timing and the output timing. The proposed messaging may also interfere with compatibility issues in the hypothetical reference decoder (HRD).

[0013] The proposed SPTI signaling embodiments presented herein are similar to those in references [2-3] but are intended to address a broader range of use cases while simultaneously avoiding all HRD compliance issues. The example SPTI-SEI messaging embodiment transmits only information about when the source picture was captured or otherwise created. The proposed messaging does not affect the timing of the decoded output picture and therefore does not impact HRD compliance.

[0014] This specification illustrates embodiments using exemplary encoding syntax for SEI messaging used in MPEG video encoding (AVC, HEVC, VVC, etc.), but similar information can also be transmitted using alternative metadata structures used in other encoding standards such as AV1 and AVS3, and / or future video encoding standards.

[0015] Example scenario where the timing of the source and the decoded output picture differs. Figures 1A to 1E illustrate a scenario where the output timing of the decoded picture differs from the timing at which the source picture was captured or otherwise created. The receiver or post-decoding process cannot retrieve the original source picture timing from the output picture timing. However, the proposed SPTI·SEI message allows such information to be transmitted within the same bitstream as the encoded video.

[0016] Figure 1A shows a typical slow-motion playback scenario. The temporal distance between the decoded output picture (gray bars) corresponding to the original source picture is magnified compared to the temporal distance between the original source pictures. In slow-motion processing, additional pictures (white bars) are typically composited (often using frame interpolation) and inserted before encoding to produce smoother motion. When the decoded output picture corresponds to both the original source picture and the composite picture, it is beneficial to have means to indicate which picture corresponds to the original source picture and which picture corresponds to the composite picture, in addition to providing means to indicate the source picture timing.

[0017] Figure 1B shows an example of frame rate conversion, where the temporal distance between the decoded output pictures (gray bars) corresponding to the original source pictures is the same as the temporal distance between the original source pictures. To increase the output picture rate, additional pictures (white bars) are synthesized before encoding. For example, references [4-5] describe an example of frame rate conversion using a generative neural network, but prior art known in the field, such as motion-based frame interpolation, is also applicable.

[0018] Figure 1C shows a special example of slow motion output in high-speed shooting. Currently, some high-speed cameras for products claim to achieve 9,600 fps at a resolution of 2,560×1,664 for scientific, industrial, automotive, and media applications (e.g., reference [6]). Figure 1C shows the encoding of high-speed source content such that the timing of the decoded output pictures corresponds to the standard frame rate, facilitating human confirmation and enabling the information contained in the SPTI·SEI messages to be used for scientific and mechanical analysis.

[0019] Figure 1D shows the encoding of time-lapse source content such that the timing of the decoded output pictures corresponds to the standard frame rate, facilitating human viewing and confirmation and enabling the information contained in the SPTI·SEI messages to be used for scientific and mechanical analysis.

[0020] Figure 1E shows a "rewind" use case where the decoded pictures are output in reverse order relative to the corresponding source pictures. For example, the time reversal of source video is used in media applications for artistic effects and storytelling. The source picture timing information contained in the SPTI·SEI messages can be used to facilitate causal playback and scientific and mechanical analysis.

[0021] Figure 2A shows an example of encoding and decoding processes using SPTI messaging. As shown in Figure 2A, when a source input generated by a video source (105) is provided, in an encoder, source picture timing information (109) is acquired and encoded as SPTI messaging (e.g., SEI messaging) (117) by an SPTI message encoder (115). The source content (107) can be content captured by a camera, content captured by a screen capture, content generated using artificial intelligence (AI) technology, or content generated by other means. After compression (110), such messaging (117) is multiplexed with an encoded bitstream and transmitted downstream. Such multiplexing can be performed as part of the header information within the bitstream or as part of auxiliary metadata (e.g., SEI messaging, video user ability information (VUI), etc.). In a decoder, the received bitstream is separated into an SPTI signal (124) and a video decoded frame generated from a video decoder (120) corresponding to the video encoder (110) (122). Thereafter, a playback device (130, 135) can generate an output video at either HRD defined timing (137) and / or source timing (132). In playback using source timing, in one embodiment, an encoded picture is stored in a buffer (140) connected to a playback system (130) that is timing-controlled by source picture timing information extracted from an SPTI message.

[0022] The system shown in Figure 2A can be used to facilitate playback for human viewing of encoded videos in fast, time-lapse, slow motion, and reverse motion on commonly available consumer and business displays.

[0023] Figure 2B shows an example of encoding and decoding processes using SPTI messaging for the purpose of mechanical analysis. Mechanical analysis includes conventional and AI-based mechanical analysis. The encoding process is similar to the context described in Figure 2A, but on the decoding side, the decoded output picture and source picture timing information extracted from the SPTI message are input to the mechanical analysis system (140). Examples of mechanical analysis systems include vehicle speed determination, heart rate timing determination in medical imaging, tracking of players and balls in sports videos, scene physics modeling to facilitate integration in gaming, video-to-text conversion applications, forensic analysis aimed at detecting missing or deleted source pictures in encoded video bitstreams, and identification of composite pictures and original source pictures in encoded bitstreams.

[0024] Figure 2C shows an example where data transmitted via SPTI·SEI messages is used as auxiliary input to the post-decoding process (145) of the decoded output picture (122). The post-decoding process includes conventional and AI-based processes, as well as neural network processing where the signal is transmitted by post-processing filtering of the metadata message. The coding process is the same as described in Figure 2A. On the decoding side, the decoded output picture and source picture timing information extracted from the SPTI message decoder (125) are input to the post-decoding processor (145). Examples of post-decoding processes include frame rate conversion, image synthesis using generative AI technology, and motion-aware spatial scaling.

[0025] SPTI Embodiment Example Source picture timing information SEI message Appearance 1 Source picture timing information can be transmitted as the temporal distance (denoted as the source picture interval) between source pictures corresponding to consecutive decoded output pictures. The source picture interval can be determined from the number of time units elapsed per second, indicated by the time scale variable (e.g., spti_time_scale), and the number of time units corresponding to the source picture interval, indicated by spti_num_units_in_source_picture_interval. The syntactic elements spti_time_scale and spti_num_units_in_source_picture_interval can be transmitted alone or in combination with other syntactic elements and conditions described elsewhere in this document. An example of such an SEI message is shown in Table 1. In some applications, it may be beneficial to constrain the values ​​of spti_time_scale and spti_num_units_in_source_picture_interval to fixed values ​​throughout the entire encoding sequence.

[0026] [Table 1]

[0027] The Source Picture Timing Information (SPTI) SEI message indicates the temporal distance between source pictures associated with the corresponding decoded output picture before encoding. For example, in the case of camera-captured content, the temporal distance between source pictures is the difference between the time the image sensor was exposed to generate the source picture associated with the current decoded picture and the time the image sensor was exposed to generate the source picture associated with the previous decoded picture in output order.

[0028] `spti_time_scale` defines the number of time units that elapse in one second. The value of `spti_time_scale` must not be zero. For example, in a time coordinate system that measures time using a 27MHz clock, `spti_time_scale` would be 27,000,000.

[0029] spti_num_units_in_source_picture_interval specifies the number of time units of a clock operating at frequency spti_time_scaleHz, corresponding to the indicated source picture interval for corresponding consecutive pictures in the output order of a coded layer-wise video sequence (CLVS). A value of 0 may be used to indicate that the source picture corresponding to the currently decoded output picture is the same as the source picture corresponding to the previously decoded output picture.

[0030] The source picture interval in seconds, represented by the variable SourcePictureInterval, is equal to the quotient obtained by dividing spti_num_units_in_source_picture_interval by spti_time_scale. For example, to represent a source picture interval equal to 0.04 seconds, spti_time_scale could be equal to 27,000,000 and spti_num_units_in_source_picture_interval could be equal to 1,080,000.

[0031] If picture n is an output picture and is not the first picture in the output bitstream, the value of the variable SourcePictureTime[n] is derived as follows: SourcePictureTime[n] = SourcePictureTime[previousPicInOutputOrder] + SourcePictureInterval (xxx1) Here, previousPicInOutputOrder is the last picture (if any) that is output before picture n in the output order. If the value of SourcePictureTime[0] is not provided by an external means not specified in this document, the value of SourcePictureTime[0] is assumed to be 0.

[0032] All source picture timing information SEI messages applied to the same CLVS have the same values ​​for the syntax elements spti_time_scale and spti_num_units_in_source_picture_interval.

[0033] Note: As used herein, the terms (xxx1) or table xxx simply indicate an unknown formula or table number that may be used in the coding specification describing this signal transmission, and that there may be an unknown number of other formulas or tables preceding this formula or table.

[0034] Appearance 2 Alternatively, the syntactic elements spti_num_units_in_elemental_source_picture_interval, which indicate the elemental source picture interval, and spti_source_picture_interval_scale_factor, which scale the source picture interval, can be signaled instead of spti_num_units_in_source_picture_interval. This syntax allows multiplication rather than division to be applied when calculating the SourcePictureInterval value. Differences from Embodiment 1 are shown in italics.

[0035] [Table 2]

[0036] Updated semantics spti_num_units_in_elemental_source_picture_interval specifies the number of time units of a clock operating at frequency spti_time_scaleHz, corresponding to the indicated elemental source picture interval for consecutive pictures in the output order within CLVS.

[0037] The elemental source picture interval expressed in seconds (also represented by the variable ElementalSourcePictureInterval) is equal to the quotient obtained by dividing spti_num_units_in_elemental_source_picture_interval by spti_time_scale. For example, to represent an elemental source picture interval equal to 0.04 seconds, spti_time_scale could be equal to 27,000,000 and spti_num_units_in_elemental_source_picture_interval could be equal to 1,080,000.

[0038] The spti_source_picture_interval_scale_factor specifies the scale factor used to determine the source picture interval for corresponding consecutive pictures in the output order in CLVS. A value of 0 may indicate that the source picture corresponding to the currently decoded output picture is identical to the source picture corresponding to the previously decoded output picture.

[0039] If the current picture is the first picture in a CLVS to which the SPTI·SEI message is applied, the value of spti_source_picture_interval_scale_factor should be 0. If the current picture is the first picture in a CLVS to which the SPTI·SEI message is applied, and the value of spti_source_picture_interval_scale_factor is greater than 0, the value should not be very large (to avoid making the se(v) code too long).

[0040] The source picture interval expressed in seconds, represented by the variable SourcePictureInterval, is equal to the product of ElementalSourcePictureInterval and spti_source_picture_interval_scale_factor.

[0041] If picture n is an output picture and is not the first picture in the output bitstream, the value of the variable SourcePictureTime[n] is derived as follows: SourcePictureTime[n] = SourcePictureTime[previousPicInOutputOrder] + SourcePictureInterval (xxx1)

[0042] Here, previousPicInOutputOrder is the last picture (if any) that is output before picture n in the output order. If the value of SourcePictureTime[0] is not provided by an external means not specified in this document, the value of SourcePictureTime[0] is assumed to be 0.

[0043] Note: In some applications, it may be preferable to signal spti_source_picture_interval_scale_factor as ue(v) instead of se(v), and to send a separate flag spti_source_picture_interval_scale_sign_flag to indicate that the SourcePictureInterval value is less than zero, as shown below.

[0044] The spti_source_picture_interval_scale_factor specifies the absolute magnitude of the scale factor used to determine the source-picture interval between corresponding consecutive pictures in the output order within CLVS. A value of 0 can be used to indicate that the source picture corresponding to the currently decoded output picture is identical to the source picture corresponding to the previously decoded output picture.

[0045] The source picture interval expressed in seconds, represented by the variable SourcePictureInterval, is equal to the product of ElementalSourcePictureInterval and spti_source_picture_interval_scale_factor. ...

[0046] If spti_source_picture_interval_sign_flag is equal to 0, it specifies that the sign of the scale factor used to determine the source-picture interval of corresponding consecutive pictures in the output order in CLVS is 0 or greater. If spti_source_picture_interval_sign_flag is equal to 1, it specifies that the sign of the scale factor used to determine the source-picture interval of corresponding consecutive pictures in the output order in CLVS is less than zero.

[0047] If picture n is an output picture and is not the first picture in the output bitstream, the value of the variable SourcePictureTime[n] is derived as follows: -If spti_source_picture_interval_sign_flag is equal to 0, SourcePictureTime[n] = SourcePictureTime[previousPicInOutputOrder] + SourcePictureInterval - If spti_source_picture_interval_sign_flag is equal to 1, SourcePictureTime[n] = SourcePictureTime[previousPicInOutputOrder] - SourcePictureInterval

[0048] Here, previousPicInOutputOrder is the last picture (if any) that is output before picture n in the output order. If the value of SourcePictureTime[0] is not provided by an external means not specified in this document, the value of SourcePictureTime[0] is assumed to be 0.

[0049] Appearance 3 Source picture timing information can be indicated by the syntactic element spti_sublayer_max_tid for all decoded output pictures for time sublayers where the TemporalId is less than or equal to a specified maximum TemporalID. Multiple SPTI·SEI messages may exist for a single CLVS to indicate source picture timing information for different maximum TemporalID values. The variable TemporalId can be defined in the same way as in VVC. Differences from Embodiment 2 are indicated in italics.

[0050] [Table 3]

[0051] Updated semantics spti_sublayer_max_tid specifies the maximum time sublayer to which SPTI·SEI messages are applied. ...

[0052] The spti_source_picture_interval_scale_factor specifies the scale factor used to determine the source picture interval for corresponding consecutive pictures in the output order in CLVS, provided that the TemporalID value is less than or equal to spti_sublayer_max_tid. A value of 0 may indicate that the source picture corresponding to the currently decoded output picture is identical to the source picture corresponding to the previously decoded output picture.

[0053] The source picture interval expressed in seconds, represented by the variable SourcePictureInterval, is equal to the product of ElementalSourcePictureInterval and spti_source_picture_interval_scale_factor.

[0054] If picture n is an output picture and is not the first picture in the output bitstream, the value of the variable SourcePictureTime[n] is derived as follows: SourcePictureTime[n] = SourcePictureTime[previousPicInOutputOrder] + SourcePictureInterval (xxx1)

[0055] Here, previousPicInOutputOrder is the last picture (if any) that is output before picture n in the output order. If the value of SourcePictureTime[0] is not provided by an external means not specified in this document, the value of SourcePictureTime[0] is assumed to be 0.

[0056] Pattern 4 As an alternative to embodiment 3, source picture timing information can be indicated for the decoded output picture of a particular time sublayer using the syntactic element spti_sublayer_tid. The temporal distance between the source picture corresponding to the first decoded output picture of a time base layer (TemporalId equal to 0) and the source picture corresponding to the first decoded output picture of another time sublayer (TemporalId greater than 0) can be defined by the syntactic element spti_sublayer_delay_factor.

[0057] For a single CLVS, multiple SPTI / SEI messages may exist to indicate source picture timing information corresponding to different TemporalID values. Differences from Embodiment 2 are indicated in italics.

[0058] [Table 4]

[0059] Updated semantics spti_sublayer_tid specifies the time sublayer to which SPTI·SEI messages are applied.

[0060] The spti_source_picture_interval_scale_factor specifies the scale factor used to determine the source picture interval for corresponding consecutive pictures in the output order in CLVS, which have a TemporalID value equal to spti_sublayer_tid. A value of 0 may indicate that the source picture corresponding to the currently decoded output picture is identical to the source picture corresponding to the previously decoded output picture.

[0061] The source picture interval expressed in seconds, represented by the variable SourcePictureInterval, is equal to the product of ElementalSourcePictureInterval and spti_source_picture_interval_scale_factor.

[0062] The spti_sublayer_source_picture_delay_factor specifies the scaling factor used to determine the temporal distance between the source picture corresponding to the first decoded output picture of a time sublayer with a TemporalId value equal to 0 and the source picture corresponding to the first decoded output picture of a time sublayer with a TemporalID value equal to spti_sublayer_tid. If spti_sublayer_tid is equal to 0, the value of spti_sublayer_source_picture_delay_factor is set to 0.

[0063] The sublayer source picture delay in seconds, represented by the variable SublayerSourcePictureDelay, is equal to the product of ElementalSourcePictureInterval and spti_sublayer_source_picture_delay_factor.

[0064] If picture n is the output picture, corresponds to the time sublayer indicated by spti_sublayer_tid, and is not the first picture in the output bitstream, then the value of the variable SourcePictureTime[n] is derived as follows: SourcePictureTime[n] = SourcePictureTime[previousPicInOutputOrder] + SourcePictureInterval + SublayerSourcePictureDelay (xxx1)

[0065] Here, previousPicInOutputOrder is the last picture (if any) that is output before picture n in the output order. If the value of SourcePictureTime[0] is not provided by an external means not specified in this document, the value of SourcePictureTime[0] is assumed to be 0.

[0066] Appearance 5 Source picture timing information for several time sublayers can be defined by looping based on the value of the syntax element spti_max_sublayers_minus1. Differences from Embodiment 3 are indicated in italics.

[0067] [Table 5]

[0068] Updated semantics spti_max_sublayers_minus_1 plus 1 specifies the maximum number of time sublayers that can exist in CLVS.

[0069] spti_sublayer_source_picture_interval_scale_factor[i] defines the scale factor used to determine the source picture interval for corresponding consecutive pictures in the output order in CLVS, where TemporalId is less than or equal to i. A value of 0 may be used to indicate that the source picture corresponding to the currently decoded output picture is the same as the source picture corresponding to the previously decoded output picture.

[0070] The indicated source picture interval in seconds associated with an output picture having a TemporalId less than or equal to i, represented by the variable SourcePictureInterval[i], is equal to the product of ElementalSourcePictureInterval and spti_source_picture_interval_scale_factor[i].

[0071] If picture n is an output picture with a TemporalId less than or equal to i, and is not the first picture in the output bitstream, the value of the variable SourcePictureTime[n] is derived as follows: SourcePictureTime[n] = SourcePictureTime[previousPicInOutputOrder] + SourcePictureInterval[i] (xxx1)

[0072] Here, previousPicInOutputOrder is the last picture (if any) that is output before picture n in the output order. If the value of SourcePictureTime[0] is not provided by an external means not specified in this document, the value of SourcePictureTime[0] is assumed to be 0.

[0073] Appearance 6 Source picture timing information can be shown as estimated for a binary time sublayer coding structure, based on the value of the syntactic element spti_sublayer_dyadic_flag. Differences from Embodiment 5 are indicated in italics.

[0074] [Table 6]

[0075] Updated semantics If spti_sublayer_dyadic_flag is equal to 1, it indicates that the time sublayer is encoded using a binary relation and that the spti_source_picture_interval_scale_factor[i] syntactic element is not present in the SPTI·SEI message. (Editor's note: This may also indicate the absence of other syntactic elements.) If spti_sublayer_dyadic_flag is equal to 0, it indicates that the time sublayer may not be encoded using a binary relation and that the spti_source_picture_interval_scale_factor[i] syntactic element is present in the SPTI·SEI message. (Editor's note: This may also indicate the possibility of other syntactic elements being present.)

[0076] If spti_sublayer_source_picture_interval_scale_factor[i] exists, it specifies the scale factor used to determine the source-picture interval of corresponding consecutive pictures in the output order in CLVS, for which TemporalId less than or equal to i. A value of 0 may be used to indicate that the source picture corresponding to the currently decoded output picture is the same as the source picture corresponding to the previously decoded output picture.

[0077] The scale factor shown, represented by the variable SublayerScaleFactor[i], is determined as follows: -If spti_sublayer_dydadic_flag is equal to 0, SublayerScaleFactor[i] will be equal to spti_sublayer_source_picture_interval_scale_factor[i]. If spti_sublayer_dydadic_flag is equal to 1, then SublayerScaleFactor[i] is equal to 2(spti_max_sublayers_minus_1-i).

[0078] The indicated source picture interval in seconds associated with an output picture having a TemporalId less than or equal to i, represented by the variable SourcePictureInterval[i], is equal to the product of ElementalSourcePictureInterval and SublayerScaleFactor[i].

[0079] If picture n is an output picture with a TemporalId less than or equal to i, and is not the first picture in the output bitstream, the value of the variable SourcePictureTime[n] is derived as follows: SourcePictureTime[n] = SourcePictureTime[previousPicInOutputOrder] + SourcePictureInterval[i] (xxx1)

[0080] Here, previousPicInOutputOrder is the last picture (if any) that is output before picture n in the output order. If the value of SourcePictureTime[0] is not provided by an external means not specified in this document, the value of SourcePictureTime[0] is assumed to be 0.

[0081] Appearance 7 Source picture timing information can be shown as estimated for a defined time sublayer coding structure, based on the value of spti_sublayer_implicit_timing_flag and the value of spti_sublayer_implicit_timing_type, which indicates the type of time sublayer coding structure. Differences from Embodiment 5 are indicated in italics.

[0082] [Table 7]

[0083] Updated semantics If spti_sublayer_implicit_timing_flag is equal to 1, it indicates that the spti_sublayer_implicit_timing_type syntax element is present in the SPTI·SEI message. If spti_sublayer_implicit_timing_flag is equal to 0, it indicates that the spti_sublayer_implicit_timing_type syntax element is not present in the SPTI·SEI message.

[0084] If present, spti_sublayer_implicit_timing_type indicates the time sublayer coding structure listed in Table 7axx. If absent, the value of spti_sublayer_implicit_timing_type is assumed to be equal to 0. The value of spti_sublayer_implicit_timing_type must be in the range of 0 to 2 (inclusive) in a bitstream conforming to this version of this document. Values ​​of spti_sublayer_implicit_timing_type from 3 to 7 (inclusive) are reserved for future use by ITU-T|ISO / IEC and must not exist in a bitstream conforming to this version of this document. Decoders conforming to this version of this document shall ignore SPTI·SEI messages with spti_sublayer_implicit_timing_type in the range of 3 to 7 (inclusive). Values ​​of spti_sublayer_implicit_timing_type greater than 7 must not exist in a bitstream conforming to this version of this document and are not reserved for future use.

[0085] If spti_sublayer_implicit_timing_type is equal to 2, a field frame information SEI message must exist for the current picture.

[0086] [Table 8]

[0087] spti_sublayer_source_picture_interval_scale_factor[i], if present, specifies the scale factor used to determine the source-picture interval of corresponding consecutive pictures in the output order in CLVS, for which TemporalId less than or equal to i. A value of 0 may be used to indicate that the source picture corresponding to the currently decoded output picture is identical to the source picture corresponding to the previously decoded output picture.

[0088] The scale factor shown is represented by the variable SublayerScaleFactor[i] and is determined as follows: -If spti_sublayer_implicit_timing_flag is equal to 0, SublayerScaleFactor[i] is equal to spti_sublayer_source_picture_interval_scale_factor[i]. If spti_sublayer_implicit_timing_flag is equal to 1 and spti_sublayer_implicit_timing_type is equal to 0, then SublayerScaleFactor[i] is equal to 2(spti_max_sublayers_minus_1-i). -If spti_sublayer_implicit_timing_flag is equal to 1 and spti_sublayer_implicit_timing_type is equal to 1, SublayerScaleFactor[i] is equal to (editor's note: undetermined; may be defined by external means).

[0089] The indicated source picture interval in seconds associated with an output picture having a TemporalId less than or equal to i, represented by the variable SourcePictureInterval[i], is equal to the product of ElementalSourcePictureInterval and SublayerScaleFactor[i].

[0090] If picture n is an output picture with a TemporalId less than or equal to i, and is not the first picture in the output bitstream, the value of the variable SourcePictureTime[n] is derived as follows: SourcePictureTime[n] = SourcePictureTime[previousPicInOutputOrder] + SourcePictureInterval[i] (xxx1)

[0091] Here, previousPicInOutputOrder is the last picture (if any) that is output before picture n in the output order. If the value of SourcePictureTime[0] is not provided by an external means not specified in this document, the value of SourcePictureTime[0] is assumed to be 0.

[0092] Appearance 8 The decoded output picture associated with the time sublayer can be indicated by the value of spti_synthesized_picture_flag as either the original source picture or the composite picture (e.g., from a frame rate conversion application). Differences relating to Embodiment 5 are shown in italics.

[0093] [Table 9]

[0094] Updated semantics If spti_sublayer_synthesized_picture_flag is equal to 1, it indicates that the decoded output picture belonging to the i-th time sublayer is a synthesis and does not correspond to the original, unmodified source picture. If spti_sublayer_synthesized_picture_flag is equal to 0, it does not indicate this.

[0095] Appearance 9 The type of relationship between the timing of the source picture and the timing of the decoded output picture can be indicated by spti_source_picture_timing_type. Differences from Embodiment 5 are indicated in italics.

[0096] [Table 10]

[0097] Updated semantics `spti_source_picture_timing_type` defines the timing relationship between the source picture and the corresponding decoded output picture, as defined in Table 9axx. Here, if `(spti_source_picture_timing_type & bitMask)` is not equal to 0, it indicates that the timing relationship has an interpretation associated with the bitMask value in Table 9axx. If `spti_source_picture_timing_type` is greater than 0 and `(spti_source_picture_timing_type & bitMask)` is equal to 0, the interpretation associated with the bitMask value does not apply to SPTI. If `spti_source_picture_timing_type` is equal to 0, the timing relationship may be defined by the application.

[0098] The value of spti_source_picture_timing_type must be in the range of 0 to 127 (inclusive) in a bitstream conforming to this version of this document. Values ​​of spti_source_picture_timing_type from 128 to 255 (inclusive) are reserved for future use by ITU-T|ISO / IEC and must not exist in a bitstream conforming to this version of this document. Decoders conforming to this version of this document must ignore SPTI·SEI messages in the range of spti_source_picture_timing_type from 128 to 255 (inclusive).

[0099] [Table 11]

[0100] If picture n is an output picture and is not the first picture in the output bitstream, the value of the variable SourcePictureTime[n] is derived as follows: - If (spti_source_picture_timing_type & bitMask) is equal to 0, SourcePictureTime[n] = SourcePictureTime[previousPicInOutputOrder] + SourcePictureInterval (If spti_source_picture_timing_type & bitMask is equal to 1, SourcePictureTime[n] = SourcePictureTime[previousPicInOutputOrder] - SourcePictureInterval

[0101] Here, previousPicInOutputOrder is the last picture (if any) that is output before picture n in the output order. If the value of SourcePictureTime[0] is not provided by an external means not specified in this document, the value of SourcePictureTime[0] is assumed to be 0.

[0102] Appearance 10 The type of relationship between the timing of the source picture and the timing of the decoded output picture can be affirmatively defined as being the same by the spti_source_timing_equals_output_timing_flag. Differences from Embodiment 5 are indicated in italics.

[0103] [Table 12]

[0104] Updated semantics If spti_source_timing_equals_output_timing_flag is equal to 1, it indicates that the timing of the source picture is the same as the timing of the corresponding decoded output picture. If spti_source_timing_equals_output_timing_flag is equal to 0, it indicates that the timing of the source picture may not be the same as the timing of the corresponding decoded output picture.

[0105] If spti_source_timing_equals_output_timing_flag is equal to 1, and a picture timing SEI message exists for the current picture, the timing of the source picture can be determined from the information transmitted in the picture timing SEI message.

[0106] Appearance 11 Discrepancies in the timing of source pictures, such as scene cuts and splices, are indicated by spti_source_timing_discontinuity_flag or spti_source_timing_discontinuity_type. Differences from Embodiment 5 are indicated in italics.

[0107] [Table 13]

[0108] Updated semantics If spti_source_timing_discontinuity_flag is equal to 1, it indicates that the timing of the source picture corresponding to the decoded output picture is discontinuous. If spti_source_timing_equals_output_timing_flag is equal to 0, it does not indicate this. Note: Examples of source picture timing discontinuities include scene cuts, splices, and event-triggered security footage.

[0109] Appearance 12 Discrepancies in the timing of source pictures, such as scene cuts and splices, are indicated by `spti_source_timing_discontinuity_type` and further defined by the `spti_source_timing_discontinuity_type` and `spti_source_transition_type` syntax elements. Differences from Embodiment 11 are shown in italics.

[0110] [Table 14]

[0111] Updated semantics If spti_source_timing_discontinuity_type exists, a value equal to 1 indicates that the timing of the source picture corresponding to the decoded output picture is discontinuous, as specified in Table 12axx. If it does not exist, the value of spti_source_timing_discontinuity_type is presumed to be equal to 0.

[0112] [Table 15]

[0113] `spti_source_transition_type`, if present, indicates the type of scene transition applied to the source picture corresponding to the decoded output picture, as defined in Table 12bxx. Here, if (spti_source_transition_type & bitMask) is not equal to 0, it indicates that the scene transition relationship has an interpretation associated with the bitMask value in Table 12bxx. If `spti_source_transition_type` is greater than 0 and (spti_source_transition_type & bitMask) is equal to 0, the interpretation associated with the bitMask value does not apply to SPTI. If `spti_source_transition_type` is equal to 0, the scene transition may be defined by the application. If it is not present, `spti_source_transition_type` is presumed to be equal to 0.

[0114] The value of spti_source_transition_type must be in the range of 0 to 127 (inclusive) in a bitstream conforming to this version of this document. Values ​​of spti_source_transition_type in the range of 128 to 255 (inclusive) are reserved for future use by ITU-T|ISO / IEC and must not exist in a bitstream conforming to this version of this document. Decoders conforming to this version of this document must ignore SPTI·SEI messages in the range of spti_source_transition_type from 128 to 255 (inclusive).

[0115] [Table 16]

[0116] Considerations regarding the duration of SEI messages Option 1: SEI messages persist throughout the entire CLVS.

[0117] [Table 17]

[0118] Semantics If a source picture timing information SEI message exists in any picture of a CLVS of a particular layer, then a source picture timing information SEI message must also exist in the first picture of that CLVS. The source picture timing information SEI message persists for the current layer from the current picture to the end of the CLVS in the decoding order. All source picture timing information SEI messages applicable to the same CLVS must have identical content.

[0119] Option 2: The duration of the SEI message is determined by the message syntax. [Table 18]

[0120] Semantics If spti_cancel_flag is equal to 1, the SPTI·SEI message indicates that the persistence state of the previous SPTI·SEI message in output order is canceled, which applies to the current layer. If spti_cancel_flag is equal to 0, it indicates that SPTI will continue.

[0121] The spti_persistence_flag specifies the persistence state of SPTI·SEI messages for the current layer.

[0122] If spti_persistence_flag is equal to 0, the SPTI·SEI message applies only to the current decrypted picture.

[0123] If spti_persistence_flag is equal to 1, the SPTI·SEI message is applied to the current decoded picture and persists for all subsequent pictures of the current layer in output order until one or more of the following conditions are met: -A new CLVS for the current layer will begin. - The bitstream ends. -The image in the current layer within the AU associated with the SPTI·SEI message will be output as the next picture in the output order.

[0124] In another embodiment, Table 13 shows another example of the proposed SPTI message.

[0125] [Table 19]

[0126] Compared to the previous embodiment, Table 13 includes the following changes. The spti_num_units_in_elemental_interval has been changed from 32 bits to 18 unsigned bits to save bits. Table 13axx below is an extension of the aforementioned Table 9axx, adding clarity to various descriptions such as "slow motion." Furthermore, new semantic constraints have been added to prevent mutually exclusive timing relationships between the source picture and the corresponding decoded output picture. For example, this semantics prevents the combination of "high-speed shooting" and "time-lapse shooting."

[0127] spti_source_timing_equals_output_timing_flag (defined in aspect 10) has been replaced with spti_source_timing_info_present_flag. The spti_source_timing_info_present_flag (defined in aspect 9) is set to spti_source_type.

[0128] The flag spti_source_type_present_flag has been added and is defined as follows: If spti_source_type_present_flag is equal to 1, it indicates that the syntactic element spti_source_type is present in the SEI message. If spti_source_type_present_flag is equal to 0, it indicates that the syntactic element spti_source_type is not present in the SEI message.

[0129] For completeness, the semantics of the renamed flag are as follows: If spti_source_type exists, it indicates the timing relationship between the source picture and the corresponding decoded output picture, as defined in Table 13axx below. Here, if (spti_source_type & bitMask) is not equal to 0, it indicates that the timing relationship has an interpretation associated with the bitMask value in the corresponding row of Table 13axx. If spti_source_type is greater than 0 and (spti_source_type & bitMask) is equal to 0, the interpretation associated with the bitMask value does not apply to the SPTI·SEI message. If it does not exist, the value of spti_source_picture_type is assumed to be equal to 0. If spti_source_type is equal to 0, the timing relationship may be defined by the application.

[0130] The value of spti_source_type must be in the range of 0 to 127 (inclusive) in a bitstream conforming to this version of this document. A decoder conforming to this version of this document must ignore SPTI·SEI messages when spti_source_picture_timing_type is in the range of 128 to 65535 (inclusive).

[0131] [Table 20]

[0132] The value of (spti_source_type & 0x04) & (spti_source_type & 0x08) is set to zero (i.e., spti_source_type cannot simultaneously indicate high-speed shooting and time-lapse shooting).

[0133] The variable temporalReversalFlag is equal to (spti_source_type & 0x10)?1:0.

[0134] Alternatively, the variable temporalReversalFactor (spti_source_type & 0x10)? can also be defined as equal to -1:1, in which case the corresponding change to equation (8-X) is as follows:

[0135] If spti_source_timing_info_present_flag is equal to 1, it indicates that the syntactic elements spti_time_scale, spti_num_units_in_elemental_interval, spti_max_sublayers_minus_1, spti_sublayer_interval_scale_factor[i], and spti_sublayer_synthesized_picture_flag[i] are present in the SEI message. If spti_source_timing_info_present_flag is equal to 0, it indicates that these syntactic elements are not present in the SEI message.

[0136] (Note: spti_num_units_in_elemental_interval is a simplified name for spti_num_units_in_elemental_source_picture_interval, which was defined earlier in aspect 2.)

[0137] If spti_sublayer_interval_scale_factor[i] exists, it specifies the scale factor used to determine the source-picture interval of corresponding consecutive pictures in the output order in CLVS, for which TemporalId less than or equal to i. A value of 0 may be used to indicate that the source picture corresponding to the currently decoded output picture is the same as the source picture corresponding to the previously decoded output picture. If (spti_source_type & 0x04) is equal to 1, the value of spti_sublayer_interval_scale_factor[i] should be 0.

[0138] The indicated source picture interval in seconds associated with an output picture having a TemporalId less than or equal to i, represented by the variable SourcePictureInterval[i], is derived as follows: SourcePictureInterval[i] = ElementalSourcePictureInterval * spti_sublayer_interval_scale_factor[i] * (1- 2 * temporalReversalFlag) (8-X)

[0139] (Note: spti_sublayer_interval_scale_factor[i] is a simplified name for spti_sublayer_source_picture_interval_scale_factor[i] defined earlier.)

[0140] When using the temporalReversalFactor variable, the expression (8-X) is modified as follows: SourcePictureInterval[i] = ElementalSourcePictureInterval * spti_sublayer_interval_scale_factor[i] * temporalReversalFactor (8-X)

[0141] References Each reference cited herein is incorporated herein by reference in its entirety. JVET refers to the Joint Video Experts Team of ITU-TSG 16 WP 3 and ISO / IECJTC 1 / SC 29. [1] “Versatile Video Coding,” Rec. ITU-TH.266, August 2020. [2] HB Teo, et al., “AHG9: Alternative Picture Timing SEI”, JVET-AC0141, 29th Meeting, by teleconference, 11-20 January 2023. [3] HB Teo, et al., “AHG9: Alternative Output Timing Hint SEI”, JVET-AD0161, 30th Meeting, Antalya, Turkey, 21-28 April 2023. [4] B. Chen, et al., “AHG9: Generative Face Video SEIMessage,” JVET-AC0088, by teleconference, 11-20 January 2023. [5] B. Chen, et al., “AHG9: Common SEIMessage of Generative Face Video”, JVET-AD0051, 30th Meeting, Antalya, Turkey, 21-28 April 2023. [6] Phantom / AMETEK T4040 camera main Web Page, accessed June 24, 2023, https: / / www.phantomhighspeed.com / products / cameras / tseries / t4040.

[0142] Computer system implementation examples Embodiments of the present invention may be implemented by computer systems, systems composed of electronic circuits and components, integrated circuit (IC) devices such as microcontrollers, field-programmable gate arrays (FPGAs), other configurable or programmable logic devices (PLDs), discrete-time or digital signal processors (DSPs), application-specific ICs (ASICs), and / or devices comprising one or more of these systems, devices, or components. Computers and / or ICs may perform, control, or execute instructions (e.g., those described herein) related to the signaling of source picture timing information in image and video coding. Computers and / or ICs may calculate any of the various parameters or values ​​related to the signaling of source picture timing information in image and video coding as described herein. Embodiments of images and videos may be implemented in hardware, software, firmware, and various combinations thereof.

[0143] Some implementations of the present invention include a computer processor that executes software instructions to cause the processor to perform the method of the present invention. For example, one or more processors in a display, encoder, set-top box, transcoder, etc., can implement the method relating to the signal transmission of source picture timing information in image and video coding as described above by executing software instructions in program memory accessible to the processor. Embodiments of the present invention may also be provided in the form of a program product. A program product may consist of any non-temporary and tangible medium that holds a set of computer-readable signals, which, when executed by a data processor, causes the data processor to perform the method of the present invention. A program product according to the present invention can be any of a wide variety of non-temporary and tangible forms. A program product may include, for example, a magnetic data storage medium such as a floppy disk or hard disk drive, an optical data storage medium such as a CD-ROM or DVD, or an electronic data storage medium such as ROM or flash RAM. The computer-readable signals on the program product may optionally be compressed or encrypted.

[0144] Where a component (e.g., a software module, processor, assembly, device, circuit, etc.) is mentioned above, unless otherwise specified, any reference to that component (including any reference to the “means”) should be interpreted as including any component that performs the function of the described component (e.g., a functionally equivalent component), including any component that is not structurally equivalent to the disclosed structure that performs that function in the illustrated embodiment of the invention.

[0145] Equivalents, extensions, substitutes, and others The above describes examples of embodiments relating to the signal transmission of source picture timing information in image and video encoding. The above specification describes embodiments of the present invention with reference to numerous specific details that may differ from implementation to implementation. Therefore, the sole exclusive indicator of what constitutes an invention and what the applicant intends to invent is the set of claims granted by this application, in the particular form (including subsequent amendments) in which such claims are granted. The definitions expressly provided herein for terms contained in such claims define the meaning of such terms as used in those claims. Therefore, no limitations, elements, characteristics, features, advantages, or attributes not expressly stated in the claims shall in any way limit the scope of those claims. The specification and drawings should therefore be interpreted as illustrative, not restrictive.

[0146] Various aspects of this disclosure can be understood from the following Enumerated Example Embodiments (EEE):

[0147] EEE1. A method for decoding a video bitstream, Receiving an encoded video bitstream, which includes an encoded picture section having an encoded sequence of video pictures and a signal transmission section including a group of source picture timing parameters, wherein the group of source picture timing parameters is The source picture time scale parameter indicates the number of time units elapsed per second, It includes a source-picture number-of-units-in-source-picture interval parameter that indicates the number of time units of a clock operating at the frequency of the aforementioned source-picture time scale parameter, Decoding the sequence of the video picture based on the source picture timing parameters, Methods that include...

[0148] Eez2. The method according to EEE1, further comprising calculating the SourcePictureInterval value as the quotient obtained by dividing the number of SourcePictureInterval units in the SourcePicture by the SourcePicture timescale.

[0149] EEE3. A method for decoding a video bitstream, Receiving an encoded video bitstream, which includes an encoded picture section containing an encoded sequence of video pictures and a signal transmission section containing a set of source picture timing (SPT) parameters, wherein the set of source picture timing parameters is: The source picture time scale parameter indicates the number of time units elapsed per second, A source-picture number-of-units-in-elemental-source-picture interval parameter indicates the number of time units of a clock operating at the frequency of the source-picture time scale parameter, corresponding to the indicated elemental source-picture interval for consecutive output pictures, This includes a source picture interval scale factor parameter that defines the scale factor, Decoding the sequence of the video picture based on the source picture timing parameters, Methods that include...

[0150] EEE4. The SourcePictureInterval value is further calculated as the product of the elemental source picture interval multiplied by the source picture interval scale coefficient, wherein the elemental source picture interval is calculated as the quotient obtained by dividing the elemental source picture interval unit number parameter in the source picture by the source picture time scale parameter. Methods described in EEE3.

[0151] EEE5. The source picture time for picture n is SourcePictureTime[n] = SourcePictureTime[previousPicInOutputOrder] + SourcePictureInterval This further includes calculating it as follows: Picture n (n>0) represents the index of an output picture in the video bitstream that is not the first output picture, and the variable previousPicInOutputOrder represents the last picture (if any) that is output before picture n in the output order. The method described in EEE2 or 4.

[0152] EEE6. The source picture timing parameter group further includes a source picture sublayer maximum time ID that identifies the maximum sublayer to which the SPT parameter group applies. Methods described in EEE3.

[0153] EEE7. The aforementioned source picture timing parameters are: The source picture time sublayer ID to which the SPT parameter set is applied, and Source picture sublayer delay coefficient, which defines the scale factor used when determining the temporal distance between a source picture corresponding to the first decoded output picture of the time sublayer having a TemporalId value equal to 0 and a source picture corresponding to the first decoded output picture of the time sublayer having a TemporalId value equal to the time sublayer ID of the said source picture. Further including, Methods described in EEE3.

[0154] Eee8. The source picture time for picture n is SourcePictureTime[n] = SourcePictureTime[previousPicInOutputOrder] + SourcePictureInterval +SublayerSourcePictureDelay This further includes calculating it as follows: Picture n (n>0) represents the index of an output picture in the video bitstream that is not the first output picture, the variable previousPicInOutputOrder represents the last picture (if any) that is output before picture n in the output order, and SublayerSourcePictureDelay represents the source picture sublayer delay coefficient. Methods for EEE7.

[0155] EEE9. The method according to EEE3, wherein the source picture interval scale coefficient parameter is defined for each of the N sublayers, and N > 0.

[0156] EEE10. The method according to EEE9, wherein the source picture timing parameter group further includes a source picture sublayer binary flag, and when the source picture sublayer binary flag is set to 1, it indicates that the time sublayer is encoded in a binary relation and that there is no source picture interval scale coefficient parameter defining a scale coefficient syntactic element.

[0157] EEE11. The method according to EEE9, wherein the source picture timing parameter group further includes a source picture sublayer implicit timing flag, and when the source picture sublayer implicit timing flag is set to 1, it indicates that sublayer implicit timing type information exists for each of the sublayers.

[0158] EEE12. The method according to EEE9, wherein the source picture timing parameter group further includes a source picture sublayer composite picture flag for each of the N sublayers, and when the source picture sublayer composite picture flag is set to 1, it indicates that the decoded output picture belonging to the i-th time sublayer is composited and does not correspond to the original source picture that has not been modified.

[0159] EEE13. The method according to EEE9, wherein the source picture timing parameter group further includes source picture timing type parameters, and the source picture timing type parameters indicate the timing relationship between the source picture and the corresponding decoded output picture using a mapping table.

[0160] EEE14. The method according to EEE9, wherein the source picture timing parameter group further includes a source picture timing equals output timing flag, and when the source picture timing equals output timing flag is set to 1, it indicates that the timing of the source picture is the same as the timing of the corresponding decoded output picture.

[0161] EEE15. The method according to EEE3, wherein the source picture interval scale factor parameter is defined using the absolute magnitude value of the scale factor and the sign flag of the scale factor.

[0162] EEE16. The method according to EEE9, wherein the source picture timing parameter group further includes a source picture timing discontinuity flag, and when the source picture timing discontinuity flag is set to 1, it indicates that the timing of the source picture corresponding to the decoded output picture is discontinuous.

[0163] EEE17. The method according to EEE16, wherein the source picture timing parameter group further includes a source picture timing discontinuity type parameter and a source picture transition type parameter, the source picture timing discontinuity type parameter indicates discontinuities in the source picture by a first table, and the source picture transition type parameter indicates the transition type of the source picture by a second table.

[0164] EEE18. The method according to EEE4, wherein the source picture timing parameter group further includes source type parameters, and the source type parameters indicate the timing relationship between the source picture and the corresponding decoded output picture.

[0165] EEE19. The method according to EEE18, wherein the source type parameter may indicate one or more of the following: slow motion, fast forward, high-speed shooting, time-lapse shooting, time reversal, still image, or sporadic shooting.

[0166] EEE20. The method according to EEE19, wherein the aforementioned source type parameter does not indicate both high-speed and time-lapse photography.

[0167] EEE21. The method according to EEE18, wherein, if the aforementioned source type parameter exists, the SourcePictureInterval value is further adjusted by a function of the source type value.

[0168] Eez22. The method according to any one of EEE1 to 21, wherein the signal transmission section includes a supplemental enhancement information (SEI) messaging section or a video user information (VUI) messaging section.

[0169] EEE23. A tangible, computer-readable storage medium containing computer-executable instructions for performing any one of the methods described in EEE1 to 21 by one or more processors.

[0170] EEE24. A device equipped with a processor and configured to perform any one of the methods described in EEE1 to 21.

Claims

1. A method for decoding a video bitstream, Receiving an encoded video bitstream, which includes an encoded picture section having an encoded sequence of video pictures and a signal transmission section including a group of source picture timing parameters, wherein the group of source picture timing parameters is The source picture time scale parameter indicates the number of time units that elapse per second, It includes a source-picture number-of-units-in-source-picture interval parameter that indicates the number of time units of a clock operating at the frequency of the aforementioned source-picture time scale parameter, Decoding the sequence of the video picture based on the source picture timing parameters, Methods that include...

2. The method according to claim 1, further comprising calculating the SourcePictureInterval value as the quotient obtained by dividing the number of SourcePictureInterval units in the SourcePicture by the SourcePicture timescale.

3. A method for decoding a video bitstream, Receiving an encoded video bitstream, which includes an encoded picture section containing an encoded sequence of video pictures and a signal transmission section containing a set of source picture timing (SPT) parameters, wherein the set of source picture timing parameters is: The source picture time scale parameter indicates the number of time units that elapse per second, A source-picture number-of-units-in-elemental-source-picture interval parameter indicates the number of time units of a clock operating at the frequency of the source-picture time scale parameter, corresponding to the indicated elemental source-picture interval for consecutive output pictures, This includes a source picture interval scale factor parameter that defines the scale factor, Decoding the sequence of the video picture based on the source picture timing parameters, Methods that include...

4. The method according to claim 3, further comprising calculating the SourcePictureInterval value as the product of multiplying the elemental source picture interval by the SourcePictureInterval scale coefficient, wherein the elemental source picture interval is calculated as the quotient obtained by dividing the elemental source picture interval unit number parameter in the source picture by the SourcePicture time scale parameter.

5. The source picture time for picture n, SourcePictureTime[n] = SourcePictureTime[previousPicInOutputOrder] + SourcePictureInterval This further includes calculating it as follows: The method according to claim 2 or 4, wherein picture n (n > 0) represents the index of an output picture in the video bitstream that is not the first output picture, and the variable previousPicInOutputOrder represents the last picture (if any) that is output before picture n in the output order.

6. The method according to claim 3, wherein the source picture timing parameter group further includes a source picture sublayer maximum time ID that identifies the maximum sublayer to which the SPT parameter group is applied.

7. The aforementioned source picture timing parameters are: The source picture time sublayer ID to which the SPT parameter set is applied, and Source picture sublayer delay coefficient, which defines the scale factor used when determining the temporal distance between a source picture corresponding to the first decoded output picture of the time sublayer having a TemporalId value equal to 0 and a source picture corresponding to the first decoded output picture of the time sublayer having a TemporalID value equal to the time sublayer ID of the said source picture. The method according to claim 3, further comprising:

8. The source picture time for picture n, SourcePictureTime[n] = SourcePictureTime[previousPicInOutputOrder] + SourcePictureInterval +SublayerSourcePictureDelay This further includes calculating it as follows: The method according to claim 7, wherein picture n (n > 0) represents the index of an output picture in the video bitstream that is not the first output picture, the variable previousPicInOutputOrder represents the last picture (if any) that is output before picture n in the output order, and SublayerSourcePictureDelay represents the source picture sublayer delay coefficient.

9. The method according to claim 3, wherein the source picture interval scale coefficient parameter is defined for each of the N sublayers, and N > 0.

10. The method according to claim 9, wherein the source picture timing parameter group further includes a source picture sublayer binary flag, and when the source picture sublayer binary flag is set to 1, it indicates that the time sublayer is encoded in a binary relation and that there is no source picture interval scale coefficient parameter defining a scale coefficient syntactic element.

11. The method according to claim 9, wherein the source picture timing parameter group further includes a source picture sublayer implicit timing flag, and when the source picture sublayer implicit timing flag is set to 1, it indicates that sublayer implicit timing type information exists for each of the sublayers.

12. The method according to claim 9, wherein the source picture timing parameter group further includes a source picture sublayer composite picture flag for each of the N sublayers, and when the source picture sublayer composite picture flag is set to 1, it indicates that the decoded output picture belonging to the i-th time sublayer is composited and does not correspond to the original unmodified source picture.

13. The method according to claim 9, wherein the source picture timing parameter group further includes source picture timing type parameters, the source picture timing type parameters indicate the timing relationship between a source picture and a corresponding decoded output picture using a mapping table.

14. The method according to claim 9, wherein the source picture timing parameter group further includes a source picture timing equals output timing flag, and when the source picture timing equals output timing flag is set to 1, it indicates that the timing of the source picture is the same as the timing of the corresponding decoded output picture.

15. The method according to claim 3, wherein the source picture interval scale factor parameter is defined using the absolute magnitude value of the scale factor and the sign flag of the scale factor.

16. The method according to claim 9, wherein the source picture timing parameter group further includes a source picture timing discontinuity flag, and when the source picture timing discontinuity flag is set to 1, it indicates that the timing of the source picture corresponding to the decoded output picture is discontinuous.

17. The method according to claim 16, wherein the source picture timing parameter group further includes a source picture timing discontinuity type parameter and a source picture transition type parameter, the source picture timing discontinuity type parameter indicates discontinuities in the source picture by a first table, and the source picture transition type parameter indicates the transition type of the source picture by a second table.

18. The method according to claim 4, wherein the source picture timing parameter group further includes source type parameters, the source type parameters indicate the timing relationship between a source picture and a corresponding decoded output picture.

19. The method according to claim 18, wherein the source type parameter may represent one or more of slow motion, fast forward, high-speed shooting, time-lapse shooting, time reversal, still image, or sporadic shooting.

20. The method according to claim 19, wherein the source type parameter does not indicate both high-speed shooting and time-lapse shooting.

21. The method according to claim 18, wherein, if the source type parameter exists, the source picture interval (SourcePictureInterval) value is further adjusted as a function of the source type value.

22. The method according to any one of claims 1 to 21, wherein the signal transmission section includes a supplemental enhancement information (SEI) messaging section or a video user information (VUI) messaging section.

23. A tangible, computer-readable storage medium storing computer-executable instructions for performing a method according to any one of the methods described in claims 1 to 21, using one or more processors.

24. An apparatus comprising a processor and configured to perform any one of the methods according to claims 1 to 21.

Images