Method for reference picture processing in video coding

Abstract

To provide a computer-implemented method for video processing.SOLUTION: The method includes deriving a total number by summing a number of reference picture list structures in a sequence parameter set (SPS) and one; allocating memory for the total number of reference picture list structures in response to a reference picture list structure being signaled in a picture header of a current picture or a slice header of a current slice; and processing a current picture or a current slice using the allocated memory.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This disclosure claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 028,509, filed May 21, 2020, the entire contents of which are incorporated by reference herein.

[0002] Technical Field

[0002] The present disclosure relates generally to video processing, and more particularly to a method, apparatus, and non-transitory computer-readable storage medium for processing reference pictures. [Background technology]

[0003] background

[0003] A video is a set of static pictures (or "frames") that capture visual information. To reduce storage memory and transmission bandwidth, a video can be compressed before storage or transmission and decompressed before display. The compression process is typically referred to as encoding, and the decompression process is typically referred to as decoding. There are various video coding formats that use standardized video coding techniques, most commonly based on prediction, transform, quantization, entropy coding, and in-loop filtering. Standardization organizations have developed video coding standards that specify specific video coding formats, such as the High Efficiency Video Coding (HEVC / H.265) standard, the Versatile Video Coding (VVC / H.266) standard, and the AVS standard. As more and more advanced video coding techniques are adopted into video standards, the coding efficiency of new video coding standards becomes increasingly higher. Summary of the Invention [Means for solving the problem]

[0004] Disclosure Overview

[0004] Embodiments of the present disclosure provide a method for video processing. In some embodiments, the method includes deriving a total number by summing the number of reference picture list structures in a sequence parameter set (SPS) and 1, allocating memory for the total number of reference picture list structures in response to the reference picture list structures being signaled in a picture header of a current picture or a slice header of a current slice, and processing the current picture or the current slice using the allocated memory.

[0005] In some embodiments, the method includes signaling a first flag in a Picture Parameter Set (PPS) to indicate whether there is a second flag and a first index in a picture header syntax or slice header of the current picture referencing the PPS, the second flag indicating whether reference picture list 1 is derived based on one of reference picture list structures associated with reference picture list 1 signaled in a Sequence Parameter Set (SPS), the first index being an index of a reference picture list structure associated with reference picture list 1 used to derive reference picture list 1 to a list of reference picture list structures associated with reference picture list 1 contained in the SPS; determining whether to signal the first index and the second index, the second index being an index of a reference picture list structure associated with reference picture list 0 used to derive reference picture list 0 to a reference picture list structure associated with reference picture list 0 contained in the SPS. the first index is an index to a list of reference picture list structures associated with reference picture list 0; determining a value of the second index in response to the second index not being signaled, wherein determining the value of the second index includes determining the value of the second index to be equal to 0 if at most one reference picture list structure associated with reference picture list 1 is included in the SPS; determining a value of the first index in response to the first index not being signaled, wherein determining the value of the first index includes determining the value of the first index to be equal to 0 if at most one reference picture list structure associated with reference picture list 1 is included in the SPS; and determining the value of the first index to be equal to the value of the second index if the first flag is equal to 0 and the second flag is equal to 1; deriving a reference picture list based on the first index and the second index; and encoding the current picture based on the reference picture list.

[0006]

[0006] In some embodiments, the method includes receiving a video bitstream; determining a value of a first flag indicating whether there is a second flag and a first index in a picture header syntax or slice header of a current picture, wherein the second flag indicates whether reference picture list 1 is derived based on one of reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index is an index to a list of reference picture list structures associated with reference picture list 1 contained in the SPS of the reference picture list structure associated with reference picture list 1 used to derive reference picture list 1; determining whether there is a first index and a second index, wherein the second index is an index to a list of reference picture list structures associated with reference picture list 0 contained in the SPS of the reference picture list structure associated with reference picture list 0 used to derive reference picture list 0. the first index is an index to a list of reference picture list structures associated with picture list 0; determining a value of the second index in response to the second index being absent, wherein determining the value of the second index includes determining the value of the second index to be equal to 0 if at most one reference picture list structure associated with reference picture list 0 is included in the SPS; determining a value of the first index in response to the first index being absent, wherein determining the value of the first index includes determining the value of the first index to be equal to 0 if at most one reference picture list structure associated with reference picture list 1 is included in the SPS; and determining the value of the first index to be equal to the value of the second index if the first flag is equal to 0 and the second flag is equal to 1; decoding the current picture based on the first index and the second index.

[0007]

[0007] In some embodiments, the method includes signaling a first flag in the slice header to indicate whether there is an active reference index number in the slice header, the active reference index number being used to derive a maximum reference index of a corresponding reference picture list used to encode the current slice; and in response to the first flag indicating the presence of the active reference index number in the slice header: determining the number of entries in reference picture list 0, and signaling the number of active reference indexes of reference picture list 0 in slice headers for P slices and B slices if the number of entries in reference picture list 0 is greater than 1; and Determining the number of entries in Reference Picture List 1 and signaling the number of active reference indexes in Reference Picture List 1 in a slice header for the B slice if the number of entries in Reference Picture List 1 is greater than one.

[0008]

[0008] In some embodiments, the method includes receiving a video bitstream including a slice header and a picture header syntax; determining a value of a first flag signaled in the slice header indicating whether there is an active reference index number in the slice header, wherein the active reference index number is used to derive a maximum reference index of a corresponding reference picture list used to decode the current slice; in response to the first flag indicating that there is an active reference index number, determining a number of entries in reference picture list 0 and decoding the active reference index number of reference picture list 0 in the slice header for the P slice and the B slice if the number of entries in reference picture list 0 is greater than 1; and determining a number of entries in reference picture list 1 and decoding the active reference index number of reference picture list 1 in the slice header for the B slice if the number of entries in reference picture list 1 is greater than 1.

[0009]

[0009] In some embodiments, the method includes determining a co-located picture referenced by a reference index of the co-located picture within a slice level, where the co-located picture is determined to be the same picture for all non-I slices of the current picture, and processing the current picture based on the co-located picture, where the co-located picture is used for temporal motion vector prediction.

[0010]

[0010] Embodiments of the present disclosure provide an apparatus for performing video processing. In some embodiments, the apparatus includes a memory configured to store instructions and one or more processors configured to execute the instructions to cause the apparatus to: derive a total number by summing a number of reference picture list structures in a sequence parameter set (SPS) and 1; allocate memory for the total number of reference picture list structures in response to the reference picture list structures being signaled in a picture header of a current picture or a slice header of a current slice; and process the current picture or the current slice using the allocated memory.

[0011]

[0011] In some embodiments, the apparatus includes a memory configured to store instructions and one or more processors, the one or more processors being configured to: signal a first flag in a picture parameter set (PPS) to indicate whether a second flag and a first index are present in a picture header syntax or slice header of the current picture referencing the PPS, the second flag indicating whether reference picture list 1 is derived based on one of reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), the first index being an index of a reference picture list structure associated with reference picture list 1 used to derive reference picture list 1 to a list of reference picture list structures associated with reference picture list 1 included in the SPS; determine whether to signal the first index and the second index, the second index being an index of a reference picture list structure associated with reference picture list 0 used to derive reference picture list 0 included in the SPS. the first index is an index to a list of reference picture list structures associated with reference picture list 0; in response to the second index not being signaled, determining a value of the second index, where determining the value of the second index includes determining the value of the second index to be equal to 0 if at most one reference picture list structure associated with reference picture list 0 is included in the SPS; in response to the first index not being signaled, determining the value of the first index includes determining the value of the first index to be equal to 0 if at most one reference picture list structure associated with reference picture list 1 is included in the SPS, and determining the value of the first index to be equal to the value of the second index if the first flag is equal to 0 and the second flag is equal to 1; deriving a reference picture list based on the first index and the second index; and encoding the current picture based on the reference picture list.

[0012]

[0012] In some embodiments, the apparatus includes a memory configured to store instructions and one or more processors, the one or more processors receiving a video bitstream; determining a value of a first flag indicating whether there is a second flag and a first index in a picture header syntax or slice header of a current picture, the second flag indicating whether reference picture list 1 is derived based on one of reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), the first index being an index of a reference picture list structure associated with reference picture list 1 used to derive reference picture list 1 to a list of reference picture list structures associated with reference picture list 1 included in the SPS; determining whether there is a first index and a second index, the second index being an index of a reference picture list structure associated with reference picture list 0 used to derive reference picture list 0 , an index to a list of reference picture list structures associated with reference picture list 0 included in the SPS; determining a value of the second index in response to the absence of the second index, wherein determining the value of the second index includes determining the value of the second index to be equal to 0 if at most one reference picture list structure associated with reference picture list 0 is included in the SPS; determining a value of the first index in response to the absence of the first index, wherein determining the value of the first index includes determining the value of the first index to be equal to 0 if at most one reference picture list structure associated with reference picture list 1 is included in the SPS, and determining the value of the first index to be equal to the value of the second index if the first flag is equal to 0 and the second flag is equal to 1; and decoding the current picture based on the first index and the second index.

[0013]

[0013] In some embodiments, the apparatus includes a memory configured to store instructions and one or more processors, wherein the one or more processors signal a first flag in the slice header to indicate whether there is an active reference index number in the slice header, wherein the active reference index number is used to derive a maximum reference index of a corresponding reference picture list used to encode the current slice; and in response to the first flag indicating there is an active reference index number in the slice header, determining the number of entries in reference picture list 0, and signaling the number of active reference indexes of reference picture list 0 in slice headers for P slices and B slices if the number of entries in reference picture list 0 is greater than 1; and The device is configured to execute instructions to cause the device to determine the number of entries in reference picture list 1, and if the number of entries in reference picture list 1 is greater than 1, signal the number of active reference indexes in reference picture list 1 in a slice header for a B slice.

[0014]

[0014] In some embodiments, the apparatus includes a memory configured to store instructions and one or more processors, the one or more processors configured to execute the instructions to cause the apparatus to receive a video bitstream including a slice header and a picture header syntax; determine a value of a first flag signaled in the slice header indicating whether there is an active reference index number in the slice header, wherein the active reference index number is used to derive a maximum reference index of a corresponding reference picture list used to decode the current slice; in response to the first flag indicating that there is an active reference index number, determine the number of entries in reference picture list 0 and decode the active reference index number of reference picture list 0 in the slice header for the P slice and the B slice if the number of entries in reference picture list 0 is greater than 1; and determine the number of entries in reference picture list 1 and decode the active reference index number of reference picture list 1 in the slice header for the B slice if the number of entries in reference picture list 1 is greater than 1.

[0015]

[0015] In some embodiments, the device includes a memory configured to store instructions and one or more processors, the one or more processors configured to execute the instructions to cause the device to determine a co-located picture referenced by a reference index of the co-located picture within a slice level, where the co-located picture is determined to be the same picture for all non-I slices of the current picture, and process the current picture based on the co-located picture, where the co-located picture is used for temporal motion vector prediction.

[0016]

[0016] Embodiments of the present disclosure provide a non-transitory computer-readable storage medium storing a set of instructions. The set of instructions is executable by one or more processors of the device to cause the device to initiate a method for video processing. In some embodiments, the method includes deriving a total number by summing the number of reference picture list structures in a sequence parameter set (SPS) and 1, allocating memory for the total number of reference picture list structures in response to the reference picture list structures being signaled in a picture header of a current picture or a slice header of a current slice, and processing the current picture or the current slice using the allocated memory.

[0017]

[0017] In some embodiments, the method includes signaling a first flag in a Picture Parameter Set (PPS) to indicate whether there is a second flag and a first index in a picture header syntax or slice header of the current picture referencing the PPS, the second flag indicating whether reference picture list 1 is derived based on one of reference picture list structures associated with reference picture list 1 signaled in a Sequence Parameter Set (SPS), the first index being an index of a reference picture list structure associated with reference picture list 1 used to derive reference picture list 1 to a list of reference picture list structures associated with reference picture list 1 contained in the SPS; determining whether to signal the first index and the second index, the second index being an index of a reference picture list structure associated with reference picture list 0 used to derive reference picture list 0 to a list of reference picture list structures associated with reference picture list 0 contained in the SPS. the first index is an index to a list of reference picture list structures associated with reference picture list 0; determining a value of the second index in response to the second index not being signaled, wherein determining the value of the second index includes determining the value of the second index to be equal to 0 if at most one reference picture list structure associated with reference picture list 1 is included in the SPS; determining a value of the first index in response to the first index not being signaled, wherein determining the value of the first index includes determining the value of the first index to be equal to 0 if at most one reference picture list structure associated with reference picture list 1 is included in the SPS, and determining the value of the first index to be equal to the value of the second index if the first flag is equal to 0 and the second flag is equal to 1; deriving a reference picture list based on the first index and the second index; and encoding the current picture based on the reference picture list.

[0018]

[0018] In some embodiments, the method includes receiving a video bitstream; determining a value of a first flag indicating whether a second flag and a first index are present in a picture header syntax or slice header of a current picture, the second flag indicating whether reference picture list 1 is derived based on one of reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), the first index being an index of the reference picture list structure associated with reference picture list 1 used to derive reference picture list 1 to a list of reference picture list structures associated with reference picture list 1 contained in the SPS; determining whether a first index and a second index are present, the second index being an index of the reference picture list structure associated with reference picture list 0 used to derive reference picture list 0 contained in the SPS. the first index being an index to a list of reference picture list structures associated with reference picture list 0; determining a value of the second index in response to the second index being absent, wherein determining the value of the second index includes determining the value of the second index to be equal to 0 if at most one reference picture list structure associated with reference picture list 0 is included in the SPS; determining a value of the first index in response to the first index being absent, wherein determining the value of the first index includes determining the value of the first index to be equal to 0 if at most one reference picture list structure associated with reference picture list 1 is included in the SPS; and determining the value of the first index to be equal to the value of the second index if the first flag is equal to 0 and the second flag is equal to 1; decoding the current picture based on the first index and the second index.

[0019]

[0019] In some embodiments, the method includes signaling a first flag in the slice header to indicate whether there is an active reference index number in the slice header, wherein the active reference index number is used to derive a maximum reference index of a corresponding reference picture list used to encode the current slice; and in response to the first flag indicating the presence of the active reference index number in the slice header: determining the number of entries in reference picture list 0, and signaling the number of active reference indexes of reference picture list 0 in slice headers for P slices and B slices if the number of entries in reference picture list 0 is greater than 1; and Determining the number of entries in Reference Picture List 1 and signaling the number of active reference indexes in Reference Picture List 1 in a slice header for the B slice if the number of entries in Reference Picture List 1 is greater than one.

[0020]

[0020] In some embodiments, the method includes receiving a video bitstream including a slice header and a picture header syntax; determining a value of a first flag signaled in the slice header indicating whether there is an active reference index number in the slice header, wherein the active reference index number is used to derive a maximum reference index of a corresponding reference picture list used to decode the current slice; in response to the first flag indicating that there is an active reference index number, determining a number of entries in reference picture list 0 and decoding the active reference index number of reference picture list 0 in the slice header for the P slice and the B slice if the number of entries in reference picture list 0 is greater than 1; and determining a number of entries in reference picture list 1 and decoding the active reference index number of reference picture list 1 in the slice header for the B slice if the number of entries in reference picture list 1 is greater than 1.

[0021]

[0021] In some embodiments, the method includes determining a co-located picture referenced by a reference index of the co-located picture within a slice level, where the co-located picture is determined to be the same picture for all non-I slices of the current picture, and processing the current picture based on the co-located picture, where the co-located picture is used for temporal motion vector prediction.

[0022] BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Embodiments and various aspects of the present disclosure are illustrated in the following detailed description and the accompanying drawings, in which the various features shown are not drawn to scale. [Brief explanation of the drawings]

[0023] [Figure 1] FIG. 1 is a schematic diagram illustrating the structure of an exemplary video sequence, according to some embodiments of the present disclosure. [Figure 2A]

[0024] FIG. 2A is a schematic diagram illustrating an example encoding process of a hybrid video encoding system according to an embodiment of the present disclosure. [Figure 2B]

[0025] FIG. 2B is a schematic diagram illustrating an example encoding process of a hybrid video encoding system according to an embodiment of the present disclosure. [Figure 3A]

[0026] FIG. 3A is a schematic diagram illustrating an example decoding process of a hybrid video coding system according to an embodiment of the present disclosure. [Figure 3B]

[0027] FIG. 3B is a schematic diagram illustrating an example decoding process of a hybrid video coding system according to an embodiment of the present disclosure. [Figure 4]

[0028] FIG. 4 is a block diagram of an exemplary apparatus for encoding or decoding video according to some embodiments of the present disclosure. [Figure 5A]

[0029] 1 illustrates an example syntax including a syntax structure for a reference picture list according to some embodiments of the present disclosure. [Figure 5B]

[0030] 10 shows exemplary pseudocode including the derivation of variables FullPocLt[i][j] according to some embodiments of the present disclosure. [Figure 6A]

[0031] 1 illustrates an example syntax including a syntax structure for a reference picture list according to some embodiments of the present disclosure. [Figure 6B]

[0032] 10 shows exemplary pseudocode including the derivation of variables NumLtrpEntries[listIdx][rplsIdx] according to some embodiments of the present disclosure. [Figure 6C]

[0033] 10 shows exemplary pseudocode including the derivation of variables AbsDeltaPocSt[listIdx][rplsIdx][i] according to some embodiments of the present disclosure. [Figure 6D]

[0034] 10 illustrates exemplary pseudocode including the derivation of variables DeltaPocValSt[listIdx][rplsIdx] according to some embodiments of the present disclosure. [Figure 7]

[0035] 1 illustrates an example syntax including a syntax structure for a reference picture list in a sequence parameter set according to some embodiments of the present disclosure. [Figure 8]

[0036] 1 illustrates an example syntax including a syntax structure for a reference picture list in a picture parameter set according to some embodiments of the present disclosure. [Figure 9A]

[0037] 1 illustrates an example syntax including a syntax structure for a reference picture list in a picture header structure according to some embodiments of the present disclosure. [Figure 9B]

[0038] 10 illustrates an example of pseudocode including the derivation of the variable MaxNumSubblockMergeCand, according to some embodiments of the present disclosure. [Figure 10A]

[0039] 10 illustrates an example syntax including a syntax structure for a reference picture list in a slice header according to some embodiments of the present disclosure. [Figure 10B]

[0040] 10 illustrates exemplary pseudocode including the derivation of the variable NumRefIdxActive[i] according to some embodiments of the present disclosure. [Figure 11A]

[0041] 1 illustrates a flowchart of an example video encoding method for signaling flags within a PH syntax structure, according to some embodiments of the present disclosure. [Figure 11B]

[0042] 1 shows a flowchart of an example video decoding method for signaling flags in a PH syntax structure, according to some embodiments of the present disclosure. [Figure 11C]

[0043] 1 illustrates an example syntax including updated signaling for ph_collocated_from_l0_flag and ph_mvd_l1_zero_flag according to some embodiments of the present disclosure. [Figure 12A]

[0044] 1 shows a flowchart of an exemplary video encoding method for indicating co-located pictures using picture order counting, according to some embodiments of this disclosure. [Figure 12B]

[0045] 1 shows a flowchart of an exemplary video encoding method for indicating co-located pictures using picture order counting, according to some embodiments of this disclosure. [Figure 12C]

[0046] 10 shows another flowchart of an exemplary video encoding method for showing co-located pictures, according to some embodiments of the present disclosure. [Figure 12D]

[0047] 1 shows a flowchart of an example video decoding method for indicating co-located pictures using picture order counting, according to some embodiments of this disclosure. [Figure 12E]

[0048] 1 shows a flowchart of an example video decoding method for indicating co-located pictures using picture order counting, according to some embodiments of this disclosure. [Figure 12F]

[0049] 10 illustrates an example syntax including an updated reference picture list in a picture parameter set according to some embodiments of the present disclosure. [Figure 12G]

[0050] 10 illustrates an example syntax including an updated slice header according to some embodiments of the present disclosure. [Figure 12H]

[0051] 10 shows exemplary pseudocode including the derivation of AbsDeltaPocStCol according to some embodiments of the present disclosure. [Figure 12I]

[0052] 10 shows exemplary pseudocode including the derivation of DeltaPocValStCol according to some embodiments of the present disclosure. [Figure 12J1]

[0053] 10 shows exemplary pseudocode for deriving collocated pictures used in a decoding method according to some embodiments of the present disclosure. [Figure 12J2]

[0053] Figure 10 shows exemplary pseudocode for deriving collocated pictures used in decoding methods according to some embodiments of the present disclosure. [Figure 13A]

[0054] 1 shows a flowchart of an example video encoding method for inferring an index of a co-located picture in an SH using the number of active entries in a reference picture list, according to some embodiments of the present disclosure. [Figure 13B]

[0055] 10 shows a flowchart of an example video decoding method for inferring an index of a co-located picture in an SH using the number of active entries in a reference picture list, according to some embodiments of the present disclosure. [Figure 13C]

[0056] 10 illustrates example semantics for the updated syntax element sh_collocated_ref_idx according to some embodiments of the present disclosure. [Figure 14A]

[0057] 1 shows a flowchart of an exemplary video processing method for a decoder that allocates memory, according to some embodiments of the present disclosure. [Figure 14B]

[0058] 1 illustrates example semantics for allocating memory according to some embodiments of the present disclosure. [Figure 15A]

[0059] 1 shows a flowchart of an exemplary video encoding method for inferring an index in a reference picture list, according to some embodiments of this disclosure. [Figure 15B]

[0060] 1 shows a flowchart of an exemplary video decoding method for inferring an index in a reference picture list, according to some embodiments of this disclosure. [Figure 15C]

[0061] 10 illustrates example semantics for the updated variable rpl_idx[i] according to some embodiments of the present disclosure. [Figure 16A]

[0062] 1 shows a flowchart of an example video encoding method for indicating whether an override number of active reference indexes in a slice header exists, according to some embodiments of the present disclosure. [Figure 16B]

[0063] 1 shows a flowchart of an example video decoding method for indicating whether an override number of active reference indexes in a slice header is present, according to some embodiments of the present disclosure. [Figure 16C]

[0064] 10 illustrates example semantics for the updated syntax element sh_num_ref_idx_active_override_flag according to some embodiments of the present disclosure. [Figure 17A]

[0065] 10 shows a flowchart of an exemplary video processing method for determining an index of a co-located picture within an SH for an I-slice, according to some embodiments of the present disclosure. [Figure 17B]

[0066] 10 illustrates example semantics for updated bitstream compliance constraints for syntax element sh_collocated_ref_idx, according to some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0024] Detailed Description

[0067] Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings in which like reference numerals in different drawings represent the same or similar elements unless otherwise indicated. The implementations set forth in the following description of exemplary embodiments do not represent all implementations in accordance with the present invention. Rather, they are merely examples of apparatus and methods in accordance with aspects related to the present invention as recited in the appended claims. Certain aspects of the present disclosure are described in more detail below. In the event of a conflict with terms and / or definitions incorporated by reference, the terms and definitions provided herein shall control.

[0025]

[0068] The ITU-T Video Coding Expert Group (ITU-T VCEG) and the ISO / IEC Moving Picture Expert Group (ISO / IEC MPEG) Joint Video Experts Team (JVET) are currently developing the Versatile Video Coding (VVC / H.266) standard. The VVC standard aims to double the compression efficiency of its predecessor, the High Efficiency Video Coding (HEVC / H.265) standard. In other words, the goal of VVC is to achieve the same subjective quality as HEVC / H.265 while using half the bandwidth.

[0026]

[0069] To achieve the same subjective quality as HEVC / H.265 using half the bandwidth, JVET is developing technology beyond HEVC using the joint exploration model (JEM) reference software. Because the coding technology was incorporated into JEM, JEM achieved substantially higher coding performance than HEVC.

[0027]

[0070] The VVC standard is a recent development and continues to incorporate more coding techniques that result in better compression performance. VVC is based on the same hybrid video coding system that has been used in modern video compression standards such as HEVC, H.264 / AVC, MPEG2, H.263, etc.

[0028]

[0071] A video is a set of static pictures (or "frames") arranged in time sequence to store visual information. A video capture device (e.g., a camera) can be used to capture and store these pictures in time sequence, and a video playback device (e.g., a television, a computer, a smartphone, a tablet computer, a video player, or any end-user terminal with display capabilities) can be used to display such pictures in time sequence. In some applications, a video capture device can also transmit the captured video in real time to a video playback device (e.g., a computer with a monitor) for purposes such as surveillance, conferencing, or live broadcasting.

[0029]

[0072] To reduce the storage space and transmission bandwidth required by such applications, video can be compressed before storage and transmission and decompressed before display. Compression and decompression can be performed by software executed by a processor (e.g., a processor in a general-purpose computer) or by specialized hardware. A module for compression is commonly referred to as an “encoder,” and a module for decompression is commonly referred to as a “decoder.” Collectively, the encoder and decoder can be referred to as a “codec.” The encoder and decoder can be implemented as any of a variety of suitable hardware, software, or combinations thereof. For example, hardware implementations of the encoder and decoder can include circuitry such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, or any combination thereof. Software implementations of the encoder and decoder can include program code, computer-executable instructions, firmware, or any suitable computer-implemented algorithm or process fixed in a computer-readable medium. Video compression and decompression may be performed by various algorithms or standards, such as MPEG-1, MPEG-2, MPEG-4, the H.26x series, or the like. In some applications, a codec may decompress video from a first encoding standard and recompress the decompressed video using a second encoding standard. In this case, the codec may be referred to as a "transcoder."

[0030]

[0073] A video coding process can identify and retain useful information that can be used to reconstruct a picture and ignore information that is not important for reconstruction. If the ignored, unimportant information cannot be perfectly reconstructed, such a coding process may be called "lossy." Otherwise, it may be called "lossless." Most coding processes are lossy; this is a tradeoff to reduce the required storage space and transmission bandwidth.

[0031]

[0074] Useful information about the picture being coded (called the "current picture") includes changes relative to a reference picture (e.g., a previously coded and reconstructed picture). Such changes may include changes in pixel position, brightness, or color, with position changes being the most important. Changes in the position of a group of pixels representing an object may reflect the movement of the object between the reference picture and the current picture.

[0032]

[0075] A picture that is coded without reference to another picture (i.e., it is its own reference picture) is called an "I-picture." A picture is called a "P-picture" if some or all of the blocks in the picture (e.g., blocks that generally refer to portions of a video picture) are predicted using intra- or inter-prediction with one reference picture (e.g., unidirectional prediction). A picture is called a "B-picture" if at least one block in the picture is predicted using two reference pictures (e.g., bidirectional prediction).

[0033]

[0076] 1 illustrates the structure of an exemplary video sequence 100 according to some embodiments of the present disclosure. The video sequence 100 can be live video or captured and archived video. The video sequence 100 can be real video, computer-generated video (e.g., computer game video), or a combination thereof (e.g., real video with augmented reality effects). The video sequence 100 can be input from a video capture device (e.g., a camera), a video archive containing previously captured video (e.g., video files stored in a storage device), or a video supply interface (e.g., a video broadcast transceiver) for receiving video from a video content provider.

[0034]

[0077] As shown in FIG. 1, video sequence 100 may include a series of pictures arranged temporally along a timeline, including pictures 102, 104, 106, and 108. Pictures 102-106 are consecutive, with additional pictures between pictures 106 and 108. In FIG. 1, picture 102 is an I-picture, and its reference picture is picture 102 itself. Picture 104 is a P-picture, and its reference picture is picture 102, as indicated by the arrow. Picture 106 is a B-picture, and its reference pictures are pictures 104 and 108, as indicated by the arrows. In some embodiments, the reference picture of a picture (e.g., picture 104) need not immediately precede or follow that picture. For example, the reference picture of picture 104 can be a picture before picture 102. It should be noted that the reference pictures of pictures 102-106 are merely examples, and this disclosure does not limit the reference picture embodiments to the examples shown in FIG.

[0035]

[0078] Typically, video codecs do not encode or decode an entire picture at once due to the computational complexity of such a task. Rather, they may divide a picture into elementary segments and encode or decode the picture segment by segment. Such elementary segments are referred to as basic processing units ("BPUs") in this disclosure. For example, structure 110 in FIG. 1 shows an example structure of a picture (e.g., any of pictures 102-108) of video sequence 100. In structure 110, the picture is divided into 4x4 basic processing units, the boundaries of which are shown as dashed lines. In some embodiments, basic processing units may be referred to as "macroblocks" in some video coding standards (e.g., MPEG family, H.261, H.263, or H.264 / AVC) or as "coding tree units" ("CTUs") in some other video coding standards (e.g., H.265 / HEVC or H.266 / VVC). The basic processing units can have variable sizes in pictures or any arbitrary shape and size of pixels, such as 128x128, 64x64, 32x32, 16x16, 4x8, 16x32, etc. The size and shape of the basic processing unit can be selected based on a balance between coding efficiency and the level of detail to be maintained in the basic processing unit for the picture.

[0036]

[0079] A basic processing unit may be a logical unit that can include groups of different types of video data stored in computer memory (e.g., in a video frame buffer). For example, a basic processing unit for a color picture may include a luma component (Y) that represents colorless luminance information, one or more chroma components (e.g., Cb and Cr) that represent color information, and related syntax elements, where the luma and chroma components may have the same size of a basic processing unit. The luma and chroma components may be referred to as "coding tree blocks" (CTBs) in some video coding standards (e.g., H.265 / HEVC or H.266 / VVC). Any operation performed on a basic processing unit may be performed repeatedly on each of its luma and chroma components.

[0037]

[0080] Video coding involves multiple computational stages, examples of which are shown in Figures 2A-2B and 3A-3B. At each stage, the size of the basic processing unit may still become too large for processing and therefore may be further divided into segments referred to as "basic processing subunits" in this disclosure. In some embodiments, the basic processing subunits may be referred to as "blocks" in some video coding standards (e.g., MPEG family, H.261, H.263, or H.264 / AVC) or as "coding units" ("CUs") in some other video coding standards (e.g., H.265 / HEVC or H.266 / VVC). The basic processing subunits may have the same or smaller size than the basic processing units. Similar to the basic processing units, the basic processing subunits are also logical units that may contain groups of different types of video data (e.g., Y, Cb, Cr, and related syntax elements) stored in computer memory (e.g., in a video frame buffer). Any operation performed on a basic processing sub-unit may be repeatedly performed on each of its luma and chroma components. Note that such division may be performed to further levels as needed for processing. Also note that different stages may use different schemes to divide the basic processing units.

[0038]

[0081] For example, in a mode decision stage (an example of which is shown in FIG. 2B ), an encoder can decide what prediction mode (e.g., intra-picture prediction or inter-picture prediction) to use for a basic processing unit, but the basic processing unit may be too large to make such a decision. The encoder can divide the basic processing unit into multiple basic processing sub-units (e.g., CUs, as in the case of H.265 / HEVC or H.266 / VVC) and decide the type of prediction for each individual basic processing sub-unit.

[0039]

[0082] As another example, in the prediction stage (an example of which is shown in FIGS. 2A-2B), the encoder may perform prediction operations at the level of basic processing sub-units (e.g., CUs). However, in some cases, the basic processing sub-units may still be too large to process. The encoder may further divide the basic processing sub-units into smaller segments (e.g., referred to as "prediction blocks" or "PBs" in H.265 / HEVC or H.266 / VVC), at which level the prediction operations may be performed.

[0040]

[0083] As another example, in the transform stage (examples of which are shown in FIGS. 2A and 2B), the encoder may perform transform operations for residual basic processing subunits (e.g., CUs). However, in some cases, the basic processing subunits may still be too large to process. The encoder may further divide the basic processing subunits into smaller segments (e.g., referred to as "transform blocks" or "TBs" in H.265 / HEVC or H.266 / VVC), at which levels the transform operations may be performed. Note that the division scheme of the same basic processing subunit may be different in the prediction stage and the transform stage. For example, in H.265 / HEVC or H.266 / VVC, the prediction blocks and transform blocks of the same CU may have different sizes and numbers.

[0041]

[0084] 1, the fundamental processing units 112 are further divided into 3x3 fundamental processing sub-units, the boundaries of which are shown as dotted lines. Different fundamental processing units of the same picture may be divided into fundamental processing sub-units in different ways.

[0042]

[0085] In some implementations, to provide parallel processing and error resilience capabilities to video encoding and decoding, a picture can be divided into regions for processing, so that the encoding or decoding process does not rely on information about a picture region from any other region of the picture. In other words, each region of a picture can be processed independently. By doing so, the codec can process different regions of a picture in parallel, thus increasing coding efficiency. Also, when data for a region is corrupted during processing or lost during network transmission, the codec can correctly encode or decode other regions of the same picture without relying on the corrupted or lost data, thus providing error resilience. Some video coding standards allow a picture to be divided into different types of regions. For example, H.265 / HEVC and H.266 / VVC provide two types of regions: "slices" and "tiles." It should also be noted that different pictures in video sequence 100 can have different partitioning schemes for dividing the picture into regions.

[0043]

[0086] For example, in Figure 1, structure 110 is divided into three regions 114, 116, and 118, the boundaries of which are shown as solid lines within structure 110. Region 114 includes four basic processing units. Regions 116 and 118 each include six basic processing units. It should be noted that the basic processing units, basic processing subunits, and regions of structure 110 in Figure 1 are merely examples, and the present disclosure does not limit the embodiments thereof.

[0044]

[0087] FIG. 2A shows a schematic diagram of an exemplary encoding process 200A according to an embodiment of the present disclosure. For example, encoding process 200A may be performed by an encoder. As shown in FIG. 2A, the encoder may encode a video sequence 202 into a video bitstream 228 according to process 200A. Similar to video sequence 100 in FIG. 1, video sequence 202 may include a set of pictures (referred to as "original pictures") arranged in a temporal order. Similar to structure 110 in FIG. 1, each original picture in video sequence 202 may be divided by the encoder into basic processing units, basic processing sub-units, or regions for processing. In some embodiments, the encoder may perform process 200A at the level of basic processing units for each original picture in video sequence 202. For example, the encoder may perform process 200A in an iterative manner, in which case the encoder may encode a basic processing unit in one iteration of process 200A. In some embodiments, the encoder may perform process 200A in parallel for regions of each original picture in video sequence 202 (eg, regions 114-118).

[0045]

[0088] 2A , an encoder may provide a fundamental processing unit (referred to as an “original BPU”) of an original picture of a video sequence 202 to a prediction stage 204 to generate prediction data 206 and a prediction BPU 208. The encoder may subtract the prediction BPU 208 from the original BPU to generate a residual BPU 210. The encoder may provide the residual BPU 210 to a transform stage 212 and a quantization stage 214 to generate quantized transform coefficients 216. The encoder may provide the prediction data 206 and the quantized transform coefficients 216 to a binary coding stage 226 to generate a video bitstream 228. Components 202, 204, 206, 208, 210, 212, 214, 216, 226, and 228 may be referred to as a “forward path.” During process 200A, after quantization stage 214, the encoder may provide quantized transform coefficients 216 to inverse quantization stage 218 and inverse transform stage 220 to generate reconstructed residual BPU 222. The encoder may add reconstructed residual BPU 222 to prediction BPU 208 to generate prediction reference 224, which is used in prediction stage 204 for the next iteration of process 200A. Components 218, 220, 222, and 224 of process 200A may be referred to as a "reconstruction path." The reconstruction path may be used to ensure that both the encoder and decoder use the same reference data for prediction.

[0046]

[0089] The encoder may iteratively perform process 200A to encode each original BPU of the original picture (in the forward path) and generate (in the reconstruction path) a prediction reference 224 for encoding the next original BPU of the original picture. After encoding all original BPUs of the original picture, the encoder may proceed to encode the next picture in video sequence 202.

[0047]

[0090] Referring to process 200A, an encoder may receive a video sequence 202 generated by a video capture device (e.g., a camera). As used herein, the term "receive" may refer to receiving, inputting, acquiring, obtaining, getting, reading, accessing, or any act in any manner to input data.

[0048]

[0091] In the prediction step 204, in the current iteration, the encoder may receive the original BPU and a prediction reference 224, perform a prediction operation, and generate predicted data 206 and a predicted BPU 208. The prediction reference 224 may be generated from a reconstruction path of a previous iteration of the process 200A. The purpose of the prediction step 204 is to reduce information redundancy by extracting the predicted data 206, which can be used to reconstruct the original BPU as a predicted BPU 208 from the predicted data 206 and the prediction reference 224.

[0049]

[0092] Ideally, predicted BPU 208 can be identical to the original BPU. However, due to non-ideal prediction and reconstruction operations, predicted BPU 208 generally differs slightly from the original BPU. To record such differences, after generating predicted BPU 208, the encoder can subtract it from the original BPU to generate residual BPU 210. For example, the encoder can subtract pixel values ​​(e.g., grayscale or RGB values) of predicted BPU 208 from corresponding pixel values ​​of the original BPU. Each pixel of residual BPU 210 can have a residual value that is the result of such a subtraction between the corresponding pixel of the original BPU and predicted BPU 208. Compared to the original BPU, predicted data 206 and residual BPU 210 can have fewer bits, which can be used to reconstruct the original BPU without significant quality degradation. Therefore, the original BPU is compressed.

[0050]

[0093] To further compress the residual BPU 210, in the transform stage 212, the encoder can reduce spatial redundancy in the residual BPU 210 by decomposing it into a set of two-dimensional "basis patterns," each associated with a "transform coefficient." The basis patterns can have the same size (e.g., the size of the residual BPU 210). Each basis pattern can represent a change frequency (e.g., a frequency of luminance change) component of the residual BPU 210. No basis pattern can be reconstructed from any combination (e.g., a linear combination) of any other basis patterns. In other words, the decomposition can decompose the changes in the residual BPU 210 into the frequency domain. Such a decomposition is similar to a discrete Fourier transform of a function, where the basis patterns are similar to the basis functions (e.g., trigonometric functions) of the discrete Fourier transform, and the transform coefficients are similar to the coefficients associated with the basis functions.

[0051]

[0094] Different transform algorithms can use different basis patterns. For example, various transform algorithms can be used in transform stage 212, such as a discrete cosine transform, a discrete sine transform, or the like. The transform in transform stage 212 is invertible. That is, an encoder can reconstruct residual BPU 210 by inverting the transform (referred to as an "inverse transform"). For example, to reconstruct pixels of residual BPU 210, the inverse transform can multiply the values ​​of corresponding pixels in the basis pattern by their associated coefficients and add the products to generate a weighted sum. For video coding standards, both the encoder and decoder can use the same transform algorithm (and therefore the same basis pattern). Therefore, the encoder can record only the transform coefficients, and the decoder can reconstruct residual BPU 210 from the transform coefficients without receiving the basis pattern from the encoder. Compared to residual BPU 210, the transform coefficients can have fewer bits, which can be used to reconstruct residual BPU 210 without significant quality degradation. Therefore, the residual BPU 210 is further compressed.

[0052]

[0095] The encoder can further compress the transform coefficients in the quantization stage 214. In the transform process, different basis patterns can represent different change frequencies (e.g., luminance change frequencies). Because the human eye is generally better at perceiving low-frequency changes, the encoder can ignore high-frequency change information without significant quality degradation during decoding. For example, in the quantization stage 214, the encoder can generate quantized transform coefficients 216 by dividing each transform coefficient by an integer value (referred to as a "quantization scale factor") and rounding the quotient to its nearest integer. After such an operation, some transform coefficients of high-frequency basis patterns can be converted to zero, and transform coefficients of low-frequency basis patterns can be converted to smaller integers. The encoder can ignore zero-valued quantized transform coefficients 216, thereby further compressing the transform coefficients. The quantization process can also be inverted, in which case the quantized transform coefficients 216 can be reconstructed into transform coefficients in the inverse operation of quantization (referred to as "dequantization").

[0053]

[0096] Because the encoder ignores the remainder of such a division in a rounding operation, quantization stage 214 may be lossy. Typically, quantization stage 214 may contribute the greatest information loss in process 200A. The greater the information loss, the fewer bits quantized transform coefficients 216 may require. To achieve different levels of information loss, the encoder may use different values ​​of the quantization syntax element or any other syntax element in the quantization process.

[0054]

[0097] In binary encoding stage 226, the encoder may encode the prediction data 206 and the quantized transform coefficients 216 using a binary encoding technique, such as, for example, entropy coding, variable length coding, arithmetic coding, Huffman coding, context-adaptive binary arithmetic coding, or any other lossless or lossy compression algorithm. In some embodiments, in addition to the prediction data 206 and the quantized transform coefficients 216, the encoder may encode other information in binary encoding stage 226, such as, for example, a prediction mode used in prediction stage 204, syntax elements for the prediction operation, the type of transform in transform stage 212, syntax elements for the quantization process (e.g., quantization syntax elements), encoder control syntax elements (e.g., bitrate control syntax elements), or the like. The encoder may use the output data of binary encoding stage 226 to generate a video bitstream 228. In some embodiments, the video bitstream 228 may be further packetized for network transmission.

[0055]

[0098] Referring to the reconstruction path of process 200A, in an inverse quantization stage 218, the encoder may perform inverse quantization on the quantized transform coefficients 216 to generate reconstructed transform coefficients. In an inverse transform stage 220, the encoder may generate a reconstructed residual BPU 222 based on the reconstructed transform coefficients. The encoder may add the reconstructed residual BPU 222 to a prediction BPU 208 to generate a prediction reference 224 to be used in the next iteration of process 200A.

[0056]

[0099] It should be noted that other variations of process 200A may be used to encode video sequence 202. In some embodiments, the stages of process 200A may be performed in a different order by the encoder. In some embodiments, one or more stages of process 200A may be combined into a single stage. In some embodiments, a single stage of process 200A may be split into multiple stages. For example, transform stage 212 and quantization stage 214 may be combined into a single stage. In some embodiments, process 200A may include additional stages. In some embodiments, process 200A may omit one or more stages in FIG. 2A.

[0057]

[0100] 2B shows a schematic diagram of another exemplary encoding process 200B according to an embodiment of the present disclosure. Process 200B may be modified from process 200A. For example, process 200B may be used by an encoder compliant with a hybrid video coding standard (e.g., the H.26x series). Compared to process 200A, the forward path of process 200B additionally includes a mode decision stage 230 and divides the prediction stage 204 into a spatial prediction stage 2042 and a temporal prediction stage 2044. The reconstruction path of process 200B additionally includes a loop filter stage 232 and a buffer 234.

[0058]

[0101] Generally, prediction techniques can be categorized into two types: spatial prediction and temporal prediction. Spatial prediction (e.g., intra-picture prediction or "intra-prediction") can use pixels from one or more already-encoded neighboring BPUs within the same picture to predict the current BPU. That is, the prediction reference 224 in spatial prediction can include neighboring BPUs. Spatial prediction can reduce the inherent spatial redundancy of a picture. Temporal prediction (e.g., inter-picture prediction or "inter-prediction") can use regions from one or more already-encoded pictures to predict the current BPU. That is, the prediction reference 224 in temporal prediction can include an encoded picture. Temporal prediction can reduce the inherent temporal redundancy of a picture.

[0059]

[0102] Referring to process 200B, within the forward path, the encoder performs prediction operations in a spatial prediction step 2042 and a temporal prediction step 2044. For example, in the spatial prediction step 2042, the encoder may perform intra prediction. For an original BPU of a picture being encoded, the prediction reference 224 may include one or more neighboring BPUs coded (in the forward path) and reconstructed (in the reconstruction path) within the same picture. The encoder may generate the predicted BPU 208 by extrapolating the neighboring BPUs. Extrapolation techniques may include, for example, linear extrapolation or interpolation, polynomial extrapolation or interpolation, or the like. In some embodiments, the encoder may perform extrapolation at the pixel level, such as by extrapolating, for each pixel of the predicted BPU 208, the value of the corresponding pixel. The neighboring BPUs used for extrapolation can be located relative to the original BPU from various directions, such as vertically (e.g., above the original BPU), horizontally (e.g., to the left of the original BPU), diagonally (e.g., below-left, below-right, above-left, or above-right of the original BPU), or any direction defined in the video coding standard used. For intra prediction, the prediction data 206 can include, for example, the locations (e.g., coordinates) of the neighboring BPUs used, the sizes of the neighboring BPUs used, syntax elements for extrapolation, the orientations of the neighboring BPUs used relative to the original BPU, or the like.

[0060]

[0103] As another example, in the temporal prediction stage 2044, the encoder may perform inter-prediction. For the original BPU of the current picture, the prediction reference 224 may include one or more pictures (referred to as "reference pictures") that have been coded (in the forward path) and reconstructed (in the reconstruction path). In some embodiments, the reference pictures may be coded and reconstructed for each BPU. For example, the encoder may add the reconstructed residual BPU 222 to the predicted BPU 208 to generate a reconstructed BPU. When all the reconstructed BPUs of the same picture have been generated, the encoder may generate the reconstructed picture as the reference picture. The encoder may perform a "motion estimation" operation to search for a matching region within a range (referred to as a "search window") of the reference picture. The location of the search window in the reference picture may be determined based on the location of the original BPU of the current picture. For example, the search window may be centered in the reference picture at a location having the same coordinates as the original BPU in the current picture and may extend outward over a predetermined distance. When the encoder identifies a region within the search window that is similar to the original BPU (e.g., by using a pel-recursive algorithm, a block matching algorithm, or the like), the encoder can determine such a region as a matching region. The matching region can have different dimensions than the original BPU (e.g., smaller than, equal to, larger than, or a different shape than the original BPU). Because the reference picture and the current picture are temporally separated in a timeline (e.g., as shown in FIG. 1), the matching region can be considered to "move" toward the location of the original BPU over time. The encoder can record the direction and distance of such movement as a "motion vector." When multiple reference pictures are used (e.g., picture 106 in FIG. 1), the encoder can search for the matching region in each reference picture and determine its associated motion vector. In some embodiments, the encoder can assign weights to the pixel values ​​of the matching region in each matching reference picture.

[0061]

[0104] Motion estimation can be used to identify various types of motion, such as, for example, translation, rotation, zooming, or the like. For inter prediction, prediction data 206 can include, for example, the location (e.g., coordinates) of the matching region, a motion vector associated with the matching region, the number of reference pictures, weights associated with the reference pictures, or the like.

[0062]

[0105] To generate the predicted BPU 208, the encoder may perform a "motion compensation" operation. Motion compensation may be used to reconstruct the predicted BPU 208 based on the prediction data 206 (e.g., a motion vector) and the prediction reference 224. For example, the encoder may shift the matching region of the reference picture according to the motion vector, in which case the original BPU of the current picture may be predicted. When multiple reference pictures are used (e.g., as in picture 106 in FIG. 1), the encoder may shift the matching region of the reference picture according to each motion vector and average the pixel values ​​of the matching region. In some embodiments, if the encoder weights the pixel values ​​of the matching region of each matching reference picture, the encoder may add a weighted sum of the pixel values ​​to the shifted matching region.

[0063]

[0106] Depending on the embodiment, inter-prediction can be unidirectional or bidirectional. Unidirectional inter-prediction can use one or more reference pictures in the same temporal direction relative to the current picture. For example, picture 104 in FIG. 1 is a unidirectional inter-predicted picture in which a reference picture (i.e., picture 102) precedes picture 104. Bidirectional inter-prediction can use one or more reference pictures in both temporal directions relative to the current picture. For example, picture 106 in FIG. 1 is a bidirectional inter-predicted picture in which reference pictures (i.e., pictures 104 and 108) are in both temporal directions relative to picture 104.

[0064]

[0107] Still referring to the forward path of process 200B, after spatial prediction 2042 and temporal prediction step 2044, in mode decision step 230, the encoder may select a prediction mode (e.g., one of intra prediction or inter prediction) for the current iteration of process 200B. For example, the encoder may perform a rate-distortion optimization technique. In this technique, the encoder may select a prediction mode to minimize the value of a cost function that depends on the bitrate of the candidate prediction mode and the distortion of the reconstructed reference picture under the candidate prediction mode. Depending on the selected prediction mode, the encoder may generate a corresponding predicted BPU 208 and predicted data 206.

[0065]

[0108] Within the reconstruction path of process 200B, if an intra-prediction mode is selected within the forward path, after generating prediction reference 224 (e.g., the current BPU encoded and reconstructed in the current picture), the encoder may provide prediction reference 224 directly to spatial prediction stage 2042 for later use (e.g., for extrapolation of the next BPU of the current picture). The encoder may provide prediction reference 224 to loop filter stage 232, where the encoder may apply a loop filter to prediction reference 224 to reduce or eliminate distortions (e.g., blocking artifacts) introduced during the encoding of prediction reference 224. The encoder may apply various loop filter techniques in loop filter stage 232, such as, for example, deblocking, sample adaptive offset, adaptive loop filter, or the like. The loop-filtered reference picture may be stored in buffer 234 (or a “decoded picture buffer”) for later use (e.g., to be used as an inter-prediction reference picture for future pictures of video sequence 202). The encoder may store one or more reference pictures in a buffer 234 for use in the temporal prediction stage 2044. In some embodiments, the encoder may encode loop filter syntax elements (e.g., loop filter strength) along with the quantized transform coefficients 216, the prediction data 206, and other information in the binary encoding stage 226.

[0066]

[0109] FIG. 3A shows a schematic diagram of an exemplary decoding process 300A according to an embodiment of the present disclosure. Process 300A may be a decompression process corresponding to compression process 200A in FIG. 2A. In some embodiments, process 300A may be similar to the reconstruction path of process 200A. A decoder may follow process 300A to decode video bitstream 228 into video stream 304. Video stream 304 may be similar to video sequence 202. However, due to information loss in the compression and decompression processes (e.g., quantization stage 214 in FIGS. 2A and 2B), video stream 304 is generally not identical to video sequence 202. Similar to processes 200A and 200B in FIGS. 2A and 2B, a decoder may perform process 300A at the level of a basic processing unit (BPU) for each picture encoded in video bitstream 228. For example, the decoder may perform process 300A in an iterative manner, in which case the decoder may decode a basic processing unit in one iteration of process 300A. In some embodiments, the decoder may perform process 300A in parallel for regions (e.g., regions 114-118) of each picture encoded in video bitstream 228.

[0067]

[0110] In FIG. 3A , a decoder may provide a portion of a video bitstream 228 associated with a basic processing unit (referred to as a “coding BPU”) of a coded picture to a binary decoding stage 302. In the binary decoding stage 302, the decoder may decode the portion into prediction data 206 and quantized transform coefficients 216. The decoder may provide the quantized transform coefficients 216 to an inverse quantization stage 218 and an inverse transform stage 220 to generate a reconstructed residual BPU 222. The decoder may provide the prediction data 206 to a prediction stage 204 to generate a prediction BPU 208. The decoder may add the reconstructed residual BPU 222 to the prediction BPU 208 to generate a prediction reference 224. In some embodiments, the prediction reference 224 may be stored in a buffer (e.g., a decoded picture buffer in computer memory). The decoder may provide the prediction reference 224 to the prediction stage 204 for performing a prediction operation in a next iteration of the process 300A.

[0068]

[0111] The decoder may perform process 300A iteratively to decode each coded BPU of a coded picture and generate a prediction reference 224 for encoding the next coded BPU of the coded picture. After decoding all coded BPUs of a coded picture, the decoder may output the picture to video stream 304 for display and proceed to decode the next coded picture in video bitstream 228.

[0069]

[0112] In binary decoding stage 302, the decoder may perform the inverse operation of the binary coding technique used by the encoder (e.g., entropy coding, variable length coding, arithmetic coding, Huffman coding, context-adaptive binary arithmetic coding, or any other lossless compression algorithm). In some embodiments, in addition to prediction data 206 and quantized transform coefficients 216, the decoder may decode other information in binary decoding stage 302, such as, for example, a prediction mode, a syntax element for the prediction operation, a type of transform, a syntax element for the quantization process (e.g., a quantization syntax element), an encoder control syntax element (e.g., a bitrate control syntax element), or the like. In some embodiments, if video bitstream 228 is transmitted in packets over a network, the decoder may depacketize video bitstream 228 before providing it to binary decoding stage 302.

[0070]

[0113] 3B shows a schematic diagram of another exemplary decoding process 300B according to an embodiment of the present disclosure. Process 300B may be modified from process 300A. For example, process 300B may be used by a decoder compliant with a hybrid video coding standard (e.g., the H.26x series). Compared to process 300A, process 300B additionally divides prediction stage 204 into spatial prediction stage 2042 and temporal prediction stage 2044, and additionally includes loop filter stage 232 and buffer 234.

[0071]

[0114] In process 300B, prediction data 206 decoded by the decoder from binary decoding stage 302 for a coding basic processing unit (referred to as the "current BPU") of a coding picture being decoded (referred to as the "current picture") may include various types of data, depending on what prediction mode was used by the encoder to encode the current BPU. For example, if intra prediction was used by the encoder to encode the current BPU, prediction data 206 may include a prediction mode indicator (e.g., a flag value) indicating intra prediction, a syntax element for an intra prediction operation, or the like. A syntax element for an intra prediction operation may include, for example, the location (e.g., coordinates) of one or more neighboring BPUs used as references, the size of the neighboring BPUs, a syntax element for extrapolation, the direction of the neighboring BPUs relative to the original BPU, or the like. As another example, if inter prediction is used by the encoder to encode the current BPU, the prediction data 206 may include a prediction mode indicator (e.g., a flag value) that indicates inter prediction, a syntax element for the inter prediction operation, or the like. The syntax element for the inter prediction operation may include, for example, the number of reference pictures associated with the current BPU, weights respectively associated with the reference pictures, locations (e.g., coordinates) of one or more matching regions within each reference picture, one or more motion vectors respectively associated with the matching regions, or the like.

[0072]

[0115] Based on the prediction mode indicator, the decoder may determine whether to perform spatial prediction (e.g., intra prediction) in spatial prediction step 2042 or temporal prediction (e.g., inter prediction) in temporal prediction step 2044. Details of performing such spatial or temporal prediction are described in FIG. 2B and will not be repeated below. After performing such spatial or temporal prediction, the decoder may generate a predicted BPU 208. The decoder may add the predicted BPU 208 and the reconstructed residual BPU 222 to generate a prediction reference 224, as described in FIG. 3A.

[0073]

[0116] In process 300B, the decoder may provide the prediction reference 224 to the spatial prediction stage 2042 or the temporal prediction stage 2044 to perform a prediction operation in the next iteration of process 300B. For example, if the current BPU is decoded using intra prediction in the spatial prediction stage 2042, after generating the prediction reference 224 (e.g., the decoded current BPU), the decoder may provide the prediction reference 224 directly to the spatial prediction stage 2042 for later use (e.g., for extrapolation of the next BPU of the current picture). If the current BPU is decoded using inter prediction in the temporal prediction stage 2044, after generating the prediction reference 224 (e.g., the reference picture from which all BPUs are decoded), the decoder may provide the prediction reference 224 to the loop filter stage 232 to reduce or eliminate distortion (e.g., blocking artifacts). The decoder may apply a loop filter to the prediction reference 224 in the manner described in FIG. 2B . The loop-filtered reference picture may be stored in a buffer 234 (e.g., a decoded picture buffer (DPB) in computer memory) for later use (e.g., to be used as an inter-prediction reference picture for a future coded picture of the video bitstream 228). The decoder may store one or more reference pictures in the buffer 234 for use in the temporal prediction stage 2044. In some embodiments, the prediction data may further include a loop filter syntax element (e.g., a loop filter strength). In some embodiments, if the prediction mode indicator in the prediction data 206 indicates that inter-prediction was used to encode the current BPU, the prediction data includes a loop filter syntax element.

[0074]

[0117] 4 is a block diagram of an exemplary device 400 for encoding or decoding video, according to an embodiment of the present disclosure. As shown in FIG. 4, the device 400 may include a processor 402. When the processor 402 executes the instructions described herein, the device 400 can become a dedicated machine for video encoding or decoding. The processor 402 can be any type of circuitry capable of manipulating or processing information. For example, processor 402 may include any number and combination of a central processing unit (or "CPU"), a graphics processing unit (or "GPU"), a neural processing unit ("NPU"), a microcontroller unit ("MCU"), an optical processor, a programmable logic controller, a microcontroller, a microprocessor, a digital signal processor, an intellectual property (IP) core, a programmable logic array (PLA), a programmable array logic (PAL), a generic array logic (GAL), a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a system on chip (SoC), an application-specific integrated circuit (ASIC), or the like. In some embodiments, processor 402 may also be a set of processors grouped as a single logical entity. For example, as shown in Figure 4, processor 402 may include multiple processors, including processor 402a, processor 402b, and processor 402n.

[0075]

[0118] The device 400 may also include a memory 404 configured to store data (e.g., a set of instructions, computer code, intermediate data, or the like). For example, as shown in Figure 4, the stored data may include program instructions (e.g., program instructions for performing steps in processes 200A, 200B, 300A, or 300B) and data for processing (e.g., video sequence 202, video bitstream 228, or video stream 304). The processor 402 may access the program instructions and data for processing (e.g., via bus 410), execute the program instructions, and perform operations or manipulations on the data for processing. The memory 404 may include a high-speed random-access storage device or a non-volatile storage device. In some embodiments, memory 404 may include any number or combination of random-access memory (RAM), read-only memory (ROM), optical disks, magnetic disks, hard drives, solid-state drives, flash drives, security digital (SD) cards, memory sticks, compact flash (CF) cards, or the like. Memory 404 may also be a group of memories (not shown in FIG. 4) grouped as a single logical entity.

[0076]

[0119] Bus 410 can be a communication device that transfers data between components internal to apparatus 400, such as an internal bus (e.g., a CPU-memory bus), an external bus (e.g., a Universal Serial Bus port, a Peripheral Component Interconnect Express port), or the like.

[0077]

[0120] For ease of explanation and without ambiguity, the processor 402 and other data processing circuitry will be collectively referred to in this disclosure as "data processing circuitry." The data processing circuitry may be implemented entirely in hardware or as a combination of software, hardware, or firmware. Additionally, the data processing circuitry may be a single, stand-alone module or may be fully or partially combined with any other component of the device 400.

[0078]

[0121] Device 400 may further include a network interface 406 for providing wired or wireless communication with a network (e.g., the Internet, an intranet, a local area network, a mobile communication network, or the like). In some embodiments, network interface 406 may include any number or combination of a network interface controller (NIC), a radio frequency (RF) module, a transponder, a transceiver, a modem, a router, a gateway, a wired network adapter, a wireless network adapter, a Bluetooth® adapter, an infrared adapter, a near-field communication ("NFC") adapter, a cellular network chip, or the like.

[0079]

[0122] In some embodiments, apparatus 400 may optionally further include a peripheral interface 408 for providing connection to one or more peripheral devices. As shown in Figure 4, the peripheral devices may include, but are not limited to, a cursor control device (e.g., a mouse, a touchpad, or a touchscreen), a keyboard, a display (e.g., a cathode ray tube display, a liquid crystal display, or a light emitting diode display), a video input device (e.g., a camera, or an input interface coupled to a video archive), or the like.

[0080]

[0123] It should be noted that a video codec (e.g., a codec performing process 200A, 200B, 300A, or 300B) may be implemented as any combination of software or hardware modules within device 400. For example, some or all of the stages of process 200A, 200B, 300A, or 300B may be implemented as one or more software modules of device 400, such as program instructions that may be loaded into memory 404. As another example, some or all of the stages of process 200A, 200B, 300A, or 300B may be implemented as one or more hardware modules of device 400, such as specialized data processing circuitry (e.g., an FPGA, an ASIC, an NPU, or the like).

[0081]

[0124] In video coding, pictures need to be identified for multiple purposes, such as identifying them as reference pictures in inter prediction, identifying them as pictures output from the DPB, identifying them as temporally co-located pictures for motion vector prediction, etc. The most common way to identify pictures is using a Picture Order Count ("POC").

[0082]

[0125] To identify reference pictures in inter prediction, motion vector ("MV") temporal prediction, and temporally co-located pictures in scaling, reference picture lists (usually two, as in AVC, HEVC, and VVC) can be derived. For example, reference picture list 0 and reference picture list 1 can be derived, each containing a list of reconstructed pictures in the DPB to be used as reference pictures. To identify the reference picture for the current block, a reference index to the reference picture list can be signaled at the block level. Reference picture marking is necessary to correctly maintain reference pictures in the DPB without requiring an unnecessarily large amount of DPB memory.

[0083]

[0126] VVC (e.g., VVC Draft 9) uses two reference picture lists ("RPL"): Reference Picture List 0 and Reference Picture List 1. They are signaled and derived directly. Information about the two reference picture lists is signaled by syntax elements and syntax structures in the Sequence Parameter Set ("SPS"), Picture Parameter Set ("PPS"), Picture Header ("PH"), and Slice Header ("SH"). A default reference picture list structure is signaled in the SPS for use by references in the PH or SH. A new reference picture list structure may be signaled in the PH or SH to derive Reference Picture List 0 and Reference Picture List 1. Whether reference picture list information is signaled in the PH or SH is determined by a flag signaled in the PPS.

[0084]

[0127] In VVC (e.g., VVC Draft 9), two reference picture lists are generated for all types of slices (e.g., B, P, and I slices). In an I slice, neither of the two reference picture lists, reference picture list 0 nor reference picture list 1, can be used for decoding. In a P slice, only reference picture list 0 can be used for decoding. In a B slice, both reference picture lists, reference picture list 0 and reference picture list 1, can be used for decoding. The two reference picture lists are constructed without using the reference picture list initialization process or the reference picture list modification process.

[0085]

[0128] Not all pictures in the reference picture list are used as reference pictures for the current picture or slice. Only active entries of the reference picture list can be used in the decoding process of slice data. The default number of active entries is signaled in the PPS in VVC (e.g., VVC Draft 9) and can be overridden by the slice header for the current slice.

[0086]

[0129] To identify pictures in the DPB for constructing the RPL, a POC containing a most significant bit ("MSB") and a least significant bit ("LSB") is used. In VVC (e.g., VVC Draft 9), the LSB of the POC is signaled in the PH, and the MSB can be explicitly signaled in the PH or derived by comparing the POC LSB of the current picture with the POC LSB of one or more previous pictures.

[0087]

[0130] In VVC (e.g., VVC Draft 9), a decoded picture in a DPB can be marked as "unused for reference," "in use for short-term reference," or "in use for long-term reference." A decoded picture can only be marked as one of these three at any given moment during the operation of the decoding process. Assigning one of these markings to a picture implicitly removes other markings, if applicable. When a picture is referred to as being marked as "in use for reference," this also refers to the picture being marked as being "in use for short-term reference" or "in use for long-term reference," but not both.

[0088]

[0131] Short-term reference pictures ("STRPs") and inter-layer reference pictures ("ILRPs") are identified by their NAL (Network Abstraction Layer) unit ID and POC value. Long-term reference pictures ("LTRPs") are identified by their NAL unit ID and the number of LSBs of their POC value.

[0089]

[0132] 5A illustrates an example syntax including a syntax structure for a reference picture list according to some embodiments of the present disclosure. The syntax illustrated in FIG. 5A may be part of the VVC standard (e.g., VVC Draft 9) or may be included in other video coding techniques.

[0090]

[0133] As shown in FIG. 5A, a syntax structure 500A for reference picture lists (eg, ref_pic_lists()) may be within the PH syntax structure or the SH.

[0091]

[0134] 5A, syntax element 510A (e.g., rpl_sps_flag[i]) equal to 1 specifies that reference picture list i (e.g., i may be 0 or 1) in the syntax structure for reference picture lists (e.g., ref_pic_lists()) is derived based on one of the syntax structures for reference picture list structures in the SPS (e.g., ref_pic_list_struct(listIdx, rplsIdx) with listIdx equal to i). Syntax element 510A equal to 0 specifies that reference picture list i (e.g., i may be 0 or 1) is derived based on a syntax structure for reference picture list structures (e.g., ref_pic_list_struct(listIdx, rplsIdx) with listIdx equal to i) that is directly included in the syntax structure for reference picture lists (e.g., ref_pic_lists()).

[0092]

[0135] If syntax element 510A is absent, the following applies: First, if the number of reference picture lists in the SPS (e.g., sps_num_ref_pic_lists[i]) is equal to 0, then the value of syntax element 510A is inferred to be equal to 0. Second, if the number of reference picture lists in the SPS (e.g., sps_num_ref_pic_lists[i]) is not equal to 0 (e.g., if the number of reference picture lists in the SPS is greater than 0), then if syntax element 520A (e.g., pps_rpl1_idx_present_flag) is equal to 0 and i is equal to 1, then the value of syntax element 510A for reference picture list 1 in the SPS (e.g., rpl_sps_flag[1]) is inferred to be equal to the value of syntax element 510A for reference picture list 0 in the SPS (e.g., rpl_sps_flag[0]).

[0093]

[0136] Syntax element 530A (e.g., rpl_idx[i]) specifies an index into the list of ref_pic_list_struct(listIdx,rplsIdx) with listIdx equal to i contained in the SPS of ref_pic_list_struct(listIdx,rplsIdx) with listIdx equal to i that is used to derive reference picture list i for the current picture. The length of syntax element 530A is the number of bits having the smallest integer greater than or equal to the base 2 logarithm of the number of ref_pic_list_struct(listIdx,rplsIdx) syntax structures in the SPS, which can be represented by Ceil(Log2(sps_num_ref_pic_lists[i])) bits. The value of syntax element 530A can be in the range of 0 to the number of reference picture lists in the SPS minus 1 (e.g., sps_num_ref_pic_lists[i]-1), inclusive. When syntax element 530A is absent, if syntax element 510A is equal to 1 and syntax element 520A is equal to 0, the value of rpl_idx[1] is inferred to be equal to the value of rpl_idx[0], otherwise the value of rpl_idx[1] is inferred to be equal to 0.

[0094]

[0137] The variable RplsIdx[i] can be derived as follows: RplsIdx[i]=rpl_sps_flag[i]?rpl_idx[i]:sps_num_ref_pic_lists[i].

[0095]

[0138] Syntax element 540A (e.g., poc_lsb_lt[i][j]) specifies the value of the picture order count modulo MaxPicOrderCntLsb of the jth LTRP entry in the ith reference picture list in ref_pic_lists(). The length of syntax element 540A is equal to the base 2 logarithm of the largest LSB in POC bits (e.g., sps_log2_max_pic_order_cnt_lsb_minus4 + 4 bits).

[0096]

[0139] The variable PocLsbLt[i][j] can be derived as follows: PocLsbLt[i][j]=ltrp_in_header_flag[i][RplsIdx[i]]?poc_lsb_lt[i][j]:rpls_poc_lsb_lt[listIdx][RplsIdx[i]][j].

[0097]

[0140] A syntax element 550A (e.g., delta_poc_msb_cycle_present_flag[i][j]) equal to 1 specifies that a syntax element 560A (e.g., delta_poc_msb_cycle_lt[i][j]) is present. A syntax element 550A equal to 0 specifies that a syntax element 560A (e.g., delta_poc_msb_cycle_lt[i][j]) is absent.

[0098]

[0141] The previous picture in decoding order that has the same nuh_layer_id as the slice header or picture header referencing the ref_pic_lists() syntax structure, has TemporalID and ph_non_ref_pic_flag both equal to 0, and is not a RASL (Random Access Skip Reading) or RADL (Random Access Decodable Reading) picture can be represented as prevTid0Pic. nuh_layer_id is a syntax element that specifies the identifier of the layer to which a VCL (Video Coding Layer) NAL (Network Abstraction Layer) unit belongs, or the identifier of the layer to which a non-VCL NAL unit applies. TemporalID is the temporal identifier of the picture. The set of previous POC values, represented as setOfPrevPocVals, is a set that includes: -prevTid0Pic POC value (e.g. PicOrderCntVal) The POC value (e.g., PicOrderCntVal) of each picture referenced by an entry in Reference Picture List 0 (e.g., RefPicList[0]) or Reference Picture List 1 (e.g., RefPicList[1]) of prevTid0Pic and having the same nuh_layer_id as the current picture - The POC value (e.g., PicOrderCntVal) of each picture that follows prevTid0Pic in decoding order, has the same nuh_layer_id as the current picture, and precedes the current picture in decoding order.

[0099]

[0142] If there are multiple values ​​in setOfPrevPocVals whose value modulo MaxPicOrderCntLsb is equal to the variable PocLsbLt[i][j], then the value of syntax element 550A (eg, delta_poc_msb_cycle_present_flag[i][j]) is equal to one.

[0100]

[0143] 5B illustrates exemplary pseudocode including the derivation of a variable FullPocLt[i][j] according to some embodiments of the present disclosure. As shown in FIG. 5B, a syntax element 560A (e.g., delta_poc_msb_cycle_lt[i][j]) specifies the value of the variable FullPocLt[i][j]. The value of the syntax element 560A (e.g., delta_poc_msb_cycle_lt[i][j]) can be greater than or equal to 0, less than or equal to 2. (32-sps_log2_max_pic_order_cnt_lsb_minus4-4) It can be in the following range: If syntax element 560 is absent, the value of syntax element 560 is inferred to be equal to 0.

[0101]

[0144] 6A illustrates an example syntax including a syntax structure for a reference picture list structure according to some embodiments of the present disclosure. The syntax structure illustrated in FIG. 6A may be part of the VVC standard (e.g., VVC Draft 9) or may be included in other video coding techniques. As illustrated in FIG. 6A, ref_pic_list_struct(listIdx, rplsIdx) may be in an SPS, in a PH syntax structure, or in an SH. Depending on whether the syntax structure is included in an SPS, a PH syntax structure, or in an SH, the following applies: - If ref_pic_list_struct(listIdx,rplsIdx) is within a PH syntax structure or SH, the ref_pic_list_struct(listIdx,rplsIdx) syntax structure specifies the reference picture list listIdx of the current picture (e.g., the coded picture that contains the PH syntax structure or SH). -If ref_pic_list_struct(listIdx,rplsIdx) is not within a PH syntax structure or SH (e.g., it is within an SPS), then the ref_pic_list_struct(listIdx,rplsIdx) syntax structure specifies candidates for the reference picture list listIdx, and the term "current picture" in the semantics specified in the remainder of this section refers to each picture that: 1) has a PH syntax structure or one or more slices that includes an rpl_idx[listIdx] equal to an index into the list of the ref_pic_list_struct(listIdx,rplsIdx) syntax structure contained within the SPS, and 2) is within a coded layer-wise video sequence (CLVS) that references the SPS.

[0102]

[0145] 6A, syntax element 610A (e.g., num_ref_entries[listIdx][rplsIdx]) specifies the number of entries in the ref_pic_list_struct(listIdx,rplsIdx) syntax structure. The value of parameter 610A can be in the range of 0 to (MaxDpbSize+13), inclusive, where MaxDpbSize is as specified at the level of the video coding standard (e.g., VVC Draft 9).

[0103]

[0146] A syntax element 620A (e.g., ltrp_in_header_flag[listIdx][rplsIdx]) equal to 0 specifies that the POC LSBs of the LTRP entries indicated in the ref_pic_list_struct(listIdx, rplsIdx) syntax structure are in the same syntax structure. A syntax element 620A equal to 1 specifies that the POC LSBs of the LTRP entries indicated in the ref_pic_list_struct(listIdx, rplsIdx) syntax structure are not in the same syntax structure. If sps_long_term_ref_pics_flag is equal to 1 and rplsIdx is equal to sps_num_ref_pic_lists[listIdx], the value of syntax element 620A is inferred to be equal to 1.

[0104]

[0147] A syntax element 630A (e.g., inter_layer_ref_pic_flag[listIdx][rplsIdx][i]) equal to 1 specifies that the i-th entry in the ref_pic_list_struct(listIdx, rplsIdx) syntax structure is an ILRP entry. A syntax element 630A equal to 0 specifies that the i-th entry in the ref_pic_list_struct(listIdx, rplsIdx) syntax structure is not an ILRP entry. If the syntax element 630A is not present, the value of the syntax element 630A is inferred to be equal to 0.

[0105]

[0148] A syntax element 640A (e.g., st_ref_pic_flag[listIdx][rplsIdx][i]) equal to 1 specifies that the i-th entry in the ref_pic_list_struct(listIdx,rplsIdx) syntax structure is a STRP entry. A syntax element 640A equal to 0 specifies that the i-th entry in the ref_pic_list_struct(listIdx,rplsIdx) syntax structure is an LTRP entry. If syntax element 630A is equal to 0 and syntax element 640A is absent, the value of syntax element 640A is inferred to be equal to 1.

[0106]

[0149] 6B illustrates exemplary pseudocode including a derivation for the number of LTRP entries (e.g., the variable NumLtrpEntries[listIdx][rplsIdx]) according to some embodiments of the present disclosure. The variable NumLtrpEntries[listIdx][rplsIdx] (e.g., variable 570A in FIG. 5A) may be derived as shown in FIG. 6B.

[0107]

[0150] 6C illustrates exemplary pseudocode including the derivation of the variable AbsDeltaPocSt[listIdx][rplsIdx][i] according to some embodiments of the present disclosure. Syntax element 650A (e.g., abs_delta_poc_st[listIdx][rplsIdx][i]) specifies the value of the variable AbsDeltaPocSt[listIdx][rplsIdx][i] (e.g., variable 690A) as shown in FIG. 6C. The value of syntax element 650A (e.g., abs_delta_poc_st[listIdx][rplsIdx][i]) is greater than or equal to 0, and less than or equal to 2. 15 -1) It can be within the following range:

[0108]

[0151] A syntax element 660A (e.g., strp_entry_sign_flag[listIdx][rplsIdx][i]) equal to 1 specifies that the i-th entry in the ref_pic_list_struct(listIdx, rplsIdx) syntax structure has a value greater than or equal to 0. A syntax element 660A equal to 0 specifies that the i-th entry in the ref_pic_list_struct(listIdx, rplsIdx) syntax structure has a value less than 0. If syntax element 660A is absent, the value of syntax element 660A is inferred to be equal to 1.

[0109]

[0152] 6D illustrates exemplary pseudocode including the derivation of the variables DeltaPocValSt[listIdx][rplsIdx] according to some embodiments of the present disclosure. DeltaPocValSt[listIdx][rplsIdx] may be derived as shown in FIG. 6D.

[0110]

[0153] 6A, syntax element 670A (e.g., rpls_poc_lsb_lt[listIdx][rplsIdx][i]) specifies the value of the picture order count modulo MaxPicOrderCntLsb of the picture referenced by the i-th entry in the ref_pic_list_struct(listIdx, rplsIdx) syntax structure. The length of syntax element 670A is sps_log2_max_pic_order_cnt_lsb_minus4+4 bits.

[0111]

[0154] Syntax element 680A (e.g., ilrp_idx[listIdx][rplsIdx][i]) specifies an index into the list of direct reference layers of the ILRP of the i-th entry in the ref_pic_list_struct(listIdx, rplsIdx) syntax structure. The value of syntax element 680A can be in the range of 0 to (NumDirectRefLayers[GeneralLayerIdx[nuh_layer_id]]-1), inclusive, where NumDirectRefLayers[LayerIdx] means the number of direct reference layers of the layer with index equal to LayerIdx.

[0112]

[0155] 7 illustrates an example syntax including a syntax structure for a reference picture list structure in a sequence parameter set according to some embodiments of the present disclosure. The syntax illustrated in FIG. 7 may be part of the VVC standard (e.g., VVC Draft 9) or may be included in other video coding techniques.

[0113]

[0156] 7, a syntax element 710 (e.g., sps_long_term_ref_pics_flag) equal to 0 specifies that LTRP is not used for inter prediction of any coded picture in the CLVS. A syntax element 710 equal to 1 specifies that LTRP may be used for inter prediction of one or more coded pictures in the CLVS.

[0114]

[0157] A syntax element 720 (e.g., sps_inter_layer_ref_pics_present_flag) equal to 0 specifies that ILRP is not used for inter prediction of any coded pictures in a CLVS. A syntax element 720 equal to 1 specifies that ILRP may be used for inter prediction of one or more coded pictures in a CLVS. If sps_video_syntax element_set_id is equal to 0, i.e., the SPS does not reference a VPS (Video Parameter Set), and when decoding each CLVS that references an SPS, the value of syntax element 720 is inferred to be equal to 0 if the VPS is not referenced (there is only one layer). If vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is equal to 1, i.e., the layer with index GeneralLayerIdx[nuh_layer_id] does not use inter-layer prediction, the value of syntax element 720 is equal to 0.

[0115]

[0158] A syntax element 730 (e.g., sps_idr_rpl_present_flag) equal to 1 specifies the presence of a reference picture list syntax element in the slice header of an IDR (Instantaneous Decoding Refresh) picture. A syntax element 730 equal to 0 specifies the absence of a reference picture list syntax element in the slice header of an IDR picture.

[0116]

[0159] A syntax element 740 (e.g., sps_rpl1_same_as_rpl0_flag) equal to 1 specifies that the syntax element sps_num_ref_pic_lists[1] and syntax structure ref_pic_list_struct(1,rplsIdx) are absent and the following applies: the value of sps_num_ref_pic_lists[1] is inferred to be equal to the value of sps_num_ref_pic_lists[0], and the value of each syntax element in ref_pic_list_struct(1,rplsIdx) is inferred to be equal to the value of the corresponding syntax element in ref_pic_list_struct(0,rplsIdx) for rplsIdx ranging from 0 to sps_num_ref_pic_lists[0]-1.

[0117]

[0160] Syntax element 750 (e.g., sps_num_ref_pic_lists[i]) specifies the number of ref_pic_list_struct(listIdx, rplsIdx) syntax structures with listIdx equal to i contained in the SPS. The value of syntax element 750 can be in the range of 0 to 64, inclusive. For each value of listIdx (equal to 0 or 1), the decoder can allocate (e.g., by process 300A of FIG. 3A or process 300B of FIG. 3B) memory for ref_pic_list_struct(listIdx, rplsIdx) syntax structures with a total number of RPLs in the SPS plus 1 (e.g., sps_num_ref_pic_lists[i]+1), since there can be one ref_pic_list_struct(listIdx, rplsIdx) syntax structure signaled directly in the slice header of the current picture.

[0118]

[0161] 8 illustrates an example syntax including a syntax structure for a reference picture list in a picture parameter set according to some embodiments of the present disclosure. The syntax illustrated in FIG. 8 may be part of the VVC standard (e.g., VVC Draft 9) or may be included in other video coding techniques.

[0119]

[0162] 8, syntax element 810 (e.g., pps_num_ref_idx_default_active_minus1[i]) plus 1 specifies the inferred value of the variable NumRefIdxActive[0] for a P slice or a B slice, with sh_num_ref_idx_active_override_flag equal to 0 when i is equal to 0, i.e., for reference picture list 0. Syntax element 810 plus 1 specifies the inferred value of the variable NumRefIdxActive[1] for a B slice, with sh_num_ref_idx_active_override_flag equal to 0 when i is equal to 1, i.e., for reference picture list 1. The value of syntax element 810 can be in the range of 0 to 14, inclusive.

[0120]

[0163] A syntax element 820 (e.g., pps_rpl1_idx_present_flag) equal to 0 specifies that rpl_sps_flag[1] and rpl_idx[1] are not present in the PH syntax structure or slice header of the picture that references the PPS. A syntax element 820 equal to 1 specifies that rpl_sps_flag[1] and rpl_idx[1] may be present in the PH syntax structure or slice header of the picture that references the PPS.

[0121]

[0164] A syntax element 830 (e.g., pps_rpl_info_in_ph_flag) equal to 1 specifies that reference picture list information is present in the PH syntax structure and not in slice headers that reference PPSs that do not contain PH syntax structures. A syntax element 830 equal to 0 specifies that reference picture list information is not present in the PH syntax structure and may be present in slice headers that reference PPSs. If the syntax element 830 is not present, the value of the syntax element 830 is inferred to be equal to 0.

[0122]

[0165] 9A shows an example syntax including a syntax structure for a reference picture list in a picture header structure according to some embodiments of the present disclosure. The syntax shown in FIG. 9A may be part of the VVC standard (e.g., VVC Draft 9) or may be included in other video coding techniques.

[0123]

[0166] As shown in Figure 9A, syntax elements 910A (e.g., ph_pic_output_flag) affect the output and removal process of decoded pictures. If syntax element 910A is absent, it is inferred that syntax element 910A is equal to 1. There are no pictures in the bitstream with ph_non_reference_picture_flag equal to 1 and syntax element 910A equal to 0. The element ph_non_reference_picture_flag equal to 1 specifies that the current picture is never used as a reference picture. The element ph_non_ref_pic_flag equal to 0 specifies that the current picture may or may not be used as a reference picture.

[0124]

[0167] A syntax element 920A (e.g., ph_temporal_mvp_enabled_flag) equal to 0 specifies that a temporal motion vector predictor is disabled and not used in decoding slices in the current picture. A syntax element 920A equal to 1 specifies that a temporal motion vector predictor is enabled and may be used in decoding slices in the current picture. If syntax element 920A is absent, the value of syntax element 920A is inferred to be equal to 0. Due to other existing constraints, the value of syntax element 920A can only be equal to 0 in a compliant bitstream when one or more of the following conditions are true: 1) no reference picture in the DPB has the same spatial resolution and the same scaling window offset as the current picture, and 2) no reference picture in the DPB is in the active entry of the RPL of any slice in the current picture. There may be other unlisted situations, complex conditions, in which syntax element 920A can only be equal to 0.

[0125]

[0168] 9B shows exemplary pseudocode including the derivation of the variable MaxNumSubblockMergeCand according to some embodiments of the present disclosure. As shown in FIG. 9B, the value of MaxNumSubblockMergeCand refers to the maximum number of subblock-based merging MVP (motion vector predictor) candidates. The value of MaxNumSubblockMergeCand can be in the range of 0 to 5, inclusive.

[0126]

[0169] Referring again to Figure 9A, a syntax element 930A (e.g., ph_collocated_from_l0_flag) equal to 1 specifies that the collocated picture used for temporal motion vector prediction is derived from reference picture list 0. A syntax element 930A equal to 0 specifies that the collocated picture used for temporal motion vector prediction is derived from reference picture list 1. If syntax element 920A and syntax element 830A (e.g., pps_rpl_info_in_ph_flag) are both equal to 1 and num_ref_entries[1][RplsIdx[1]] is equal to 0, then the value of syntax element 930A is inferred to be equal to 1.

[0127]

[0170] Syntax element 940A (e.g., ph_collocated_ref_idx) specifies the reference index of the collocated picture used for temporal motion vector prediction. If syntax element 930A is equal to 1, syntax element 940A references an entry in reference picture list 0, and the value of syntax element 940A may be in the range of 0 or greater and (num_ref_entries[0][RplsIdx[0]]-1), inclusive. If syntax element 930A is equal to 0, syntax element 940A references an entry in reference picture list 1, and the value of syntax element 940A may be in the range of 0 or greater and (num_ref_entries[1][RplsIdx[1]]-1), inclusive. If syntax element 940A is absent, the value of syntax element 940A is inferred to be equal to 0.

[0128]

[0171] A syntax element 950A (e.g., ph_mvd_l1_zero_flag) equal to 1 specifies that the motion vector difference (e.g., mvd_coding(x0, y0, 1, cpIdx)) syntax structure is not parsed, and MvdL1[x0][y0][compIdx] and MvdCpL1[x0][y0][cpIdx][compIdx] are set equal to 0 for compIdx=0..1 and cpIdx=0..2. A syntax element 950A equal to 0 specifies that the mvd_coding(x0, y0, 1, cpIdx) syntax structure is parsed. If syntax element 950A is absent, the value of syntax element 950A is inferred to be 1. MvdL1 is the motion vector difference decoded from the bitstream associated with the reference picture in reference picture list 1. MvdCpL1 is the control point motion vector difference decoded from the bitstream relative to the reference picture in reference picture list 1. This is for coding blocks that use affine motion compensation. x0,y0 is the top-left position of the current coding block, compIdx is the component index, and cpIdx is the index of the control point.

[0129]

[0172] 10A shows an example syntax including a syntax structure for a reference picture list in a slice header according to some embodiments of the present disclosure. The syntax shown in FIG. 10A may be part of the VVC standard (e.g., VVC Draft 9) or may be included in other video coding techniques.

[0130]

[0173] 10A, a syntax element 1010A (e.g., sh_num_ref_idx_active_override_flag) equal to 1 specifies that the syntax element sh_num_ref_idx_active_minus1[0] is present for P slices and B slices, and the syntax element sh_num_ref_idx_active_minus1[1] is present for B slices. A syntax element 1010A equal to 0 specifies that the syntax elements sh_num_ref_idx_active_minus1[0] and sh_num_ref_idx_active_minus1[1] are absent. If the syntax element 1010A is absent, the value of the syntax element 1010A is inferred to be equal to 1.

[0131]

[0174] Syntax element 1020A (e.g., sh_num_ref_idx_active_minus1[i]) is used to derive the variable NumRefIdxActive[i]. The value of syntax element 1020A can be in the range of 0 to 14, inclusive. For i equal to 0 or 1, syntax element 1010A is equal to 1 when the current slice is a B slice, and if syntax element 1020A is absent, syntax element 1020A is inferred to be equal to 0.

[0132]

[0175] 10B shows exemplary pseudocode including the derivation of the variable NumRefIdxActive[i] according to some embodiments of the present disclosure. As shown in FIG. 10B, the value of NumRefIdxActive[i]-1 specifies the maximum reference index for reference picture list i that can be used to decode a slice. As shown by equation (1) in FIG. 10B, syntax element 1020A is used to derive NumRefIdxActive[i]. If the value of NumRefIdxActive[i] is equal to 0, no reference index for reference picture list i can be used to decode the slice. If the current slice is a P slice, the value of NumRefIdxActive[0] is greater than 0. If the current slice is a B slice, both NumRefIdxActive[0] and NumRefIdxActive[1] are greater than 0.

[0133]

[0176] 10A, syntax element 1030A (e.g., sh_cabac_init_flag) specifies a method for determining the initialization table to be used in the initialization process for a context variable. If syntax element 1030A is not present, syntax element 1030A is inferred to be equal to 0.

[0134]

[0177] A syntax element 1040A (e.g., sh_collocated_from_l0_flag) equal to 1 specifies that the collocated picture used for temporal motion vector prediction is derived from reference picture list 0. A syntax element 1040A equal to 0 specifies that the collocated picture used for temporal motion vector prediction is derived from reference picture list 1. If sh_slice_type is equal to B or P, syntax element 920A (e.g., ph_temporal_mvp_enabled_flag) is equal to 1, syntax element 1040A is not present, and the following applies: if sh_slice_type is equal to B, syntax element 1040A is inferred to be equal to syntax element 930A (e.g., ph_collocated_from_l0_flag), and if sh_slice_type is not equal to P (e.g., sh_slice_type is equal to P), the value of syntax element 1040A is inferred to be equal to 1.

[0135]

[0178] Syntax element 1050A (e.g., sh_collocated_ref_idx) specifies the reference index of the collocated picture used for temporal motion vector prediction. If sh_slice_type is equal to P or if sh_slice_type is equal to B and syntax element 1040A is equal to 1, syntax element 1050A references an entry in reference picture list 0, and the value of syntax element 1050A may be in the range of 0 to (NumRefIdxActive[0]-1), inclusive. If sh_slice_type is equal to B and syntax element 1040A is equal to 0, syntax element 1050A references an entry in reference picture list 1, and the value of syntax element 1050A may be in the range of 0 to (NumRefIdxActive[1]-1), inclusive. In the absence of syntax element 1050A, the following applies: if syntax element 830 (e.g., pps_rpl_info_in_ph_flag) is equal to 1, the value of syntax element 1050A is inferred to be equal to syntax element 940A (e.g., ph_collocated_ref_idx); if syntax element 830 is not equal to 1 (e.g., syntax element 830 is equal to 0), the value of syntax element 1050A is inferred to be equal to 0. It is a bitstream-compliant requirement that the picture referenced by syntax element 1050A is the same for all slices of a coded picture and that RprConstraintsActive[sh_collocated_from_l0_flag?0:1][sh_collocated_ref_idx] is equal to 0. This constraint requires that the collocated picture have the same spatial resolution and the same scaling window offset as the current picture.

[0136]

[0179] In VVC (e.g., VVC Draft 9), syntax element 930A (e.g., ph_collocated_from_l0_flag) and syntax element 950A (e.g., ph_mvd_l1_zero_flag) are two flags signaled within PH. Syntax element 930A indicates which reference picture list the collocated picture used for temporal motion vector prediction is from. Syntax element 950A indicates whether the mvd_coding() syntax structure is parsed for reference picture list 1. As a result, these two flags are only relevant if the number of active entries in reference picture list 1 is greater than 0. However, as shown in Figure 10A, because the number of active entries in a reference picture list is overridden in the slice header by sh_num_ref_idx_active_minus1[i], when syntax element 930A and syntax element 950A are signaled in the PH, the decoder has no knowledge of the exact number of active entries in reference picture list 1. Therefore, in VVC (e.g., VVC Draft 9), the total number of entries in reference picture list 1 is used as the condition for signaling these two flags, as shown in Figure 9A.

[0137]

[0180] Although this disclosure refers to various syntax elements that provide inferences based on values ​​being equal to 0 or 1, it will be understood that values ​​may be configured in any manner (e.g., 1 or 0) to provide the appropriate inference.

[0138]

[0181] In VVC (e.g., VVC Draft 9), it is guaranteed that the number of active entries in both two reference picture lists is equal to 0 for an I slice. For a P slice, the number of active entries in reference picture list 0 is greater than 0, and the number of active entries in reference picture list 1 is equal to 0. For a B slice, the number of active entries in both two reference picture lists is greater than 0. There is no guarantee regarding the total number of entries in the reference picture lists. For example, for an I slice, the number of entries in either of the two reference picture lists may be greater than 0. As a result, the signaling condition that the total number of entries in reference picture list 1 is greater than 0 is too relaxed for syntax element 930A and syntax element 950A, causing unnecessary signaling of those two syntax elements.

[0139]

[0182] To overcome this drawback in conventional encoding techniques, some embodiments of the present disclosure (as shown below in Figures 11A to 11C) avoid unnecessary signaling when the number of entries in reference picture list 0 is equal to 0.

[0140]

[0183] FIG. 11A shows a flowchart of an exemplary video encoding method 1100A for signaling a flag in a PH syntax structure according to some embodiments of the present disclosure. Method 1100A may be performed by an encoder (e.g., by process 200A of FIG. 2A or process 200B of FIG. 2B) or by one or more software or hardware components of an apparatus (e.g., apparatus 400 of FIG. 4). For example, one or more processors (e.g., processor 402 of FIG. 4) may perform method 1100A. In some embodiments, method 1100A may be implemented by a computer program product embodied in a computer-readable medium that includes computer-executable instructions, such as program code, for execution by a computer (e.g., apparatus 400 of FIG. 4). Referring to FIG. 11A, method 1100A may include the following steps 1102A and 1104A.

[0141]

[0184] In step 1102A, the encoder encodes the current picture based on the co-located picture. For example, the reference picture can be derived by reference picture 0 and reference picture list 1, each of which contains a list of reconstructed pictures in the DPB (e.g., buffer 234 in FIG. 3B) to be used as reference pictures. The current picture is used for temporal motion vector prediction.

[0142]

[0185] In step 1104A, if the number of entries in reference picture list 0 and the number of entries in reference picture list 1 are both greater than 0, the syntax element ph_collocated_from_l0_flag (e.g., syntax element 930A) and the syntax element ph_mvd_l1_zero_flag (e.g., syntax element 950A) are signaled. The syntax element ph_collocated_from_l0_flag indicates which reference picture list the collocated picture used for temporal motion vector prediction comes from, i.e., the collocated picture used for temporal motion vector prediction comes from the reference picture list indicated by the first flag. The syntax element ph_mvd_l1_zero_flag indicates whether a motion vector difference syntax structure related to reference picture list 1 is signaled. In this way, when signaling the two flags, entries in both reference picture list 1 and reference picture 0 are guaranteed. Therefore, unnecessary signaling when the number of entries in reference picture list 0 is equal to 0 is avoided, and decoding efficiency is improved.

[0143]

[0186] FIG. 11B shows a flowchart of an exemplary video decoding method 1100B for decoding flags in a PH syntax structure according to some embodiments of the present disclosure. Method 1100B may be performed by a decoder (e.g., by process 300A of FIG. 3A or process 300B of FIG. 3B) or by one or more software or hardware components of an apparatus (e.g., apparatus 400 of FIG. 4). For example, one or more processors (e.g., processor 402 of FIG. 4) may perform method 1100B. In some embodiments, method 1100B may be implemented by a computer program product embodied in a computer-readable medium that includes computer-executable instructions, such as program code, for execution by a computer (e.g., apparatus 400 of FIG. 4). Referring to FIG. 11B, method 1100B may include the following steps 1102B-1106B.

[0144]

[0187] In step 1102B, a decoder receives a video bitstream (eg, video bitstream 228 of FIG. 3B), which may be encoded using inter prediction.

[0145]

[0188] In step 1104B, if the number of entries in reference picture list 0 and the number of entries in reference picture list 1 are both greater than 0, the decoder decodes from the bitstream the syntax element ph_collocated_from_l0_flag (e.g., syntax element 930A) and the syntax element ph_mvd_l1_zero_flag (e.g., syntax element 950A). The syntax element ph_collocated_from_l0_flag indicates which reference picture list the collocated picture used for temporal motion vector prediction comes from, i.e., the collocated picture used for temporal motion vector prediction comes from the reference picture list indicated by the first flag. The syntax element ph_mvd_l1_zero_flag indicates whether the motion vector difference syntax structure associated with reference picture list 1 is present in the bitstream. In this way, when signaling the two flags, entries in both reference picture list 1 and reference picture 0 are guaranteed.

[0146]

[0189] In step 1106B, the current picture is decoded based on the co-located picture, thus avoiding unnecessary signaling when the number of entries in reference picture list 0 is equal to 0, and improving efficiency.

[0147]

[0190] 11C illustrates a portion of an exemplary picture header syntax structure 1100C according to some embodiments of the present disclosure. Picture header (PH) syntax structure 1100C may be used within method 1100A. PH syntax structure 1100B is modified based on syntax structure 900A of FIG. 9A, with changes from previous VVCs indicated in italics within blocks 1110C and 1120C.

[0148]

[0191] See 1110C, and in some embodiments, the syntax element ph_collocated_from_l0_flag (e.g., syntax element 930A) is signaled if num_ref_entries[0][RplsIdx[0]] is greater than 0 and num_refe_entries[1][RplsIdx[1]] is greater than 0. See 1120C, and the syntax element ph_mvd_l1_zero_flag (e.g., syntax element 950A) is signaled if pps_rpl_info_in_ph_flag is not equal to 0 or num_ref_entries[0][RplsIdx[0]] is greater than 0, with num_refe_entries[1][RplsIdx[1]] greater than 0. Therefore, syntax element 930A and syntax element 950A may be signaled when the number of entries in reference picture list 0 and the number of reference picture list 1 are both greater than 0. Unnecessary signaling when the number of entries in reference picture list 0 is equal to 0 is avoided, and coding efficiency is improved.

[0149]

[0192] In VVC (e.g., VVC Draft 9), a collocated picture can be indicated in a PH or an SH. When reference picture list information is signaled in a PH, a collocated picture is indicated in the PH by syntax element 930A (e.g., ph_collocated_from_l0_flag) and syntax element 940A (e.g., ph_collocated_ref_idx). When reference picture list information is signaled in an SH, a collocated picture is indicated in the SH by syntax element 1040A (e.g., sh_collocated_from_l0_flag) and syntax element 1050A (e.g., sh_collocated_ref_idx). A syntax element 930A equal to 1 specifies that a collocated picture used for temporal motion vector prediction is derived from reference picture list 0. A syntax element 930A equal to 0 specifies that a collocated picture used for temporal motion vector prediction is derived from reference picture list 1. When syntax element 930A is signaled within PH, the signaling condition is that the number of entries in reference picture list 1 is greater than 0. However, the number of active entries in a reference picture list can be overridden within the slice level. Therefore, even if syntax element 930A is signaled to be 0, it cannot be guaranteed that a co-located picture can be selected from reference picture list 1, because SH may override the number of active entries in reference picture list 1 to be 0. Similarly, when syntax element 940A is signaled within PH, the maximum allowed value is the number of entries in the reference picture list minus 1. If SH overrides the number of active entries to a value less than syntax element 940A, it is an invalid bitstream.

[0150]

[0193] To avoid such illegal scenarios, VVC (e.g., VVC Draft 9) imposes some bitstream compliance constraints. However, such constraints put a burden on the encoder to satisfy all the constraints. In practice, the decoder should also consider how to handle the bitstream when such illegal cases occur.

[0151]

[0194] To overcome this drawback in conventional encoding techniques, some embodiments of the present disclosure (as shown below in Figures 12A-12J) indicate co-located pictures without signaling an index to a reference picture list, thereby avoiding illegal scenarios in a more robust manner.

[0152]

[0195] FIG. 12A shows a flowchart of an exemplary video encoding method 1200A for indicating a co-located picture without signaling an index to a reference picture list, according to some embodiments of the present disclosure. Method 1200A may be performed by an encoder (e.g., by process 200A of FIG. 2A or process 200B of FIG. 2B) or by one or more software or hardware components of an apparatus (e.g., apparatus 400 of FIG. 4). For example, one or more processors (e.g., processor 402 of FIG. 4) may perform method 1200A. In some embodiments, method 1200A may be implemented by a computer program product embodied in a computer-readable medium that includes computer-executable instructions, such as program code, for execution by a computer (e.g., apparatus 400 of FIG. 4). Referring to FIG. 12A, method 1200A may include the following steps 1202A and 1204A.

[0153]

[0196] In step 1202A, the encoder encodes the current picture into a bitstream based on the co-located picture, and the co-located picture is used for temporal motion vector prediction. In step 1204A, the co-located picture is indicated in the bitstream without signaling an index of the reference picture list. Because the co-located picture is indicated by an index without referring to an entry in the reference picture list, the co-located picture can be legitimately indicated even if the SH overrides the number of active entries in reference picture list 1 to be 0. Therefore, the robustness of the encoding process is improved.

[0154]

[0197] 12B shows an example flowchart of an encoding method 1200B according to some embodiments of the present disclosure. It will be understood that method 1200B may be part of step 1204 within method 1200A of FIG. 12A. FIG. 12C shows another flowchart of an example video encoding method 1200B for showing co-located pictures according to some embodiments of the present disclosure. Referring to FIG. 12B and FIG. 12C, in some embodiments, method 1200B may further include the following steps 1202B-1206B.

[0155]

[0198] In step 1202B, if the co-located picture is an inter-layer reference picture, a first parameter is signaled to indicate the co-located picture. The first parameter indicates the index of the co-located picture relative to the list of direct reference layers of the layer in which the current picture resides. For example, the index may be the syntax element inter_layer_col_pic_idx. Thus, the co-located picture is indicated without using a reference picture list. This can avoid an invalid scenario when the SH overrides the number of active entries in the reference picture list. Before step 1202B, a flag indicating whether the co-located picture is an inter-layer reference picture can be signaled. Step 1202B can also be referred to as 1201C and 1202C in FIG. 12C.

[0156]

[0199] In step 1204B, if the co-located picture is a short-term reference picture (STRP), a delta picture order count (delta POC) is signaled. Furthermore, the delta POC can be used to derive the POC. In this scenario, the co-located picture is indicated using the POC without using a reference picture list. Therefore, an illegal scenario can be avoided when the SH overrides the number of active entries in the reference picture list. Step 1204B can also be referred to as 1203C and 1204C in Figure 12C.

[0157]

[0200] In step 1206B, if the co-located picture is a long-term reference picture (LTRP), the least significant bit (LSB) and the most significant bit (MSB) of the POC are signaled. Furthermore, the POC can be derived by the LSB and MSB. In this scenario, the co-located picture is indicated using the POC without using a reference picture list. Therefore, an illegal scenario in which the SH overrides the number of active entries in the reference picture list can be avoided. Step 1206B can also be referred to as 1203C and 1205C in FIG. 12C. Using the POC to indicate the co-located picture can effectively increase the robustness for determining the co-located picture. In some embodiments, a flag indicating whether the co-located picture is a short-term reference picture can be signaled before steps 1204B and 1206B.

[0158]

[0201] FIG. 12D shows a flowchart of an example video decoding method 1200D for representing a co-located picture without decoding an index to a reference picture list, according to some embodiments of the present disclosure. Method 1200D may be performed by a decoder (e.g., by process 300A of FIG. 3A or process 300B of FIG. 3B) or by one or more software or hardware components of an apparatus (e.g., apparatus 400 of FIG. 4). For example, one or more processors (e.g., processor 402 of FIG. 4) may perform method 1200D. In some embodiments, method 1200D may be implemented by a computer program product embodied in a computer-readable medium that includes computer-executable instructions, such as program code, for execution by a computer (e.g., apparatus 400 of FIG. 4). Referring to FIG. 12D, method 1200D may include the following steps 1202D-1206D:

[0159]

[0202] In step 1202D, the decoder receives a video bitstream for processing (e.g., video bitstream 228 in FIG. 3B), which may be encoded using inter prediction. Reference pictures may be derived, for example, by reference picture 0 and reference picture list 1, each of which contains a list of reconstructed pictures in the DPB (e.g., buffer 234 in FIG. 3B) to be used as reference pictures.

[0160]

[0203] In step 1204D, the co-located picture used for temporal motion vector prediction is determined based on the bitstream without decoding an index to the reference picture list.

[0161]

[0204] In step 1206D, the current picture is decoded based on the co-located picture. Since the co-located picture is represented without using the reference picture list structure, the co-located picture can be represented legitimately even if SH overrides the number of active entries in reference picture list 1 to be 0. Therefore, the robustness of the decoding process is improved.

[0162]

[0205] Since the co-located picture is represented without using the reference picture list structure, the co-located picture can be represented legally even if the SH overrides the number of active entries in the reference picture list 1 to be 0. Thus, the robustness of the decoding process is improved.

[0163]

[0206] 12E shows an example flowchart of a decoding method 1200E according to some embodiments of the present disclosure. It will be understood that the method 1200E may be part of step 1204D within the method 1200D of FIG. 12D.

[0164]

[0207] In step 1202E, if the co-located picture is an inter-layer reference picture, a first parameter for indicating the co-located picture is decoded. The first parameter indicates the index of the co-located picture relative to the list of direct reference layers of the layer in which the current picture resides. For example, the index may be the syntax element inter_layer_col_pic_idx. Thus, the co-located picture is indicated without using a reference picture list. This can avoid an invalid scenario when the SH overrides the number of active entries in the reference picture list. In some embodiments, before step 1202E, a first flag indicating whether the co-located picture is an inter-layer reference picture is decoded, and whether the co-located picture is an inter-layer reference picture is determined based on the first flag.

[0165]

[0208] In step 1204E, if the co-located picture is a short-term reference picture (STRP), decode a delta picture order count (delta POC). Furthermore, the delta POC can be used to derive the POC. In this scenario, the co-located picture is indicated using the POC without using a reference picture list. Therefore, an illegal scenario can be avoided when the SH overrides the number of active entries in the reference picture list.

[0166]

[0209] In step 1206E, if the co-located picture is a long-term reference picture (LTRP), the least significant bit (LSB) and the most significant bit (MSB) of the POC are decoded. Furthermore, the POC can be derived by the LSB and the MSB. In this scenario, the co-located picture is indicated using the POC without using a reference picture list. Therefore, an illegal scenario when the SH overrides the number of active entries in the reference picture list can be avoided. In some embodiments, before steps 1204E and 1206E, a second flag indicating whether the co-located picture is a short-term reference picture is decoded, and whether the co-located picture is a short-term reference picture is determined based on the second flag.

[0167]

[0210] 12F and 12G illustrate a portion of an example picture parameter set syntax structure 1200F and a portion of an example slice header syntax structure 1200G, according to some embodiments of the present disclosure. Picture parameter set syntax structure 1200F, along with slice header syntax structure 1200G, can be used within methods 1200A, 1200B, 1200D, and 1200E. Picture parameter set syntax structure 1200F is modified based on portion 960A of syntax structure 900A of FIG. 9A , with changes from previous VVC syntax indicated in italics and suggested deletion syntax further indicated by strikethrough. Slice header syntax structure 1200G is modified based on portion 1060A of syntax structure 1000A of FIG. 10A , with suggested deletion syntax further indicated by strikethrough. As shown in Figures 12F and 12G, the syntax elements ph_collocated_from_l0_flag, ph_collocated_ref_idx, sh_collocated_from_l0_flag, and sh_collocated_ref_idx are no longer signaled within the PPS or SH.

[0168]

[0211] As shown in FIG. 12F, a syntax element 1210F (e.g., inter_layer_col_pic_flag) equal to 1 specifies that the co-located picture used for temporal motion vector prediction is referenced by an ILRP entry in the reference picture list. A syntax element 1210F equal to 0 specifies that the co-located picture used for temporal motion vector prediction is not referenced by an ILRP entry in the reference picture list. In the absence of syntax element 1210F, the value of syntax element 1210F is inferred to be equal to 0. Syntax element 1210F may be signaled within 1201C for determining whether the co-located picture is an inter-layer reference picture.

[0169]

[0212] A syntax element 1220F (e.g., st_col_pic_flag) equal to 1 specifies that the co-located picture used for temporal motion vector prediction is referenced by a STRP entry in the reference picture list. A syntax element 1220F equal to 0 specifies that the co-located picture used for temporal motion vector prediction is referenced by an LTRP entry in the reference picture list. If syntax element 1210F is equal to 0 and syntax element 1220F is absent, the value of syntax element 1220F is inferred to be equal to 1. Syntax element 1220F may be signaled in 1203C for determining whether the co-located picture is a short-term reference picture. If syntax element 1220F is equal to 1 (e.g., 1203C of FIG. 12C is true), step 1204B (shown in FIG. 12B) is processed and the delta picture order count (delta POC) is signaled (e.g., in 1204C of FIG. 12C). If syntax element 1220F is equal to 0 (e.g., 1203C of FIG. 12C is false), step 1206B (shown in FIG. 12B) is processed and the least significant bit (LSB) of the POC and the most significant bit (MSB) of the POC are signaled (e.g., in 1205C of FIG. 12C).

[0170]

[0213] Syntax element 1230F (e.g., abs_delta_poc_st_col) specifies the value of the variable AbsDeltaPocStCol. FIG. 12H shows an example of pseudocode including the derivation of AbsDeltaPocStCol according to some embodiments of the present disclosure. The value of syntax element 1230F (e.g., abs_delta_poc_st_col) is greater than or equal to 0 and less than or equal to 2. 15 -1) It can be within the following range:

[0171]

[0214] 12F, a syntax element 1240F (e.g., sign_delta_poc_st_col_flag) equal to 1 specifies that the value of the variable DeltaPocValStCol is greater than or equal to 0. A syntax element 1240F equal to 0 specifies that the value of the variable DeltaPocValStCol is less than 0. If the syntax element 1240F is absent, the value of the syntax element 1240F is inferred to be equal to 1. FIG. 12I shows an example of pseudocode including the derivation of DeltaPocValStCol according to some embodiments of the present disclosure. The variable DeltaPocValStCol can be derived as shown in FIG. 12I.

[0172]

[0215] 12F, in some embodiments, a syntax element 1250F (e.g., poc_lsb_lt_col) specifies the value of the picture order count modulo MaxPicOrderCntLsb of collocated pictures used for temporal motion vector prediction. The length of syntax element 1250F is sps_log2_max_pic_order_cnt_lsb_minus4+4 bits.

[0173]

[0216] Syntax element 1260F (e.g., delta_poc_msb_cycle_lt_col) specifies the value of the variable FullPocLtCol as follows: FullPocLtCol=PicOrderCntVal-delta_poc_msb_cycle_lt_col * MaxPicOrderCntLsb (PicOrderCntVal&(MaxPicOrderCntLsb-1))+poc_lsb_lt_col

[0174]

[0217] A syntax element 1270F (e.g., delta_poc_msb_cycle_col_present_flag) equal to 1 specifies that the syntax element 1260F (e.g., delta_poc_msb_cycle_lt_col) is present. A syntax element 1270B equal to 0 specifies that the syntax element 1260F is absent.

[0175]

[0218] Further assume for syntax element 1270F that prevTid0Pic is the previous picture in decoding order that has the same nuh_layer_id as the slice or picture header that references the ref_pic_lists() syntax structure, has TemporalId equal to 0, and is not a RASL or RADL picture. Assume that setOfPrevPocVals is a set consisting of: -PicOrderCntVal of prevTid0Pic - PicOrderCntVal of each picture referenced by an entry in RefPicList[0] or RefPicList[1] of prevTid0Pic and having the same nuh_layer_id as the current picture - The PicOrderCntVal of each picture that follows prevTid0Pic in decoding order, has the same nuh_layer_id as the current picture, and precedes the current picture in decoding order.

[0176]

[0219] If there are multiple values ​​in setOfPrevPocVals and the value modulo MaxPicOrderCntLsb is equal to syntax element 1250F (eg, poc_lsb_lt_col), the value of delta_poc_msb_cycle_present_flag[i][j] shall be equal to 1.

[0177]

[0220] Syntax element 1280F (e.g., inter_layer_col_pic_idx) specifies an index into the list of direct reference layers of the co-located picture used for temporal motion vector prediction when the co-located picture used for temporal motion vector prediction is referenced by an ILRP entry in a reference picture list. The value of syntax element 1280F can be in the range of 0 to (NumDirectRefLayers[GeneralLayerIdx[nuh_layer_id]]-1), inclusive.

[0178]

[0221] As shown in Figure 12F, if sps_inter_layer_ref_pics_present_flag (e.g., syntax element 720) is equal to 1, syntax element 1210F is signaled, i.e., ILRP may be used for inter prediction of one or more coded pictures in CLVS, and an index to indicate which inter layer reference picture is treated as the co-located picture (e.g., syntax element 1280F inter_layer_col_pic_idx) is signaled, which corresponds to step 1202B in Figure 12B. If the co-located picture is a short-term reference picture, i.e., syntax element 1220F (e.g., st_col_pic_flag) is equal to 1, delta POC (e.g., syntax element 1230F) is signaled, which corresponds to step 1204B in Figure 12B. If the co-located picture is a long-term reference picture, that is, if syntax element 1220F (e.g., st_col_pic_flag) is equal to 0, then the LSB of the POC (e.g., syntax element 1250F) and the delta MSB of the POC (e.g., syntax element 1260F) are signaled, which corresponds to step 1206B in FIG. 12B. Furthermore, the MSB of the POC may be derived by the delta MSB, and the POC may be derived by the MSB and LSB. Thus, the co-located picture can be indicated independently of the reference picture list structure.

[0179]

[0222] Considering that VVC (e.g., VVC Draft 9) has a constraint that the co-located pictures referenced by all slices in a picture should be the same picture, the updated syntax structures 1200F and 1200G allow a co-located picture to be indicated only in the PH, not in the SH. As a result, it can be guaranteed that all slices in a picture have the same co-located picture, and no constraint is necessary, thus increasing the efficiency and robustness for indicating a co-located picture.

[0180]

[0223] 12J shows an example of pseudocode for deriving a collocated picture, denoted as colPic, and a flag, colPicFlag, used in methods 1200A, 1200B, 1200C, and 1200D. As shown in FIG. 12J, for various scenarios of collocated pictures, such as a collocated picture referenced by a STRP entry in the reference picture list (as shown in scenario 1210J), a collocated picture referenced by an LTRP entry in the reference picture list (as shown in scenario 1220J), or a collocated picture referenced by an ILPR entry in the reference picture list (as shown in scenario 1230J), all slices in a picture have the same collocated picture (e.g., picA). Therefore, robustness for determining the collocated picture is improved.

[0181]

[0224] In some embodiments, there is a bitstream compliance requirement that enforces the following constraint: when ph_temporal_mvp_enabled_flag is equal to 1, colPic is not a "non-reference picture", is referenced by an active entry in RefPicList[0] or RefPicList[1], and colPicFlag is equal to 0. A "non-reference picture" can be considered as a mark to indicate that there is no reference picture in the RPL. colPicFlag equal to 0 indicates that the current picture and the co-located picture have the same picture size and the same scaling window. In other words, when temporal MVP is enabled, the co-located picture should be in a reference picture list and be referenced by an active entry in Reference Picture List 0 or Reference Picture List 1. Therefore, robustness for co-located pictures is improved.

[0182]

[0225] In VVC (e.g., VVC Draft 9), ref_pic_list_struct() and syntax elements used to identify collocated pictures (e.g., syntax element 930A (e.g., ph_collocated_from_l0_flag) and syntax element 940A (e.g., ph_collocated_ref_idx) in the PH and syntax element 1040A (e.g., sh_collocated_from_l0_flag) and syntax element 1050A (e.g., sh_colocated_ref_idx) in the SH) can be signaled within the PH or the SH depending on the value of pps_rpl_info_ph_flag. When the value of pps_rpl_info_ph_flag is equal to 1, syntax element 930A, syntax element 940A, and ref_pic_list_struct() are signaled within the PH, and syntax element 1040A and syntax element 1050A are not signaled. In this case, the values ​​of syntax element 1040A and syntax element 1050A are inferred depending on syntax element 930A, syntax element 940A, and the slice type of the current slice. If it is a B slice, syntax element 1040A is inferred to be equal to syntax element 930A. If it is a P slice, syntax element 1040A is directly inferred to be equal to 1, regardless of the value of syntax element 930A. Syntax element 1050A is inferred to be equal to syntax element 940A for both P slices and B slices. However, for syntax element 940A signaled in the PH, the maximum allowed value is the number of entries in the reference picture list minus 1, while for syntax element 1050A, the maximum allowed value is the number of active entries in the reference picture list, which may be overridden in the slice header, minus 1. As a result, if syntax element 1050A is inferred to be equal to syntax element 940A, such inference may violate the maximum value constraint.

[0183]

[0226] For example, if syntax element 930A is signaled as 0, the number of entries in reference picture list 1 (num_ref_entries[1] signaled in ref_pic_list_structure()) is N, and ph_colocated_ref_idx is signaled as N-1, then syntax element 1040A is inferred to be equal to 0 and syntax element 1050A is inferred to be equal to N-1. However, the number of active entries in reference picture list 1 may be overridden to be a number less than N. In that case, the bitstream is illegal.

[0184]

[0227] In another example, if syntax element 930A is signaled as 0, the number of entries in reference picture list 1 (num_ref_entries[1] signaled in ref_pic_list_structure()) is N, syntax element 940A is signaled as N-1, and the number of active entries is not overridden in the slice header (assuming the number of active entries is the same as the number of entries in both reference picture lists). However, if the current slice is a P slice, syntax element 940A is inferred to be equal to 1, and syntax element 1050A is inferred to be equal to N-1. However, the number of entries in reference picture list 0 (num_ref_entries[0] signaled in ref_pic_list_structure()) may be less than N. As a result, the bitstream in this case is also illegal.

[0185]

[0228] To overcome this drawback in conventional encoding techniques, some embodiments of the present disclosure (as shown below in Figures 13A-13C) infer co-located pictures in the SH based also on the number of active entries in the reference picture list.

[0186]

[0229] FIG. 13A shows a flowchart of an exemplary video encoding method 1300A for determining an index of a co-located picture in an SH using the number of active entries in a reference picture list, according to some embodiments of the present disclosure. Method 1300A may be performed by an encoder (e.g., by process 200A of FIG. 2A or process 200B of FIG. 2B) or by one or more software or hardware components of an apparatus (e.g., apparatus 400 of FIG. 4). For example, one or more processors (e.g., processor 402 of FIG. 4) may perform method 1300A. In some embodiments, method 1300A may be implemented by a computer program product embodied in a computer-readable medium including computer-executable instructions, such as program code, for execution by a computer (e.g., apparatus 400 of FIG. 4). Referring to FIG. 13A, method 1300A may include the following steps 1302A-1306A.

[0187]

[0230] In step 1302A, it is determined whether to signal a parameter for indicating a reference index of a collocated picture in a slice header. In VVC, the parameter for indicating a reference index of a collocated picture in a slice header may be the syntax element sh_collocated_ref_idx.

[0188]

[0231] In step 1304, if the parameter is not signaled in the slice header, determine the collocated picture as the picture referenced by the index having a value equal to the smaller of the value of the collocated picture's reference index (e.g., ph_collocated_ref_idx) signaled in the picture header and the number of active entries in the target reference picture list minus 1 (e.g., NumRefIdxActive[!sh_collocated_from_l0_flag]−1). The target reference picture list in the reference picture list is indicated by a flag indicating from which reference picture list the collocated picture used for temporal motion vector prediction is derived. Thus, the number of active entries in the reference picture is taken into account when inferring the value of syntax element 1050A (e.g., sh_collocated_ref_idx). If the value of syntax element 940A (e.g., ph_collocated_ref_idx) signaled in PH is greater than or equal to the number of active entries in the target reference picture list, the inferred value of syntax element 1050A (e.g., sh_collocated_ref_idx) is clipped to be less than the number of active entries in the target reference picture list. The target reference picture list in the reference picture list is indicated by syntax element 1040A (e.g., sh_collocated_from_l0_flag).

[0189]

[0232] In step 1306, the current picture is coded based on the collocated picture, and the collocated picture is used for temporal motion vector prediction, thus avoiding an invalid bitstream and improving the robustness of the collocated picture.

[0190]

[0233] FIG. 13B shows a flowchart of an exemplary video decoding method 1300B for determining an index of a co-located picture in an SH using the number of active entries in a reference picture list, according to some embodiments of the present disclosure. Method 1300B may be performed by a decoder (e.g., by process 300A of FIG. 3A or process 300B of FIG. 3B) or by one or more software or hardware components of an apparatus (e.g., apparatus 400 of FIG. 4). For example, one or more processors (e.g., processor 402 of FIG. 4) may perform method 1300B. In some embodiments, method 1300B may be implemented by a computer program product embodied in a computer-readable medium including computer-executable instructions, such as program code, for execution by a computer (e.g., apparatus 400 of FIG. 4). Referring to FIG. 13B, method 1300B may include the following steps 1302B-1310B:

[0191]

[0234] In step 1302B, the decoder receives a video bitstream (e.g., video bitstream 228 of FIG. 3B), which may be encoded using inter-prediction. Thus, reference pictures may be derived, for example, by reference picture 0 and reference picture list 1, each of which contains a list of reconstructed pictures in the DPB (e.g., buffer 234 in FIG. 3B) to be used as reference pictures.

[0192]

[0235] In step 1304B, determine whether there is a parameter in the slice header indicating the reference index of the collocated picture used for temporal motion vector prediction. In VVC, the parameter for indicating the reference index of the collocated picture in the slice header may be the syntax element sh_collocated_ref_idx.

[0193]

[0236] In step 1306B, if the parameter is not present, the value of the parameter is determined to be equal to the smaller of the value of the reference index of the collocated picture used for temporal motion vector prediction in the picture header (e.g., ph_collocated_ref_idx) and the number of active entries in the target reference picture list minus 1 (e.g., NumRefIdxActive[!sh_collocated_from_l0_flag]-1). The target reference picture list in the reference picture list is indicated by a flag indicating which reference picture list the collocated picture used for temporal motion vector prediction is derived from. Therefore, the number of active entries in the reference picture is taken into account when determining the value of syntax element 1050A (e.g., sh_collocated_ref_idx). If the value of syntax element 940A (e.g., ph_collocated_ref_idx) signaled in PH is greater than or equal to the number of active entries in the target reference picture list, the determined value of syntax element 1050A (e.g., sh_collocated_ref_idx) is clipped to be less than the number of active entries in the target reference picture list. The target reference picture list in the reference picture list is indicated by syntax element 1040A (e.g., sh_collocated_from_l0_flag). Thus, invalid bitstreams are avoided.

[0194]

[0237] In step 1308B, the co-located picture is determined as the picture referenced by the index having a value equal to the parameter in the target reference picture list, thereby improving the robustness of the co-located picture.

[0195]

[0238] In step 1310B, the current picture is decoded based on the co-located picture, which improves the reliability of the decoding process.

[0196]

[0239] 13C illustrates a portion of example semantics 1300C according to some embodiments of the present disclosure. Semantics 1300C may be used in methods 1300A and 1300B. As shown in FIG. 13C, in syntax 1310C, changes from the previous VVC are shown in italics, and suggested deletion syntax is further indicated by strikethrough. Syntax 1310C corresponds to step 1306A of FIG. 13A and step 1306B of FIG. 13B. If pps_rpl_info_in_ph_flag (e.g., syntax element 830) is equal to 1, it means that the reference picture list information is in the PH syntax structure and not in the SH that references a PPS that does not contain a PH syntax structure, and the value of sh_collocated_ref_idx (e.g., syntax element 1050A) is inferred to be equal to min(ph_collocated_ref_idx,NumRefIdxActive[!sh_collocated_from_l0_flag]-1), that is, the value of sh_collocated_ref_idx is set equal to the smaller of the value of the reference index of the collocated picture used for temporal motion vector prediction in the picture header (e.g., ph_collocated_ref_idx) and the number of active entries in the target reference picture list minus 1 (e.g., NumRefIdxActive[!sh_collocated_from_l0_flag]-1). The target reference picture list is indicated by syntax element 1040A (e.g., sh_collocated_from_l0_flag), which is the reference picture list from which the collocated picture used for temporal motion vector prediction is derived. If the collocated picture used for temporal MVP is derived from reference picture list 0, the target reference picture list is reference picture list 0. If the collocated picture used for temporal MVP is derived from reference picture list 1, the target reference picture list is reference picture list 1.

[0197]

[0240] In VVC (e.g., VVC Draft 9), ref_pic_list_struct() may be signaled in the SPS or may be included in the syntax structure ref_pic_lists(). If the ref_pic_list_structure() signaled in the SPS is not selected in the PH or SH, another ref_pic_list_structure() may be directly signaled within the ref_pic_lists() signaled in the PH or SH. However, VVC (e.g., VVC Draft 9) provides the following: for each value of listIdx (equal to 0 or 1), the decoder should allocate memory for a total number of ref_pic_list_struct(listIdx, rplsIdx) syntax structures equal to sps_num_ref_pic_lists[i] plus 1, because there can be one ref_pic_list_struct(listIdx, rplsIdx) syntax structure directly signaled in the slice header of the current picture. In light of the above, this is not accurate.

[0198]

[0241] To overcome this drawback in conventional encoding techniques, in some embodiments of the present disclosure (as shown below in Figures 14A and 14B), if one ref_pic_list_struct(listIdx, rplsIdx) syntax structure is directly signaled in the picture header or slice header of the current picture, then for each value of listIdx (equal to 0 or 1), the decoder allocates memory for a total number of ref_pic_list_struct(listIdx, rplsIdx) syntax structures equal to sps_num_ref_pic_lists[i] plus 1.

[0199]

[0242] FIG. 14A shows a flowchart of an exemplary video processing method 1400A for allocating memory according to some embodiments of the present disclosure. Method 1400A may be performed by an encoder (e.g., by process 200A of FIG. 2A or process 200B of FIG. 2B), a decoder (e.g., by process 300A of FIG. 3A or process 300B of FIG. 3B), or one or more software or hardware components of an apparatus (e.g., apparatus 400 of FIG. 4). For example, one or more processors (e.g., processor 402 of FIG. 4) may perform method 1400A. In some embodiments, method 1400A may be implemented by a computer program product embodied in a computer-readable medium that includes computer-executable instructions, such as program code, for execution by a computer (e.g., apparatus 400 of FIG. 4). Referring to FIG. 14A, method 1400A may include the following steps 1402A-1406A:

[0200]

[0243] In step 1402, a total number is derived by summing the number of reference picture list structures in the sequence parameter set (SPS) and 1. Since another RPL may be signaled later (in a picture header or slice header), an additional number 1 is added to the number of reference picture list structures in the SPS to get the total number.

[0201]

[0244] In step 1404A, memory for the total number of reference picture list structures is allocated in response to the reference picture list structures being signaled in the picture header of the current picture or the slice header of the current slice, so that more memory is allocated for additional RPLs signaled in the picture header of the current picture or the slice header of the current slice by the encoder / decoder before encoding / decoding, which is useful for video processing.

[0202]

[0245] In step 1406A, the allocated memory is used to process the current picture or current slice, and since the allocated memory is more reliable with respect to the additional RPL, the encoding / decoding process may be more accurate and robust.

[0203]

[0246] 14B illustrates a portion of example semantics 1400B according to some embodiments of the present disclosure. Semantics 1400B may be used within method 1400A, with changes from previous VVCs indicated in italics (see block 1410B). Due to the possibility that another RPL will be signaled later (in a picture header or slice header), more memory is allocated for the additional RPL.

[0204]

[0247] In VVC (e.g., VVC Draft 9), syntax element 530A (e.g., rpl_idx[i]) specifies an index into a list of ref_pic_list_struct(listIdx, rplsIdx) syntax structures with listIdx equal to i contained in the SPS of ref_pic_list_struct(listIdx, rplsIdx) syntax structures with listIdx equal to i that are used to derive reference picture list i for the current picture. This semantics may not be precise because reference picture lists can be derived for pictures or slices.

[0205]

[0248] In VVC (e.g., VVC Draft 9), in the absence of syntax element 530A, there is an inference rule for inferring the value of syntax element 530A: if syntax element 510A (e.g., rpl_sps_flag[i]) is equal to 1 and syntax element 520A (e.g., pps_rpl1_idx_present_flag) is equal to 0, then the value of rpl_idx[1] is inferred to be equal to rpl_idx[0]; otherwise, the value of rpl_idx[1] is inferred to be equal to 0. This inference rule has several problems. First, there is only an inference rule for rpl_idx[1], but no inference rule for rpl_idx[0]. Second, if syntax element 510A is equal to 1 and syntax element 520A is equal to 0, there is no guarantee that rpl_idx[0] is signaled. So in this case, inferring the value of rpl_idx[1] to be equal to rpl_idx[0] may be problematic. In short, the inference rules in VVC (e.g., VVC Draft 9) cannot guarantee that both rpl_idx[0] and rpl_idx[1] will get the appropriate values ​​at the decoder side if rpl_idx[0] and rpl_idx[1] are absent.

[0206]

[0249] To overcome this shortcoming in conventional encoding techniques, some embodiments of the present disclosure (as shown below in Figures 15A-15C) provide updated semantics for syntax element 530A (e.g., rpl_idx[i]).

[0207]

[0250] FIG. 15A shows a flowchart of an exemplary video encoding method 1500A for determining an index in a reference picture list according to some embodiments of the present disclosure. Method 1500A may be performed by an encoder (e.g., by process 200A of FIG. 2A or process 200B of FIG. 2B) or by one or more software or hardware components of an apparatus (e.g., apparatus 400 of FIG. 4). For example, one or more processors (e.g., processor 402 of FIG. 4) may perform method 1500A. In some embodiments, method 1500A may be implemented by a computer program product embodied in a computer-readable medium that includes computer-executable instructions, such as program code, for execution by a computer (e.g., apparatus 400 of FIG. 4). Referring to FIG. 15A, method 1500A may include the following steps 1502A-1514A.

[0208]

[0251] In step 1502A, a first flag (e.g., pps_rpl1_idx_present_flag) in a picture parameter set (PPS) is signaled to indicate whether there is a second flag (e.g., rpl_sps_flag[1]) and a first index (e.g., rpl_idx[1]) in the picture header syntax or slice header of the current picture that references the PPS. The first flag (e.g., pps_rpl1_idx_present_flag) indicates whether reference picture list 1 is derived based on one of the reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index (e.g., rpl_idx[1]) is an index of the reference picture list structure associated with reference picture list 1 used to derive reference picture list 1 relative to a list of reference picture list structures associated with reference picture list 1 contained in the SPS. A second flag (eg, rpl_sps_flag[1]) may then be signaled.

[0209]

[0252] In step 1504A, it is determined whether to signal a first index (e.g., rpl_idx[1]) and a second index (e.g., rpl_idx[0]). The second index (e.g., rpl_idx[0]) is an index of a reference picture list structure associated with reference picture list 0 used to derive reference picture list 0 into a list of reference picture list structures associated with reference picture list 0 contained within the SPS.

[0210]

[0253] If the second index (eg, rpl_idx[0]) is not signaled, the value of the second index (eg, rpl_idx[0]) may be determined by step 1506A.

[0211]

[0254] In step 1506A, if at most one reference picture list structure associated with reference picture list 0 is contained within the SPS, the value of the second index (e.g., rpl_idx[0]) is determined to be equal to 0. Referring to Figure 5A, if sps_num_ref_pic_lists[0] is less than or equal to 1, rpl_idx[0] is not signaled. Thus, in step 1506A, the value of rpl_idx[0] is determined for situations where rpl_idx[0] is not signaled, increasing the reliability of inferring rpl_idx[0].

[0212]

[0255] If the first index (eg, rpl_idx[1]) is not signaled, the value of the first index (eg, rpl_idx[0]) can be determined by steps 1508A and 1510A.

[0213]

[0256] In step 1508A, if at most one reference picture list structure associated with reference picture list 1 is included in the SPS, the value of the first index (e.g., rpl_idx[1]) is determined to be equal to 0. Referring to Figure 5A, if sps_num_ref_pic_lists[1] is less than or equal to 1, rpl_idx[1] is not signaled. Thus, in step 1508A, the value of rpl_idx[1] is determined for situations where rpl_idx[1] is not signaled, increasing the reliability of inferring rpl_idx[1].

[0214]

[0257] In step 1510A, if the first flag (e.g., pps_rpl1_idx_present_flag) is equal to 0 and the second flag (e.g., rpl_sps_flag[1]) is equal to 1, determine the value of the first index (e.g., rpl_idx[1]) to be equal to the value of the second index (e.g., rpl_idx[0]). If sps_num_ref_pic_lists[0] is less than or equal to 1 (e.g., in step 1508A), the value of rpl_idx[0] is set to 0; otherwise (e.g., sps_num_ref_pic_list[0] > 1), the value of rpl_idx[0] is determined for all scenarios since rpl_idx[0] is signaled (see FIG. 5A). Thus, in this case, the value of rpl_idx[1] is set equal to the determined value of rpl_idx[0]. Therefore, the value of rpl_idx[1] is determined for all scenarios (e.g., whether rpl_idx[0] is signaled or not). If rpl_idx[i] is not signaled, the value of rpl_idx[i] (both rpl_idx[0] and rpl_idx[1]) can be guaranteed to get the appropriate value.

[0215]

[0258] After determining the values ​​of the first index (e.g., rpl_idx[1]) and the second index (e.g., rpl_idx[0]), a reference picture list is determined based on the first index and the second index in step 1512A. Regardless of whether the first index (e.g., rpl_idx[1]) and the second index (e.g., rpl_idx[0]) are signaled or not, the determination of the values ​​of the first index (e.g., rpl_idx[1]) and the second index (e.g., rpl_idx[0]) is guaranteed, so the decision regarding the reference picture list may be more reliable.

[0216]

[0259] In step 1514A, the current picture is coded based on the reference picture list, thus improving the robustness of the coding process.

[0217]

[0260] In some embodiments, since it is inferred that rpl_idx[0] is equal to 0 when one reference picture list structure of reference picture list 0 is in the SPS (reference to step 1508A), step 1510A can be replaced with "in response to one reference picture list structure of reference picture list i being in the SPS, rpl_idx[i] is determined to be equal to 0", which can further improve the efficiency of the encoding process.

[0218]

[0261] FIG. 15B shows a flowchart of an example video decoding method 1500B for determining an index in a reference picture list according to some embodiments of the present disclosure. Method 1500B may be performed by a decoder (e.g., by process 300A of FIG. 3A or process 300B of FIG. 3B) or by one or more software or hardware components of an apparatus (e.g., apparatus 400 of FIG. 4). For example, one or more processors (e.g., processor 402 of FIG. 4) may perform method 1500B. In some embodiments, method 1500B may be implemented by a computer program product embodied in a computer-readable medium that includes computer-executable instructions, such as program code, for execution by a computer (e.g., apparatus 400 of FIG. 4). Referring to FIG. 15B, method 1500B may include the following steps 1502B-1514B:

[0219]

[0262] In step 1502B, a decoder receives a video bitstream (e.g., video bitstream 228 in FIG. 3B), which may be encoded using inter prediction. Reference pictures may be derived, for example, by reference picture 0 and reference picture list 1, each of which contains a list of reconstructed pictures in a DPB (e.g., buffer 234 in FIG. 3B) to be used as reference pictures.

[0220]

[0263] In step 1504B, a value of a first flag (e.g., pps_rpl1_idx_present_flag) indicating whether a second flag (e.g., rpl_sps_flag[1]) and a first index (e.g., rpl_idx[1]) are present in the picture header syntax or slice header of the current picture is determined. The second flag indicates whether reference picture list 1 is derived based on one of the reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index is an index of the reference picture list structure associated with reference picture list 1 used to derive reference picture list 1 relative to a list of reference picture list structures associated with reference picture list 1 contained in the SPS. The value of the second flag (e.g., rpl_sps_flag[1]) can then be determined.

[0221]

[0264] In step 1506B, it is determined whether there is a first index (e.g., rpl_idx[1]) and a second index (e.g., rpl_idx[0]). The second index (e.g., rpl_idx[0]) is an index of a reference picture list structure associated with reference picture list 0 used to derive reference picture list 0 into a list of reference picture list structures associated with reference picture list 0 contained within the SPS.

[0222]

[0265] If there is no second index (eg, rpl_idx[0]), the value of the second index (eg, rpl_idx[0]) may be determined by step 1508B.

[0223]

[0266] In step 1508B, if at most one reference picture list structure associated with reference picture list 0 is included in the SPS, the value of the second index (e.g., rpl_idx[0]) is determined to be equal to 0. Referring to FIG. 5A, if sps_num_ref_pic_lists[0] is less than or equal to 1, rpl_idx[0] is not signaled and therefore rpl_idx[0] does not exist. In this case, rpl_idx[0] is set to be equal to 0. Thus, in step 1508B, the value of rpl_idx[0] is determined for the situation where rpl_idx[0] does not exist, increasing the reliability of inferring rpl_idx[0].

[0224]

[0267] If the first index (eg, rpl_idx[1]) does not exist, the value of the first index (eg, rpl_idx[1]) can be determined by steps 1510B and 1512B.

[0225]

[0268] In step 1510B, if at most one reference picture list structure associated with reference picture list 1 is contained within the SPS, the value of the first index (e.g., rpl_idx[1]) is determined to be equal to 0. Referring to Figure 5A, if sps_num_ref_pic_lists[1] is less than or equal to 1, rpl_idx[1] is not signaled and therefore rpl_idx[1] does not exist. Thus, in step 1510B, the value of rpl_idx[1] is determined for situations where rpl_idx[1] is not signaled, increasing the reliability of inferring rpl_idx[1].

[0226]

[0269] In step 1512B, if the first flag (e.g., pps_rpl1_idx_present_flag) is equal to 0 and the second flag (e.g., rpl_sps_flag[1]) is equal to 1, determine the value of the first index (e.g., rpl_idx[1]) to be equal to the value of the second index (e.g., rpl_idx[0]). If sps_num_ref_pic_lists[0] is less than or equal to 1 (in step 1508A), the value of rpl_idx[0] is set to 0; otherwise (e.g., sps_num_ref_pic_list[0] > 1), the value of rpl_idx[0] is determined for all scenarios since rpl_idx[0] is signaled (see FIG. 5A). Thus, in this case, the value of rpl_idx[1] is set equal to the determined value of rpl_idx[0]. Therefore, the value of rpl_idx[1] is determined for all scenarios (e.g., whether or not rpl_idx[0] is present). If rpl_idx[i] is absent, the value of rpl_idx[i] (both rpl_idx[0] and rpl_idx[1]) can be guaranteed to get the correct value.

[0227]

[0270] At 1514B, the current picture is decoded based on the first index (e.g., rpl_idx[1]) and the second index (e.g., rpl_idx[0]). Regardless of whether the first index (e.g., rpl_idx[1]) and the second index (e.g., rpl_idx[0]) are present or not, the determination of the values ​​of the first index (e.g., rpl_idx[1]) and the second index (e.g., rpl_idx[0]) is guaranteed, so that the determination regarding the reference picture list may be more reliable.

[0228]

[0271] In some embodiments, since rpl_idx[0] is inferred to be equal to 0 when one reference picture list structure of reference picture list 0 is in the SPS (reference to step 1508B), step 1514B can be replaced with "rpl_idx[i] is inferred to be equal to 0 in response to one reference picture list structure of reference picture list i being in the SPS." The efficiency of the decoding process can be further improved.

[0229]

[0272] 15C illustrates a portion of example semantics 1500C according to some embodiments of the present disclosure. Semantics 1500C may be used within methods 1500A and 1500B. As shown in FIG. 15C, changes from previous VVCs are shown in italics, and suggested deletion syntax is further indicated by strikethrough (reference to blocks 1510C and 1520C). Two alternative derivation descriptions are shown. In some embodiments, as shown in block 1510C, in the case where there is no rpl_idx[i], if there is at most one reference picture list structure for reference picture list i (e.g., sps_num_ref_pic_list[i] is less than or equal to 1), the value of rpl_idx[i] is inferred to be equal to 0; otherwise (if there are multiple reference picture list structures for reference picture i, i.e., sps_num_ref_pic_list[i] is greater than 1), if i is equal to 1, i.e., sps_num_ref_pic_list[1] is greater than 1, the value of rpl_idx[1] is inferred to be equal to rpl_idx[0]. The difference between block 1520C and block 1510C is that the expressions "otherwise" and "i is equal to 1" are interpreted in more detail as "sps_num_ref_pic_list[1] is greater than 1". In some embodiments, the condition "sps_num_ref_pic_list[i] is less than or equal to 1" (reference to blocks 1511C and 1521C) may be replaced by "if sps_num_ref_pic_list[i] is equal to 1."

[0230]

[0273] In VVC (e.g., VVC Draft 9), a syntax element 1010A (e.g., sh_num_ref_idx_active_override_flag) equal to 1 specifies that the syntax element sh_num_ref_idx_active_minus1[0] is present for P slices and B slices, and that the syntax element sh_num_ref_idx_active_minus1[1] is present for B slices. A syntax element 1010A equal to 0 specifies that the syntax elements sh_num_ref_idx_active_minus1[0] and sh_num_ref_idx_active_minus1[1] are absent. However, as shown in FIG. 10A , when syntax element 1010A is equal to 1, the values ​​of num_ref_entries[i][RplsIdx[i]] are further examined to signal sh_num_ref_idx_active_minus1[i]. The syntax element sh_num_ref_idx_active_minus1[i] is signaled only if syntax element 1010A is equal to 1 and num_ref_entries[i][RplsIdx[i]] is greater than 1. As a result, syntax element 1010A equal to 1 does not necessarily mean that sh_num_ref_idx_active_minus1[i] is signaled.

[0231]

[0274] To overcome this shortcoming in conventional encoding techniques, some embodiments of the present disclosure (as shown below in Figures 16A-16C) provide updated semantics for syntax element 1010A to improve the efficiency of the encoding / decoding process.

[0232]

[0275] FIG. 16A shows a flowchart of an example video encoding method 1600A for indicating the presence of an active reference index number in a slice header, according to some embodiments of the present disclosure. Method 1600A may be performed by an encoder (e.g., by process 200A of FIG. 2A or process 200B of FIG. 2B) or by one or more software or hardware components of an apparatus (e.g., apparatus 400 of FIG. 4). For example, one or more processors (e.g., processor 402 of FIG. 4) may perform method 1600A. In some embodiments, method 1600A may be implemented by a computer program product embodied in a computer-readable medium that includes computer-executable instructions, such as program code, for execution by a computer (e.g., apparatus 400 of FIG. 4). Referring to FIG. 16A, method 1600A may include the following steps 1602A-1608A:

[0233]

[0276] In step 1602A, a first flag is signaled in the slice header to indicate whether there is an active reference index number in the slice header. For example, a syntax element sh_num_ref_idx_active_override_flag is signaled to indicate whether the active reference index number (e.g., sh_num_ref_idx_active_minus1[i]) of reference picture list i (i equals 0 or 1) is present in the slice header. The active reference index number is used to derive the maximum reference index of the corresponding reference picture list that may be used to encode the current slice. The number of reference indexes used to encode the current slice may be less than or equal to the maximum number derived from the active reference index number.

[0234]

[0277] In step 1604A, it is determined whether there is an active reference index number. If the first flag indicates that there is an active reference index number, the syntax element sh_num_ref_idx_active_minus1[0] is present for P slices and B slices, and the syntax element sh_num_ref_idx_active_minus1[1] is present for B slices. Then, steps 1606A and 1608A are performed.

[0235]

[0278] In step 1606A, the number of entries in reference picture list 0 is first determined, and if the number of entries in reference picture list 0 (e.g., num_ref_entries[0][RplsIdx[0]]) is determined to be greater than 1, the number of active reference indices in reference picture list 0 (e.g., sh_num_ref_idx_active_minus1[0]) is signaled in the slice header for P slices and B slices.

[0236]

[0279] In step 1608A, the number of entries in reference picture list 1 is first determined, and if the number of entries in reference picture list 1 (e.g., num_ref_entries[1][RplsIdx[1]]) is determined to be greater than 1, the number of active reference indices in reference picture list 1 (e.g., sh_num_ref_idx_active_minus1[1]) is signaled in the slice header for the B slice.

[0237]

[0280] According to steps 1606A and 1608A, if the number of entries in reference picture list i (e.g., num_ref_entries[i][RplsIdx[i]]) is greater than 1, the number of active reference indexes in reference picture list i (e.g., sh_num_ref_idx_active_minus1[i]) is signaled within the slice level.

[0238]

[0281] Thus, there is no uncertainty regarding sh_num_ref_idx_active_minus1[i] signaled when sh_num_ref_idx_active_override_flag is equal to 1, and the accuracy and robustness of the encoding process may be improved.

[0239]

[0282] In some embodiments, method 1600A may further include steps 1610A and 1612A. If the first flag indicates that there are no active reference index numbers, the syntax element sh_num_ref_idx_active_minus1[i] is not present. Then, steps 1610A and 1612A are performed.

[0240]

[0283] In step 1610A, skip signaling the number of active reference indices (e.g., sh_num_ref_idx_active_minus1[0]) of reference picture list 0 in the slice headers for P slices and B slices. In other words, no sh_num_ref_idx_active_minus1[0] is signaled in the slice headers for P slices and B slices.

[0241]

[0284] In step 1612A, skip signaling the number of active reference indexes of reference picture list 1 in the slice header for the B slice, in other words, no sh_num_ref_idx_active_minus1[1] is signaled in the slice header for the B slice.

[0242]

[0285] Therefore, in the absence of active reference index numbers, the encoding process may be more efficient by skipping the signaling of active reference numbers.

[0243]

[0286] FIG. 16B illustrates a flowchart of an exemplary video decoding method 1600B for indicating an active reference index number in a slice header, according to some embodiments of the present disclosure. Method 1600B may be performed by a decoder (e.g., by process 300A of FIG. 3A or process 300B of FIG. 3B) or by one or more software or hardware components of an apparatus (e.g., apparatus 400 of FIG. 4). For example, one or more processors (e.g., processor 402 of FIG. 4) may perform method 1600B. In some embodiments, method 1600B may be implemented by a computer program product embodied in a computer-readable medium that includes computer-executable instructions, such as program code, for execution by a computer (e.g., apparatus 400 of FIG. 4). Referring to FIG. 16A, method 1600B may include the following steps 1602B-1608B:

[0244]

[0287] In step 1602B, a decoder receives a video bitstream (e.g., video bitstream 228 in FIG. 3B) including slice header and picture header syntax, where the video bitstream may be encoded using inter prediction. Reference pictures may be derived, for example, by reference picture 0 and reference picture list 1, each of which includes a list of reconstructed pictures in a DPB (e.g., buffer 234 in FIG. 3B) to be used as reference pictures.

[0245]

[0288] In step 1604B, the value of a first flag signaled in the slice header indicating whether there is an active reference index number is determined. In some embodiments, the first flag is a syntax element sh_num_ref_idx_active_override_flag, which may indicate whether there is an active reference index (e.g., sh_num_ref_idx_active_minus1[i]) for reference picture list i (i equal to 0 or 1). The active reference index number is used to derive the maximum reference index of the corresponding reference picture list that may be used to decode the current slice. The number of reference indexes used to decode the current slice may be less than or equal to the maximum number derived from the active reference index number.

[0246]

[0289] If the value of the first flag is determined to be a value indicating that there is an active reference index number, the syntax element sh_num_ref_idx_active_minus1[0] is present for P slices and B slices, and the syntax element sh_num_ref_idx_active_minus1[1] is present for B slices. Then, step 1606B and step 1608B are performed.

[0247]

[0290] In step 1606B, the number of entries in reference picture list 0 (e.g., num_ref_entries[0][RplsIdx[0]]) is determined, and if the number of entries in reference picture list 0 is determined to be greater than 1, the number of active reference indices in reference picture list 0 (e.g., sh_num_ref_idx_active_minus1[0]) in the slice header for P slices and B slices is decoded.

[0248]

[0291] In step 1608B, the number of entries in reference picture list 1 (e.g., num_ref_entries[1][RplsIdx[1]]) is determined, and if the number of entries in reference picture list 1 is determined to be greater than 1, the number of active reference indices in reference picture list 1 in the slice header for the B slice (e.g., sh_num_ref_idx_active_minus1[1]) is decoded.

[0249]

[0292] Steps 1606B and 1608B signal the number of active reference indices (e.g., sh_num_ref_idx_active_minus1[i]) for reference picture list i if the number of entries for reference picture list i (e.g., num_ref_entries[i][RplsIdx[i]]) is greater than 1. Thus, there is no uncertainty about sh_num_ref_idx_active_minus1[i] being signaled when sh_num_ref_idx_active_override_flag is equal to 1.

[0250]

[0293] In some embodiments, method 1600B may further include step 1610B and step 1612B. If it is determined that the value of the first flag is a value indicating that there is no active reference index number, the syntax element sh_num_ref_idx_active_minus1[i] is not signaled. Then, step 1610B and step 1612B are performed.

[0251]

[0294] In step 1610B, skip decoding the active reference index number (e.g., sh_num_ref_idx_active_minus1[0]) of reference picture list 0 in the slice headers for P slices and B slices. In other words, there is no sh_num_ref_idx_active_minus1[0] in the slice header for B slices.

[0252]

[0295] In step 1612B, skip decoding the active reference index number of reference picture list 1 (e.g., sh_num_ref_idx_active_minus1[1]) in the slice header for the B slice. In other words, there is no sh_num_ref_idx_active_minus1[1] in the slice header for the B slice. Therefore, the efficiency of the decoding process may be improved.

[0253]

[0296] 16C illustrates a portion of example semantics 1600C according to some embodiments of the present disclosure. Semantics 1600B may be used within methods 1600A and 1600B. As shown in FIG. 16C, changes from previous VVC are shown in italics, and the proposed deletion syntax is further indicated by strikethrough (reference to blocks 1610C and 1620C). Two alternative descriptions are shown. As shown in block 1610C, sh_num_ref_idx_active_override_flag equal to 1 does not necessarily specify that there is a syntax element sh_num_ref_idx_active_minus1[0] for P slices and B slices, or that there is a syntax element sh_num_ref_idx_active_minus1[1] for B slices. As shown in block 1620C, if there is a syntax element sh_num_ref_idx_active_minus1[0] for P slices and B slices, a condition "if num_ref_entries[0][RplsIdx[0] is greater than 1" is added, and if there is a syntax element sh_num_ref_idx_active_minus1[1] for B slices, a condition "if num_ref_entries[1][RplsIdx[1] is greater than 1" is added, and sh_num_ref_idx_active_override_flag is equal to 1. Thus, the accuracy and robustness of the decoding process may be improved.

[0254]

[0297] In VVC (e.g., VVC Draft 9), there is a bitstream compliance constraint that the picture referenced by syntax element 1050A (e.g., sh_collocated_ref_idx) is the same for all slices of a coded picture, and RprConstraintsActive[sh_collocated_from_l0_flag?0:1][sh_collocated_ref_idx] is equal to 0. To identify the picture referenced by syntax element 1050A, the values ​​of syntax element 1040A (e.g., sh_collocated_from_l0_flag) and syntax element 1050A (e.g., sh_collocated_ref_idx) must first be determined. However, as shown in FIG. 10A , syntax element 1040A is signaled only for B slices, and syntax element 1050A is signaled only for P and B slices. For I slices, syntax element 1040A and syntax element 1050A are not signaled. Furthermore, there are no inferred values ​​for these two syntax elements in an I-slice. As a result, the value of syntax element 1050A is undefined in an I-slice. Therefore, the encoder / decoder cannot identify the picture referenced by syntax element 1050A and cannot check the compliance constraint.

[0255]

[0298] To overcome this shortcoming in conventional encoding techniques, embodiments of the present disclosure (as illustrated below in Figures 17A and 17B) provide updated semantics to improve the accuracy and robustness of video processing.

[0256]

[0299] For example, Figure 17A shows a flowchart of an exemplary video processing method 1700A for picture processing. Method 1700A may be performed by an encoder (e.g., by process 200A of Figure 2A or process 200B of Figure 2B), by a decoder (e.g., by process 300A of Figure 3A or process 300B of Figure 3B), or by one or more software or hardware components of an apparatus (e.g., apparatus 400 of Figure 4). For example, one or more processors (e.g., processor 402 of Figure 4) may perform method 1700A. In some embodiments, method 1700A may be implemented by a computer program product embodied in a computer-readable medium including computer-executable instructions, such as program code, for execution by a computer (e.g., apparatus 400 of Figure 4). Referring to Figure 24A, method 1700A may include steps 1702A and 1704A.

[0257]

[0300] In step 1702, the collocated picture referenced by the collocated picture's reference index (e.g., sh_collocated_ref_idx) within the slice level is determined, and the collocated picture is determined to be the same picture for all non-I slices of the current picture, thus avoiding uncertainty regarding the values ​​of sh_collocated_ref_idx and sh_collocated_from_l0_flag.

[0258]

[0301] In step 1704A, the current picture is processed based on the collocated picture, and the collocated picture is used for temporal motion vector prediction, so that the robustness of the video processing can be improved.

[0259]

[0302] That is, the pictures used for temporal motion vector prediction referenced by the reference indexes of the co-located picture are determined to be the same for all non-I slices of the coded picture. In some embodiments, the pictures used for temporal motion vector prediction referenced by the reference indexes of the co-located picture are determined to be the same for all P slices and B slices of the current picture.

[0260]

[0303] FIG. 17B illustrates a portion of example semantics 1700B according to some embodiments of the present disclosure. Semantics 1700B may be used within method 1700A. As shown in FIG. 17B, changes from previous VVC are italicized, and proposed deletion syntax is further indicated by strikethrough (reference to blocks 1710B and 1720B). Two alternative descriptions are shown. As shown in block 1710B, the bitstream compliance requirement is further elaborated to "all non-I slices" instead of "all slices." Thus, the efficiency and robustness of the decoding process are improved. The difference between block 1720B and block 1710B is that the expression "non-I slices" is replaced by "P slices and B slices" to be more precise.

[0261]

[0304] Some embodiments also provide a non-transitory computer-readable storage medium containing instructions that can be executed by a device (such as the encoders and decoders of the present disclosure) to perform the above-described methods. Common forms of non-transitory media include, for example, a floppy disk, a flexible disk, a hard disk, a solid-state drive, a magnetic tape, or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, any physical medium with a pattern of holes, RAM, PROM, and EPROM, FLASH-EPROM or any other flash memory, NVRAM, cache, registers, any other memory chip or cartridge, and networked versions thereof. A device may include one or more processors (CPUs), input / output interfaces, a network interface, and / or memory.

[0262]

[0305] It should be noted that relational terms herein, such as "first" and "second," are used merely to distinguish one entity or operation from another, and do not require or imply any actual relationship or order among those entities or operations. Furthermore, the words "comprising," "having," "containing," and "including," and other similar forms, are intended to be equivalent in meaning and to be open-ended in that the element or elements following any of these words are not meant to be an exclusive list of such elements or elements, or to be limited to only the listed element or elements.

[0263]

[0306] As used herein, unless specifically stated otherwise, the term "or" includes all possible combinations unless impracticable. For example, if it is stated that a database may include A or B, then, unless specifically stated otherwise or impracticable, the database may include A, or B, or A and B. As a second example, if it is stated that a database may include A, B, or C, then, unless specifically stated otherwise or impracticable, the database may include A, B, C, A and B, A and C, B and C, or A, B, and C.

[0264]

[0307] It is understood that the above-described embodiments can be implemented by hardware, or software (program code), or a combination of hardware and software. If implemented by software, it can be stored in the above-described computer-readable medium. The software, when executed by a processor, can perform the methods of the present disclosure. The computational units and other functional units described in this disclosure can be implemented by hardware, or software, or a combination of hardware and software. Those skilled in the art will also understand that multiple of the above-described modules / units can be combined into one module / unit, or that each of the above-described modules / units can be further divided into multiple sub-modules / sub-units.

[0265]

[0308] The embodiments may be further described using the following clauses: 1. A computer-implemented method for encoding video, comprising: encoding the current picture based on the co-located picture, where the co-located picture is used for temporal motion vector prediction; and signaling a first flag and a second flag in response to both the number of entries in reference picture list 0 and the number of entries in reference picture list 1 being greater than 0, wherein the first flag indicates that the co-located picture is derived from reference picture list 0 or reference picture list 1, and the second flag indicates whether a motion vector difference syntax structure is signaled. 20. A computer-implemented method comprising: 2. A computer-implemented method for decoding video, comprising: receiving a video bitstream; decoding a first flag and a second flag in response to both the number of entries in reference picture list 0 and the number of entries in reference picture list 1 being greater than 0, wherein the first flag indicates that a co-located picture used for temporal motion vector prediction is derived from reference picture list 0 or reference picture list 1, and the second flag indicates whether a motion vector difference syntax structure is in the bitstream of the current picture; and Decoding a current picture based on a collocated picture 20. A computer-implemented method comprising: 3. A computer-implemented method for encoding video, comprising: encoding the current picture based on the co-located picture, where the co-located picture is used for temporal motion vector prediction; and Indicating a co-located picture in a bitstream without signaling an index to a reference picture list - Patent Application 20070122997 20. A computer-implemented method comprising: 4. Indicating a co-located picture in a bitstream without signaling an index to a reference picture list signaling a first flag to indicate whether the co-located picture is an inter-layer reference picture; and signaling a first parameter for indicating the co-located picture in response to the co-located picture being an inter-layer reference picture, the first parameter indicating an index of the co-located picture relative to a list of direct reference layers of the layer in which the current picture resides. 4. The method of clause 3, further comprising: 5. Indicating a co-located picture in a bitstream without signaling an index to a reference picture list signaling a second flag to indicate whether the co-located picture is a short-term reference picture or a long-term reference picture; and signaling a second parameter for indicating the co-located picture in response to the co-located picture being a short-term reference picture, the second parameter indicating a difference between a picture order count of the co-located picture and a picture order count of the current picture; 5. The method of clause 4, further comprising: 6. In response to the co-located picture being a long-term reference picture, signaling a third parameter and a fourth parameter for indicating the co-located picture, wherein the third parameter indicates a least significant bit (LSB) of a picture order count (POC) of the co-located picture, and the fourth parameter indicates a delta most significant bit (MSB) of the picture order count (POC) of the co-located picture. 6. The method of clause 5, further comprising: 7. The method of clause 6, wherein the first flag, the second flag, the first parameter, the second parameter, the third parameter, and the fourth parameter are signaled in a picture header, and all slices in the picture have the same co-located picture. 8. The method of clause 3, wherein the reference picture list is Reference Picture List 0 or Reference Picture List 1. 9. A computer-implemented method for decoding video, comprising: receiving a video bitstream; determining a co-located picture to be used for temporal motion vector prediction without decoding an index to a reference picture list; and Decoding a current picture based on a collocated picture 20. A computer-implemented method comprising: 10. Determining the co-located picture used for temporal motion vector prediction without decoding an index to a reference picture list decoding a first flag indicating whether the co-located picture is an inter-layer reference picture; determining whether the co-located picture is an inter-layer reference picture based on a first flag; and and in response to the co-located picture being an inter-layer reference picture, decoding a first parameter and determining the co-located picture based on the first parameter, the first parameter indicating an index of the co-located picture relative to a list of direct reference layers of the layer in which the current picture resides. 10. The method of clause 9, further comprising: 11. Determining the co-located picture used for temporal motion vector prediction without decoding an index to a reference picture list structure decoding a second flag indicating whether the co-located picture is a short-term reference picture or a long-term reference picture; determining whether the co-located picture is a short-term reference picture or a long-term reference picture based on a second flag; and and in response to the co-located picture being a short-term reference picture, decoding a second parameter and determining the co-located picture based on the second parameter, the second parameter indicating a difference between a picture order count of the co-located picture and a picture order count of the current picture. 11. The method of clause 10, further comprising: 12. In response to the co-located picture being a long-term reference picture, decoding a third parameter and a fourth parameter, and determining the co-located picture based on the third parameter and the fourth parameter, wherein the third parameter indicates a least significant bit (LSB) of a picture order count (POC) of the co-located picture, and the fourth parameter indicates a delta most significant bit (MSB) of the picture order count (POC) of the co-located picture. 12. The method of clause 11, further comprising: 13. The method of clause 12, wherein the first flag, the second flag, the first parameter, the second parameter, the third parameter, and the fourth parameter are in a picture header, and all slices in the picture have the same co-located picture. 14. The method of clause 9, wherein the reference picture list is Reference Picture List 0 or Reference Picture List 1. 15. A computer-implemented method for encoding video, comprising: determining whether to signal a parameter in a slice header for indicating a reference index of the co-located picture; responsive to the parameter not being signaled in the slice header, determining the co-located picture as the picture referenced by an index having a value equal to the smaller of the value of the co-located picture's reference index signaled in the picture header and the number of active entries in the target reference picture list minus 1; and Encoding a current picture based on a collocated picture, the collocated picture being used for temporal motion vector prediction. 20. A computer-implemented method comprising: 16. The method of clause 15, wherein the target reference picture list is indicated by a flag indicating from which reference picture list the co-located picture used for temporal motion vector prediction is derived. 17. A computer-implemented method for decoding video, comprising: receiving a video bitstream; determining whether a parameter indicating a reference index of a co-located picture used for temporal motion vector prediction is present in the slice header; responsive to the absence of the parameter, determining a value of the parameter to be equal to the smaller of a value of a reference index of the co-located picture used for temporal motion vector prediction in the picture header and a number of active entries in the target reference picture list minus one; determining the co-located picture as the picture referenced by an index having a value equal to the value of the parameter in the target reference picture list; and Decoding a current picture based on a collocated picture 20. A computer-implemented method comprising: 18. The method of clause 17, wherein the target reference picture list is indicated by a flag indicating from which reference picture list the co-located picture used for temporal motion vector prediction is derived. 19. A computer-implemented method for video processing, comprising: deriving the total number by summing the number of reference picture list structures in a sequence parameter set (SPS) and 1; allocating memory for a total number of reference picture list structures in response to the reference picture list structures being signaled in a picture header of the current picture or a slice header of the current slice; and Processing the current picture or current slice using the allocated memory 20. A computer-implemented method comprising: 20. A computer-implemented method for encoding video, comprising: signaling a first flag in a picture parameter set (PPS) to indicate whether a second flag and a first index are present in a picture header syntax or a slice header of the current picture that references the PPS, wherein the second flag indicates whether reference picture list 1 is derived based on one of reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index is an index of the reference picture list structure associated with reference picture list 1 used to derive reference picture list 1 to a list of reference picture list structures associated with reference picture list 1 included in the SPS; determining whether to signal a first index and a second index, wherein the second index is an index of a reference picture list structure associated with reference picture list 0 used to derive reference picture list 0 relative to a list of reference picture list structures associated with reference picture list 0 included in the SPS; and determining a value of the second index in response to the second index not being signaled, wherein determining the value of the second index includes: determining a value of the second index to be equal to 0 if at most one reference picture list structure associated with reference picture list 0 is included in the SPS; determining a value of the first index in response to the first index not being signaled, wherein determining the value of the first index includes: determining a value of a first index to be equal to 0 if at most one reference picture list structure associated with reference picture list 1 is included in the SPS; and determining a value of the first index to be equal to a value of the second index if the first flag is equal to 0 and the second flag is equal to 1; containing, deriving a reference picture list based on the first index and the second index; and Encoding a current picture based on a reference picture list 20. A computer-implemented method comprising: 21. A computer-implemented method for decoding video, comprising: receiving a video bitstream; determining a value of a first flag indicating whether a second flag and a first index are present in a picture header syntax or a slice header of the current picture, wherein the second flag indicates whether reference picture list 1 is derived based on one of reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index is an index of the reference picture list structure associated with reference picture list 1 used to derive reference picture list 1 to a list of reference picture list structures associated with reference picture list 1 included in the SPS; determining whether there is a first index and a second index, wherein the second index is an index of a reference picture list structure associated with reference picture list 0 used to derive reference picture list 0 relative to a list of reference picture list structures associated with reference picture list 0 that are included in the SPS; and determining a value of the second index in response to the absence of the second index, wherein determining the value of the second index includes: If at most one reference picture list structure associated with reference picture list 0 is included in the SPS, determining the value of the second index to be equal to 0. containing, and determining a value of the first index in response to the absence of the first index, wherein determining the value of the first index includes: determining a value of the first index to be equal to 0 if at most one reference picture list structure associated with reference picture list 1 is included in the SPS; and determining a value of the first index to be equal to a value of the second index if the first flag is equal to 0 and the second flag is equal to 1; containing, Decoding the current picture based on the first index and the second index 20. A computer-implemented method comprising: 22. A computer-implemented method for encoding video, comprising: signaling a first flag in the slice header to indicate whether there is an active reference index number in the slice header, the active reference index number being used to derive a maximum reference index of a corresponding reference picture list that may be used to encode the current slice; in response to a first flag indicating an active reference index number in the slice header; determining the number of entries in reference picture list 0, and signaling the number of active reference indexes of reference picture list 0 in slice headers for P slices and B slices if the number of entries in reference picture list 0 is greater than 1; and determining the number of entries in reference picture list 1, and signaling the number of active reference indexes of reference picture list 1 in a slice header for a B slice if the number of entries in reference picture list 1 is greater than 1; 20. A computer-implemented method comprising: 23. In response to the first flag indicating that there is no active reference index number in the slice header, skipping signaling the number of active reference indexes of reference picture list 0 in slice headers for P slices and B slices; and Skipping signaling active reference index number reference picture list 1 in slice header for B slices 23. The method of clause 22, further comprising: 24. A computer-implemented method for decoding video, comprising: receiving a video bitstream including slice header and picture header syntax; determining a value of a first flag signaled in a slice header indicating whether there is an active reference index number in the slice header, the active reference index number being used to derive a maximum reference index of a corresponding reference picture list that may be used to decode the current slice; in response to the first flag indicating that there is an active reference index number; determining the number of entries in reference picture list 0, and decoding the number of active reference indexes of reference picture list 0 in the slice header for P slices and B slices if the number of entries in reference picture list 0 is greater than 1; and determining the number of entries in reference picture list 1, and decoding the number of active reference indexes of reference picture list 1 in the slice header for the B slice if the number of entries in reference picture list 1 is greater than 1; 20. A computer-implemented method comprising: 25. In response to the first flag indicating that there is no active reference index number, skipping decoding the active reference index number of reference picture list 0 in the slice header for P slices and B slices; and Skipping decoding the active reference index number of reference picture list 1 in the slice header for B slices 25. The method of clause 24, further comprising: 26. A computer-implemented method for video processing, comprising: Determining a collocated picture referenced by a reference index of the collocated picture in a slice level, where the collocated picture is determined to be the same picture for all non-I slices of the current picture; and Processing a current picture based on a collocated picture, the collocated picture being used for temporal motion vector prediction. 20. A computer-implemented method comprising: 27. A computer-implemented method for video processing, comprising: Determining a co-located picture referenced by a reference index of the co-located picture in a slice level, where the co-located picture is determined to be the same picture for all P slices and B slices of the current picture; and Processing a current picture based on a collocated picture, the collocated picture being used for temporal motion vector prediction. 20. A computer-implemented method comprising: 28. An apparatus for processing video data, comprising: a memory configured to store instructions; and one or more processors, wherein the one or more processors encoding the current picture based on the co-located picture, where the co-located picture is used for temporal motion vector prediction; and signaling a first flag and a second flag in response to both the number of entries in reference picture list 0 and the number of entries in reference picture list 1 being greater than 0, wherein the first flag indicates that the co-located picture is derived from reference picture list 0 or reference picture list 1, and the second flag indicates whether a motion vector difference syntax structure is signaled. 10. An apparatus configured to execute instructions to cause the apparatus to: 29. An apparatus for processing video data, comprising: a memory configured to store instructions; and one or more processors, wherein the one or more processors receiving a video bitstream; decoding a first flag and a second flag in response to both the number of entries in reference picture list 0 and the number of entries in reference picture list 1 being greater than 0, wherein the first flag indicates that a co-located picture used for temporal motion vector prediction is derived from reference picture list 0 or reference picture list 1, and the second flag indicates whether a motion vector difference syntax structure is in the bitstream of the current picture; and Decoding a current picture based on a collocated picture 10. An apparatus configured to execute instructions to cause the apparatus to: 30. An apparatus for processing video data, comprising: a memory configured to store instructions; and one or more processors, wherein the one or more processors encoding the current picture based on the co-located picture, where the co-located picture is used for temporal motion vector prediction; and Indicating a co-located picture in a bitstream without signaling an index to a reference picture list - Patent Application 20070122997 10. An apparatus configured to execute instructions to cause the apparatus to: 31. The processor: signaling a first flag to indicate whether the co-located picture is an inter-layer reference picture; and signaling a first parameter for indicating the co-located picture in response to the co-located picture being an inter-layer reference picture, the first parameter indicating an index of the co-located picture relative to a list of direct reference layers of the layer in which the current picture resides; 31. The apparatus of clause 30, further configured to execute instructions to cause the apparatus to: 32. The processor: signaling a second flag to indicate whether the co-located picture is a short-term reference picture or a long-term reference picture; and signaling a second parameter for indicating the co-located picture in response to the co-located picture being a short-term reference picture, the second parameter indicating a difference between a picture order count of the co-located picture and a picture order count of the current picture; 32. The apparatus of clause 31, further configured to execute instructions to cause the apparatus to: 33. The processor: and signaling a third parameter and a fourth parameter for indicating the co-located picture in response to the co-located picture being a long-term reference picture, the third parameter indicating a least significant bit (LSB) of a picture order count (POC) of the co-located picture, and the fourth parameter indicating a delta most significant bit (MSB) of the picture order count (POC) of the co-located picture. 33. The apparatus of clause 32, further configured to execute instructions to cause the apparatus to: 34. The apparatus of clause 33, wherein the first flag, the second flag, the first parameter, the second parameter, the third parameter, and the fourth parameter are signaled in a picture header, and all slices in the picture have the same co-located picture. 35. The apparatus of clause 30, wherein the reference picture list is Reference Picture List 0 or Reference Picture List 1. 36. An apparatus for processing video data, comprising: a memory configured to store instructions; and one or more processors, wherein the one or more processors receiving a video bitstream; determining a co-located picture to be used for temporal motion vector prediction without decoding an index to a reference picture list; and Decoding a current picture based on a collocated picture 10. An apparatus configured to execute instructions to cause the apparatus to: 37. The processor: decoding a first flag indicating whether the co-located picture is an inter-layer reference picture; determining whether the co-located picture is an inter-layer reference picture based on a first flag; and and in response to the co-located picture being an inter-layer reference picture, decoding a first parameter and determining the co-located picture based on the first parameter, the first parameter indicating an index of the co-located picture relative to a list of direct reference layers of the layer in which the current picture resides. 37. The apparatus of clause 36, further configured to execute instructions to cause the apparatus to: 38. The processor: decoding a second flag indicating whether the co-located picture is a short-term reference picture or a long-term reference picture; determining whether the co-located picture is a short-term reference picture or a long-term reference picture based on a second flag; and and in response to the co-located picture being a short-term reference picture, decoding a second parameter and determining the co-located picture based on the second parameter, the second parameter indicating a difference between a picture order count of the co-located picture and a picture order count of the current picture. 38. The apparatus of clause 37, further configured to execute instructions to cause the apparatus to: 39. The processor: and in response to the co-located picture being a long-term reference picture, decoding a third parameter and a fourth parameter, and determining the co-located picture based on the third parameter and the fourth parameter, wherein the third parameter indicates a least significant bit (LSB) of a picture order count (POC) of the co-located picture, and the fourth parameter indicates a delta most significant bit (MSB) of the picture order count (POC) of the co-located picture. 39. The apparatus of clause 38, further configured to execute instructions to cause the apparatus to: 40. The apparatus of clause 39, wherein the first flag, the second flag, the first parameter, the second parameter, the third parameter, and the fourth parameter are in a picture header, and all slices in the picture have the same co-located picture. 41. The apparatus of clause 36, wherein the reference picture list is Reference Picture List 0 or Reference Picture List 1. 42. An apparatus for processing video data, comprising: a memory configured to store instructions; and one or more processors, wherein the one or more processors determining whether to signal a parameter in a slice header for indicating a reference index of the co-located picture; responsive to the parameter not being signaled in the slice header, determining the co-located picture as the picture referenced by an index having a value equal to the smaller of the value of the co-located picture's reference index signaled in the picture header and the number of active entries in the target reference picture list minus 1; and Encoding a current picture based on a collocated picture, the collocated picture being used for temporal motion vector prediction. 10. An apparatus configured to execute instructions to cause the apparatus to: 43. The apparatus of clause 42, wherein the target reference picture list is indicated by a flag indicating from which reference picture list the co-located picture used for temporal motion vector prediction is derived. 44. An apparatus for processing video data, comprising: a memory configured to store instructions; and one or more processors, wherein the one or more processors receiving a video bitstream; determining whether a parameter indicating a reference index of a co-located picture used for temporal motion vector prediction is present in the slice header; responsive to the absence of the parameter, determining a value of the parameter to be equal to the smaller of a value of a reference index of the co-located picture used for temporal motion vector prediction in the picture header and a number of active entries in the target reference picture list minus one; determining the co-located picture as the picture referenced by an index having a value equal to the value of the parameter in the target reference picture list; and Decoding a current picture based on a collocated picture 10. An apparatus configured to execute instructions to cause the apparatus to: 45. The apparatus of clause 44, wherein the target reference picture list is indicated by a flag indicating from which reference picture list the co-located picture used for temporal motion vector prediction is derived. 46. ​​An apparatus for processing video data, comprising: a memory configured to store instructions; and one or more processors, wherein the one or more processors deriving the total number by summing the number of reference picture list structures in a sequence parameter set (SPS) and 1; allocating memory for a total number of reference picture list structures in response to the reference picture list structures being signaled in a picture header of the current picture or a slice header of the current slice; and Processing the current picture or current slice using the allocated memory 10. An apparatus configured to execute instructions to cause the apparatus to: 47. An apparatus for processing video data, comprising: a memory configured to store instructions; and one or more processors, wherein the one or more processors signaling a first flag in a picture parameter set (PPS) to indicate whether a second flag and a first index are present in a picture header syntax or a slice header of the current picture that references the PPS, wherein the second flag indicates whether reference picture list 1 is derived based on one of reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index is an index of the reference picture list structure associated with reference picture list 1 used to derive reference picture list 1 to a list of reference picture list structures associated with reference picture list 1 included in the SPS; determining whether to signal a first index and a second index, wherein the second index is an index of a reference picture list structure associated with reference picture list 0 used to derive reference picture list 0 relative to a list of reference picture list structures associated with reference picture list 0 included in the SPS; and determining a value of the second index in response to the second index not being signaled, wherein determining the value of the second index includes: If at most one reference picture list structure associated with reference picture list 0 is included in the SPS, determining the value of the second index to be equal to 0. containing, determining a value of the first index in response to the first index not being signaled, wherein determining the value of the first index includes: determining a value of the first index to be equal to 0 if at most one reference picture list structure associated with reference picture list 1 is included in the SPS; and determining a value of the first index to be equal to a value of the second index if the first flag is equal to 0 and the second flag is equal to 1; containing, deriving a reference picture list based on the first index and the second index; and Encoding a current picture based on a reference picture list 10. An apparatus configured to execute instructions to cause the apparatus to: 48. An apparatus for processing video data, comprising: a memory configured to store instructions; and one or more processors, wherein the one or more processors receiving a video bitstream; determining a value of a first flag indicating whether a second flag and a first index are present in a picture header syntax or a slice header of the current picture, wherein the second flag indicates whether reference picture list 1 is derived based on one of reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index is an index of the reference picture list structure associated with reference picture list 1 used to derive reference picture list 1 to a list of reference picture list structures associated with reference picture list 1 included in the SPS; determining whether there is a first index and a second index, wherein the second index is an index of a reference picture list structure associated with reference picture list 0 used to derive reference picture list 0 relative to a list of reference picture list structures associated with reference picture list 0 that are included in the SPS; and determining a value of the second index in response to the absence of the second index, wherein determining the value of the second index includes: If at most one reference picture list structure associated with reference picture list 0 is included in the SPS, determining the value of the second index to be equal to 0. containing, and determining a value of the first index in response to the absence of the first index, wherein determining the value of the first index includes: determining a value of the first index to be equal to 0 if at most one reference picture list structure associated with reference picture list 1 is included in the SPS; and determining a value of the first index to be equal to a value of the second index if the first flag is equal to 0 and the second flag is equal to 1; containing, Decoding the current picture based on the first index and the second index 10. An apparatus configured to execute instructions to cause the apparatus to: 49. An apparatus for processing video data, comprising: a memory configured to store instructions; and one or more processors, wherein the one or more processors signaling a first flag in the slice header to indicate whether there is an active reference index number in the slice header, the active reference index number being used to derive a maximum reference index of a corresponding reference picture list that may be used to encode the current slice; in response to a first flag indicating an active reference index number in the slice header; determining the number of entries in reference picture list 0, and signaling the number of active reference indexes of reference picture list 0 in slice headers for P slices and B slices if the number of entries in reference picture list 0 is greater than 1; and determining the number of entries in reference picture list 1, and signaling the number of active reference indexes of reference picture list 1 in a slice header for a B slice if the number of entries in reference picture list 1 is greater than 1; 10. An apparatus configured to execute instructions to cause the apparatus to: 50. The processor: in response to a first flag indicating that there is no active reference index number in the slice header; skipping signaling the number of active reference indexes of reference picture list 0 in slice headers for P slices and B slices; and Skipping signaling active reference index number reference picture list 1 in slice header for B slices 49. The apparatus of claim 49, further configured to execute instructions to cause the apparatus to: 51. An apparatus for processing video data, comprising: a memory configured to store instructions; and one or more processors, wherein the one or more processors receiving a video bitstream including slice header and picture header syntax; determining a value of a first flag signaled in a slice header indicating whether there is an active reference index number in the slice header, the active reference index number being used to derive a maximum reference index of a corresponding reference picture list that may be used to decode the current slice; in response to the first flag indicating that there is an active reference index number; determining the number of entries in reference picture list 0, and decoding the number of active reference indexes of reference picture list 0 in the slice header for P slices and B slices if the number of entries in reference picture list 0 is greater than 1; and determining the number of entries in reference picture list 1, and decoding the number of active reference indexes of reference picture list 1 in the slice header for the B slice if the number of entries in reference picture list 1 is greater than 1; 10. An apparatus configured to execute instructions to cause the apparatus to: 52. The processor in response to the first flag indicating that there is no active reference index number; skipping decoding the active reference index number of reference picture list 0 in the slice header for P slices and B slices; and Skipping decoding the active reference index number of reference picture list 1 in the slice header for B slices 51. The apparatus of clause 50, further configured to execute instructions to cause the apparatus to: 53. An apparatus for processing video data, comprising: a memory configured to store instructions; and one or more processors, wherein the one or more processors Determining a collocated picture referenced by a reference index of the collocated picture in a slice level, where the collocated picture is determined to be the same picture for all non-I slices of the current picture; and Processing a current picture based on a collocated picture, the collocated picture being used for temporal motion vector prediction. 10. An apparatus configured to execute instructions to cause the apparatus to: 54. An apparatus for processing video data, comprising: a memory configured to store instructions; and one or more processors, wherein the one or more processors Determining a co-located picture referenced by a reference index of the co-located picture in a slice level, where the co-located picture is determined to be the same picture for all P slices and B slices of the current picture; and Processing a current picture based on a collocated picture, the collocated picture being used for temporal motion vector prediction. 10. An apparatus configured to execute instructions to cause the apparatus to: 55. A non-transitory computer-readable medium storing a set of instructions, the set of instructions executable by one or more processors of a device to cause the device to initiate a method for video data processing, the method comprising: encoding the current picture based on the co-located picture, where the co-located picture is used for temporal motion vector prediction; and signaling a first flag and a second flag in response to both the number of entries in reference picture list 0 and the number of entries in reference picture list 1 being greater than 0, wherein the first flag indicates that the co-located picture is derived from reference picture list 0 or reference picture list 1, and the second flag indicates whether a motion vector difference syntax structure is signaled. 1. A non-transitory computer-readable medium comprising: 56. A non-transitory computer-readable medium storing a set of instructions, the set of instructions executable by one or more processors of a device to cause the device to initiate a method for video data processing, the method comprising: receiving a video bitstream; decoding a first flag and a second flag in response to both the number of entries in reference picture list 0 and the number of entries in reference picture list 1 being greater than 0, wherein the first flag indicates that a co-located picture used for temporal motion vector prediction is derived from reference picture list 0 or reference picture list 1, and the second flag indicates whether a motion vector difference syntax structure is in the bitstream of the current picture; and Decoding a current picture based on a collocated picture 1. A non-transitory computer-readable medium comprising: 57. A non-transitory computer-readable medium storing a set of instructions, the set of instructions executable by one or more processors of a device to cause the device to initiate a method for video data processing, the method comprising: encoding the current picture based on the co-located picture, where the co-located picture is used for temporal motion vector prediction; and Indicating a co-located picture in a bitstream without signaling an index to a reference picture list - Patent Application 20070122997 1. A non-transitory computer-readable medium comprising: 58. The method comprises: signaling a first flag to indicate whether the co-located picture is an inter-layer reference picture; and signaling a first parameter for indicating the co-located picture in response to the co-located picture being an inter-layer reference picture, the first parameter indicating an index of the co-located picture relative to a list of direct reference layers of the layer in which the current picture resides; 58. The non-transitory computer-readable medium of clause 57, further comprising: 59. The method comprises: signaling a second flag to indicate whether the co-located picture is a short-term reference picture or a long-term reference picture; and signaling a second parameter for indicating the co-located picture in response to the co-located picture being a short-term reference picture, the second parameter indicating a difference between a picture order count of the co-located picture and a picture order count of the current picture. 59. The non-transitory computer-readable medium of clause 58, further comprising: 60. The method comprises: and signaling a third parameter and a fourth parameter for indicating the co-located picture in response to the co-located picture being a long-term reference picture, the third parameter indicating a least significant bit (LSB) of a picture order count (POC) of the co-located picture, and the fourth parameter indicating a delta most significant bit (MSB) of the picture order count (POC) of the co-located picture. 59. The non-transitory computer-readable medium of claim 59, further comprising: 61. The non-transitory computer-readable medium of clause 60, wherein the first flag, the second flag, the first parameter, the second parameter, the third parameter, and the fourth parameter are signaled in a picture header, and all slices in the picture have the same co-located picture. 62. The non-transitory computer-readable medium of clause 57, wherein the reference picture list is Reference Picture List 0 or Reference Picture List 1. 63. A non-transitory computer-readable medium storing a set of instructions, the set of instructions executable by one or more processors of a device to cause the device to initiate a method for video data processing, the method comprising: receiving a video bitstream; determining a co-located picture to be used for temporal motion vector prediction without decoding an index to a reference picture list; and Decoding a current picture based on a collocated picture 1. A non-transitory computer-readable medium comprising: 64. The method comprises: decoding a first flag indicating whether the co-located picture is an inter-layer reference picture; determining whether the co-located picture is an inter-layer reference picture based on a first flag; and and in response to the co-located picture being an inter-layer reference picture, decoding a first parameter and determining the co-located picture based on the first parameter, the first parameter indicating an index of the co-located picture relative to a list of direct reference layers of the layer in which the current picture resides. 64. The non-transitory computer-readable medium of clause 63, further comprising: 65. The method comprises: decoding a second flag indicating whether the co-located picture is a short-term reference picture or a long-term reference picture; determining whether the co-located picture is a short-term reference picture or a long-term reference picture based on a second flag; and and in response to the co-located picture being a short-term reference picture, decoding a second parameter and determining the co-located picture based on the second parameter, the second parameter indicating a difference between a picture order count of the co-located picture and a picture order count of the current picture. 65. The non-transitory computer-readable medium of clause 64, further comprising: 66. The method comprises: and in response to the co-located picture being a long-term reference picture, decoding a third parameter and a fourth parameter, and determining the co-located picture based on the third parameter and the fourth parameter, wherein the third parameter indicates a least significant bit (LSB) of a picture order count (POC) of the co-located picture, and the fourth parameter indicates a delta most significant bit (MSB) of the picture order count (POC) of the co-located picture. 66. The non-transitory computer-readable medium of clause 65, further comprising: 67. The non-transitory computer-readable medium of clause 66, wherein the first flag, the second flag, the first parameter, the second parameter, the third parameter, and the fourth parameter are in a picture header, and all slices in the picture have the same co-located picture. 68. The non-transitory computer-readable medium of clause 63, wherein the reference picture list is Reference Picture List 0 or Reference Picture List 1. 69. A non-transitory computer-readable medium storing a set of instructions, the set of instructions executable by one or more processors of a device to cause the device to initiate a method for video data processing, the method comprising: determining whether to signal a parameter in a slice header for indicating a reference index of the co-located picture; responsive to the parameter not being signaled in the slice header, determining the co-located picture as the picture referenced by an index having a value equal to the smaller of the value of the co-located picture's reference index signaled in the picture header and the number of active entries in the target reference picture list minus 1; and Encoding a current picture based on a collocated picture, the collocated picture being used for temporal motion vector prediction. 1. A non-transitory computer-readable medium comprising: 70. The non-transitory computer-readable medium of clause 69, wherein the target reference picture list is indicated by a flag indicating from which reference picture list the co-located picture used for temporal motion vector prediction is derived. 71. A non-transitory computer-readable medium storing a set of instructions, the set of instructions executable by one or more processors of a device to cause the device to initiate a method for video data processing, the method comprising: receiving a video bitstream; determining whether a parameter indicating a reference index of a co-located picture used for temporal motion vector prediction is present in the slice header; responsive to the absence of the parameter, determining a value of the parameter to be equal to the smaller of a value of a reference index of the co-located picture used for temporal motion vector prediction in the picture header and a number of active entries in the target reference picture list minus one; determining the co-located picture as the picture referenced by an index having a value equal to the value of the parameter in the target reference picture list; and Decoding a current picture based on a collocated picture 1. A non-transitory computer-readable medium comprising: 72. The non-transitory computer-readable medium of clause 71, wherein the target reference picture list is indicated by a flag indicating from which reference picture list the co-located picture used for temporal motion vector prediction is derived. 73. A non-transitory computer-readable medium storing a set of instructions, the set of instructions executable by one or more processors of a device to cause the device to initiate a method for video data processing, the method comprising: deriving the total number by summing the number of reference picture list structures in a sequence parameter set (SPS) and 1; allocating memory for a total number of reference picture list structures in response to the reference picture list structures being signaled in a picture header of the current picture or a slice header of the current slice; and Processing the current picture or current slice using the allocated memory 1. A non-transitory computer-readable medium comprising: 74. A non-transitory computer-readable medium storing a set of instructions, the set of instructions executable by one or more processors of a device to cause the device to initiate a method for video data processing, the method comprising: signaling a first flag in a picture parameter set (PPS) to indicate whether a second flag and a first index are present in a picture header syntax or a slice header of the current picture that references the PPS, wherein the second flag indicates whether reference picture list 1 is derived based on one of reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index is an index of the reference picture list structure associated with reference picture list 1 used to derive reference picture list 1 to a list of reference picture list structures associated with reference picture list 1 included in the SPS; determining whether to signal a first index and a second index, wherein the second index is an index of a reference picture list structure associated with reference picture list 0 used to derive reference picture list 0 relative to a list of reference picture list structures associated with reference picture list 0 included in the SPS; and determining a value of the second index in response to the second index not being signaled, wherein determining the value of the second index includes: If at most one reference picture list structure associated with reference picture list 0 is included in the SPS, determining the value of the second index to be equal to 0. containing, determining a value of the first index in response to the first index not being signaled, wherein determining the value of the first index comprises m determining a value of the first index to be equal to 0 if at most one reference picture list structure associated with reference picture list 1 is included in the SPS; and determining a value of the first index to be equal to a value of the second index if the first flag is equal to 0 and the second flag is equal to 1; containing, deriving a reference picture list based on the first index and the second index; and Encoding a current picture based on a reference picture list 1. A non-transitory computer-readable medium comprising: 75. A non-transitory computer-readable medium storing a set of instructions, the set of instructions executable by one or more processors of a device to cause the device to initiate a method for video data processing, the method comprising: receiving a video bitstream; determining a value of a first flag indicating whether a second flag and a first index are present in a picture header syntax or a slice header of the current picture, wherein the second flag indicates whether reference picture list 1 is derived based on one of reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index is an index of the reference picture list structure associated with reference picture list 1 used to derive reference picture list 1 to a list of reference picture list structures associated with reference picture list 1 included in the SPS; determining whether there is a first index and a second index, wherein the second index is an index of a reference picture list structure associated with reference picture list 0 used to derive reference picture list 0 relative to a list of reference picture list structures associated with reference picture list 0 that are included in the SPS; and determining a value of the second index in response to the absence of the second index, wherein determining the value of the second index includes: If at most one reference picture list structure associated with reference picture list 0 is included in the SPS, determining the value of the second index to be equal to 0. containing, and determining a value of the first index in response to the absence of the first index, wherein determining the value of the first index includes: determining a value of the first index to be equal to 0 if at most one reference picture list structure associated with reference picture list 1 is included in the SPS; and determining a value of the first index to be equal to a value of the second index if the first flag is equal to 0 and the second flag is equal to 1; containing, Decoding the current picture based on the first index and the second index 1. A non-transitory computer-readable medium comprising: 76. A non-transitory computer-readable medium storing a set of instructions, the set of instructions executable by one or more processors of a device to cause the device to initiate a method for video data processing, the method comprising: signaling a first flag in the slice header to indicate whether there is an active reference index number in the slice header, the active reference index number being used to derive a maximum reference index of a corresponding reference picture list that may be used to encode the current slice; in response to a first flag indicating an active reference index number in the slice header; determining the number of entries in reference picture list 0, and signaling the number of active reference indexes of reference picture list 0 in slice headers for P slices and B slices if the number of entries in reference picture list 0 is greater than 1; and determining the number of entries in reference picture list 1, and signaling the number of active reference indexes of reference picture list 1 in a slice header for a B slice if the number of entries in reference picture list 1 is greater than 1; 1. A non-transitory computer-readable medium comprising: 77. The method comprises: in response to a first flag indicating that there is no active reference index number in the slice header; skipping signaling the number of active reference indexes of reference picture list 0 in slice headers for P slices and B slices; and Skipping signaling active reference index number reference picture list 1 in slice header for B slices 77. The non-transitory computer-readable medium of clause 76, further comprising: 78. A non-transitory computer-readable medium storing a set of instructions, the set of instructions executable by one or more processors of a device to cause the device to initiate a method for video data processing, the method comprising: receiving a video bitstream including slice header and picture header syntax; determining a value of a first flag signaled in a slice header indicating whether there is an active reference index number in the slice header, the active reference index number being used to derive a maximum reference index of a corresponding reference picture list that may be used to decode the current slice; in response to the first flag indicating that there is an active reference index number; determining the number of entries in reference picture list 0, and decoding the number of active reference indexes of reference picture list 0 in the slice header for P slices and B slices if the number of entries in reference picture list 0 is greater than 1; and determining the number of entries in reference picture list 1, and decoding the number of active reference indexes of reference picture list 1 in the slice header for the B slice if the number of entries in reference picture list 1 is greater than 1; 1. A non-transitory computer-readable medium comprising: 79. The method comprises: in response to the first flag indicating that there is no active reference index number; skipping decoding the active reference index number of reference picture list 0 in the slice header for P slices and B slices; and Skipping decoding the active reference index number of reference picture list 1 in the slice header for B slices 79. The non-transitory computer-readable medium of clause 78, further comprising: 80. A non-transitory computer-readable medium storing a set of instructions, the set of instructions executable by one or more processors of a device to cause the device to initiate a method for video data processing, the method comprising: Determining a collocated picture referenced by a reference index of the collocated picture in a slice level, where the collocated picture is determined to be the same picture for all non-I slices of the current picture; and Processing a current picture based on a collocated picture, the collocated picture being used for temporal motion vector prediction. 1. A non-transitory computer-readable medium comprising: 81. A non-transitory computer-readable medium storing a set of instructions, the set of instructions executable by one or more processors of a device to cause the device to initiate a method for video data processing, the method comprising: Determining a co-located picture referenced by a reference index of the co-located picture in a slice level, where the co-located picture is determined to be the same picture for all P slices and B slices of the current picture; and Processing a current picture based on a collocated picture, the collocated picture being used for temporal motion vector prediction. 1. A non-transitory computer-readable medium comprising:

[0266]

[0309] In the foregoing specification, embodiments have been described with reference to numerous specific details that may vary from implementation to implementation. Certain adaptations and modifications of the above-described embodiments may be made. Other embodiments may be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. It is intended that the specification and examples be considered as examples only, with the true scope and spirit of the invention being indicated by the appended claims. It is also intended that the sequences of steps depicted in the figures are for illustrative purposes only and are not intended to be limited to any particular sequence of steps. Thus, one skilled in the art will recognize that these steps may be performed in different orders while implementing the same method.

[0267]

[0310] In the drawings and specification, illustrative embodiments have been disclosed. However, many variations and modifications to these embodiments may be made. Accordingly, although specific terms are employed, they are used in a generic and descriptive sense only and not for purposes of limitation.

Claims

1. 1. A computer-implemented method for video processing, comprising: deriving the total number by summing the number of reference picture list structures in a sequence parameter set (SPS) and 1; allocating memory for the total number of reference picture list structures in response to a reference picture list structure being signaled in a picture header of the current picture or a slice header of the current slice; and processing the current picture or the current slice using the allocated memory; 20. A computer-implemented method comprising:

2. An apparatus for processing video data, comprising: a memory configured to store instructions; one or more processors, wherein the one or more processors: signaling a first flag in a Picture Parameter Set (PPS) to indicate whether a second flag and a first index are present in a picture header syntax or a slice header of the current picture referencing a PPS, wherein the second flag indicates whether reference picture list 1 is derived based on one of reference picture list structures associated with reference picture list 1 signaled in a Sequence Parameter Set (SPS), and the first index is the index of the reference picture list structure associated with reference picture list 1 used to derive reference picture list 1 relative to the list of reference picture list structures associated with reference picture list 1 included in the SPS; determining whether to signal the first index and the second index, wherein the second index is an index of the reference picture list structure associated with reference picture list 0 used to derive reference picture list 0 relative to the list of reference picture list structures associated with reference picture list 0 included in the SPS; determining a value of the second index in response to the second index not being signaled, wherein determining the value of the second index includes: determining the value of the second index to be equal to 0 if at most one reference picture list structure associated with reference picture list 0 is included in the SPS; determining a value of the first index in response to the first index not being signaled, wherein determining the value of the first index includes: determining the value of the first index to be equal to 0 if at most one reference picture list structure associated with reference picture list 1 is included in the SPS; and determining the value of the first index to be equal to the value of the second index if the first flag is equal to 0 and the second flag is equal to 1; containing, deriving the reference picture list based on the first index and the second index; and encoding the current picture based on the reference picture list; an apparatus configured to execute the instructions to cause the apparatus to:

3. A non-transitory computer-readable medium storing a set of instructions, the set of instructions executable by one or more processors of an apparatus to cause the apparatus to initiate a method for video data processing, the method comprising: receiving a video bitstream; determining a value of a first flag indicating whether a second flag and a first index are present in a picture header syntax or a slice header of a current picture, wherein the second flag indicates whether reference picture list 1 is derived based on one of the reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index is the index of the reference picture list structure associated with reference picture list 1 used to derive reference picture list 1 relative to the list of reference picture list structures associated with reference picture list 1 included in the SPS; determining whether the first index and the second index are present, wherein the second index is the index of the reference picture list structure associated with reference picture list 0 used to derive reference picture list 0 relative to the list of reference picture list structures associated with reference picture list 0 included within the SPS; determining a value of the second index in response to the second index being absent, wherein determining the value of the second index includes: determining the value of the second index to be equal to 0 if at most one reference picture list structure associated with reference picture list 0 is included in the SPS; determining a value of the first index in response to the first index being absent, wherein determining the value of the first index includes: determining the value of the first index to be equal to 0 if at most one reference picture list structure associated with reference picture list 1 is included in the SPS; and determining the value of the first index to be equal to the value of the second index if the first flag is equal to 0 and the second flag is equal to 1; containing, decoding the current picture based on the first index and the second index; 1. A non-transitory computer-readable medium comprising:

4. 1. A computer-implemented method for encoding video, comprising: signaling a first flag in a slice header to indicate whether there is an active reference index number in the slice header, the active reference index number being used to derive a maximum reference index of a corresponding reference picture list used to encode the current slice; in response to the first flag indicating the presence of the active reference index number in the slice header; determining the number of entries in reference picture list 0, and signaling the number of active reference indexes in reference picture list 0 in the slice header for P slices and B slices if the number of entries in reference picture list 0 is greater than 1; and determining the number of entries in reference picture list 1, and signaling the number of active reference indexes in reference picture list 1 in the slice header for a B slice if the number of entries in reference picture list 1 is greater than 1; 20. A computer-implemented method comprising:

5. In response to the first flag indicating that the active reference index number is absent in the slice header, skipping signaling the number of active reference indexes of reference picture list 0 in the slice header for P slices and B slices; and Skipping signaling the active reference index number reference picture list 1 in a slice header for a B slice The method of claim 4 further comprising:

6. An apparatus for processing video data, comprising: a memory configured to store instructions; one or more processors, wherein the one or more processors: receiving a video bitstream including slice header and picture header syntax; determining a value of a first flag signaled in the slice header indicating whether there is an active reference index number in the slice header, the active reference index number being used to derive a maximum reference index of a corresponding reference picture list that may be used to decode the current slice; In response to the first flag indicating that there is an active reference index number, determining a number of entries in reference picture list 0, and if the number of entries in reference picture list 0 is greater than 1, decoding the number of active reference indexes in reference picture list 0 in the slice header for P slices and B slices; and determining the number of entries in reference picture list 1, and if the number of entries in reference picture list 1 is greater than 1, decoding the number of active reference indexes in reference picture list 1 in a slice header for a B slice; an apparatus configured to execute the instructions to cause the apparatus to:

7. the processor: in response to the first flag indicating that there are no active reference indexes; skipping decoding the active reference index number of reference picture list 0 in the slice header for P slices and B slices; and Skipping decoding the active reference index number of reference picture list 1 in the slice header for a B slice 7. The apparatus of claim 6, further configured to execute the instructions to cause the apparatus to:

8. A non-transitory computer-readable medium storing a set of instructions, the set of instructions executable by one or more processors of an apparatus to cause the apparatus to initiate a method for video data processing, the method comprising: Determining a collocated picture referenced by a reference index of the collocated picture in a slice level, where the collocated picture is determined to be the same picture for all non-I slices of the current picture; and processing the current picture based on the co-located picture, the co-located picture being used for temporal motion vector prediction; 1. A non-transitory computer-readable medium comprising:

Images