Computer implementation method and device for video data processing and storage medium
By optimizing the encoding and decoding process of reference image list in video data processing, the problem of low efficiency in video data processing in the prior art is solved, and more efficient storage and transmission efficiency is achieved.
Patent Information
- Application Number
- CN202411126842.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2020-05-21
- Filing Date
- 2021-05-21
- Publication Date
- 2024-11-29
AI Technical Summary
When existing video encoding technologies process video data, it is difficult to efficiently manage and decode reference image lists, resulting in low storage and transmission bandwidth efficiency and limited encoding efficiency.
By deriveing the number of reference image list structures in the sequence parameter set (SPS), and performing memory allocation and processing based on the signals in the image header or strip header, the encoding and decoding process of the reference image list includes judging and sending related indexes. To improve encoding and decoding efficiency.
It improves the storage and transmission efficiency of video data processing, reduces the computational complexity of encoding and decoding, and enhances the performance of video encoding and decoding.
Smart Images

Figure CN119052501A_ABST
Abstract
Description
[0001] This application is a divisional application of the original Chinese invention patent application entitled “Computer-implemented method, device and storage medium for video data processing”. The application number of the original application is 202180031988.6, and the application date of the original application is May 21, 2021.
[0002] CROSS-REFERENCE TO RELATED APPLICATIONS
[0003] This disclosure claims priority to U.S. Provisional Application No. 63 / 028,509, filed on May 21, 2020, the entire contents of which are incorporated herein by reference. Technical Field
[0004] The present disclosure relates generally to video processing, and more particularly to computer-implemented methods, apparatus, and storage media for performing video data processing. Background Art
[0005] A video is a set of static images (or "frames") that capture visual information. In order to reduce storage memory and transmission bandwidth, the video can be compressed before storage or transmission, and then decompressed before display. The compression process is usually called encoding, and the decompression process is usually called decoding. There are various video coding formats that use standardized video coding techniques, the most common of which are based on prediction, transform, quantization, entropy coding, and loop filtering. Video coding standards, such as the High Efficiency Video Coding (HEVC / H.265) standard, the Versatile Video Coding (VVC / H.266) standard, and the AVS standard, specify specific video coding formats and are developed by standardization organizations. As more and more advanced video coding technologies are adopted in video standards, the coding efficiency of new video coding standards is also getting higher and higher. Summary of the invention
[0006] The disclosed embodiments provide a computer-implemented method for performing video data processing. In some embodiments, the method includes: deriving a total number by adding the number of reference picture list structures in a sequence parameter set (SPS) to 1; allocating memory for the total number of reference picture list structures in response to signaling the reference picture list structures in a picture header of a current picture or a slice header of a current slice; and processing the current picture or the current slice using the allocated memory.
[0007] In some embodiments, the method includes: signaling a first flag in a picture parameter set (PPS) to indicate whether a second flag and a first index are present in a picture header syntax or a slice header of a current picture referencing the PPS, wherein the second flag indicates whether reference picture list 1 is derived based on one of the reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index is an index in a list of reference picture list structures associated with reference picture list 1 included in the SPS for deriving reference picture list 1; determining whether to signal the first index and the second index, wherein the second index is an index in a list of reference picture list structures associated with reference picture list 0 for deriving reference picture list 0 The table structure includes an index in a list of reference image list structures associated with reference image list 0 included in the SPS; in response to not signaling the second index, determining the value of the second index includes: when the SPS includes at most one reference image list structure associated with reference image list 0, determining that the value of the second index is equal to 0; in response to not signaling the first index, determining the value of the first index includes: when the SBS includes at most one reference image list structure associated with reference image list 1, determining that the value of the first index is equal to 0; and when the first flag is equal to 0 and the second flag is equal to 1, determining that the value of the first index is equal to the value of the second index; deriving the reference image list based on the first index and the second index; and encoding the current image based on the reference image list.
[0008] In some embodiments, the method comprises: receiving a video bitstream; determining a value of a first flag, the first flag indicating whether a second flag and a first index are present in a picture header syntax or a slice header of a current picture, wherein the second flag indicates whether reference picture list 1 is derived based on one of the reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index is an index in a list of reference picture list structures associated with reference picture list 1 included in the SPS for deriving reference picture list 1; determining whether the first index and the second index are present, wherein the second index is an index in a list of reference picture list structures associated with reference picture list 1 included in the SPS for deriving reference picture list 0 an index of a reference image list structure associated with image list 0 in a list of reference image list structures associated with reference image list 0 included in an SPS; in response to the non-existence of a second index, determining the value of the second index comprises: when the SPS includes at most one reference image list structure associated with reference image list 0, determining that the value of the second index is equal to 0; in response to the non-existence of a first index, determining the value of the first index comprises: when the SBS includes at most one reference image list structure associated with reference image list 1, determining that the value of the first index is equal to 0; and when the first flag is equal to 0 and the second flag is equal to 1, determining that the value of the first index is equal to the value of the second index; and decoding the current image based on the first index and the second index.
[0009] In some embodiments, the method includes: signaling a first flag in a slice header to indicate whether there is an active reference index number in the slice header, wherein the active reference index number is used to derive a maximum reference index of a corresponding reference image list for encoding a current slice; in response to the first flag indicating the presence of the active reference index number in the slice header, determining the number of entries in reference image list 0, and when the number of entries in reference image list 0 is greater than 1, signaling the active reference index number of reference image list 0 in slice headers of P and B slices; and determining the number of entries in reference image list 1, and when the number of entries in reference image list 1 is greater than 1, signaling the active reference index number of reference image list 1 in a slice header of a B slice.
[0010] In some embodiments, the method includes: receiving a video bitstream including a slice header and a picture header syntax; determining a value of a first flag signaled in the slice header, the first flag indicating whether there is an activated reference index number in the slice header, wherein the activated reference index number is used to derive a maximum reference index of a corresponding reference picture list for decoding a current slice; in response to the first flag indicating the presence of the activated reference index number, determining the number of entries in reference picture list 0, and when the number of entries in reference picture list 0 is greater than 1, decoding the activated reference index number of reference picture list 0 in the slice header of P and B slices; and determining the number of entries in reference picture list 1, and when the number of entries in reference picture list 1 is greater than 1, decoding the activated reference index number of reference picture list 1 in the slice header of the B slice.
[0011] In some embodiments, the method includes: determining a collocated picture referenced by a reference index of the collocated picture at a slice level, wherein the collocated picture is determined to be the same picture for all non-I slices of the current picture; and processing the current picture based on the collocated picture, wherein the collocated picture is used for temporal motion vector prediction.
[0012] The disclosed embodiments provide a device for performing video processing. In some embodiments, the device includes: a memory for storing instructions; and one or more processors for executing the instructions so that the device performs: deriving a total number by adding the number of reference picture list structures in a sequence parameter set (SPS) to 1; in response to signaling the reference picture list structure in an image header of a current image or a slice header of a current slice, allocating memory for the total number of reference picture list structures; and processing the current image or the current slice using the allocated memory.
[0013] In some embodiments, the device includes: a memory for storing instructions; and one or more processors for executing the instructions so that the device performs: signaling a first flag in a picture parameter set (PPS) to indicate whether a second flag and a first index are present in a picture header syntax or a slice header of a current picture that references the PPS, wherein the second flag indicates whether reference picture list 1 is derived based on one of the reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index is an index in a list of reference picture list structures associated with reference picture list 1 included in the SPS for deriving reference picture list 1; determining whether to signal the first index and the second index, wherein the second index is an index in a list of reference picture list structures associated with reference picture list 1 included in the SPS for deriving reference picture list 1; an index of a reference image list structure associated with reference image list 0 in a list of reference image list structures associated with reference image list 0 included in an SPS; in response to not signaling a second index, determining a value of the second index comprises: when the SPS includes at most one reference image list structure associated with reference image list 0, determining that the value of the second index is equal to 0; in response to not signaling a first index, determining a value of the first index comprises: when the SBS includes at most one reference image list structure associated with reference image list 1, determining that the value of the first index is equal to 0; and when the first flag is equal to 0 and the second flag is equal to 1, determining that the value of the first index is equal to the value of the second index; deriving a reference image list based on the first index and the second index; and encoding a current image based on the reference image list.
[0014] In some embodiments, the apparatus includes: a memory for storing instructions; and one or more processors for executing the instructions so that the apparatus performs: receiving a video bitstream; determining a value of a first flag, the first flag indicating whether a second flag and a first index are present in a picture header syntax or a slice header of a current picture, wherein the second flag indicates whether a reference picture list 1 is derived based on one of the reference picture list structures associated with the reference picture list 1 signaled in a sequence parameter set (SPS), and the first index is an index in a list of reference picture list structures associated with the reference picture list 1 included in the SPS for deriving the reference picture list 1; determining whether the first index and the second index are present, wherein the second index indicates whether a reference picture list 1 is derived based on one of the reference picture list structures associated with the reference picture list 1 signaled in a sequence parameter set (SPS); and determining whether the first index and the second index are present, wherein the second index indicates whether a reference picture list 1 is derived based on one of the reference picture list structures associated with the reference picture list 1 signaled in the SPS. The reference is an index in a list of reference image list structures associated with reference image list 0 included in the SPS for deriving reference image list 0; in response to the second index not existing, determining the value of the second index includes: when the SPS includes at most one reference image list structure associated with reference image list 0, determining that the value of the second index is equal to 0; in response to the first index not existing, determining the value of the first index includes: when the SBS includes at most one reference image list structure associated with reference image list 1, determining that the value of the first index is equal to 0; and when the first flag is equal to 0 and the second flag is equal to 1, determining that the value of the first index is equal to the value of the second index; and decoding the current image based on the first index and the second index.
[0015] In some embodiments, the device includes: a memory for storing instructions; and one or more processors for executing the instructions so that the device performs: sending a first flag in a slice header with a signal to indicate whether there is an activated reference index number in the slice header, wherein the activated reference index number is used to derive the maximum reference index of the corresponding reference image list used to encode the current slice; in response to the first flag indicating that there is an activated reference index number in the slice header, determining the number of entries in reference image list 0, and when the number of entries in reference image list 0 is greater than 1, sending the activated reference index number of reference image list 0 with a signal in the slice header of the P and B slices; and determining the number of entries in reference image list 1, and when the number of entries in reference image list 1 is greater than 1, sending the activated reference index number of reference image list 1 with a signal in the slice header of the B slice.
[0016] In some embodiments, the device includes: a memory for storing instructions; and one or more processors for executing the instructions so that the device performs: receiving a video bitstream including a slice header and a picture header syntax; determining a value of a first flag sent by a signal in the slice header, the first flag indicating whether there is an activated reference index number in the slice header, wherein the activated reference index number is used to derive a maximum reference index of a corresponding reference picture list for decoding a current slice; in response to the first flag indicating the presence of the activated reference index number, determining the number of entries in reference picture list 0, and when the number of entries in reference picture list 0 is greater than 1, decoding the activated reference index number of reference picture list 0 in the slice header of P and B slices; and determining the number of entries in reference picture list 1, and when the number of entries in reference picture list 1 is greater than 1, decoding the activated reference index number of reference picture list 1 in the slice header of the B slice.
[0017] In some embodiments, the device includes: a memory for storing instructions; and one or more processors for executing the instructions so that the device performs: determining a collocated image referenced by a reference index of the collocated image at a slice level, wherein the collocated image is determined to be the same image for all non-I slices of a current image; and processing the current image based on the collocated image, wherein the collocated image is used for temporal motion vector prediction.
[0018] The disclosed embodiments provide a non-transitory computer-readable storage medium storing an instruction set, which is executed by one or more processors of a device, so that the device starts to execute a method for performing video processing. In some embodiments, the method includes: deriving a total number by adding the number of reference picture list structures in a sequence parameter set (SPS) to 1; in response to signaling the reference picture list structure in an image header of a current image or a slice header of a current slice, allocating memory for the total number of reference picture list structures; and processing the current image or the current slice using the allocated memory.
[0019] In some embodiments, the method includes: signaling a first flag in a picture parameter set (PPS) to indicate whether a second flag and a first index are present in a picture header syntax or a slice header of a current picture that references the PPS, wherein the second flag indicates whether reference picture list 1 is derived based on one of the reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index is an index in a list of reference picture list structures associated with reference picture list 1 included in the SPS for deriving reference picture list 1; determining whether to signal the first index and the second index, wherein the second index is an index in a list of reference picture list structures associated with reference picture list 0 included in the SPS for deriving reference picture list 0 An index of an associated reference image list structure in a list of reference image list structures associated with reference image list 0 included in an SPS; in response to not signaling a second index, determining a value of the second index includes: when the SPS includes at most one reference image list structure associated with reference image list 0, determining that the value of the second index is equal to 0; in response to not signaling a first index, determining a value of the first index includes: when the SBS includes at most one reference image list structure associated with reference image list 1, determining that the value of the first index is equal to 0; and when the first flag is equal to 0 and the second flag is equal to 1, determining that the value of the first index is equal to the value of the second index; deriving a reference image list based on the first index and the second index; and encoding a current image based on the reference image list.
[0020] In some embodiments, the method comprises: receiving a video bitstream; determining a value of a first flag, the first flag indicating whether a second flag and a first index are present in a picture header syntax or a slice header of a current picture, wherein the second flag indicates whether reference picture list 1 is derived based on one of the reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set SPS, and the first index is an index in a list of reference picture list structures associated with reference picture list 1 included in the SPS for deriving reference picture list 1; determining whether the first index and the second index are present, wherein the second index is an index in a list of reference picture list structures associated with reference picture list 1 included in the SPS for deriving reference picture list 0; an index of a reference image list structure associated with reference image list 0 in a list of reference image list structures associated with reference image list 0 included in an SPS; in response to the absence of a second index, determining the value of the second index includes: when the SPS includes at most one reference image list structure associated with reference image list 0, determining that the value of the second index is equal to 0; in response to the absence of a first index, determining the value of the first index includes: when the SBS includes at most one reference image list structure associated with reference image list 1, determining that the value of the first index is equal to 0; and when the first flag is equal to 0 and the second flag is equal to 1, determining that the value of the first index is equal to the value of the second index; and decoding the current image based on the first index and the second index.
[0021] In some embodiments, the method includes: signaling a first flag in a slice header to indicate whether there is an activated reference index number in the slice header, wherein the activated reference index number is used to derive a maximum reference index of a corresponding reference image list for encoding a current slice; in response to the first flag indicating the presence of the activated reference index number in the slice header, determining the number of entries in reference image list 0, and when the number of entries in reference image list 0 is greater than 1, signaling the activated reference index number of reference image list 0 in slice headers of P and B slices; and determining the number of entries in reference image list 1, and when the number of entries in reference image list 1 is greater than 1, signaling the activated reference index number of reference image list 1 in the slice header of the B slice.
[0022] In some embodiments, the method includes: receiving a video bitstream including a slice header and a picture header syntax; determining a value of a first flag signaled in the slice header, the first flag indicating whether there is an activated reference index number in the slice header, wherein the activated reference index number is used to derive a maximum reference index of a corresponding reference picture list for decoding a current slice; in response to the first flag indicating the presence of the activated reference index number, determining the number of entries in reference picture list 0, and when the number of entries in reference picture list 0 is greater than 1, decoding the activated reference index number of reference picture list 0 in the slice header of P and B slices; and determining the number of entries in reference picture list 1, and when the number of entries in reference picture list 1 is greater than 1, decoding the activated reference index number of reference picture list 1 in the slice header of the B slice.
[0023] In some embodiments, the method comprises: determining a collocated picture referenced by a reference index of the collocated picture at a slice level, wherein the collocated picture is determined to be the same picture for all non-I slices of the current picture; and processing the current picture based on the collocated picture, wherein the collocated picture is used for temporal motion vector prediction.
[0024] An embodiment of the present disclosure provides a computer-implemented device for processing video data, comprising: a first processing module, configured to derive a total number by adding the number of reference picture list structures in a sequence parameter set (SPS) to 1; a second processing module, configured to allocate memory for reference picture list structures equal to the total number in response to signaling the reference picture list structure in an image header of a current image or a stripe header of a current stripe; and to process the current image or the current stripe using the allocated memory.
[0025] The embodiment of the present disclosure provides a computer-implemented device for processing video data, comprising: a third processing module, configured to send a first flag in a slice header by signaling whether there is an activated reference index number in the slice header, wherein the activated reference index number is used to derive a maximum reference index of a corresponding reference image list for encoding a current slice; a fourth processing module, configured to determine the number of entries in reference image list 0 in response to the first flag indicating the presence of the activated reference index number in the slice header, and when the number of entries in the reference image list 0 is greater than 1, send the activated reference index number of reference image list 0 by signaling in the slice headers of P and B slices; and determine the number of entries in reference image list 1, and when the number of entries in the reference image list 1 is greater than 1, send the activated reference index number of reference image list 1 by signaling in the slice header of the B slice.
[0026] An embodiment of the present disclosure provides a storage medium product, which is used to store a bit stream generated by executing the above method. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Embodiments and aspects of the present disclosure are described in the following detailed description and accompanying drawings.The various features shown in the drawings are not drawn to scale.
[0028] Figure 1 is a schematic diagram of the structure of an exemplary video sequence according to some embodiments of the present disclosure.
[0029] Figure 2A is a schematic diagram of an exemplary encoding process of a hybrid video coding system consistent with an embodiment of the present disclosure.
[0030] Figure 2B is a schematic diagram of another exemplary encoding process of a hybrid video coding system consistent with an embodiment of the present disclosure.
[0031] Figure 3A is a schematic diagram of an exemplary decoding process of a hybrid video coding system consistent with an embodiment of the present disclosure.
[0032] Figure 3B is a schematic diagram of another exemplary decoding process of a hybrid video coding system consistent with an embodiment of the present disclosure.
[0033] Figure 4 is a block diagram of an exemplary apparatus for encoding or decoding a video according to some embodiments of the present disclosure.
[0034] Figure 5A An exemplary syntax including a syntax structure of a reference picture list according to some embodiments of the present disclosure is shown.
[0035] Figure 5B An exemplary pseudo code including the derivation of the variable FullPocLt[i][j] according to some embodiments of the present disclosure is shown.
[0036] Fig. 6A An exemplary syntax including a syntax structure of a reference picture list according to some embodiments of the present disclosure is shown.
[0037] Figure 6B An exemplary pseudo code including the derivation of the variable NumLtrpEntries[listIdx][rplsIdx] according to some embodiments of the present disclosure is shown.
[0038] Figure 6C An exemplary pseudo code including the derivation of the variable AbsDeltaPocSt[listIdx][rplsIdx][i] according to some embodiments of the present disclosure is shown.
[0039] Fig.6DAn exemplary pseudo code including the derivation of the variable DeltaPocValSt[listIdx][rplsIdx] according to some embodiments of the present disclosure is shown.
[0040] Figure 7 An exemplary syntax including a syntax structure of a reference picture list in a sequence parameter set according to some embodiments of the present disclosure is shown.
[0041] Figure 8 An exemplary syntax including a syntax structure of a reference picture list in a picture parameter set according to some embodiments of the present disclosure is shown.
[0042] Fig. 9A An exemplary syntax including a syntax structure of a reference picture list in a picture header structure according to some embodiments of the present disclosure is shown.
[0043] Fig. 9B Exemplary pseudo code including the derivation of the variable MaxNumSubblockMergeCand according to some embodiments of the present disclosure is shown.
[0044] Fig. 10A An exemplary syntax including a syntax structure of a reference picture list in a slice header according to some embodiments of the present disclosure is shown.
[0045] Fig. 10B An exemplary pseudo code including the derivation of the variable NumRefIdxActive[i] according to some embodiments of the present disclosure is shown.
[0046] Fig.11A A flowchart of an exemplary video encoding method for signaling a flag in a PH syntax structure according to some embodiments of the present disclosure is shown.
[0047] Fig. 11B A flowchart of an exemplary video decoding method for decoding a flag in a PH syntax structure according to some embodiments of the present disclosure is shown.
[0048] Fig. 11C An exemplary syntax of updated signaling including ph_collocated_from_l0_flag and ph_mvd_l1_zero_flag according to some embodiments of the present disclosure is shown.
[0049] Fig. 12A A flowchart of an exemplary video encoding method for indicating collocated pictures using picture order counts according to some embodiments of the present disclosure is shown.
[0050] Fig. 12B A flowchart of an exemplary video encoding method for indicating collocated pictures using picture order counts according to some embodiments of the present disclosure is shown.
[0051] Fig. 12C A flowchart of an exemplary video encoding method for indicating collocated images according to some embodiments of the present disclosure is shown.
[0052] Fig.12D A flowchart of an exemplary video decoding method for indicating collocated pictures using picture order counts according to some embodiments of the present disclosure is shown.
[0053] Fig.12E A flowchart of an exemplary video decoding method for indicating collocated pictures using picture order counts according to some embodiments of the present disclosure is shown.
[0054] Fig.12F An exemplary syntax including an updated reference picture list in a picture parameter set according to some embodiments of the present disclosure is shown.
[0055] Figure 12G An exemplary syntax including an updated slice header according to some embodiments of the present disclosure is shown.
[0056] Fig.12H Exemplary pseudo code including derivation of AbsDeltaPocStCol according to some embodiments of the present disclosure is shown.
[0057] Fig.12I Exemplary pseudo code including the derivation of DeltaPocValStCol according to some embodiments of the present disclosure is shown.
[0058] Fig.12J An exemplary pseudo code for deriving a collocated image used in a decoding method according to some embodiments of the present disclosure is shown.
[0059] Fig.13A A flowchart of an exemplary video encoding method for inferring the index of a collocated picture in a SH using the number of activated entries in a reference picture list according to some embodiments of the present disclosure is shown.
[0060] Fig. 13B A flowchart of an exemplary video decoding method for inferring the index of a collocated picture in a SH using the number of activated entries in a reference picture list according to some embodiments of the present disclosure is shown.
[0061] Fig. 13C Exemplary semantics of the syntax element sh_collocated_ref_idx for updating according to some embodiments of the present disclosure are shown.
[0062] Fig.14A A flowchart of an exemplary video processing method for allocating memory for a decoder according to some embodiments of the present disclosure is shown.
[0063] Fig. 14B Exemplary semantics for allocating memory according to some embodiments of the present disclosure are shown.
[0064] Fig.15A A flowchart of an exemplary video encoding method for inferring an index in a reference picture list according to some embodiments of the present disclosure is shown.
[0065] Fig. 15B A flowchart of an exemplary video encoding method for inferring an index in a reference picture list according to some embodiments of the present disclosure is shown.
[0066] Fig. 15C Exemplary semantics of the variable rpl_idx[i] for updating according to some embodiments of the present disclosure are shown.
[0067] Fig.16A A flowchart of an exemplary video encoding method for indicating whether an override number of an activation reference index exists in a slice header according to some embodiments of the present disclosure is shown.
[0068] Fig. 16B A flowchart of an exemplary video decoding method for indicating whether a coverage number of an activated reference index exists in a slice header according to some embodiments of the present disclosure is shown.
[0069] Fig. 16C Exemplary semantics of the syntax element sh_num_ref_idx_active_override_flag for updating according to some embodiments of the present disclosure are shown.
[0070] Fig.17A A flowchart of an exemplary video processing method for defining a collocated image index in SH for an I slice according to some embodiments of the present disclosure is shown.
[0071] Fig. 17B Exemplary semantics of bitstream conformance constraints for updates of the syntax element sh_collocated_ref_idx according to some embodiments of the present disclosure are shown. DETAILED DESCRIPTION
[0072] Reference will now be made in detail to exemplary embodiments, examples of which are shown in the accompanying drawings. The following description refers to the accompanying drawings, and unless otherwise specified, the same numbers in different drawings represent the same or similar elements. The embodiments set forth in the following description of exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with aspects related to the present invention as in the attached claims. Specific aspects of the present disclosure are described in more detail below. In the event of a conflict with terms and / or definitions incorporated by reference, the terms and definitions provided herein shall prevail.
[0073] The joint video experts team (JVET) of the ITU-T video coding expert group (ITU-T VCEG) and the ISO / IEC moving picture expert group (ISO / IEC MPEG) is currently developing the versatile video coding (VVC / H.266) standard. The VVC standard aims to double the compression efficiency of its predecessor, the high efficiency video coding (HEVC / H.265) standard. In other words, VVC aims to achieve the same subjective quality as HEVC / H.265 using half the bandwidth.
[0074] To achieve the same subjective quality as HEVC / H.265 using half the bandwidth, JVET has been developing technologies beyond HEVC using the Joint Exploration Model (JEM) reference software. As coding technologies are incorporated into JEM, JEM achieves higher coding performance than HEVC.
[0075] The VVC standard has been recently developed and continues to include more coding techniques that provide better compression performance. VVC is based on the same hybrid video coding system used in modern video compression standards such as HEVC, H.264 / AVC, MPEG2, H.263, etc.
[0076] A video is a set of static images (or "frames") arranged in time sequence to store visual information. A video capture device (e.g., a camera) can be used to capture and store these images in time sequence, and a video playback device (e.g., a television, computer, smartphone, tablet, video player, or any end-user terminal with display capability) can be used to display such images in time sequence. In addition, in some applications, the video capture device can transmit the captured video to a video playback device (e.g., a computer with a monitor) in real time, such as for monitoring, conferencing, or live broadcasting.
[0077] In order to reduce the storage space and transmission bandwidth required for such applications, the video can be compressed before storage and transmission, and decompressed before display. Compression and decompression can be implemented by software executed by a processor (e.g., a processor of a general-purpose computer) or dedicated hardware. The module for compression is usually called an "encoder", and the module for decompression is usually called a "decoder". Encoders and decoders can be collectively referred to as "codecs". Encoders and decoders can be implemented as any of various suitable hardware, software, or combinations thereof. For example, the hardware implementation of encoders and decoders can include circuits, such as one or more microprocessors, digital signal processors (digital signal processors, DSPs), application-specific integrated circuits (application-specific integrated circuits, ASICs), field-programmable gate arrays (field-programmable gate arrays, FPGAs), discrete logic, or any combination thereof. The software implementation of encoders and decoders can include program code, computer executable instructions, firmware, or any suitable computer-implemented algorithm or process fixed in a computer-readable medium. Video compression and decompression can be implemented by various algorithms or standards, such as MPEG-1, MPEG-2, MPEG-4, H.26x series, etc. In some applications, a codec may decompress video from a first coding standard and recompress the decompressed video using a second coding standard, in which case the codec may be referred to as a "transcoder."
[0078] The video encoding process can identify and retain useful information that can be used to reconstruct the image, and ignore information that is not important for reconstruction. If the ignored, unimportant information cannot be fully reconstructed, the encoding process can be called "lossy". Otherwise, it can be called "lossless". Most encoding processes are lossy, which is a trade-off made to reduce the required storage space and transmission bandwidth.
[0079] Useful information about the image being encoded (referred to as the "current image") includes changes relative to a reference image (e.g., a previously encoded and reconstructed image). Such changes can include changes in pixel position, brightness, or color, with changes in position being of greatest interest. Changes in the position of a group of pixels representing an object can reflect the object's motion between the reference image and the current image.
[0080] A picture that is encoded without reference to another picture (i.e., it is its own reference picture) is called an "I-picture." If some or all blocks in a picture (e.g., blocks that are typically part of a video picture) are predicted using intra-prediction or inter-prediction (e.g., uni-prediction) using one reference picture, then the picture is called a "P-picture." If at least one block in a picture is predicted using two reference pictures (e.g., bi-prediction), then the picture is called a "B-picture."
[0081] Figure 1 The structure of an exemplary video sequence 100 according to some embodiments of the present disclosure is shown. The video sequence 100 may be a live video or a captured and archived video. The video sequence 100 may be a real video, a computer-generated video (e.g., a computer game video), or a combination thereof (e.g., a real video with an augmented reality effect). The video sequence 100 may be input from a video capture device (e.g., a camera), a video archive containing previously captured videos (e.g., a video file stored in a storage device), or a video feed interface (e.g., a video broadcast transceiver) that receives video from a video content provider.
[0082] like Figure 1 As shown, video sequence 100 may include a series of images arranged in time along a time axis, including images 102, 104, 106, and 108. Images 102-106 are consecutive, and there are more images between images 106 and 108. Figure 1 , picture 102 is an I-picture, and its reference picture is picture 102 itself. Picture 104 is a P-picture, and its reference picture is picture 102, as indicated by the arrows. Picture 106 is a B-picture, and its reference pictures are pictures 104 and 108, as indicated by the arrows. In some embodiments, the reference picture of a picture (e.g., picture 104) may not be immediately before or after the picture. For example, the reference picture of picture 104 may be the picture before picture 102. It should be noted that the reference pictures of pictures 102-106 are only examples, and the present disclosure does not limit the embodiments of the reference pictures to Figure 1 The example shown in .
[0083] Typically, due to the computational complexity of the encoding and decoding tasks, video codecs do not encode or decode the entire image at once. Instead, they can divide the image into basic segments and encode or decode the image segment by segment. Such basic segments are referred to as basic processing units (BPUs) in this disclosure. For example, Figure 1Structure 110 in shows an example structure of an image (e.g., any one of images 102-108) of video sequence 100. In structure 110, the image is divided into 4×4 basic processing units, whose boundaries are shown as dashed lines. In some embodiments, the basic processing unit may be referred to as a "macroblock" in some video coding standards (e.g., the MPEG series, H.261, H.263, or H.264 / AVC), or as a "coding tree unit" (CTU) in some other video coding standards (e.g., H.265 / HEVC or H.266 / VVC). The basic processing unit in an image may have a variable size, such as 128×128, 64×64, 32×32, 16×16, 4×8, 16×32, or any arbitrary shape and size of pixels. The size and shape of the basic processing unit may be selected for an image based on a balance between coding efficiency and the level of detail to be maintained in the basic processing unit.
[0084] A basic processing unit may be a logical unit that may include a set of different types of video data stored in a computer memory (e.g., in a video frame buffer). For example, a basic processing unit for a color image may include a luma component (Y) representing achromatic luminance information, one or more chroma components (e.g., Cb and Cr) representing color information, and associated syntax elements, where the luma component and the chroma components may have the same size of basic processing unit size. In some video coding standards (e.g., H.265 / HEVC or H.266 / VVC), luma and chroma components may be referred to as "coding tree blocks" ("CTBs"). Any operation performed on a basic processing unit may be repeated for each of its luma and chroma components.
[0085] Video encoding has several stages of operation, examples of which are given in Figure 2A-2B and Figure 3A-3BAs shown in . For each stage, the size of the basic processing unit may still be too large to be processed, so it can be further divided into segments referred to as "basic processing subunits" in this disclosure. In some embodiments, the basic processing subunit may be called a "block" in some video coding standards (e.g., MPEG series, H.261, H.263, or H.264 / AVC), or a "coding unit" ("CU") in some other video coding standards (e.g., H.265 / HEVC or H.266 / VVC). The size of the basic processing subunit may be the same as or smaller than the basic processing unit. Similar to the basic processing unit, the basic processing subunit is also a logical unit, which may include a set of different types of video data (e.g., Y, Cb, Cr, and related syntax elements) stored in a computer memory (e.g., in a video frame buffer). Any operation performed on the basic processing subunit can be repeated for each of its luminance and chrominance components. It should be noted that this division can be further performed according to processing needs. It should also be noted that different stages can divide the basic processing units using different schemes.
[0086] For example, in the mode decision phase (an example of which is in Figure 2B ), the encoder can decide what prediction mode to use for the basic processing unit (e.g., intra-image prediction or inter-image prediction), but the basic processing unit may be too large to make such a decision. The encoder can split the basic processing unit into multiple basic processing sub-units (e.g., CUs in H.265 / HEVC or H.266 / VVC) and determine the prediction type for each individual basic processing sub-unit.
[0087] For another example, in the prediction phase (the example in Figure 2A and Figure 2B As shown in Figure 2, the encoder can perform prediction operations at the basic processing sub-unit (e.g., CU) level. However, in some cases, the basic processing sub-unit may still be too large to process. The encoder can further split the basic processing sub-unit into smaller segments (e.g., called "prediction blocks" or "PBs" in H.265 / HEVC or H.266 / VVC), at which prediction operations can be performed.
[0088] For another example, in the transformation phase (the example in Figure 2A and Figure 2B), the encoder can perform transform operations on the residual basic processing sub-unit (e.g., CU). However, in some cases, the basic processing sub-unit may still be too large to process. The encoder can further divide the basic processing sub-unit into smaller segments (e.g., called "transform blocks" or "TBs (tranform blocks)" in H.265 / HEVC or H.266 / VVC), at the level of which transform operations can be performed. It should be noted that the division scheme of the same basic processing sub-unit may be different in the prediction stage and the transform stage. For example, in H.265 / HEVC or H.266 / VVC, the prediction blocks and transform blocks of the same CU may have different sizes and numbers.
[0089] exist Figure 1 In the structure 110, the basic processing unit 112 is further divided into 3×3 basic processing sub-units, whose boundaries are shown by dotted lines. Different basic processing units of the same image can be divided into basic processing sub-units of different schemes.
[0090] In some implementations, in order to provide parallel processing capabilities and fault tolerance for video encoding and decoding, the image can be divided into multiple regions for processing, so that for one region of the image, the encoding or decoding process can be independent of information from any other region of the image. In other words, each region of the image can be processed independently. By doing so, the codec can process different regions of the image in parallel, thereby improving coding efficiency. In addition, when the data of one region is damaged during processing or lost during network transmission, the codec can correctly encode or decode other regions of the same image without relying on the damaged or lost data, thereby providing fault tolerance. In some video coding standards, the image can be divided into different types of regions. For example, H.265 / HEVC and H.266 / VVC provide two types of regions: "slices" and "tiles". It should also be noted that different images of the video sequence 100 may have different partitioning schemes for dividing the image into multiple regions.
[0091] For example, in Figure 1 , structure 110 is divided into three regions 114, 116, and 118, whose boundaries are shown as solid lines within structure 110. Region 114 includes four basic processing units. Each of regions 116 and 118 includes six basic processing units. It should be noted that Figure 1 The basic processing units, basic processing sub-units, and regions of the structure 110 are merely examples, and the present disclosure does not limit the embodiments thereof.
[0092] Figure 2ASchematic diagram of an example encoding process 200A consistent with an embodiment of the present disclosure is shown. For example, the encoding process 200A may be performed by an encoder. Figure 2A As shown, the encoder may encode the video sequence 202 into a video bitstream 228 according to process 200A. Figure 1 Similar to the video sequence 100 in FIG. 2 , the video sequence 202 may include a set of images (referred to as “original images”) arranged in time order. Figure 1 In the structure 110 in the video sequence 202, each original image of the video sequence 202 can be divided into basic processing units, basic processing sub-units or regions for processing by the encoder. In some embodiments, the encoder can perform process 200A on each original image of the video sequence 202 at the basic processing unit level. For example, the encoder can perform process 200A in an iterative manner, wherein the encoder can encode the basic processing unit in one iteration of process 200A. In some embodiments, the encoder can perform process 200A in parallel on regions (e.g., regions 114-118) of each original image of the video sequence 202.
[0093] exist Figure 2A 2, an encoder may feed a basic processing unit (referred to as an "original BPU") of an original image of a video sequence 202 to a prediction stage 204 to generate prediction data 206 and a predicted BPU 208. The encoder may subtract the predicted BPU 208 from the original BPU to generate a residual BPU 210. The encoder may feed the residual BPU 210 to a transform stage 212 and a quantization stage 214 to generate quantized transform coefficients 216. The encoder may feed the prediction data 206 and the quantized transform coefficients 216 to a binary encoding stage 226 to generate a video bitstream 228. The components 202, 204, 206, 208, 210, 212, 214, 216, 226, and 228 may be referred to as a "forward path." During process 200A, after the quantization stage 214, the encoder may feed the quantized transform coefficients 216 to an inverse quantization stage 218 and an inverse transform stage 220 to generate a reconstructed residual BPU 222. The encoder may add the reconstructed residual BPU 222 to the predicted BPU 208 to generate a prediction reference 224, which is used in the prediction phase 204 of the next iteration of the process 200A. The components 218, 220, 222, and 224 of the process 200A may be referred to as a "reconstruction path." The reconstruction path may be used to ensure that the encoder and decoder use the same reference data for prediction.
[0094] The encoder can iteratively perform process 200A to encode each original BPU of the original image (in the forward path) and generate a prediction reference 224 for encoding the next original BPU of the original image (in the reconstruction path). After encoding all the original BPUs of the original image, the encoder can continue to encode the next image in the video sequence 202.
[0095] Referring to process 200A, an encoder may receive a video sequence 202 generated by a video capture device (eg, a camera). As used herein, the term "receive" may refer to any action of receiving, inputting, acquiring, retrieving, obtaining, reading, accessing, or in any manner inputting data.
[0096] In the prediction phase 204, in the current iteration, the encoder may receive the original BPU and the prediction reference 224 and perform a prediction operation to generate the prediction data 206 and the predicted BPU 208. The prediction reference 224 may be generated from the reconstruction path of the previous iteration of the process 200A. The purpose of the prediction phase 204 is to reduce information redundancy by extracting the prediction data 206, which may be used to reconstruct the original BPU into the predicted BPU 208 from the prediction data 206 and the prediction reference 224.
[0097] Ideally, the predicted BPU 208 can be the same as the original BPU. However, due to non-ideal prediction and reconstruction operations, the predicted BPU 208 is usually slightly different from the original BPU. In order to record this difference, after generating the predicted BPU 208, the encoder can subtract it from the original BPU to generate a residual BPU 210. For example, the encoder can subtract the value (e.g., grayscale value or RGB value) of the pixel of the predicted BPU 208 from the value of the corresponding pixel of the original BPU. Each pixel of the residual BPU 210 can have a residual value, which is the result of this subtraction between the corresponding pixels of the original BPU and the predicted BPU 208. Compared with the original BPU, the prediction data 206 and the residual BPU 210 can have fewer bits, but they can be used to reconstruct the original BPU without significant quality degradation. Therefore, the original BPU is compressed.
[0098] To further compress the residual BPU 210, in the transform stage 212, the encoder can reduce the spatial redundancy of the residual BPU 210 by decomposing it into a set of two-dimensional "basic patterns", each of which is associated with a "transform coefficient". The basic patterns can have the same size (e.g., the size of the residual BPU 210). Each basic pattern can represent a frequency-varying (e.g., frequency-varying) component of the residual BPU 210. No basic pattern can be reproduced from any combination (e.g., linear combination) of any other basic pattern. In other words, the decomposition can decompose the variation of the residual BPU 210 into the frequency domain. This decomposition is similar to the discrete Fourier transform of a function, where the basic patterns are similar to the basic functions of the discrete Fourier transform (e.g., trigonometric functions), and the transform coefficients are similar to the coefficients associated with the basic functions.
[0099] Different transform algorithms can use different basic modes. Various transform algorithms can be used in the transform stage 212, for example, discrete cosine transform, discrete sine transform, etc. The transform in the transform stage 212 is reversible. That is, the encoder can restore the residual BPU 210 by the inverse operation of the transform (called "inverse transform"). For example, in order to restore the pixels of the residual BPU 210, the inverse transform can be to multiply the values of the corresponding pixels of the basic mode by the corresponding correlation coefficients, and add the products to produce a weighted sum. For video coding standards, both the encoder and the decoder can use the same transform algorithm (and therefore the same basic mode). Therefore, the encoder can only record the transform coefficients, and the decoder can reconstruct the residual BPU 210 from these coefficients without receiving the basic mode from the encoder. Compared with the residual BPU 210, the transform coefficients can have fewer bits, but they can be used to reconstruct the residual BPU 210 without significantly reducing the quality. Therefore, the residual BPU 210 is further compressed.
[0100] The encoder may further compress the transform coefficients in the quantization stage 214. During the transform process, different basic modes may represent different frequencies of change (e.g., frequency of brightness changes). Since the human eye is generally better at recognizing low-frequency changes, the encoder may ignore information about high-frequency changes without causing a significant decrease in decoding quality. For example, in the quantization stage 214, the encoder may generate quantized transform coefficients 216 by dividing each transform coefficient by an integer value (referred to as a "quantization scale factor") and rounding the quotient to its nearest integer. After such an operation, some transform coefficients of high-frequency basic modes may be converted to zero, and transform coefficients of low-frequency basic modes may be converted to smaller integers. The encoder may ignore quantized transform coefficients 216 of zero value, further compressing the transform coefficients by this operation. The quantization process is also reversible, where the quantized transform coefficients 216 may be reconstructed into transform coefficients in an inverse operation of quantization (referred to as "inverse quantization").
[0101] Because the encoder ignores the remainder of this division in the rounding operation, the quantization stage 214 may be lossy. Generally, the quantization stage 214 may cause the greatest information loss in the process 200A. The greater the information loss, the fewer bits may be required for the quantized transform coefficients 216. To achieve different degrees of information loss, the encoder may use different values of the quantization syntax element or any other syntax element of the quantization process.
[0102] In the binary encoding stage 226, the encoder may encode the prediction data 206 and the quantized transform coefficients 216 using a binary encoding technique (e.g., entropy coding, variable length coding, arithmetic coding, Huffman coding, context-adaptive binary arithmetic coding, or any other lossless or lossy compression algorithm). In some embodiments, in addition to the prediction data 206 and the quantized transform coefficients 216, the encoder may encode other information in the binary encoding stage 226, such as the prediction mode used in the prediction stage 204, the syntax elements of the prediction operation, the transform type in the transform stage 212, the syntax elements of the quantization process (e.g., quantization syntax elements), the encoder control syntax elements (e.g., bitrate control syntax elements), etc. The encoder may use the output data of the binary encoding stage 226 to generate a video bitstream 228. In some embodiments, the video bitstream 228 may be further packaged for network transmission.
[0103] Referring to the reconstruction path of process 200A, at the inverse quantization stage 218, the encoder may perform inverse quantization on the quantized transform coefficients 216 to generate reconstructed transform coefficients. At the inverse transform stage 220, the encoder may generate a reconstructed residual BPU 222 based on the reconstructed transform coefficients. The encoder may add the reconstructed residual BPU 222 to the predicted BPU 208 to generate a prediction reference 224 to be used in the next iteration of process 200A.
[0104] It should be noted that other variations of process 200A may be used to encode video sequence 202. In some embodiments, the various stages of process 200A may be performed by the encoder in a different order. In some embodiments, one or more stages of process 200A may be combined into a single stage. In some embodiments, a single stage of process 200A may be divided into multiple stages. For example, transform stage 212 and quantization stage 214 may be combined into a single stage. In some embodiments, process 200A may include additional stages. In some embodiments, process 200A may be omitted. Figure 2A one or more stages in a process.
[0105] Figure 2BA schematic diagram of another example encoding process 200B consistent with an embodiment of the present disclosure is shown. Process 200B can be modified from process 200A. For example, process 200B can be used by an encoder that conforms to a hybrid video coding standard (e.g., the H.26x series). Compared to process 200A, the forward path of process 200B additionally includes a mode decision stage 230 and divides the prediction stage 204 into a spatial prediction stage 2042 and a temporal prediction stage 2044. The reconstruction path of process 200B additionally includes a loop filter stage 232 and a buffer 234.
[0106] In general, prediction techniques can be divided into two categories: spatial prediction and temporal prediction. Spatial prediction (e.g., intra-image prediction or "intra-frame prediction") can use pixels from one or more encoded adjacent BPUs in the same image to predict the current BPU. That is, the prediction reference 224 in spatial prediction can include adjacent BPUs. Spatial prediction can reduce the spatial redundancy inherent in the image. Temporal prediction (e.g., inter-image prediction or "inter-frame prediction") can use regions from one or more encoded images to predict the current BPU. That is, the prediction reference 224 in temporal prediction can include encoded images. Temporal prediction can reduce the inherent temporal redundancy of the image.
[0107] Referring to process 200B, in the forward path, the encoder performs prediction operations in the spatial prediction stage 2042 and the temporal prediction stage 2044. For example, in the spatial prediction stage 2042, the encoder may perform intra-frame prediction. For the original BPU of the image being encoded, the prediction reference 224 may include one or more neighboring BPUs that have been encoded (in the forward path) and reconstructed (in the reconstruction path) in the same image. The encoder may generate the predicted BPU 208 by extrapolating the neighboring BPUs. Extrapolation techniques may include, for example, linear extrapolation or interpolation, polynomial extrapolation or interpolation, etc. In some embodiments, the encoder may perform extrapolation at the pixel level, such as by extrapolating the value of the corresponding pixel for each pixel of the predicted BPU 208. The neighboring BPUs used for extrapolation may be located in various directions relative to the original BPU, such as in the vertical direction (e.g., at the top of the original BPU), in the horizontal direction (e.g., on the left side of the original BPU), in the diagonal direction (e.g., at the lower left, lower right, upper left, or upper right of the original BPU), or in any direction defined in the video coding standard used. For intra prediction, the prediction data 206 may include, for example, the location (eg, coordinates) of the used neighboring BPUs, the size of the used neighboring BPUs, extrapolation parameters, the direction of the used neighboring BPUs relative to the original BPU, and the like.
[0108] For another example, in the temporal prediction stage 2044, the encoder may perform inter-frame prediction. For the original BPU of the current image, the prediction reference 224 may include one or more images (referred to as "reference images") that have been encoded (in the forward path) and reconstructed (in the reconstruction path). In some embodiments, the reference images may be encoded and reconstructed BPU by BPU. For example, the encoder may add the reconstructed residual BPU 222 to the predicted BPU 208 to generate a reconstructed BPU. When all reconstructed BPUs of the same image are generated, the encoder may generate the reconstructed image as a reference image. The encoder may perform a "motion estimation" operation to search for a matching area in a range of the reference image (referred to as a "search window"). The position of the search window in the reference image may be determined based on the position of the original BPU in the current image. For example, the search window may be centered at a position in the reference image having the same coordinates as the original BPU in the current image, and may extend outward by a predetermined distance. When the encoder identifies an area in the search window that is similar to the original BPU (e.g., by using a pixel recursive algorithm, a block matching algorithm, etc.), the encoder may determine such an area as a matching area. The matching region may have a different size (e.g., smaller, equal, larger, or different in shape) than the original BPU. Since the reference image and the current image are temporally separated on the time axis (e.g., Figure 1 ), so the matching area can be considered to "move" to the location of the original BPU over time. The encoder can record the direction and distance of this movement as a "motion vector". Figure 1 When the encoder searches for matching regions for each reference image and determines its associated motion vector, the encoder may assign weights to the pixel values of the matching regions of the respective matching reference images.
[0109] Motion estimation may be used to identify various types of motion, such as translation, rotation, scaling, etc. For inter-frame prediction, the prediction data 206 may include, for example, the location (e.g., coordinates) of the matching region, the motion vector associated with the matching region, the number of reference images, the weights associated with the reference images, etc.
[0110] To generate the predicted BPU 208, the encoder may perform a "motion compensation" operation. Motion compensation may be used to reconstruct the predicted BPU 208 based on the prediction data 206 (e.g., motion vectors) and the prediction reference 224. For example, the encoder may move a matching area of a reference image according to the motion vector, where the encoder may predict the original BPU of the current image. When multiple reference images are used (e.g., Figure 1The encoder may move the matching region of the reference image according to the respective motion vectors and the average pixel value of the matching region. In some embodiments, if the encoder has assigned weights to the pixel values of the matching regions of the respective matching reference images, the encoder may add the weighted sum of the pixel values of the moved matching regions.
[0111] In some embodiments, inter-frame prediction can be unidirectional or bidirectional. Unidirectional inter-frame prediction can use one or more reference images in the same temporal direction relative to the current image. For example, Figure 1 The picture 104 in is a unidirectional inter-frame prediction picture, where the reference picture (e.g., picture 102) precedes the picture 104. Bidirectional inter-frame prediction can use one or more reference pictures in two temporal directions relative to the current picture. For example, Figure 1 Picture 106 in is a bi-directional inter-predicted picture, where reference pictures (eg, pictures 104 and 108 ) are located in both temporal directions relative to picture 104 .
[0112] Still referring to the forward path of process 200B, after spatial prediction 2042 and temporal prediction stage 2044, at mode decision stage 230, the encoder can select a prediction mode (e.g., one of intra prediction or inter prediction) for the current iteration of process 200B. For example, the encoder can perform a rate-distortion optimization technique, in which the encoder can select a prediction mode based on the bit rate of a candidate prediction mode and the distortion of a reconstructed reference image under the candidate prediction mode to minimize the value of a cost function. Based on the selected prediction mode, the encoder can generate a corresponding predicted BPU 208 and predicted data 206.
[0113] In the reconstruction path of process 200B, if the intra prediction mode is selected in the forward path, after generating the prediction reference 224 (e.g., the current BPU that has been encoded and reconstructed in the current image), the encoder can directly feed the prediction reference 224 to the spatial prediction stage 2042 for later use (e.g., for extrapolating the next BPU of the current image). The encoder can feed the prediction reference 224 to the loop filter stage 232, where the encoder can apply a loop filter to the prediction reference 224 to reduce or eliminate distortion (e.g., blocking artifacts) introduced during the encoding of the prediction reference 224. The encoder can apply various loop filter techniques in the loop filter stage 232, such as deblocking, sample adaptive offset, adaptive loop filter, etc. The reference pictures for loop filtering may be stored in a buffer 234 (or "decoded picture buffer (DPB)") for later use (e.g., as inter-frame prediction reference pictures for future pictures of the video sequence 202). The encoder may store one or more reference pictures in the buffer 234 for use in the temporal prediction stage 2044. In some embodiments, the encoder may encode syntax elements of the loop filter (e.g., loop filter strength) as well as the quantized transform coefficients 216, the prediction data 206, and other information in the binary encoding stage 226.
[0114] Figure 3A A schematic diagram of an example decoding process 300A consistent with an embodiment of the present disclosure is shown. Process 300A may correspond to Figure 2A 300A. In some embodiments, process 300A may be similar to the reconstruction path of process 200A. The decoder may decode video bitstream 228 into video stream 304 according to process 300A. Video stream 304 may be very similar to video sequence 202. However, due to information loss during compression and decompression (e.g., Figure 2A and Figure 2B 214), typically, the video stream 304 is different from the video sequence 202. Figure 2A and Figure 2B 200A and 200B in the decoder, the decoder may perform process 300A at a basic processing unit (BPU) level for each picture encoded in the video bitstream 228. For example, the decoder may perform process 300A in an iterative manner, where the decoder may decode a basic processing unit in one iteration of process 300A. In some embodiments, the decoder may perform process 300A in parallel for regions (e.g., regions 114-118) of each picture encoded in the video bitstream 228.
[0115] exist Figure 3AIn the process 300A, the decoder may feed a portion of the video bitstream 228 associated with the basic processing unit of the encoded picture (referred to as the "encoded BPU") to the binary decoding stage 302. In the binary decoding stage 302, the decoder may decode the portion into prediction data 206 and quantized transform coefficients 216. The decoder may feed the quantized transform coefficients 216 to the inverse quantization stage 218 and the inverse transform stage 220 to generate a reconstructed residual BPU 222. The decoder may feed the prediction data 206 to the prediction stage 204 to generate a predicted BPU 208. The decoder may add the reconstructed residual BPU 222 to the predicted BPU 208 to generate a prediction reference 224. In some embodiments, the prediction reference 224 may be stored in a buffer (e.g., a decoded picture buffer in a computer memory). The decoder may feed the predicted reference 224 to the prediction stage 204 to perform a prediction operation in the next iteration of the process 300A.
[0116] The decoder may iteratively perform process 300A to decode each coded BPU of the coded picture and generate a prediction reference 224 for encoding the next coded BPU of the coded picture. After decoding all coded BPUs of the coded picture, the decoder may output the picture to a video stream 304 for display and continue decoding the next coded picture in the video bitstream 228.
[0117] In the binary decoding stage 302, the decoder may perform the inverse operation of the binary coding technique used by the encoder (e.g., entropy coding, variable length coding, arithmetic coding, Huffman coding, context-adaptive binary arithmetic coding, or any other lossless compression algorithm). In some embodiments, in addition to the prediction data 206 and the quantized transform coefficients 216, the decoder may decode other information in the binary decoding stage 302, such as the prediction mode, the syntax elements of the prediction operation, the transform type, the syntax elements of the quantization process (e.g., the quantization syntax elements), the encoder control syntax elements (e.g., the bit rate control syntax elements), etc. In some embodiments, if the video bitstream 228 is transmitted in the form of packets over the network, the decoder may depacketize the video bitstream 228 before feeding it to the binary decoding stage 302.
[0118] Figure 3B A schematic diagram of another example decoding process 300B consistent with an embodiment of the present disclosure is shown. Process 300B can be modified from process 300A. For example, process 300B can be used by a decoder that conforms to a hybrid video coding standard (e.g., H.26x series). Compared to process 300A, process 300B additionally divides the prediction stage 204 into a spatial prediction stage 2042 and a temporal prediction stage 2044, and additionally includes a loop filter stage 232 and a buffer 234.
[0119] In process 300B, for an encoded basic processing unit (referred to as a "current BPU") of an encoded image being decoded (referred to as a "current image"), the prediction data 206 decoded by the decoder from the binary decoding stage 302 may include various types of data, depending on what prediction mode the encoder used to encode the current BPU. For example, if the encoder encodes the current BPU using intra-frame prediction, the prediction data 206 may include a prediction mode indicator (e.g., a flag value) indicating intra-frame prediction, syntax elements of intra-frame prediction operations, etc. The syntax elements of the intra-frame prediction operations may include, for example, the location (e.g., coordinates) of one or more neighboring BPUs used as references, the size of the neighboring BPUs, an extrapolation syntax element, the direction of the neighboring BPU relative to the original BPU, etc. For another example, if the encoder encodes the current BPU using inter-frame prediction, the prediction data 206 may include a prediction mode indicator (e.g., a flag value) indicating inter-frame prediction, syntax elements of inter-frame prediction operations, etc. The syntactic elements of the inter-frame prediction operation may include, for example, the number of reference images associated with the current BPU, the weights respectively associated with the reference images, the positions (e.g., coordinates) of one or more matching regions in each reference image, one or more motion vectors respectively associated with the matching regions, and the like.
[0120] Based on the prediction mode indicator, the decoder can decide whether to perform spatial prediction (eg, intra prediction) in the spatial prediction stage 2042 or to perform temporal prediction (eg, inter prediction) in the temporal prediction stage 2044 . Figure 2B The details of performing such spatial prediction or temporal prediction are described in , and will not be repeated below. After performing such spatial prediction or temporal prediction, the decoder can generate a predicted BPU 208. Figure 3A , the decoder may add the predicted BPU 208 and the reconstructed residual BPU 222 to generate a prediction reference 224 .
[0121] In process 300B, the decoder may feed the prediction reference 224 to the spatial prediction stage 2042 or the temporal prediction stage 2044 to perform a prediction operation in the next iteration of process 300B. For example, if the current BPU is decoded using intra-frame prediction in the spatial prediction stage 2042, then after generating the prediction reference 224 (e.g., the decoded current BPU), the decoder may directly feed the prediction reference 224 to the spatial prediction stage 2042 for later use (e.g., for extrapolating the next BPU of the current image). If the current BPU is decoded using inter-frame prediction in the temporal prediction stage 2044, then after generating the prediction reference 224 (e.g., a reference image in which all BPUs have been decoded), the decoder may feed the prediction reference 224 to the loop filter stage 232 to reduce or eliminate distortion (e.g., blocking artifacts). The decoder may Figure 2B The loop filter is applied to the prediction reference 224 in a manner. The loop filtered reference picture can be stored in a buffer 234 (e.g., a decoded picture buffer (DPB) in a computer memory) for later use (e.g., as an inter-frame prediction reference picture for a future encoded picture of the video bitstream 228). The decoder can store one or more reference pictures in the buffer 234 for use in the temporal prediction stage 2044. In some embodiments, the prediction data may also include syntax elements of the loop filter (e.g., loop filter strength). In some embodiments, when the prediction mode indicator of the prediction data 206 indicates that inter-frame prediction was used to encode the current BPU, the prediction data includes syntax elements of the loop filter.
[0122] Figure 4 4 is a block diagram of an example apparatus 400 for encoding or decoding video consistent with an embodiment of the present disclosure. Figure 4As shown, the device 400 may include a processor 402. When the processor 402 executes the instructions herein, the device 400 may become a dedicated machine for video encoding or decoding. The processor 402 may be any type of circuit capable of manipulating or processing information. For example, the processor 402 may include any number of central processing units (or central processing units, "CPUs"), graphics processing units (or graphics processing units, "GPUs"), neural processing units (neural processing units, "NPUs"), microcontroller units (mibrocontroller units, "MCUs"), optical processors, programmable logic controllers, microcontrollers, microprocessors, digital signal processors, intellectual property (IP) cores, programmable logic arrays (PLA), programmable array logic (PAL), generic array logic (GAL), complex programmable logic devices (CPLD), field-programmable gate arrays (FPGA), systems on chip (SoC), application-specific integrated circuits (ASIC), etc. Any combination. In some embodiments, processor 402 may also be a group of processors grouped into a single logical component. Figure 4 As shown, processor 402 may include multiple processors, including processor 402a, processor 402b, and processor 402n.
[0123] The apparatus 400 may also include a memory 404 configured to store data (eg, a set of instructions, computer code, intermediate data, etc.). Figure 4As shown, the stored data may include program instructions (e.g., program instructions for implementing stages in process 200A, 200B, 300A, or 300B) and data for processing (e.g., video sequence 202, video bitstream 228, or video stream 304). Processor 402 may access program instructions and data for processing (e.g., via bus 410) and execute program instructions to perform operations or controls on data for processing. Memory 404 may include a high-speed random access memory device or a non-volatile memory device. In some embodiments, memory 404 may include any number of random-access memories (RAM), read-only memories (ROM), optical disks, magnetic disks, hard disk drives, solid-state drives, flash drives, security digital (SD) cards, memory sticks, compact flash (CF) cards, etc. Any combination. Memory 404 may also be a group of memories ( Figure 4 not shown).
[0124] The bus 410 may be a communication device that transmits data between components within the apparatus 400 , such as an internal bus (eg, a CPU-memory bus), an external bus (eg, a universal serial bus port, a peripheral component interconnect express port), and the like.
[0125] For ease of explanation and without ambiguity, the processor 402 and other data processing circuits are collectively referred to as "data processing circuits" in this disclosure. The data processing circuits may be fully implemented as hardware, or a combination of software, hardware, or firmware. In addition, the data processing circuit may be a single independent module, or may be fully or partially combined into any other component of the device 400.
[0126] The device 400 may also include a network interface 406 to provide wired or wireless communication with a network (e.g., the Internet, an intranet, a local area network, a mobile communication network, etc.). In some embodiments, the network interface 406 may include any number of network interface controllers (NICs), radio frequency (RF) modules, repeaters, transceivers, modems, routers, gateways, wired network adapters, wireless network adapters, Bluetooth adapters, infrared adapters, near-field communication (NFC) adapters, cellular network chips, etc.
[0127] In some embodiments, the apparatus 400 may optionally further include a peripheral interface 408 to provide a connection to one or more peripheral devices. Figure 4 As shown, peripheral devices may include, but are not limited to, a cursor control device (e.g., a mouse, touchpad, or touch screen), a keyboard, a display (e.g., a cathode ray tube display, a liquid crystal display, or a light emitting diode display), a video input device (e.g., a camera or an input interface coupled to a video archive), etc.
[0128] It should be noted that the video codec (e.g., a codec that performs the process 200A, 200B, 300A, or 300B) can be implemented as any combination of any software or hardware modules in the device 400. For example, some or all stages of the process 200A, 200B, 300A, or 300B can be implemented as one or more software modules of the device 400, such as program instructions that can be loaded into the memory 404. For another example, some or all stages of the process 200A, 200B, 300A, or 300B can be implemented as one or more hardware modules of the device 400, such as dedicated data processing circuits (e.g., FPGA, ASIC, NPU, etc.).
[0129] In video coding, pictures need to be identified for various purposes, including being identified as reference pictures in inter prediction, pictures to be output from the DPB, temporally collocated pictures for motion vector prediction, etc. The most common method of identifying pictures is to use picture order count (POC).
[0130] In order to identify reference pictures in inter prediction, motion vector ("MV") temporal prediction, and temporally collocated pictures in scaling, reference picture lists (usually two, as in AVC, HEVC, and VVC) can be derived. For example, reference picture list 0 and reference picture list 1 can be derived, each of which includes a list of reconstructed pictures in the DPB to be used as reference pictures. And the reference indexes of these reference picture lists can be signaled at the block level to identify the reference picture of the current block. Reference picture marking is required to correctly maintain the reference pictures in the DPB without requiring unnecessarily large amounts of DPB memory.
[0131] In VVC (e.g., VVC draft 9), two reference picture lists ("RPL") are used, namely reference picture list 0 and reference picture list 1. These two lists are directly signaled and derived. The information of the two reference picture lists is signaled by syntax elements and syntax structures in the sequence parameter set ("SPS"), picture parameter set ("PPS"), picture header ("PH") and slice header ("SH"). A predefined reference picture list structure is signaled in the SPS for reference in the PH or SH. A new reference picture list structure may also be signaled in the PH or SH for deriving reference picture list 0 and reference picture list 1. Whether the reference picture list information is signaled in the PH or SH is determined by a flag signaled in the PPS.
[0132] In VVC (e.g., VVC draft 9), two reference picture lists are generated for all types of slices (e.g., B, P, and I slices). For I slices, neither reference picture list 0 nor reference picture list 1 can be used for decoding. For P slices, only reference picture list 0 can be used for decoding. For B slices, both reference picture lists 0 and reference picture list 1 can be used for decoding. The two reference picture lists are constructed without using the reference picture list initialization process or the reference picture list modification process.
[0133] Not all pictures in the reference picture list are used as reference pictures for the current picture or slice. Only the active entries in the reference picture list can be used during the decoding process of the slice data. In VVC (e.g., VVC draft 9), the default number of active entries is signaled in the PPS and can be overwritten by the slice header of the current slice.
[0134] To identify the picture in the DPB to construct the RPL, a POC including the most significant bits ("MSB") and the least significant bits ("LSB") is used. In VVC (e.g., VVC draft 9), the LSB of the POC is signaled in the PH, and the MSB can be explicitly signaled in the PH or derived by comparing the POC LSB of the current picture with the POC LSB of one or more previous pictures.
[0135] In VVC (e.g., VVC draft 9), decoded pictures in the DPB can be marked as "unused for reference", "used for short-term reference", or "used for long-term reference". At any given moment during the operation of the decoding process, a decoded picture can be marked as only one of these three pictures. Assigning one of these markings to a picture implicitly removes the other markings when applicable. When a picture is marked as "used for reference", this also means that the picture is marked as "used for short-term reference" or "used for long-term reference", but not both at the same time.
[0136] Short-term reference pictures ("STRP") and inter-layer reference pictures ("ILRP") are identified by their NAL (network abstraction layer) unit ID and POC value. Long-term reference pictures ("LTRP") are identified by their NAL unit ID and a number of LSBs of their POC value.
[0137] Figure 5A An exemplary syntax including a syntax structure of a reference picture list according to some embodiments of the present disclosure is shown. Figure 5A The syntax shown may be part of the VVC standard (eg, VVC draft 9) or other video coding techniques.
[0138] like Figure 5A As shown, a syntax structure 500A of a reference picture list (eg, ref_pic_lists()) may exist in a PH syntax structure or a SH.
[0139] like Figure 5A As shown, the syntax element 510A (e.g., rpl_sps_flag[i]) equal to 1 specifies that the reference picture list i (e.g., i can be 0 or 1) in the syntax structure of the reference picture list (e.g., ref_pic_lists()) is derived based on one of the syntax structures of the reference picture list structure in the SPS (e.g., ref_pic_list_struct(listIdx, rplsIdx) with listIdx equal to i). The syntax element 510A equal to 0 specifies that the reference picture list i (e.g., i can be 0 or 1) is derived based on the syntax structure of the reference picture list (e.g., ref_pic_list_struct(listIdx, rplsIdx) with listIdx equal to i) directly included in the syntax structure of the reference picture list (e.g., ref_pic_list_struct(listIdx, rplsIdx)).
[0140] When the syntax element 510A is not present, the following scheme is used. First, if the number of reference picture lists in the SPS (e.g., sps_num_ref_pic_lists[i]) is equal to 0, then the value of the syntax element 510A is inferred to be equal to 0. Second, if the number of reference picture lists in the SPS (e.g., sps_num_ref_pic_lists[i]) is not equal to 0 (e.g., if the number of reference picture lists in the SPS is greater than 0), when the syntax element 520A (e.g., pps_rpl1_idx_present_flag) is equal to 0 and i is equal to 1, then the value of the syntax element 510A of reference picture list 1 in the SPS (e.g., rpl_sps_flag[1]) is inferred to be equal to the value of the syntax element 510A of reference picture list 0 in the SPS (e.g., rpl_sps_flag[0]).
[0141] The syntax element 530A (e.g., rpl_idx[i]) specifies the index of the ref_pic_list_struct(listIdx, rplsIdx) (listIdx equals i) used to derive the reference picture list i for the current picture, in the list of ref_pic_list_struct(listIdx, rplsIdx) with listIdx equal to i included in the SPS. The length of the syntax element 530A is a certain number of bits, which is the smallest integer with base 2 greater than or equal to the logarithm of the number of ref_pic_list_struct(listIdx, rplsIdx) syntax structures in the SPS, which can be expressed as Ceil(Log2(sps_num_ref_pic_lists[i])) bits. The value of the syntax element 530A can be in the inclusive range of 0 to the number of reference picture lists in the SPS - 1 (e.g., sps_num_ref_pic_lists[i] - 1). When syntax element 530A is not present, the value of rpl_idx[1] is inferred to be equal to the value of rpl_idx[0] if syntax element 510A is equal to 1 and syntax element 520A is equal to 0, otherwise the value of rpl_idx[1] is inferred to be equal to 0.
[0142] The variable RplsIdx[i] can be derived as follows:
[0143] RplsIdx[i]=rpl_sps_flag[i]? rpl_idx[i]: sps_num_ref_pic_lists[i].
[0144] The syntax element 540A (e.g., poc_lsb_lt[i][j]) specifies the value of the picture order count modulo MaxPicOrderCntLsb for the j-th LTRP entry in the i-th reference picture list in ref_pic_lists(). The length of the syntax element 540A is equal to the base 2 logarithm of the maximum LSB in the POC in bits (e.g., sps_log2_max_pic_order_cnt_lsb_minus4+4 bits).
[0145] The variable PocLsbLt[i][j] can be derived as follows:
[0146] PocLsbLt[i][j]=ltrp_in_header_flag[i][RplsIdx[i]]? poc_lsb_lt[i][j]:
[0147] rpls_poc_lsb_lt[listIdx][RplsIdx[i]][j].
[0148] The syntax element 550A (eg, delta_poc_msb_cycle_present_flag[i][j]) equal to 1 specifies that the syntax element 560A (eg, delta_poc_msb_cycle_lt[i][j]) is present. The syntax element 550A equal to 0 specifies that the syntax element 560 is not present.
[0149] The previous picture in decoding order that has the same nuh_layer_id as the slice header or picture header that references the ref_pic_lists() syntax structure, and has TemporalID and ph_non_ref_pic_flag both equal to 0, and is not a RASL (Random Access Skipped Leading) or RADL (Random Access Decodable Leading) picture, can be referred to as prevTid0Pic. nuh_layer_id is a syntax element that specifies the identifier of the layer to which a VCL (Video Coding Layer) NAL (Network Abstraction Layer) unit belongs, or the identifier of the layer to which a non-VCL NAL unit applies. TemporalID is the temporal identifier of the picture. The set of previous POC values described as setOfPrevPocVals is a set consisting of:
[0150] - POC value of prevTid0Pic (e.g., PicOrderCntVal);
[0151] - the POC value (e.g., PicOrderCntVal) of each picture referenced by an entry in reference picture list 0 (e.g., RefPicList[0]) or reference picture list 1 (e.g., RefPicList[1]) of prevTid0Pic and having the same nuh_layer_id as the current picture;
[0152] - The POC value (eg, PicOrderCntVal) of each picture that follows prevTid0Pic in decoding order and has the same nuh_layer_id as the current picture and precedes the current picture in decoding order.
[0153] When there is more than one value in setOfPrevPocVals whose value modulo MaxPicOrderCntLsb is equal to the variable PocLsbLt[i][j], then the value of the syntax element 550A (eg, delta_poc_msb_cycle_present_flag[i][j]) is equal to 1.
[0154] Figure 5B FIG. 4 shows an exemplary pseudo code including the derivation of the variable FullPocLt[i][j] according to some embodiments of the present disclosure. Figure 5B As shown, the syntax element 560A (eg, delta_poc_msb_cycle_lt[i][j]) specifies the value of the variable FullPocLt[i][j]. The value of the syntax element 560A (eg, delta_poc_msb_cycle_lt[i][j]) can be between 0 and 2. (32-sps _ log2 _ max _ pic _ order _ cnt _ lsb _ minus4-4) When syntax element 560 is not present, the value of syntax element 560 is inferred to be equal to 0.
[0155] Fig. 6A An exemplary syntax including a syntax structure of a reference picture list structure according to some embodiments of the present disclosure is shown. Fig. 6AThe syntax structure shown may be part of the VVC standard (e.g., VVC draft 9) or other video coding techniques. Fig. 6A As shown, ref_pic_list_struct(listIdx, rplsIdx) can exist in SPS, PH syntax structure, or SH. Depending on whether the syntax structure is contained in SPS, PH syntax structure, or SH, the following applies:
[0156] - If ref_pic_list_struct(listIdx, rplsIdx) is present in a PH syntax structure or SH, the ref_pic_list_struct(listIdx, rplsIdx) syntax structure specifies the reference picture list listIdx of the current picture (eg, the coded picture containing the PH syntax structure or SH).
[0157] - If ref_pic_list_struct(listIdx, rplsIdx) is not present in the PH syntax structure or SH (for example, it is present in the SPS), the ref_pic_list_struct(listIdx, rplsIdx) syntax structure specifies candidates for the reference picture list listIdx, and the term "current picture" in the semantics specified in the remainder of this clause refers to each picture that: 1) has a PH syntax structure or contains one or more slices of rpl_idx[listIdx], where rpl_idx[listIdx] is equal to the index in the list of the ref_pic_list_struct(listIdx, rplsIdx) syntax structure included in the SPS, and 2) is in a coded layer-wise videosequence (CLVS) that references the SPS.
[0158] like Fig. 6A As shown, the syntax element 610A (e.g., num_ref_entries[listIdx][rplsIdx]) specifies the number of entries in the ref_pic_list_struct(listIdx, rplsIdx) syntax structure. The value of parameter 610A can be in the inclusive range of 0 to MaxDpbSize+13, where MaxDpbSize is as specified in the level of the video coding standard (e.g., VVC draft 9).
[0159] The syntax element 620A (e.g., ltrp_in_header_flag[listIdx][rplsIdx]) equal to 0 specifies that the POC LSBs of the LTRP entries indicated in the ref_pic_list_struct(listIdx, rplsIdx) syntax structure are present in the same syntax structure. The syntax element 620A (e.g., ltrp_in_header_flag[listIdx][rplsIdx]) equal to 1 specifies that the POC LSBs of the LTRP entries indicated in the ref_pic_list_struct(listIdx, rplsIdx) syntax structure are not present in the same syntax structure. When sps_long_term_ref_pics_flag is equal to 1 and rplsIdx is equal to sps_num_ref_pic_lists[listIdx], the value of the syntax element 620A is inferred to be equal to 1.
[0160] The syntax element 630A (e.g., inter_layer_ref_pic_flag[listIdx][rplsIdx][i]) equal to 1 specifies that the i-th entry in the ref_pic_list_struct(listIdx, rplsIdx) syntax structure is an ILRP entry. The syntax element 630A equal to 0 specifies that the i-th entry in the ref_pic_list_struct(listIdx, rplsIdx) syntax structure is not an ILRP entry. When the syntax element 630A is not present, the value of the syntax element 630A is inferred to be equal to 0.
[0161] The syntax element 640A (e.g., st_ref_pic_flag[listIdx][rplsIdx][i]) equal to 1 specifies that the i-th entry in the ref_pic_list_struct(listIdx, rplsIdx) syntax structure is a STRP entry. The syntax element 640A equal to 0 specifies that the i-th entry in the ref_pic_list_struct(listIdx, rplsIdx) syntax structure is a LTRP entry. When the syntax element 630A is equal to 0 and the syntax element 640A is not present, the value of the syntax element 640A is inferred to be equal to 1.
[0162] Figure 6B FIG. 4 shows an exemplary pseudo code including derivation of the number of LTRP entries (eg, variable NumLtrpEntries[listIdx][rplsIdx]) according to some embodiments of the present disclosure. Figure 5AThe variable 570A in Figure 6B Export as shown.
[0163] Figure 6C FIG. 4 shows an exemplary pseudo code including the derivation of the variable AbsDeltaPocSt[listIdx][rplsIdx][i] according to some embodiments of the present disclosure. Figure 6C As shown, the syntax element 650A (e.g., abs_delta_poc_st[listIdx][rplsIdx][i]) specifies the value of the variable AbsDeltaPocSt[listIdx][rplsIdx][i] (e.g., variable 690A). The value of the syntax element 650A (e.g., abs_delta_poc_st[listIdx][rplsIdx][i]) can be between 0 and 2. 15 -1 inclusive.
[0164] The syntax element 660A (e.g., strp_entry_sign_flag[listIdx][rplsIdx][i]) equal to 1 specifies that the i-th entry in the ref_pic_list_struct (listIdx, rplsIdx) syntax structure has a value greater than or equal to 0. The syntax element 660A equal to 0 specifies that the i-th entry in the ref_pic_list_struct (listIdx, rplsIdx) syntax structure has a value less than 0. When the syntax element 660A is not present, the value of the syntax element 660A is inferred to be equal to 1.
[0165] Fig.6D FIG. 1 shows an exemplary pseudo code including the derivation of the variable DeltaPocValSt[listIdx][rplsIdx] according to some embodiments of the present disclosure. Fig.6D As shown, DeltaPocValSt[listIdx][rplsIdx] is derived.
[0166] Reference Fig. 6A , the syntax element 670A (e.g., rpls_poc_lsb_lt[listIdx][rplsIdx][i]) specifies the value of the picture order count modulo MaxPicOrderCntLsb of the picture referenced by the i-th entry in the ref_pic_list_struct(listIdx, rplsIdx) syntax structure. The length of the syntax element 670A is sps_log2_max_pic_order_cnt_lsb_minus4+4 bits.
[0167] The syntax element 680A (e.g., ilrp_idx[listIdx][rplsIdx][i]) specifies the index of the ILRP of the i-th entry in the list of direct reference layers in the ref_pic_list_struct(listIdx, rplsIdx) syntax structure. The value of the syntax element 680A can be in the inclusive range of 0 to NumDirectRefLayers[GeneralLayerIdx[nuh_layer_id]]-1, where NumDirectRefLayers[LayerIdx] represents the number of direct reference layers of the layer with index equal to LayerIdx.
[0168] Figure 7 An exemplary syntax including a syntax structure of a reference picture list structure in a sequence parameter set according to some embodiments of the present disclosure is shown. Figure 7 The syntax shown may be part of the VVC standard (eg, VVC draft 9) or other video coding techniques.
[0169] like Figure 7 As shown, syntax element 710 (eg, sps_long_term_ref_pics_flag) equal to 0 specifies that no LTRP is used for inter prediction of any coded picture in the CLVS. Syntax element 710 equal to 1 specifies that LTRP is used for inter prediction of one or more coded pictures in the CLVS.
[0170] Syntax element 720 (e.g., sps_inter_layer_ref_pics_present_flag) equal to 0 specifies that no ILRP is used for inter-frame prediction of any coded picture in the CLVS. Syntax element 720 equal to 1 specifies that ILRP can be used for inter-frame prediction of one or more coded pictures in the CLVS. The value of syntax element 720 is inferred to be equal to 0 when sps_video_syntax element_set_id is equal to 0, that is, the SPS does not reference a VPS (video parameter set), and the VPS is not referenced when decoding each CLVS (only one layer) that references the SPS. The value of syntax element 720 is equal to 0 when vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is equal to 1, that is, the layer indexed by GeneralLayerIdx[nuh_layer_id] does not use inter-layer prediction.
[0171] The syntax element 730 (eg, sps_idr_rpl_present_flag) equal to 1 specifies that the reference picture list syntax element is present in the slice header of an IDR (instaneous decoding refresh) picture. The syntax element 730 equal to 0 specifies that the reference picture list syntax element is not present in the slice header of an IDR picture.
[0172] Syntax element 740 (e.g., sps_rpl1_same_as_rpl0_flag) equal to 1 specifies that the syntax element sps_num_ref_pic_lists[1] and the syntax structure ref_pic_list_struct(1, rplsIdx) do not exist, and the following apply: the value of sps_num_ref_pic_lists[1] is inferred to be equal to the value of sps_num_ref_pic_lists[0]; and for rplsIdx in the range from 0 to sps_num_ref_pic_lists[0]–1, the value of each syntax element in ref_pic_list_struct(1, rplsIdx) is inferred to be equal to the value of the corresponding syntax element in ref_pic_list_struct(0, rplsIdx).
[0173] The syntax element 750 (e.g., sps_num_ref_pic_lists[i]) specifies the number of ref_pic_list_struct(listIdx, rplsIdx) syntax structures included in the SPS with listIdx equal to i. The value of the syntax element 750 can be in the inclusive range of 0 to 64. For each value of listIdx (equal to 0 or 1), the decoder (e.g., by Figure 3A Process 300A or Figure 3B Process 300B) can allocate memory for a total number of ref_pic_list_struct(listIdx, rplsIdx) syntax structures equal to the number of RPLs in the SPS + 1 (e.g., sps_num_ref_pic_lists[i]+1), because there may be a ref_pic_list_struct(listIdx, rplsIdx) syntax structure directly signaled in the slice header of the current picture.
[0174] Figure 8 An exemplary syntax including a syntax structure of a reference picture list in a picture parameter set according to some embodiments of the present disclosure is shown. Figure 8 The syntax shown may be part of the VVC standard (eg, VVC draft 9) or other video coding techniques.
[0175] like Figure 8 As shown, when i is equal to 0, i.e., for reference picture list 0, syntax element 810 (e.g., pps_num_ref_idx_default_active_minus1[i]) plus 1 specifies the inferred value of variable NumRefIdxActive[0] for P or B slices with sh_num_ref_idx_active_override_flag equal to 0. When i is equal to 1, i.e., for reference picture list 1, syntax element 810 plus 1 specifies the inferred value of variable NumRefIdxActive[1] for B slices with sh_num_ref_idx_active_override_flag equal to 0. The value of syntax element 810 can be in the inclusive range of 0 to 14.
[0176] Syntax element 820 (eg, pps_rpl1_idx_present_flag) equal to 0 specifies that rpl_sps_flag[1] and rpl_idx[1] are not present in the PH syntax structure or in the slice header of a picture referencing a PPS. Syntax element 820 equal to 1 specifies that rpl_sps_flag[1] and rpl_idx[1] may be present in the PH syntax structure or in the slice header of a picture referencing a PPS.
[0177] Syntax element 830 (e.g., pps_rpl_info_in_ph_flag) equal to 1 specifies that the reference picture list information is present in the PH syntax structure and is not present in a slice header that references a PPS and does not contain a PH syntax structure. Syntax element 830 equal to 0 specifies that the reference picture list information is not present in the PH syntax structure and may be present in a slice header that references a PPS. When syntax element 830 is not present, the value of syntax element 830 is inferred to be equal to 0.
[0178] Fig. 9A An exemplary syntax including a syntax structure of a reference picture list in a picture header structure according to some embodiments of the present disclosure is shown. Fig. 9A The syntax shown may be part of the VVC standard (eg, VVC draft 9) or other video coding techniques.
[0179] like Fig. 9AAs shown, syntax element 910A (e.g., ph_pic_output_flag) affects the output and removal process of decoded pictures. When syntax element 910A is not present, it is inferred to be equal to 1. There is no picture in the bitstream with ph_non_reference_picture_flag equal to 1 and syntax element 910A equal to 0. The element ph_non_reference_picture_flag equal to 1 specifies that the current picture has never been used as a reference picture. The element ph_non_reference_picture_flag equal to 0 specifies that the current picture may or may not be used as a reference picture.
[0180] A syntax element 920A (e.g., ph_temporal_mvp_enabled_flag) equal to 0 specifies that the temporal motion vector predictor is disabled and is not used for decoding of slices in the current picture. A syntax element 920A equal to 1 specifies that the temporal motion vector predictor is enabled and may be used for decoding of slices in the current picture. When the syntax element 920A is not present, the value of the syntax element 920A is inferred to be equal to 0. Due to other constraints, the value of the syntax element 920A in a consistent bitstream can only be equal to 0 when one or more of the following conditions are met: 1) no reference picture in the DPB has the same spatial resolution and the same scaling window offset as the current picture, and 2) no reference picture in the DPB is present in the activation entry of the RPL of all slices of the current picture. There may be other cases and complex conditions not listed where the syntax element 920A can only be equal to 0.
[0181] Fig. 9B FIG. 4 shows an exemplary pseudo code including the derivation of the variable MaxNumSubblockMergeCand according to some embodiments of the present disclosure. Fig. 9B As shown, the value of MaxNumSubblockMergeCand refers to the maximum number of candidates for sub-block based merge MVP (motion vector predictor). The value of MaxNumSubblockMergeCand can be in the inclusive range of 0 to 5.
[0182] Reference Fig. 9A, the syntax element 930A (e.g., ph_collocated_from_l0_flag) equal to 1 specifies that the collocated picture used for temporal motion vector prediction is derived from reference picture list 0. The syntax element 930A equal to 0 specifies that the collocated picture used for temporal motion vector prediction is derived from reference picture list 1. When the syntax element 920A and the syntax element 830A (e.g., pps_rpl_info_in_ph_flag) are both equal to 1 and num_ref_entries[1][RplsIdx[1]] is equal to 0, the value of the syntax element 930A is inferred to be equal to 1.
[0183] Syntax element 940A (e.g., ph_collocated_ref_idx) specifies the reference index of a collocated picture for temporal motion vector prediction. When syntax element 930A is equal to 1, syntax element 940A references an entry in reference picture list 0, and the value of syntax element 940A may be in the inclusive range of 0 to num_ref_entries[0][RplsIdx[0]]-1. When syntax element 930A is equal to 0, syntax element 940A references an entry in reference picture list 1, and the value of syntax element 940A may be in the inclusive range of 0 to num_ref_entries[1][RplsIdx[1]]-1. When syntax element 940A is not present, the value of syntax element 940A is inferred to be equal to 0.
[0184] Syntax element 950A (e.g., ph_mvd_l1_zero_flag) equal to 1 specifies that the motion vector difference (e.g., mvd_coding(x0, y0, 1, cpIdx)) syntax structure is not parsed, and MvdL1[x0][y0][compIdx] and MvdCpL1[x0][y0][cpIdx][compIdx] are set equal to 0, where compIdx = 0..1 and cpIdx = 0..2. Syntax element 950A equal to 0 specifies that the mvd_coding(x0, y0, 1, cpIdx) syntax structure is parsed. When syntax element 950A is not present, the value of syntax element 950A is inferred to be 1. MvdL1 is the motion vector difference decoded from the bitstream associated with the reference picture in reference picture list 1. MvdCpL1 is the control point motion vector difference decoded from the bitstream associated with the reference picture in reference picture list 1. It is used for coding blocks using affine motion compensation. x0 and y0 are the top left corner position of the current coding block, compIdx is the component index, and cpIdx is the index of the control point.
[0185] Fig. 10AAn exemplary syntax including a syntax structure of a reference picture list in a slice header according to some embodiments of the present disclosure is shown. Fig. 10A The syntax shown may be part of the VVC standard (eg, VVC draft 9) or other video coding techniques.
[0186] like Fig. 10A As shown, the syntax element 1010A (e.g., sh_num_ref_idx_active_override_flag) equal to 1 specifies that the syntax element sh_num_ref_idx_active_minus1[0] is present in the case of slices P and B, and the syntax element sh_num_ref_idx_active_minus1[1] is present in the case of slice B. The syntax element 1010A equal to 0 specifies that the syntax elements sh_num_ref_idx_active_minus1[0] and sh_num_ref_idx_active_minus1[1] are not present. When the syntax element 1010A is not present, the value of the syntax element 1010A is inferred to be equal to 1.
[0187] The syntax element 1020A (e.g., sh_num_ref_idx_active_minus1[i]) is used for the derivation of the variable NumRefIdxActive[i]. The value of the syntax element 1020A can be in the inclusive range of 0 to 14. For i equal to 0 or 1, when the current slice is a B slice, the syntax element 1010A is equal to 1, and when the syntax element 1020A is not present, the value of the syntax element 1020A is inferred to be equal to 0.
[0188] Fig. 10B FIG. 4 shows an exemplary pseudo code including the derivation of the variable NumRefIdxActive[i] according to some embodiments of the present disclosure. Fig. 10B As shown, the value of NumRefIdxActive[i]-1 specifies the maximum reference index of the reference picture list i that may be used for decoding the slice. Syntax element 1020A is used for the derivation of NumRefIdxActive[i], such as Fig. 10B As shown in formula (1). When the value of NumRefIdxActive[i] is equal to 0, no reference index of reference picture list i can be used to decode the slice. When the current slice is a P slice, the value of NumRefIdxActive[0] is greater than 0. When the current slice is a B slice, the values of NumRefIdxActive[0] and NumRefIdxActive[1] are both greater than 0.
[0189] like Fig. 10AAs shown, the syntax element 1030A (eg, sh_cabac_init_flag) specifies a method for determining the initialization table used during the initialization of context variables. When the syntax element 1030A is not present, it is inferred to be equal to 0.
[0190] The syntax element 1040A (e.g., sh_collocated_from_l0_flag) equal to 1 specifies that the collocated picture used for temporal motion vector prediction is derived from reference picture list 0. The syntax element 1040A equal to 0 specifies that the collocated picture used for temporal motion vector prediction is derived from reference picture list 1. When sh_slice_type is equal to B or P, the syntax element 920A (e.g., ph_temporal_mvp_enabled_flag) is equal to 1, and the syntax element 1040A is not present, the following applies: if sh_slice_type is equal to B, then the syntax element 1040A is inferred to be equal to the syntax element 930A (e.g., ph_collocated_from_l0_flag); if sh_slice_type is not equal to B (e.g., sh_slice_type is equal to P), then the value of the syntax element 1040A is inferred to be equal to 1.
[0191] The syntax element 1050A (e.g., sh_collocated_ref_idx) specifies the reference index of the collocated picture for temporal motion vector prediction. When sh_slice_type is equal to P, or when sh_slice_type is equal to B and the syntax element 1040A is equal to 1, the syntax element 1050A refers to an entry in reference picture list 0, and the value of the syntax element 1050A can be in the inclusive range of 0 to NumRefIdxActive[0]-1. When sh_slice_type is equal to B and the syntax element 1040A is equal to 0, the syntax element 1050A refers to an entry in reference picture list 1, and the value of the syntax element 1050A can be in the inclusive range of 0 to NumRefIdxActive[1]-1. When the syntax element 1050A is not present, the following applies: if the syntax element 830 (e.g., pps_rpl_info_in_ph_flag) is equal to 1, then the value of the syntax element 1050A is inferred to be equal to the syntax element 940A (e.g., ph_collocated_ref_idx); if the syntax element 830 is not equal to 1 (e.g., the syntax element 830 is equal to 0), then the value of the syntax element 1050A is inferred to be equal to 0. As a requirement for bitstream conformance, the picture referenced by the syntax element 1050A is the same for all slices of the coded picture, and RprConstraintsActive[sh_collocated_from_10_flag? 0:1][sh_collocated_ref_idx] is equal to 0. This constraint requires that the collocated picture has the same spatial resolution and the same scaling window offset as the current picture.
[0192] In VVC (e.g., VVC draft 9), syntax element 930A (e.g., ph_collocated_from_l0_flag) and syntax element 950A (e.g., ph_mvd_l1_zero_flag) are two flags signaled in PH. Syntax element 930A indicates which reference picture list the collocated picture used for temporal motion vector prediction comes from. Syntax element 950A indicates whether the mvd_coding() syntax structure is parsed for reference picture list 1. Therefore, these two flags are only relevant when the number of activated entries in reference picture list 1 is greater than 0. However, if Fig. 10AAs shown, since the number of active entries in the reference picture list is covered by sh_num_ref_idx_active_minus1[i] in the slice header, when syntax element 930A and syntax element 950A are signaled in PH, the decoder does not know the exact number of active entries in reference picture list 1. Therefore, in VVC (e.g., VVC draft 9), the total number of entries in reference picture list 1 is used as a condition for signaling these two flags, such as Fig. 9A shown.
[0193] It should be understood that although the present disclosure relates to various syntax elements that provide inferences based on values equal to 0 or 1, these values may be configured in any manner (eg, 1 or 0) to provide appropriate inferences.
[0194] In VVC (e.g., VVC draft 9), for I slices, it is guaranteed that the number of active entries in both reference picture lists is equal to 0. For P slices, the number of active entries in reference picture list 0 is greater than 0, and the number of active entries in reference picture list 1 is equal to 0. For B slices, the number of active entries in both reference picture lists is greater than 0. There is no guarantee about the total number of entries in the reference picture lists. For example, for I slices, the number of entries in either of the two reference picture lists can be greater than 0. Therefore, the signaling condition that the total number of entries in reference picture list 1 is greater than 0 is too loose for syntax element 930A and syntax element 950A, and will cause unnecessary signaling of these two syntax elements.
[0195] In order to overcome this defect of the traditional encoding technology, in some embodiments of the present disclosure (for example, FIG. 11A to FIG. 11C In the embodiment provided in ), unnecessary signaling is avoided when the number of entries in reference picture list 0 is equal to 0.
[0196] Fig.11A 1 shows a flowchart of an exemplary video encoding method 1100A for signaling a flag in a PH syntax structure according to some embodiments of the present disclosure. The method 1100A may be performed by an encoder (e.g., Figure 2A Process 200A or Figure 2B 200B), or by a device (e.g., Figure 4 For example, one or more processors (e.g., Figure 4 The method 1100A may be performed by a processor 402 of the computer system. In some embodiments, the method 1100A may be implemented by a computer program product embodied in a computer-readable medium including a program executed by a computer (e.g., Figure 4Computer executable instructions, such as program codes, executed by the device 400. Fig.11A , method 1100A may include the following steps 1102A and 1104A.
[0197] At step 1102A, the encoder encodes the current picture based on the collocated picture. The reference picture may be derived, for example, by reference picture list 0 and reference picture list 1, each of which includes a DPB (e.g., Figure 3B A list of reconstructed pictures in the buffer 234 in ( ) to be used as reference pictures. The current picture is used for temporal motion vector prediction.
[0198] In step 1104A, when the number of entries in reference picture list 0 and the number of entries in reference picture list 1 are both greater than 0, a syntax element ph_collocated_from_l0_flag (e.g., syntax element 930A) and a syntax element ph_mvd_l1_zero_flag (e.g., syntax element 950A) are signaled. The syntax element ph_collocated_from_l0_flag indicates which reference picture list the collocated picture used for temporal motion vector prediction comes from, that is, the collocated picture used for temporal motion vector prediction comes from the reference picture list indicated by the first flag. The syntax element ph_mvd_l1_zero_flag indicates whether the motion vector difference syntax structure associated with reference picture list 1 is signaled. In this way, when the two flags are signaled, the entries in reference picture list 1 and reference picture list 0 are both guaranteed. Therefore, unnecessary signaling is avoided when the number of entries in reference picture list 0 is equal to 0, and decoding efficiency is improved.
[0199] Fig. 11B 1 shows a flowchart of an exemplary video decoding method 1100B for decoding a flag in a PH syntax structure according to some embodiments of the present disclosure. The method 1100B may be performed by a decoder (e.g., Figure 3A Process 300A or Figure 3B 300B), or by a device (e.g., Figure 4 For example, one or more processors (e.g., Figure 4 The method 1100B may be performed by a processor 402 of the computer system. In some embodiments, the method 1100B may be implemented by a computer program product, which is embodied in a computer-readable medium, which includes a computer program product (e.g., Figure 4 Computer executable instructions, such as program codes, executed by the device 400. Fig. 11B, method 1100B may include the following steps 1102B to 1106B.
[0200] At step 1102B, the decoder receives a video bitstream (eg, Figure 3B The video bit stream 228 may be encoded using inter-frame prediction.
[0201] At step 1104B, when the number of entries in reference picture list 0 and the number of entries in reference picture list 1 are both greater than 0, a syntax element ph_collocated_from_l0_flag (e.g., syntax element 930A) and a syntax element ph_mvd_l1_zero_flag (e.g., syntax element 950A) are decoded by the decoder from the bitstream. The syntax element ph_collocated_from_l0_flag indicates which reference picture list the collocated picture used for temporal motion vector prediction comes from, i.e., the collocated picture used for temporal motion vector prediction comes from the reference picture list indicated by the first flag. The syntax element ph_mvd_l1_zero_flag indicates whether there is a motion vector difference syntax structure associated with reference picture list 1 in the bitstream. In this way, when two flags are signaled, entries in reference picture list 1 and reference picture list 0 are both guaranteed.
[0202] In step 1106B, the current picture is decoded based on the collocated picture. Therefore, unnecessary signaling in the case where the number of entries in the reference picture list 0 is equal to 0 is avoided, and efficiency is improved.
[0203] Fig. 11C FIG. 1 shows a portion of an exemplary picture header syntax structure 1100C according to some embodiments of the present disclosure. The picture header (PH) syntax structure 1100C may be used in the method 1100A. Fig. 9A The syntax structure 900A modifies the PH syntax structure 1100B, and the changes compared to the previous VVC are shown in italics in boxes 1110C and 1120C.
[0204] Referring to 1110C, in some embodiments, when num_ref_entries[0][RplsIdx[0]] is greater than 0 and num_refe_entries[1][RplsIdx[1]] is greater than 0, a syntax element ph_collocated_from_l0_flag (e.g., syntax element 930A) is signaled. Referring to 1120C, when pps_rpl_info_in_ph_flag is not equal to 0, or when num_ref_entries[0][RplsIdx[0]] is greater than 0 and num_refe_entries[1][RplsIdx[1]] is greater than 0, a syntax element ph_mvd_l1_zero_flag (e.g., syntax element 950A) is signaled. Thus, when the number of entries in reference picture list 0 and the number of entries in reference picture list 1 are both greater than 0, syntax element 930A and syntax element 950A may be signaled. Unnecessary signaling when the number of entries in reference picture list 0 is equal to 0 is avoided, and coding efficiency is improved.
[0205] In VVC (e.g., VVC draft 9), collocated pictures may be indicated in PH or SH. If reference picture list information is signaled in PH, collocated pictures are indicated in PH by syntax element 930A (e.g., ph_collocated_from_l0_flag) and syntax element 940A (e.g., ph_collocated_ref_idx). If reference picture list information is signaled in SH, collocated pictures are indicated in SH by syntax element 1040A (e.g., sh_collocated_from_l0_flag) and syntax element 1050A (e.g., sh_collocated_ref_idx). Syntax element 930A equal to 1 specifies that collocated pictures for temporal motion vector prediction are derived from reference picture list 0. Syntax element 930A equal to 0 specifies that collocated pictures for temporal motion vector prediction are derived from reference picture list 1. When syntax element 930A is signaled in PH, the signaling condition is that the number of entries in reference picture list 1 is greater than 0. However, the number of active entries in the reference picture list can be overridden at the slice level. Therefore, even if the signaled syntax element 930A is 0, it is not guaranteed that the collocated picture can be selected from reference picture list 1 because the SH may overwrite the number of active entries in reference picture list 1 to 0. Similarly, when the syntax element 940A is signaled in the PH, the maximum allowed value is the number of entries in the reference picture list minus 1. If the SH overwrites the number of active entries to be less than the value of the syntax element 940A, it is an illegal bitstream.
[0206] To avoid such illegal scenarios, VVC (e.g., VVC Draft 9) proposes several bitstream consistency constraints. However, satisfying all constraints will impose a burden on the encoder. In fact, the decoder should also consider how to handle the bitstream when such illegal situations occur.
[0207] In order to overcome this defect of the traditional encoding technology, in some embodiments of the present disclosure (for example, FIG. 12A to FIG. 12J In the embodiment provided in ), the collocated picture is indicated without signaling the index of the reference picture list to avoid illegal scenes in a more robust way.
[0208] Fig. 12A 1 shows a flowchart of an exemplary video encoding method 1200A for indicating a collocated image without signaling an index to a reference image list according to some embodiments of the present disclosure. The method 1200A may be performed by an encoder (e.g., Figure 2A Process 200A or Figure 2B 200B), or by a device (e.g., Figure 4 For example, one or more processors (e.g., Figure 4 The method 1200A may be performed by a processor 402 of the computer system. In some embodiments, the method 1200A may be implemented by a computer program product embodied in a computer-readable medium including a program executed by a computer (e.g., Figure 4 Computer executable instructions, such as program codes, executed by the device 400. Fig. 12A , method 1200A may include the following steps 1202A and 1204A.
[0209] At step 1202A, the encoder encodes the current picture into a bitstream based on the collocated picture, wherein the collocated picture is used for temporal motion vector prediction. At step 1204A, the collocated picture in the bitstream is indicated without signaling an index to a reference picture list. Since the collocated picture is indicated without referencing an entry in the reference picture list by an index, the collocated picture can be legally indicated even if the SH overwrites the number of active entries in reference picture list 1 to 0. Therefore, the robustness of the encoding process is improved.
[0210] Fig. 12B FIG. 1 shows an exemplary flow chart of an encoding method 1200B according to some embodiments of the present disclosure. It should be understood that the method 1200B may be Fig. 12A Part of step 1204A of method 1200A. Fig. 12C Another flow chart of an exemplary video encoding method 1200B for indicating collocated images according to some embodiments of the present disclosure is shown. Fig. 12B and Fig. 12C In some embodiments, method 1200B may further include the following steps 1202B to 1206B.
[0211] In step 1202B, when the juxtaposed picture is an inter-layer reference picture, a first parameter is signaled to indicate the juxtaposed picture. The first parameter indicates the index of the juxtaposed picture in the list of direct reference layers of the layer in which the current picture is located. For example, the index may be the syntax element inter_layer_col_pic_idx. Thus, the juxtaposed picture is indicated without utilizing a reference picture list. When SH overrides the number of activated entries in the reference picture list, illegal scenarios may be avoided. Prior to step 1202B, a flag may be signaled to indicate whether the juxtaposed picture is an inter-layer reference picture. Step 1202B may also refer to Fig. 12C 1201C and 1202C.
[0212] In step 1204B, when the collocated picture is a short-term reference picture (STRP), a delta picture order count (delta POC) is signaled. In addition, the POC can be derived from the delta POC. In this scenario, the POC is used to indicate the collocated picture without using the reference picture list. Therefore, illegal scenarios can be avoided when the SH covers the number of active entries in the reference picture list. Step 1204B can also refer to Fig. 12C 1203C and 1204C.
[0213] In step 1206B, when the collocated picture is a long-term reference picture (LTRP), the least significant bit (LSB) of the POC and the most significant bit (MSB) of the POC are signaled. In addition, the POC can be derived from the LSB and the MSB. In this scenario, the POC is used to indicate the collocated picture without using the reference picture list. Therefore, when the SH covers the number of active entries in the reference picture list, an illegal scenario can be avoided. Step 1206B can also refer to Fig. 12C 1203C and 1205C in. Using POC to indicate the collocated image can effectively enhance the robustness of determining the collocated image. In some embodiments, before steps 1204B and 1206B, a flag indicating whether the collocated image is a short-term reference image can be sent by a signal.
[0214] Fig.12D 1 shows a flowchart of an exemplary video decoding method 1200D for indicating a collocated image without decoding an index of a reference image list according to some embodiments of the present disclosure. The method 1200D may be performed by a decoder (e.g., Figure 3A Process 300A or Figure 3B 300B), or by a device (e.g., Figure 4 For example, one or more processors (e.g., Figure 4 The method 1200D may be performed by a processor 402 of the computer system. In some embodiments, the method 1200D may be implemented by a computer program product embodied in a computer-readable medium including a program executed by a computer (e.g., Figure 4 Computer executable instructions, such as program codes, executed by the device 400. Fig.12D , method 1200D may include the following steps 1202D to 1206D.
[0215] At step 1202D, the decoder receives a video bitstream for processing (e.g., Figure 3B The video bitstream 228 of FIG. 1 may be encoded using inter-frame prediction. The reference images may be derived, for example, by reference image list 0 and reference image list 1, each of which includes a DPB (e.g., Figure 3B A list of reconstructed images in the buffer 234 in ( ) to be used as reference images.
[0216] In step 1204D, the collocated picture for temporal motion vector prediction is determined based on the bitstream, but the index to the reference picture list is not decoded.
[0217] In step 1206D, the current picture is decoded based on the collocated picture. Since the collocated picture is indicated without utilizing the reference picture list structure, the collocated picture can be legally indicated even if the SH overwrites the number of active entries in the reference picture list 1 to 0. Therefore, the robustness of the decoding process is improved.
[0218] Fig.12E FIG. 1 shows an exemplary flowchart of a decoding method 1200E according to some embodiments of the present disclosure. It should be understood that the method 1200E may be Fig.12D Part of step 1204D of method 1200D.
[0219] In step 1202E, when the juxtaposed picture is an inter-layer reference picture, a first parameter is decoded to indicate the juxtaposed picture. The first parameter indicates the index of the juxtaposed picture in the list of direct reference layers of the layer in which the current picture is located. For example, the index can be the syntax element inter_layer_col_pic_idx. Therefore, the juxtaposed picture is indicated without utilizing a reference picture list. When SH overrides the number of activated entries in the reference picture list, illegal scenes can be avoided. In some embodiments, before step 1202E, a first flag indicating whether the juxtaposed picture is an inter-layer reference picture is decoded, and it is determined whether the juxtaposed picture is an inter-layer reference picture based on the first flag.
[0220] In step 1204E, when the collocated picture is a short-term reference picture (STRP), an incremental picture order count (incremental POC) is decoded. In addition, the POC can be derived from the incremental POC. In this scenario, the collocated picture is indicated using the POC without using the reference picture list. Therefore, when the SH covers the number of active entries in the reference picture list, illegal scenarios can be avoided.
[0221] In step 1206E, when the collocated image is a long-term reference picture (LTRP), the least significant bit (LSB) of the POC and the most significant bit (MSB) of the POC are decoded. In addition, the POC can be derived through the LSB and the MSB. In this scenario, the POC is used to indicate the collocated image without using the reference image list. Therefore, when the SH covers the number of activated entries in the reference image list, an illegal scenario can be avoided. In some embodiments, before steps 1204E and 1206E, a second flag indicating whether the collocated image is a short-term reference image is decoded, and it is determined whether the collocated image is a short-term reference image based on the second flag.
[0222] Fig.12F and Figure 12G A portion of an exemplary picture parameter set syntax structure 1200F and a portion of an exemplary slice header syntax structure 1200G according to some embodiments of the present disclosure are shown. The picture parameter set syntax structure 1200F and the slice header syntax structure 1200G can be used together with the methods 1200A, 1200B, 1200D, and 1200E. Fig. 9A Part 960A of the syntax structure 900A modifies the picture parameter set syntax structure 1200F, with changes compared to the previous VVC shown in italics and proposed deleted syntax further shown in strikethrough. Fig. 10A The portion 1060A of the syntax structure 1000A modifies the stripe header syntax structure 1200G, and the proposed deleted syntax is further shown in strikethrough. Fig.12F and Figure 12G As shown, the syntax elements ph_collocated_from_l0_flag, ph_collocated_ref_idx, sh_collocated_from_l0_flag and sh_collocated_ref_idx are no longer signaled in the PPS or SH.
[0223] like Fig.12FAs shown, a syntax element 1210F (e.g., inter_layer_col_pic_flag) equal to 1 specifies that the collocated picture used for temporal motion vector prediction is referenced by an ILRP entry in the reference picture list. A syntax element 1210F equal to 0 specifies that the collocated picture used for temporal motion vector prediction is not referenced by an ILRP entry in the reference picture list. When the syntax element 1210F is not present, the value of the syntax element 1210F is inferred to be equal to 0. The syntax element 1210F may be signaled in 1201C to determine whether the collocated picture is an inter-layer reference picture.
[0224] Syntax element 1210F (e.g., st_col_pic_flag) equal to 1 specifies that the collocated picture for temporal motion vector prediction is referenced by a STRP entry in the reference picture list. Syntax element 1220F equal to 0 specifies that the collocated picture for temporal motion vector prediction is referenced by a LTRP entry in the reference picture list. When syntax element 1210F is equal to 0 and syntax element 1220F is not present, the value of syntax element 1220F is inferred to be equal to 1. Syntax element 1220F may be signaled in 1203C to determine whether the collocated picture is a short-term reference picture. If syntax element 1220F is equal to 1 (e.g., Fig. 12C If 1203C is true), then execute step 1204B (if Fig. 12B ), and an incremental picture order count (incremental POC) is signaled (e.g., Fig. 12C If the syntax element 1220F is equal to 0 (e.g., Fig. 12C 1203C is false), then execute step 1206B (if Fig. 12B ), and signaling the least significant bit (LSB) of the POC and the most significant bit (MSB) of the POC (e.g., Fig. 12C 1205C in.
[0225] The syntax element 1230F (eg, abs_delta_poc_st_col) specifies the value of the variable AbsDeltaPocStCol. Fig.12H FIG. 12 shows an exemplary pseudo code including AbsDeltaPocStCol derivation 1210H according to some embodiments of the present disclosure. The value of the syntax element 1230F (eg, abs_delta_poc_st_col) can be between 0 and 2. 15 -1 inclusive.
[0226] Reference Fig.12F, a syntax element 1240F (eg, sign_delta_poc_st_col_flag) equal to 1 specifies that the value of the variable DeltaPocValStCol is greater than or equal to 0. A syntax element 1240F equal to 0 specifies that the value of the variable DeltaPocValStCol is less than 0. When the syntax element 1240F is not present, the value of the syntax element 1240F is inferred to be equal to 1. Fig.12I FIG. 1 shows an exemplary pseudo code including the derivation of DeltaPocValStCol according to some embodiments of the present disclosure. Fig.12I The derived variable DeltaPocValStCol is shown.
[0227] Reference Fig.12F In some embodiments, the syntax element 1250F (eg, poc_lsb_lt_col) specifies the value of the picture order count modulo MaxPicOrderCntLsb for the collocated picture used for temporal motion vector prediction. The length of the syntax element 1250F is sps_log2_max_pic_order_cnt_lsb_minus4+4 bits.
[0228] The syntax element 1260 (eg, delta_poc_msb_cycle_lt_col) specifies the value of the variable FullPocLtCol as follows:
[0229] FullPocLtCol=PicOrderCntVal–delta_poc_msb_cycle_lt_col*MaxPicOrderCntLsb-
[0230] (PicOrderCntVal&(MaxPicOrderCntLsb-1))+poc_lsb_lt_col
[0231] The syntax element 1270F (eg, delta_poc_msb_cycle_col_present_flag) equal to 1 specifies that the syntax element 1260F (eg, delta_poc_msb_cycle_lt_col) is present. The syntax element 1270B equal to 0 specifies that the syntax element 1260F is not present.
[0232] Further, for syntax element 1270F, let prevTid0Pic be the previous picture in decoding order whose nuh_layer_id is the same as the slice or picture header referencing the ref_pic_lists() syntax structure, whose TemporalId is equal to 0, and which is not a RASL or RADL picture. Let setOfPrevPocVals be the set containing:
[0233] -PicOrderCntVal of prevTid0Pic,
[0234] - PicOrderCntVal of each picture that is represented by RefPicList[0] or RefPicList[1] of prevTid0Pic
[0235] and nuh_layer_id is the same as the current image,
[0236] - PicOrderCntVal of each picture that follows prevTid0Pic in decoding order and has the same nuh_layer_id as the current picture and that precedes the current picture in decoding order.
[0237] When there is more than one value in setOfPrevPocVals whose modulo MaxPicOrderCntLsb value is equal to syntax element 1250F (eg, poc_lsb_lt_col), then the value of delta_poc_msb_cycle_present_flag[i][j]) shall be equal to 1.
[0238] When the collocated picture used for temporal motion vector prediction is referenced by an ILRP entry in the reference picture list, the syntax element 1280F (e.g., inter_layer_col_pic_idx) specifies the index of the collocated picture in the list of direct reference layers used for the temporal motion vector. The value of the syntax element 1280F can be in the inclusive range of 0 to NumDirectRefLayers[GeneralLayerIdx[nuh_layer_id]]-1.
[0239] like Fig.12F As shown, when sps_inter_layer_ref_pics_present_flag (e.g., syntax element 720) is equal to 1, a syntax element 1210F is signaled that ILRP can be used for inter-frame prediction of one or more coded pictures in CLVS, and an index (e.g., syntax element 1280F inter_layer_col_pic_idx) is signaled to indicate which inter-layer reference picture is used as a collocated picture, which is the same as Fig. 12B If the collocated picture is a short-term reference picture, that is, the syntax element 1220F (e.g., st_col_pic_flag) is equal to 1, a delta POC is signaled (e.g., syntax element 1230F), which corresponds to step 1202B. Fig. 12BIf the collocated picture is a long-term reference picture, i.e., syntax element 1220F (e.g., st_col_pic_flag) is equal to 0, the LSB of the POC (e.g., syntax element 1250F) and the incremental MSB of the POC (e.g., syntax element 1260F) are signaled, which corresponds to step 1204B of FIG. Fig. 12B Corresponding to step 1206B of . In addition, the MSB of the POC can be derived from the incremental MSB, and the POC can be derived from the MSB and the LSB. Therefore, the collocated picture can be indicated independently of the reference picture list structure.
[0240] Considering that VVC (e.g., VVC draft 9) has a constraint that the collocated image referenced by all slices within a picture should be the same image, according to the updated syntax structures 1200F and 1200G, the collocated image can be indicated only in the PH, but not in the SH. Therefore, it can be guaranteed that all slices within the picture have the same collocated image, and no constraint is required, thereby improving the efficiency and robustness of indicating the collocated image.
[0241] Fig.12J FIG. 1 shows exemplary pseudo code for deriving a collocated image labeled colPic and a flag colPicFlag for methods 1200A, 1200B, 1200C, and 1200D according to some embodiments of the present disclosure. Fig.12J As shown, for different scenarios of collocated images, such as collocated images referenced by STRP entries in the reference image list (as shown in scenario 1210J), collocated images referenced by LTRP entries in the reference image list (as shown in scenario 1220J), or collocated images referenced by ILPR entries in the reference image list (as shown in scenario 1230J), all strips within the image have the same collocated image (e.g., picA). Therefore, the robustness of determining collocated images is improved.
[0242] In some embodiments, there is a requirement for bitstream consistency, namely the following constraints apply: when ph_temporal_mvp_enabled_flag is equal to 1, colPic is not "no reference picture", but is referenced by an active entry in RefPicList[0] or RefPicList[1], and colPicFlag is equal to 0. "No reference picture" can be regarded as a flag indicating that there is no reference picture in the RPL. colPicFlag equal to 0 indicates that the current picture and the collocated picture have the same picture size and the same scaling window. That is, when temporal MVP is enabled, the collocated picture should be present in the reference picture list and referenced by an active entry in reference picture list 0 or reference picture list 1. Therefore, the robustness of the collocated pictures is improved.
[0243] In VVC (e.g., VVC draft 9), ref_pic_list_struct() and syntax elements for identifying collocated pictures may be signaled in PH or SH according to the value of pps_rpl_info_ph_flag (syntax element 930A (e.g., ph_collocated_from_l0_flag) and syntax element 940A (e.g., ph_collocated_ref_idx) in PH, and syntax element 1040A (e.g., sh_collocated_from_l0_flag) and syntax element 1050A (e.g., sh_colocated_ref_idx) in SH. When the value of pps_rpl_info_ph_flag is equal to 1, syntax element 930A, syntax element 940A, and ref_pic_list_struct() are signaled in PH. The syntax element 1040A and the syntax element 1050A are not signaled. In this case, the values of the syntax element 1040A and the syntax element 1050A are inferred based on the values of the syntax element 930A and the syntax element 940A and the slice type of the current slice. If it is a B slice, the syntax element 1040A is inferred to be equal to the syntax element 930A. If it is a P slice, the syntax element 1040A is directly inferred to be equal to 1 without considering the value of the syntax element 930A. For P slices For both A and B slices, the value of syntax element 1050A is inferred to be equal to syntax element 940A. However, for syntax element 940A signaled in PH, the maximum allowed value is the number of entries in the reference picture list minus 1, but for syntax element 1050A, the maximum allowed value is the number of active entries in the reference picture list minus 1, which may be overwritten in the slice header. Therefore, when syntax element 1050A is inferred to be equal to syntax element 940A, the maximum value constraint may be violated.
[0244] For example, when the signaled syntax element 930A is 0, the number of entries in reference picture list 1 (num_ref_entris[1] signaled in ref_pic_list_structure()) is N, and the signaled ph_colocated_ref_idx is N-1, then in this case the value of the inferred syntax element 1040A is equal to 0, and the value of the inferred syntax element 1050A is equal to N-1. However, the number of active entries in reference picture list 1 may be overwritten to a number less than N. In this case, the bitstream is illegal.
[0245] In yet another example, when the signaled syntax element 930A is 0, the number of entries in reference picture list 1 (num_ref_entries[1] signaled in ref_pic_list_structure()) is N, the signaled syntax element 940A is N-1, and the number of active entries is not overwritten in the slice header (assuming the number of active entries is the same as the number of entries in the reference picture list). However, if the current slice is a P slice, the inferred syntax element 940A is equal to 1, and the inferred syntax element 1050A is equal to N-1. However, the number of entries in reference picture list 0 (num_ref_entries[0] signaled in ref_pic_list_structure()) may be less than N. Therefore, the bitstream is also illegal.
[0246] In order to solve this defect in the traditional coding technology, in some embodiments of the present disclosure (for example, FIG. 13A to FIG. 13C In the embodiment provided in ), the collocated images in SH are also inferred based on the number of activated entries in the reference image list.
[0247] Fig.13A A flowchart of an exemplary video encoding method 1300A for determining the index of a collocated image in a SH using the number of active entries in a reference image list according to some embodiments of the present disclosure is shown. The method 1300A may be performed by an encoder (e.g., Figure 2A Process 200A or Figure 2B 200B), or by a device (e.g., Figure 4 For example, one or more processors (e.g., Figure 4 The method 1300A may be performed by a processor 402 of the computer system. In some embodiments, the method 1300A may be implemented by a computer program product embodied in a computer-readable medium including a program executed by a computer (e.g., Figure 4 Computer executable instructions, such as program codes, executed by the device 400. Fig.13A , method 1300A may include the following steps 1302A to 1306A.
[0248] At step 1302A, it is determined whether to signal a parameter in a slice header to indicate a reference index of a collocated picture. In VVC, the parameter indicating a reference index of a collocated picture in a slice header may be a syntax element sh_collocated_ref_idx.
[0249] At step 1304A, when the parameter is not signaled in the slice header, the collocated picture is determined to be the picture referenced by an index whose value is equal to the smaller of the value of the reference index of the collocated picture signaled in the picture header (e.g., ph_collocated_ref_idx) and the number of active entries in the target reference picture list minus 1 (e.g., NumRefIdxActive[!sh_collocated_from_10_flag]-1). The target reference picture list in the reference picture list is indicated by a flag indicating from which reference picture list the collocated picture for temporal motion vector prediction is derived. Therefore, when inferring the value of the syntax element 1050A (e.g., sh_collocated_ref_idx), the number of active entries in the reference picture list is also taken into account. If the syntax element 940A signaled in the PH (e.g., ph_collocated_ref_idx) is greater than or equal to the number of active entries in the target reference picture list, the inferred value of the syntax element 1050A (e.g., sh_collocated_ref_idx) is constrained to be less than the number of active entries in the target reference picture list. The target reference picture list in the reference picture list is indicated by the syntax element 1040A (e.g., sh_collocated_from_l0_flag).
[0250] At step 1306A, the current image is encoded based on the collocated image, wherein the collocated image is used for temporal motion vector prediction. Thus, illegal bitstreams are avoided and robustness of the collocated image is improved.
[0251] Fig. 13B A flowchart of an exemplary video decoding method 1300B for determining the index of a collocated picture in a SH using the number of active entries in a reference picture list according to some embodiments of the present disclosure is shown. The method 1300B may be performed by a decoder (e.g., Figure 3A Process 300A or Figure 3B 300B), or by a device (e.g., Figure 4 For example, one or more processors (e.g., Figure 4 The method 1300B may be performed by a processor 402 of the computer system. In some embodiments, the method 1300B may be implemented by a computer program product embodied in a computer-readable medium including a program executed by a computer (e.g., Figure 4 Computer executable instructions, such as program codes, executed by the device 400. Fig. 13B , method 1300B may include the following steps 1302B to 1310B.
[0252] At step 1302B, the decoder receives a video bitstream (eg, Figure 3B The video bitstream 228 of FIG. 1 may be encoded using inter-frame prediction. Thus, the reference images may be derived, for example, from reference image list 0 and reference image list 1, each of which includes a DPB (e.g., Figure 3B A list of reconstructed images in the buffer 234 in ( ) to be used as reference images.
[0253] At step 1304B, it is determined whether there is a parameter indicating a reference index of a collocated picture for temporal motion vector prediction in the slice header. In VVC, the parameter indicating a reference index of a collocated picture in the slice header may be a syntax element sh_collocated_ref_idx.
[0254] In step 1306B, when the parameter is not present, the value of the parameter is determined to be equal to the smaller value between the value of the reference index of the collocated picture for temporal motion vector prediction present in the picture header (e.g., ph_collocated_ref_idx) and the number of active entries in the target reference picture list minus 1 (e.g., NumRefIdxActive[!sh_collocated_from_l0_flag]–1). The target reference picture list in the reference picture list is indicated by a flag indicating from which reference picture list the collocated picture for temporal motion vector prediction is derived. Therefore, when determining the value of the syntax element 1050A (e.g., sh_collocated_ref_idx), the number of active entries in the reference picture is also taken into account. If the value of the syntax element 940A (e.g., ph_collocated_ref_idx) signaled in the PH is greater than or equal to the number of active entries in the target reference picture list, the determined value of the syntax element 1050A (e.g., sh_collocated_ref_idx) is constrained to be less than the number of active entries in the target reference picture list. The target reference picture list in the reference picture list is indicated by the syntax element 1040A (eg, sh_collocated_from_10_flag). Thus, illegal bitstreams are avoided.
[0255] At step 1308B, the collocated image is determined to be the image referenced by the index whose value is equal to the parameter value in the target reference image list. The robustness of the collocated image is improved.
[0256] In step 1310B, the current image is decoded based on the collocated image, thereby improving the reliability of the decoding process.
[0257] Fig. 13C1 shows a portion of an exemplary semantics 1300C according to some embodiments of the present disclosure. The semantics 1300C may be used in both the method 1300A and the method 1300B. Fig. 13C As shown, in syntax 1310C, the changes compared to the previous VVC are shown in italics, and the proposed deleted syntax is further shown in strikethrough. Syntax 1310C corresponds to Fig.13A Step 1306A and Fig. 13B If pps_rpl_info_in_ph_flag (e.g., syntax element 830) is equal to 1, indicating that the reference picture list information exists in the PH syntax structure and does not exist in the SH that references the PPS and does not contain the PH syntax structure, it is inferred that the value of sh_collocated_ref_idx (e.g., syntax element 1050A) is equal to min(ph_collocated_ref_idx, NumRefIdxActive[!sh_collocated_from_l0_flag]-1), that is, the value of sh_collocated_ref_idx is set equal to the smaller value between the reference index value of the collocated picture used for temporal motion vector prediction in the picture header (e.g., ph_collocated_ref_idx) and the number of active entries in the target reference picture list minus 1 (e.g., NumRefIdxActive[!sh_collocated_from_l0_flag]–1). The target reference picture list is indicated by the syntax element 1040A (e.g., sh_collocated_from_l0_flag), which is the reference picture list from which the collocated picture for temporal motion vector prediction is derived. When the collocated picture for temporal MVP is derived from reference picture list 0, the target reference picture list is reference picture list 0. When the collocated picture for temporal MVP is derived from reference picture list 1, the target reference picture list is reference picture list 1.
[0258] In VVC (e.g., VVC draft 9), ref_pic_list_struct() can be signaled in SPS or included in the syntax structure ref_pic_lists(). When the ref_pic_list_structure() signaled in SPS is not selected from PH or SH, another ref_pic_list_structure() can be directly signaled in ref_pic_lists(), which is signaled in PH or SH. However, VVC (e.g., VVC draft 9) provides the following: For each value of listIdx (equal to 0 or 1), the decoder should allocate memory for a total number of ref_pic_list_struct(listIdx, rplsIdx) syntax structures equal to sps_num_ref_pic_lists[i] plus 1, because there may be a ref_pic_list_struct(listIdx, rplsIdx) syntax structure directly signaled in the slice header of the current picture. In view of the above, this is not accurate.
[0259] In order to overcome this defect of the traditional encoding technology, in some embodiments of the present disclosure (for example, Fig.14A and Fig. 14B In the embodiment provided, for each value of listIdx (equal to 0 or 1), for the case where a ref_pic_list_struct(listIdx, rplsIdx) syntax structure is signaled directly in the picture header or slice header of the current picture, the decoder allocates memory for a total number of ref_pic_list_struct(listIdx, rplsIdx) syntax structures equal to sps_num_ref_pic_lists[i] plus 1.
[0260] Fig.14A 1400A is a flowchart of an exemplary video processing method 1400A for allocating memory to a decoder according to some embodiments of the present disclosure. The method 1400A may be performed by an encoder (e.g., by Figure 2A Process 200A or Figure 2B Process 200B), a decoder (eg Figure 3A Process 300A or Figure 3B Process 300B) is performed, or by a device (e.g., Figure 4 For example, one or more processors (e.g., Figure 4The method 1400A may be performed by a processor 402 of the computer system. In some embodiments, the method 1400A may be implemented by a computer program product embodied in a computer-readable medium including a program executed by a computer (e.g., Figure 4 Computer executable instructions, such as program codes, executed by the device 400. Fig.14A , method 1400A may include the following steps 1402A to 1406A.
[0261] At step 1402A, the total number is derived by adding the number of reference picture list structures in the sequence parameter set (SPS) to 1. Since there is the possibility of additional RPLs being signaled later (in a picture header or slice header), the number of reference picture list structures in the SPS is incremented by an additional value of 1 to obtain the total number.
[0262] In step 1404A, in response to signaling the reference picture list structure in the picture header of the current picture or the slice header of the current slice, memory is allocated for the total number of reference picture list structures. Therefore, allocating more memory for the additional RPLs signaled in the picture header of the current picture or the slice header of the current picture by the encoder / decoder before encoding / decoding will help video processing.
[0263] At step 1406A, the current image or current slice is processed using the allocated memory. Since the allocated memory is more reliable for the additional RPL, the encoding / decoding process can be more accurate and robust.
[0264] Fig. 14B Portions of exemplary semantics 1400B according to some embodiments of the present disclosure are shown. Semantics 1400B may be used for method 1400A, with changes compared to previous VVC shown in italics (see box 1410B). For the possibility of subsequently signaling additional RPLs (in a picture header or slice header), more memory is allocated for the additional RPLs.
[0265] In VVC (e.g., VVC draft 9), the syntax element 530A (e.g., rpl_idx[i]) specifies the index of the ref_pic_list_struct(listIdx, rplsIdx) syntax structure (listIdx equal to i) used to derive the reference picture list i of the current picture in the list of ref_pic_list_struct(listIdx, rplsIdx) syntax structures with listIdx equal to i included in the SPS. Since the reference picture list can be derived from a picture or a slice, this semantics may not be accurate.
[0266] In VVC (e.g., VVC draft 9), when syntax element 530A is not present, there is an inference rule for inferring the value of syntax element 530A: if syntax element 510A (e.g., rpl_sps_flag[i]) is equal to 1 and syntax element 520A (e.g., pps_rpl1_idx_present_flag) is equal to 0, then the value of rpl_idx[1] is inferred to be equal to rpl_idx[0], otherwise the value of rpl_idx[1] is inferred to be equal to 0. There are some problems with this inference rule. First, there is only an inference rule for rpl_idx[1], and there is no inference rule for rpl_idx[0]. Second, when syntax element 510A is equal to 1 and syntax element 520A is equal to 0, it is not guaranteed that rpl_idx[0] is signaled. Therefore, in this case, inferring the value of rpl_idx[1] to be equal to rpl_idx[0] may be problematic. In summary, the inference rules of VVC (such as VVC draft 9) cannot guarantee that rpl_idx[0] and rpl_idx[1] both get appropriate values at the decoder side when they do not exist.
[0267] In order to overcome this defect of the traditional encoding technology, in some embodiments of the present disclosure (for example, FIG. 15A to FIG. 15C In the embodiment provided in ), updated semantics for syntax element 530A (e.g., rpl_idx[i]) are provided.
[0268] Fig.15A 1 is a flowchart of an exemplary video encoding method 1500A for determining an index in a reference picture list according to some embodiments of the present disclosure. The method 1500A may be performed by an encoder (e.g., Figure 2A Process 200A or Figure 2B 200B), or by a device (e.g., Figure 4 For example, one or more processors (e.g., Figure 4 The method 1500A may be performed by a processor 402 of the computer system. In some embodiments, the method 1500A may be implemented by a computer program product embodied in a computer-readable medium including a program executed by a computer (e.g., Figure 4 Computer executable instructions, such as program codes, executed by the device 400. Fig.15A , method 1500A may include the following steps 1502A to 1514A.
[0269] In step 1502A, a first flag (e.g., pps_rpl1_idx_present_flag) is signaled in a picture parameter set (PPS) to indicate whether a second flag (e.g., rpl_sps_flag[1]) and a first index (e.g., rpl_idx[1]) are present in a picture header syntax or a slice header syntax of a current picture that references the PPS. The first flag (e.g., pps_rpl1_idx_present_flag) indicates whether reference picture list 1 is derived based on one of the reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index (e.g., rpl_idx[1]) is an index of the reference picture list structure associated with reference picture list 1 in a list of reference picture list structures associated with reference picture list 1 included in the SPS for deriving reference picture list 1. Then, a second flag (rpl_sps_flag[1]) may be signaled.
[0270] At step 1504A, it is determined whether to signal a first index (e.g., rpl_idx[1]) and a second index (e.g., rpl_idx[0]). The second index is an index in a list of reference picture list structures associated with reference picture list 0 included in the SPS for deriving a reference picture list 0 associated with reference picture list 0.
[0271] When the second index (eg, rpl_idx[0]) is not signaled, the value of the second index (eg, rpl_idx[0]) may be determined through step 1506A.
[0272] At step 1506A, when the SPS includes at most one reference picture list structure associated with reference picture list 0, the value of the second index (eg, rpl_idx[0]) is determined to be equal to 0. Figure 5A , when sps_num_ref_pic_lists[0] is less than or equal to 1, rpl_idx[0] is not signaled. Therefore, through step 1506A, the value of rpl_idx[0] is determined without signaling rpl_idx[0], which enhances the reliability of inferring rpl_idx[0].
[0273] When the first index (eg, rpl_idx[1]) is not signaled, the value of the first index (eg, rpl_idx[1]) may be determined through steps 1508A and 1510A.
[0274] At step 1508A, when the SPS includes at most one reference picture list structure associated with reference picture list 1, the value of the first index (eg, rpl_idx[1]) is determined to be equal to 0. Figure 5A , when sps_num_ref_pic_lists[1] is less than or equal to 1, rpl_idx[1] is not signaled. Therefore, through step 1508A, the value of rpl_idx[1] is determined without signaling rpl_idx[1], which enhances the reliability of inferring rpl_idx[1].
[0275] At step 1510A, when the first flag (e.g., pps_rpl1_idx_present_flag) is equal to 0 and the second flag (e.g., rpl_sps_flag[1]) is equal to 1, it is determined that the value of the first index (e.g., rpl_idx[1]) is equal to the value of the second index (e.g., rpl_idx[0]). Since if sps_num_ref_pic_lists[0] is less than or equal to 1, the value of rpl_idx[0] is set to 0 (step 1508A), otherwise (e.g., sps_num_ref_pic_lists[0]>1) rpl_idx[0] (reference Figure 5A ), so the value of rpl_idx[0] is determined for all scenarios. Therefore, in this case, it is determined that the value of rpl_idx[1] is set equal to the value of rpl_idx[0]. Therefore, for all scenarios (e.g., whether rpl_idx[0] is signaled or not), the value of rpl_idx[1] is determined. If rpl_idx[i] is not signaled, it is guaranteed that rpl_idx[i] (rpl_idx[0] and rpl_idx[1]) will get the appropriate value.
[0276] After determining the values of the first index (e.g., rpl_idx[1]) and the second index (e.g., rpl_idx[0]), in step 1512A, a reference picture list is determined based on the first index and the second index. Since the values of the first index (e.g., rpl_idx[1]) and the second index (e.g., rpl_idx[0]) are guaranteed to be determined regardless of whether the first index (e.g., rpl_idx[1]) and the second index (e.g., rpl_idx[0]) are signaled, the determination of the reference picture list can be more reliable.
[0277] At step 1514A, the current picture is encoded based on the reference picture list. Thus, the robustness of the encoding process is improved.
[0278] In some embodiments, since rpl_idx[0] is inferred to be equal to 0 when a reference picture list structure of reference picture list 0 exists in the SPS (refer to step 1508A), step 1510A can be replaced by "in response to the existence of a reference picture list structure of reference picture list i in the SPS, determine that rpl_idx[i] is equal to 0". The efficiency of the encoding process can be further improved.
[0279] Fig. 15B 1 is a flowchart of an exemplary video decoding method 1500B for determining an index of a reference picture list according to some embodiments of the present disclosure. The method 1500B may be performed by a decoder (e.g., Figure 3A Process 300A or Figure 3B 300B), or by a device (e.g., Figure 4 For example, one or more processors (e.g., Figure 4 The method 1500B may be performed by a processor 402 of the computer system. In some embodiments, the method 1500B may be implemented by a computer program product embodied in a computer-readable medium including a program executed by a computer (e.g., Figure 4 Computer executable instructions, such as program codes, executed by the device 400. Fig. 15B , method 1500B may include the following steps 1502B to 1514B.
[0280] At step 1502B, the decoder receives a video bitstream (eg, Figure 3B The video bitstream 228 in the video bitstream may be encoded with inter-frame prediction. The reference pictures may be derived, for example, by reference picture list 0 and reference picture list 1, each of which includes a DPB (e.g., Figure 3B A list of reconstructed images in the buffer 234 in ( ) to be used as reference images.
[0281] In step 1504B, a value of a first flag (e.g., pps_rpl1_idx_present_flag) is determined, the first flag indicating whether a second flag (e.g., rpl_sps_flag[1]) and a first index (e.g., rpl_idx[1]) are present in a picture header syntax or a slice header syntax of a current picture. The second flag indicates whether reference picture list 1 is derived based on one of the reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index is an index of the reference picture list structure associated with reference picture list 1 in a list of reference picture list structures associated with reference picture list 1 included in the SPS for deriving reference picture list 1. Then, a value of the second flag (e.g., rpl_sps_flag[1]) may be determined.
[0282] In step 1506B, it is determined whether a first index (e.g., rpl_idx[1]) and a second index (e.g., rpl_idx[0]) exist. The second index is an index in a list of reference picture list structures associated with reference picture list 0 included in the SPS for deriving a reference picture list structure associated with reference picture list 0 for deriving reference picture list 0.
[0283] When the second index (eg, rpl_idx[0]) does not exist, the value of the second index (eg, rpl_idx[0]) may be determined through step 1508B.
[0284] At step 1508B, when the SPS includes at most one reference picture list structure associated with reference picture list 0, the value of the second index (eg, rpl_idx[0]) is determined to be equal to 0. Figure 5A , when sps_num_ref_pic_lists[0] is less than or equal to 1, rpl_idx[0] is not signaled, and therefore rpl_idx[0] does not exist. In this case, rpl_idx[0] is set equal to 0. Therefore, through step 1508B, the value of rpl_idx[0] is determined when rpl_idx[0] does not exist, which enhances the reliability of inferring rpl_idx[0].
[0285] When the first index (eg, rpl_idx[1]) does not exist, the value of the first index (eg, rpl_idx[1]) may be determined through steps 1510B and 1512B.
[0286] At step 1510B, when the SPS includes at most one reference picture list structure associated with reference picture list 1, the value of the first index (eg, rpl_idx[1]) is determined to be equal to 0. Figure 5A, when sps_num_ref_pic_lists[1] is less than or equal to 1, rpl_idx[1] is not signaled, and therefore rpl_idx[0] does not exist. Therefore, through step 1510B, the value of rpl_idx[1] is determined without signaling rpl_idx[1], which enhances the reliability of inferring rpl_idx[1].
[0287] At step 1512B, when the first flag (e.g., pps_rpl1_idx_present_flag) is equal to 0 and the second flag (e.g., rpl_sps_flag[1]) is equal to 1, it is determined that the value of the first index (e.g., rpl_idx[1]) is equal to the value of the second index (rpl_idx[0]). Since if sps_num_ref_pic_lists[0] is less than or equal to 1, the value of rpl_idx[0] is set to 0 (step 1508A), otherwise (e.g., sps_num_ref_pic_lists[0]>1) rpl_idx[0] (reference Figure 5A ), so the value of rpl_idx[0] is determined for all scenarios. Therefore, in this case, it is determined that the value of rpl_idx[1] is set equal to the value of rpl_idx[0]. Therefore, for all scenarios (e.g., whether or not rpl_idx[0] exists), the value of rpl_idx[1] is determined. If rpl_idx[i] does not exist, it is guaranteed that rpl_idx[i] (rpl_idx[0] and rpl_idx[1]) will get the appropriate value.
[0288] At 1514B, the current picture is decoded based on the first index (e.g., rpl_idx[1]) and the second index (e.g., rpl_idx[0]). Since the values of the first index (e.g., rpl_idx[1]) and the second index (e.g., rpl_idx[0]) are guaranteed to be determined regardless of whether the first index (e.g., rpl_idx[1]) and the second index (e.g., rpl_idx[0]) exist, the determination of the reference picture list can be more reliable.
[0289] In some embodiments, since rpl_idx[0] is inferred to be equal to 0 when a reference picture list structure of reference picture list 0 exists in the SPS (refer to step 1508B), step 1514B can be replaced by "in response to the presence of a reference picture list structure of reference picture list i in the SPS, inferring rpl_idx[i] is equal to 0". The efficiency of the decoding process can be further improved.
[0290] Fig. 15C1 shows a portion of an exemplary semantics 1500C according to some embodiments of the present disclosure. The semantics 1500C may be used in both the method 1500A and the method 1500B. Fig. 15C As shown, the changes compared to the previous VVC are shown in italics, and the proposed deleted syntax is further shown in strikethrough (see boxes 1510C and 1520C). Two alternative derivation descriptions are provided. In some embodiments, as shown in box 1510C, for the case where rpl_idx[i] does not exist, if there is at most one reference picture list structure for reference picture list i (e.g., sps_num_ref_pic_list[i] is less than or equal to 1), then the value of rpl_idx[i] is inferred to be equal to 0; otherwise (there is more than one reference picture list structure for reference picture i, that is, sps_num_ref_pic_list[i] is greater than 1), and i is equal to 1, that is, sps_num_ref_pic_list[1] is greater than 1, then the value of rpl_idx[1] is inferred to be equal to rpl_idx[0]. The difference between boxes 1520C and 1510C is that the expression "otherwise" and "i is equal to 1" is explained in detail as "sps_num_ref_pic_list[1] is greater than 1". In some embodiments, the condition “if sps_num_ref_pic_list[i] is less than or equal to 1” (see blocks 1511C and 1521C) may be replaced with “if sps_num_ref_pic_list[i] is equal to 1”.
[0291] In VVC (e.g., VVC draft 9), the syntax element 1010A (e.g., sh_num_ref_idx_active_override_flag) equal to 1 specifies that the syntax element sh_num_ref_idx_active_minus1[0] is present in the case of slices P and B, and the syntax element sh_num_ref_idx_active_minus1[1] is present in the case of slice B. The syntax element 1010A equal to 0 specifies that the syntax elements sh_num_ref_idx_active_minus1[0] and sh_num_ref_idx_active_minus1[1] are not present. However, if Fig. 10AAs shown, when the syntax element 1010A is equal to 1, the value of num_ref_entries[i][RplsIdx[i]] is further checked for signaling sh_num_ref_idx_active_minus1[i]. The syntax element sh_num_ref_idx_active_minus1[i] is signaled only when the syntax element 1010A is equal to 1 and num_ref_entries[i][RplsIdx[i]] is greater than 1. Therefore, the syntax element 1010A being equal to 1 does not necessarily mean that sh_num_ref_idx_active_minus1[i] is signaled.
[0292] In order to overcome this defect in the traditional coding technology, in some embodiments of the present disclosure (for example, FIG. 16A to FIG. 16C In the provided embodiment), updated semantics of the syntactic element 1010A are provided to improve the efficiency of the encoding / decoding process.
[0293] Fig.16A 1 shows a flowchart of an exemplary video encoding method 1600A for indicating the number of activation reference indexes present in a slice header according to some embodiments of the present disclosure. The method 1600A may be performed by an encoder (e.g., Figure 2A Process 200A or Figure 2B 200B), or by a device (e.g., Figure 4 For example, one or more processors (e.g., Figure 4 The method 1600A may be performed by a processor 402 of the computer system. In some embodiments, the method 1600A may be implemented by a computer program product embodied in a computer-readable medium including a program executed by a computer (e.g., Figure 4 Computer executable instructions, such as program codes, executed by the device 400. Fig.16A , method 1600A may include the following steps 1602A to 1608A.
[0294] In step 1602, a first flag is signaled in a slice header to indicate whether there is an active reference index number in the slice header. For example, a syntax element sh_num_ref_idx_active_override_flag is signaled to indicate whether there is an active reference index number (e.g., sh_num_ref_idx_active_minus1[i]) of a reference picture list i in the slice header (i is equal to 0 or 1). The active reference index number is used to derive the maximum reference index of the corresponding reference picture list that may be used to encode the current slice. The number of reference indexes used to encode the current slice may be less than or equal to the maximum number derived from the active reference index number.
[0295] In step 1604A, it is determined whether an active reference index number exists. When the first flag indicates that an active reference index number exists, the syntax element sh_num_ref_idx_active_minus1[0] exists in the case of P and B slices, and the syntax element sh_num_ref_idx_active_minus1[1] exists in the case of B slices. Then, steps 1606A and 1608A are performed.
[0296] In step 1606A, the number of entries in reference picture list 0 is first determined. If the number of entries in reference picture list 0 (e.g., num_ref_entries[0][RplsIdx[0]]) is determined to be greater than 1, the number of active reference indexes of reference picture list 0 (e.g., sh_num_ref_idx_active_minus1[0]) is signaled in the slice header in the case of P and B slices.
[0297] In step 1608A, the number of entries in reference picture list 1 is first determined. If the number of entries in reference picture list 1 (e.g., num_ref_entries[1][RplsIdx[1]]) is determined to be greater than 1, the number of activated reference indexes of reference picture list 1 (e.g., sh_num_ref_idx_active_minus1[1]) is signaled in the slice header in the case of a B slice.
[0298] Through steps 1606A and 1608A, when the number of entries in reference picture list i (e.g., num_ref_entries[i][RplsIdx[i]]) is greater than 1, the number of active reference indexes of reference picture list i (e.g., sh_num_ref_idx_active_minus1[i]) is signaled at the slice level.
[0299] Therefore, the ambiguity of signaling sh_num_ref_idx_active_minus1[i] when sh_num_ref_idx_active_override_flag is equal to 1 is eliminated, which can improve the accuracy and robustness of the encoding process.
[0300] In some embodiments, the method 1600A may further include steps 1610A and 1612A. When the first flag indicates that the number of active reference indices does not exist, the syntax element sh_num_ref_idx_active_minus1[i] does not exist. Then, steps 1610A and 1612A are performed.
[0301] In step 1610A, signaling the active reference index number (eg, sh_num_ref_idx_active_minus1[0]) of reference picture list 0 in the slice header in case of P and B slices is skipped. That is, sh_num_ref_idx_active_minus1[0] is not signaled in the slice header in case of P and B slices.
[0302] In step 1612A, signaling the number of active reference indices of reference picture list 1 in the slice header in case of a B slice is skipped. That is, sh_num_ref_idx_active_minus1[1] is not signaled in the slice header in case of a B slice.
[0303] Therefore, the encoding process is made more efficient by skipping signaling the activation reference index number when it does not exist.
[0304] Fig. 16B 1 shows a flowchart of an exemplary video decoding method 1600B for indicating an activation reference index number in a slice header according to some embodiments of the present disclosure. The method 1600B may be performed by a decoder (e.g., Figure 3A Process 300A or Figure 3B 300B), or by a device (e.g., Figure 4 For example, one or more processors (e.g., Figure 4 The processor 402 of the embodiment of the present invention may perform the method 1600B. In some embodiments, the method 1600B may be implemented by a computer program product, which is embodied in a computer-readable medium, and the computer-readable medium includes a computer (e.g., Figure 4 Computer executable instructions, such as program codes, executed by the device 400. Fig. 16B As shown, method 1600B may include the following steps 1602B to 1608B.
[0305] At step 1602B, the decoder receives a video bitstream (eg, Figure 3B The video bitstream 228 includes a slice header and a picture header syntax, and the video bitstream can be encoded using inter-frame prediction. The reference picture can be derived, for example, by reference picture list 0 and reference picture list 1, each of which includes a DPB (e.g., Figure 3B A list of reconstructed images in the buffer 234 in ( ) to be used as reference images.
[0306] In step 1604B, a value of a first flag signaled in a slice header is determined, the first flag indicating whether there is an active reference index number. In some embodiments, the first flag is a syntax element h_num_ref_idx_active_override_flag, which may indicate whether there is an active reference index number (e.g., sh_num_ref_idx_active_minus1[i]) for reference picture list i (i is equal to 0 or 1). The active reference index number is used to derive the maximum reference index of the corresponding reference picture list that may be used to decode the current slice. The number of reference indexes used to decode the current slice may be less than or equal to the maximum number derived from the active reference index number.
[0307] When it is determined that the value of the first flag is a value indicating the presence of the active reference index number, the syntax element sh_num_ref_idx_active_minus1[0] exists in the case of P and B slices, and the syntax element sh_num_ref_idx_active_minus1[1] exists in the case of B slices. Then, step 1606B and step 1608B are performed.
[0308] In step 1606B, the number of entries in reference picture list 0 is determined (e.g., num_ref_entries[0][RplsIdx[0]]), and if the number of entries in reference picture list 0 is determined to be greater than 1, then in the case of P and B slices, the number of active reference indexes of reference picture list 0 in the decoded slice header is (e.g., sh_num_ref_idx_active_minus1[0]).
[0309] In step 1608B, the number of entries in reference picture list 1 is determined (e.g., num_ref_entries[1][RplsIdx[1]]), and if the number of entries in reference picture list 1 is determined to be greater than 1, then in the case of a B slice, the number of active reference indexes of reference picture list 1 in the decoded slice header is (e.g., sh_num_ref_idx_active_minus1[1]).
[0310] Through steps 1606B and 1608B, when the number of entries in reference picture list i (e.g., num_ref_entries[i][RplsIdx[i]]) is greater than 1, the number of active reference indexes of reference picture list i (e.g., sh_num_ref_idx_active_minus1[i]) is signaled. Therefore, the ambiguity of signaling sh_num_ref_idx_active_minus1[i] when sh_num_ref_idx_active_override_flag is equal to 1 is eliminated.
[0311] In some embodiments, the method 1600B may further include steps 1610B and 1612B. When the value of the first flag is determined to indicate that there is no value of the number of active reference indices, the syntax element sh_num_ref_idx_active_minus1[i] is not signaled. Then, steps 1610B and 1612B are performed.
[0312] In step 1610B, the active reference index number (eg, sh_num_ref_idx_active_minus1[0]) of the reference picture list 0 in the slice header is skipped for decoding in the case of P and B slices. That is, sh_num_ref_idx_active_minus1[0] does not exist in the slice header in the case of B slices.
[0313] In step 1612B, the active reference index number (eg, sh_num_ref_idx_active_minus1[1]) of the reference picture list 1 in the slice header is skipped for decoding in the case of a B slice. That is, sh_num_ref_idx_active_minus1[1] does not exist in the case of a B slice. Therefore, the efficiency of the decoding process can be improved.
[0314] Fig. 16C 1 shows a portion of an exemplary semantics 1600C according to some embodiments of the present disclosure. The semantics 1600C may be used in both the method 1600A and the method 1600B. Fig. 16CAs shown, the changes compared to the previous VVC are shown in italics, and the proposed deleted syntax is further shown in strikethrough (see blocks 1610C and 1620C). Two alternative descriptions are provided. As shown in block 1610C, sh_num_ref_idx_active_override_flag equal to 1 does not necessarily specify the presence of the syntax element sh_num_ref_idx_active_minus1[0] in the case of P and B slices, or the presence of the syntax element sh_num_ref_idx_active_minus1[1] in the case of B slices. As shown in block 1620C, for sh_num_ref_idx_active_override_flag equal to 1, the syntax element sh_num_ref_idx_active_minus1[0] is present for the case of P and B slices, a condition of "when num_ref_entries[0][RplsIdx[0] is greater than 1" is added, and the syntax element sh_num_ref_idx_active_minus1[1] is present for the case of B slices, a condition of "when num_ref_entries[1][RplsIdx] is greater than 1" is added. Thus, the accuracy and robustness of the decoding process may be improved.
[0315] In VVC (e.g., VVC draft 9), there is a bitstream consistency constraint that the picture referenced by the syntax element 1050A (e.g., sh_collocated_ref_idx) is the same for all slices of the coded picture, and RprConstraintsActive[sh_collocated_from_l0_flag? 0:1][sh_collocated_ref_idx] is equal to 0. In order to identify the picture referenced by the syntax element 1050A, it is necessary to first determine the values of the syntax element 1040A (e.g., sh_collocated_from_l0_flag) and the syntax element 1050A (e.g., sh_collocated_ref_idx). However, as Fig. 10A As shown, syntax element 1040A is signaled only for B slices, and syntax element 1050A is signaled only for P and B slices. For I slices, syntax element 1040A and syntax element 1050A are not signaled. And for I slices, there are no inferred values for these two syntax elements. Therefore, for I slices, the value of syntax element 1050A is undefined. Therefore, the encoder / decoder cannot identify the picture referenced by syntax element 1050A and cannot perform consistency constraint checking.
[0316] In order to overcome this defect of the traditional encoding technology, in some embodiments of the present disclosure (for example, Fig.17A and Fig. 17B In the provided embodiment), updated semantics are provided to improve the accuracy and robustness of video processing.
[0317] For example, Fig.17A 1 shows a flowchart of an exemplary video processing method 1700A for image processing. The method 1700A may be performed by an encoder (e.g., by Figure 2A Process 200A or Figure 2B Process 200B), a decoder (eg Figure 3A Process 300A or Figure 3B Process 300B) is performed, or by a device (e.g., Figure 4 For example, one or more processors (e.g., Figure 4 The method 1700A may be performed by a processor 402 of the computer system. In some embodiments, the method 1700A may be implemented by a computer program product embodied in a computer-readable medium including a program executed by a computer (e.g., Figure 4 Computer executable instructions, such as program codes, executed by the device 400. Fig.17A , method 1700A may include steps 1702A and 1704A.
[0318] In step 1702A, the collocated picture referenced by the reference index (e.g., sh_collocated_ref_idx) of the collocated picture at the slice level is determined, wherein the collocated picture is determined to be the same picture for all non-I slices of the current picture. Thus, the ambiguity of the sh_collocated_ref_idx and sh_collocated_from_l0_flag values is avoided.
[0319] At step 1704A, the current image is processed based on the collocated image, wherein the collocated image is used for temporal motion vector prediction. Therefore, the robustness of video processing can be improved.
[0320] That is, for all non-I slices of the coded picture, the picture used for temporal motion vector prediction referenced by the reference index of the collocated picture is determined to be the same. In some embodiments, for all P slices and B slices of the current picture, the picture used for temporal motion vector prediction referenced by the reference index of the collocated picture is determined to be the same.
[0321] Fig. 17BFIG. 1 shows a portion of an exemplary semantics 1700B according to some embodiments of the present disclosure. The semantics 1700B may be used in method 1700A. Fig. 17B As shown, the changes compared to the previous VVC are shown in italics, and the proposed deleted syntax is further shown in strikethrough (see boxes 1710B and 1720B). Two alternative descriptions are provided. As shown in box 1710B, the requirement for bitstream consistency is further refined to "all non-I slices" instead of "all slices". Therefore, the efficiency and robustness of the decoding process are improved. The difference between box 1720B and 1710B is that the expression of "non-I slices" is replaced with "P slices and B slices" to make it more accurate.
[0322] In some embodiments, a non-transitory computer-readable storage medium including instructions is also provided, and the instructions can be executed by a device (e.g., the disclosed encoder and decoder) to perform the above method. Common forms of non-transitory media include, for example, floppy disks, foldable disks, hard disks, solid-state drives, tapes or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with a hole pattern, RAM, PROM and EPROM, FLASH-EPROM or any other flash memory, NVRAM, cache, registers, any other storage chip or box and network versions thereof. The device may include one or more processors (CPU), input / output interfaces, network interfaces and / or memories.
[0323] It should be noted that relational terms in this document, such as "first" and "second", etc., are only used to distinguish one entity or operation from another entity or operation, and do not require or imply any actual relationship or order between these entities or operations. In addition, the words "include", "have", "include" and "includes" and other similar forms are intended to have the same meaning and are open-ended, in that one or more items following any of these words are not intended to be an exhaustive list of the one or more items, or to be limited to the one or more items listed.
[0324] As used herein, unless specifically stated otherwise, the term "or" includes all possible combinations unless not feasible. For example, if it is stated that a database may include A or B, then unless otherwise explicitly stated or not feasible, the database may include A, or B, or A and B. As a second example, if it is stated that a database may include A, B, or C, then unless otherwise explicitly stated or not feasible, the database may include A, or B, or C, or A and B, or A and C, or B and C, or A, B, and C.
[0325] It is understood that the above embodiments can be implemented by hardware or software (program code) or a combination of hardware and software. If implemented by software, it can be stored in the above-mentioned computer-readable medium. When the software is executed by a processor, the disclosed method can be executed. The computing unit and other functional units described in this disclosure can be implemented by hardware, software or a combination of hardware and software. It will also be understood by those of ordinary skill in the art that multiple of the above modules / units can be combined into one module / unit, and each of the above modules / units can be further divided into multiple sub-modules / sub-units.
[0326] The embodiments may be further described using the following terms:
[0327] 1. A computer-implemented method for encoding a video, comprising:
[0328] encoding the current picture based on the collocated picture, wherein the collocated picture is used for temporal motion vector prediction; and
[0329] In response to the number of entries in reference image list 0 and the number of entries in reference image list 1 being greater than 0, a first flag and a second flag are sent by signal, wherein the first flag indicates that the collocated image is derived from reference image list 0 or reference image list 1, and the second flag indicates whether a motion vector difference syntax structure is sent by signal.
[0330] 2. A computer-implemented method for decoding a video, comprising:
[0331] receiving a video bitstream;
[0332] In response to the number of entries in reference picture list 0 and the number of entries in reference picture list 1 being greater than 0, decoding a first flag and a second flag, wherein the first flag indicates that the collocated picture used for temporal motion vector prediction is derived from reference picture list 0 or reference picture list 1, and the second flag indicates whether a motion vector difference syntax structure is present in a bitstream of a current picture; and
[0333] Decode the current image based on the concatenated image.
[0334] 3. A computer-implemented method for encoding a video, comprising:
[0335] encoding the current picture based on the collocated picture, wherein the collocated picture is used for temporal motion vector prediction; and
[0336] Indicates the collocated picture in the bitstream without signaling the index into the reference picture list.
[0337] 4. The method of clause 3, wherein indicating a collocated picture in the bitstream without signaling an index in a reference picture list further comprises:
[0338] signaling a first flag to indicate whether the collocated picture is an inter-layer reference picture; and
[0339] In response to the collocated picture being an inter-layer reference picture, a first parameter is signaled to indicate the collocated picture, wherein the first parameter indicates an index of the collocated picture in a list of direct reference layers of a layer where the current picture is located.
[0340] 5. The method of clause 4, wherein indicating the collocated picture in the bitstream without signaling an index in a reference picture list further comprises:
[0341] signaling a second flag to indicate whether the collocated picture is a short-term reference picture or a long-term reference picture; and
[0342] In response to the collocated picture being the short-term reference picture, a second parameter is signaled to indicate the collocated picture, wherein the second parameter indicates a difference between a picture order count of the collocated picture and a picture order count of the current picture.
[0343] 6. The method according to clause 5, further comprising:
[0344] In response to the collocated image being a long-term reference image, a third parameter and a fourth parameter are signaled to indicate the collocated image, wherein the third parameter indicates a least significant bit (LSB) of a picture order count (POC) of the collocated image, and the fourth parameter indicates an incremental most significant bit (MSB) of the picture order count (POC) of the collocated image.
[0345] 7. The method of clause 6, wherein the first flag, the second flag, the first parameter, the second parameter, the third parameter and the fourth parameter are signaled in a picture header and all slices in the picture have the same collocated picture.
[0346] 8. A method according to clause 3, wherein the reference picture list is reference picture list 0 or reference picture list 1.
[0347] 9. A computer-implemented method for decoding a video, comprising:
[0348] receiving a video bitstream;
[0349] determining a collocated picture for temporal motion vector prediction without decoding an index into a reference picture list; and
[0350] Decode the current image based on the concatenated image.
[0351] 10. The method of clause 9, wherein determining the collocated picture for temporal motion vector prediction without decoding the index in the reference picture list further comprises:
[0352] decoding a first flag indicating whether a collocated picture is an inter-layer reference picture;
[0353] determining whether the collocated picture is an inter-layer reference picture based on the first flag; and
[0354] In response to the collocated picture being an inter-layer reference picture, a first parameter is decoded and the collocated picture is determined based on the first parameter, wherein the first parameter indicates an index of the collocated picture in a list of direct reference layers of a layer where the current picture is located.
[0355] 11. The method of clause 10, wherein determining the collocated picture for temporal motion vector prediction without decoding the index in the reference picture list further comprises:
[0356] decoding a second flag indicating whether the collocated picture is a short-term reference picture or a long-term reference picture;
[0357] determining whether the collocated image is a short-term reference image or a long-term reference image based on a second flag; and
[0358] In response to the collocated picture being a short-term reference picture, a second parameter is decoded and the collocated picture is determined based on the second parameter, wherein the second parameter indicates a difference between a picture order count of the collocated picture and a picture order count of the current picture.
[0359] 12. The method according to clause 11, further comprising:
[0360] In response to the collocated image being a long-term reference image, a third parameter and a fourth parameter are decoded, and the collocated image is determined based on the third parameter and the fourth parameter, wherein the third parameter indicates a least significant bit (LSB) of a picture order count (POC) of the collocated image, and the fourth parameter indicates an incremental most significant bit (MSB) of the picture order count (POC) of the collocated image.
[0361] 13. The method of clause 12, wherein the first flag, the second flag, the first parameter, the second parameter, the third parameter and the fourth parameter are present in an image header and all stripes in the image have the same concatenated image.
[0362] 14. The method of clause 9, wherein the reference picture list is reference picture list 0 or reference picture list 1.
[0363] 15. A computer-implemented method for encoding a video, comprising:
[0364] determining whether to signal a parameter in a slice header to indicate a reference index of a collocated picture;
[0365] In response to the parameter not being signaled in the slice header, determining the collocated picture to be a picture referenced by an index whose value is equal to the smaller of a value of a reference index of the collocated picture signaled in the picture header and a number of active entries in the target reference picture list minus one; and
[0366] The current picture is encoded based on the collocated picture, wherein the collocated picture is used for temporal motion vector prediction.
[0367] 16. A method according to clause 15, wherein the target reference picture list is indicated by a flag indicating from which reference picture list the collocated picture used for temporal motion vector prediction is derived.
[0368] 17. A computer-implemented method for decoding a video, comprising:
[0369] receiving a video bitstream;
[0370] determining whether there is a parameter in a slice header indicating a reference index of a collocated picture for temporal motion vector prediction;
[0371] In response to the parameter not being present, determining the value of the parameter to be equal to the smaller of the value of the reference index of the collocated picture for temporal motion vector prediction present in the picture header and the number of active entries in the target reference picture list minus 1;
[0372] Determining the concatenated picture is the picture referenced by the index whose value is equal to the parameter value in the target reference picture list; and
[0373] Decode the current image based on the concatenated image.
[0374] 18. A method according to clause 17, wherein the target reference picture list is indicated by a flag indicating from which reference picture list the collocated picture used for temporal motion vector prediction is derived.
[0375] 19. A computer-implemented method for video processing, comprising:
[0376] The total number is derived by adding the number of reference picture list structures in the sequence parameter set (SPS) to 1;
[0377] In response to signaling the reference picture list structures in a picture header of the current picture or a slice header of the current slice, allocating memory for the total number of reference picture list structures; and
[0378] Processes the current image or current strip using the allocated memory.
[0379] 20. A computer-implemented method for encoding a video, comprising:
[0380] a first flag is signaled in a picture parameter set (PPS) to indicate whether a second flag and a first index are present in a picture header syntax or a slice header of a current picture referencing the PPS; wherein the second flag indicates whether reference picture list 1 is derived based on one of the reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index is an index in a list of reference picture list structures associated with reference picture list 1 included in the SPS for deriving reference picture list 1;
[0381] determining whether to signal a first index and a second index, wherein the second index is an index of a reference picture list structure associated with reference picture list 0 for deriving reference picture list 0 in a list of reference picture list structures associated with reference picture list 0 included in the SPS;
[0382] In response to not signaling the second index, determining a value of the second index includes:
[0383] When the SPS includes at most one reference picture list structure associated with reference picture list 0, determining that the value of the second index is equal to 0;
[0384] In response to not signaling the first index, determining a value of the first index includes:
[0385] When at most one reference picture list structure associated with reference picture list 1 is included in the SPS, determining the value of the first index to be equal to 0; and
[0386] When the first flag is equal to 0 and the second flag is equal to 1, determining that the value of the first index is equal to the value of the second index;
[0387] deriving a reference picture list based on the first index and the second index; and
[0388] The current picture is encoded based on the reference picture list.
[0389] 21. A computer-implemented method for decoding a video, comprising:
[0390] receiving a video bitstream;
[0391] determining a value of a first flag, the first flag indicating whether a second flag and a first index are present in a picture header syntax or a slice header of a current picture, wherein the second flag indicates whether reference picture list 1 is derived based on one of reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index is an index in a list of reference picture list structures associated with reference picture list 1 included in the SPS for deriving reference picture list 1;
[0392] determining whether a first index and a second index exist, wherein the second index is an index of a reference picture list structure associated with reference picture list 0 for deriving reference picture list 0 in a list of reference picture list structures associated with reference picture list 0 included in the SPS;
[0393] In response to the second index not existing, determining a value of the second index includes:
[0394] When the SPS includes at most one reference picture list structure associated with reference picture list 0, determining that the value of the second index is equal to 0;
[0395] In response to the first index not existing, determining a value of the first index includes:
[0396] When at most one reference picture list structure associated with reference picture list 1 is included in the SPS, determining the value of the first index to be equal to 0; and
[0397] When the first flag is equal to 0 and the second flag is equal to 1, determining that the value of the first index is equal to the value of the second index; and
[0398] The current image is decoded based on the first index and the second index.
[0399] 22. A computer-implemented method for encoding a video, comprising:
[0400] Signaling a first flag in a slice header to indicate whether there is an activated reference index number in the slice header, wherein the activated reference index number is used to derive a maximum reference index of a corresponding reference picture list that can be used to encode a current slice;
[0401] In response to the first flag indicating that there is an activation reference index number in the slice header,
[0402] determining the number of entries in reference picture list 0 and, if the number of entries in reference picture list 0 is greater than 1, signaling the number of activated reference indices of reference picture list 0 in a slice header in case of P and B slices; and
[0403] The number of entries in reference picture list 1 is determined, and when the number of entries in reference picture list 1 is greater than 1, the number of active reference indices of reference picture list 1 is signaled in the slice header in case of B slices.
[0404] 23. The method according to clause 22, further comprising:
[0405] In response to the first flag indicating that the activation reference index number is not present in the slice header,
[0406] skipping signaling the activated reference index number of reference picture list 0 in the slice header in case of P and B slices; and
[0407] Skip signaling the activated reference index number of reference picture list 1 in the slice header in case of B slices.
[0408] 24. A computer-implemented method for decoding a video, comprising:
[0409] receiving a video bitstream including slice header and picture header syntax;
[0410] determining a value of a first flag signaled in a slice header, the first flag indicating whether there is an activated reference index number in the slice header, wherein the activated reference index number is used to derive a maximum reference index of a corresponding reference picture list that can be used to decode a current slice;
[0411] In response to the first flag indicating the presence of an activation reference index number,
[0412] determining the number of entries in reference picture list 0 and, if the number of entries in reference picture list 0 is greater than 1, decoding the active reference index number of reference picture list 0 in the slice header in case of P and B slices; and
[0413] Determine the number of entries in reference picture list 1, and when the number of entries in reference picture list 1 is greater than 1, decode the active reference index number of reference picture list 1 in the slice header in case of B slices.
[0414] 25. The method according to clause 24, further comprising:
[0415] In response to the first flag indicating that there is no activation reference index number,
[0416] Skip decoding the activated reference index number of reference picture list 0 in the slice header in case of P and B slices; and
[0417] Skip decoding the activated reference index number of reference picture list 1 in the slice header in case of B slices.
[0418] 26. A computer-implemented method for video processing, comprising:
[0419] determining a collocated picture referenced by a reference index of the collocated picture at a slice level, wherein the collocated picture is determined to be the same picture for all non-I slices of the current picture; and
[0420] The current picture is processed based on a collocated picture, wherein the collocated picture is used for temporal motion vector prediction.
[0421] 27. A computer-implemented method for video processing, comprising:
[0422] determining a collocated picture referred to by a reference index of the collocated picture at a slice level, wherein the collocated picture is determined to be the same picture for all P slices and B slices of the current picture; and
[0423] The current picture is processed based on a collocated picture, wherein the collocated picture is used for temporal motion vector prediction.
[0424] 28. A device for processing video data, the device comprising:
[0425] a memory for storing instructions; and
[0426] One or more processors for executing instructions to cause the apparatus to:
[0427] encoding the current picture based on the collocated picture, wherein the collocated picture is used for temporal motion vector prediction; and
[0428] In response to the number of entries in reference image list 0 and the number of entries in reference image list 1 being greater than 0, a first flag and a second flag are sent by signal, wherein the first flag indicates that the collocated image is derived from reference image list 0 or reference image list 1, and the second flag indicates whether a motion vector difference syntax structure is sent by signal.
[0429] 29. A device for processing video data, the device comprising:
[0430] a memory for storing instructions; and
[0431] One or more processors for executing instructions to cause the apparatus to:
[0432] receiving a video bitstream;
[0433] In response to the number of entries in reference picture list 0 and the number of entries in reference picture list 1 being greater than 0, decoding a first flag and a second flag, wherein the first flag indicates that the collocated picture used for temporal motion vector prediction is derived from reference picture list 0 or reference picture list 1, and the second flag indicates whether a motion vector difference syntax structure is present in a bitstream of a current picture; and
[0434] Decode the current image based on the concatenated image.
[0435] 30. A device for processing video data, the device comprising:
[0436] a memory for storing instructions; and
[0437] One or more processors for executing instructions to cause the apparatus to:
[0438] encoding the current picture based on the collocated picture, wherein the collocated picture is used for temporal motion vector prediction; and
[0439] Indicates the collocated picture in the bitstream without signaling the index to the reference picture list.
[0440] 31. An apparatus according to clause 30, wherein the processor is further configured to execute instructions to cause the apparatus to perform:
[0441] signaling a first flag to indicate whether the collocated picture is an inter-layer reference picture; and
[0442] In response to the collocated picture being an inter-layer reference picture, a first parameter is signaled to indicate the collocated picture, wherein the first parameter indicates an index of the collocated picture in a list of direct reference layers of a layer where the current picture is located.
[0443] 32. An apparatus according to clause 31, wherein the processor is further configured to execute instructions to cause the apparatus to perform:
[0444] signaling a second flag to indicate whether the collocated picture is a short-term reference picture or a long-term reference picture; and
[0445] In response to the collocated picture being the short-term reference picture, a second parameter is signaled to indicate the collocated picture, wherein the second parameter indicates a difference between a picture order count of the collocated picture and a picture order count of the current picture.
[0446] 33. An apparatus according to clause 32, wherein the processor is further configured to execute instructions to cause the apparatus to perform:
[0447] In response to the collocated image being a long-term reference image, a third parameter and a fourth parameter are signaled to indicate the collocated image, wherein the third parameter indicates a least significant bit (LSB) of a picture order count (POC) of the collocated image, and the fourth parameter indicates an incremental most significant bit (MSB) of the picture order count (POC) of the collocated image.
[0448] 34. The apparatus of clause 33, wherein the first flag, the second flag, the first parameter, the second parameter, the third parameter and the fourth parameter are signaled in an image header and all stripes in the image have the same collocated image.
[0449] 35. An apparatus as described in clause 30, wherein the reference picture list is reference picture list 0 or reference picture list 1.
[0450] 36. A device for processing video data, the device comprising:
[0451] a memory for storing instructions; and
[0452] One or more processors for executing instructions to cause the apparatus to:
[0453] receiving a video bitstream;
[0454] determining a collocated picture for temporal motion vector prediction without decoding an index into a reference picture list; and
[0455] Decode the current image based on the concatenated image.
[0456] 37. An apparatus according to clause 36, wherein the processor is further configured to execute instructions to cause the apparatus to perform:
[0457] decoding a first flag indicating whether a collocated picture is an inter-layer reference picture;
[0458] determining whether the collocated picture is an inter-layer reference picture based on the first flag; and
[0459] In response to the collocated picture being an inter-layer reference picture, a first parameter is decoded and the collocated picture is determined based on the first parameter, wherein the first parameter indicates an index of the collocated picture in a list of direct reference layers of a layer where the current picture is located.
[0460] 38. An apparatus according to clause 37, wherein the processor is further configured to execute instructions to cause the apparatus to perform:
[0461] decoding a second flag indicating whether the collocated picture is a short-term reference picture or a long-term reference picture;
[0462] determining whether the collocated image is a short-term reference image or a long-term reference image based on a second flag; and
[0463] In response to the collocated picture being a short-term reference picture, a second parameter is decoded and the collocated picture is determined based on the second parameter, wherein the second parameter indicates a difference between a picture order count of the collocated picture and a picture order count of the current picture.
[0464] 39. An apparatus according to clause 38, wherein the processor is further configured to execute instructions to cause the apparatus to perform:
[0465] In response to the collocated image being a long-term reference image, a third parameter and a fourth parameter are decoded, and the collocated image is determined based on the third parameter and the fourth parameter, wherein the third parameter indicates a least significant bit (LSB) of a picture order count (POC) of the collocated image, and the fourth parameter indicates an incremental most significant bit (MSB) of the picture order count (POC) of the collocated image.
[0466] 40. The apparatus of clause 39, wherein the first flag, the second flag, the first parameter, the second parameter, the third parameter, and the fourth parameter are present in an image header, and all stripes in the image have the same concatenated image.
[0467] 41. An apparatus as described in clause 36, wherein the reference picture list is reference picture list 0 or reference picture list 1.
[0468] 42. A device for processing video data, the device comprising:
[0469] a memory for storing instructions; and
[0470] One or more processors for executing instructions to cause the apparatus to:
[0471] determining whether to signal a parameter in a slice header to indicate a reference index of a collocated picture;
[0472] In response to the parameter not being signaled in the slice header, determining the collocated picture to be a picture referenced by an index whose value is equal to the smaller of a value of a reference index of the collocated picture signaled in the picture header and a number of active entries in the target reference picture list minus one; and
[0473] The current picture is encoded based on the collocated picture, wherein the collocated picture is used for temporal motion vector prediction.
[0474] 43. Apparatus according to clause 42, wherein the target reference picture list is indicated by a flag indicating from which reference picture list the collocated picture used for temporal motion vector prediction is derived.
[0475] 44. A device for processing video data, the device comprising:
[0476] a memory for storing instructions; and
[0477] One or more processors for executing instructions to cause the apparatus to:
[0478] receiving a video bitstream;
[0479] determining whether a parameter indicating a reference index of a collocated picture for temporal motion vector prediction exists in a slice header; and
[0480] In response to the parameter not being present, determining the value of the parameter to be equal to the smaller of the value of the reference index of the collocated picture for temporal motion vector prediction present in the picture header and the number of active entries in the target reference picture list minus 1;
[0481] Determining the concatenated picture is the picture referenced by the index whose value is equal to the parameter value in the target reference picture list; and
[0482] Decode the current image based on the concatenated image.
[0483] 45. Apparatus according to clause 44, wherein the target reference picture list is indicated by a flag indicating from which reference picture list the collocated picture used for temporal motion vector prediction is derived.
[0484] 46. A device for processing video data, the device comprising:
[0485] a memory for storing instructions; and
[0486] One or more processors for executing instructions to cause the apparatus to:
[0487] The total number is derived by adding the number of reference picture list structures in the sequence parameter set (SPS) to 1;
[0488] In response to signaling the reference picture list structures in a picture header of the current picture or a current slice header, allocating memory for the total number of reference picture list structures; and
[0489] Processes the current image or current strip using the allocated memory.
[0490] 47. A device for processing video data, the device comprising:
[0491] a memory for storing instructions; and
[0492] One or more processors for executing instructions to cause the apparatus to:
[0493] signaling a first flag in a picture parameter set (PPS) to indicate whether a second flag and a first index are present in a picture header syntax or a slice header of a current picture referencing the PPS, wherein the second flag indicates whether reference picture list 1 is derived based on one of reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index is an index in a list of reference picture list structures associated with reference picture list 1 included in the SPS for deriving reference picture list 1;
[0494] determining whether to signal a first index and a second index, wherein the second index is an index of a reference picture list structure associated with reference picture list 0 for deriving reference picture list 0 in a list of reference picture list structures associated with reference picture list 0 included in the SPS;
[0495] In response to not signaling the second index, determining a value of the second index includes:
[0496] When the SPS includes at most one reference picture list structure associated with reference picture list 0, determining that the value of the second index is equal to 0;
[0497] In response to not signaling the first index, determining a value of the first index includes:
[0498] When at most one reference picture list structure associated with reference picture list 1 is included in the SPS, determining the value of the first index to be equal to 0; and
[0499] When the first flag is equal to 0 and the second flag is equal to 1, determining that the value of the first index is equal to the value of the second index; deriving a reference picture list based on the first index and the second index; and
[0500] The current picture is encoded based on the reference picture list.
[0501] 48. A device for processing video data, the device comprising:
[0502] a memory for storing instructions; and
[0503] One or more processors for executing instructions to cause the apparatus to:
[0504] receiving a video bitstream;
[0505] determining a value of a first flag, the first flag indicating whether a second flag and a first index are present in a picture header syntax or a slice header of a current picture; wherein the second flag indicates whether reference picture list 1 is derived based on one of reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index is an index of the reference picture list structure associated with reference picture list 1 in a list of reference picture list structures associated with reference picture list 1 included in the SPS for deriving reference picture list 1;
[0506] determining whether a first index and a second index exist, wherein the second index is an index of a reference picture list structure associated with reference picture list 0 for deriving reference picture list 0 in a list of reference picture list structures associated with reference picture list 0 included in the SPS;
[0507] In response to the second index not existing, determining a value of the second index includes:
[0508] When the SPS includes at most one reference picture list structure associated with reference picture list 0, determining that the value of the second index is equal to 0;
[0509] In response to the first index not existing, determining a value of the first index includes:
[0510] When at most one reference picture list structure associated with reference picture list 1 is included in the SPS, determining the value of the first index to be equal to 0; and
[0511] When the first flag is equal to 0 and the second flag is equal to 1, determining that the value of the first index is equal to the value of the second index;
[0512] as well as
[0513] The current image is decoded based on the first index and the second index.
[0514] 49. A device for processing video data, the device comprising:
[0515] a memory for storing instructions; and
[0516] One or more processors for executing instructions to cause the apparatus to:
[0517] Signaling a first flag in a slice header to indicate whether an activated reference index number exists in the slice header, wherein the activated reference index number is used to derive a maximum reference index of a corresponding reference picture list that can be used to encode a current slice;
[0518] In response to the first flag indicating that there is an activation reference index number in the slice header,
[0519] determining the number of entries in reference picture list 0 and, if the number of entries in reference picture list 0 is greater than 1, signaling the number of activated reference indices of reference picture list 0 in a slice header in case of P and B slices; and
[0520] The number of entries in reference picture list 1 is determined, and when the number of entries in reference picture list 1 is greater than 1, the number of active reference indices of reference picture list 1 is signaled in the slice header in case of B slices.
[0521] 50. The apparatus of clause 49, wherein the processor is further configured to execute instructions to cause the apparatus to:
[0522] In response to the first flag indicating that the activation reference index number is not present in the slice header,
[0523] skipping signaling the activated reference index number of reference picture list 0 in the slice header in case of P and B slices; and
[0524] Skip signaling the activated reference index number of reference picture list 1 in the slice header in case of B slices.
[0525] 51. A device for processing video data, the device comprising:
[0526] a memory for storing instructions; and
[0527] One or more processors for executing instructions to cause the apparatus to:
[0528] receiving a video bitstream including slice header and picture header syntax;
[0529] determining a value of a first flag signaled in a slice header, the first flag indicating whether there is an activated reference index number in the slice header, wherein the activated reference index number is used to derive a maximum reference index of a corresponding reference picture list that can be used to decode a current slice;
[0530] In response to the first flag indicating the presence of an activation reference index number,
[0531] determining the number of entries in reference picture list 0 and, if the number of entries in reference picture list 0 is greater than 1, decoding the number of active reference indices of reference picture list 0 in the slice header in case of P and B slices;
[0532] as well as
[0533] Determine the number of entries in reference picture list 1, and when the number of entries in reference picture list 1 is greater than 1, decode the active reference index number of reference picture list 1 in the slice header in case of B slices.
[0534] 52. An apparatus according to clause 50, wherein the processor is further configured to execute instructions to cause the apparatus to perform:
[0535] In response to the first flag indicating that there is no activation reference index number,
[0536] Skip decoding the activated reference index number of reference picture list 0 in the slice header in case of P and B slices; and
[0537] Skip decoding the activated reference index number of reference picture list 1 in the slice header in case of B slices.
[0538] 53. A device for processing video data, the device comprising:
[0539] a memory for storing instructions; and
[0540] One or more processors for executing instructions to cause the apparatus to:
[0541] determining a collocated picture referenced by a reference index of the collocated picture at a slice level, wherein the collocated picture is determined to be the same picture for all non-I slices of the current picture; and
[0542] The current picture is processed based on a collocated picture, wherein the collocated picture is used for temporal motion vector prediction.
[0543] 54. A device for processing video data, the device comprising:
[0544] a memory for storing instructions; and
[0545] One or more processors for executing instructions to cause the apparatus to:
[0546] determining a collocated picture referred to by a reference index of the collocated picture at a slice level, wherein the collocated picture is determined to be the same picture for all P slices and B slices of the current picture; and
[0547] The current picture is processed based on a collocated picture, wherein the collocated picture is used for temporal motion vector prediction.
[0548] 55. A non-transitory computer readable medium storing an instruction set, the instruction set being executed by one or more processors of a device so that the device performs a method for performing video data processing, the method comprising:
[0549] encoding the current picture based on the collocated picture, wherein the collocated picture is used for temporal motion vector prediction; and
[0550] In response to the number of entries in reference image list 0 and the number of entries in reference image list 1 being greater than 0, a first flag and a second flag are sent by signal, wherein the first flag indicates that the collocated image is derived from reference image list 0 or reference image list 1, and the second flag indicates whether a motion vector difference syntax structure is sent by signal.
[0551] 56. A non-transitory computer readable medium storing an instruction set, the instruction set being executed by one or more processors of a device so that the device performs a method for performing video data processing, the method comprising:
[0552] receiving a video bitstream;
[0553] In response to the number of entries in reference picture list 0 and the number of entries in reference picture list 1 being greater than 0, decoding a first flag and a second flag, wherein the first flag indicates that the collocated picture used for temporal motion vector prediction is derived from reference picture list 0 or reference picture list 1, and the second flag indicates whether a motion vector difference syntax structure is present in a bitstream of a current picture; and
[0554] Decode the current image based on the concatenated image.
[0555] 57. A non-transitory computer readable medium storing an instruction set, the instruction set being executed by one or more processors of a device so that the device performs a method for performing video data processing, the method comprising:
[0556] encoding the current picture based on the collocated picture, wherein the collocated picture is used for temporal motion vector prediction; and
[0557] Indicates the collocated picture in the bitstream without signaling the index into the reference picture list.
[0558] 58. The non-transitory computer-readable medium of clause 57, wherein the method further comprises:
[0559] signaling a first flag to indicate whether the collocated picture is an inter-layer reference picture; and
[0560] In response to the collocated picture being an inter-layer reference picture, a first parameter is signaled to indicate the collocated picture, wherein the first parameter indicates an index of the collocated picture in a list of direct reference layers of a layer where the current picture is located.
[0561] 59. The non-transitory computer-readable medium of clause 58, wherein the method further comprises:
[0562] signaling a second flag to indicate whether the collocated picture is a short-term reference picture or a long-term reference picture; and
[0563] In response to the collocated picture being the short-term reference picture, a second parameter is signaled to indicate the collocated picture, wherein the second parameter indicates a difference between a picture order count of the collocated picture and a picture order count of the current picture.
[0564] 60. The non-transitory computer-readable medium of clause 59, wherein the method further comprises:
[0565] In response to the collocated image being a long-term reference image, a third parameter and a fourth parameter are signaled to indicate the collocated image, wherein the third parameter indicates a least significant bit (LSB) of a picture order count (POC) of the collocated image, and the fourth parameter indicates an incremental most significant bit (MSB) of the picture order count (POC) of the collocated image.
[0566] 61. The non-transitory computer readable medium of clause 60, wherein the first flag, the second flag, the first parameter, the second parameter, the third parameter, and the fourth parameter are signaled in an image header and all strips in the image have the same collocated image.
[0567] 62. The non-transitory computer-readable medium of clause 57, wherein the reference picture list is reference picture list 0 or reference picture list 1.
[0568] 63. A non-transitory computer readable medium storing an instruction set, the instruction set being executed by one or more processors of a device so that the device performs a method for performing video data processing, the method comprising:
[0569] receiving a video bitstream;
[0570] determining a collocated picture for temporal motion vector prediction without decoding an index into a reference picture list; and
[0571] Decode the current image based on the concatenated image.
[0572] 64. The non-transitory computer-readable medium of clause 63, wherein the method further comprises:
[0573] decoding a first flag indicating whether a collocated picture is an inter-layer reference picture;
[0574] determining whether the collocated picture is an inter-layer reference picture based on the first flag; and
[0575] In response to the collocated picture being an inter-layer reference picture, a first parameter is decoded and the collocated picture is determined based on the first parameter, wherein the first parameter indicates an index of the collocated picture in a list of direct reference layers of a layer where the current picture is located.
[0576] 65. The non-transitory computer-readable medium of clause 64, wherein the method further comprises:
[0577] decoding a second flag indicating whether the collocated picture is a short-term reference picture or a long-term reference picture;
[0578] determining whether the collocated image is a short-term reference image or a long-term reference image based on a second flag; and
[0579] In response to the collocated picture being a short-term reference picture, a second parameter is decoded and the collocated picture is determined based on the second parameter, wherein the second parameter indicates a difference between a picture order count of the collocated picture and a picture order count of the current picture.
[0580] 66. The non-transitory computer-readable medium of clause 65, wherein the method further comprises:
[0581] In response to the collocated image being a long-term reference image, a third parameter and a fourth parameter are decoded, and the collocated image is determined based on the third parameter and the fourth parameter, wherein the third parameter indicates a least significant bit (LSB) of a picture order count (POC) of the collocated image, and the fourth parameter indicates an incremental most significant bit (MSB) of the picture order count (POC) of the collocated image.
[0582] 67. The non-transitory computer-readable medium of clause 66, wherein the first flag, the second flag, the first parameter, the second parameter, the third parameter, and the fourth parameter are present in an image header, and all strips in the image have the same concatenated image.
[0583] 68. The non-transitory computer-readable medium of clause 63, wherein the reference picture list is reference picture list 0 or reference picture list 1.
[0584] 69. A non-transitory computer readable medium storing an instruction set, the instruction set being executed by one or more processors of a device so that the device performs a method for performing video data processing, the method comprising:
[0585] determining whether to signal a parameter in a slice header to indicate a reference index of a collocated picture;
[0586] In response to the parameter not being signaled in the slice header, determining the collocated picture to be a picture referenced by an index whose value is equal to the smaller of a value of a reference index of the collocated picture signaled in the picture header and a number of active entries in the target reference picture list minus one; and
[0587] The current picture is encoded based on the collocated picture, wherein the collocated picture is used for temporal motion vector prediction.
[0588] 70. The non-transitory computer-readable medium of clause 69, wherein the target reference picture list is indicated by a flag indicating from which reference picture list the collocated picture for temporal motion vector prediction is derived.
[0589] 71. A non-transitory computer readable medium storing an instruction set, the instruction set being executed by one or more processors of a device so that the device performs a method for performing video data processing, the method comprising:
[0590] receiving a video bitstream;
[0591] determining whether there is a parameter in a slice header indicating a reference index of a collocated picture for temporal motion vector prediction;
[0592] In response to the parameter not being present, determining the value of the parameter to be equal to the smaller of the value of the reference index of the collocated picture for temporal motion vector prediction present in the picture header and the number of active entries in the target reference picture list minus 1;
[0593] Determining the concatenated picture is the picture referenced by the index whose value is equal to the parameter value in the target reference picture list; and
[0594] Decode the current image based on the concatenated image.
[0595] 72. A non-transitory computer-readable medium as described in clause 71, wherein the target reference picture list is indicated by a flag indicating from which reference picture list the collocated picture used for temporal motion vector prediction is derived.
[0596] 73. A non-transitory computer readable medium storing an instruction set, the instruction set being executed by one or more processors of a device so that the device performs a method for performing video data processing, the method comprising:
[0597] The total number is derived by adding the number of reference picture list structures in the sequence parameter set (SPS) to 1;
[0598] In response to signaling the reference picture list structures in a picture header of the current picture or a slice header of the current slice, allocating memory for the total number of reference picture list structures; and
[0599] Processes the current image or current strip using the allocated memory.
[0600] 74. A non-transitory computer readable medium storing a set of instructions, the set of instructions being executed by one or more processors of a device so that the device performs a method for performing video data processing, the method comprising:
[0601] signaling a first flag in a picture parameter set (PPS) to indicate whether a second flag and a first index are present in a picture header syntax or a slice header of a current picture referencing the PPS, wherein the second flag indicates whether reference picture list 1 is derived based on one of reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index is an index in a list of reference picture list structures associated with reference picture list 1 included in the SPS for deriving reference picture list 1;
[0602] determining whether to signal a first index and a second index, wherein the second index is an index of a reference picture list structure associated with reference picture list 0 for deriving reference picture list 0 in a list of reference picture list structures associated with reference picture list 0 included in the SPS;
[0603] In response to not signaling the second index, determining a value of the second index includes:
[0604] When the SPS includes at most one reference picture list structure associated with reference picture list 0, determining that the value of the second index is equal to 0;
[0605] In response to not signaling the first index, determining a value of the first index includes:
[0606] When at most one reference picture list structure associated with reference picture list 1 is included in the SPS, determining the value of the first index to be equal to 0; and
[0607] When the first flag is equal to 0 and the second flag is equal to 1, determining that the value of the first index is equal to the value of the second index;
[0608] deriving a reference picture list based on the first index and the second index; and
[0609] The current picture is encoded based on the reference picture list.
[0610] 75. A non-transitory computer readable medium storing an instruction set, the instruction set being executed by one or more processors of a device so that the device performs a method for performing video data processing, the method comprising:
[0611] receiving a video bitstream;
[0612] determining a value of a first flag, the first flag indicating whether a second flag and a first index are present in a picture header syntax or a slice header of a current picture, wherein the second flag indicates whether reference picture list 1 is derived based on one of reference picture list structures associated with reference picture list 1 signaled in a sequence parameter set (SPS), and the first index is an index in a list of reference picture list structures associated with reference picture list 1 included in the SPS for deriving reference picture list 1;
[0613] determining whether a first index and a second index exist, wherein the second index is an index of a reference picture list structure associated with reference picture list 0 for deriving reference picture list 0 in a list of reference picture list structures associated with reference picture list 0 included in the SPS;
[0614] In response to the second index not existing, determining a value of the second index includes:
[0615] When the SPS includes at most one reference picture list structure associated with reference picture list 0, determining that the value of the second index is equal to 0;
[0616] In response to the first index not existing, determining a value of the first index includes:
[0617] When at most one reference picture list structure associated with reference picture list 1 is included in the SPS, determining the value of the first index to be equal to 0; and
[0618] When the first flag is equal to 0 and the second flag is equal to 1, determining that the value of the first index is equal to the value of the second index; and
[0619] The current image is decoded based on the first index and the second index.
[0620] 76. A non-transitory computer readable medium storing a set of instructions, the set of instructions being executed by one or more processors of a device so that the device performs a method for performing video data processing, the method comprising:
[0621] Signaling a first flag in a slice header to indicate whether there is an activated reference index number in the slice header, wherein the activated reference index number is used to derive a maximum reference index of a corresponding reference picture list that can be used to encode a current slice;
[0622] In response to the first flag indicating that there is an activation reference index number in the slice header,
[0623] determining the number of entries in reference picture list 0 and, if the number of entries in reference picture list 0 is greater than 1, signaling the number of activated reference indices of reference picture list 0 in a slice header in case of P and B slices; and
[0624] The number of entries in reference picture list 1 is determined, and when the number of entries in reference picture list 1 is greater than 1, the number of active reference indices of reference picture list 1 is signaled in the slice header in case of B slices.
[0625] 77. The non-transitory computer-readable medium of clause 76, wherein the method further comprises:
[0626] In response to the first flag indicating that the activation reference index number is not present in the slice header,
[0627] skipping signaling the activated reference index number of reference picture list 0 in the slice header in case of P and B slices; and
[0628] Skip signaling the activated reference index number of reference picture list 1 in the slice header in case of B slices.
[0629] 78. A non-transitory computer readable medium storing an instruction set, the instruction set being executed by one or more processors of a device so that the device performs a method for performing video data processing, the method comprising:
[0630] receiving a video bitstream including slice header and picture header syntax;
[0631] determining a value of a first flag signaled in a slice header, the first flag indicating whether there is an activated reference index number in the slice header, wherein the activated reference index number is used to derive a maximum reference index of a corresponding reference picture list that can be used to decode a current slice;
[0632] In response to the first flag indicating the presence of an activation reference index number,
[0633] determining the number of entries in reference picture list 0 and, if the number of entries in reference picture list 0 is greater than 1, decoding the active reference index number of reference picture list 0 in the slice header in case of P and B slices; and
[0634] Determine the number of entries in reference picture list 1, and when the number of entries in reference picture list 1 is greater than 1, decode the active reference index number of reference picture list 1 in the slice header in case of B slices.
[0635] 79. The non-transitory computer-readable medium of clause 78, wherein the method further comprises:
[0636] In response to the first flag indicating that there is no activation reference index number,
[0637] Skip decoding the activated reference index number of reference picture list 0 in the slice header in case of P and B slices; and
[0638] Skip decoding the activated reference index number of reference picture list 1 in the slice header in case of B slices.
[0639] 80. A non-transitory computer readable medium storing an instruction set, the instruction set being executed by one or more processors of a device so that the device performs a method for performing video data processing, the method comprising:
[0640] determining a collocated picture referenced by a reference index of the collocated picture at a slice level, wherein the collocated picture is determined to be the same picture for all non-I slices of the current picture; and
[0641] The current picture is processed based on a collocated picture, wherein the collocated picture is used for temporal motion vector prediction.
[0642] 81. A non-transitory computer readable medium storing an instruction set, the instruction set being executed by one or more processors of a device so that the device performs a method for performing video data processing, the method comprising:
[0643] determining a collocated picture referred to by a reference index of the collocated picture at a slice level, wherein the collocated picture is determined to be the same picture for all P slices and B slices of the current picture; and
[0644] The current picture is processed based on a collocated picture, wherein the collocated picture is used for temporal motion vector prediction.
[0645] The present disclosure also provides a computer-implemented apparatus for video processing, comprising:
[0646] A first processing module, configured to derive a total number by adding the number of reference picture list structures in a sequence parameter set SPS to 1;
[0647] a second processing module, responsive to signaling the reference picture list structure in a picture header of the current picture or a slice header of the current slice, allocating memory for the total number of reference picture list structures; and
[0648] The current image or current strip is processed using the allocated memory.
[0649] The present disclosure also provides a computer-implemented apparatus for video processing, comprising:
[0650] a third processing module for signaling a first flag in a slice header to indicate whether there is an activated reference index number in the slice header, wherein the activated reference index number is used to derive a maximum reference index of a corresponding reference picture list for encoding a current slice;
[0651] a fourth processing module, configured to, in response to the first flag indicating that the activation reference index number exists in the slice header,
[0652] determining a number of entries in reference picture list 0 and, when the number of entries in reference picture list 0 is greater than 1, signaling the number of active reference indexes for reference picture list 0 in the slice headers of P and B slices; and
[0653] The number of entries in reference picture list 1 is determined, and when the number of entries in reference picture list 1 is greater than 1, the number of active reference indexes of reference picture list 1 is signaled in the slice header of the B slice.
[0654] In some embodiments, the computer-implemented apparatus for video processing further includes: a fifth processing module; the fifth processing module is configured to:
[0655] In response to the first flag indicating that the activation reference index number does not exist in the slice header,
[0656] skipping signaling the activated reference index number of reference picture list 0 in the slice header of P and B slices; and
[0657] Skip signaling the activated reference index number of reference picture list 1 in the slice header of the B slice.
[0658] The present disclosure also provides a computer program product, including a computer program, which implements any of the above methods when executed by a processor.
[0659] An embodiment of the present disclosure provides a storage medium product, which is used to store a bit stream generated by executing the above method.
[0660] In the above description, embodiments are described with reference to a large number of specific details, which may vary with the implementation. The described embodiments may be subject to certain adjustments and modifications. In view of the description and practice of the invention disclosed herein, other embodiments will be apparent to those skilled in the art. The description and examples are considered to be exemplary only, and the true scope and spirit of the invention are indicated by the appended claims. The sequence of steps shown in the figures is also intended only for illustrative purposes and is not intended to limit any particular sequence of steps. Therefore, it will be appreciated by those skilled in the art that these steps may be performed in different orders when implementing the same method.
[0661] Exemplary embodiments have been disclosed in the drawings and the specification. However, many variations and modifications may be made to these embodiments. Therefore, although specific terms are used, they are used only in a general and descriptive sense and not for the purpose of limitation.
Claims
1. A method for encoding a video sequence into a bit stream, comprising: receiving a video sequence; as well as Encoding one or more images in the video sequence, wherein the encoding comprises: signaling a first flag in a slice header to indicate whether an active reference index number exists in the slice header, wherein the active reference index number is used to derive a maximum reference index of a corresponding reference picture list for encoding a current slice; In response to the first flag indicating that the activation reference index number exists in the slice header, determining a number of entries in reference picture list 0 and, when the number of entries in reference picture list 0 is greater than 1, signaling an active reference index number for reference picture list 0 in the slice headers of P and B slices; and The number of entries in reference picture list 1 is determined, and when the number of entries in reference picture list 1 is greater than 1, an activated reference index number of reference picture list 1 is signaled in the slice header of the B slice.
2. The method according to claim 1, wherein: When the first flag indicates that the activation reference index number does not exist in the slice header, skipping signaling the activated reference index number of reference picture list 0 in the slice header of P and B slices; and Skip signaling the activated reference index number of reference picture list 1 in the slice header of the B slice.
3. The method according to claim 1, further comprising: A bitstream is generated, the bitstream including: the first flag, the activated reference index number of the reference picture list 0 in the slice header of the P and B slices, and the activated reference index number of the reference picture list 1 in the slice header of the B slice.
4. A method for decoding a bit stream to output one or more images of a video stream, comprising: Receive a bit stream; as well as Decoding one or more images using the encoded information in the bitstream, wherein the decoding comprises: determining a value of a first flag in a slice header indicating whether there is an activated reference index number, wherein the activated reference index number is used to derive a maximum reference index of a corresponding reference picture list for decoding a current slice; In response to the first flag indicating the presence of the active reference index number, determining the number of entries in reference picture list 0 and the number of entries in reference picture list 1; when the number of entries in the reference picture list 0 is greater than 1, the number of activated reference indices of the reference picture list 0 in the slice header for decoding P and B slices; When the number of entries in the reference picture list 1 is greater than 1, the number of activated reference indexes of the reference picture list 1 in the slice header of the decoded B slice.
5. The method according to claim 4, wherein: When the first flag indicates that the activation reference index number does not exist, skip decoding of the activated reference index number of reference picture list 0 in the slice header of P and B slices; as well as Skip decoding the activated reference index number of reference picture list 1 in the slice header of the B slice.
6. A device for processing video data, the device comprising: A memory for storing instructions; as well as One or more processors, configured to execute the instructions so that the apparatus performs: receiving a video sequence; as well as Encoding one or more images in the video sequence, wherein the encoding comprises: signaling a first flag in a slice header to indicate whether an active reference index number exists in the slice header, wherein the active reference index number is used to derive a maximum reference index of a corresponding reference picture list for encoding a current slice; In response to the first flag indicating that the activation reference index number exists in the slice header, determining a number of entries in reference picture list 0 and, when the number of entries in reference picture list 0 is greater than 1, signaling an active reference index number for reference picture list 0 in the slice headers of P and B slices; and The number of entries in reference picture list 1 is determined, and when the number of entries in reference picture list 1 is greater than 1, an activated reference index number of reference picture list 1 is signaled in the slice header of the B slice.
7. The apparatus of claim 6, wherein the processor is further configured to execute the instructions so that the apparatus performs: When the first flag indicates that the activation reference index number does not exist in the slice header, skipping signaling the activated reference index number of reference picture list 0 in the slice header of P and B slices; and Skip signaling the activated reference index number of reference picture list 1 in the slice header of the B slice.
8. The apparatus of claim 6, wherein the processor is further configured to execute the instructions so that the apparatus performs: Generate a bitstream, the bitstream comprising: The first flag, the activated reference index number of the reference picture list 0 in the slice header of the P and B slices, and the activated reference index number of the reference picture list 1 in the slice header of the B slice.
9. A device for processing video data, the device comprising: A memory for storing instructions; as well as One or more processors, configured to execute the instructions so that the apparatus performs: Receive a bit stream; as well as Decoding one or more images using the encoded information in the bitstream, wherein the decoding comprises: determining a value of a first flag in a slice header indicating whether there is an activated reference index number, wherein the activated reference index number is used to derive a maximum reference index of a corresponding reference picture list for decoding a current slice; In response to the first flag indicating the presence of the active reference index number, determining the number of entries in reference picture list 0 and the number of entries in reference picture list 1, when the number of entries in the reference picture list 0 is greater than 1, the number of activated reference indices of the reference picture list 0 in the slice header for decoding P and B slices; as well as When the number of entries in the reference picture list 1 is greater than 1, the number of activated reference indexes of the reference picture list 1 in the slice header of the decoded B slice.
10. The apparatus of claim 9, wherein the processor is further configured to execute the instructions so that the apparatus performs: When the first flag indicates that the activation reference index number does not exist, skip decoding of the activated reference index number of reference picture list 0 in the slice header of P and B slices; and Skip decoding the activated reference index number of reference picture list 1 in the slice header of the B slice.
11. A non-transitory computer-readable medium storing an instruction set, wherein the instruction set is executed by one or more processors of a device, so that the device starts to execute a method for performing video data processing, the method comprising: receiving a video sequence; as well as Encoding one or more images in the video sequence, wherein the encoding comprises: signaling a first flag in a slice header to indicate whether an active reference index number exists in the slice header, wherein the active reference index number is used to derive a maximum reference index of a corresponding reference picture list for encoding a current slice; In response to the first flag indicating that the activation reference index number exists in the slice header, determining a number of entries in reference picture list 0 and, when the number of entries in reference picture list 0 is greater than 1, signaling an active reference index number for reference picture list 0 in the slice headers of P and B slices; and The number of entries in reference picture list 1 is determined, and when the number of entries in reference picture list 1 is greater than 1, an activated reference index number of reference picture list 1 is signaled in the slice header of the B slice.
12. The non-transitory computer readable medium of claim 11, wherein: When the first flag indicates that the activation reference index number does not exist in the slice header, the method further includes: skipping signaling the activated reference index number of reference picture list 0 in the slice header of P and B slices; and Skip signaling the activated reference index number of reference picture list 1 in the slice header of the B slice.
13. The non-transitory computer readable medium of claim 11, wherein: The method further comprises: A bitstream is generated, the bitstream including: the first flag, the activated reference index number of the reference picture list 0 in the slice header of the P and B slices, and the activated reference index number of the reference picture list 1 in the slice header of the B slice.
14. A non-transitory computer-readable medium storing an instruction set, wherein the instruction set is executed by one or more processors of a device, so that the device starts to execute a method for performing video data processing, the method comprising: Receive a bit stream; as well as Decoding one or more images using the encoded information in the bitstream, wherein the decoding comprises: determining a value of a first flag in a slice header indicating whether there is an activated reference index number, wherein the activated reference index number is used to derive a maximum reference index of a corresponding reference picture list for decoding a current slice; In response to the first flag indicating the presence of the active reference index number, determining the number of entries in reference picture list 0 and the number of entries in reference picture list 1; when the number of entries in the reference picture list 0 is greater than 1, the number of activated reference indices of the reference picture list 0 in the slice header for decoding P and B slices; When the number of entries in the reference picture list 1 is greater than 1, the number of activated reference indexes of the reference picture list 1 in the slice header of the decoded B slice.
15. The non-transitory computer readable medium of claim 14, wherein: When the first flag indicates that the activation reference index number does not exist, the method further includes: skip decoding of the activated reference index number of reference picture list 0 in the slice header of P and B slices; and Skip decoding the activated reference index number of reference picture list 1 in the slice header of the B slice.
