Method and apparatus for high-level syntax in video coding and decoding
By rearranging and grouping the syntax elements in the video encoding and codec standards, the designs related to intra- and inter-frame prediction are optimized, and the encoding and codec efficiency and quality problems caused by unreasonable design in the prior art are solved, and more efficient video programming is achieved Decoding effect.
Patent Information
- Application Number
- CN202311191514.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-04-04
- Filing Date
- 2021-04-02
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2041-04-02
AI Technical Summary
The existing video codec standards have unreasonable order and grouping methods in the design of inter prediction and intra prediction syntax elements, which affect the encoding and codec efficiency and video quality.
By rearranging syntax elements, the intra prediction-related elements are defined before the inter prediction-related elements and grouped by prediction type, adding flags that allow inter prediction to control the prediction type at the codec level, optimizing partition constraints and reference pictures Use of
Improve the efficiency and quality of video encoding and decoding, and through reasonable grammatical design and logo control, unnecessary redundant information transmission is reduced, and bit rate and video quality is improved.
Smart Images

Figure CN117221604B_ABST
Abstract
Description
[0001] This application is a divisional of Chinese patent application No. 202180039854.9, entitled “Methods and devices for advanced syntax in video encoding and decoding,” filed on April 2, 2021. Technical Field
[0002] The present disclosure relates to video coding and compression. More specifically, the present application relates to high-level syntax in video bitstreams applicable to one or more video codec standards. Background Art
[0003] Various video codec techniques can be used to compress video data. Video coding is performed according to one or more video codec standards. For example, video codec standards include Versatile Video Codec (VVC), Joint Exploration Test Model (JEM), High Efficiency Video Codec (H.265 / HEVC), Advanced Video Codec (H.264 / AVC), Moving Picture Experts Group (MPEG) codec, etc. Video coding is usually performed using prediction methods (e.g., inter-frame prediction, intra-frame prediction, etc.), which utilize redundancy present in video images or sequences. An important goal of video coding technology is to compress video data into a form that uses a lower bit rate while avoiding or minimizing video quality degradation. Summary of the invention
[0004] Examples of the present disclosure provide methods and apparatus for lossless coding in video coding.
[0005] According to a first aspect of the present disclosure, a method for decoding a video signal is provided. The method may include a decoder receiving at least one universal video codec (VVC) syntax flag. The at least one VVC syntax flag may include a first VVC syntax flag that may indicate whether inter-frame prediction is allowed in the corresponding codec level. The decoder may also receive an inter-frame related syntax element in response to a syntax element indicating that inter-frame prediction is allowed. The decoder may also obtain a first reference picture I associated with a video block in a bitstream. (0) and the second reference picture I (1) In the display order, the first reference picture I (0) The second reference picture I may be before the current picture (1) The decoder may also obtain the first reference picture I (0) The first prediction sample I of the video block is obtained by (0) (i, j). i and j may represent the coordinates of a sample point of the current picture. The decoder may also obtain the coordinates of the second reference picture I (1) The reference block in obtains the second prediction sample I of the video block(1) The decoder may also be configured to decode the first prediction sample I based on the at least one VVC syntax flag, the inter-frame related syntax element, and the first prediction sample I. (0) (i, j) and the second prediction sample I (1) (i, j) to obtain the bidirectional prediction samples.
[0006] According to a second aspect of the present disclosure, a method for decoding a video signal is provided. The method may include a decoder receiving an arrangement partition constraint syntax element at a sequence parameter set (SPS) level. The arrangement partition constraint syntax element is arranged so that inter-frame prediction related syntax elements are grouped according to VVC syntax at the codec level. The decoder may also obtain a first reference picture I associated with a video block in a bitstream. (0) and the second reference picture I (1) In the display order, the first reference picture I (0) The second reference picture I may be before the current picture (1) The decoder may also obtain the first reference picture I (0) The first prediction sample I of the video block is obtained by (0) (i, j). i and j may represent the coordinates of a sample point of the current picture. The decoder may also obtain the coordinates of the second reference picture I (1) The reference block in obtains the second prediction sample I of the video block (1) The decoder may also be configured to perform the following operations based on the arrangement partition constraint syntax element, the first prediction sample point I (0) (i, j) and the second prediction sample I (1) (i, j) to obtain the bidirectional prediction samples.
[0007] According to a third aspect of the present disclosure, a computing device is provided. The computing device may include one or more processors and a non-transitory computer-readable memory storing instructions executable by the one or more processors. The one or more processors may be configured to receive at least one VVC syntax flag. The at least one VVC syntax flag includes a first VVC syntax flag indicating whether inter-frame prediction is allowed in the corresponding codec level. The one or more processors may be further configured to receive an inter-frame related syntax element in response to a syntax element indicating that inter-frame prediction is allowed. The one or more processors may be further configured to obtain a first reference picture I associated with a video block in a bitstream. (0) and the second reference picture I (1) In the display order, the first reference picture I (0) The second reference picture I may be before the current picture (1)The one or more processors may be further configured to obtain a first reference picture from the first reference picture I (0) The first prediction sample I of the video block is obtained by (0) (i, j). i and j may represent the coordinates of a sample point of the current picture. The one or more processors may be further configured to obtain the coordinates of a sample point of the current picture from the second reference picture I (1) The reference block in obtains the second prediction sample I of the video block (1) The one or more processors may be further configured to generate a VVC syntax flag based on the at least one VVC syntax flag, the inter-frame related syntax element, the first prediction sample I (0) (i, j) and the second prediction sample I (1) (i, j) to obtain the bidirectional prediction samples.
[0008] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium having instructions stored therein is provided. When the instructions are executed by one or more processors of the device, the instructions may cause the device to receive an SPS-level permutation partition constraint syntax element. The permutation partition constraint syntax element is arranged so that inter-frame prediction-related syntax elements are grouped according to VVC syntax at the codec level. The instructions may also cause the device to obtain a first reference picture I associated with a video block in a bitstream. (0) and the second reference picture I (1) In the display order, the first reference picture I (0) The second reference picture I may be before the current picture (1) The instruction may also cause the device to obtain the first reference picture I (0) The first prediction sample I of the video block is obtained by (0) (i, j). i and j may represent the coordinates of a sample point of the current picture. The instruction may also cause the device to obtain the second reference picture I (1) The reference block in obtains the second prediction sample I of the video block (1) The instruction may also cause the device to arrange the partition constraint syntax element, the first prediction sample I (0) (i, j) and the second prediction sample I (1) (i, j) to obtain the bidirectional prediction samples.
[0009] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not intended to restrict the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate examples consistent with the disclosure and, together with the description, serve to explain the principles of the disclosure.
[0011] Figure 1 is a block diagram of an encoder according to an example of the present disclosure.
[0012] Figure 2 is a block diagram of a decoder according to an example of the present disclosure.
[0013] Figure 3A is a diagram illustrating block partitioning in a multi-type tree structure according to an example of the present disclosure.
[0014] Figure 3B is a diagram illustrating block partitioning in a multi-type tree structure according to an example of the present disclosure.
[0015] Figure 3C is a diagram illustrating block partitioning in a multi-type tree structure according to an example of the present disclosure.
[0016] Figure 3D is a diagram illustrating block partitioning in a multi-type tree structure according to an example of the present disclosure.
[0017] Figure 3E is a diagram illustrating block partitioning in a multi-type tree structure according to an example of the present disclosure.
[0018] Figure 4 A method for decoding a video signal according to an example of the present disclosure.
[0019] Figure 5 A method for decoding a video signal according to an example of the present disclosure.
[0020] Figure 6 A method for decoding a video signal according to an example of the present disclosure.
[0021] Figure 7 is a diagram illustrating a computing environment coupled with a user interface according to examples of the present disclosure. DETAILED DESCRIPTION
[0022] Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. The following descriptions all refer to the accompanying drawings, in which, unless otherwise specified, the same reference numerals in different drawings represent the same or similar elements. The embodiments set forth in the following descriptions of the embodiments do not represent all embodiments consistent with the present disclosure. Rather, they are merely examples of devices and methods consistent with the aspects related to the present disclosure recited in the appended claims.
[0023] The terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. As used in this disclosure and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein is intended to represent and include any or all possible combinations of one or more items in the associated enumerated items.
[0024] It should be understood that although the terms "first", "second", "third", etc. may be used herein to describe various information, such information should not be limited by these terms. These terms are merely used to distinguish one type of information from another type of information. For example, without departing from the scope of the present disclosure, the first information may be referred to as the second information; and similarly, the second information may also be referred to as the first information. As used herein, the term "if" may be understood to mean "when" or "at" or "in response to a judgment", depending on the context.
[0025] The first version of the HEVC standard was completed in October 2013, which provides about 50% bitrate savings or equivalent perceptual quality compared to the previous generation video codec standard H.264 / MPEG AVC. Although the HEVC standard provides significant codec improvements over its predecessor, there is evidence that codec efficiency better than HEVC can be achieved using additional codec tools. On this basis, both VCEG and MPEG have started exploration work on new codec technologies to achieve future video codec standardization. ITU-T VECG and ISO / IEC MPEG established a Joint Video Exploration Team (JVET) in October 2015 to start major research on advanced technologies that can significantly improve codec efficiency. JVET maintains a reference software called the Joint Exploration Model (JEM) by integrating multiple additional codec tools on the basis of the HEVC Test Model (HM).
[0026] In October 2017, ITU-T and ISO / IEC issued a joint call for proposals (CfP) on video compression with capabilities beyond HEVC. In April 2018, 23 CfP responses were received and evaluated at the 10th JVET meeting, demonstrating that compression efficiency was about 40% higher than HEVC. Based on such evaluation results, JVET launched a new project to develop a new generation of video codec standards called Versatile Video Codec (VVC). In the same month, a reference software code base called VVC Test Model (VTM) was established to demonstrate a reference implementation of the VVC standard. Like HEVC, VVC is built on a block-based hybrid video codec framework.
[0027] Figure 1 An overall diagram of a block-based video encoder for VVC is shown. Specifically, Figure 1 A typical encoder 100 is shown. The encoder 100 has a video input 110, motion compensation 112, motion estimation 114, intra / inter mode decision 116, block prediction value 140, adder 128, transform 130, quantization 132, prediction related information 142, intra prediction 118, picture buffer 120, inverse quantization 134, inverse transform 136, adder 126, memory 124, loop filter 122, entropy coding 138, and bitstream 144.
[0028] In encoder 100, a video frame is partitioned into multiple video blocks for processing. For each given video block, a prediction is formed based on an inter-prediction method or an intra-prediction method.
[0029] The prediction residual representing the difference between the current video block (part of the video input 110) and its prediction value (part of the block prediction value 140) is sent from the adder 128 to the transform 130. The transform coefficients are then sent from the transform 130 to the quantization 132 for entropy reduction. The quantized coefficients are then fed to the entropy coding 138 to generate a compressed video bitstream. Figure 1 As shown, prediction related information 142 from intra / inter mode decision 116, such as video block partition information, motion vector (MV), reference picture index, and intra prediction mode, is also fed through entropy coding 138 and saved into a compressed bitstream 144. The compressed bitstream 144 comprises a video bitstream.
[0030] In the encoder 100, decoder related circuitry is also required to reconstruct pixels for prediction purposes. First, the prediction residual is reconstructed by inverse quantization 134 and inverse transformation 136. The reconstructed prediction residual is combined with the block prediction value 140 to generate the unfiltered reconstructed pixels of the current video block.
[0031] Spatial prediction (or "intra-prediction") uses pixels from samples of already coded neighboring blocks (called reference samples) in the same video frame as the current video block to predict the current video block.
[0032] Temporal prediction (also known as "inter prediction") uses reconstructed pixels from already coded video pictures to predict the current video block. Temporal prediction reduces temporal redundancy inherent in video signals. The temporal prediction signal for a given coding unit (CU) or coding block is typically signaled by one or more MVs indicating the amount and direction of motion between the current CU and its temporal reference. Further, if multiple reference pictures are supported, a reference picture index is additionally sent, which is used to identify which reference picture in the reference picture store the temporal prediction signal comes from.
[0033] Motion estimation 114 takes the video input 110 and the signal from the picture buffer 120 and outputs the motion estimation signal to motion compensation 112. Motion compensation 112 takes the video input 110, the signal from the picture buffer 120 and the motion estimation signal from motion estimation 114 and outputs the motion compensation signal to intra / inter mode decision 116.
[0034] After performing spatial prediction and / or temporal prediction, the intra / inter mode decision 116 in the encoder 100 selects the best prediction mode, for example based on a rate-distortion optimization method. The block prediction value 140 is then subtracted from the current video block, and the resulting prediction residual is decorrelated using a transform 130 and a quantization 132. The resulting quantized residual coefficients are dequantized by dequantization 134 and inversely transformed by an inverse transform 136 to form a reconstructed residual, which is then added back to the prediction block to form a reconstructed signal for the CU. Further loop filtering 122, such as a deblocking filter, sample adaptive offset (SAO), and / or an adaptive loop filter (ALF), may be applied to the reconstructed CU before the reconstructed CU is placed in the reference picture storage of the picture buffer 120 and used to encode and decode future video blocks. To form an output video bitstream 144, the encoding and decoding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are sent to an entropy coding unit 138 for further compression and packaging to form a bitstream.
[0035] Figure 1 A block diagram of a general block-based hybrid video coding system is given. The input video signal is processed block by block, which is called a coding unit (CU). In VTM-1.0, a CU can be up to 128×128 pixels. However, unlike HEVC, which partitions blocks based solely on a quadtree, in VVC, a coding tree unit (CTU) is divided into multiple CUs to accommodate different local characteristics based on quad / bin / ternary trees. In addition, the concept of multiple partition unit types in HEVC is removed, that is, the splitting of CU, prediction unit (PU), and transform unit (TU) no longer exists in VVC; instead, each CU is always used as a basic unit for both prediction and transformation without further partitioning. In the multi-type tree structure, a CTU is first partitioned by a quadtree structure. Then, each quadtree leaf node can be further partitioned by a binary tree structure and a ternary tree structure.
[0036] like Figure 3A , Figure 3B , Figure 3C , Figure 3D and Figure 3EAs shown, there are five types of partitioning, four-way partitioning, horizontal two-way partitioning, vertical two-way partitioning, horizontal three-way partitioning, and vertical three-way partitioning.
[0037] Figure 3A A diagram illustrating block quad partitioning in a multi-type tree structure according to the present disclosure is shown.
[0038] Figure 3B A diagram illustrating vertical binary partitioning of blocks in a multi-type tree structure according to the present disclosure is shown.
[0039] Figure 3C A diagram illustrating block-level binary partitioning in a multi-type tree structure according to the present disclosure is shown.
[0040] Figure 3D A diagram illustrating vertical ternary partitioning of blocks in a multi-type tree structure according to the present disclosure is shown.
[0041] Figure 3E A diagram illustrating block-level ternary partitioning in a multi-type tree structure according to the present disclosure is shown.
[0042] exist Figure 1In the video code, spatial prediction and / or temporal prediction can be performed. Spatial prediction (or "intra-frame prediction") uses pixels from samples (called reference samples) of already coded adjacent blocks in the same video picture / strip to predict the current video block. Spatial prediction reduces the spatial redundancy inherent in the video signal. Temporal prediction (also known as "inter-frame prediction" or "motion compensated prediction") uses reconstructed pixels from already coded video images to predict the current video block. Temporal prediction reduces the temporal redundancy inherent in the video signal. The temporal prediction signal of a given CU is usually signaled by one or more motion vectors (MVs) indicating the amount and direction of motion between the current CU and its temporal reference. Similarly, if multiple reference pictures are supported, a reference picture index is additionally sent, which is used to identify which reference picture in the reference picture storage the temporal prediction signal comes from. After performing spatial and / or temporal prediction, the mode decision block in the encoder selects the best prediction mode, for example based on a rate-distortion optimization method. Then, the prediction block is subtracted from the current video block; and the prediction residual is decorrelated and quantized using a transform. The quantized residual coefficients are dequantized and inverse transformed to form a reconstructed residual, which is then added back to the prediction block to form the reconstructed signal of the CU. Further, loop filtering such as deblocking filter, sample adaptive offset (SAO) and adaptive loop filter (ALF) can be applied to the reconstructed CU before it is placed in the reference picture storage and used for encoding and decoding future video blocks. In order to form the output video bitstream, the coding mode (inter-frame or intra-frame), prediction mode information, motion information, and quantized residual coefficients are sent to the entropy coding unit for further compression and packaging to form a bitstream.
[0043] Figure 2 FIG. 1 shows a general block diagram of a video decoder for VVC. Specifically, Figure 2 A block diagram of a typical decoder 200 is shown. The decoder 200 has a bitstream 210, entropy decoding 212, inverse quantization 214, inverse transform 216, adder 218, intra / inter mode selection 220, intra prediction 222, memory 230, loop filter 228, motion compensation 224, picture buffer 226, prediction related information 234, and video output 232.
[0044] Decoder 200 is similar to that present in Figure 1The decoder 200 first decodes the incoming video bitstream 210 by entropy decoding 212 to obtain the quantization coefficient levels and prediction related information. Then, the quantization coefficient levels are processed by inverse quantization 214 and inverse transform 216 to obtain the reconstructed prediction residuals. The block prediction value mechanism implemented in the intra / inter mode selector 220 is configured to perform intra prediction 222 or motion compensation 224 based on the decoded prediction information. A set of unfiltered reconstructed pixels is obtained by summing the reconstructed prediction residuals from the inverse transform 216 with the prediction output generated by the block prediction value mechanism using a summer 218.
[0045] The reconstructed block may further pass through a loop filter 228 and then be stored in a picture buffer 226 used as a reference picture storage. The reconstructed video in the picture buffer 226 may be sent to drive a display device and used to predict future video blocks. When the loop filter 228 is turned on, a filtering operation is performed on these reconstructed pixels, thereby obtaining a final reconstructed video output 232.
[0046] Figure 2 A general block diagram of a block-based video decoder is given. The video bitstream is first entropy decoded at the entropy decoding unit. The codec mode and prediction information are sent to the spatial prediction unit (in the case of intra-frame codec) or the temporal prediction unit (in the case of inter-frame codec) to form a prediction block. The residual transform coefficients are sent to the dequantization unit and the inverse transform unit to reconstruct the residual block. The prediction block and the residual block are then added. The reconstructed block can be further passed through loop filtering and then stored in the reference picture storage. The reconstructed video in the reference picture storage is then sent out to drive the display device and is used to predict future video blocks.
[0047] Generally, the basic intra prediction scheme applied in VVC remains the same as that of HEVC, except that several modules are further extended and / or improved, such as matrix weighted intra prediction (MIP) codec mode, intra sub-partitioning (ISP) codec mode, extended intra prediction using wide-angle intra direction, position-dependent intra prediction combination (PDPC), and 4-tap intra interpolation. The main focus of this disclosure is to improve the existing high-level syntax design in the VVC standard. The relevant background knowledge is elaborated in detail in the following sections.
[0048] Like HEVC, VVC uses a bitstream structure based on Network Abstraction Layer (NAL) units. The coded bitstream is partitioned into multiple NAL units that should be smaller than the maximum transmission unit size when transmitted over a lossy packet network. Each NAL unit consists of a NAL unit header followed by a NAL unit payload. There are two conceptual categories of NAL units. Video Codec Layer (VCL) NAL units containing coded sample data, such as coded slice NAL units, whereas non-VCL NAL units containing metadata that typically belongs to more than one coded picture, or in cases where association with a single coded picture would be meaningless, such as parameter set NAL units, or in cases where the information is not needed for the decoding process, such as SEI NAL units.
[0049] In VVC, a two-byte NAL unit header is introduced, and it is expected that this design is sufficient to support future extensions. The syntax and associated semantics of the NAL unit header in the current VVC draft specification are illustrated in Table 1 and Table 2, respectively. How to read Table 1 is illustrated in the appendix of this invention, which can also be found in the VVC specification.
[0050] Table 1. NAL unit header syntax
[0051]
[0052] Table 2. NAL unit header semantics
[0053]
[0054] Table 3. NAL unit type codes and NAL unit type categories
[0055]
[0056]
[0057] VVC inherits the parameter set concept of HEVC with some modifications and additions. Parameter sets can be part of the video bitstream or can be received by the decoder through other means (including out-of-band transmission using a reliable channel, hard encoding and decoding in the encoder and decoder, etc.). Parameter sets contain identifiers referenced directly or indirectly from the slice header, as discussed in more detail later. The reference process is called "activation". Depending on the parameter set type, activation occurs per picture or per sequence. Among other reasons, the concept of activation by reference is introduced because implicit activation cannot be performed by virtue of the location of information in the bitstream (such as other syntax elements common to the video codec) in the case of out-of-band transmission.
[0058] Video parameter sets (VPS) are introduced to convey information applicable to multiple layers and sub-layers. VPS is introduced to address these shortcomings and enable a clean and scalable high-level design of a multi-layer codec. Regardless of whether each layer of a given video sequence has the same or different sequence parameter sets (SPS), they all reference the same VPS. The syntax and associated semantics of video parameter sets in the current VVC draft specification are illustrated in Table 4 and Table 5, respectively. How to read Table 4 is illustrated in the appendix of the present invention, which can also be found in the VVC specification.
[0059] Table 4. Video parameter set RBSP syntax
[0060]
[0061]
[0062]
[0063]
[0064] Table 5 Video parameter set RBSP language
[0065]
[0066]
[0067]
[0068]
[0069]
[0070]
[0071]
[0072]
[0073]
[0074] In VVC, the SPS contains information applicable to all slices of a coded video sequence. A coded video sequence starts with an instantaneous decoding refresh (IDR) picture or a BLA picture or a CRA picture that is the first picture in the bitstream, and includes all subsequent pictures that are not IDR or BLA pictures. The bitstream consists of one or more coded video sequences. The content of the SPS can be roughly divided into six categories: 1) self-reference (its own ID); 2) decoder operation point related information (profile, level, picture size, number of sub-layers, etc.); 3) enablement flags for certain tools within the profile, and associated codec tool parameters when the tools are enabled; 4) information that defines the flexibility of structure and transform coefficient coding and decoding; 5) temporal scalability control; and 6) visual usability information (VUI) including HRD information. The syntax and associated semantics of the sequence parameter set in the current VVC draft specification are illustrated in Tables 6 and 7, respectively. How to read Table 6 is illustrated in the appendix of the present invention, which can also be found in the VVC specification.
[0075] Table 6. Sequence parameter set RBSP syntax
[0076]
[0077]
[0078]
[0079]
[0080]
[0081]
[0082]
[0083] Table 7. Sequence parameter set RBSP semantics
[0084]
[0085]
[0086]
[0087]
[0088]
[0089]
[0090]
[0091]
[0092]
[0093]
[0094]
[0095]
[0096]
[0097]
[0098]
[0099]
[0100]
[0101]
[0102]
[0103] The picture parameter set (PPS) of VVC contains such information that can change between pictures. The PPS includes information roughly equivalent to a portion of the PPS in HEVC, including: 1) self-reference; 2) initial picture control information, such as the initial quantization parameter (QP), the number of flags indicating the use or presence of certain tools, or control information in the slice header; and 3) tile information. The syntax and associated semantics of the sequence parameter set in the current VVC draft specification are illustrated in Table 8 and Table 9, respectively. How to read Table 8 is illustrated in the appendix of the present invention, which can also be found in the VVC specification.
[0104] Table 8. Picture parameter set RBSP syntax
[0105]
[0106]
[0107]
[0108]
[0109]
[0110] Table 9. Picture parameter set RBSP semantics
[0111]
[0112]
[0113]
[0114]
[0115]
[0116]
[0117]
[0118]
[0119]
[0120]
[0121] The slice header contains information that may vary between slices and such picture related information that is relatively small or only relevant for a specific slice or picture type. The size of the slice header may be significantly larger than the PPS, especially when there are tile or wavefront entry point offsets in the slice header and RPS, prediction weights or reference picture list modifications are explicitly signaled. The syntax and associated semantics of the sequence parameter sets in the current VVC draft specification are illustrated in Table 10 and Table 11, respectively. How to read Table 10 is illustrated in the Appendix section of the present invention, which can also be found in the VVC specification.
[0122] Table 10. Picture header structure syntax
[0123]
[0124]
[0125]
[0126]
[0127]
[0128] Table 11. Image header structure semantics
[0129]
[0130]
[0131]
[0132]
[0133]
[0134]
[0135]
[0136]
[0137]
[0138]
[0139]
[0140] Improvements to syntax elements
[0141] In the current VVC, when similar syntax elements exist for intra prediction and inter prediction, respectively, syntax elements related to inter prediction are defined before syntax elements related to intra prediction in some places. Such an order may not be preferred considering the fact that intra prediction is allowed in all picture / slice types but inter prediction is not allowed. From a standardization perspective, it would be beneficial to always define intra prediction related syntax before syntax for inter prediction.
[0142] It is also observed that in current VVC, some syntax elements that are highly related to each other are defined in a sprawling manner at different places. From a standardization perspective, it would also be beneficial to group some syntax together.
[0143] Proposed method
[0144] Methods are provided to simplify and / or further improve existing designs of high-level grammars. It should be noted that the invented methods can be applied independently or in combination.
[0145] Grouping partition constraint syntax elements by prediction type
[0146] In the present disclosure, it is proposed to rearrange the syntax elements so that the intra-prediction related syntax elements are defined before the inter-prediction related syntax elements. According to the present disclosure, the partition constraint syntax elements are grouped by prediction type, where the intra-prediction related ones are first and the inter-prediction related ones are later. In one embodiment, the order of the partition constraint syntax elements in the SPS is consistent with the order of the partition constraint syntax elements in the picture header. An example of the decoding process for the VVC draft is illustrated in Table 12 below. Changes to the VVC draft are shown using bold and italic fonts.
[0147] Table 12. Proposed sequence parameter set RBSP syntax
[0148]
[0149]
[0150] Grouping dual-tree chroma syntax elements
[0151] In the present disclosure, it is proposed to group syntax elements related to dual-tree chroma types. In one embodiment, the partition constraint syntax elements of dual-tree chroma in the SPS should be signaled together in the dual-tree chroma case. An example of the decoding process for the VVC draft is illustrated in Table 13 below. Changes to the VVC draft are shown using bold and italic fonts.
[0152] Table 13. Proposed sequence parameter set RBSP syntax
[0153]
[0154]
[0155] If it is also considered that the intra prediction related syntax is defined before the inter prediction related syntax, according to the method of the present disclosure, another example of the decoding process with respect to the VVC draft is illustrated in the following Table 14. Changes to the VVC draft are shown using bold and italic fonts.
[0156] Table 14. Proposed sequence parameter set RBSP syntax
[0157]
[0158]
[0159] Conditionally signaling inter prediction related syntax elements
[0160] As mentioned in the earlier description, according to the current VVC, intra prediction is allowed in all picture / slice types but inter prediction is not allowed. According to the present disclosure, it is proposed to add a flag in the VVC syntax at a specific codec level to indicate whether inter prediction is allowed in a sequence, picture and / or slice. In the case where inter prediction is not allowed, inter prediction related syntax is not signaled at the corresponding codec level (e.g., sequence, picture and / or slice level).
[0161] It is also proposed to add a flag in the VVC syntax at a specific codec level to indicate whether inter-frame slices such as P slices and B slices are allowed in a sequence, picture and / or slice. If inter-frame slices are not allowed, inter-frame slice related syntax is not signaled at the corresponding codec level (e.g., sequence, picture and / or slice level).
[0162] In the following sections, some examples are given based on the proposed flag for allowing inter-frame slices. And, the proposed flag for allowing inter-frame prediction can be used in a similar way.
[0163] When the proposed flags to allow inter-frame strips are added at different levels, these flags can be signaled in a hierarchical manner. When the signaled flag at a higher level indicates that inter-frame strips are not allowed, the flags at lower levels do not need to be signaled and can be inferred to be 0 (which means that inter-frame strips are not allowed).
[0164] In one or more examples, a flag is added to the SPS to indicate whether inter-slices are allowed in encoding and decoding the current video sequence. If not allowed, inter-slice related syntax elements are not signaled in the SPS. An example of the decoding process for the VVC draft is illustrated in Table 15 below. Changes to the VVC draft are shown using bold and italic fonts. It should be noted that there are syntax elements other than those introduced in this example. For example, there are many inter-slice (or inter-prediction tool) related syntax elements, such as etc.; there are also syntax elements related to the reference picture list, such as Etc. All these syntax elements related to inter prediction can be selectively controlled by the proposed flags.
[0165] Table 15. Proposed sequence parameter set RBSP syntax
[0166]
[0167]
[0168]
[0169]
[0170]
[0171]
[0172]
[0173]
[0174] 7.4.3.3 Sequence parameter set RBSP semantics
[0175] equal to 0 specifies that all coded slices of the video sequence have slice_type equal to 2 (which indicates that the coded slice is an I slice). sps_inter_slice_allowed_flag equal to 1 specifies that one or more coded slices with slice_type equal to 0 (which indicates that the coded slice is a P slice) or 1 (which indicates that the coded slice is a B slice) may or may not be present in the video sequence.
[0176] In another example, according to the method of the present disclosure, a flag is added to the picture parameter set PPS to indicate whether inter-frame slices are allowed in encoding and decoding pictures associated with this PPS. If not allowed, the selected inter-frame prediction related syntax elements are not signaled in the PPS.
[0177] In yet another example, according to the method of the present disclosure, a flag that allows inter-frame strips can be signaled in a hierarchical manner. A flag is added to the SPS to indicate whether inter-frame strips are allowed in encoding and decoding pictures associated with this SPS, such as sps_inter_slice_allowed_flag. When sps_inter_slice_allowed_flag is equal to 0 (which means that inter-frame strips are not allowed), the inter-frame strips allowed flag in the picture header can be omitted when signaling and inferred to be 0. An example of the decoding process for the VVC draft is illustrated in Table 16 below. Changes to the VVC draft are shown using bold and italic fonts.
[0178] Table 16. Proposed sequence parameter set RBSP syntax
[0179]
[0180] 7.4.3.7 Image header structure semantics
[0181] ph_inter_slice_allowed_flag equal to 1 specifies that one or more coded slices with slice_type equal to 0 or 1 may or may not be present in the picture. When not present, the value of ph_inter_slice_allowed_flag is inferred to be equal to 0.
[0182] Figure 4 A method for decoding a video signal according to the present disclosure is shown. For example, the method can be applied to a decoder.
[0183] In step 410, the decoder may receive at least one VVC syntax flag. The at least one VVC syntax flag may include a first VVC syntax flag indicating whether inter-frame prediction is allowed in a corresponding coding level.
[0184] In step 412, the decoder may receive an inter-correlated syntax element in response to the syntax element indicating that inter prediction is allowed.
[0185] In step 414, the decoder may obtain a first reference picture I associated with a video block in the bitstream. (0) and the second reference picture I (1) In the display order, the first reference picture I (0) Before the current picture and the second reference picture I (1) After the current image.
[0186] In step 416, the decoder can obtain the first reference picture I (0) The reference block in the video block obtains the first prediction sample I (0) (i, j). i and j represent the coordinates of a sample point in the current image.
[0187] In step 418, the decoder can obtain the second reference picture I (1) The reference block in the video block obtains the second prediction sample I (1) (i, j).
[0188] In step 420, the decoder may generate a decoded image based on the at least one VVC syntax flag, the inter-frame related syntax element, the first prediction sample I (0) (i, j) and the second predicted sample point I (1) (i, j) to obtain the bidirectional prediction samples.
[0189] Grouping inter-frame related syntax elements
[0190] In the present disclosure, it is proposed to rearrange syntax elements so that inter-prediction related syntax elements are grouped according to VVC syntax at a certain codec level (e.g., sequence, picture and / or slice level). According to the present disclosure, it is proposed to rearrange syntax elements related to inter-frame slices in a sequence parameter set (SPS). An example of the decoding process for the VVC draft is illustrated in Table 17 below. Changes to the VVC draft are shown below. Added parts are shown using bold and italic fonts, while deleted parts are shown in strikethrough fonts.
[0191] Table 17. Proposed sequence parameter set RBSP syntax
[0192]
[0193]
[0194]
[0195]
[0196]
[0197]
[0198]
[0199]
[0200] Figure 5 A method for decoding a video signal according to the present disclosure is shown. For example, the method can be applied to a decoder.
[0201] In step 510, the decoder may receive an arranged partition constraint syntax element at the SPS level. The arranged partition constraint syntax element is arranged so that inter-prediction related syntax elements are grouped according to the VVC syntax at the codec level.
[0202] In step 512, the decoder may obtain a first reference picture I associated with a video block in the bitstream. (0) and the second reference picture I (1) In the display order, the first reference picture I (0) Before the current picture and the second reference picture I (1) After the current image.
[0203] In step 514, the decoder can obtain the first reference picture I (0) The reference block in the video block obtains the first prediction sample I (0) (i, j). i and j represent the coordinates of a sample point in the current image.
[0204] In step 516, the decoder can obtain the second reference picture I (1) The reference block in the video block obtains the second prediction sample I (1) (i, j).
[0205] In step 518, the decoder may arrange the partition constraint syntax element, the first prediction sample I (0) (i, j) and the second predicted sample point I (1) (i, j) to obtain the bidirectional prediction samples.
[0206] Another example of a decoding process for the VVC draft is illustrated in Table 18 below. Changes to the VVC draft are shown below. Added portions are shown using bold and italic fonts, while deleted portions are shown in strikethrough fonts.
[0207] Table 18. Proposed sequence parameter set RBSP syntax
[0208]
[0209]
[0210]
[0211]
[0212]
[0213]
[0214]
[0215]
[0216] According to the present disclosure, it is also proposed to add a flag in the VVC syntax at a specific codec level to indicate whether inter-frame strips such as P strips and B strips are allowed in a sequence, picture and / or slice. If inter-frame strips are not allowed, the inter-frame strip-related syntax is not signaled at the corresponding codec level (e.g., sequence, picture and / or strip level). In one example, according to the method of the present disclosure, a flag sps_inter_slice_allowed_flag is added to the SPS to indicate whether inter-frame strips are allowed in encoding and decoding the current video sequence. If not allowed, inter-frame strip-related syntax elements are not signaled in the SPS. An example of the decoding process for the VVC draft is illustrated in Table 19 below. The added parts are shown in bold and italic fonts, and the deleted parts are shown in strikethrough fonts.
[0217] Table 19. Proposed sequence parameter set RBSP syntax
[0218]
[0219]
[0220]
[0221]
[0222]
[0223]
[0224]
[0225]
[0226] Another example of a decoding process for the VVC draft is illustrated in Table 20 below. Changes to the VVC draft are shown below. Added portions are shown using bold and italic fonts, while deleted portions are shown in strikethrough fonts.
[0227] Table 20. Proposed sequence parameter set RBSP syntax
[0228]
[0229]
[0230]
[0231]
[0232]
[0233]
[0234]
[0235]
[0236] Figure 6 A method for decoding a video signal according to the present disclosure is shown. For example, the method can be applied to a decoder.
[0237] In step 610 , the decoder may receive a bitstream including a VPS, an SPS, a PPS, a picture header, and a slice header for encoded video data.
[0238] In step 612, the decoder may decode the VPS.
[0239] In step 614, the decoder may decode the SPS and obtain the permutation partition constraint syntax element at the SPS level.
[0240] In step 616, the decoder may decode the PPS.
[0241] In step 618, the decoder may decode the picture header.
[0242] In step 620, the decoder may decode the slice header.
[0243] In step 622, the decoder may decode the video data based on the VPS, SPS, PPS, picture header, and slice header.
[0244] The above method can be implemented using a device including one or more circuits, including an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components. The device can use these circuits in combination with other hardware or software components for performing the above method. Each module, submodule, unit or subunit disclosed above can be implemented at least in part using the one or more circuits.
[0245] Figure 7 A computing environment 710 is shown coupled to a user interface 760. The computing environment 710 may be part of a data processing server. The computing environment 710 includes a processor 720, a memory 740, and an I / O interface 750.
[0246] The processor 720 generally controls the overall operation of the computing environment 710, such as operations associated with display, data acquisition, data communication, and image processing. The processor 720 may include one or more processors to execute instructions to perform all or some steps in the above method. Moreover, the processor 720 may include one or more modules that facilitate the interaction between the processor 720 and other components. The processor may be a central processing unit (CPU), a microprocessor, a single-chip microcomputer, a GPU, etc.
[0247] The memory 740 is configured to store various types of data to support the operation of the computing environment 710. The memory 740 may include predetermined software 742. Examples of such data include instructions for any application or method operating on the computing environment 710, video data sets, image data, etc. The memory 740 may be implemented using any type of volatile or non-volatile memory device or combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.
[0248] I / O interface 750 provides an interface between processor 720 and peripheral interface modules (such as keyboard, click wheel, buttons, etc.). Buttons may include but are not limited to a home button, a start scan button, and a stop scan button. I / O interface 750 may be coupled to an encoder and a decoder.
[0249] In some embodiments, a non-transitory computer-readable storage medium is also provided, which includes a plurality of programs, which are included in the memory 740 and can be executed by the processor 720 in the computing environment 710 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
[0250] The non-transitory computer-readable storage medium stores a plurality of programs for execution by a computing device having one or more processors, wherein the plurality of programs, when executed by the one or more processors, enable the computing device to perform the above-mentioned motion prediction method.
[0251] In some embodiments, the computing environment 710 may be implemented using one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), graphics processing units (GPUs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.
[0252] Other examples of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any changes, uses or adaptations of the disclosure made in accordance with its general principles, including deviations from the present disclosure within known or customary practices in the art. It is intended that the specification and examples be considered exemplary only.
[0253] It should be understood that the present disclosure is not limited to the exact examples described above and shown in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof.
Claims
1. A method for decoding a video signal, comprising: The inter-frame prediction related syntax elements at the sequence parameter set SPS level are received by the decoder, wherein the inter-frame prediction related syntax elements are sequentially arranged at the codec level, wherein the inter-frame prediction related syntax elements include a sps_weighted_pred_flag flag, a sps_weighted_bipred_flag flag, a sps_mmvd_enabled_flag flag, a six_minus_max_num_merge_cand syntax element, and a log2_parallel_merge_level_minus2 syntax element that are sequentially arranged, the sps_weighted_pred_flag flag is equal to 1 to specify that weighted prediction can be applied to the P slice referencing the SPS, the sps_weighted_pred_flag flag is equal to 0 to specify that the weighted prediction is not applied to the P slice referencing the SPS, the sps_weighted_bipred_flag flag is equal to 1 to specify that explicit weighted prediction can be applied to the B slice referencing the SPS, and the sps_w The eighted_bipred_flag flag is equal to 0 to specify that the explicit weighted prediction is not applied to the B slice of the referenced SPS, the sps_mmvd_enabled_flag flag is equal to 1 to specify that the merge mode with motion vector difference is enabled, the sps_mmvd_enabled_flag flag is equal to 0 to specify that the merge mode with motion vector difference is disabled, the six_minus_max_num_merge_cand syntax element specifies the maximum number of merge motion vector prediction candidates supported in the SPS subtracted from 6, the log2_parallel_merge_level_minus2 syntax element plus 2 specifies the value of the variable Log2ParMrgLevel, the variable Log2ParMrgLevel is used in at least one of the following processes: the process of obtaining spatial merge candidates, the process of obtaining motion vectors and reference indices in sub-block merge mode, and the call of the process of controlling the update of the history-based motion vector prediction value list, and wherein the inter-frame prediction related syntax elements are arranged in the following way: Signaling said sps_weighted_pred_flag flag; signaling said sps_weighted_bipred_flag flag; Signaling the sps_mmvd_enabled_flag flag; Setting the value of the six_minus_max_num_merge_cand syntax element; and Setting the value of the log2_parallel_merge_level_minus2 syntax element; obtaining, by the decoder, a first reference picture and a second reference picture associated with a video block in a bitstream, wherein the first reference picture precedes a current picture and the second reference picture follows the current picture in display order; Obtaining, by the decoder, a first prediction sample of the video block from the first reference picture; obtaining, by the decoder, a second prediction sample of the video block from the second reference picture; and The decoder obtains a bidirectional prediction sample based on at least one of the inter-frame prediction related syntax elements, the first prediction sample, and the second prediction sample, Wherein, the method further comprises: After the log2_parallel_merge_level_minus2 syntax element, a sps_isp_enabled_flag flag is received, the sps_isp_enabled_flag flag being equal to 1 to specify that intra prediction with sub-partitions is enabled, and the sps_isp_enabled_flag flag being equal to 0 to specify that intra prediction with sub-partitions is disabled.
2. The method of claim 1, further comprising: The sps_wpp_entry_point_offsets_present_flag flag and the log2_max_pic_order_cnt_lsb_minus4 syntax element are continuously received, wherein the sps_wpp_entry_point_offsets_present_flag flag is equal to 1 to specify that the signaling of the entry point offset of the CTU row can be present in the slice header of the picture referencing the SPS, the sps_wpp_entry_point_offsets_present_flag flag is equal to 0 to specify that the signaling of the entry point offset of the CTU row is not present in the slice header of the picture referencing the SPS, and the log2_max_pic_order_cnt_lsb_minus4 syntax element specifies the value of the variable MaxPicOrderCntLSb used in the decoding process for the picture order count, the variable MaxPicOrderCntLSb=2 (lo g 2 _ max _ pic _ order _ cnt _ lsb _ minus4+4) .
3. The method of claim 2, wherein receiving the inter prediction related syntax element comprises: After the log2_max_pic_order_cnt_lsb_minus4 syntax element, the sps_weighted_pred_flag flag is received.
4. A computing device comprising: one or more processors; as well as A non-transitory computer-readable storage medium storing instructions executable by the one or more processors, wherein the one or more processors are configured to perform the method of any one of claims 1-3.
5. A non-transitory computer-readable storage medium storing a plurality of programs for execution by a computing device having one or more processors and a bit stream to be decoded by the method of any one of claims 1 to 3, wherein: The plurality of programs, when executed by the one or more processors, cause the computing device to perform the method of any one of claims 1-3.
6. A computer program product, comprising a plurality of programs for execution by a computing device having one or more processors, wherein when the plurality of programs are executed by the one or more processors, the computing device receives a bit stream and executes the method according to any one of claims 1 to 3 based on the bit stream.
