Scaling window in video codec

Abstract

An example method of video processing includes performing a conversion between a current picture of a video and a codec representation of the video. The current picture includes a first window having dimensions of W×H samples, where W and H are integers. A reference picture associated with the current picture includes a second window having dimensions of W'×H' samples, where W' and H' are integers. Samples in the first window and the second window are subject to the same processing rules during the conversion, and W, H, W', and H' satisfy constraints.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on International Patent Application No. PCT / CN2020 / 116468, filed on September 21, 2020, which claims priority to and the benefit of International Patent Application No. PCT / CN2019 / 106671, filed on September 19, 2019, and International Patent Application No. PCT / CN2019 / 109837, filed on October 5, 2019. All of the above patent applications are hereby incorporated by reference in their entirety. Technical Field

[0003] This patent document relates to video encoding and decoding technology, equipment and systems. Background Art

[0004] Currently, efforts are underway to improve the performance of current video codec technologies to provide better compression rates or to provide video encoding and decoding schemes that allow for lower complexity or parallelized implementations. Several new video codec tools have recently been proposed by industry experts and are currently being tested to determine their effectiveness. Summary of the Invention

[0005] Devices, systems, and methods related to digital video coding and decoding, and in particular, to the management of motion vectors, are described. The methods described can be applied to existing video coding standards (e.g., High Efficiency Video Codec (HEVC) or Universal Video Codec) as well as future video coding standards or video codecs.

[0006] In one representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes performing a conversion between a current picture of a video and a codec representation of the video. The current picture includes a first window having dimensions of W×H samples, where W and H are integers. A reference picture associated with the current picture includes a second window having dimensions of W'×H' samples, where W' and H' are integers. Samples in the first window and the second window are subject to the same processing rules during the conversion, and W, H, W', and H' satisfy constraints.

[0007] In another representative aspect, the disclosed technology may be used to provide a method for video processing. The method includes performing a conversion between a current picture of a video and a codec representation of the video. During the conversion, an offset value for determining a window of the current picture is signaled in the codec representation with N pixel precision, where N is a positive integer greater than 1.

[0008] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes performing a conversion between a video including a first video unit and a second video unit and a codec representation of the video. A first set of offset values ​​for a first window of the first video unit and a second set of offset values ​​for a second window of the second video unit are determined according to rules associated with dimensions of the first video unit and dimensions of the second video unit, wherein samples in the first window and the second window are subject to the same processing rules during the conversion.

[0009] In another representative aspect, the disclosed technology may be used to provide a method for video processing. The method includes determining, for a conversion between a current picture of a video and a codec representation of the video, a position of a reference sample of a reference picture associated with the current picture based on a top left position of a window of pictures, wherein the pictures include at least the current picture or the reference picture. The window is subjected to a processing rule during the conversion. The method also includes performing the conversion based on the determination.

[0010] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes performing a conversion between a current picture of a video and a codec representation of the video. The current picture includes a first window having dimensions of W×H samples, where W and H are integers. A reference picture associated with the current picture includes a second window having dimensions of W'×H' samples, where W' and H' are integers, and the upper left position of the second window is represented as (X0, Y0). Samples in the first window and the second window are subject to the same processing rules during the conversion. The integer portion of the coordinates of the reference sample is constrained to be within a range determined in part based on the first window or the second window.

[0011] In another representative aspect, the disclosed technology may be used to provide a method for video processing, the method comprising performing a conversion between a current video block and a codec representation of the current video block, wherein, during the conversion, if a resolution and / or size of a reference picture is different from the resolution and / or size of the current video block, applying the same interpolation filter to a group of adjacent or non-adjacent samples predicted using the current video block.

[0012] In another representative aspect, the disclosed technology may be used to provide a method for video processing, the method comprising performing a conversion between a current video block and a codec representation of the current video block, wherein, during the conversion, if a resolution and / or size of a reference picture is different from the resolution and / or size of the current video block, only blocks predicted using the current video block are allowed to use integer-valued motion information associated with the current block.

[0013] In another representative aspect, the disclosed technology can be used to provide another method for video processing. The method includes performing a conversion between a current video block and a codec representation of the current video block, wherein, during the conversion, if a resolution and / or size of a reference picture is different from the resolution and / or size of the current video block, applying an interpolation filter to derive a block predicted using the current video block, and wherein the interpolation filter is selected based on a rule.

[0014] In another representative aspect, the disclosed technology can be used to provide another method for video processing, the method comprising performing a conversion between a current video block and a codec representation of the current video block, wherein during the conversion, if a resolution and / or size of a reference picture is different from the resolution and / or size of the current video block, selectively applying a deblocking filter, wherein a strength of the deblocking filter is set according to a rule related to the resolution and / or size of the reference picture relative to the resolution and / or size of the current video block.

[0015] In another representative aspect, the disclosed technology can be used to provide another method for video processing, the method comprising performing a conversion between a current video block and a codec representation of the current video block, wherein during the conversion, a reference picture of the current video block is resampled according to a rule based on a dimension of the current video block.

[0016] In another representative aspect, the disclosed technology can be used to provide another method for video processing. The method includes performing a conversion between a current video block and a codec representation of the current video block, wherein during the conversion, use of codec tools for the current video block is selectively enabled or disabled depending on a resolution / size of a reference picture of the current video block relative to a resolution / size of the current video block.

[0017] In another representative aspect, the disclosed technology can be used to provide another method for video processing. The method includes performing a conversion between a plurality of video blocks and a codec representation of the plurality of video blocks, wherein during the conversion, a first consistency window for a first video block and a second consistency window for a second video block are defined, and wherein a ratio of a width and / or a height of the first consistency window to the second consistency window is based on a rule based at least on a conforming bitstream.

[0018] In another representative aspect, the disclosed technology can be used to provide another method for video processing. The method includes performing a conversion between a plurality of video blocks and codec representations of the plurality of video blocks, wherein during the conversion, a first conformance window is defined for the first video block and a second conformance window is defined for the second video block, and wherein a ratio of a width and / or a height of the first conformance window to the second conformance window is based on a rule based at least on a conformance bitstream.

[0019] Furthermore, in a representative aspect, an apparatus in a video system is disclosed, the apparatus comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement any one or more of the disclosed methods.

[0020] In one representative aspect, a video decoding apparatus is disclosed that includes a processor configured to implement the methods disclosed herein.

[0021] In one representative aspect, a video encoding apparatus is disclosed that includes a processor configured to implement the methods disclosed herein.

[0022] Furthermore, a computer program product stored on a non-transitory computer-readable medium is disclosed, the computer program product comprising program code for performing any one or more of the disclosed methods.

[0023] The above and other aspects and features of the disclosed technology are described in more detail in the drawings, the description, and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Examples of sub-block motion vectors (VSB) and motion vector differences are shown.

[0025] Figure 2 An example of a 16x16 video block divided into 16 4x4 regions is shown.

[0026] Figure 3A Examples of specific locations in the sample points are shown.

[0027] Figure 3B Another example of a specific location in a spot is shown.

[0028] Figure 3C Yet another example of a specific position in a sample point is shown.

[0029] Figure 4A An example of the position of the current sample point and its reference sample point is shown.

[0030] Figure 4B Another example of the positions of the current sample and its reference sample is shown.

[0031] Figure 5 is a block diagram of an example hardware platform for implementing the visual media decoding or visual media encoding techniques described in this document.

[0032] Figure 6 A flow chart illustrating an example method for video encoding and decoding is shown.

[0033] Figure 7 is a block diagram of an example video processing system in which the disclosed technology may be implemented.

[0034] Figure 8 is a block diagram illustrating an example video encoding and decoding system.

[0035] Figure 9 is a block diagram illustrating an encoder according to some embodiments of the present disclosure.

[0036] Figure 10 is a block diagram illustrating a decoder according to some embodiments of the present disclosure.

[0037] Figure 11 is a flowchart representation of a video processing method according to the present technology.

[0038] Figure 12 is a flowchart representation of another video processing method according to the present technology.

[0039] Figure 13 is a flowchart representation of another video processing method according to the present technology.

[0040] Figure 14 is a flowchart representation of another video processing method according to the present technology.

[0041] Figure 15 is a flowchart representation of another video processing method according to the present technology. DETAILED DESCRIPTION

[0042] 1. Video Codec in HEVC / H.265

[0043] Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, ISO / IEC developed MPEG-1 and MPEG-4 Vision, and the two organizations jointly developed H.262 / MPEG-2 Video, H.264 / MPEG-4 Advanced Video Codec (AVC), and H.265 / HEVC. Since H.262, video codec standards have been based on a hybrid video codec architecture that uses temporal prediction plus transform coding. To explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and incorporated them into reference software called the Joint Exploration Model (JEM). In April 2018, a Joint Video Experts Team (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) was established to develop the VVC standard with the goal of reducing the bit rate by 50% compared to HEVC.

[0044] 2. Summary

[0045] 2.1 Adaptive Resolution Change (ARC)

[0046] AVC and HEVC do not have the ability to change resolution without having to introduce IDR or intra random access point (IRAP) pictures; such a capability can be denoted as Adaptive Resolution Change (ARC). There are several use cases or application scenarios that would benefit from the ARC feature, including the following:

[0047] - Rate adaptation in video calling and conferencing: In order to adapt the encoded video to changing network conditions, when the network conditions deteriorate and the available bandwidth becomes lower, the encoder can adapt to the bandwidth by encoding lower resolution pictures. Currently, changing the picture resolution can only be done after the IRAP picture; this has some problems. An IRAP picture at reasonable quality will be much larger than an inter-frame codec picture and will be correspondingly more complex to decode: this will cost time and resources. This is also a problem if the resolution change is requested by the decoder for load reasons. It may also destroy the low-latency buffering condition, forcing the audio to resynchronize and the end-to-end delay of the stream will increase, at least temporarily. This may result in a poor user experience.

[0048] Active speaker changes in multi-party video conferencing: For multi-party video conferencing, it is common to show the active speaker at a larger video size than the video for the remaining conference participants. When the active speaker changes, the image resolution for each participant may also need to be adjusted. The need for ARC becomes more important when such changes in the active speaker occur frequently.

[0049] - Fast start in streaming: For streaming applications, it is common for the application to buffer up to a certain length of decoded pictures before starting to display them. Starting the bitstream with a lower resolution will allow the application to have enough pictures in the buffer to start displaying faster.

[0050] Adaptive stream switching in streaming: Dynamic Adaptive Streaming over HTTP (DASH) includes a feature called @mediaStreamStructureId. This enables switching between different representations at open GOP random access points with non-decodable leading pictures (e.g., CRA pictures associated with RASL pictures in HEVC). When two different representations of the same video have different bitrates but the same spatial resolution, and they have the same value of @mediaStreamStructureId, switching between these two representations can be performed at the CRA pictures associated with the RASL pictures, and the RASL pictures associated with the switch at the CRA pictures can be decoded with acceptable quality, thus achieving seamless switching. With ARC, the @mediaStreamStructureId feature will also be available for switching between DASH representations with different spatial resolutions.

[0051] ARC is also known as dynamic resolution conversion.

[0052] ARC can be viewed as a special case of Reference Picture Resampling (RPR) (eg, H.263 Annex P).

[0053] 2.2 Reference Picture Resampling in H.263 Appendix P

[0054] This mode describes an algorithm that warps the reference picture before using it for prediction. It can be used to resample reference pictures that have a different source format than the picture being predicted. It can also be used for global motion estimation or estimation of rotational motion by warping the shape, size and position of the reference picture. The syntax includes the warping parameters to be used as well as the resampling algorithm. The simplest level of operation for the reference picture resampling mode is an implicit factor 4 resampling, where only FIR filters need to be applied to the upsampling and downsampling processes. In this case, when the size of the new picture (indicated in the picture header) is different from the size of the previous picture, no additional signaling overhead is required because its purpose is understood.

[0055] 2.3 Consistency Window in VVC

[0056] The consistency window in VVC defines a rectangle. Samples within the consistency window belong to the image of interest. Samples outside the consistency window are discarded during output.

[0057] When a consistency window is applied, the scaling ratio in the RPR is derived based on the consistency window.

[0058] Picture Parameter Set RBSP Syntax

[0059]

[0060] pic_width_in_luma_samples specifies the width of each decoded picture referring to the PPS in units of luma samples. pic_width_in_luma_samples shall not be equal to 0, shall be an integer multiple of Max(8, MinCbSizeY), and shall be less than or equal to pic_width_max_in_luma_samples.

[0061] When subpics_present_flag is equal to 1, the value of pic_width_in_luma_samples shall be equal to pic_width_max_in_luma_samples.

[0062] pic_height_in_luma_samples specifies the height of each decoded picture referring to the PPS in units of luma samples. pic_height_in_luma_samples shall not be equal to 0 and shall be an integer multiple of Max(8, MinCbSizeY) and shall be less than or equal to pic_height_max_in_luma_samples.

[0063] When subpics_present_flag is equal to 1, the value of pic_height_in_luma_samples shall be equal to pic_height_max_in_luma_samples.

[0064] Let refPicWidthInLumaSamples and refPicHeightInLumaSamples be the pic_width_in_luma_samples and pic_height_in_luma_samples, respectively, of the reference picture of the current picture that refers to this PPS. Bitstream conformance requirements that satisfy all of the following conditions:

[0065] –pic_width_in_luma_samples*2 should be greater than or equal to refPicWidthInLumaSamples.

[0066] –pic_height_in_luma_samples*2 should be greater than or equal to refPicHeightInLumaSamples.

[0067] –pic_width_in_luma_samples should be less than or equal to refPicWidthInLumaSamples*8.

[0068] –pic_height_in_luma_samples should be less than or equal to refPicHeightInLumaSamples*8.

[0069] conformance_window_flag equal to 1 indicates that the conformance cropping window offset parameter follows in the SPS. conformance_window_flag equal to 0 indicates that the conformance cropping window offset parameter is not present.

[0070] conf_win_left_offset, conf_win_right_offset, conf_win_top_offset, and conf_win_bottom_offset specify the picture samples in the CVS output from the decoding process, according to the rectangular area specified in picture coordinates for output. When conformance_window_flag is equal to 0, the values ​​of conf_win_left_offset, conf_win_right_offset, conf_win_top_offset, and conf_win_bottom_offset are inferred to be equal to 0.

[0071] The conforming cropping window contains luma samples with horizontal picture coordinates from SubWidthC*conf_win_left_offset to pic_width_in_luma_samples-(SubWidthC*conf_win_right_offset+1) and vertical picture coordinates from SubHeightC*conf_win_top_offset to pic_height_in_luma_samples-(SubHeightC*conf_win_bottom_offset+1), inclusive.

[0072] The value of SubWidthC*(conf_win_left_offset+conf_win_right_offset) should be less than pic_width_in_luma_samples, and the value of SubHeightC*(conf_win_top_offset+conf_win_bottom_offset) should be less than pic_height_in_luma_samples.

[0073] The variables PicOutputWidthL and PicOutputHeightL are derived as follows:

[0074] PicOutputWidthL=pic_width_in_luma_samples- (7-43)

[0075] SubWidthC*(conf_win_right_offset+conf_win_left_offset)PicOutputHeightL=pic_height_in_pic_size_units- (7-44)

[0076] SubHeightC*(conf_win_bottom_offset+conf_win_top_offset)

[0077] When ChromaArrayType is not equal to 0, the corresponding specified samples of the two chroma arrays are samples with picture coordinates (x / SubWidthC, y / SubHeightC), where (x, y) are the picture coordinates of the specified luma sample.

[0078] NOTE – The consistent crop window offset parameter applies only to output. All internal decoding processes will apply to the unconstrained The cropped image size.

[0079] Let ppsA and ppsB be any two PPSs that reference the same SPS. The bitstream conformance requirement is that when the values ​​of pic_width_in_luma_samples and pic_height_in_luma_samples of ppsA and ppsB are respectively the same, the values ​​of conf_win_left_offset, conf_win_right_offset, conf_win_top_offset, and conf_win_bottom_offset of ppsA and ppsB are respectively the same.

[0080] 2.4 Reference Picture Resampling (RPR)

[0081] In some embodiments, ARC is also referred to as reference picture resampling (RPR). With RPR, TMVP is disabled if the collocated picture has a different resolution than the current picture. Additionally, BDOF and DMVR are disabled when the reference picture has a different resolution than the current picture.

[0082] To handle general MC when the reference picture has a different resolution than the current picture, the interpolation part is defined as follows:

[0083] 8.5.6.3 Fractional Sample Interpolation Process

[0084] 8.5.6.3.1 Overview

[0085] The inputs to this process are:

[0086] – Luma position (xSb, ySb), specifies the top left sample of the current codec sub-block relative to the top left luma sample of the current picture,

[0087] -Variable sbWidth specifies the width of the current codec sub-block.

[0088] -Variable sbHeight specifies the height of the current codec sub-block.

[0089] – motion vector offset mvOffset,

[0090] – refined motion vector refMvLX,

[0091] – the selected reference picture sample array refPicLX,

[0092] – Half-sample interpolation filter index hpelIfIdx,

[0093] – bidirectional optical flow flag bdofFlag,

[0094] – The variable cIdx specifies the color component index of the current block.

[0095] The output of this process is:

[0096] – predSamplesLX, an array of (sbWidth+brdExtSize)x(sbHeight+brdExtSize) predicted sample values.

[0097] The prediction block border extension size brdExtSize is derived as follows:

[0098] brdExtSize=(bdofFlag||(inter_affine_flag[xSb][ySb]&&sps_affine_prof_enabled_flag))? 2:0(8-752)

[0099] Set the variable fRefWidth equal to the PicOutputWidthL of the reference picture measured in luma samples.

[0100] Set the variable fRefHeight equal to the PicOutputHeightL of the reference picture measured in luma samples.

[0101] Set the motion vector mvLX equal to (refMvLX-mvOffset).

[0102] – If cIdx is equal to 0, then the following applies:

[0103] - Define the scaling factor and its fixed-point representation as:

[0104] hori_scale_fp=((fRefWidth<<14)+(PicOutputWidthL>>1)) / PicOutputWidthL(8-753)

[0105] vert_scale_fp=((fRefHeight<<14)+(PicOutputHeightL>>1)) / PicOutputHeightL (8-754)

[0106] – Let (xIntL, yIntL) be the luma position in full sample units, and (xFracL, yFracL) be the offset in 1 / 16 sample units. These variables are used only in this clause to specify fractional sample positions within the reference sample array refPicLX.

[0107] – The upper left coordinate of the bounding block to be used for reference sample filling (xSbIntL ,ySbInt L ) is set equal to (xSb+(mvLX[0]>>4),ySb+(mvLX[1]>>4)).

[0108] – For each luma sample position (x L =0..sbWidth-1+brdExtSize,y L =0..sbHeight-1+brdExtSize), the corresponding predicted luminance sample value predSamplesLX[x L ][y L ]:

[0109] -Let (refxSb L ,refySb L ) and (refx L ,refy L ) is the luminance position pointed to by the motion vector (refMvLX[0], refMvLX[1]) given in units of 1 / 16 samples. The variable refxSb is derived as follows L 、refx L 、refySb L and refy L :

[0110] refxSb L =((xSb<<4)+refMvLX[0])*hori_scale_fp (8-755)

[0111] refx L =((Sign(refxSb)*((Abs(refxSb)+128)>>8)+x L *((hori_scale_fp+8)>>4))+32)>>6 (8-756)

[0112] refySb L =((ySb<<4)+refMvLX[1])*vert_scale_fp (8-757)

[0113] refy L =((Sign(refySb)*((Abs(refySb)+128)>>8)+yL*((vert_scale_fp+8)>>4))+32)>>6 (8-758)

[0114] – Derived variable xInt as followsL 、yInt L 、xFrac L and yFrac L :

[0115] xInt L =refx L >>4 (8-759)

[0116] yInt L =refy L >>4 (8-760)

[0117] xFrac L =refx L &15 (8-761)

[0118] yFrac L =refy L &15 (8-762)

[0119] – If bdofFlag is equal to true or (sps_affine_prof_enabled_flag is equal to true and inter_affine_flag[xSb][ySb] is equal to true), and one or more of the following conditions are true, then L +(xFrac L >>3)-1),yInt L +(yFrac L >>3)-1) and refPicLX are input by calling the luminance integer sample retrieval process specified in clause 8.5.6.3.3 to derive the predicted luminance sample value predSamplesLX[x L ][y L ].

[0120] –x L Equal to 0.

[0121] –x L Equal to sbWidth+1.

[0122] –y L Equal to 0.

[0123] –y L Equal to sbHeight+1.

[0124] – Otherwise, in the format of (xIntL-(brdExtSize>0?1:0),yIntL-(brdExtSize>0?1:0)), (xFracL,yFracL), (xSbInt L ,ySbIntL ), refPicLX, hpelIfIdx, sbWidth, sbHeight and (xSb, ySb) are input to derive the predicted luma sample value predSamplesLX[xL][yL] by calling the luma sample 8-tap interpolation filtering process specified in clause 8.5.6.3.2.

[0125] – Otherwise (cIdx is not equal to 0), then the following applies:

[0126] – Let (xIntC, yIntC) be the chroma position given in full sample units, and (xFracC, yFracC) be the offset given in 1 / 32 sample units. These variables are used only in this clause to specify regular fractional sample positions within the reference sample array refPicLX.

[0127] – Set the upper left coordinate (xSbIntC, ySbIntC) of the bounding block used for reference sample filling to be equal to ((xSb / SubWidthC)+(mvLX[0]>>5),(ySb / SubHeightC)+(mvLX[1]>>5)).

[0128] For each chroma sample position in the predicted chroma sample array predSamplesLX (xC = 0..sbWidth-1, yC = 0..sbHeight-1), the corresponding predicted chroma sample value predSamplesLX[xC][yC] is derived as follows:

[0129] –Let (refxSb C ,refySb C ) and (refx C ,refy C ) is the chroma position pointed to by the motion vector (mvLX[0], mvLX[1]) given in units of 1 / 32 samples. The variable refxSb is derived as follows C 、refySb C 、refx C and refy C :

[0130] refxSb C =((xSb / SubWidthC<<5)+mvLX[0])*hori_scale_fp (8-763)

[0131] refx C =((Sign(refxSb C )*((Abs(refxSb C)+256)>>9)+xC*((hori_scale_fp+8)>>4))+16)>>5 (8-764)

[0132] refySb C =((ySb / SubHeightC<<5)+mvLX[1])*vert_scale_fp (8-765)

[0133] refy C =((Sign(refySb C )*((Abs(refySb C )+256)>>9)+yC*((vert_scale_fp+8)>>4))+16)>> 5(8-766)

[0134] – Derived variable xInt as follows C 、yInt C 、xFrac C and yFrac C :

[0135] xInt C =refx C >>5 (8-767)

[0136] yInt C =refy C >>5 (8-768)

[0137] xFrac C =refy C &31 (8-769)

[0138] yFrac C =refy C &31 (8-770)

[0139] – Derives the predicted sample values ​​predSamplesLX[xC][yC] by invoking the procedure specified in clause 8.5.6.3.4 with (xIntC,yIntC), (xFracC,yFracC), (xSbIntC,ySbIntC), sbWidth, sbHeight and refPicLX as input.

[0140] 8.5.6.3.2 Luma Sample Interpolation Filtering Process

[0141] The inputs to this process are:

[0142] – Luminance position in full sample units (xInt L ,yIntL ),

[0143] – Luma position in fractional samples (xFrac L ,yFrac L ),

[0144] – Luminance position in units of full samples (xSbInt L ,ySbInt L ), specifies the upper left sample of the bounding block for reference sample filling relative to the upper left luma sample of the reference picture,

[0145] – Luminance reference sample array refPicLX L ,

[0146] – Half-sample interpolation filter index hpelIfIdx,

[0147] –Variable sbWidth specifies the width of the current sub-block.

[0148] –Variable sbHeight specifies the height of the current sub-block.

[0149] – Luma position (xSb, ySb), specifies the top left sample of the current sub-block relative to the top left luma sample of the current picture,

[0150] The output of this process is the predicted luminance sample value predSampleLX L

[0151] The variables shift1, shift2, and shift3 are derived as follows:

[0152] – Set the variable shift1 to be equal to Min(4, BitDepth Y -8), set the variable shift2 equal to 6, and set the variable shift3 equal to Max(2,14-BitDepth Y ).

[0153] – Set the variable picW equal to pic_width_in_luma_samples and the variable picH equal to pic_height_in_luma_samples.

[0154] It is equal to xFrac as follows L or yFrac L The luminance interpolation filter coefficient f for each 1 / 16 fractional sample position p L [p]:

[0155] – If MotionModelIdc[xSb][ySb] is greater than 0, and sbWidth and sbHeight are both equal to 4, then the luminance interpolation filter coefficient f L [p] is specified in Table 2.

[0156] – Otherwise, depending on hpelIfIdx, the luma interpolation filter coefficients f L [p] is specified in Table 1.

[0157] For i=0..7, the luminance position (xInt i ,yInt i ):

[0158] – If subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, then the following applies:

[0159] xInt i =Clip3(SubPicLeftBoundaryPos,SubPicRightBoundaryPos,xInt L +i-3)(8-771)

[0160] yInt i =Clip3(SubPicTopBoundaryPos,SubPicBotBoundaryPos,yInt L +i-3)(8-772)

[0161] – Otherwise (subpic_treated_as_pic_flag[SubPicIdx] is equal to 0), then the following applies:

[0162] xInt i =Clip3(0,picW-1,sps_ref_wraparound_enabled_flag?

[0163] ClipH((sps_ref_wraparound_offset_minus1+1)*MinCbSizeY,picW,xInt L +i-3):(8-773)xInt L +i-3)

[0164] yInt i =Clip3(0,picH-1,yInt L +i-3) (8-774)

[0165] For i = 0..7, the luminance position in units of full samples is further modified as follows:

[0166] xInt i =Clip3(xSbInt L -3,xSbInt L +sbWidth+4,xInt i ) (8-775)

[0167] yInt i =Clip3(ySbInt L -3,ySbInt L +sbHeight+4,yInt i ) (8-776)

[0168] The predicted luminance sample value predSampleLX is derived as follows L :

[0169] – If xFrac L and yFrac L If both are equal to 0, then predSampleLX is derived as follows L Value:

[0170] predSampleLX L =refPicLX L [xInt3][yInt3]< <shift3(8-777)

[0171] – Otherwise, if xFrac L Not equal to 0 and yFrac L If it is equal to 0, then predSampleLX is derived as follows L Value:

[0172]

[0173] – Otherwise, if xFrac L is equal to 0, and yFrac L If it is not equal to 0, then predSampleLX is derived as follows L Value:

[0174]

[0175] – Otherwise, if xFrac L Not equal to 0 and yFrac L If it is not equal to 0, then predSampleLX is derived as follows L Value:

[0176] – The sample array temp[n] is derived as follows, where n = 0..7:

[0177]

[0178] – Derived as follows: The predicted luminance sample value predSampleLX L :

[0179]

[0180] Table 8-11 Luma interpolation filter coefficients f for each 1 / 16 fractional sample position p L [p]

[0181]

[0182] Table 8-12 Luma interpolation filter coefficients f for each 1 / 16 fractional sample position p for affine motion mode L [p]

[0183]

[0184] 8.5.6.3.3 Luminance integer sample retrieval process

[0185] The inputs to this process are:

[0186] – Luminance position in full sample units (xInt L ,yInt L ),

[0187] – Luminance reference sample array refPicLX L ,

[0188] The output of this process is the predicted luminance sample value predSampleLX L

[0189] Set the variable shift equal to Max(2,14-BitDepth Y ).

[0190] The variable picW is set equal to pic_width_in_luma_samples, and the variable picH is set equal to pic_height_in_luma_samples.

[0191] The luminance position (xInt, yInt) in units of full samples is derived as follows:

[0192] xInt=Clip3(0,picW-1,sps_ref_wraparound_enabled_flag? (8-782)

[0193] ClipH((sps_ref_wraparound_offset_minus1+1)*MinCbSizeY,picW,xInt L ):xInt L )

[0194] yInt=Clip3(0,picH-1,yInt L ) (8-783)

[0195] The predicted luminance sample value predSampleLX is derived as follows L :

[0196] predSampleLX L =refPicLX L [xInt][yInt]< <shift3 (8-784)

[0197] 8.5.6.3.4 Chroma sample interpolation process

[0198] The inputs to this process are:

[0199] – Chroma position in units of full sample points (xInt C ,yInt C ),

[0200] – Chroma position in 1 / 32 fractional samples (xFrac C ,yFrac C ),

[0201] – the chroma position in full sample units (xSbIntC, ySbIntC), which specifies the top left sample of the bounding block used for reference sample filling relative to the top left chroma sample of the reference picture,

[0202] –Variable sbWidth specifies the width of the current sub-block.

[0203] –Variable sbHeight specifies the height of the current sub-block.

[0204] – Chroma reference sample array refPicLX C .

[0205] The output of this process is the predicted chroma sample value predSampleLX C

[0206] The variables shift1, shift2, and shift3 are derived as follows:

[0207] – Set the variable shift1 to be equal to Min(4, BitDepth C -8), set the variable shift2 equal to 6, and set the variable shift3 equal to Max(2,14-BitDepth C ).

[0208] – Set the variable picW C Set it equal to pic_width_in_luma_samples / SubWidthC, and set the variable picH C Set equal to pic_height_in_luma_samples / SubHeightC.

[0209] In Table 3, it is specified that the value equal to xFrac C or yFrac C The chroma interpolation filter coefficient f for each 1 / 32 fractional sample position p C [p].

[0210] Set the variable xOffset equal to (sps_ref_wraparound_offset_minus1+1)*MinCbSizeY) / SubWidthC.

[0211] For i=0..3, the chroma position (xInt i ,yInt i ):

[0212] – If subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, then the following applies:

[0213] xInt i =Clip3(SubPicLeftBoundaryPos / SubWidthC,SubPicRightBoundaryPos / SubWidthC,xInt L +i) (8-785)

[0214] yInt i =Clip3(SubPicTopBoundaryPos / SubHeightC,SubPicBotBoundaryPos / SubHeightC,yInt L +i) (8-786)

[0215] – Otherwise (subpic_treated_as_pic_flag[SubPicIdx] is equal to 0), then the following applies:

[0216] xInt i =Clip3(0,picW C -1,sps_ref_wraparound_enabled_flag? ClipH(xOffset,picW C ,xInt C +i-1):(8-787)

[0217] xInt C +i-1)

[0218] yInt i =Clip3(0,picH C -1,yInt C +i-1) (8-788)

[0219] For i=0..3, the chroma position (xInt i ,yInt i ):

[0220] xInt i =Clip3(xSbIntC-1,xSbIntC+sbWidth+2,xInt i ) (8-789)

[0221] yInt i =Clip3(ySbIntC-1,ySbIntC+sbHeight+2,yInt i ) (8-790)

[0222] The predicted chroma sample value predSampleLX is derived as follows C :

[0223] – If xFrac C and yFrac C If both are equal to 0, then predSampleLX is derived as follows C Value:

[0224] predSampleLX C =refPicLX C [xInt1][yInt1]< <shift3 (8-791)

[0225] – Otherwise, if xFracC Not equal to 0 and yFrac C If it is equal to 0, then predSampleLX is derived as follows C Value:

[0226]

[0227] – Otherwise, if xFrac C is equal to 0, and yFrac C If it is not equal to 0, then predSampleLX is derived as follows C Value:

[0228]

[0229] – Otherwise, if xFrac C Not equal to 0 and yFrac C If it is not equal to 0, then predSampleLX is derived as follows C Value:

[0230] – The sample array temp[n] is derived as follows, where n=0..3:

[0231]

[0232] – Derived as follows: predicted chroma sample value predSampleLX C :

[0233] predSampleLX C =(f C [yFrac C ][0]*temp[0]+

[0234] f C [yFrac C ][1]*temp[1]+

[0235] f C [yFrac C ][2]*temp[2]+ (8-795)

[0236] f C [yFrac C ][3]*temp[3])>>shift2

[0237] Table 8-13 Chroma interpolation filter coefficients f for each 1 / 32 fractional sample position p C [p] provisions.

[0238]

[0239]

[0240] 2.5 Affine Motion Compensated Prediction Based on Refined Sub-blocks

[0241] The technology disclosed herein includes a method for refining sub-block-based affine motion compensation predictions using optical flow. After performing sub-block-based affine motion compensation, the prediction samples are refined by adding the differences derived from the optical flow equation, a process called prediction refinement using optical flow (PROF). The proposed method enables pixel-level inter-frame prediction without increasing memory access bandwidth.

[0242] To achieve finer granularity in motion compensation, this contribution proposes a method to refine sub-block-based affine motion compensation predictions using optical flow. After performing sub-block-based affine motion compensation, the luma prediction samples are refined by adding the differences derived from the optical flow equation. The proposed PROF (Prediction Refinement using Optical Flow) is described in the following four steps.

[0243] Step 1) Perform sub-block based affine motion compensation to generate sub-block prediction I(i, j).

[0244] Step 2) Use a 3-tap filter [-1, 0, 1] to calculate the spatial gradient g of the sub-block prediction at each sample point x (i, j) and g y (i, j).

[0245] g x (i,j)=I(i+1,j)-I(i-1,j)

[0246] g y (i,j)=I(i,j+1)-I(i,j-1)

[0247] The sub-block prediction is extended by one pixel on each side for gradient calculation. To reduce memory bandwidth and complexity, pixels on the extended boundary are copied from the nearest integer pixel position in the reference picture. Thus, additional interpolation of the padded area is avoided.

[0248] Step 3) Refine the brightness prediction (denoted as ΔI) calculated by the optical flow equation.

[0249] ΔI(i, j)=g x (i, j)*Δv x (i, j)+g y (i, j)*Δv y (i, j)

[0250] Among them Figure 1As shown, ΔMV (expressed as Δv(i, j)) is the difference between the pixel MV (expressed as v(i, j)) calculated for the sample position (i, j) and the sub-block MV of the sub-block to which the pixel (i, j) belongs.

[0251] Since the affine model parameters and the pixel position relative to the sub-block center do not change between sub-blocks, Δv(i, j) can be calculated for the first sub-block and reused for other sub-blocks in the same CU. Let x and y be the horizontal and vertical offsets from the pixel position to the sub-block center, then Δv(x, y) can be obtained by the following equation:

[0252]

[0253] For a 4-parameter affine model,

[0254]

[0255]

[0256] For the 6-parameter affine model,

[0257]

[0258] Among them, (v 0x , v 0y )、(v 1x , v 1y )、(v 2x , v 2y ) are the upper left, upper right and lower left control point motion vectors, w and h are the width and height of the CU.

[0259] Step 4) Finally, the luma prediction refinement is added to the sub-block prediction I(i, j). The final prediction I' is generated according to the following equation.

[0260] I′(i,j)=I(i,j)+ΔI(i,j)

[0261] Here are some details:

[0262] a) How does PROF derive gradients?

[0263] In some embodiments, the gradient is calculated for each sub-block (4×4 sub-blocks in VTM-4.0) of each reference list. For each sub-block, the nearest integer sample of the reference block is retrieved to fill in the four outer edges of the sample.

[0264] Assume that the MV of the current sub-block is (MVx, MVy). Then, calculate the fractional part as (FracX, FracY) = (MVx & 15, MVy & 15). Calculate the integer part as (IntX, IntY) = (MVx >> 4, MVy >> 4). Derive the offset (OffsetX, OffsetY) as follows:

[0265] OffsetX = FracX > 7? 1 : 0;

[0266] OffsetY = FracY > 7? 1 : 0;

[0267] Assume that the upper-left coordinates of the current sub-block are (xCur, yCur) and the dimension of the current sub-block is W × H. Then calculate (xCor0, yCor0), (xCor1, yCor1), (xCor2, yCor2) and (xCor3, yCor3) as follows:

[0268] (xCor0, yCor0) = (xCur + IntX + OffsetX - 1, yCur + IntY + OffsetY - 1);

[0269] (xCor1, yCor1) = (xCur + IntX + OffsetX - 1, yCur + IntY + OffsetY + H);

[0270] (xCor2,yCor2) = (xCur + IntX + OffsetX - 1,yCur + IntY + OffsetY);

[0271] (xCor3,yCor3) = (xCur + IntX + OffsetX + W,yCur + IntY + OffsetY);

[0272] Assume that PredSample[x][y] (where x = 0..W - 1, y = 0..H - 1) stores the prediction samples for the sub-block. Then derive the filling samples as follows:

[0273] PredSample[x][-1] = (Ref(xCor0 + x,yCor0) << Shift0) – Rounding, where x = -1..W;

[0274] PredSample[x][H] = (Ref(xCor1 + x,yCor1) << Shift0) – Rounding, where x = -1..W;

[0275] PredSample[-1][y] = (Ref(xCor2, yCor2 + y) << Shift0) – Rounding, where y = 0..H-1;

[0276] PredSample[W][y] = (Ref(xCor3, yCor3 + y) << Shift0) - Rounding, where y = 0..H-1;

[0277] Where Rec represents the reference picture. Rounding is an integer, which equals 2 in an exemplary PROF implementation 13 . Shift0 = Max(2, (14 - BitDepth)); PROF attempts to improve the precision of gradients, which is different from BIO in VTM-4.0, which outputs gradients with the same precision as the input luminance samples.

[0278] Calculate the gradients in PROF as follows:

[0279] Shift1 = Shift0 - 4.

[0280] gradientH[x][y] = (predSamples[x + 1][y] - predSample[x - 1][y]) >> Shift1

[0281] gradientV[x][y] = (predSample[x][y + 1] - predSample[x][y - 1]) >> Shift1

[0282] It should be noted that predSamples[x][y] maintains the interpolated precision.

[0283] b) How PROF derives Δv

[0284] Δv can be described as follows (denoted as dMvH[posX][posY] and dMvV[posX][posY], where posX = 0..W-1, posY = 0..H-1).

[0285] Assume that the dimension of the current block is cbWidth × cbHeight, the number of control point motion vectors is numCpMv, and the control point motion vectors are cpMvLX[cpIdx], where cpIdx = 0..numCpMv-1 and X is 0 or 1 representing two reference lists.​​​​​log2CbW=Log2(cbWidth)

[0288] log2CbH=Log2(cbHeight)

[0289] The variables mvScaleHor, mvScaleVer, dHorX, and dVerX are derived as follows:

[0290] mvScaleHor=cpMvLX[0][0]<<7

[0291] mvScaleVer=cpMvLX[0][1]<<7

[0292] dHorX=(cpMvLX[1][0]-cpMvLX[0][0])<<(7-log2CbW)

[0293] dVerX=(cpMvLX[1][1]-cpMvLX[0][1])<<(7-log2CbW)

[0294] The variables dHorY and dVerY are derived as follows:

[0295] - If numCpMv is equal to 3, the following applies:

[0296] dHorY=(cpMvLX[2][0]-cpMvLX[0][0])<<(7-log2CbH)

[0297] dVerY=(cpMvLX[2][1]-cpMvLX[0][1])<<(7-log2CbH)

[0298] Otherwise (numCpMv is equal to 2), the following applies:

[0299] dHorY=-dVerX

[0300] dVerY=dHorX

[0301] The variables qHorX, qVerX, qHorY, and qVerY are derived as follows:

[0302] qHorX=dHorX<<2;

[0303] qVerX=dVerX<<2;

[0304] qHorY=dHorY<<2;

[0305] qVerY=dVerY<<2;

[0306] Calculate dMvH[0][0] and dMvV[0][0] as follows:

[0307] dMvH[0][0]=((dHorX+dHorY)<<1)-((qHorX+qHorY)<<1);

[0308] dMvV[0][0]=((dVerX+dVerY)<<1)-((qVerX+qVerY)<<1);

[0309] Derive dMvH[xPos][0] and dMvV[xPos][0] as follows, where xPos ranges from 1 to W-1:

[0310] dMvH[xPos][0]=dMvH[xPos-1][0]+qHorX;

[0311] dMvV[xPos][0]=dMvV[xPos-1][0]+qVerX;

[0312] For yPos from 1 to H-1, the following applies:

[0313] dMvH[xPos][yPos]=dMvH[xPos][yPos-1]+qHorY, where xPos=0..W-1

[0314] dMvV[xPos][yPos]=dMvV[xPos][yPos-1]+qVerY, where xPos=0..W-1

[0315] Finally, dMvH[xPos][yPos] and dMvV[xPos][yPos] with posM=0..W-1 and posY=0..H-1 are shifted right to:

[0316] dMvH[xPos][yPos]=SatShift(dMVH[xPos][yPos], 7+2-1);

[0317] dMvV[xPos][yPos]=SatShift(dMvV[xPos][yPos], 7+2-1);

[0318] Where SatShift(x, n) and Shift(x, n) are defined as:

[0319]

[0320] Shift(x,n)=(x+offset0)>>n

[0321] In one example, offset0 and / or offset1 are set to (1<<n)>>1.

[0322] c) How does PROF derive ΔI?

[0323] For the position (posX, posY) within the sub-block, its corresponding Δv(i, j) is expressed as (dMvH[posX][posY], dMvV[posX][posY]). Its corresponding gradient is expressed as (gradientH[posX][posY], gradientV[posX][posY]).

[0324] Then, ΔI(posX, posY) is derived as follows.

[0325] (dMvH[posX][posY], dMvV[posX][posY]) is clipped to:

[0326] dMvH[posX][posY]=Clip3(-32768, 32767, dMvH[posX][posY]);

[0327] dMvV[posX][posY]=Clip3(-32768, 32767, dMvV[posX][posY]);

[0328] ΔI(posX, posY)=dMvH[posX][posY]×gradientH[posX][posY]+dMvV[posX][posY]×gradientV[posX][posY];

[0329] ΔI(posX, posY)=Shift(ΔI(posX, posY), 1+1+4);

[0330] ΔI(posX, posY)=Clip3(-(2 13 -1), 2 13 -1, ΔI(posX, posY));

[0331] d) How does PROF derive I'?

[0332] If the current block is not encoded using bidirectional prediction or weighted prediction,

[0333] I'(posX, posY)=Shift((I(posX, posY)+ΔI(posX, posY)), Shift0),

[0334] I'(posX, posY)=ClipSample(I'(posX, posY)),

[0335] Among them, ClipSample clips the sample value into a valid output sample value, and then outputs I'(posX, posY) as the inter-frame prediction value.

[0336] Otherwise (the current block is coded using bidirectional prediction or weighted prediction), I'(posX, posY) will be stored and used to generate inter-frame prediction values ​​based on other prediction values ​​and / or weighted values.

[0337] 2.6 Example Strip Header

[0338]

[0339]

[0340]

[0341]

[0342]

[0343]

[0344] 2.7 Example Sequence Parameter Set

[0345]

[0346]

[0347]

[0348]

[0349]

[0350] 2.8 Example Image Parameter Set

[0351]

[0352]

[0353]

[0354]

[0355] 2.9 Example Adaptive Parameter Set

[0356]

[0357]

[0358]

[0359]

[0360]

[0361] 2.10 Example Image Header

[0362] In some embodiments, the image header is designed to have the following attributes:

[0363] 1. The temporal ID and layer ID of the picture header NAL unit are the same as those of the layer access unit containing the picture header.

[0364] 2. The picture header NAL unit should precede the NAL unit containing the first slice of its associated picture. This establishes the association between the picture header and the slice of the picture associated with the picture header without signaling the picture header ID in the picture header and referencing the picture header ID from the slice header.

[0365] 3. The picture header NAL unit should follow the picture level parameter set or higher level, such as DPS, VPS, SPS, PPS, etc. Therefore, this requires that those parameter sets are not repeated / non-existent within a picture or within an access unit.

[0366] 4. The image header contains information about the image type of the image it is associated with. The image type can be used to define the following (this is not an exhaustive list):

[0367] a. The image is an IDR image

[0368] b. The image is a CRA image

[0369] c. The picture is a GDR picture

[0370] d. The picture is non-IRAP, non-GDR picture and contains only I stripes

[0371] e. The picture is non-IRAP, non-GDR picture, and can only contain P stripes and I stripes

[0372] f. The picture is a non-IRAP, non-GDR picture and contains any one of the B slice, P slice and / or I slice

[0373] 5. Move the signaling of picture level syntax elements in the slice header to the picture header.

[0374] 6. Signal non-picture level syntax elements in the slice header, which are usually the same for all slices of the same picture in the picture header. When those syntax elements are not present in the picture header, they can be signaled in the slice header.

[0375] In some embodiments, the concept of a mandatory picture header is used to be sent once per picture as the first VCL NAL unit of the picture. It is also proposed to move syntax elements currently in the slice header to this picture header. Functionally, syntax elements that only need to be sent once per picture can be moved to the picture header, rather than being sent multiple times for a given picture. For example, syntax elements in the slice header are sent once per slice. The moved slice header syntax elements are constrained to be identical within a picture.

[0376] The syntax elements are already constrained to be the same across all slices of a picture. It can be inferred that moving these fields to the picture header so that they are signaled only once per picture, rather than once per slice, avoids unnecessary transmission of redundant bits without changing the functionality of these syntax elements.

[0377] 1. In some embodiments, the following semantic constraints exist:

[0378] When present, the values ​​of each of the slice header syntax elements slice_pic_parameter_set_id, non_reference_picture_flag, colour_plane_id, slice_pic_order_cnt_lsb, recovery_poc_cnt, no_output_of_prior_pics_flag, pic_output_flag, and slice_temporal_mvp_enabled_flag should be the same in all slice headers of a codec picture. Therefore, each of these syntax elements can be moved to the picture header to avoid unnecessary redundant bits.

[0379] In this contribution, recovery_poc_cnt and no_output_of_prior_pics_flag are not moved to the picture header. Their presence in the slice header depends on a conditional check of the slice header nal_unit_type, so if it is desired to move these syntax elements to the picture header, it is recommended to study them.

[0380] 2. In some embodiments, the following semantic constraints exist:

[0381] When present, the value of slice_lmcs_aps_id shall be the same for all slices of a picture.

[0382] When present, the value of slice_scaling_list_aps_id should be the same for all slices of a picture.Therefore, each of these syntax elements can be moved to the picture header to avoid unnecessary redundant bits.

[0383] In some embodiments, syntax elements are not currently constrained to be the same in all slices of a picture. It is recommended to evaluate the expected usage of these syntax elements to determine which ones can be moved to the picture header to simplify the overall VVC design, as it is claimed that processing a large number of syntax elements in each slice header has complexity implications.

[0384] 1. It is proposed to move the following syntax elements to the picture header. There is currently no restriction on them having different values ​​on different slices, but it is claimed that there is no / small benefit and codec loss to transmit them in each slice header, as their intended usage will change at the picture level:

[0385] a.six_minus_max_num_merge_cand

[0386] b.five_minus_max_num_subblock_merge_cand

[0387] c.slice_fpel_mmvd_enabled_flag

[0388] d.slice_disable_bdof_dmvr_flag

[0389] e.max_num_merge_cand_minus_max_num_triangle_cand

[0390] f.slice_six_minus_max_num_ibc_merge_cand

[0391] 2. It is proposed to move the following syntax elements to the picture header. There is currently no restriction on them having different values ​​on different slices, but it is claimed that there is no / smaller benefit and codec loss to transmit them in each slice header, as their intended usage will change at the picture level:

[0392] a.partition_constraints_override_flag

[0393] b.slice_log2_diff_min_qt_min_cb_luma

[0394] c.slice_max_mtt_hierarchy_depth_luma

[0395] d.slice_log2_diff_max_bt_min_qt_luma

[0396] e.slice_log2_diff_max_tt_min_qt_luma

[0397] f.slice_log2_diff_min_qt_min_cb_chroma

[0398] g.slice_max_mtt_hierarchy_depth_chroma

[0399] h.slice_log2_diff_max_bt_min_qt_chroma

[0400] i.slice_log2_diff_max_tt_min_qt_chroma

[0401] The conditional check "slice_type == 1" associated with some of these syntax elements has been removed with the move to the picture header.

[0402] 3. It is proposed to move the following syntax elements to the picture header. There is currently no restriction on them having different values ​​on different slices, but it is claimed that there is no / smaller benefit and codec loss to transmit them in each slice header, as their intended usage will change at the picture level:

[0403] a.mvd_l1_zero_flag

[0404] The conditional check "slice_type == B" associated with some of these syntax elements has been removed with the move to the picture header.

[0405] 4. It is proposed to move the following syntax elements to the picture header. There is currently no restriction on them having different values ​​on different slices, but it is claimed that there is no / smaller benefit and codec loss to transmit them in each slice header, as their intended usage will change at the picture level:

[0406] a.dep_quant_enabled_flag

[0407] b.sign_data_hiding_enabled_flag

[0408] 3. Deficiencies of existing implementation methods

[0409] During motion vector refinement, DMVR and BIO do not involve the original signal, which may cause the codec block to have inaccurate motion information. In addition, DMVR and BIO sometimes use fractional motion vectors after motion refinement, while screen video usually has integer motion vectors, which makes the current motion information even more inaccurate and degrades codec performance.

[0410] When RPR is applied in VVC, RPR (ARC) may have the following problems:

[0411] 1. With RPR, the interpolation filters may be different for adjacent samples in a block, which is undesirable in SIMD (Single Instruction Multiple Data) implementation.

[0412] 2. RPR is not considered for the enclosed area.

[0413] 3. Note that "The consistency crop window offset parameters apply only to output. All internal decoding processes are applied to the uncropped picture size." However, when RPR is applied, these parameters can be used during decoding.

[0414] 4. When deriving the reference sample point position, RPR only considers the ratio between the two consistency windows. However, the difference in the upper left offset between the two consistency windows should also be considered.

[0415] 5. The ratio between the width / height of the reference picture and the width / height of the current picture is constrained in VVC. However, the ratio between the width / height of the consistency window of the reference picture and the width / height of the consistency window of the current picture is not constrained.

[0416] 6. Not all syntax elements in the picture header are processed correctly.

[0417] 7. In the current VVC, for TPM and GEO prediction modes, chroma blending weights are derived regardless of the chroma sample position type of the video sequence. For example, in TPM / GEO, if the chroma weight is derived from the luma weight, the luma weight may need to be downsampled to match the sampling of the chroma signal. Chroma downsampling is usually applied when the chroma sample position type is 0, which is widely used in ITU-R BT.601 or ITU-R BT.709. However, if a different chroma sample position type is used, this may cause misalignment between the chroma samples and the downsampled luma samples, which may reduce the encoding and decoding performance.

[0418] 4 Example Techniques and Embodiments

[0419] The embodiments described in detail below are intended to serve as examples for explaining the general concept. These embodiments should not be interpreted in a narrow sense. Furthermore, these embodiments can be combined in any manner.

[0420] In addition to DMVR and BIO mentioned below, the method described below can also be applied to other decoder motion information derivation techniques.

[0421] The motion vector is represented as (mv_x, mv_y), where mv_x is the horizontal component and mv_y is the vertical component.

[0422] In this disclosure, the resolution (or dimension, or width / height or size) of a picture may refer to the resolution (or dimension, or width / height or size) of the encoded / decoded picture, or may refer to the resolution (or dimension, width / height or size) of a consistency window in the encoded / decoded picture.

[0423] Motion Compensation in RPR

[0424] 1. When the resolution of the reference picture is different from the resolution of the current picture, or when the width and / or height of the reference picture is larger than the width and / or height of the current picture, the same horizontal and / or vertical interpolation filter can be used to generate the prediction value of the sample set (at least two samples) of the current block.

[0425] a. In one example, the set may include all samples in the region of the block.

[0426] i. For example, the block can be divided into S M×N rectangles that do not overlap each other. Each M×N rectangle is a set. Figure 2 In the example shown, a 16x16 block can be divided into 16 4x4 rectangles, each rectangle being a set.

[0427] ii. For example, a row of N samples is a set. N is an integer not greater than the block width. In one example, N is 4 or 8 or the block width.

[0428] iii. For example, a column with N samples is a set. N is an integer not greater than the block height. In one example, N is 4 or 8 or the block height.

[0429] iv. M and / or N may be predefined or derived on the fly, such as based on block dimensions / codec information, or signaled.

[0430] b. In one example, the samples in a set may have the same MV (denoted as shared MV).

[0431] c. In one example, samples in a set may have MVs with the same horizontal component (denoted as shared horizontal component).

[0432] d. In one example, samples in a set may have MVs with the same vertical component (denoted as shared vertical component).

[0433] e. In one example, samples in a set may have MVs with the same fractional portion of the horizontal component (denoted as shared fractional horizontal component).

[0434] i. For example, assuming that the MV of the first sample point is (MV1x, MV1y) and the MV of the second sample point is (MV2x, MV2y), then MV1x&(2 M -1) is equal to MV2x&(2 M -1), where M represents the MV precision. For example, M=4.

[0435] f. In one example, the samples in the group may have MVs with the same fractional portion of the vertical component (denoted as shared fractional vertical component).

[0436] i. For example, assuming the MV of the first sample is (MV1x, MV1y) and the MV of the second sample is (MV2x, MV2y), then MV1y & (2M-1) should equal MV2y & (2M-1), where M represents the MV precision. For example, M = 4.

[0437] g. In one example, for the samples in the set to be predicted, the prediction can first be performed based on the current picture and the reference picture (e.g., the refx derived in Section 8.5.6.3.1 of JVET-O2001-v14). L ,refy L )) is derived from the resolution of MV b Then, MV can be b is further modified (e.g., rounded / truncated / clipped) to MV' to meet requirements such as those in the above items, and MV' will be used to derive the predicted sample for this sample.

[0438] i. In one example, MV' has the same b The integer part is the same, and the fractional part of MV' is set to the shared fractional horizontal and / or vertical components.

[0439] ii. In one example, MV' is set to have a shared fractional horizontal and / or vertical component and is closest to MV b .

[0440] h. The shared motion vector (and / or shared horizontal component and / or shared vertical component and / or shared fractional vertical component and / or shared fractional vertical component) can be set as the motion vector (and / or horizontal component and / or vertical component and / or fractional vertical component and / or fractional vertical component) of a specific sample in the set.

[0441] i. For example, a specific sample point can be located at the corner point of a rectangular set, such as Figure 3A "A", "B", "C" and "D" shown in.

[0442] ii. For example, a specific sample point can be located at the center of a set of rectangles, such as Figure 3A The "E", "F", "G" and "H" shown in the figure.

[0443] iii. For example, a particular point may be located at the end of a row or column shape set, such as Figure 3B and Figure 3C "A" and "D" shown in the figure.

[0444] iv. For example, a particular point may be located in the middle of a set of row or column shapes, such as Figure 3B and Figure 3C "B" and "C" shown in the figure.

[0445] v. In one example, the motion vector of a particular sample point can be the MV mentioned in item g. b .

[0446] i. The shared motion vector (and / or shared horizontal component and / or shared vertical component and / or shared fractional vertical component and / or shared fractional vertical component) may be set to the motion vector (and / or horizontal component and / or vertical component and / or fractional vertical component and / or fractional vertical component) of a virtual sample located at a different position than all samples in the set.

[0447] i. In one example, the virtual sample point is not in the set, but is located in a region covering all the samples in the set.

[0448] 1) Alternatively, the virtual sample point is located outside the region covering all the sample points in the set, for example, adjacent to the lower right corner of the region.

[0449] ii. In one example, the MVs of virtual samples are derived in the same way as real samples, but at different locations.

[0450] iii. Figures 3A-3C The “V” in shows three examples of virtual sample points.

[0451] j. The shared MV (and / or shared horizontal component and / or shared vertical component and / or shared fractional vertical component and / or shared fractional vertical component) can be set as a function of the MV (and / or horizontal component and / or vertical component and / or fractional vertical component and / or fractional vertical component) of multiple samples and / or virtual samples.

[0452] i. For example, the shared MV (and / or shared horizontal component and / or shared vertical component and / or shared fractional vertical component and / or shared fractional vertical component) may be set to the average MV (and / or horizontal component and / or vertical component and / or fractional vertical component and / or fractional vertical component) of the following samples: all or part of the samples in the set; or Figure 3A Sample points "E", "F", "G", "H" in ; or Figure 3A Sample points "E", "H" in ; or Figure 3A Sample points "A", "B", "C", "D" in ; or Figure 3A Sample points "A", "D" in ; or Figure 3B Sample points "B" or "C" in the Figure 3A Sample points "A" and "D" in ; Figure 3C Sample points "B" or "C" in the Figure 3C Sample points "A" and "D" in .

[0453] 2. It is proposed that when the resolution of the reference picture is different from that of the current picture, or when the width and / or height of the reference picture is larger than that of the current picture, only integer MVs are allowed to perform the motion compensation process to derive the prediction block of the current block.

[0454] a. In one example, the decoded motion vector for the sample to be predicted is rounded to an integer MV before being used.

[0455] b. In one example, the decoded motion vector for the sample to be predicted is rounded to the integer MV closest to the decoded motion vector.

[0456] c. In one example, the decoded motion vector for the sample to be predicted is rounded to the integer MV closest to the decoded motion vector in the horizontal direction.

[0457] d. In one example, the decoded motion vector for the sample to be predicted is rounded to the integer MV that is closest to the decoded motion vector in the vertical direction.

[0458] 3. The motion vectors used in the motion compensation process of the samples in the current block (e.g., the shared MV / shared horizontal or vertical or fractional component / MV' mentioned in the above items) can be stored in the decoded picture buffer and used for motion vector prediction of subsequent blocks in the current / different pictures.

[0459] a. Alternatively, the motion vector used in the motion compensation process of the samples in the current block (e.g., the shared MV / shared horizontal or vertical or fractional component / MV' mentioned in the above items) is not allowed to be used for prediction of the motion vector of the subsequent block in the current / different picture.

[0460] i. In one example, the decoded motion vector (e.g., MV in the above entry) can be b ) is used for motion vector prediction of subsequent blocks in the current / different picture.

[0461] b. In one example, the motion vector used in the motion compensation process of the samples in the current block can be used in the filtering process (eg, deblocking filter / SAO / ALF).

[0462] i. Alternatively, the decoded motion vectors (e.g., MV in the above entry) can be used in the filtering process. b ).

[0463] c. In one example, such MVs may be derived at the sub-block level and may be stored for each sub-block.

[0464] 4. It is proposed that the interpolation filter used to derive the prediction block of the current block in the motion compensation process can be selected depending on whether the resolution of the reference picture is different from the resolution of the current picture, or whether the width and / or height of the reference picture is greater than the width and / or height of the current picture.

[0465] a. In one example, when condition A is met, an interpolation filter with fewer taps may be applied, where condition A depends on the current picture and / or reference picture dimensions.

[0466] i. In one example, condition A is that the resolution of the reference picture is different from the resolution of the current picture.

[0467] ii. In one example, condition A is that the width and / or height of the reference picture is greater than the width and / or height of the current picture.

[0468] iii. In one example, condition A is W1>a*W2 and / or H1>b*H2, where (W1, H1) represents the width and height of the reference image, and (W2, H2) represents the width and height of the current image, and a and b are two coefficients, for example, a=b=1.5.

[0469] iv. In one example, condition A may also depend on whether bidirectional prediction is used.

[0470] v. In one example, a 1-tap filter is applied. In other words, unfiltered integer pixels are output as the interpolation result.

[0471] vi. In one example, when the resolution of the reference picture is different from the resolution of the current picture, a bilinear filter is applied.

[0472] vii. In one example, when the resolution of the reference picture is different from that of the current picture, or the width and / or height of the reference picture is larger than that of the current picture, a 4-tap filter or a 6-tap filter is applied.

[0473] 1) A 6-tap filter can also be used for affine motion compensation.

[0474] 2) A 4-tap filter can also be used for interpolation of chroma samples.

[0475] b. In one example, when the resolution of the reference picture is different from that of the current picture, or the width and / or height of the reference picture is larger than that of the current picture, padding samples are used to perform interpolation.

[0476] c. Whether and / or how to apply the method disclosed in item 4 may depend on the color components.

[0477] i. For example, these methods are applied only to the luma component.

[0478] d. Whether and / or how to apply the method disclosed in item 4 may depend on the interpolation filtering direction.

[0479] i. For example, these methods are only applied to horizontal filtering.

[0480] ii. For example, these methods are only applied to vertical filtering.

[0481] 5. It is proposed to apply a two-stage process for prediction block generation when the resolution of the reference picture is different from that of the current picture, or the width and / or height of the reference picture is larger than that of the current picture.

[0482] a. In the first stage, a virtual reference block is generated by upsampling or downsampling an area in the reference picture, depending on the width and / or height of the current picture and the reference picture.

[0483] b. In the second stage, prediction samples are generated from the virtual reference blocks by applying interpolation filtering, independent of the width and / or height of the current picture and the reference picture.

[0484] 6. It is proposed that, as in some embodiments, the calculation of the upper left coordinate (xSbIntL, ySbIntL) of the bounding block used for reference sample filling can be derived depending on the width and / or height of the current picture and the reference picture.

[0485] a. In one example, the brightness position in the full sample unit is modified to:

[0486] xInt i =Clip3(xSbInt L -Dx,xSbInt L +sbWidth+Ux,xInt i ),

[0487] yInt i =Clip3(ySbInt L -Dy,ySbInt L +sbHeight+Uy,yInt i ),

[0488] Dx and / or Dy and / or Ux and / or Uy may depend on the width and / or height of the current picture and the reference picture.

[0489] b. In one example, the chroma position in the full sample unit is modified to:

[0490] xInti=Clip3(xSbInt C -Dx,xSbInt C +sbWidth+Ux,xInti)

[0491] yInti=Clip3(ySbInt C -Dy,ySbInt C +sbHeight+Uy,yInti)

[0492] Dx and / or Dy and / or Ux and / or Uy may depend on the width and / or height of the current picture and the reference picture.

[0493] 7. Instead of storing / using motion vectors for blocks based on the same reference picture resolution as the current picture, it is proposed to use the real motion vectors, where the resolution difference is taken into account.

[0494] Alternatively, when a motion vector is used to generate a prediction block, it is not necessary to calculate the current picture and the reference picture (e.g., (refx L , refy L ))'s resolution to further change the motion vectors.

[0495] Interaction between RPR and other codec tools

[0496] 8. Whether / how to apply a filtering process (eg, a deblocking filter) may depend on the resolution of the reference picture and / or the resolution of the current picture.

[0497] a. In one example, in addition to the motion vector difference, the boundary strength (BS) setting in the deblocking filter can take into account the resolution difference.

[0498] i. In one example, the motion vector difference scaled according to the resolution of the current and reference pictures can be used to determine the boundary strength.

[0499] b. In one example, compared to a case where the same resolution is used for both blocks, if the resolution of at least one reference picture of block A is different from (or smaller than or larger than) the resolution of a reference picture indicated by block B, the strength of the deblocking filter used for the boundary between block A and block B may be set differently (e.g., increased / decreased).

[0500] c. In one example, if the resolution of at least one reference picture of block A is different from (or smaller than or larger than) the resolution of at least one reference picture of block B, the boundary between block A and block B is marked as to be filtered (e.g., BS is set to 2).

[0501] d. In one example, compared to a case where the reference picture and the current picture use the same resolution, if the resolution of at least one reference picture of block A and / or block B is different from (or smaller than or larger than) the resolution of the current picture, the strength of the deblocking filter used for the boundary between block A and block B may be set differently (e.g., increased / decreased).

[0502] e. In one example, if at least one reference picture of at least one of the two blocks has a resolution different from the current picture resolution, the boundary between the two blocks is marked to be filtered (eg, BS is set to 2).

[0503] 9. When sub-pictures exist, the conforming bitstream may satisfy that the reference picture must have the same resolution as the current picture.

[0504] a. Alternatively, when the reference picture has a different resolution than the current picture, the sub-picture must not exist in the current picture.

[0505] b. Alternatively, for a sub-picture in the current picture, it is not allowed to use a reference picture with a different resolution as the current picture.

[0506] i. Alternatively, in addition, reference picture management can be invoked to exclude those reference pictures that have a different resolution.

[0507] 10. In one example, sub-pictures may be defined for pictures with different resolutions (eg, how to divide a picture into multiple sub-pictures).

[0508] In one example, if the reference picture has a different resolution than the current picture, the corresponding sub-picture in the reference picture may be derived by scaling and / or offsetting a sub-picture of the current picture.

[0509] 11. PROF (Prediction Refinement using Optical Flow) can be enabled when the reference picture has a different resolution than the current picture.

[0510] a. In one example, a set of MVs (denoted as MVs) can be generated for the sample set. g ), and can be used for motion compensation as described in item 1. On the other hand, an MV (denoted as MV p ), and you can convert MV p and MV g The difference between (eg, corresponding to Δv used in PROF) and the gradient (eg, the spatial gradient of the motion compensated block) is used to derive prediction refinement.

[0511] b. In one example, MV p Can have with MV g Different precision. For example, MV p It can have 1 / N pixel (N>0) accuracy, N=32, 64, etc.

[0512] c. In one example, MV g It can have a different precision than the internal MV precision (e.g., 1 / 16 pixel).

[0513] d. In one example, prediction refinements are added to the prediction block to generate a refined prediction block.

[0514] e. In one example, such a method can be applied to each prediction direction.

[0515] f. In one example, such a method can only be applied to the case of unidirectional prediction.

[0516] g. In one example, such a method can be applied to unidirectional prediction or / and bidirectional prediction.

[0517] h. In one example, such a method can be applied only when the reference picture has a different resolution than the current picture.

[0518] 12. It is proposed that when the resolution of the reference picture is different from the resolution of the current picture, only one MV can be used for a block / sub-block to perform the motion compensation process to derive the prediction block of the current block.

[0519] a. In one example, a unique MV for a block / sub-block may be defined as a function (eg, an average) of all MVs associated with each sample within the block / sub-block.

[0520] b. In one example, the unique MV for a block / sub-block may be defined as a selected MV associated with a selected sample point (eg, a center sample point) within the block / sub-block.

[0521] c. In one example, only one MV may use a 4x4 block or sub-block (eg, 4x1).

[0522] d. In one example, BIO can be further applied to compensate for the accuracy loss due to block-based motion vectors.

[0523] 13. When the width and / or height of the reference picture is different from the width and / or height of the current picture, a lazy mode that does not signal any block-based motion vectors may be applied.

[0524] a. In one example, motion vectors may not be signaled and the motion compensation process is a pure resolution change scenario approximating a still image.

[0525] b. In one example, only the motion vector at the picture / slice / tile / CTU level may be signaled, and when the resolution changes, the relevant blocks may use the motion vector.

[0526] 14. For blocks coded using affine prediction mode and / or non-affine prediction mode, PROF can be applied to approximate motion compensation when the width and / or height of the reference picture is different from the width and / or height of the current picture.

[0527] a. In one example, PROF may be enabled when the width and / or height of the reference picture is different from the width and / or height of the current picture.

[0528] b. In one example, a set of affine motions can be generated by combining the indicated motions and resolution scaling and used by PROF.

[0529] 15. When the width and / or height of the reference picture is different from the width and / or height of the current picture, interlaced prediction (eg, as proposed in JVET-K0102) may be applied for approximate motion compensation.

[0530] a. In one example, resolution changes (scaling) are represented as affine motions and interleaved motion prediction can be applied.

[0531] 16. When the width and / or height of the current picture is the same as the width and / or height of the IRAP picture in the same IRAP period

[0532] or heights are different, LMCS and / or chroma residual scaling can be disabled.

[0533] a. In one example, when the LMCS is disabled, slice-level flags such as slice_lmcs_enabled_flag, slice_lmcs_aps_id, and slice_chroma_residual_scale_flag may not be signaled and are inferred as 0.

[0534] b. In one example, when the chroma residual scaling is disabled, slice-level flags such as slice_chroma_residual_scale_flag may not be signaled and are inferred as 0.

[0535] Constraints on RPR

[0536] 17. RPR can be applied to coded blocks with block dimension constraints.

[0537] a. In one example, for a coded block of M×N, where M is the block width and N is the block height, when M*N < T or M*N <= T (such as T = 256), RPR may not be used.

[0538] b. In one example, when M < K (or M <= K) (such as K = 16) and / or N < L (or N <= L) (such as L = 16), RPR may not be used.

[0539] 18. Bitstream consistency can be added to limit the ratio between the width and / or height of the active reference picture (or its consistency window) and the current picture (or its consistency window). Assume refPicW and refPicH represent the width and height of the reference picture, and curPicW and curPicH represent the width and height of the current picture.

[0540] a. In one example, when (refPicW ÷ curPicW) is equal to an integer, the reference picture can be marked as an active reference picture.

[0541] i. Alternatively, when (refPicW ÷ curPicW) is equal to a fraction, the reference picture can be marked as unavailable.

[0542] b. In one example, when (refPicW ÷ curPicW) is equal to (X*n), where X represents a fraction, such as X = 1 / 2, and n represents an integer, such as n = 1, 2, 3, 4..., the reference picture can be marked as an active reference picture.

[0543] i. In one example, when (refPicW ÷ curPicW) is not equal to (X*n), the reference picture can be marked as unavailable.

[0544] 19. For an M×N block, whether to enable and / or how to enable encoding / decoding tools (e.g., bidirectional prediction / integer triangle prediction mode (TPM) / fusion process in TPM) may depend on the resolution of the reference picture (or its consistency window) and / or the resolution of the current picture (or its consistency window).

[0545] a. In one example, M*N < T or M*N <= T (such as T = 64).

[0546] b. In one example, M < K (or M <= K) (such as K = 16) and / or N < L (or N <= L) (such as L = 16).

[0547] c. In one example, when the width / height of at least one reference picture is different from that of the current picture, the encoding / decoding tools are not allowed to be used.

[0548] i. In one example, when the width / height of at least one reference picture of the block is greater than the width / height of the current picture, the encoding / decoding tools are not allowed to be used. <​​​​​​​​​​​​​​​​​​​​​​​​22. It is proposed that when the width and / or height of the picture represented as (pic_width_in_luma_samples, pic_height_in_luma_samples) in the first video unit and the second video unit are the same, the consistency cropping window offset parameters in the first video unit (e.g., PPS) and the second video unit can be different.

[0558] 23. It is proposed that when the width and / or height of the picture represented as (pic_width_in_luma_samples, pic_height_in_luma_samples) in the first video unit and the second video unit are different, in the consistency bitstream, the consistency cropping window offset in the first video unit (e.g., PPS) and the second video unit should be the same.

[0559] a. It is proposed that regardless of whether the width and / or height of the picture represented as (pic_width_in_luma_samples, pic_height_in_luma_samples) in the first video unit and the second video unit are the same, in the conforming bitstream, the conforming cropping window offset parameters in the first video unit (e.g., PPS) and the second video unit should be the same.

[0560] 24. Assume that the width and height of the consistency window defined in the first video unit (e.g., PPS) are denoted as W1 and H1, respectively. The width and height of the consistency window defined in the second video unit (e.g., PPS) are denoted as W2 and H2, respectively. The upper left position of the consistency window defined in the first video unit (e.g., PPS) is denoted as X1 and Y1. The upper left position of the consistency window defined in the second video unit (e.g., PPS) is denoted as X2 and Y2. The width and height of the codec / decoded picture (e.g., pic_width_in_luma_samples and pic_height_in_luma_samples) defined in the first video unit (e.g., PPS) are denoted as PW1 and PH1, respectively. The width and height of the codec / decoded picture defined in the second video unit (e.g., PPS) are denoted as PW2 and PH2.

[0561] a. In one example, in a conforming bitstream, W1 / W2 should be equal to X1 / X2.

[0562] i. Alternatively, in a conforming bitstream, W1 / X1 should equal W2 / X2.

[0563] ii. Alternatively, in a conforming bitstream, W1*X2 should be equal to W2*X1.

[0564] b. In one example, in a conforming bitstream, H1 / H2 should be equal to Y1 / Y2.

[0565] i. Alternatively, in a conforming bitstream, H1 / Y1 should equal H2 / Y2.

[0566] ii. Alternatively, in a conforming bitstream, H1*Y2 should be equal to H2*Y1.

[0567] c. In one example, in a conforming bitstream, PW1 / PW2 should be equal to X1 / X2.

[0568] i. Alternatively, in a conforming bitstream, PW1 / X1 should equal PW2 / X2.

[0569] ii. Alternatively, in a conforming bitstream, PW1*X2 should be equal to PW2*X1.

[0570] d. In one example, PH1 / PH2 should be equal to Y1 / Y2 in a conforming bitstream.

[0571] i. Alternatively, in a conforming bitstream, PH1 / Y1 should be equal to PH2 / Y2.

[0572] ii. Alternatively, in a conforming bitstream, PH1*Y2 should be equal to PH2*Y1.

[0573] e. In one example, in a conforming bitstream, PW1 / PW2 should be equal to W1 / W2.

[0574] i. Alternatively, in a conforming bitstream, PW1 / W1 should equal PW2 / W2.

[0575] ii. Alternatively, in a conforming bitstream, PW1*W2 should be equal to PW2*W1.

[0576] In one example, PH1 / PH2 should be equal to H1 / H2 in a conforming bitstream.

[0577] i. Alternatively, in a conforming bitstream, PH1 / H1 should be equal to PH2 / H2.

[0578] ii. Alternatively, in a conforming bitstream, PH1*H2 should be equal to PH2*H1.

[0579] g. In a conforming bitstream, if PW1 is greater than PW2, then W1 must be greater than W2.

[0580] h. In a conforming bitstream, if PW1 is less than PW2, then W1 must be less than W2.

[0581] i. In a conforming bitstream, (PW1-PW2)*(W1-W2) must be no less than 0.

[0582] j. In a conforming bitstream, if PH1 is greater than PH2, then H1 must be greater than H2.

[0583] k. In a conforming bitstream, if PH1 is less than PH2, then H1 must be less than H2.

[0584] l. In a conformant bitstream, (PH1-PH2)*(H1-H2) must be no less than 0.

[0585] m. In a conforming bitstream, if PW1>=PW2, then W1 / W2 must not be greater than (or less than) PW1 / PW2.

[0586] n. In a conforming bitstream, if PH1>=PH2, then H1 / H2 must not be greater than (or less than) PH1 / PH2.

[0587] 25. Assume that the width and height of the consistency window of the current picture are denoted as W and H respectively. The width and height of the consistency window of the reference picture are denoted as W' and H' respectively. Then, the conforming bitstream shall comply with at least one of the following constraints.

[0588] aW*pw>=W'; pw is an integer, for example, 2.

[0589] bW*pw>W'; pw is an integer, such as 2.

[0590] c. W'*pw'>=W; pw' is an integer, for example 8.

[0591] d.W'*pw'>W; pw' is an integer, such as 8.

[0592] eH*ph>=H'; ph is an integer, for example, 2.

[0593] fH*ph>H'; ph is an integer, such as 2.

[0594] g.H'*ph'>=H; ph' is an integer, for example 8.

[0595] h.H'*ph'>H; ph' is an integer, such as 8.

[0596] i. In one example, pw is equal to pw'.

[0597] j. In one example, ph is equal to ph'.

[0598] k. In one example, pw is equal to ph.

[0599] 1. In one example, pw' is equal to ph'.

[0600] In one example, when W and H represent the width and height of the current picture, respectively, the conforming bitstream needs to satisfy the above subitems. W' and H' represent the width and height of the reference picture, respectively.

[0601] 26. Partial signaling of consistency window parameters is proposed.

[0602] a. In one example, the upper left sample point in the consistency window of the picture is the same as the upper left sample point in the picture.

[0603] b. For example, the conf_win_left_offset defined in VVC is not signaled and is inferred to be zero.

[0604] c. For example, the conf_win_top_offset defined in VVC is not signaled and is inferred to be zero.

[0605] 27. The location of the reference sample point is proposed (for example, the reference sample point defined in VVC (refx L , refy L The derivation of )) can depend on the top left position of the consistency window of the current picture and / or the reference picture (e.g., (conf_win_left_offset, conf_win_top_offset) defined in VVC). Figure 4 shows an example of derivation of sample point positions according to VVC (a) and the proposed method (b). The dotted rectangle represents the consistency window.

[0606] a. In one example, a dependency exists only when the width and / or height of the current picture is different from the width and / or height of the reference picture.

[0607] b. In one example, deriving the horizontal position of the reference sample (e.g., refx as defined in VVC) L ) may depend on the left position of the consistency window of the current picture and / or reference picture (e.g., conf_win_left_offset as defined in VVC).

[0608] i. In one example, the horizontal position of the current sample relative to the upper left position of the consistency window in the current picture (denoted as xSb') is calculated and used to derive the position of the reference sample.

[0609] 1) For example, calculate xSb’ = xSb - (conf_win_left_offset << Prec), and use it to derive the position of the reference sample, where xSb represents the horizontal position of the current sample in the current picture. conf_win_left_offset represents the horizontal position of the top-left sample in the consistency window of the current picture. Prec represents the precision of xSb and xSb’, where (xSb >> Prec) can show the actual horizontal coordinate of the current sample relative to the current picture. For example, Prec = 0 or Prec = 4.

[0610] ii. In one example, calculate the horizontal position (denoted as Rx’) of the reference sample relative to the top-left position of the consistency window in the reference picture.

[0611] 1) The calculation of Rx’ can depend on xSb’ and / or the motion vector and / or the resampling ratio. iii. In one example, calculate the horizontal position (denoted as Rx) of the relative position of the reference sample in the reference picture according to Rx’.

[0612] position.

[0613] 1) For example, calculate Rx = Rx’ + (conf_win_left_offset_ref << Prec), where conf_win_left_offset_ref represents the horizontal position of the top-left sample in the consistency window of the reference picture. Prec represents the precision of Rx and Rx’. For example, Prec = 0 or Prec = 4.

[0614] iv. In one example, Rx can be directly calculated according to xSb’ and / or the motion vector and / or the resampling ratio. In other words, the two steps of deriving Rx’ and Rx are combined into one-step calculation.

[0615] v. Whether and / or how to use the left position of the consistency window of the current picture and / or the reference picture (e.g., conf_win_left_offset defined in VVC) can depend on the color component and / or the color format.

[0616] and / or color format.

[0617] 2) For example, conf_win_left_offset can be modified to conf_win_left_offset = conf_win_left_offset / SubWidthC, where SubWidthC defines the horizontal sampling step of the color component. For example, for the luminance component, SubWidthC is equal to 1. When the color format is 4:2:0 or 4:2:2, for the chrominance component, SubWidthC is equal to 2.

[0618] c. In one example, the derivation of the vertical position of the reference sample point (e.g., refy defined in VVC) L can depend on the top position of the consistency window of the current picture and / or the reference picture (e.g., conf_win_top_offset defined in VVC).

[0619] i. In one example, calculate the vertical position of the current sample point relative to the upper left position of the consistency window in the current picture (denoted as ySb’), and use it to derive the position of the reference sample point.

[0620] 1) For example, calculate ySb’ = ySb - (conf_win_top_offset << Prec) and use it to derive the position of the reference sample point, where ySb represents the vertical position of the current sample point in the current picture. conf_win_top_offset represents the vertical position of the upper left sample point in the consistency window of the current picture. Prec represents the precision of ySb and ySb’. For example, Prec = 0 or Prec = 4.

[0621] ii. In one example, calculate the vertical position of the reference sample point relative to the upper left position of the consistency window in the reference picture (denoted as Ry’).

[0622] 1) The calculation of Ry’ can depend on ySb’ and / or the motion vector and / or the resampling ratio.

[0623] iii. In one example, calculate the vertical position of the reference sample point relative to the reference picture based on Ry’ (denoted as Ry).

[0624] 1) For example, calculate Ry = Ry’ + (conf_win_top_offset_ref << Prec), where conf_win_top_offset_ref represents the vertical position of the upper left sample point in the consistency window of the reference picture. Prec represents the precision of Ry and Ry’. For example, Prec = 0 or Prec = 4.

[0625] iv. In one example, Ry can be directly calculated based on ySb' and / or the motion vector and / or the resampling ratio. In other words, the two steps of deriving Ry' and Ry are combined into one step.

[0626] v. Whether and / or how to use the top position of the consistency window of the current picture and / or reference picture (e.g., conf_win_top_offset defined in VVC) may depend on the color component and / or color format.

[0627] 1) For example, conf_win_top_offset can be modified to conf_win_top_offset = conf_win_top_offset * SubHeightC, where SubHeightC defines the vertical sampling step of the color component. For example, for the luma component, SubHeightC is equal to 1. When the color format is 4:2:0, for the chroma components, SubHeightC is equal to 2.

[0628] 2) For example, conf_win_top_offset can be modified to conf_win_top_offset = conf_win_top_offset / SubHeightC, where SubHeightC defines the vertical sampling step of the color component. For example, for the luma component, SubHeightC is equal to 1. When the color format is 4:2:0, for the chroma components, SubHeightC is equal to 2.

[0629] 28. It is proposed to clip the integer part of the horizontal coordinate of the reference sample to [minW, maxW]. Assume that the width and height of the consistency window of the reference picture are denoted by W and H respectively. The width and height of the consistency window of the reference picture are denoted by W' and H' respectively. The upper left position of the consistency window in the reference picture is denoted by (x0, y0).

[0630] a. In one example, minW is equal to 0.

[0631] b. In one example, minW is equal to X0.

[0632] c. In one example, maxW is equal to W-1.

[0633] d. In one example, maxW is equal to W'-1.

[0634] e. In one example, maxW is equal to X0+W'-1.

[0635] f. In one example, minW and / or maxW may be modified based on the color format and / or color components.

[0636] i. For example, change minW to minW*SubC.

[0637] ii. For example, change minW to minW / SubC.

[0638] iii. For example, change maxW to maxW*SubC.

[0639] iv. For example, change maxW to maxW / SubC.

[0640] v. In one example, for the luma component, SubC is equal to 1.

[0641] vi. In one example, when the color format is 4:2:0, SubC is equal to 2 for the chroma component.

[0642] vii. In one example, when the color format is 4:2:2, SubC is equal to 2 for the chroma components.

[0643] viii. In one example, when the color format is 4:4:4, SubC is equal to 1 for the chroma components.

[0644] g. In one example, whether and / or how pruning is performed may depend on the dimensions of the current picture (or its consistency window) and the dimensions of the reference picture (or its consistency window).

[0645] i. In one example, cropping is performed only if the dimensions of the current picture (or its consistency window) are different from the dimensions of the reference picture (or its consistency window).

[0646] 29. It is proposed to clip the integer part of the vertical coordinate of the reference sample to [minH, maxH]. Assume that the width and height of the consistency window of the reference picture are denoted by W and H respectively. The width and height of the consistency window of the reference picture are denoted by W' and H' respectively. The upper left position of the consistency window in the reference picture is denoted by (X0, Y0).

[0647] a. In one example, minH is equal to 0.

[0648] b. In one example, minH is equal to Y 0.

[0649] c. In one example, maxH is equal to H-1.

[0650] d. In one example, maxH is equal to H'-1.

[0651] e. In one example, maxH is equal to Y0+H'-1.

[0652] f. In one example, minH and / or maxH can be modified based on the color format and / or color components.

[0653] i. For example, change minH to minH*SubC.

[0654] ii. For example, change minH to minH / SubC.

[0655] iii. For example, modify maxH to ​​maxH*SubC.

[0656] iv. For example, change maxH to ​​maxH / SubC.

[0657] v. In one example, for the luma component, SubC is equal to 1.

[0658] vi. In one example, when the color format is 4:2:0, SubC is equal to 2 for the chroma component.

[0659] vii. In one example, when the color format is 4:2:2, SubC is equal to 1 for the chroma components.

[0660] viii. In one example, when the color format is 4:4:4, SubC is equal to 1 for the chroma components.

[0661] g. In one example, whether and / or how pruning is performed may depend on the dimensions of the current picture (or its consistency window) and the dimensions of the reference picture (or its consistency window).

[0662] i. In one example, cropping is performed only if the dimensions of the current picture (or its consistency window) are different from the dimensions of the reference picture (or its consistency window).

[0663] In the following discussion, if two syntax elements have equivalent functions but can be signaled in different video units (e.g., VPS / SPS / PPS / slice header / picture header, etc.), a first syntax element is said to "correspond to" a second syntax element.

[0664] 30. It is proposed to signal a syntax element in a first video unit (eg, picture header or PPS) and not signal the corresponding syntax element in a second video unit at a higher level (eg, SPS) or a lower level (eg, slice header).

[0665] a. Alternatively, a first syntax element may be signaled in a first video unit (eg, a picture header or PPS), and a corresponding second syntax element may be signaled in a lower-level second video unit (eg, a slice header).

[0666] i. Alternatively, an indicator may be signaled in the second video unit to inform whether the second syntax element is signaled thereafter.

[0667] ii. In one example, if a second syntax element is signaled, the slice (eg, slice header) associated with the second video unit may follow the indication of the second syntax element instead of the indication of the first syntax element.

[0668] iii. An indicator associated with the first syntax element may be signaled in the first video unit to inform whether the second syntax element is signaled in any slices (or other video units) associated with the first video unit.

[0669] b. Alternatively, a first syntax element (eg, VPS / SPS / PPS) may be signaled in a higher-level first video unit, and a corresponding second syntax element (eg, picture header) may be signaled in a second video unit.

[0670] i. Alternatively, an indicator may be signaled to inform whether the second syntax element is signaled thereafter.

[0671] ii. In one example, if a second syntax element is signaled, pictures associated with the second video unit (which may be partitioned into slices) may follow the indication of the second syntax element instead of the indication of the first syntax element.

[0672] c. The first syntax element in the picture header may have the same functionality as the second syntax element in the slice header specified in Section 2.6 (such as, but not limited to, slice_temporal_mvp_enabled_flag, cabac_init_flag, six_minus_max_num_merge_cand, five_minus_max_num_subblock_merge_cand, slice_fpel_mmvd_enabled_flag, slice_disable_bdof_dmvr_flag, max_num_merge_cand_minus_max_num_triangle_cand, slice_fpel_mmvd_enabled_flag, slice_six_minus_max_num_ibc_merge_cand, slice_joint_cbcr_sign_flag, slice_qp_delta, etc.), but may control all slices of the picture.

[0673] d. The first syntax element in the SPS specified in Section 2.6 may have the same functionality as the second syntax element in the picture header (such as, but not limited to, sps_bdof_dmvr_slice_present_flag, sps_mmvd_enabled_flag, sps_isp_enabled_flag, sps_mrl_enabled_flag, sps_mip_enabled_flag, sps_cclm_enabled_flag, sps_mts_enabled_flag, etc.), but may only control the associated picture (which may be partitioned into multiple slices).

[0674] e. The first syntax element in the PPS specified in Section 2.7 may have the same functionality as the second syntax element in the picture header (such as but not limited to entropy_coding_sync_enabled_flag, entry_point_offsets_present_flag, cabac_init_present_flag, rpl1_idx_present_flag, etc.), but may only control the associated picture (which may be partitioned into multiple slices).

[0675] 31. In this disclosure (Item 1-Item 29), the term "consistency window" may be replaced by other terms such as "scaling window." The scaling window may be signaled differently from the consistency window and used to derive a scaling ratio and / or an upper left offset, which are used to derive a reference sample position for RPR.

[0676] a. In one example, the scaling window can be constrained by a consistency window. For example, in a conformant bitstream, the scaling window must be contained within the consistency window.

[0677] 32. Whether and / or how the allowed maximum block size of blocks for transform-skip coding is signaled may depend on the maximum block size of blocks for transform-coding.

[0678] a. Alternatively, in a conforming bitstream, the maximum block size of a block used for transform-skip-coding cannot be larger than the maximum block size of a block used for transform-coding.

[0679] 33. Whether and how to signal the indication of enabling joint Cb-Cr residual (JCCR) codec (eg, sps_joint_cbcr_enabled_flag) may depend on the color format (eg, 4:0:0, 4:2:0, etc.).

[0680] a. For example, if the color format is 4:0:0, the indication of enabling joint Cb-Cr residual (JCCR) may not be signaled. The exemplary syntax design is as follows:

[0681]

[0682] Downsampling filter type used for chroma blending mask generation in TPM / GEO

[0683] 34. The type of downsampling filter used for mixing weight derivation of chroma samples can be signaled at the video unit level (e.g., SPS / VPS / PPS / picture header / sub-picture / slice / slice header / slice / brick / CTU / VPDU level).

[0684] a. In one example, a high-level flag may be signaled to switch between different chroma format types for content.

[0685] i. In one example, a high level flag may be signaled to switch between chroma format type 0 and chroma format type 2.

[0686] ii. In one example, a flag may be signaled to specify whether the luma weights for the left upper and lower samples are co-located with the left upper luma weight in TPM / GEO prediction mode (ie, chroma sample position type 0).

[0687] iii. In one example, a flag may be signaled to specify whether the luma samples of the lower left and upper samples in the TPM / GEO prediction mode are horizontally co-located with the upper left luma sample, but vertically shifted by 0.5 units relative to the upper left luma sample (i.e., sample position type 2).

[0688] b. In one example, the type of downsampling filter may be signaled for 4:2:0 chroma format and / or 4:2:2 chroma format.

[0689] c. In one example, a flag may be signaled to specify the type of chroma downsampling filter used for TPM / GEO prediction.

[0690] i. In one example, a flag may be signaled to determine whether to use downsampling filter A or downsampling filter B for chroma weight derivation in TPM / GEO prediction mode.

[0691] 35. The type of downsampling filter used for mixing weight derivation of chroma samples can be derived at the video unit level (e.g., SPS / VPS / PPS / picture header / sub-picture / slice / slice header / slice / brick / CTU / VPDU level).

[0692] a. In one example, a lookup table may be defined to specify the correspondence between chroma subsampling filter types and chroma format types of the content.

[0693] 36. In case of different chroma sample location types, the specified downsampling filter can be used for TPM / GEO prediction mode.

[0694] a. In one example, in the case of a certain chroma sample location type (eg, chroma sample location type 0), the chroma weights of the TPM / GEO may be subsampled from the co-located top-left luma weight.

[0695] b. In one example, in the case of a certain chroma sample position type (e.g., chroma sample position type 0 or 2), a specified X-tap filter (X is a constant, e.g., X=6 or 5) may be used for chroma weight subsampling in TPM / GEO prediction mode.

[0696] 37. In a video unit (eg, SPS, PPS, picture header, slice header, etc.), a first syntax element (eg, a flag) may be signaled to indicate whether MTS is disabled for all blocks (slices / pictures).

[0697] a. Conditionally signaling a second syntax element indicating how MTS is applied to intra-coded blocks (slices / pictures) (e.g., MTS enabled / disabled / implicit / explicit MTS) based on the first syntax element. For example, the second syntax element is signaled only when the first syntax element indicates that MTS is not disabled for all blocks (slices / pictures).

[0698] b. Conditionally signaling a third syntax element indicating how MTS is applied to inter-coded blocks (slices / pictures) (e.g., MTS enabled / disabled / implicit / explicit MTS) based on the first syntax element. For example, the third syntax element is signaled only when the first syntax element indicates that MTS is not disabled for all blocks (slices / pictures).

[0699] c. The exemplary grammar design is as follows:

[0700]

[0701] d. Conditionally signaling the third syntax element based on whether sub-block transform (SBT) is applied. An exemplary syntax design is as follows:

[0702]

[0703] e. The exemplary grammar design is as follows

[0704]

[0705] 5. Additional Embodiments

[0706] In the following, text changes are shown in underlined bold italic font.

[0707] 5.1 Implementation of Constraints on the Consistency Window

[0708] conf_win_left_offset, conf_win_right_offset, conf_win_top_offset, and conf_win_bottom_offset specify the picture samples in the CVS output from the decoding process, according to the rectangular area specified in picture coordinates for output. When conformance_window_flag is equal to 0, the values ​​of conf_win_left_offset, conf_win_right_offset, conf_win_top_offset, and conf_win_bottom_offset are inferred to be equal to 0.

[0709] The conforming cropping window contains luma samples with horizontal picture coordinates from SubWidthC*conf_win_left_offset to pic_width_in_luma_samples-(SubWidthC*conf_win_right_offset+1) and vertical picture coordinates from SubHeightC*conf_win_top_offset to pic_height_in_luma_samples-(SubHeightC*conf_win_bottom_offset+1), inclusive.

[0710] The value of SubWidthC*(conf_win_left_offset+conf_win_right_offset) should be less than pic_width_in_luma_samples, and the value of SubHeightC*(conf_win_top_offset+conf_win_bottom_offset) should be less than pic_height_in_luma_samples.

[0711] The variables PicOutputWidthL and PicOutputHeightL are derived as follows:

[0712] PicOutputWidthL=pic_width_in_luma_samples- (7-43)

[0713] SubWidthC*(conf_win_right_offset+conf_win_left_offset)PicOutputHeightL=pic_height_in_pic_size_units- (7-44)

[0714] SubHeightC*(conf_win_bottom_offset+conf_win_top_offset)

[0715] When ChromaArrayType is not equal to 0, the corresponding specified samples of the two chroma arrays are samples with picture coordinates (x / SubWidthC, y / SubHeightC), where (x, y) are the picture coordinates of the specified luma sample.

[0716]

[0717] 5.2 Example 1 of Derivation of Reference Sample Point Positions

[0718] 8.5.6.3.1 Overview

[0719] …

[0720] Set the variable fRefWidth equal to the PicOutputWidthL of the reference picture measured in luma samples.

[0721] Set the variable fRefHeight equal to the PicOutputHeightL of the reference picture measured in luma samples.

[0722]

[0723] Set the motion vector mvLX equal to (refMvLX-mvOffset).

[0724] – If cIdx is equal to 0, then the following applies:

[0725] - Define the scaling factor and its fixed-point representation as:

[0726] hori_scale_fp=((fRefWidth<<14)+(PicOutputWidthL>>1)) / PicOutputWidthL(8-753)

[0727] vert_scale_fp=((fRefHeight<<14)+(PicOutputHeightL>>1)) / PicOutputHeightL (8-754)

[0728] – Let (xIntL, yIntL) be the luma position in full sample units, and (xFracL, yFracL) be the offset in 1 / 16 sample units. These variables are used only in this clause to specify fractional sample positions within the reference sample array refPicLX.

[0729] – The upper left coordinate of the bounding block to be used for reference sample filling (xSbInt L ,ySbInt L ) is set equal to (xSb+(mvLX[0]>>4),ySb+(mvLX[1]>>4)).

[0730] – For each luma sample position (x L =0..sbWidth-1+brdExtSize,y L =0..sbHeight-1+brdExtSize), the corresponding predicted luminance sample value predSamplesLX[x L ][y L ]:

[0731] -Let (refxSb L ,refySb L ) and (refx L ,refy L ) is the luminance position pointed to by the motion vector (refMvLX[0], refMvLX[1]) given in units of 1 / 16 samples. The variable refxSb is derived as follows L 、refx L 、refySb L and refy L :

[0732]

[0733] refx L =((Sign(refxSb)*((Abs(refxSb)+128)>>8)+x L *((hori_scale_fp+8)>>4))+32)>>6 (8-756)

[0734]

[0735] refy L=((Sign(refySb)*((Abs(refySb)+128)>>8)+yL*((vert_scale_fp+8)>>4))+32)>>6 (8-758)

[0736]

[0737] – Derived variable xInt as follows L 、yInt L 、xFrac L and yFrac L :

[0738] xInt L =refx L >>4 (8-759)

[0739] yInt L =refy L >>4 (8-760)

[0740] xFrac L =refx L &15 (8-761)

[0741] yFrac L =refy L &15 (8-762)

[0742] If bdofFlag is true or (sps_affine_prof_enabled_flag is true and inter_affine_flag[xSb][ySb] is true), and one or more of the following conditions are true, then L +(xFrac L >>3)-1),yInt L +(yFrac L >>3)-1) and refPicLX as input, the predicted luminance sample value predSamplesLX[x L ][y L ].

[0743] –x L Equal to 0.

[0744] –x L Equal to sbWidth+1.

[0745] –y L Equal to 0.

[0746] –y L Equal to sbHeight+1.

[0747] – Otherwise, in the format of (xIntL-(brdExtSize>0?1:0),yIntL-(brdExtSize>0?1:0)), (xFracL,yFracL), (xSbInt L ,ySbInt L ), refPicLX, hpelIfIdx, sbWidth, sbHeight and (xSb, ySb) are input to derive the predicted luma sample value predSamplesLX[xL][yL] by calling the luma sample 8-tap interpolation filtering process specified in clause 8.5.6.3.2.

[0748] – Otherwise (cIdx is not equal to 0), then the following applies:

[0749] – Let (xIntC, yIntC) be the chroma position given in full sample units, and (xFracC, yFracC) be the offset given in 1 / 32 sample units. These variables are used only in this clause to specify regular fractional sample positions within the reference sample array refPicLX.

[0750] – Set the upper left coordinate (xSbIntC, ySbIntC) of the bounding block used for reference sample filling to be equal to ((xSb / SubWidthC)+(mvLX[0]>>5),(ySb / SubHeightC)+(mvLX[1]>>5)).

[0751] – For each chroma sample position in the predicted chroma sample array predSamplesLX (xC = 0..sbWidth-1, yC = 0..sbHeight-1), the corresponding predicted chroma sample value predSamplesLX[xC][yC] is derived as follows:

[0752] –Let (refxSb C ,refySb C ) and (refx C ,refy C ) is the chroma position pointed to by the motion vector (mvLX[0], mvLX[1]) given in units of 1 / 32 samples. The variable refxSb is derived as follows C 、refySb C 、refx C and refy C :

[0753]

[0754] refx C =((Sign(refxSb C )*((Abs(refxSb C )+256)>>9)+xC*((hori_scale_fp+8)>>4))+16)>>5 (8-764)

[0755]

[0756] refy C =((Sign(refySb C )*((Abs(refySb C )+256)>>9)+yC*((vert_scale_fp+8)>>4))+16)>> 5(8-766)

[0757]

[0758]

[0759] – Derived variable xInt as follows C 、yInt C 、xFrac C and yFrac C :

[0760] xInt C =refx C >>5 (8-767)

[0761] yInt C =refy C >>5 (8-768)

[0762] xFrac C =refy C &31 (8-769)

[0763] yFrac C =refy C &31 (8-770)

[0764] 5.3 Example 2 of Derivation of Reference Sample Point Positions

[0765] 8.5.6.3.1 Overview

[0766] …

[0767] Set the variable fRefWidth equal to the PicOutputWidthL of the reference picture measured in luma samples.

[0768] Set the variable fRefHeight equal to the PicOutputHeightL of the reference picture measured in luma samples.

[0769]

[0770] Set the motion vector mvLX equal to (refMvLX-mvOffset).

[0771] – If cIdx is equal to 0, then the following applies:

[0772] - Define the scaling factor and its fixed-point representation as:

[0773] hori_scale_fp=((fRefWidth<<14)+(PicOutputWidthL>>1)) / PicOutputWidthL(8-753)

[0774] vert_scale_fp=((fRefHeight<<14)+(PicOutputHeightL>>1)) / PicOutputHeightL (8-754)

[0775] – Let (xIntL, yIntL) be the luma position in full sample units, and (xFracL, yFracL) be the offset in 1 / 16 sample units. These variables are used only in this clause to specify fractional sample positions within the reference sample array refPicLX.

[0776] – The upper left coordinate of the bounding block to be used for reference sample filling (xSbInt L ,ySbInt L ) is set equal to (xSb+(mvLX[0]>>4),ySb+(mvLX[1]>>4)).

[0777] – For each luma sample position (x L =0..sbWidth-1+brdExtSize,y L =0..sbHeight-1+brdExtSize), the corresponding predicted luminance sample value predSamplesLX[x L ][y L ]:

[0778] -Let (refxSb L ,refySb L ) and (refx L ,refy L ) is the luminance position pointed to by the motion vector (refMvLX[0], refMvLX[1]) given in units of 1 / 16 samples. The variable refxSb is derived as follows L 、refx L 、refySb L and refy L :

[0779]

[0780] – Derived variable xInt as follows L 、yInt L 、xFrac L and yFrac L :

[0781] xInt L =refx L >>4 (8-759)

[0782] yInt L =refy L >>4 (8-760)

[0783] xFrac L =refx L &15 (8-761)

[0784] yFrac L =refy L &15 (8-762)

[0785] If bdofFlag is true or (sps_affine_prof_enabled_flag is true and inter_affine_flag[xSb][ySb] is true), and one or more of the following conditions are true, then L +(xFrac L >>3)-1),yInt L +(yFrac L >>3)-1) and refPicLX as input, the predicted luminance sample value predSamplesLX[x L ][y L ].

[0786] –x L Equal to 0.

[0787] –x L Equal to sbWidth+1.

[0788] –y L Equal to 0.

[0789] –y L Equal to sbHeight+1.

[0790] – Otherwise, in the format of (xIntL-(brdExtSize>0?1:0),yIntL-(brdExtSize>0?1:0)), (xFracL,yFracL), (xSbInt L ,ySbInt L ), refPicLX, hpelIfIdx, sbWidth, sbHeight and (xSb, ySb) are input to derive the predicted luma sample value predSamplesLX[xL][yL] by calling the luma sample 8-tap interpolation filtering process specified in clause 8.5.6.3.2.

[0791] – Otherwise (cIdx is not equal to 0), then the following applies:

[0792] – Let (xIntC, yIntC) be the chroma position given in full sample units, and (xFracC, yFracC) be the offset given in 1 / 32 sample units. These variables are used only in this clause to specify regular fractional sample positions within the reference sample array refPicLX.

[0793] – Set the upper left coordinate (xSbIntC, ySbIntC) of the bounding block used for reference sample filling to be equal to ((xSb / SubWidthC)+(mvLX[0]>>5),(ySb / SubHeightC)+(mvLX[1]>>5)).

[0794] – For each chroma sample position in the predicted chroma sample array predSamplesLX (xC = 0..sbWidth-1, yC = 0..sbHeight-1), the corresponding predicted chroma sample value predSamplesLX[xC][yC] is derived as follows:

[0795] –Let (refxSb C ,refySb C ) and (refx C ,refy C) is the chroma position pointed to by the motion vector (mvLX[0], mvLX[1]) given in units of 1 / 32 samples. The variable refxSb is derived as follows C 、refySb C 、refx C and refy C :

[0796]

[0797] refx C =((Sign(refxSb C )*((Abs(refxSb C )+256)>>9)+xC*((hori_scale_fp+8)>>4))+16)>>5 (8-764)

[0798]

[0799] refy C =((Sign(refySb C )*((Abs(refySb C )+256)>>9)+yC*((vert_scale_fp+8)>>4))+16)>> 5(8-766)

[0800]

[0801] – Derived variable xInt as follows C 、yInt C 、xFrac C and yFrac C :

[0802] xInt C =refx C >>5 (8-767)

[0803] yInt C =refy C >>5 (8-768)

[0804] xFrac C =refy C &31 (8-769)

[0805] yFrac C =refy C &31 (8-770)

[0806] 5.4 Example 3 of Derivation of Reference Sample Point Positions

[0807] 8.5.6.3.1 Overview

[0808] …

[0809] Set the variable fRefWidth equal to the PicOutputWidthL of the reference picture measured in luma samples.

[0810] Set the variable fRefHeight equal to the PicOutputHeightL of the reference picture measured in luma samples.

[0811]

[0812] Set the motion vector mvLX equal to (refMvLX-mvOffset).

[0813] – If cIdx is equal to 0, then the following applies:

[0814] - Define the scaling factor and its fixed-point representation as:

[0815] hori_scale_fp=((fRefWidth<<14)+(PicOutputWidthL>>1)) / PicOutputWidthL(8-753)

[0816] vert_scale_fp=((fRefHeight<<14)+(PicOutputHeightL>>1)) / PicOutputHeightL (8-754)

[0817] – Let (xIntL, yIntL) be the luma position in full sample units, and (xFracL, yFracL) be the offset in 1 / 16 sample units. These variables are used only in this clause to specify fractional sample positions within the reference sample array refPicLX.

[0818] – The upper left coordinate of the bounding block to be used for reference sample filling (xSbInt L ,ySbInt L ) is set equal to (xSb+(mvLX[0]>>4),ySb+(mvLX[1]>>4)).

[0819] – For each luma sample position (x L =0..sbWidth-1+brdExtSize,y L=0..sbHeight-1+brdExtSize), the corresponding predicted luminance sample value predSamplesLX[x L ][y L ]:

[0820] -Let (refxSb L ,refySb L ) and (refx L ,refy L ) is the luminance position pointed to by the motion vector (refMvLX[0], refMvLX[1]) given in units of 1 / 16 samples. The variable refxSb is derived as follows L 、refx L 、refySb L and refy L :

[0821]

[0822] – Derived variable xInt as follows L 、yInt L 、xFrac L and yFrac L :

[0823] xInt L =refx L >>4 (8-759)

[0824] yInt L =refy L >>4 (8-760)

[0825] xFrac L =refx L &15 (8-761)

[0826] yFrac L =refy L &15 (8-762)

[0827] If bdofFlag is true or (sps_affine_prof_enabled_flag is true and inter_affine_flag[xSb][ySb] is true), and one or more of the following conditions are true, then L +(xFrac L >>3)-1),yInt L +(yFrac L>>3)-1) and refPicLX as input, the predicted luminance sample value predSamplesLX[x L ][y L ].

[0828] –x L Equal to 0.

[0829] –x L Equal to sbWidth+1.

[0830] –y L Equal to 0.

[0831] –y L Equal to sbHeight+1.

[0832] – Otherwise, in the format of (xIntL-(brdExtSize>0?1:0),yIntL-(brdExtSize>0?1:0)), (xFracL,yFracL), (xSbInt L ,ySbInt L ), refPicLX, hpelIfIdx, sbWidth, sbHeight and (xSb, ySb) are input to derive the predicted luma sample value predSamplesLX[xL][yL] by calling the luma sample 8-tap interpolation filtering process specified in clause 8.5.6.3.2.

[0833] – Otherwise (cIdx is not equal to 0), then the following applies:

[0834] – Let (xIntC, yIntC) be the chroma position given in full sample units, and (xFracC, yFracC) be the offset given in 1 / 32 sample units. These variables are used only in this clause to specify regular fractional sample positions within the reference sample array refPicLX.

[0835] – Set the upper left coordinate (xSbIntC, ySbIntC) of the bounding block used for reference sample filling to be equal to ((xSb / SubWidthC)+(mvLX[0]>>5),(ySb / SubHeightC)+(mvLX[1]>>5)).

[0836] – For each chroma sample position in the predicted chroma sample array predSamplesLX (xC = 0..sbWidth-1, yC = 0..sbHeight-1), the corresponding predicted chroma sample value predSamplesLX[xC][yC] is derived as follows:

[0837] –Let (refxSb C ,refySb C ) and (refx C ,refy C ) is the chroma position pointed to by the motion vector (mvLX[0], mvLX[1]) given in units of 1 / 32 samples. The variable refxSb is derived as follows C 、refySb C 、refx C and refy C :

[0838]

[0839]

[0840] – Derived variable xInt as follows C 、yInt C 、xFrac C and yFrac C :

[0841] xInt C =refx C >>5 (8-767)

[0842] yInt C =refy C >>5 (8-768)

[0843] xFrac C =refy C &31 (8-769)

[0844] yFrac C =refy C &31 (8-770)

[0845] 5.5 Example 1 of Reference Sample Point Position Cropping

[0846] 8.5.6.3.1 Overview

[0847] The inputs to this process are:

[0848] – Luma position (xSb, ySb), specifies the top left sample of the current codec sub-block relative to the top left luma sample of the current picture,

[0849] -Variable sbWidth specifies the width of the current codec sub-block.

[0850] -Variable sbHeight specifies the height of the current codec sub-block.

[0851] – motion vector offset mvOffset,

[0852] – refined motion vector refMvLX,

[0853] – the selected reference picture sample array refPicLX,

[0854] – Half-sample interpolation filter index hpelIfIdx,

[0855] – bidirectional optical flow flag bdofFlag,

[0856] – The variable cIdx specifies the color component index of the current block.

[0857] The output of this process is:

[0858] – predSamplesLX, an array of (sbWidth+brdExtSize)x(sbHeight+brdExtSize) predicted sample values.

[0859] The prediction block boundary extension size brdExtSize is derived as follows:

[0860] brdExtSize=(bdofFlag||(inter_affine_flag[xSb][ySb]&&sps_affine_prof_enabled_flag))? 2:0(8-752)

[0861] Set the variable fRefWidth equal to the PicOutputWidthL of the reference picture measured in luma samples.

[0862] Set the variable fRefHeight equal to the PicOutputHeightL of the reference picture measured in luma samples.

[0863] Set the motion vector mvLX equal to (refMvLX-mvOffset).

[0864] – If cIdx is equal to 0, then the following applies:

[0865] - Define the scaling factor and its fixed-point representation as:

[0866] hori_scale_fp=((fRefWidth<<14)+(PicOutputWidthL>>1)) / PicOutputWidthL(8-753)

[0867] vert_scale_fp=((fRefHeight<<14)+(PicOutputHeightL>>1)) / PicOutputHeightL (8-754)

[0868] – Let (xIntL, yIntL) be the luma position in full sample units, and (xFracL, yFracL) be the offset in 1 / 16 sample units. These variables are used only in this clause to specify fractional sample positions within the reference sample array refPicLX.

[0869] – The upper left coordinate of the bounding block to be used for reference sample filling (xSbInt L ,ySbInt L ) is set equal to (xSb+(mvLX[0]>>4),ySb+(mvLX[1]>>4)).

[0870] – For each luma sample position (x L =0..sbWidth-1+brdExtSize,y L =0..sbHeight-1+brdExtSize), the corresponding predicted luminance sample value predSamplesLX[x L ][y L ]:

[0871] -Let (refxSb L ,refySb L ) and (refx L ,refy L ) is the luminance position pointed to by the motion vector (refMvLX[0], refMvLX[1]) given in units of 1 / 16 samples. The variable refxSb is derived as follows L 、refx L 、refySb L and refy L :

[0872] refxSb L=((xSb<<4)+refMvLX[0])*hori_scale_fp (8-755)

[0873] refx L =((Sign(refxSb)*((Abs(refxSb)+128)>>8)+x L *((hori_scale_fp+8)>>4))+32)>>6 (8-756)

[0874] refySb L =((ySb<<4)+refMvLX[1])*vert_scale_fp (8-757)

[0875] refy L =((Sign(refySb)*((Abs(refySb)+128)>>8)+yL*((vert_scale_fp+8)>>4))+32)>>6 (8-758)

[0876] – Derived variable xInt as follows L 、yInt L 、xFrac L and yFrac L :

[0877]

[0878] xFrac L =refx L &15 (8-761)

[0879] yFrac L =refy L &15 (8-762)

[0880] – If bdofFlag is equal to true or (sps_affine_prof_enabled_flag is equal to true and inter_affine_flag[xSb][ySb] is equal to true), and one or more of the following conditions are true, then L +(xFrac L >>3)-1),yInt L +(yFrac L >>3)-1) and refPicLX are input by calling the luminance integer sample retrieval process specified in clause 8.5.6.3.3 to derive the predicted luminance sample value predSamplesLX[x L ][y L].

[0881] –x L Equal to 0.

[0882] –x L Equal to sbWidth+1.

[0883] –y L Equal to 0.

[0884] –y L Equal to sbHeight+1.

[0885] – Otherwise, in the format of (xIntL-(brdExtSize>0?1:0),yIntL-(brdExtSize>0?1:0)), (xFracL,yFracL), (xSbInt L ,ySbInt L ), refPicLX, hpelIfIdx, sbWidth, sbHeight and (xSb, ySb) are input to derive the predicted luma sample value predSamplesLX[xL][yL] by calling the luma sample 8-tap interpolation filtering process specified in clause 8.5.6.3.2.

[0886] – Otherwise (cIdx is not equal to 0), then the following applies:

[0887] – Let (xIntC, yIntC) be the chroma position given in full sample units, and (xFracC, yFracC) be the offset given in 1 / 32 sample units. These variables are used only in this clause to specify regular fractional sample positions within the reference sample array refPicLX.

[0888] – Set the upper left coordinate (xSbIntC, ySbIntC) of the bounding block used for reference sample filling to be equal to ((xSb / SubWidthC)+(mvLX[0]>>5),(ySb / SubHeightC)+(mvLX[1]>>5)).

[0889] For each chroma sample position in the predicted chroma sample array predSamplesLX (xC = 0..sbWidth-1, yC = 0..sbHeight-1), the corresponding predicted chroma sample value predSamplesLX[xC][yC] is derived as follows:

[0890] –Let (refxSb C ,refySb C ) and (refx C ,refyC ) is the chroma position pointed to by the motion vector (mvLX[0], mvLX[1]) given in units of 1 / 32 samples. The variable refxSb is derived as follows C 、refySb C 、refx C and refy C :

[0891] refxSb C =((xSb / SubWidthC<<5)+mvLX[0])*hori_scale_fp (8-763)

[0892] refx C =((Sign(refxSb C )*((Abs(refxSb C )+256)>>9)+xC*((hori_scale_fp+8)>>4))+16)>>5 (8-764)

[0893] refySb C =((ySb / SubHeightC<<5)+mvLX[1])*vert_scale_fp(8-765)

[0894] refy C =((Sign(refySb C )*((Abs(refySb C )+256)>>9)+yC*((vert_scale_fp+8)>>4))+16)>> 5(8-766)

[0895] – Derived variable xInt as follows C 、yInt C 、xFrac C and yFrac C :

[0896]

[0897] xFrac C =refy C &31 (8-769)

[0898] yFrac C =refy C &31 (8-770)

[0899] – Derives the predicted sample values ​​predSamplesLX[xC][yC] by invoking the procedure specified in clause 8.5.6.3.4 with (xIntC,yIntC), (xFracC,yFracC), (xSbIntC,ySbIntC), sbWidth, sbHeight and refPicLX as input.

[0900] 5.6 Example 2 of Reference Sample Point Position Cropping

[0901] 8.5.6.3.1 Overview

[0902] The inputs to this process are:

[0903] – Luma position (xSb, ySb), specifies the top left sample of the current codec sub-block relative to the top left luma sample of the current picture,

[0904] -Variable sbWidth specifies the width of the current codec sub-block.

[0905] -Variable sbHeight specifies the height of the current codec sub-block.

[0906] – motion vector offset mvOffset,

[0907] – refined motion vector refMvLX,

[0908] – the selected reference picture sample array refPicLX,

[0909] – Half-sample interpolation filter index hpelIfIdx,

[0910] – bidirectional optical flow flag bdofFlag,

[0911] – The variable cIdx specifies the color component index of the current block.

[0912] The output of this process is:

[0913] – predSamplesLX, an array of (sbWidth+brdExtSize)x(sbHeight+brdExtSize) predicted sample values.

[0914] The prediction block boundary extension size brdExtSize is derived as follows:

[0915] brdExtSize=(bdofFlag||(inter_affine_flag[xSb][ySb]&&sps_affine_prof_enabled_flag))? 2:0 (8-752)

[0916] Set the variable fRefWidth equal to the PicOutputWidthL of the reference picture measured in luma samples.

[0917] Set the variable fRefHeight equal to the PicOutputHeightL of the reference picture measured in luma samples.

[0918]

[0919] Set the motion vector mvLX equal to (refMvLX-mvOffset).

[0920] – If cIdx is equal to 0, then the following applies:

[0921] - Define the scaling factor and its fixed-point representation as:

[0922] hori_scale_fp=((fRefWidth<<14)+(PicOutputWidthL>>1)) / PicOutputWidthL(8-753)

[0923] vert_scale_fp=((fRefHeight<<14)+(PicOutputHeightL>>1)) / PicOutputHeightL (8-754)

[0924] – Let (xIntL, yIntL) be the luma position in full sample units, and (xFracL, yFracL) be the offset in 1 / 16 sample units. These variables are used only in this clause to specify fractional sample positions within the reference sample array refPicLX.

[0925] – The upper left coordinate of the bounding block to be used for reference sample filling (xSbInt L ,ySbInt L ) is set equal to (xSb+(mvLX[0]>>4),ySb+(mvLX[1]>>4)).

[0926] – For each luma sample position (x L =0..sbWidth-1+brdExtSize,y L =0..sbHeight-1+brdExtSize), the corresponding predicted luminance sample value predSamplesLX[x L ][y L ]:

[0927] -Let (refxSb L ,refySb L ) and (refx L ,refy L ) is the luminance position pointed to by the motion vector (refMvLX[0], refMvLX[1]) given in units of 1 / 16 samples. The variable refxSb is derived as follows L 、refx L 、refySb L and refy L :

[0928] refxSb L =((xSb<<4)+refMvLX[0])*hori_scale_fp (8-755)

[0929] refx L =((Sign(refxSb)*((Abs(refxSb)+128)>>8)+x L *((hori_scale_fp+8)>>4))+32)>>6 (8-756)

[0930] refySb L =((ySb<<4)+refMvLX[1])*vert_scale_fp (8-757)

[0931] refy L =((Sign(refySb)*((Abs(refySb)+128)>>8)+yL*((vert_scale_fp+8)>>4))+32)>>6 (8-758)

[0932] – Derived variable xInt as follows L 、yInt L 、xFrac L and yFrac L :

[0933]

[0934] xFrac L =refx L &15 (8-761)

[0935] yFrac L =refy L &15 (8-762)

[0936] – If bdofFlag is equal to true or (sps_affine_prof_enabled_flag is equal to true and inter_affine_flag[xSb][ySb] is equal to true), and one or more of the following conditions are true, then L +(xFrac L >>3)-1),yInt L +(yFrac L >>3)-1) and refPicLX are input by calling the luminance integer sample retrieval process specified in clause 8.5.6.3.3 to derive the predicted luminance sample value predSamplesLX[x L ][y L ].

[0937] –x L Equal to 0.

[0938] –x L Equal to sbWidth+1.

[0939] –y L Equal to 0.

[0940] –y L Equal to sbHeight+1.

[0941] – Otherwise, in the format of (xIntL-(brdExtSize>0?1:0),yIntL-(brdExtSize>0?1:0)), (xFracL,yFracL), (xSbInt L ,ySbInt L ), refPicLX, hpelIfIdx, sbWidth, sbHeight and (xSb, ySb) are input to derive the predicted luma sample value predSamplesLX[xL][yL] by calling the luma sample 8-tap interpolation filtering process specified in clause 8.5.6.3.2.

[0942] – Otherwise (cIdx is not equal to 0), then the following applies:

[0943] – Let (xIntC, yIntC) be the chroma position given in full sample units, and (xFracC, yFracC) be the offset given in 1 / 32 sample units. These variables are used only in this clause to specify regular fractional sample positions within the reference sample array refPicLX.

[0944] – Set the upper left coordinate (xSbIntC, ySbIntC) of the bounding block used for reference sample filling to be equal to ((xSb / SubWidthC)+(mvLX[0]>>5),(ySb / SubHeightC)+(mvLX[1]>>5)).

[0945] For each chroma sample position in the predicted chroma sample array predSamplesLX (xC = 0..sbWidth-1, yC = 0..sbHeight-1), the corresponding predicted chroma sample value predSamplesLX[xC][yC] is derived as follows:

[0946] –Let (refxSb C ,refySb C ) and (refx C ,refy C ) is the chroma position pointed to by the motion vector (mvLX[0], mvLX[1]) given in units of 1 / 32 samples. The variable refxSb is derived as follows C 、refySb C 、refx C and refy C :

[0947] refxSb C =((xSb / SubWidthC<<5)+mvLX[0])*hori_scale_fp (8-763)

[0948] refx C =((Sign(refxSb C )*((Abs(refxSb C )+256)>>9)+xC*((hori_scale_fp+8)>>4))+16)>>5 (8-764)

[0949] refySb C =((ySb / SubHeightC<<5)+mvLX[1])*vert_scale_fp (8-765)

[0950] refy C =((Sign(refySb C )*((Abs(refySb C )+256)>>9)+yC*((vert_scale_fp+8)>>4))+16)>> 5(8-766)

[0951] – Derived variable xInt as followsC 、yInt C 、xFrac C and yFrac C :

[0952]

[0953] xFrac C =refy C &31 (8-769)

[0954] yFrac C =refy C &31 (8-770)

[0955] – Derives the predicted sample values ​​predSamplesLX[xC][yC] by invoking the procedure specified in clause 8.5.6.3.4 with (xIntC,yIntC), (xFracC,yFracC), (xSbIntC,ySbIntC), sbWidth, sbHeight and refPicLX as input.

[0956] 6. Example Implementations of the Disclosed Technology

[0957] Figure 5 is a block diagram of a video processing device 500. The device 500 can be used to implement one or more methods described herein. The device 500 can be embodied in a smartphone, a tablet, a computer, an Internet of Things (IoT) receiver, etc. The device 500 may include one or more processors 502, one or more memories 504, and video processing hardware 506. The processor(s) 502 can be configured to implement one or more methods described in this document. The memory(s) 504 can be used to store data and code for implementing the methods and techniques described herein. The video processing hardware 506 can be used to implement some of the techniques described in this document in hardware circuits and can be partially or completely part of the processor 502 (e.g., a graphics processor core GPU or other signal processing circuitry).

[0958] In this document, the term "video processing" or codec may refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during the conversion from a pixel representation of a video to a corresponding bitstream representation, or vice versa. The bitstream representation of the current video block may correspond to bits located at the same position within the bitstream or distributed at different positions, as defined by the syntax. For example, a macroblock may be encoded based on error residual values ​​from transforms and codecs, and also using bits in the header and other fields in the bitstream.

[0959] It will be appreciated that the disclosed methods and techniques will benefit video encoder and / or decoder embodiments incorporated within video processing devices such as smartphones, laptops, desktops, and similar devices by enabling use of the techniques disclosed in this document.

[0960] Figure 6 6 is a flow chart of an example method 600 for video processing. The method 600 includes, at 610, performing a conversion between a current video block and a codec representation of the current video block, wherein during the conversion, if a resolution and / or size of a reference picture is different from a resolution and / or size of the current video block, applying the same interpolation filter to a group of adjacent or non-adjacent samples predicted using the current video block.

[0961] Some embodiments may be described using the following clause-based format.

[0962] 1. A video processing method, comprising:

[0963] Performing a conversion between a current video block and a codec representation of the current video block, wherein, during the conversion, if a resolution and / or size of a reference picture differs from a resolution and / or size of the current video block, applying the same interpolation filter to a group of adjacent or non-adjacent samples predicted using the current video block.

[0964] 2. The method of clause 1, wherein the same interpolation filter is a vertical interpolation filter.

[0965] 3. The method of clause 1, wherein the same interpolation filter is a horizontal interpolation filter.

[0966] 4. The method of clause 1, wherein the group of adjacent or non-adjacent samples comprises all samples located in the region of the current video block.

[0967] 5. The method of clause 4, wherein the current video block is divided into a plurality of rectangles each having a size of M x N .

[0968] 6. The method according to clause 5, wherein M and / or N are predetermined.

[0969] 7. The method of clause 5, wherein M and / or N are derived from the dimensions of the current video block.

[0970] 8. A method according to clause 5, wherein M and / or N are signalled in a codec representation of the current video block.

[0971] 9. The method of clause 1, wherein the groups of samples share the same motion vector.

[0972] 10. The method of clause 9, wherein the groups of samples share the same horizontal component and / or the same fractional part of the horizontal component.

[0973] 11. The method of clause 9, wherein the groups of samples share the same vertical component and / or the same fractional part of the vertical component.

[0974] 12. A method according to one or more of clauses 9-11, wherein the same motion vector or its components satisfy at least one or more rules based on at least one of the following: resolution of the reference picture, size of the reference picture, resolution of the current video block, size of the current video block, or precision value.

[0975] 13. Method according to one or more of clauses 9-11, wherein the same motion vector or components thereof correspond to motion information of samples located in the current video block.

[0976] 14. Method according to one or more of clauses 9-11, wherein the same motion vector or its components are set as motion information for virtual samples located inside or outside the group.

[0977] 15. A video processing method, comprising:

[0978] Performing a conversion between a current video block and a codec representation of the current video block, wherein, during the conversion, if a resolution and / or size of a reference picture differs from the resolution and / or size of the current video block, only blocks predicted using the current video block are permitted to use integer-valued motion information related to the current block.

[0979] 16. The method of clause 15, wherein the integer-valued motion information is derived by rounding raw motion information of the current video block.

[0980] 17. The method of clause 15, wherein the original motion information of the current video block is in the horizontal direction and / or the vertical direction.

[0981] 18. A video processing method, comprising:

[0982] Performing a conversion between a current video block and a codec representation of the current video block, wherein, during the conversion, if a resolution and / or size of a reference picture differs from a resolution and / or size of the current video block, an interpolation filter is applied to derive a block predicted using the current video block, and wherein the interpolation filter is selected based on a rule.

[0983] 19. The method of clause 18, wherein the rule relates to the resolution and / or size of the reference picture relative to the resolution and / or size of the current video block.

[0984] 20. The method of clause 18, wherein the interpolation filter is a vertical interpolation filter.

[0985] 21. The method of clause 18, wherein the interpolation filter is a horizontal interpolation filter.

[0986] 22. The method of clause 18, wherein the interpolation filter is one of: a 1-tap filter, a bilinear filter, a 4-tap filter, or a 6-tap filter.

[0987] 23. A method according to clause 22, wherein an interpolation filter is used as part of the other steps of the conversion.

[0988] 24. The method of clause 18, wherein the interpolation filter comprises the use of padding samples.

[0989] 25. The method of clause 18, wherein use of the interpolation filter depends on the color components of the samples of the current video block.

[0990] 26. A video processing method, comprising:

[0991] Performing a conversion between a current video block and a codec representation of the current video block, wherein during the conversion, if a resolution and / or size of a reference picture differs from a resolution and / or size of the current video block, selectively applying a deblocking filter, wherein a strength of the deblocking filter is set according to a rule related to the resolution and / or size of the reference picture relative to the resolution and / or size of the current video block.

[0992] 27. The method of clause 27, wherein the strength of the deblocking filter varies from one video block to another.

[0993] 28. A video processing method, comprising:

[0994] Performing a conversion between a current video block and a codec representation of the current video block, wherein during the conversion, if a sub-picture of the current video block exists, the conforming bitstream satisfies rules regarding the resolution and / or size of reference pictures relative to the resolution and / or size of the current video block.

[0995] 29. The method according to clause 28, further comprising:

[0996] The current video block is divided into one or more sub-pictures, where the division depends at least on the resolution of the current video block.

[0997] 30. A video processing method, comprising:

[0998] A conversion between the current video block and a codec representation of the current video block is performed, wherein during the conversion, reference pictures of the current video block are resampled according to rules based on a dimension of the current video block.

[0999] 31. A video processing method, comprising:

[1000] A conversion between a current video block and a codec representation of the current video block is performed, wherein during the conversion, use of codec tools for the current video block is selectively enabled or disabled depending on a resolution / size of a reference picture of the current video block relative to a resolution / size of the current video block.

[1001] 32. The method according to one or more of the preceding clauses, wherein the group of samples is located within a coherence window.

[1002] 33. The method of clause 32, wherein the consistency window is rectangular in shape.

[1003] 34. Method according to any one or more of the preceding clauses, wherein the resolution belongs to the resolution of a coded / decoded video block or to the resolution of a consistency window in a coded / decoded video block.

[1004] 35. Method according to any one or more of the preceding clauses, wherein the size is of a coded / decoded video block or of a consistency window in a coded / decoded video block.

[1005] 36. Method according to any one or more of the preceding clauses, wherein the dimensions belong to dimensions of a coded / decoded video block or dimensions of a consistency window in a coded / decoded video block.

[1006] 37. The method of clause 32, wherein the consistency window is defined by a consistency clipping window parameter set.

[1007] 38. A method according to clause 37, wherein at least a part of the conforming cropping window parameter set is signalled implicitly or explicitly in the codec representation.

[1008] 39. A method according to any one or more of the preceding clauses, wherein signalling of a consistent cropping window parameter set in the codec representation is not allowed.

[1009] 40. A method according to any one or more of the preceding clauses, wherein the position of the reference sample is derived relative to the top left sample of the current video block in the consistency window.

[1010] 41. A video processing method, comprising:

[1011] A conversion between a plurality of video blocks and codec representations of the plurality of video blocks is performed, wherein, during the conversion, a first consistency window for a first video block and a second consistency window for a second video block are defined, and wherein a ratio of a width and / or a height of the first consistency window to the second consistency window is according to a rule based at least on a conforming bitstream.

[1012] 42. A video decoding apparatus comprising a processor configured to implement the method of one or more of clauses 1-41.

[1013] 43. A video encoding apparatus comprising a processor configured to implement the method of one or more of clauses 1-41.

[1014] 44. A computer program product having computer code stored thereon which, when executed by a processor, causes the processor to implement the method of any one of clauses 1 to 41.

[1015] 45. A method, apparatus or system as herein described.

[1016] Figure 7 is a block diagram illustrating an example video processing system 700 in which the various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 700. System 700 may include an input 702 for receiving video content. The video content may be received in a raw or uncompressed format, such as 8-bit or 10-bit multi-component pixel values, or may be received in a compressed or encoded format. Input 702 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces (e.g., Ethernet, passive optical networks (PONs), etc.) and wireless interfaces (e.g., Wi-Fi or cellular interfaces).

[1017] System 700 may include a codec component 704 that implements various codecs or encoding methods described in this document. Codec component 704 can reduce the average bit rate of the video from input 702 to the output of codec component 704 to generate a codec representation of the video. Therefore, codec technology is sometimes referred to as video compression or video transcoding technology. As represented by component 706, the output of codec component 704 can be stored or sent via the connected communication. The stored or transmitted bitstream (or codec) representation of the video received at input 702 can be used by the component. 708 is used to generate pixel values ​​or displayable video, which are sent to display interface 710. The process of generating user-visible video from the bitstream representation is sometimes referred to as video decompression. In addition, although some video processing operations are referred to as "codec" operations or tools, it should be understood that the codec tools or operations are used at the encoder, and the corresponding decoding tools or operations of the inverse codec results will be performed by the decoder.

[1018] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB), High-Definition Multimedia Interface (HDMI), or DisplayPort, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE interfaces, etc. The technology described herein may be implemented in various electronic devices, such as mobile phones, laptop computers, smartphones, or other devices capable of performing digital data processing and / or video display.

[1019] Figure 8 is a block diagram illustrating an example video coding system 100 that may utilize the techniques of this disclosure.

[1020] like Figure 8 As shown, the video encoding and decoding system 100 may include a source device 110 and a target device 120. The source device 110 generates encoded video data and may be referred to as a video encoding device. The target device 120 may decode the encoded video data generated by the source device 110 and may be referred to as a video decoding device.

[1021] Source device 110 may include a video source 112 , a video encoder 114 , and an input / output (I / O) interface 116 .

[1022] The video source 112 may include a source such as a video capture device, an interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of these sources. The video data may include one or more pictures. The video encoder 114 encodes the video data from the video source 112 to generate a bitstream. The bitstream may include a sequence of bits that form a codec representation of the video data. The bitstream may include codec pictures and associated data. The codec picture is a codec representation of the picture. The associated data may include sequence parameter sets, picture parameter sets, and other syntax structures. The I / O interface 116 may include a modulator / demodulator (modem) and / or a transmitter. The encoded video data may be directly transmitted to the target device 120 via the network 130a via the I / O interface 116. The encoded video data may also be stored on a storage medium / server 130b for access by the target device 120.

[1023] Target device 120 may include an I / O interface 126 , a video decoder 124 , and a display device 122 .

[1024] The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may obtain encoded video data from the source device 110 or the storage medium / server 130b. The video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to a user. The display device 122 may be integrated with the target device 120, or may be external to the target device 120, the target device 120 being configured to interact with an external display device.

[1025] The video encoder 114 and the video decoder 124 may operate according to a video compression standard, such as the High Efficiency Video Coding (HEVC) standard, the Versatile Video Coding (VVC) standard, and other current and / or future standards.

[1026] Figure 9 is a block diagram illustrating an example of a video encoder 200, which may be Figure 8 The video encoder 114 in the system 100 is shown in FIG.

[1027] Video encoder 200 may be configured to perform any or all of the techniques of this disclosure. Figure 9 In the example of , video encoder 200 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of video encoder 200. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[1028] The functional components of the video encoder 200 may include a segmentation unit 201, a prediction unit 202 (which may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205 and an intra-frame prediction unit 206), a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213 and an entropy coding unit 214.

[1029] In other examples, the video encoder 200 may include more, fewer, or different functional components. In an example, the prediction unit 202 may include an intra block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode where at least one reference picture is a picture in which the current video block is located.

[1030] Furthermore, some components such as the motion estimation unit 204 and the motion compensation unit 205 may be highly integrated, but for ease of illustration, they are shown in FIG. Figure 5 They are shown in the examples.

[1031] The segmentation unit 201 may segment a picture into one or more video blocks. The video encoder 200 and the video decoder 300 may support various video block sizes.

[1032] The mode selection unit 203 can, for example, select one of the intra or inter coding modes based on the error result, and provide the resulting intra or inter coding block to the residual generation unit 207 to generate residual block data, and to the reconstruction unit 212 to reconstruct the coding block for use as a reference picture. In some examples, the mode selection unit 203 can select a combination of intra and inter prediction (CIIP) modes, where the prediction is based on an inter prediction signal and an intra prediction signal. In the case of inter prediction, the mode selection unit 203 can also select the resolution of the motion vector for the block (e.g., sub-pixel or integer pixel precision).

[1033] To perform inter-frame prediction on the current video block, the motion estimation unit 204 may generate motion information for the current video block by comparing the current video block with one or more reference frames from the buffer 213. The motion compensation unit 205 may determine a predicted video block for the current video block based on the motion information and decoded samples of pictures from the buffer 213 other than the picture associated with the current video block.

[1034] Motion estimation unit 204 and motion compensation unit 205 may perform different operations on the current video block depending on, for example, whether the current video block is in an I slice, a P slice, or a B slice.

[1035] In some examples, motion estimation unit 204 may perform unidirectional prediction on the current video block, and motion estimation unit 204 may search for a reference video block for the current video block in the reference pictures in list 0 or list 1. Motion estimation unit 204 may then generate a reference index indicating a reference picture in list 0 or list 1, which contains the reference video block and a motion vector indicating a spatial displacement between the current video block and the reference video block. Motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as motion information for the current video block. Motion compensation unit 205 may generate a predicted video block for the current block based on the reference video block indicated by the motion information for the current video block.

[1036] In other examples, the motion estimation unit 204 may perform bidirectional prediction on the current video block. The motion estimation unit 204 may search for a reference video block for the current video block in the reference pictures in list 0 and may also search for another reference video block for the current video block in the reference pictures in list 1. The motion estimation unit 204 may then generate reference indexes indicating reference pictures in list 0 and list 1, each of which contains reference video blocks and motion vectors indicating spatial displacements between the reference video blocks and the current video block. The motion estimation unit 204 may output the reference index and motion vector for the current video block as motion information for the current video block. The motion compensation unit 205 may generate a predicted video block for the current video block based on the reference video block indicated by the motion information for the current video block.

[1037] In some examples, motion estimation unit 204 may output a complete set of motion information for use in the decoding process of a decoder.

[1038] In some examples, motion estimation unit 204 may not output a complete set of motion information for the current video. Instead, motion estimation unit 204 may signal the motion information of the current video block relative to the motion information of another video block. For example, motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of a neighboring video block.

[1039] In one example, motion estimation unit 204 may indicate a value in a syntax structure associated with the current video block that indicates to video decoder 300 that the current video block has the same motion information as another video block.

[1040] In another example, the motion estimation unit 204 can identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. The video decoder 300 can use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[1041] As discussed above, video encoder 200 may predictively signal motion vectors. Two examples of predictive signaling techniques that may be implemented by video encoder 200 include Advanced Motion Vector Prediction (AMVP) and merge mode signaling.

[1042] The intra-frame prediction unit 206 may perform intra-frame prediction on the current video block. When the intra-frame prediction unit 206 performs intra-frame prediction on the current video block, the intra-frame prediction unit 206 may generate prediction data for the current video block based on decoded samples of other video blocks in the same picture. The prediction data for the current video block may include a predicted video block and various syntax elements.

[1043] The residual generation unit 207 can generate residual data for the current video block by subtracting (e.g., as indicated by a minus sign) the predicted video block(s) of the current video block from the current video block. The residual data for the current video block may include residual video blocks corresponding to different sample components of the samples in the current video block.

[1044] In other examples, for the current video block, for example, in skip mode, there may be no residual data for the current video block, and the residual generation unit 207 may not perform the subtraction operation.

[1045] Transform processing unit 208 may generate one or more transform coefficient video blocks for a current video block by applying one or more transforms to a residual video block associated with the current video block.

[1046] After transform processing unit 208 generates a transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[1047] The inverse quantization unit 210 and the inverse transform unit 211 may apply inverse quantization and inverse transform, respectively, to the transform coefficient video block to reconstruct a residual video block from the transform coefficient video block. The reconstruction unit 212 may add the reconstructed residual video block to corresponding samples of one or more prediction video blocks generated by the prediction unit 202 to generate a reconstructed video block associated with the current block for storage in the buffer 213.

[1048] After reconstruction unit 212 reconstructs the video block, a loop filtering operation may be performed to reduce video block artifacts in the video block.

[1049] The entropy coding unit 214 may receive data from other functional components of the video encoder 200. When the entropy coding unit 214 receives data, the entropy coding unit 214 may perform one or more entropy coding operations to generate entropy-coded data and output a bitstream including the entropy-coded data.

[1050] Figure 10 is a block diagram illustrating an example of a video decoder 300, which may be Figure 8 The video decoder 114 in the system 100 is shown in FIG.

[1051] Video decoder 300 may be configured to perform any or all of the techniques of this disclosure. Figure 10 In the example of FIG, video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of video decoder 300. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[1052] exist Figure 10 In the example of FIG, the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra-frame prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. In some examples, the video decoder 300 can perform operations relative to those performed with respect to the video encoder 200 (e.g., Figure 9 ) The encoding process described is roughly the opposite of the decoding process.

[1053] The entropy decoding unit 301 can obtain an encoded bitstream. The encoded bitstream may include entropy-encoded video data (e.g., coded blocks of video data). The entropy decoding unit 301 can decode the entropy-encoded video data, and the motion compensation unit 302 can determine motion information including motion vectors, motion vector precision, reference picture list index, and other motion information from the entropy-decoded video data. For example, the motion compensation unit 302 can determine such information by performing AMVP and merge mode.

[1054] The motion compensation unit 302 may generate a motion compensated block, possibly performing interpolation based on an interpolation filter. An identifier of the interpolation filter used with sub-pixel precision may be included in a syntax element.

[1055] Motion compensation unit 302 may use the interpolation filters used by video encoder 20 during encoding of a video block to calculate interpolated values ​​for sub-integer pixels of a reference block. Motion compensation unit 302 may determine the interpolation filters used by video encoder 200 based on received syntax information and use the interpolation filters to generate a prediction block.

[1056] The motion compensation unit 302 may use some syntax information to determine the size of blocks used to encode frame(s) and / or slice(s) of the coded video sequence, partitioning information describing how each macroblock of a picture of the coded video sequence is partitioned, a mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-coded block, and other information used to decode the coded video sequence.

[1057] The intra prediction unit 303 can use, for example, an intra prediction mode received in the bitstream to form a prediction block from spatially neighboring blocks. The inverse quantization unit 303 inversely quantizes, i.e., dequantizes, the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 303 applies an inverse transform.

[1058] The reconstruction unit 306 may add the residual block to the corresponding prediction block generated by the motion compensation unit 202 or the intra prediction unit 303 to form a decoded block. If necessary, a deblocking filter may be applied to the decoded block to remove blocking artifacts. The decoded video block is then stored in the buffer 307, which provides a reference block for subsequent motion compensation.

[1059] Figure 11 1 is a flowchart representation of a video processing method according to the present technology. Method 1100 includes, at operation 1110, performing a conversion between a current picture of a video and a codec representation of the video. The current picture includes a first window having dimensions of W×H samples, where W and H are integers. A reference picture associated with the current picture includes a second window having dimensions of W'×H' samples, where W' and H' are integers. Samples in the first window and the second window are subject to the same processing rules during the conversion, and W, H, W', and H' satisfy constraints.

[1060] In some embodiments, the first window and the second window are scaling windows. The same processing rules stipulate that samples in the scaling window are subjected to resampling operations. In some embodiments, the first window and the second window are consistency windows. The same processing rules stipulate that samples outside the consistency window are discarded when outputting the video.

[1061] In some embodiments, the constraint states W' / a ≤ W ≤ W' × b, where a and b are positive integers. In some embodiments, a is equal to 2. In some embodiments, b is equal to 8. In some embodiments, the constraint states H' / c ≤ H ≤ H' × d, where c and d are positive integers. In some embodiments, c is equal to 2. In some embodiments, d is equal to 8. In some embodiments, a = c. In some embodiments, b = d.

[1062] In some embodiments, the dimension of the first scaling window or the dimension of the second scaling window is signaled in the codec representation, and the dimension of the first scaling window or the dimension of the second scaling window is used to determine the set of scaling ratios.

[1063] In some embodiments, the dimensions of the first scaling window or the dimensions of the second scaling window are signaled in the codec representation, and the dimensions of the first scaling window or the dimensions of the second scaling window are used to determine an upper left offset for reference samples in applying reference picture resampling (RPR) during conversion.

[1064] Figure 12 1 is a flowchart representation of a video processing method according to the present technology. The method 1200 includes, at operation 1210, performing a conversion between a current picture of a video and a codec representation of the video. During the conversion, an offset value for determining a window of the current picture is signaled in the codec representation with N pixel precision, where N is a positive integer greater than 1.

[1065] In some embodiments, the window comprises a scaling window, and samples within the scaling window are subjected to resampling operations. In some embodiments, the window comprises a consistency window, and samples outside the consistency window are discarded when outputting the video. In some embodiments, in determining the consistency window for the current picture, the offset value signaled in the codec representation is multiplied by N. In some embodiments, N is equal to 4 or 8. In some embodiments, the offset value is used in one or more codec processes during conversion.

[1066] Figure 13 1 is a flowchart representation of a video processing method according to the present technology. The method 300 includes, at operation 1310, performing a conversion between a video including a first video unit and a second video unit and a codec representation of the video. A first set of offset values ​​for a first window of the first video unit and a second set of offset values ​​for a second window of the second video unit are determined according to rules associated with dimensions of the first video unit and dimensions of the second video unit. Samples in the first window and the second window are subject to the same processing rules during the conversion.

[1067] In some embodiments, the first window and the second window are scaling windows, and the same processing rules dictate that samples within the scaling windows undergo resampling operations. In some embodiments, the first window and the second window are consistency windows, and the same processing rules dictate that samples outside the consistency windows are discarded when outputting the video.

[1068] In some embodiments, the first video unit or the second video unit comprises a picture of a video. In some embodiments, the rule provides that if the dimensions of the first video unit are the same as the dimensions of the second video unit, the first set of offset values ​​is different from the second set of offset values. In some embodiments, the rule provides that if the dimensions of the first video unit are different from the dimensions of the second video unit, the first set of offset values ​​is the same as the second set of offset values. In some embodiments, the rule provides that the first set of offset values ​​is the same as the second set of offset values ​​regardless of whether the dimensions of the first video unit are the same as the dimensions of the second video unit.

[1069] In some embodiments, the first consistency window has dimensions W1×H1, the upper left position of the first consistency window is represented as (X1, Y1), the second consistency window has dimensions W2×H2, the upper left position of the second consistency window is represented as (X2, Y2), the dimensions of the first video unit are represented as PW1×PH1, and the dimensions of the second video unit are represented as PW2×PH2. In some embodiments, W1 / W2 = X1 / X2. In some embodiments, W1 / X1 = W2 / X2. W1×X2 = W2×X1. In some embodiments, H1 / H2 = Y1 / Y2. In some embodiments, H1 / Y1 = H2 / Y2. In some embodiments, H1×Y2 = H2×Y1. In some embodiments, PW1 / PW2 = X1 / X2. In some embodiments, PW1 / X1 = PW2 / X2. In some embodiments, PW1×X2 = PW2×X1. In some embodiments, PH1 / PH2 = Y1 / Y2. In some embodiments, PH1 / Y1 = PH2 / Y2. In some embodiments, PH1×Y2 = PH2×Y1. In some embodiments, PW1 / PW2 = W1 / W2. In some embodiments, PW1 / W1 = PW2 / W2. In some embodiments, PW1×W2 = PW2×W1. In some embodiments, PH1 / PH2 = H1 / H2. In some embodiments, PH1 / H1 = PH2 / H2. In some embodiments, PH1×H2 = PH2×H1. In some embodiments, when PW1 > PW2, W1 > W2. In some embodiments, when PW1 < PW2, W1 < W2. In some embodiments, (PW1 - PW2)×(W1 - W2)≥0. In some embodiments, when PH1 > PH2, H1 > H2. In some embodiments, when PH1 < PH2, H1 < H2. In some embodiments, (PH1 - PH2)×(H1 - H2)≥0. In some embodiments, when PW1≥PW2, (W1 / W2)≤(PW1 / PW2). In some embodiments, when PW1≥PW2, (W1 / W2)≥(PW1 / PW2). In some embodiments, when PH1≥PW2, (H1 / H2)≤(PH1 / PH2). In some embodiments, when PH1≥PW2, (H1 / H2)≥(PH1 / PH2).

[1070] Figure 1414 is a flowchart representing a video processing method according to the present technology. Method 1400 includes, at operation 1410, determining, for a conversion between a current picture of a video and a codec representation of the video, a position of a reference sample of a reference picture associated with the current picture based on a top-left position of a window of pictures, the pictures including at least the current picture or the reference picture. The window is subjected to processing rules during the conversion. Method 1400 also includes, at operation 1420, performing the conversion based on the determination.

[1071] In some embodiments, the window comprises a scaling window, and the processing rules specify that samples within the scaling window are subjected to resampling operations. In some embodiments, the window comprises a consistency window, and the processing rules specify that samples outside the consistency window are discarded when outputting the video.

[1072] In some embodiments, the upper left position of the window is determined based on a left offset value and an upper offset value associated with the window. In some embodiments, the horizontal position xSb' of the current sample relative to the upper left position of the window is determined based on the horizontal position xSb of the current sample in the current picture and the left offset of the window of the current picture. In some embodiments, the horizontal position xSb' of the current sample relative to the upper left position of the window is further determined based on the sample precision.

[1073] In some embodiments, the horizontal position xSb' of the current sample relative to the top left position of the window is used to determine the position of the reference sample. In some embodiments, the horizontal position of the reference sample relative to the reference picture is determined based on xSb' and the left offset of the window of the reference picture. In some embodiments, the sample precision indicates a four-bit displacement.

[1074] In some embodiments, the vertical position ySb' of the current sample relative to the upper left position of the window is determined based on the vertical position ySb of the current sample in the current picture and the upper offset of the window of the current picture. In some embodiments, the vertical position ySb' of the current sample relative to the upper left position of the window is also determined based on the sample precision. In some embodiments, the vertical position ySb' of the current sample relative to the upper left position of the window is used to determine the position of the reference sample. In some embodiments, the vertical position of the reference sample relative to the reference picture is determined based on ySb' and the upper offset of the window of the reference picture. In some embodiments, the sample precision indicates a four-bit displacement.

[1075] In some embodiments, the use of the upper left position of the window is based on the color components of the video.In some embodiments, the color components include a luminance component.

[1076] Figure 15is a flowchart representation of a video processing method according to the present technology. Method 1500 includes: at operation 1510, performing a conversion between a current picture of a video and a codec representation of the video. The current picture includes a first window having dimensions of W×H samples, where W and H are integers. A reference picture associated with the current picture includes a second window having dimensions of W'×H' samples, where W' and H' are integers, and the upper left position of the second window is represented as (X0, Y0). The samples in the first window and the second window are subject to the same processing rules during the conversion. The integer portion of the coordinates of the reference sample is constrained to be within a range determined in part based on the first window or the second window.

[1077] In some embodiments, the integer portion of the horizontal coordinate of the reference sample is constrained to be within the range [minW, maxW]. In some embodiments, minW is equal to 0. In some embodiments, minW is equal to X0. In some embodiments, maxW is equal to W-1. In some embodiments, maxW is equal to W'-1. In some embodiments, maxW is equal to X0+W'-1. In some embodiments, at least one of minW or maxW is modified based on the color components and / or color format of the video. In some embodiments, minW is modified to minW×subC, where subC is an integer. In some embodiments, minW is modified to minW / subC, where subC is an integer. In some embodiments, maxW is modified to maxW×subC, where subC is an integer. In some embodiments, maxW is modified to maxW / subC, where subC is an integer. In some embodiments, when the color components include a luma component, subC is 1. In some embodiments, when the color components include a chroma component and the color format is 4:2:0 or 4:2:2, subC is 2. In some embodiments, when the color components include a chroma component and the color format is 4:4:4, subC is 1.

[1078] In some embodiments, the integer portion of the vertical coordinate of the reference sample is constrained to be within the range [minH, maxH]. In some embodiments, minH is equal to 0. In some embodiments, minH is equal to Y0. In some embodiments, maxH is equal to H-1. In some embodiments, maxH is equal to H'-1. In some embodiments, maxH is equal to Y0+H'-1. In some embodiments, at least one of minH or maxH is modified based on the color components and / or color format of the video. In some embodiments, minH is modified to minH×subC, where subC is an integer. In some embodiments, minH is modified to minH / subC, where subC is an integer. In some embodiments, maxH is modified to maxH×subC, where subC is an integer. In some embodiments, maxH is modified to maxH / subC, where subC is an integer. In some embodiments, subC is 1 when the color components include a luma component. In some embodiments, subC is 2 when the color components include chroma components and the color format is 4:2:0 or 4:2:2. In some embodiments, subC is 1 when the color components include chroma components and the color format is 4:4:4.

[1079] In some embodiments, the applicability of the constraint on the integer part of the coordinates of the reference sample is determined based on the dimensions of the current picture and the dimensions of the reference picture. In some embodiments, the constraint on the integer part of the coordinates of the reference sample is based on the dimensions of the first window and the dimensions of the second window. In some embodiments, the constraint applies to the following situations: (1) the dimensions of the current picture are not the same as the dimensions of the reference picture, or (2) the dimensions of the first window are not the same as the dimensions of the second window. In some embodiments, the first window and the second window are scaling windows, and wherein the same processing rules stipulate that the samples in the scaling windows are subjected to resampling operations. In some embodiments, the first window and the second window are consistency windows, and wherein the same processing rules stipulate that samples outside the consistency window are discarded when outputting the video.

[1080] In some embodiments, the conversion comprises encoding the video into a codec representation. In some embodiments, the conversion comprises decoding the codec representation into the video.

[1081] Some embodiments of the disclosed technology include making a decision or determination to enable a video processing tool or mode. In one example, when a video processing tool or mode is enabled, the encoder will use or implement the tool or mode in the processing of the video block, but will not necessarily modify the resulting bitstream based on the use of the tool or mode. That is, when the video processing tool or mode is enabled based on the decision or determination, the conversion from the video block to the bitstream representation of the video will use the video processing tool or mode. In another example, when the video processing tool or mode is enabled, the decoder will process the bitstream knowing that the bitstream has been modified based on the video processing tool or mode. That is, the conversion from the bitstream representation of the video to the video block will be performed using the video processing tool or mode enabled based on the decision or determination.

[1082] Some embodiments of the disclosed technology include making a decision or determination to disable a video processing tool or mode. In one example, when a video processing tool or mode is disabled, an encoder will not use the tool or mode in converting video blocks into a bitstream representation of the video. In another example, when a video processing tool or mode is disabled, a decoder will process the bitstream knowing that no modifications have been made to the bitstream using the video processing tool or mode that was disabled based on the decision or determination.

[1083] The disclosed and other solutions, examples, embodiments, modules, and functional operations described in this document can be implemented in digital electronic circuits or computer software, firmware, or hardware, including the structures disclosed in this document and their structural equivalents, or a combination of one or more thereof. The disclosed embodiments and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium for execution by a data processing apparatus or to control the operation of the data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a storage device, a composition of matter that affects a machine-readable propagated signal, or a combination of one or more thereof. The term "data processing apparatus" encompasses all apparatuses, devices, and machines for processing data, including (for example) a programmable processor, a computer, or multiple processors or computers. In addition to hardware, the apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more thereof. A propagated signal is an artificially generated signal, such as a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to a suitable receiver device.

[1084] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language (including compiled or interpreted languages) and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). A computer program may be deployed to execute on one or more computers that are located at a single site or distributed across multiple sites and interconnected by a communications network.

[1085] The processes and logic flows described in this document can be performed by executing one or more computer programs by one or more programmable processors to perform functions by operating on input data and generating output. These processes and logic flows can also be performed by special purpose logic circuitry, and the apparatus can also be implemented as special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

[1086] For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, as well as any one or more processors of any type of digital computer. Generally speaking, a processor will receive instructions and data from a read-only memory or a random access memory, or both. The essential elements of a computer are a processor that executes instructions and one or more memory devices that store instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic, magneto-optical, or optical disks, or be operatively coupled to receive data from or transfer data to one or more mass storage devices, or both. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD ROM disks. The processor and memory may be supplemented by, or incorporated into, special-purpose logic circuitry.

[1087] Although this patent document contains many details, they should not be interpreted as limitations on any subject matter or the scope of the claims, but rather as descriptions of specific features of specific embodiments of particular technologies. Certain features described in this patent document in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments, either individually or in any suitable subcombination. Furthermore, although certain features may be described above as functioning in certain combinations and even initially claimed as such, one or more features from a claimed combination may in some cases be removed from the combination, and a claimed combination may involve subcombinations or variations of subcombinations.

[1088] Similarly, while operations may be depicted in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, in order to achieve desired results. Furthermore, the separation of various system components among the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[1089] Only a few implementations and examples are described, and other implementations, enhancements, and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A video processing method, comprising: For a conversion between a current picture of a video and a bitstream of the video, determining a size of a current window of the current picture based on a first set of offset values, For a reference picture associated with the current picture, determining a size of a reference window of the reference picture based on a second set of offset values, The conversion is performed based on the size of the current window and the size of the reference window, wherein indications of the first set of offset values ​​and the second set of offset values ​​are included in the bitstream, Wherein, the size of the current window and the size of the reference window satisfy the constraints, and The width of the current window is W, the width of the reference window is W', the constraint stipulates that W' / a≤W≤W'×b, and a and b are positive integers.

2. The method according to claim 1, wherein: The current window and the reference window are scaling windows used to derive a scaling ratio in inter-frame prediction.

3. The method according to claim 1, wherein: The height of the current window is H, the height of the reference window is H', and The constraint stipulates that H' / a≤H≤H'×b.

4. The method according to claim 3, wherein: a equals 2, b equals 8.

5. The method according to claim 1, wherein: Indications of the first set of offset values ​​and the second set of offset values ​​are included in a picture parameter set (PPS) of the bitstream.

6. The method according to claim 1, wherein: The top offset and the left offset of the first set of offset values ​​and the position of the top left sample of the current block in the current picture are used to determine the position of the reference sample associated with the first sample in the current block.

7. The method according to claim 2, wherein: Different groups of offset values ​​are used to indicate the consistency window and the scaling window, respectively, and When the video is output, the samples outside the consistency window can be processed arbitrarily.

8. The method according to claim 1, wherein: The converting includes encoding the current picture into the bitstream.

9. The method according to claim 1, wherein: The converting includes decoding the current picture from the bitstream.

10. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to: For a conversion between a current picture of a video and a bitstream of the video, determining a size of a current window of the current picture based on a first set of offset values, For a reference picture associated with the current picture, determining a size of a reference window of the reference picture based on a second set of offset values, The conversion is performed based on the size of the current window and the size of the reference window, in, an indication of the first set of offset values ​​and the second set of offset values ​​being included in the bitstream, Wherein, the size of the current window and the size of the reference window satisfy the constraints, and The width of the current window is W, the width of the reference window is W', the constraint stipulates that W' / a≤W≤W'×b, and a and b are positive integers.

11. The device according to claim 10, wherein: The current window and the reference window are scaling windows used to derive a scaling ratio in inter-frame prediction.

12. The device according to claim 10, wherein: The height of the current window is H, the height of the reference window is H', and The constraint stipulates that H' / a≤H≤H'×b.

13. The device according to claim 12, wherein: a equals 2, b equals 8.

14. The device according to claim 10, wherein: Indications of the first set of offset values ​​and the second set of offset values ​​are included in a picture parameter set (PPS) of the bitstream.

15. The device according to claim 10, wherein: The top offset and the left offset of the first set of offset values ​​and the position of the top left sample of the current block in the current picture are used to determine the position of the reference sample associated with the first sample in the current block.

16. The device according to claim 11, wherein Different groups of offset values ​​are used to indicate the consistency window and the scaling window, respectively, and When the video is output, the samples outside the consistency window can be processed arbitrarily.

17. A non-transitory computer-readable storage medium storing instructions that cause a processor to: For a conversion between a current picture of a video and a bitstream of the video, determining a size of a current window of the current picture based on a first set of offset values, For a reference picture associated with the current picture, determining a size of a reference window of the reference picture based on a second set of offset values, The conversion is performed based on the size of the current window and the size of the reference window, in, an indication of the first set of offset values ​​and the second set of offset values ​​being included in the bitstream, Wherein, the size of the current window and the size of the reference window satisfy the constraints, and The width of the current window is W, the width of the reference window is W', the constraint stipulates that W' / a≤W≤W'×b, and a and b are positive integers.

18. A method for storing a bitstream of a video, comprising: For a current picture of the video, determining a size of a current window of the current picture based on a first set of offset values, For a reference picture associated with the current picture, determining a size of a reference window of the reference picture based on a second set of offset values, generating the bitstream based on the size of the current window and the size of the reference window, storing the bitstream in a non-transitory computer-readable recording medium, wherein indications of the first set of offset values ​​and the second set of offset values ​​are included in the bitstream, Wherein, the size of the current window and the size of the reference window satisfy the constraints, and The width of the current window is W, the width of the reference window is W', the constraint stipulates that W' / a≤W≤W'×b, and a and b are positive integers.

19. A computer readable medium having stored thereon codes which, when executed by a processor, cause the processor to implement the method according to any one of claims 2 to 9.

Images