Subpixel motion vector correction for chroma based on scaling ratio and chroma sampling type

Abstract

The device is configured to modify the horizontal component of the chroma position, which is pointed to by the motion vector of the current block of the current picture, based on chroma phase shifts in the horizontal direction. The chroma position is located in a reference picture having a different resolution or a different chroma sampling type from the current picture. The device may also modify the vertical component of the chroma position based on chroma phase shifts in the vertical direction. Additionally, the device may generate chroma samples of the prediction block for the current block of the current picture based on the chroma samples of the reference picture corresponding to the modified chroma position. The modified chroma position is defined by the modified horizontal and vertical components of the chroma position. The device may also code the current block based on the prediction block for the current block.

Description

Technology Field

[0001] This application claims priority to U.S. Patent Application No. 17 / 017,278 filed on September 10, 2020, which claims the benefit of U.S. Provisional Patent Application No. 62 / 902,271 filed on September 18, 2019, the entire contents of each of these are incorporated by reference.

[0002] Technology field

[0003] The present disclosure relates to video encoding and video decoding. Background Technology

[0004] Digital video capabilities can be integrated into a wide range of devices, including digital televisions, digital direct broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video gaming devices, video game consoles, cellular or satellite wireless phones, so-called "smartphones," video teleconferencing devices, video streaming devices, etc. Digital video devices implement video coding technologies as described in standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4, Part 10, AVC (Advanced Video Coding), ITU-T H.265, HEVC (High Efficiency Video Coding), and extensions of these standards. By implementing such video coding technologies, video devices may more efficiently transmit, receive, encode, decode, and / or store digital video information.

[0005] Video coding techniques include spatial (intra-picture) prediction and / or temporal (inter-picture) prediction to reduce or eliminate redundancy inherent in video sequences. For block-based video coding, a video slice (e.g., a video picture or a portion of a video picture) may be partitioned into video blocks, which may also be referred to as coding tree units (CTUs), coding units (CUs), and / or coding nodes. Video blocks in an intra-coded (I) slice of a picture are encoded using spatial prediction for reference samples in neighboring blocks of the same picture. Video blocks in an inter-coded (P or B) slice of a picture may use spatial prediction for reference samples in neighboring blocks within the same picture or temporal prediction for reference samples in other reference pictures. Pictures may be referred to as frames, and reference pictures may be referred to as reference frames. means of solving the problem

[0006] Generally, the present disclosure describes techniques for determining motion vectors to be used in conjunction with reference picture resampling. As described herein, a video coder, e.g., a video encoder or a video decoder, may perform inter-prediction to generate a prediction block for a current block of a current picture. The video coder may use samples of a reference picture to generate the prediction block. However, the reference picture may have a different resolution or chroma sampling type than the current picture. Due to differences in resolutions or chroma phase shifts, the video coder may adjust the motion vector of the current block (or the position indicated by the motion vector) to indicate the exact position in the reference picture.

[0007] In one example, the present disclosure describes a method for coding video data, the method comprising: modifying a horizontal component of a chroma position pointed to by a motion vector of a current block of a current picture of video data based on chroma phase shifts in a horizontal direction, said chroma position being in a reference picture having a different resolution or a different chroma sampling type from the current picture; modifying a vertical component of a chroma position based on chroma phase shifts in a vertical direction; generating chroma samples of a prediction block for a current block based on chroma samples of a reference picture corresponding to the modified chroma position, said chroma position being defined by the modified horizontal and vertical components of the chroma position; and coding a current block based on a prediction block for a current block.

[0008] In another example, the present disclosure describes a device for coding video data, the device comprising: a memory configured to store video data; and one or more processors implemented as circuits, wherein the one or more processors modify a horizontal component of a chroma position pointed to by a motion vector of a current block of a current picture of video data based on chroma phase shifts in the horizontal direction, said chroma position being in a reference picture and said chroma position being in a reference picture having a different resolution or a different chroma sampling type from the current picture; modify a vertical component of a chroma position based on chroma phase shifts in the vertical direction; generate chroma samples of a prediction block for a current block based on chroma samples of a reference picture corresponding to the modified chroma position, said modified chroma position being defined by the modified horizontal and vertical components of the chroma position; and are configured to code a current block based on a prediction block for a current block.

[0009] In another example, the present disclosure describes a device for coding video data, the device comprising: means for modifying a horizontal component of a chroma position pointed to by a motion vector of a current block of a current picture of video data based on chroma phase shifts in a horizontal direction, said chroma position being in a reference picture having a different resolution or a different chroma sampling type from the current picture; means for modifying a vertical component of a chroma position based on chroma phase shifts in a vertical direction; means for generating chroma samples of a prediction block for a current block based on chroma samples of a reference picture corresponding to the modified chroma position, said modified chroma position being defined by the modified horizontal and vertical components of the chroma position; and means for coding a current block based on a prediction block for a current block.

[0010] In another example, the present disclosure describes a computer-readable storage medium storing instructions, the instructions which, when executed, cause one or more processors to modify a horizontal component of a chroma position pointed to by a motion vector of a current block of a current picture of video data based on chroma phase shifts in the horizontal direction, said chroma position being in a reference picture having a different resolution or a different chroma sampling type from the current picture, said chroma position being in a reference picture having a different resolution or a different chroma sampling type from the current picture; said chroma position being in a vertical component of a chroma position based on chroma phase shifts in the vertical direction; said chroma samples of a prediction block for a current block based on chroma samples of a reference picture corresponding to the modified chroma position being in a vertical direction being in a chroma position of a current block being in a chroma sample of a prediction block being in a chroma position defined by the modified horizontal and vertical components of the chroma position; and said current block being coded based on a prediction block for a current block.

[0011] Details of one or more examples are described in the accompanying drawings and the following description. Other features, purposes, and advantages will become apparent from the description, drawings, and claims. Brief explanation of the drawing

[0012] FIG. 1 is a block diagram illustrating an exemplary video encoding and decoding system capable of performing the techniques of the present disclosure. Figure 2 is a conceptual diagram illustrating exemplary chroma sampling types. FIG. 3 is a block diagram illustrating an exemplary video encoder capable of performing the techniques of the present disclosure. FIG. 4 is a block diagram illustrating an exemplary video decoder capable of performing the techniques of the present disclosure. FIG. 5 is a flowchart illustrating an exemplary method for encoding a current block according to one or more techniques of the present disclosure. FIG. 6 is a flowchart illustrating an exemplary method for decoding a current block of video data according to one or more techniques of the present disclosure. FIG. 7 is a flowchart illustrating an exemplary method of coding video data according to one or more techniques of the present disclosure. Specific details for implementing the invention

[0013] A video coder, for example, a video encoder or a video decoder, may use inter-prediction to generate a prediction block for the current block of the current picture. The video coder may also use the motion vector of the current block to determine its position in the reference picture. The video coder may then use samples of the reference picture at the position indicated by the motion vector to generate the prediction block. For example, if the motion vector indicates a sub-pixel position (i.e., a position between two samples), the video coder may interpolate samples of the prediction block based on samples of the reference picture. In cases where the video coder is a video encoder, the video encoder may use the prediction block to generate residual data for the current block. The residual data may then be encoded and signaled. In cases where the video coder is a video decoder, the video decoder may use the prediction block and residual data for the current block to reconstruct the current block.

[0014] In some examples, the current picture and the reference picture have different resolutions or different chroma sampling types. For example, the current picture and the reference picture may be coded using different chroma formats. For example, in one example, the current picture may be coded in a 4:2:2 chroma format, and the reference picture may be coded in a 4:2:0 chroma format. Thus, in this example, the current picture has twice as many chroma samples in the vertical direction as the reference picture. These differences in chroma phase shifts may cause motion vectors to display inaccurate positions within the reference pictures.

[0015] Additionally, in pictures coded in the 4:2:0 chroma format, different chroma sampling types exist. In the 4:2:0 chroma format, chroma samples are down-sampled 2:1 relative to lumina samples in both horizontal and vertical directions. Different chroma sampling types correspond to different positions of chroma samples relative to lumina samples. These differences in chroma sample types between the current picture and the reference picture are not taken into account and may reduce coding efficiency or picture quality. Therefore, the current picture and the reference picture may have different resolutions or chroma sampling types.

[0016] The present disclosure describes techniques that may solve these problems. For example, in one example, a video coder (e.g., a video encoder or a video decoder) may modify the horizontal component of a chroma position pointed to by the motion vector of the current block of the current picture based on chroma phase shifts in the horizontal direction. The chroma position is in a reference picture having a different resolution or a different chroma sampling type from the current picture. Additionally, the video coder may modify the vertical component of the motion vector based on chroma phase shifts in the vertical direction. The video coder may generate chroma samples of a predicted block for the current block based on chroma samples of the reference picture corresponding to the modified chroma position. The modified chroma position is defined by the modified horizontal and vertical components of the chroma position. The video coder may code (e.g., encode or decode) the current block based on the predicted block for the current block. By modifying the horizontal and vertical components of the chroma position in this way, the video coder may be able to avoid issues that may arise when the current picture and the reference picture have different resolutions or chroma sampling types.

[0017] FIG. 1 is a block diagram illustrating an exemplary video encoding and decoding system (100) capable of performing the techniques of the present disclosure. The techniques of the present disclosure generally relate to coding (encoding and / or decoding) video data. Generally, video data includes any data for processing video. Accordingly, video data may include raw, unencoded video, encoded video, decoded (e.g., restored) video, and video metadata such as signaling data.

[0018] As illustrated in FIG. 1, the system (100) includes a source device (102) that provides encoded video data to be decoded and displayed by a destination device (116) in this example. In particular, the source device (102) provides video data to the destination device (116) via a computer-readable medium (110). The source device (102) and the destination device (116) may include any of a wide range of devices, including desktop computers, mobile devices (e.g., notebook computers, tablet computers, telephone handsets, e.g., smartphones, televisions, cameras, etc.), broadcast receiver devices, set-top boxes, display devices, digital media players, video gaming consoles, video streaming devices, etc. In some cases, the source device (102) and the destination device (116) may be equipped for wireless communication and thus may be referred to as wireless communication devices.

[0019] In the example of FIG. 1, the source device (102) includes a video source (104), memory (106), a video encoder (200), and an output interface (108). The destination device (116) includes an input interface (122), a video decoder (300), memory (120), and a display device (118). According to the present disclosure, the video encoder (200) of the source device (102) and the video decoder (300) of the destination device (116) may be configured to apply techniques for determining motion vectors to be used for reference picture resampling. Thus, the source device (102) represents an example of a video encoding device, while the destination device (116) represents an example of a video decoding device. In other examples, the source device and the destination device may include other components or arrangements. For example, the source device (102) may receive video data from an external video source, such as an external camera. Likewise, the destination device (116) may interface with an external display device rather than including an integrated display device.

[0020] The system (100) as illustrated in FIG. 1 is merely one example. In general, any digital video encoding and / or decoding device may perform techniques for determining motion vectors for use with reference picture resampling. The source device (102) and the destination device (116) are merely examples of such coding devices in which the source device (102) generates coded video data for transmission to the destination device (116). The present disclosure refers to a "coding" device as a device that performs the coding (encoding and / or decoding) of data. Accordingly, the video encoder (200) and the video decoder (300) represent examples of coding devices, in particular a video encoder and a video decoder, respectively. In some examples, the source device (102) and the destination device (116) may operate substantially symmetrically such that each of the source device (102) and the destination device (116) includes video encoding and decoding components. Accordingly, the system (100) may support unidirectional or bidirectional video transmission between a source device (102) and a destination device (116) for, for example, video streaming, video playback, video broadcasting, or video calling.

[0021] Generally, a video source (104) represents a source of video data (e.g., raw, unencoded video data) and provides a sequential series of pictures of video data (also referred to as “frames”) to a video encoder (200) that encodes data for the pictures. The video source (104) of the source device (102) may include a video capture device such as a video camera, a video archive containing previously captured raw video, and / or a video feed interface for receiving video from a video content provider. As an additional alternative, the video source (104) may generate computer graphics-based data as source video, or as a combination of live video, archived video, and computer-generated video. In each case, the video encoder (200) encodes the captured, pre-captured, or computer-generated video data. The video encoder (200) may rearrange the pictures from the received order (sometimes referred to as “display order”) into a coding order for encoding. The video encoder (200) may generate a bitstream containing encoded video data. The source device (102) may output the encoded video data onto a computer-readable medium (110) through an output interface (108) for reception and / or extraction by, for example, the input interface (122) of the destination device (116).

[0022] The memory (106) of the source device (102) and the memory (120) of the destination device (116) represent general-purpose memories. In some examples, the memories (106, 120) may store raw video data, e.g., raw video from a video source (104) and raw, decoded video data from a video decoder (300). Additionally or alternatively, the memories (106, 120) may store software instructions executable by, e.g., a video encoder (200) and a video decoder (300), respectively. Although the memory (106) and memory (120) are shown separately from the video encoder (200) and video decoder (300) in this example, it should be understood that the video encoder (200) and video decoder (300) may also include internal memories for functionally similar or equivalent purposes. Furthermore, the memories (106, 120) may store encoded video data, for example, output from the video encoder (200) and input to the video decoder (300). In some examples, portions of the memories (106, 120) may be allocated as one or more video buffers to store, for example, raw, decoded, and / or encoded video data.

[0023] The computer-readable medium (110) may represent any type of medium or device capable of transmitting encoded video data from a source device (102) to a destination device (116). In one example, the computer-readable medium (110) represents a communication medium that enables the source device (102) to transmit encoded video data directly to the destination device (116) in real time, for example, through a radio frequency network or a computer-based network. According to a communication standard such as a wireless communication protocol, an output interface (108) may modulate a transmission signal containing the encoded video data, and an input interface (122) may demodulate the received transmission signal. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form a packet-based network, such as a local area network, a wide area network, or a global network, such as part of the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful for enabling communication from a source device (102) to a destination device (116).

[0024] In some examples, the computer-readable medium (110) may include a storage device (112). The source device (102) may output encoded data to the storage device (112) from the output interface (108). Similarly, the destination device (116) may access the encoded data from the storage device (112) through the input interface (122). The storage device (112) may include any of various distributed or local access data storage media, such as a hard drive, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage media for storing encoded video data.

[0025] In some examples, the computer-readable medium (110) may include a file server (114) or other intermediate storage device that may store encoded video data generated by the source device (102). The source (102) may output encoded video data to the file server (114) or other intermediate storage device that may store the encoded video generated by the source device (102). The destination device (116) may access the stored video data from the file server (114) via streaming or downloading. The file server (114) may be any type of server device capable of storing encoded video data and transmitting the encoded video data to the destination device (116). The file server (114) may represent a web server (e.g., for a website), a file transfer protocol (FTP) server, a content delivery network device, or a NAS (network attached storage) device. The destination device (116) may also access encoded video data from the file server (114) via any standard data connection, including an internet connection. This may include a wireless channel (e.g., Wi-Fi connection), a wired connection (e.g., Digital Subscriber Line (DSL), cable modem, etc.), or a combination of both, suitable for accessing encoded video data stored on the file server (114). The file server (114) and the input interface (122) may be configured to operate according to a streaming transmission protocol, a download transmission protocol, or a combination thereof.

[0026] The output interface (108) and input interface (122) may represent a wireless transmitter / receiver, a modem, a wired networking component (e.g., an Ethernet card), a wireless communication component operating according to any of the various IEEE 802.11 standards, or other physical components. In examples where the output interface (108) and input interface (122) include wireless components, the output interface (108) and input interface (122) may be configured to transmit data, such as encoded video data, according to cellular communication standards such as 4G, 4G-LTE (Long Term Evolution), LTE Advanced, 5G, etc. In some examples where the output interface (108) includes a wireless transmitter, the output interface (108) and the input interface (122) may be configured to transmit data, such as encoded video data, according to other wireless standards such as IEEE 802.11 specifications, IEEE 802.15 specifications (e.g., ZigBee™), Bluetooth™ standards, etc. In some examples, the source device (102) and / or the destination device (116) may include individual system-on-chip (SoC) devices. For example, the source device (102) may include an SoC device for performing functions attributed to the video encoder (200) and / or the output interface (108), and the destination device (116) may include an SoC device for performing functions attributed to the video decoder (300) and / or the input interface (122).

[0027] The techniques of the present disclosure may be applied to video coding by supporting any of various multimedia applications, such as over-the-air television broadcasts, cable television transmissions, satellite television transmissions, internet streaming video transmissions, e.g., DASH (dynamic adaptive streaming over HTTP), digital video encoded on a data storage medium, decoding of digital video stored on a data storage medium, or other applications.

[0028] The input interface (122) of the destination device (116) receives an encoded video bitstream from a computer-readable medium (110) (e.g., a communication medium, a storage device (112), a file server (114), etc.). The encoded video bitstream may include signaling information defined by a video encoder (200), which is also used by a video decoder (300), such as syntax elements having values ​​that describe the processing and / or characteristics of video blocks or other coded units (e.g., slices, pictures, groups of pictures, sequences, etc.). A display device (118) displays decoded pictures of the decoded video data to a user. The display device (118) may include any of the various display devices, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

[0029] Although not illustrated in FIG. 1, in some examples, the video encoder (200) and the video decoder (300) may each be integrated with an audio encoder and / or an audio decoder, and may include suitable MUX-DEMUX units or other hardware and / or software to handle multiplexed streams containing both audio and video in a common data stream. Where applicable, the MUX-DEMUX units may follow other protocols such as the ITU H.223 multiplexer protocol or the User Datagram Protocol (UDP).

[0030] Each of the video encoder (200) and the video decoder (300) may be implemented as any of various suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. Where the techniques are partially implemented in software, the device may store instructions for the software on a suitable non-transient computer-readable medium and execute those instructions in hardware using one or more processors to perform the techniques of the present disclosure. Each of the video encoder (200) and the video decoder (300) may be included in one or more encoders or decoders, any one of which may be integrated as part of a combined encoder / decoder (CODEC) in each device. A device including a video encoder (200) and / or a video decoder (300) may include an integrated circuit, a microprocessor, and / or a wireless communication device, such as a cellular phone.

[0031] The video encoder (200) and video decoder (300) may operate according to a video coding standard such as ITU-T H.265, also referred to as High Efficiency Video Coding (HEVC), or extensions thereof, such as multi-view and / or scalable video coding extensions. Alternatively, the video encoder (200) and video decoder (300) may operate according to other proprietary or industry standards such as ITU-T H.266, also referred to as Versatile Video Coding (VVC). A draft of the VVC standard is Bross et al., "Versatile Video Coding (Draft 6)," Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 15 th It is described in Meeting: Gothenburg, SE, 3-12 July 2019, JVET-O2001-vB ("VVC Draft 6"). Chen et al., "Algorithm Description for Versatile Video Coding and Test Model 6," Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 15 th Meeting: Gothenburg, SE, 3-12 July 2019, JVET-O2002 (hereinafter "VTM-6") describes a test model for VVC Draft 6. However, the techniques of the present disclosure are not limited to any specific coding standard.

[0032] Generally, the video encoder (200) and video decoder (300) may perform block-based coding of pictures. The term “block” generally refers to a structure containing data to be processed (e.g., to be encoded, to be decoded, or otherwise to be used in the encoding and / or decoding process). For example, a block may contain a two-dimensional matrix of samples of luminance and / or chrominance data. Generally, the video encoder (200) and video decoder (300) may code video data represented in a YUV (e.g., Y, Cb, Cr) format. That is, rather than coding red, green, and blue (RGB) data for samples of a picture, the video encoder (200) and video decoder (300) may code luminance and chrominance components, wherein the chrominance components may include both red and blue chrominance components. In some examples, the video encoder (200) converts the received RGB-formatted data into a YUV representation prior to encoding, and the video decoder (300) converts the YUV representation into an RGB format. Alternatively, pre- and post-processing units (not shown) may perform these conversions.

[0033] The present disclosure may generally refer to the coding of pictures (e.g., encoding and decoding), which includes a process of encoding or decoding data of the pictures. Similarly, the present disclosure may refer to the coding of blocks of pictures, which includes a process of encoding or decoding data for the blocks, such as, for example, prediction and / or residual coding. An encoded video bitstream generally includes a series of values ​​for syntax elements representing coding decisions (e.g., coding modes) and partitioning of the pictures into blocks. Accordingly, references to coding a picture or block should generally be understood as coding values ​​for the syntax elements forming the picture or block.

[0034] HEVC defines various blocks including coding units (CUs), prediction units (PUs), and transformation units (TUs). According to HEVC, a video coder (e.g., a video encoder (200)) partitions coding tree units (CTUs) into CUs according to a quadtree structure. That is, the video coder partitions the CTUs and CUs into four identical non-overlapping squares, and each node of the quadtree has either zero or four child nodes. Nodes without child nodes may be referred to as "leaf nodes," and the CUs of such leaf nodes may contain one or more PUs and / or one or more TUs. The video coder may further partition the PUs and TUs. For example, in HEVC, a residual quadtree (RQT) represents the partitioning of the TUs. In HEVC, PUs represent inter-prediction data, while TUs represent residual data. Intra-predicted CUs include intra-predicted information such as intra-mode indication.

[0035] As another example, the video encoder (200) and video decoder (300) may be configured to operate according to VVC. According to VVC, a video encoder (such as the video encoder (200)) partitions a picture into multiple coding tree units (CTUs). The video encoder (200) may partition the CTUs according to a tree structure such as a quadtree binary tree (QTBT) structure or a multi-type tree (MTT) structure. The QTBT structure eliminates the concepts of multiple partition types, such as the separation between CUs, PUs, and TUs in HEVC. The QTBT structure includes two levels: a first level partitioned according to quadtree partitioning, and a second level partitioned according to binary tree partitioning. The root node of the QTBT structure corresponds to a CTU. The leaf nodes of binary trees correspond to coding units (CUs).

[0036] In an MTT partitioning structure, blocks may be partitioned using quadtree (QT) partitions, binary tree (BT) partitions, and one or more types of tripletree (TT) (also referred to as ternary tree (TT)) partitions. A triple or ternary tree partition is a partition in which a block is split into three subblocks. In some examples, a triple or ternary tree partition divides the block into three subblocks without splitting the original block through the center. The partitioning types in MTT (e.g., QT, BT, and TT) may be symmetric or asymmetric.

[0037] In some examples, the video encoder (200) and the video decoder (300) may use a single QTBT or MTT structure to represent each of the luminance and chrominance components, whereas in other examples, the video encoder (200) and the video decoder (300) may use two or more QTBT or MTT structures, such as one QTBT / MTT structure for the luminance component and another QTBT / MTT structure for both of the chrominance components (or two QTBT / MTT structures for individual chrominance components).

[0038] The video encoder (200) and video decoder (300) may be configured to use quadtree partitioning, QTBT partitioning, MTT partitioning, or other partitioning structures per HEVC. For the purposes of explanation, the description of the techniques of the present disclosure is presented with respect to QTBT partitioning. However, it should be understood that the techniques of the present disclosure may also be applied to video coders configured to use quadtree partitioning or other types of partitioning.

[0039] Blocks (e.g., CTUs or CUs) may be grouped in various ways within a picture. For example, a brick may refer to a rectangular area of ​​rows of CTUs within a specific tile in the picture. A tile may refer to a rectangular area of ​​CTUs within a specific tile column or a specific tile row in the picture. A tile column refers to a rectangular area of ​​CTUs having a height equal to the height of the picture and a width specified by syntax elements (e.g., as in a picture parameter set). A tile row refers to a rectangular area of ​​CTUs having a height specified by syntax elements (e.g., as in a picture parameter set) and a width equal to the width of the picture.

[0040] In some examples, a tile may be partitioned into multiple bricks, each of which may contain one or more CTU rows within the tile. A tile that is not partitioned into multiple bricks may also be referred to as a brick. However, a brick, which is a true subset of a tile, may not be referred to as a tile.

[0041] Bricks in a picture may also be arranged into slices. A slice may be an integer number of bricks in a picture that may be exclusively contained in a single Network Abstraction Layer (NAL) unit. In some examples, a slice includes only a number of complete tiles or a continuous sequence of complete bricks of a single tile.

[0042] The present disclosure may interchangeably use "NxN" and "N by N" to refer to the sample dimensions of a block (such as a CU or other video block) in terms of vertical and horizontal dimensions, e.g., 16x16 samples or 16 by 16 samples. Generally, a 16x16 CU will have 16 samples in the vertical direction (y = 16) and 16 samples in the horizontal direction (x = 16). Similarly, an NxN CU generally has N samples in the vertical direction and N samples in the horizontal direction, where N represents a non-negative integer value. Samples in a CU may be arranged in rows and columns. Furthermore, CUs do not necessarily have to have the same number of samples in the horizontal direction as in the vertical direction. For example, CUs may contain NxM samples, where M is not necessarily the same as N.

[0043] A video encoder (200) encodes video data for CUs and other information representing prediction and / or residual information. The prediction information indicates how the CU will be predicted to form a prediction block for the CU. The residual information generally represents sample-by-sample differences between the samples of the CU prior to encoding and the prediction block.

[0044] To predict the CU, the video encoder (200) may generally form a prediction block for the CU through inter-prediction or intra-prediction. Inter-prediction generally refers to predicting the CU from data of a previously coded picture, whereas intra-prediction generally refers to predicting the CU from data of the same picture previously coded. To perform inter-prediction, the video encoder (200) may generate a prediction block using one or more motion vectors. The video encoder (200) may generally perform motion search to identify a reference block that closely matches the CU, for example, in terms of differences between the CU and the reference block. The video encoder (200) may calculate a difference metric using the sum of absolute differences (SAD), sum of squared differences (SSD), mean absolute difference (MAD), mean squared differences (MSD), or other such difference calculations to determine whether the reference block closely matches the current CU. In some examples, the video encoder (200) may predict the current CU using unidirectional prediction or bidirectional prediction.

[0045] Motion vectors may have sub-pixel precision. In other words, a motion vector may represent a position in a reference picture between two samples of the reference picture in the horizontal and / or vertical directions. The accuracy of a motion vector refers to how accurate a position the motion vector may have relative to a position in the reference picture. For example, if the accuracy of a motion vector is 16, the motion vector may represent one of 15 positions between a sample of the reference picture or between a sample of the reference picture and the next sample. When performing inter-prediction using motion vectors, the video encoder (200) or video decoder (300) may apply an interpolation filter to the samples of the reference picture to determine the samples of the prediction block corresponding to the position indicated by the motion vector. For example, if the motion vector indicates a position vertically midway between two samples of a reference picture, the video encoder (200) or video decoder (300) may apply an interpolation filter to determine a value for a position vertically midway between two samples of a reference picture based at least on two samples of a reference picture.

[0046] Some examples of VVC also provide an affine motion compensation mode that may be considered as an inter-prediction mode. In the affine motion compensation mode, the video encoder (200) may determine two or more motion vectors representing non-translational motion, such as zoom in or out, rotation, perspective motion, or other irregular motion types.

[0047] To perform intra-prediction, the video encoder (200) may select an intra-prediction mode to generate prediction blocks. Some examples of VVC provide 67 intra-prediction modes, including planar mode and DC mode, as well as various directional modes. Generally, the video encoder (200) selects an intra-prediction mode that describes neighbor samples for the current block (e.g., a block of CU) to predict samples of the current block. Such samples may generally be located above, above and to the left of, or to the left of the current block in the same picture as the current block, assuming that the video encoder (200) codes CTUs and CUs in raster scan order (left, right, up, down).

[0048] The video encoder (200) encodes data indicating the prediction mode for the current block. For example, for inter-prediction modes, the video encoder (200) may encode motion information for the corresponding mode as well as data indicating which of the various available inter-prediction modes is used. For unidirectional or bidirectional inter-prediction, for example, the video encoder (200) may encode motion vectors using Advanced Motion Vector Prediction (AMVP) or a merge mode. The video encoder (200) may also encode motion vectors for an affine motion compensation mode using similar modes.

[0049] Following a prediction such as an intra prediction or inter prediction of a block, the video encoder (200) may calculate residual data for the block. Residual data, such as a residual block, represents sample-by-sample differences between the block and the prediction block for the block, formed using the corresponding prediction mode. The video encoder (200) may apply one or more transformations to the residual block to generate transformed data in a transformation domain instead of a sample domain. For example, the video encoder (200) may apply a Discrete Cosine Transform (DCT), an Integer Transform, a Wavelet Transform, or a conceptually similar transformation to the residual video data. Additionally, the video encoder (200) may apply a quadratic transformation following a first transformation, such as a Mode-Dependent Non-Separable Quadratic Transform (MDNSST), a Signal-Dependent Transform, or a Karhunen-Loeve Transform (KLT). The video encoder (200) generates transformation coefficients following the application of one or more transformations.

[0050] As mentioned above, following any transformations to generate transformation coefficients, the video encoder (200) may perform quantization of the transformation coefficients. Quantization generally refers to a process in which transformation coefficients are quantized to provide additional compression, such that the amount of data used to represent those transformation coefficients is reduced as much as possible. By performing the quantization process, the video encoder (200) may reduce the bit depth associated with some or all of the transformation coefficients. For example, the video encoder (200) may round down n-bit values ​​to m-bit values ​​during quantization, where n is greater than m. In some examples, to perform quantization, the video encoder (200) may perform a bitwise right-shift of the values ​​to be quantized.

[0051] Following quantization, the video encoder (200) may scan the transform coefficients to generate a one-dimensional vector from a two-dimensional matrix containing the quantized transform coefficients. The scan may be designed to place higher energy (and therefore lower frequency) transform coefficients at the front of the vector and lower energy (and therefore higher frequency) transform coefficients at the rear of the vector. In some examples, the video encoder (200) may generate a serialized vector by utilizing a predefined scan order to scan the quantized transform coefficients, and then entropy-encode the quantized transform coefficients of the vector. In other examples, the video encoder (200) may perform an adaptive scan. After scanning the quantized transform coefficients to form a one-dimensional vector, the video encoder (200) may entropy-encode the one-dimensional vector according to, for example, Context Adaptive Binary Arithmetic Coding (CABAC). The video encoder (200) may also entropy-encode values ​​for syntax elements describing metadata associated with the encoded video data for use by the video decoder (300) in decoding the video data.

[0052] To perform CABAC, the video encoder (200) may assign a context within a context model to the symbol to be transmitted. The context may, for example, be related to whether the neighboring values ​​of the symbol are zero values. The probability determination may be based on the context assigned to the symbol.

[0053] The video encoder (200) may also generate more syntax data, such as block-based syntax data, picture-based syntax data, and sequence-based syntax data. The video decoder (300) may likewise decode such syntax data to determine how to decode the corresponding video data.

[0054] In this way, the video encoder (200) may generate a bitstream containing syntax elements describing the partitioning of the encoded video data, for example, into blocks of a picture (e.g., CUs), and prediction and / or residual information for the blocks. Ultimately, the video decoder (300) may receive the bitstream and decode the encoded video data.

[0055] Generally, the video decoder (300) performs a process opposite to that performed by the video encoder (200) to decode the encoded video data of the bitstream. For example, the video decoder (300) may decode values ​​for the syntax elements of the bitstream using CABAC in a manner substantially similar to, but opposite to, the CABAC encoding process of the video encoder (200). The syntax elements may define the CUs of the CTU by defining partitioning information for partitioning the picture into CTUs and partitioning each CTU according to a corresponding partition structure such as a QTBT structure. The syntax elements may additionally define prediction and residual information for blocks of video data (e.g., CUs).

[0056] Residual information may be represented, for example, by quantized transform coefficients. The video decoder (300) may inversely quantize and inversely transform the quantized transform coefficients of the block to reconstruct a residual block for the block. The video decoder (300) forms a prediction block for the block using a signaled prediction mode (intra- or inter-prediction) and associated prediction information (e.g., motion information for inter-prediction). Then, the video decoder (300) may reconstruct the original block by combining the prediction block and the residual block (on a sample-by-sample basis). The video decoder (300) may perform additional processing, such as performing a deblocking process to reduce visual artifacts along the boundaries of the block.

[0057] A bitstream may contain a sequence of Network Abstraction Layer (NAL) units. A NAL unit is a syntax structure containing a representation of bytes that include the type of data in the NAL unit and, as needed, a raw byte sequence payload (RBSP) interspersed with anti-emulation bits. Each NAL unit may include a NAL unit header and may encapsulate an RBSP. The NAL unit header may include a syntax element that indicates a NAL unit type code. The NAL unit type code specified by the NAL unit header indicates the type of the NAL unit. An RBSP may be a syntax structure containing an integer of the bytes encapsulated within the NAL unit. In some cases, the RBSP includes zero bits.

[0058] As mentioned above, the bitstream may include video data and representations of encoded pictures of associated data. The associated data may include parameter sets. NAL units may encapsulate video parameter sets (VPS), sequence parameter sets (SPS), and picture parameter sets (PPS). A VPS is a syntax structure containing syntax elements that apply to zero or more whole coded video sequences (CVS). An SPS is also a syntax structure containing syntax elements that apply to zero or more whole CVS. An SPS may include syntax elements that identify the active VPS when the SPS is active. Therefore, the syntax elements of a VPS may be more generally applicable than the syntax elements of an SPS. A PPS is a syntax structure containing syntax elements that apply to zero or more coded pictures. A PPS may include a syntax element that identifies the active SPS when the PPS is active. A slice header of a slice may include a syntax element that indicates the active PPS when the slice segment is being coded.

[0059] FIG. 2 is a conceptual diagram illustrating exemplary chroma sample position types for a 4:2:0 chroma format. A picture may be divided into an upper field and a lower field. The upper field and the lower field may correspond to alternating rows of luminance samples. As illustrated in the example of FIG. 2, when a picture is encoded using a 4:2:0 chroma format, there may be six different types of chroma sample arrays (150) for each field. FIG. 2 has been described with reference to field coding, but chroma types may be used in frame coding. In examples where frame coding is used, each luminance sample of a frame (picture) may be considered to belong to the upper field, and chroma types 0 through 3 may be used.

[0060] Specifically, in the example of FIG. 2, when the picture is encoded using the 4:2:0 chroma format and type 0 chroma samples, all other chroma sample positions in the row of upper field chroma sample positions correspond to chroma samples, and the chroma samples are aligned horizontally with the upper field chroma sample positions and vertically aligned in the middle between the row of upper field chroma sample positions and the row of lower field chroma sample positions below the row of upper field chroma sample positions. When the picture is encoded using the 4:2:0 chroma format and type 0 chroma samples, all other chroma sample positions in the row of lower field chroma sample positions correspond to chroma samples, and the chroma samples are aligned horizontally with the lower field chroma sample positions and vertically aligned in the middle between the row of lower field chroma sample positions and the row of upper field chroma sample positions above the row of lower field chroma sample positions.

[0061] When a picture is encoded using the 4:2:0 chroma format and Type 1 chroma samples, all other lumina sample positions in the rows of upper field lumina sample positions correspond to chroma samples, and the chroma samples are aligned horizontally between the upper field lumina sample positions and vertically aligned in the middle between the rows of upper field lumina sample positions and the rows of lower field lumina sample positions below the rows of upper field lumina sample positions. When a picture is encoded using the 4:2:0 chroma format and Type 0 chroma samples, all other lumina sample positions in the rows of lower field lumina sample positions correspond to chroma samples, and the chroma samples are aligned horizontally between the lower field lumina sample positions and vertically aligned in the middle between the rows of lower field lumina sample positions and the rows of upper field lumina sample positions above the rows of lower field lumina sample positions.

[0062] When a picture is encoded using the 4:2:0 chroma format and Type 2 chroma samples, all other lumina sample positions in the row of upper field lumina sample positions correspond to chroma samples, and the chroma samples are directly aligned with the upper field lumina sample positions in the row of upper field lumina sample positions. When a picture is encoded using the 4:2:0 chroma format and Type 2 chroma samples, all other lumina sample positions in the row of lower field lumina sample positions correspond to chroma samples, and the chroma samples are directly aligned with the upper field lumina sample positions in the row of lower field lumina sample positions above the row of lower field lumina sample positions.

[0063] When a picture is encoded using the 4:2:0 chroma format and Type 3 chroma samples, all other lumina samples in the rows of upper field lumina sample locations correspond to a chroma sample, and the chroma sample is directly aligned with the rows of upper field lumina sample locations and the midpoint between the upper field lumina sample locations. When a picture is encoded using the 4:2:0 chroma format and Type 3 chroma samples, all other lumina samples in the rows of lower field lumina sample locations correspond to a chroma sample, and the chroma sample is vertically aligned with the rows of upper field lumina sample locations above the rows of lower field lumina sample locations and horizontally aligned with the midpoint between the lower field lumina sample locations.

[0064] When a picture is encoded using the 4:2:0 chroma format and Type 4 chroma samples, all other lumina samples in the rows of upper field lumina sample positions correspond to a chroma sample, and the chroma sample is aligned with the lower field lumina sample positions below the corresponding upper field lumina sample positions. When a picture is encoded using the 4:2:0 chroma format and Type 4 chroma samples, all other lumina samples in the rows of lower field lumina sample positions correspond to a chroma sample, and the chroma sample is aligned with the corresponding lower field lumina sample positions.

[0065] When a picture is encoded using the 4:2:0 chroma format and Type 5 chroma samples, all other lumina samples in the rows of upper field lumina sample positions correspond to chroma samples, and the chroma samples are horizontally aligned at the intermediate position between the upper field lumina sample positions and vertically aligned with the rows of lower field lumina sample positions below the rows of upper field lumina sample positions. When a picture is encoded using the 4:2:0 chroma format and Type 5 chroma samples, all other lumina samples in the rows of lower field lumina sample positions correspond to chroma samples, and the chroma samples are horizontally aligned at the intermediate position between the lower field lumina sample positions and vertically aligned with the rows of upper field lumina sample positions above the rows of lower field lumina sample positions.

[0066] In examples where frame coding is used (i.e., when upper and lower fields do not exist), all other luminance samples in the column of luminance sample locations correspond to chroma samples, and the chroma samples are aligned vertically and horizontally with respect to the upper field as described above.

[0067] Reference Picture Resampling (RPR) was introduced in VVC Draft 6. RPR causes the current picture and the reference picture to have different resolutions. Therefore, in motion compensation, the resolution difference is accounted for during interpolation filtering by adjusting the interpolation filter phase and the reference block start position. Additionally, scaling ratios are derived for horizontal and vertical directions based on the widths and heights of the current and reference pictures, and the fit windows of the current and reference pictures. Scaling ratios for horizontal and vertical directions may also be determined based on the picture's output width (PicOutputWidthL) and output height (PicOutputHeightL). In the VTM-6 specification draft, PicOutputWidthL and PicOutputHeightL may be derived as follows:

[0068]

[0069] In Equation (7-43), pic_width_in_luma_samples is a syntax element in PPS representing the width of each decoded picture, referencing PPS in units of lumina samples. In Equation (7-44), pic_height_in_luma_samples is a syntax element in PPS representing the height of each decoded picture, referencing PPS in units of lumina samples. That is, pic_width_in_luma_samples and pic_height_in_luma_samples are the width and height of the picture. SubWidthC represents the horizontal subsampling rate (e.g., SubWidthC equal to 1 indicates no horizontal subsampling, SubWidthC equal to 2 indicates that all other samples are removed in the horizontal direction, etc.). Similarly, SubHeightC represents the vertical subsampling rate (for example, SubHeightC indicates no vertical subsampling, and SubWidthC, which is equal to 2, indicates that all other samples in the vertical direction are removed, etc.). The syntax elements conf_win_left_offset, conf_win_right_offset, conf_win_top_offset, and conf_win_bottom_offset specify the samples of the pictures in the CVS output from the decoding process in terms of a rectangular area specified in the picture coordinates for output. That is, conf_win_right_offset, conf_win_left_offset, conf_win_bottom_offset, and conf_win_top_offset are the left, right, bottom, and top offsets of the conformity window.

[0070] In VTM-6, the scaling ratios (i.e., the horizontal scaling ratio (hori_scale_fp) and the vertical scaling ratio (vert_scale_fp)) are derived as follows:

[0071]

[0072] The variable fRefWidth is set to be equal to the PicOutputWidthL of the reference picture in the luma samples. The variable fRefHeight is set to be equal to the PicOutputHeightL of the reference picture in the luma samples. PicOutputWidthL and PicOutputHeightL are variables derived for the current picture.

[0073] The starting position of the block (top left corner) in the reference picture denoted by ( xIntL, yIntL ) and the filter phase or offset given by 1 / 16-sample units denoted by ( xFracL, yFracL ) are derived as follows:

[0074] - ( refxSb L , refySb L ) and ( refx L , refy L ) are luminance positions pointed to by motion vectors ( refMvLX[0], refMvLX[1] ) given in 1 / 16-sample units. Variables refxSb L , refx L , refySb L , and refy L It is derived as follows:

[0075]

[0076] - Variables xInt L , yInt L , xFrac L and yFrac L It is derived as follows:

[0077]

[0078]

[0079] In the text above, (xSb, ySb) specifies the top-left sample of the current coding subblock for the top-left luma sample of the current picture, and refMvLX is the motion vector for the reference picture list LX (where X is 0 and 1).

[0080] In VVC Draft 6, the same process is repeated for the chroma color components as follows.

[0081] - For each chroma sample position (xC = 0..sbWidth - 1, yC = 0...sbHeight - 1) within the predicted chroma sample arrays predSamplesLX, the corresponding predicted chroma sample value predSamplesLX[ xC ][ yC ] is derived as follows:

[0082] - (refxSb C , refySb C ) and ( refx C , refy C ) are chroma positions pointed to by motion vectors (mvLX

[0000] , mvLX

[0001] ) given in 1 / 32-sample units. Variables refxSb C , refySb C, refx C and refy C It is derived as follows:

[0083]

[0084] - Variables xInt C , yInt C , xFrac C and yFrac C It is derived as follows:

[0085]

[0086]

[0087] In VVC Draft 6, Chroma has 32 filter phases, whereas Luma has 16 filter phases. That is, a motion vector may represent one of the 16 positions for a Luma sample, and a motion vector may represent one of the 32 positions for a Chroma sample. Fractional position xFrac considering that resampling is applied when the Chroma filter phase is derived, i.e., when hori_scale_fp or vert_scale_fp differs from the 1x ratio. C and yFrac C In this, as illustrated in FIG. 2, a zero-phase shift between the grids of the luminance and chroma components is assumed to be type 2 chroma sampling. In this disclosure, the term 1x ratio refers to a 1-to-1 ratio.

[0088] However, there may be other chroma types as illustrated in FIG. 2, which may have phase shifts different from 0 between the lumina and chroma components. For example, Type 2 chroma sampling has a phase shift of 0 in the horizontal and vertical directions, Type 0 chroma sampling has a phase shift of 0 in the horizontal direction and a phase shift of 0.5 in the vertical direction, and for Type 1, both the horizontal and vertical phase shifts between the components are equal to 0.5.

[0089] When interpolation filtering is applied in VVC Draft 6, phase shifts between components are not considered, that is, they are always assumed to be 0. Therefore, when this zero-phase filtering is applied to non-Type 2 chroma sampling, the results of the filtering will change the chroma sampling type from the original input type, which may be undesirable. Changing the chroma sampling type in this way can affect coding efficiency, affect the output picture, and / or the display / rendering of the picture, as there may be a mismatch between the input picture chroma type and the output picture chroma type.

[0090] The techniques of the present disclosure are described in relation to frame coding. However, the techniques of the present disclosure may also be applicable to frame field coding. In the latter case, any solution of the present disclosure may be applied individually to upper and lower fields. For example, if the present disclosure describes a syntax element for frame coding, a similar syntax element is signaled for the upper field and another similar syntax element is signaled for the lower field.

[0091] According to the technique of the present disclosure, to solve the chroma sampling type problem, information on chroma type or horizontal and vertical chroma phase shifts may be signaled in a parameter set such as VPS, SPS, PPS, or others. These phase shifts may also indicate the input type for each picture.

[0092] In the current VVC Draft 6, the chroma_sample_loc_type_frame syntax element (also referred to as the chroma_sample_loc_type syntax element) is signaled in the SPS Video Availability Information (VUI) to indicate phase shifts. Because the chroma_sample_loc_type_frame syntax element is signaled in the VUI, the video decoder may ignore the chroma_sample_loc_type_frame syntax element. In examples where field coding is used, similar syntax elements (e.g., chroma_sample_loc_type_top_field and chroma_sample_loc_type_bottom_field) may be signaled for the top and bottom fields.

[0093] In one example of the present disclosure, the chroma_sample_loc_type_frame syntax element is signaled in the PPS. Therefore, the chroma_sample_loc_type_frame syntax element is not signaled in the negligible VUI portion of the SPS. This relates to Rapaka et al., "AHG13: chroma phase offset for SHVC resampling process," Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 16 thMeeting, San Jose, US, 9-17 Jan. 2014, document JCTVC-P0164 (hereinafter, "JCTVC-P0164"). It is similar to the signaling of the chroma_phase_idc syntax element described in JCTVC-P0164 and is signaled in the VPS for HEVC scalable extension. In JCTVC-P0164, chroma_phase_idc specifies the locations of the chroma sample grids for all pictures referencing the VPS. In JCTVC-P0164, the value of chroma_phase_idc is in the range including 0 to 5. In JCTVC-P0164, the variables chroma_phase_X and chroma_phase_Y are derived as follows:

[0094]

[0095] In another example of the present disclosure, horizontal and vertical phase shifts are explicitly signaled, for example, as is done in HEVC, where these shifts are signaled for each inter-layer reference picture in the PPS extension. For example, HEVC specifies the phase_hor_chroma_plus8 syntax element and the phase_ver_chroma_plus8 syntax element as follows:

[0096]

[0097] phase_hor_chroma_plus8[ ref_loc_offset_layer_id[ i ] minus 8 specifies the chroma phase shift in the horizontal direction used in resampling of a direct reference layer picture having a nuh_layer_id equal to ref_loc_offset_layer_id[ i ] ]. The value of phase_hor_chroma_plus8[ ref_loc_offset_layer_id[ i ] ] must be in the range including 0 to 63. If not present, the value of phase_hor_chroma_plus8[ ref_loc_offset_layer_id[ i ] ] is inferred to be equal to 8.

[0098] phase_ver_chroma_plus8 [ ref_loc_offset_layer_id[ i ] ] minus 8 specifies the chroma phase shift in the vertical direction used in the resampling process of the direct reference layer picture having a nuh_layer_id identical to ref_loc_offset_layer_id[ i ]. The value of phase_ver_chroma_plus8[ ref_loc_offset_layer_id[ i ] ] must be in the range of 0 to 63. If it does not exist, the value of phase_ver_chroma_plus8[ ref_loc_offset_layer_id[ i ] ] is inferred as follows:

[0099] - If chroma_format_idc is equal to 3 (4:4:4 chroma format), the value of phase_ver_chroma_plus8[ ref_loc_offset_layer_id[ i ] ] is inferred to be equal to 8.

[0100] - Otherwise, the value of phase_ver_chroma_plus8[ ref_loc_offset_layer_id[ i ] ] is inferred to be equal to ( 4 * scaledRefRegHeight + refRegHeight / 2 ) / refRegHeight + 4, where the value of scaledRefRegHeight is equal to the value of ScaledRefRegionHeightInSamplesY derived for a direct reference layer picture having a nuh_layer_id equal to ref_loc_offset_layer_id[ i ] of the picture referencing this PPS, and the value of refRegHeight is equal to the value of RefLayerRegionHeightInSamplesY derived for a direct reference layer picture having a nuh_layer_id equal to ref_loc_offset_layer_id[ i ] of the picture referencing this PPS.

[0101]

[0102] According to one or more techniques of the present disclosure, phase shifts indicate the chroma sampling type of each picture. In one example, the phase shifts indicate that a given picture has this specific chroma type, and that other pictures may have different types in the same sequence. In another example, this chroma phase shift information may be signaled in an SPS. In this example, the chroma phase shift information indicates that all pictures have that specific chroma type.

[0103] Additionally, in some examples of the present disclosure, it may always be assumed and limited in the resampling (down-sampling or up-sampling) process that the chroma type remains unchanged across all pictures in the video sequence, and when the resampling process is applied, the required chroma phase shifts are derived for each picture, and the phase shifts may differ for each picture to maintain the chroma type unchanged. The chroma type may be indicated as in the examples described above. A picture may be an input picture, a current picture, a reference picture, or an output picture.

[0104] Chroma phase shift may be signaled as a positive value. However, the sign of the phase shift may be derived based on whether down-sampling (e.g., a scaling ratio of less than 1x) or up-sampling (e.g., a scaling ratio of greater than 1x) is applied. In one example, the sign of the chroma phase shift is positive when down-sampling is applied and negative when up-sampling is applied in the horizontal or vertical direction. In some examples, down-sampling may be applied in one direction and up-sampling in the other direction.

[0105] There may be cases where the current picture and the reference picture have different chroma phase shifts. When the current picture and the reference picture have different chroma phase shifts, the current and reference phase shifts of both are used in the filtering and resampling processes.

[0106] In both examples (i.e., cases where down-sampling is applied in one direction and up-sampling is applied in the other (e.g., different resolutions) and cases where the current picture and the reference picture have different chroma phase shifts), chroma phase shift information is used in an interpolation filtering process that considers only zero-phase shifts. However, chroma phase shift information may also be considered when resampling is applied, that is, only when hori_scale_fp or vert_scale_fp differs from the 1x ratio. This means that the signaled chroma phase shift information is applied conditionally in filtering, namely, chroma_phase_X is used when hori_scale_fp differs from the 1x ratio, and chroma_phase_Y is used when vert_scale_fp differs from the 1x ratio.

[0107] It should be noted that when the chroma format is not 4:2:0, such as 4:2:2 or 4:4:4 (e.g., when chroma_format_idc is not equal to 4:2:0), chroma subsampling is not applied in the vertical direction (4:2:2) or chroma subsampling is not applied at all (4:4:4). In VVC Draft 6, the chroma_format_idc syntax element indicates the chroma sampling format (i.e., chroma format). In some examples where the chroma format is not 4:2:0 (e.g., when chroma_format_idc is not equal to 4:2:0), the techniques of the present disclosure do not apply to vertical filtering for the 4:2:2 format or to both horizontal and vertical filtering for the 4:4:4 format or the monochrome format.

[0108] Table 1. chroma_format_idc and separate_colour_plane_flag

[0109]

[0110] A video decoder (300) may use Table 1 above to determine the chroma format, SubWidthC, and SubHeightC based on the chroma_format_idc syntax element and the separate_color_plane_flag syntax element that may be signaled in the bitstream. The separate_color_plane_flag syntax element equal to 1 specifies that the three color components of the 4:4:4 chroma format are coded individually. The separate_color_plane_flag equal to 0 specifies that the color components are not coded individually. For example, if the chroma_format_idc syntax element is equal to 3 and the separate_color_plane_flag syntax element is equal to 0, the chroma format is 4:4:4, SubWidthC is equal to 1, and SubHeightC is equal to 1.

[0111] In one implementation example, refxSb C , refx C , refySb C , refy C is modified to account for chroma phase shift as shown in Table 2 below, where<!----> ...<!--!--> Text surrounded by tags is added to VVC Draft 6:

[0112] Table 2

[0113]

[0114] In Table 2, addX is set based on horizontal chroma phase shifts, and addY is set based on vertical chroma phase shifts. In one example, the chroma phase shifts include a chroma phase shift for the current picture and a chroma phase shift for the reference picture.

[0115] In a more detailed example, if the horizontal scaling ratio (hori_scale_fp) is equal to the 1x ratio or chroma_format_idc is equal to 3 or 0, addX is set to 0. Otherwise, if the horizontal scaling ratio is not equal to the 1x ratio and chroma_format_idc is not equal to 3 or 1, addX is set based on the chroma phase shifts in the horizontal direction. If the vertical scaling ratio (vert_scale_fp) is equal to the 1x ratio or chroma_format_idc is equal to 2, 3, or 0, addY is set to 0. Otherwise, if the vertical scaling ratio is not equal to the 1x ratio and chroma_format_idc is not equal to 2, not equal to 3, and not equal to 0, addY is set based on the chroma phase shifts in the vertical direction. The values ​​addX and addY may also be set as described in the following examples.

[0116] In some examples, addX and addY are set based on chroma phase shifts (e.g., by the video encoder (200) and video decoder (300)) as illustrated in the examples of Table 3 below:

[0117] Table 3

[0118]

[0119] In the mathematical formulas of Table 3, (1<<14) is the accuracy factor for horizontal and vertical scaling. Additionally, in the mathematical formulas of Table 3, the right shift of 2 (divided by 4) comes from the definitions of chroma_phase_X and chroma_phase_Y, which have accuracy factors of 2 (i.e., 1 corresponds to a 0.5 shift) and a chroma subsampling ratio for luminance (2 in the 4:2:0 format). addX and addY are right-shifted by 5 because the number of chroma fractional positions is 32, which is the same as the chroma MV accuracy.

[0120] When considering other chroma formats such as 4:2:2 and 4:4:4, in one example, the above formulas may be modified to include chroma subsampling ratios SubWidthC and SubHeightC as illustrated in the example of Table 4 below:

[0121] Table 4

[0122]

[0123] In the formulas of Table 4, when there is no chroma subsampling in the horizontal or vertical direction, division by 4 (right shift by 2) is replaced by division by 2. That is, as shown in Table 4, the value of SubWidthC may be equal to 1 when there is no horizontal subsampling, and may be equal to 2 when there is horizontal subsampling. Therefore, division by 2 or 4 occurs depending on the presence or absence of horizontal subsampling. Similarly, the value of SubHeightC may be equal to 1 when there is no vertical subsampling, and may be equal to 2 when there is vertical subsampling. Therefore, division by 2 or 4 occurs depending on the presence or absence of vertical subsampling.

[0124] A different number of fractional positions may be used, and thus the right shift may vary depending on the number of fractional positions, and different accuracy expressions may be used for scaling factors and phases. In general, the formulas in Tables 3 and 4 are just examples of integer implementations, and other similar formulas may be used.

[0125] In some examples, the last two steps of the mathematical expressions in Tables 3 and 4 can be combined by incorporating shifts (5-2) as shown in Table 5 below:

[0126] Table 5

[0127]

[0128] In the examples of Tables 3, 4, and 5, chroma_phase_X and chroma_phase_Y may be chroma phase shifts of the current picture (after resampling), while ref_chroma_phase_X and ref_chroma_phase_Y may be chroma phase shifts of the reference picture (before resampling).

[0129] If the chroma phase accuracy is 16, the above formulas are rewritten as shown in the example in Table 6 below:

[0130] Table 6

[0131]

[0132] When considering other chroma formats (e.g., 4:2:2 and 4:4:4), the video encoder (200) and video decoder (300) may calculate addX and addY according to the following formulas in Table 7:

[0133] Table 7

[0134]

[0135] In another example, instead of dividing into SubWidthC and SubHeightC, the horizontal and vertical chroma phase values ​​can be scaled by SubWidthC and SubHeightC. Scaling can be performed before or after phase signaling. In the first case (i.e., in examples where scaling is performed before phase signaling), a smaller magnitude value is signaled, which may be advantageous because fewer bits may be required to signal a smaller magnitude value than a larger magnitude value.

[0136] In another example, when the chroma phase accuracy is 32, the formulas can be rewritten as shown in Table 8 below:

[0137] Table 8

[0138]

[0139] In some examples, a fixed chroma phase may be used. That is, the chroma phase is always the same and is therefore not signaled. For example, since chroma sample type 0 may be the most commonly used chroma format, a chroma phase corresponding to chroma sample type 0 may be used. In this example, chroma_phase_X = ref_chroma_phase_X = 0 and chroma_phase_Y = ref_chroma_phase_Y = 1.

[0140] In another example, the current chroma phase can be fixed, but the reference chroma phase is signaled. In yet another example, the reference chroma phase can be fixed, but the current chroma phase is signaled.

[0141] Accordingly, according to the techniques of the present disclosure, a video coder (e.g., a video encoder (200) or a video decoder (300)) may modify the horizontal component of a chroma position pointed to by the motion vector of the current block of the current picture of the video data based on chroma phase shifts in the horizontal direction. The chroma position is in a reference picture having a different resolution or a different chroma sampling type from the current picture. The video coder may also modify the vertical component of the chroma position based on chroma phase shifts in the vertical direction. Additionally, the video coder may generate chroma samples of a prediction block for the current block of the current picture based on chroma samples of the reference picture corresponding to the modified chroma position. The modified chroma position is defined by the modified horizontal and vertical components of the chroma position. The video coder may code the current block based on the prediction block for the current block.

[0142] Similar methods can be applied to the luminance phase. Phase shifts in the luminance may occur, for example, when downsampling is performed outside the codec (for the input picture to be encoded in one example) and non-zero phase alignment is utilized during downsampling. Similarly, upsampling filtering may be used outside the codec (for display in one example) and non-zero phase alignment filtering may be applied. In these examples, a discrepancy in the luminance phase between the input / output picture and the encoded or reference picture may occur.

[0143] To avoid this phase mismatch, luminance phases may be signaled or used in a filtering process. In one example, signaling of luminance phases may be performed using HEVC syntax. In HEVC, luminance phases are signaled only for inter-layer pictures (i.e., pictures that depend on pictures in different layers). However, according to one example of the present disclosure, HEVC syntax for signaling luminance phases may be applied to the current picture as well as inter-layer pictures. HEVC syntax and semantics for signaling luminance phases are presented below.

[0144]

[0145] phase_hor_luma [ ref_loc_offset_layer_id[ i ] ] specifies the luma phase shift in the horizontal direction used in the resampling process of a direct reference layer picture having the same nuh_layer_id as ref_loc_offset_layer_id[ i ] ]. The value of phase_hor_luma[ ref_loc_offset_layer_id[ i ] ] must be in the range including 0 to 31. If it does not exist, the value of phase_hor_luma[ ref_loc_offset_layer_id[ i ] ] is inferred to be equal to 0.

[0146] phase_ver_luma[ ref_loc_offset_layer_id[ i ] ] specifies the lumina phase shift in the vertical direction used in resampling of a direct reference layer picture having the same nuh_layer_id as ref_loc_offset_layer_id[ i ] ]. The value of phase_ver_luma[ ref_loc_offset_layer_id[ i ] ] must be in the range including 0 to 31. If it does not exist, the value of phase_ver_luma[ ref_loc_offset_layer_id[ i ] ] is inferred to be equal to 0.

[0147] In other examples, luminance phases may be signaled in other ways. For example, luminance phases may be signaled using a luma_phase_idc syntax element similar to the chroma_phase_idc syntax element discussed above. The luma_phase_idc syntax element indicates a position shift between one picture and another picture, such as the current picture and a reference picture. Luma phases may also be derived from luma_phase_idc (in the same way that, for example, chroma phases are derived from the chroma_phase_idc syntax element).

[0148] Information regarding horizontal and vertical luminance phase shifts may be signaled in parameter sets such as VPS, SPS, PPS, or elsewhere. Phase shifts may also indicate the input type for each picture. Horizontal and vertical luminance phase shifts may be signaled in the same parameter set as chroma phase shifts, such as PPS. In other examples, horizontal and vertical luminance phases may be signaled in a higher parameter set, such as SPS or VPS, while chroma phase shifts are signaled in PPS. The underlying idea behind signaling horizontal and vertical luminance phase shifts in a higher parameter set is that while luminance phase shifts may be global position shift characteristics for the entire video sequence, chroma phase shifts may be more specific to a picture.

[0149] As previously described regarding chroma, considering the luminance phases of the current picture and the reference picture (e.g., in the mathematical formulas of Tables 3 through 8), modifications similar to those performed in chroma filters may be performed in luminance filtering. Luminance modification is applied when the scaling factor differs from the 1x ratio, that is, when the reference picture and the current picture have different widths or heights or both (e.g., in the video encoder (200) and video decoder (300)).

[0150]

[0151] In a more detailed example, if hori_scale_fp is equal to a 1x ratio, addX is set to 0; otherwise, addX is set based on the luminance phase shifts in the horizontal direction. If vert_scale_fp is equal to a 1x ratio, addY is set to 0; otherwise, addY is set based on the luminance phase shifts in the vertical direction, for example, as shown in Table 9 below.

[0152] Table 9

[0153]

[0154] Here, (1<<14) is the accuracy factor for horizontal and vertical scaling, and the right shift by 1 (i.e., divided by 2) comes from the definitions of chroma_phase_X and chroma_phase_Y having an accuracy factor of 2 (i.e., 1 corresponds to a 0.5 shift), and is right shifted by 4 because of the number of luminance fractional positions, which is 16, equal to the luminance MV accuracy.

[0155] In other examples, addX and addY are set based on luma phase shifts as shown in Table 10 below:

[0156] Table 10

[0157]

[0158] In this example, the luminance phase is considered to have an accuracy factor of 16. The same annotation may apply that some of the luminance phases can be fixed (current or reference), and in another example, one can be fixed and the other can be signaled.

[0159] The accuracy of the luminance phase and the chroma phase may differ; for example, the accuracy of the luminance phase may be 8, but the accuracy of the chroma phase may be 16. In this example, a 0.5 shift of the luminance corresponds to 4, and a 0.5 shift of the chroma corresponds to 8. The accuracy factors of the luminance and chroma may be set to be equal to the number of fractions in the luminance and chroma motion interpolation filters, respectively.

[0160] Accordingly, in some examples, the video coder may modify the horizontal component of the luminance position, which is pointed to by the motion vector of the current block of the current picture, based on luminance phase shifts in the horizontal direction. The luminance position is located in a reference picture having a different resolution or a different chroma sampling type from the current picture. The video coder may also modify the vertical component of the luminance position based on luminance phase shifts in the vertical direction. Additionally, the video coder may generate luminance samples of the prediction block for the current block based on the luminance samples of the reference picture corresponding to the modified luminance position. The modified luminance position is defined by the modified horizontal and vertical components of the luminance position. The video coder may also code the current block based on the prediction block for the current block.

[0161] The following section of the present disclosure describes an exemplary solution for luminance and chroma using VVC Draft 6, which has an accuracy factor of 16 for both luminance and chroma phases. The text added to VVC Draft 6 is<!----> ...<!--!--> Indicated by tags. A subset of exemplary solutions may also be used (e.g., only chroma phase offsets or only predefined fixed phase offsets).

[0162]

[0163] Same as 1 <!----> resample_phase_set_present_flag specifies that a resample phase set exists in PPS. resample_phase_set_present_flag, which is equal to 0, specifies that a resample phase set does not exist in PPS.

[0164] The resampling phase set specifies the phase offsets used in the resampling process of the picture referencing the PPS.

[0165] phase_hor_lumaspecifies the luma phase shift in the horizontal direction used in the resampling process. The value of phase_hor_luma must be in the range including 0 to 31. If it does not exist, the value of phase_hor_luma is inferred to be equal to 0.

[0166] phase_ver_luma Specifies the lumina phase shift in the vertical direction used in the resampling process. The value of phase_ver_luma must be in the range including 0 to 31. If it does not exist, the value of phase_ver_luma is inferred to be equal to 0.

[0167] phase_hor_chroma Specifies the chroma phase shift in the horizontal direction used in the resampling process. The value of phase_hor_chroma must be in the range including 0 to 63. If it does not exist, the value of phase_hor_chroma is inferred to be equal to 0.

[0168] phase_ver_chroma specifies the chroma phase shift in the vertical direction used in the resampling process. The value of phase_ver_chroma must be in the range including 0 to 63. If it does not exist, and chroma_format_idc is equal to 3 (4:4:4 chroma format) or otherwise equal to 8, the value of phase_ver_chroma is inferred to be equal to 0.<!--!-->

[0169] 8.5.6.3 Fractional Sample Interpolation Process

[0170] 8.5.6.3.1 General

[0171] The inputs for this process are as follows:

[0172] - Luma position ( xSb, ySb ), specifying the upper left sample of the current coding block for the upper left luma sample of the current picture

[0173] - Variable sbWidth that specifies the width of the current coding subblock,

[0174] - Variable sbHeight that specifies the height of the current coding subblock,

[0175] - Motion vector offset mvOffset,

[0176] - Refined motion vector refMvLX,

[0177] - Selected reference picture sample array refPicLX,

[0178] - Half-sample interpolation filter index hpelIfIdx,

[0179] - Bidirectional optical flow flag bdofFlag,

[0180] - Variable cIdx that specifies the color component index of the current block.

[0181] The outputs of this process are as follows:

[0182] - Array of (sbWidth + brdExtSize)x(sbHeight + brdExtSize) of predicted sample values ​​predSamplesLX.

[0183] The predicted block boundary expansion size brdExtSize is derived as follows:

[0184]

[0185] The variable fRefWidth is set to be equal to the PicOutputWidthL of the reference picture in the luma samples.

[0186] The variable fRefHeight is set to be equal to the PicOutputHeightL of the reference picture in the luma samples.

[0187] The motion vector mvLX is set to be equal to ( refMvLX - mvOffset ).

[0188] - If cIdx is equal to 0, the following applies:

[0189] - Scaling factors and fixed point representations are defined as follows:

[0190]

[0191] Let (xIntL, yIntL) be the luminance position given in whole-sample units, and (xFracL, yFracL) be the offset given in 1 / 16-sample units. These variables are used only in this clause to specify the fractional sample position within the reference sample array refPicLX.

[0192] - Top-left coordinates of the bounding block for reference sample padding ( xSbInt L , ySbInt L ) is set to be the same as ( xSb + ( mvLX

[0000] >> 4 ), ySb + ( mvLX

[0001] >> 4 ) )

[0193] - Each luminous sample position within the predicted luminous sample array predSamplesLX ( x L = 0..sbWidth - 1 + brdExtSize, y L For = 0..sbHeight - 1 + brdExtSize ), the corresponding predicted luminance sample value predSamplesLX[ x L ][ y L ] is derived as follows:

[0194] - ( refxSb L , refySb L ) and ( refx L , refy L Let ) be the luminance positions pointed to by the motion vector ( refMvLX[0], refMvLX[1] ) given in 1 / 16-sample units. Variables refxSb L , refx L , refySb L , and refy L It is derived as follows:

[0195] <!---->ref_ phase_hor_luma is set to be the same as phase_hor_luma of refPicLX, and ref_ phase_ver_luma is set to be the same as phase_ver_luma of refPicLX.

[0196]

[0197] - Variables xInt L , yInt L , xFrac L and yFrac L It is derived as follows:

[0198]

[0199]

[0200] - If bdofFlag is equal to TRUE (sps_affine_prof_enabled_flag is equal to TRUE and inter_affine_flag[ xSb ][ ySb ] is equal to TRUE), and if one or more of the following conditions are true, the predicted luminance sample value predSamplesLX[ x L ][ y L ] is as inputs in Clause 8.5.6.3.3 ( xInt L + ( xFrac L >> 3) - 1), yInt L + ( yFrac L >> 3 ) - 1 ) and as specified by refPicLX, it is derived by calling the luma integer sample fetching process.

[0201] - x L is the same as 0.

[0202] - x L is equivalent to sbWidth + 1.

[0203] - y L is the same as 0.

[0204] - y Lis the same as sbHeight + 1.

[0205] - Otherwise, the predicted luminance sample values ​​predSamplesLX[ xL ][ yL ] are inputs ( xIntL - ( brdExtSize > 0 ? 1 : 0 ), yIntL - ( brdExtSize > 0 ? 1 : 0 ) ), ( xFracL, yFracL ), ( xSbInt L , ySbInt L It is derived by calling the luminance sample 8-tap interpolation filtering process as specified in Clause 8.5.6.3.2 with ), refPicLX, hpelIfIdx, sbWidth, sbHeight and ( xSb, ySb )

[0206] - Otherwise (if cIdx is not equal to 0), the following applies:

[0207] Let (xIntC, yIntC) be the chroma position given in whole-sample units, and (xFracC, yFracC) be the offset given in 1 / 32-sample units. These variables are used only in this clause to specify the general fractional sample positions within the reference sample array refPicLX.

[0208] - The top-left coordinates ( xSbIntC, ySbIntC ) of the boundary block for reference sample padding are set to be the same as ( (xSb / SubWidthC ) + ( mvLX

[0000] >> 5 ), ( ySb / SubHeightC ) + ( mvLX

[0001] >> 5 ) )

[0209] - For each chroma sample position (xC = 0..sbWidth - 1, yC = 0...sbHeight - 1) within the predicted chroma sample arrays predSamplesLX, the corresponding predicted chroma sample value predSamplesLX[ xC ][ yC ] is derived as follows:

[0210] - (refxSb C , refySb C ) and ( refx C , refy C Let ) be the chroma positions pointed to by the motion vector (mvLX

[0000] , mvLX

[0001] ) given in 1 / 32-sample units. Variables refxSb C , refySb C, refx C and refy C It is derived as follows:

[0211] <!----> ref_ phase_hor_chroma is set to be the same as refPicLX's phase_hor_luma, and ref_ phase_ver_chroma is set to be the same as refPicLX's phase_ver_luma.

[0212]

[0213] - Variables xInt C , yInt C , xFrac C and yFrac C It is derived as follows:

[0214]

[0215] - The predicted sample values ​​predSamplesLX[ xC ][ yC ] are derived by calling the process specified in clause 8.5.6.3.4 with inputs ( xIntC, yIntC ), ( xFracC, yFracC ), ( xSbIntC, ySbIntC ), sbWidth, sbHeight and refPicLX.

[0216] The present disclosure may generally refer to "signaling" certain information, such as syntax elements. The term "signaling" may refer to the communication of values ​​for syntax elements and / or other data used to decode encoded video data. That is, the video encoder (200) may signal values ​​for syntax elements in a bitstream. Generally, signaling refers to generating values ​​in a bitstream. As mentioned above, the source device (102) may transmit the bitstream to the destination device (116) non-real-time or substantially real-time, as may occur when storing syntax elements in the storage device (112) for subsequent retrieval by the destination device (116).

[0217] FIG. 3 is a block diagram illustrating an exemplary video encoder (200) capable of performing the techniques of the present disclosure. FIG. 3 is provided for illustrative purposes and should not be considered as a limitation to the techniques roughly illustrated and described in the present disclosure. For illustrative purposes, the present disclosure describes a video encoder (200) in the context of video coding standards such as the HEVC video coding standard and the VVC / H.266 video coding standard under development. However, the techniques of the present disclosure are not limited to these video coding standards and are generally applicable to video encoding and decoding.

[0218] In the example of FIG. 3, the video encoder (200) includes a video data memory (230), a mode selection unit (202), a residual generation unit (204), a conversion processing unit (206), a quantization unit (208), an inverse quantization unit (210), an inverse conversion processing unit (212), a restoration unit (214), a filter unit (216), a decoded picture buffer (DPB) (218), and an entropy encoding unit (220). Any or all of the video data memory (230), the mode selection unit (202), the residual generation unit (204), the conversion processing unit (206), the quantization unit (208), the inverse quantization unit (210), the inverse conversion processing unit (212), the restoration unit (214), the filter unit (216), the DPB (218), and the entropy encoding unit (220) may be implemented in one or more processors or in a processing circuit. Furthermore, the video encoder (200) may include additional or alternative processors or processing circuits to perform these and other functions.

[0219] The video data memory (230) may store video data to be encoded by components of the video encoder (200). The video encoder (200) may receive video data stored in the video data memory (230) from, for example, a video source (104) (Fig. 1). The DPB (218) may act as a reference picture memory to store reference video data for use in predicting subsequent video data by the video encoder (200). The video data memory (230) and the DPB (218) may be formed by any of various memory devices, such as dynamic random access memory (DRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices, including synchronous dynamic random access memory (SDRAM). The video data memory (230) and the DPB (218) may be provided by the same memory device or by separate memory devices. In various examples, the video data memory (230) may be on-chip with other components of the video encoder (200) as exemplified, or off-chip with respect to those components.

[0220] In the present disclosure, a reference to the video data memory (230) should not be interpreted as being limited to memory inside the video encoder (200) unless specifically described otherwise, or to memory outside the video encoder (200) unless specifically described otherwise. Rather, a reference to the video data memory (230) should be understood as a reference memory that stores video data received by the video encoder (200) for encoding (e.g., video data for the current block to be encoded). The memory (106) of FIG. 1 may also provide temporary storage of outputs from various units of the video encoder (200).

[0221] Various units of FIG. 3 are illustrated to aid in understanding the operations performed by the video encoder (200). The units may be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Fixed-function circuits refer to circuits that provide specific functions and are pre-configured for the operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functions for the operations that can be performed. For example, programmable circuits may execute software or firmware that causes the programmable circuits to operate in a manner defined by the commands of the software or firmware. Fixed-function circuits may execute software commands (e.g., to receive parameters or output parameters), but the types of operations performed by the fixed-function circuits are generally invariant. In some examples, one or more of the units may be separate circuit blocks (fixed-function or programmable), and in some examples, one or more of the units may be integrated circuits.

[0222] The video encoder (200) may include arithmetic logic units (ALUs), elementary function units (EFUs), digital circuits, analog circuits, and / or programmable cores formed from programmable circuits. In examples where the operations of the video encoder (200) are performed using software executed by the programmable circuits, memory (106; FIG. 1) may store instructions (e.g., object code) of the software that the video encoder (200) receives and executes, or another memory (not shown) within the video encoder (200) may store such instructions.

[0223] The video data memory (230) is configured to store received video data. The video encoder (200) may extract a picture of video data from the video data memory (230) and provide the video data to the residual generation unit (204) and the mode selection unit (202). The video data in the video data memory (230) may be raw video data to be encoded.

[0224] The mode selection unit (202) includes a motion estimation unit (222), a motion compensation unit (224), and an intra-prediction unit (226). The mode selection unit (202) may include additional function units to perform video prediction according to different prediction modes. For example, the mode selection unit (202) may include a palette unit, an intra-block copy unit (which may be part of the motion estimation unit (222) and / or the motion compensation unit (224)), an affine unit, a linear model (LM) unit, etc.

[0225] The mode selection unit (202) generally adjusts multiple encoding passes to test combinations of encoding parameters and rate-distortion values ​​of the results for such combinations. The encoding parameters may include partitioning of CTUs into CUs, prediction modes for CUs, transformation types for residual data of CUs, quantization parameters for residual data of CUs, etc. The mode selection unit (202) may ultimately select a combination of encoding parameters that has better rate-distortion values ​​than other tested combinations.

[0226] The video encoder (200) partitions a picture extracted from the video data memory (230) into a series of CTUs and may encapsulate one or more CTUs within a slice. The mode selection unit (202) may partition the CTUs of the picture according to a tree structure, such as the quadtree structure or QTBT structure of HEVC described above. As described above, the video encoder (200) may form one or more CUs by partitioning the CTUs according to a tree structure. Such CUs may also generally be referred to as "video blocks" or "blocks".

[0227] Generally, the mode selection unit (202) also controls its components (e.g., motion estimation unit (222), motion compensation unit (224), and intra-prediction unit (226)) to generate a prediction block for the current block (e.g., the current CU, or, in HEVC, the overlapping part of PU and TU). For the inter-prediction of the current block, the motion estimation unit (222) may perform motion search to identify one or more closely matching reference blocks from one or more reference pictures (one or more previously coded pictures stored in DPB (218). In particular, the motion estimation unit (222) may calculate a value indicating how similar a potential reference block is to the current block, for example, based on the sum of absolute differences (SAD), sum of squared differences (SSD), mean absolute difference (MAD), mean squared differences (MSD), etc. The motion estimation unit (222) may perform these calculations using sample-by-sample differences between a reference block generally considered and a current block. The motion estimation unit (222) may also identify a reference block having the lowest value resulting from these calculations, indicating the reference block that most closely matches the current block.

[0228] The motion estimation unit (222) may form one or more motion vectors (MVs) that define the positions of reference blocks in reference pictures for the position of the current block in the current picture. The motion estimation unit (222) may then provide the motion vectors to the motion compensation unit (224). For example, for unidirectional inter-prediction, the motion estimation unit (222) may provide a single motion vector, whereas for bidirectional inter-prediction, the motion estimation unit (222) may provide two motion vectors. Then, the motion compensation unit (224) may generate a prediction block using the motion vectors. For example, the motion compensation unit (224) may extract data for the reference block using the motion vectors. As another example, if the motion vector has fractional sample precision, the motion compensation unit (224) may interpolate the values ​​for the prediction block according to one or more interpolation filters. Additionally, for bidirectional inter-prediction, the motion compensation unit (224) may extract data for two reference blocks identified by individual motion vectors and may combine the extracted data, for example, through sample-by-sample averaging or weighted averaging.

[0229] As another example, for intra-prediction or intra-prediction coding, the intra-prediction unit (226) may generate a prediction block from samples adjacent to the current block. For example, for directional modes, the intra-prediction unit (226) may generally generate a prediction block by mathematically combining the values ​​of neighboring samples and populating these calculated values ​​in a direction defined across the current block. As another example, for DC mode, the intra-prediction unit (226) may calculate the average of neighboring samples for the current block and generate a prediction block containing the average of these results for each sample of the prediction block.

[0230] The mode selection unit (202) provides the prediction block to the residual generation unit (204). The residual generation unit (204) receives the raw, unencoded version of the current block from the video data memory (230) and the prediction block from the mode selection unit (202). The residual generation unit (204) calculates the sample-by-sample differences between the current block and the prediction block. The resulting sample-by-sample differences define the residual block for the current block. In some examples, the residual generation unit (204) may also determine the differences between sample values ​​in the residual block to generate the residual block using residual differential pulse code modulation (RDPCM). In some examples, the residual generation unit (204) may be formed using one or more subtractor circuits that perform binary subtraction.

[0231] In the examples where the mode selection unit (202) partitions the CUs into PUs, each PU may be associated with a luminance prediction unit and a corresponding chroma prediction unit. The video encoder (200) and the video decoder (300) may support PUs of various sizes. As indicated above, the size of the CU may represent the size of the luminance coding block of the CU, and the size of the PU may represent the size of the luminance prediction block of the PU. Assuming the size of a specific CU is 2Nx2N, the video encoder (200) may support PU sizes of 2Nx2N or NxN for intra prediction, and symmetric PU sizes of 2Nx2N, 2NxN, Nx2N, NxN, or similar for inter prediction. The video encoder (200) and video decoder (300) may also support asymmetric partitioning for PU sizes of 2NxnU, 2NxnD, nLx2N, and nRx2N for inter-prediction.

[0232] In examples where the mode selection unit (202) does not further partition the CU into PUs, each CU may be associated with a luminance coding block and a corresponding chroma coding block. As described above, the size of the CU may refer to the size of the luminance coding block of the CU. The video encoder (200) and the video decoder (300) may support CU sizes of 2Nx2N, 2NxN, or Nx2N.

[0233] For some examples, for other video coding techniques such as intra-block copy mode coding, affine mode coding, and linear model (LM) mode coding, the mode selection unit (202) generates a prediction block for the current block being encoded through individual units associated with the coding techniques. In some examples, such as palette mode coding, the mode selection unit (202) may not generate a prediction block, but instead generates syntax elements indicating a method for restoring the block based on the selected palette. In such modes, the mode selection unit (202) may provide these syntax elements to the entropy encoding unit (220) to be encoded.

[0234] As described above, the residual generation unit (204) receives video data for the current block and the corresponding prediction block. The residual generation unit (204) then generates a residual block for the current block. To generate the residual block, the residual generation unit (204) calculates the sample-by-sample differences between the prediction block and the current block.

[0235] The transformation processing unit (206) applies one or more transformations to the residual block to generate a block of transformation coefficients (referred to herein as a “transformation coefficient block”). The transformation processing unit (206) may form the transformation coefficient block by applying various transformations to the residual block. For example, the transformation processing unit (206) may apply the Discrete Cosine Transform (DCT), the Directional Transform, the Karhunen-Loeve Transform (KLT), or a conceptually similar transformation to the residual block. In some examples, the transformation processing unit (206) may perform multiple transformations on the residual block, such as a first transformation and a second transformation, such as a rotation transformation. In some examples, the transformation processing unit (206) does not apply transformations to the residual block.

[0236] The quantization unit (208) may quantize the transformation coefficients in the transformation coefficient block to generate a quantized transformation coefficient block. The quantization unit (208) may quantize the transformation coefficients of the transformation coefficient block according to the quantization parameter (QP) value associated with the current block. The video encoder (200) may adjust the degree of quantization applied to the transformation coefficient blocks associated with the current block by adjusting the QP value associated with the CU (e.g., via the mode selection unit (202)). Quantization may introduce a loss of information, and thus, the quantized transformation coefficients may have lower precision than the original transformation coefficients generated by the transformation processing unit (206).

[0237] The inverse quantization unit (210) and the inverse transform processing unit (212) may each apply inverse quantization and inverse transforms to the quantized transform factor block to restore a residual block from the transform factor block. The restoration unit (214) may generate a restored block corresponding to the current block (potentially having some degree of distortion) based on the prediction block generated by the mode selection unit (202) and the restored residual block. For example, the restoration unit (214) may generate a restored block by adding samples of the restored residual block to corresponding samples from the prediction block generated by the mode selection unit (202).

[0238] The filter unit (216) may perform one or more filter operations on the restored block. For example, the filter unit (216) may perform deblocking operations to reduce blocking artifacts along the edges of the CUs. In some examples, the operations of the filter unit (216) may be skipped.

[0239] The video encoder (200) stores the restored blocks in the DPB (218). For example, in examples where the operations of the filter unit (216) are not required, the restoration unit (214) may store the restored blocks in the DPB (218). In examples where the operations of the filter unit (216) are required, the filter unit (216) may store the filtered restored blocks in the DPB (218). The motion estimation unit (222) and the motion compensation unit (224) may take a reference picture from the DPB (218) formed from the restored (and potentially filtered) blocks and inter-predict blocks of subsequent encoded pictures. Additionally, the intra-prediction unit (226) may use the restored blocks from the DPB (218) of the current picture to intra-predict other blocks in the current picture.

[0240] Generally, the entropy encoding unit (220) may entropy encode syntax elements received from other functional components of the video encoder (200). For example, the entropy encoding unit (220) may entropy encode blocks of quantized transform factors from the quantization unit (208). As another example, the entropy encoding unit (220) may entropy encode prediction syntax elements from the mode selection unit (202) (e.g., motion information for inter-prediction or intra-mode information for intra-prediction). The entropy encoding unit (220) may perform one or more entropy encoding operations on syntax elements, which are other examples of video data, to generate entropy-encoded data. For example, the entropy encoding unit (220) may perform a context-adaptive variable-length coding (CAVLC) operation, a CABAC operation, a V2V (variable-to-variable) length coding operation, a syntax-based context-adaptive binary arithmetic coding (SBAC) operation, a probabilistic interval partitioning entropy (PIPE) coding operation, an exponential-Golomb encoding operation, or other types of entropy encoding operations on the data. In some examples, the entropy encoding unit (220) may operate in a bypass mode where syntax elements are not entropy encoded.

[0241] The video encoder (200) may output a bitstream containing entropy-encoded syntax elements necessary to restore blocks of a picture or slice. In particular, the entropy encoding unit (220) may output a bitstream.

[0242] The operations described above are described with respect to blocks. Such descriptions should be understood as operations for Luma coding blocks and / or Chroma coding blocks. As described above, in some examples, the Luma coding blocks and Chroma coding blocks are Luma and Chroma components of CU. In some examples, the Luma coding blocks and Chroma coding blocks are Luma and Chroma components of PU.

[0243] In some examples, operations performed on a luminal coding block do not need to be repeated for a chroma coding block. As one example, operations to identify the motion vector (MV) and reference picture for a luminal coding block do not need to be repeated to identify the MV and reference picture for a chroma block. Rather, the MV for a luminal coding block may be scaled to determine the MV for a chroma block, and the reference picture may be the same. As another example, the intra-prediction process may be the same for a luminal coding block and a chroma coding block.

[0244] A video encoder (200) represents an example of a device configured to encode video data, comprising a memory configured to store video data and one or more processors implemented in a circuit, wherein the one or more processors are configured to modify the horizontal component of a chroma position pointed to by a motion vector of a current block of a current picture based on chroma phase shifts in the horizontal direction. The chroma position is in a reference picture having a different resolution or a different chroma sampling type from the current picture. The one or more processors may also modify the vertical component of the chroma position based on chroma phase shifts in the vertical direction. Additionally, the one or more processors may generate chroma samples of a prediction block for the current block based on chroma samples of a reference picture corresponding to the modified chroma position. The modified chroma position is defined by the modified horizontal and vertical components of the chroma position. The one or more processors of the video encoder (200) may encode the current block based on the prediction block for the current block.

[0245] In some examples, the video encoder (200) represents an example of a video encoding device comprising a memory configured to store video data and one or more processors implemented in the circuit, wherein the one or more processors are configured to modify the horizontal component of the luminance position pointed to by the current block of the current picture based on luminance phase shifts in the horizontal direction. The reference picture has a different resolution or a different chroma sampling type from the current picture. The one or more processors may also modify the vertical component of the luminance position based on luminance phase shifts in the vertical direction. Additionally, the one or more processors may generate luminance samples of a prediction block for the current block based on luminance samples of the reference picture corresponding to the modified luminance position. The modified luminance position is defined by the modified horizontal and vertical components of the luminance position. The one or more processors of the video encoder (200) may encode the current block based on the prediction block for the current block.

[0246] FIG. 4 is a block diagram illustrating an exemplary video decoder (300) that may perform the techniques of the present disclosure. FIG. 4 is provided for illustrative purposes and is not a limitation to the techniques roughly illustrated and described in the present disclosure. For illustrative purposes, the present disclosure describes a video decoder (300) according to the techniques of VVC and HEVC. However, the techniques of the present disclosure may be performed by video coding devices configured for other video coding standards.

[0247] In the example of FIG. 4, the video decoder (300) includes a coded picture buffer (CPB) memory (320), an entropy decoding unit (302), a prediction processing unit (304), an inverse quantization unit (306), an inverse transformation processing unit (308), a restoration unit (310), a filter unit (312), and a decoded picture buffer (DPB) (314). Any or all of the CPB memory (320), the entropy decoding unit (302), the prediction processing unit (304), the inverse quantization unit (306), the inverse transformation processing unit (308), the restoration unit (310), the filter unit (312), and the DPB (314) may be implemented in one or more processors or in a processing circuit. Furthermore, the video decoder (300) may include additional or alternative processors or processing circuits to perform these and other functions.

[0248] The prediction processing unit (304) includes a motion compensation unit (316) and an intra-prediction unit (318). The prediction processing unit (304) may include additional units to perform predictions according to different prediction modes. As examples, the prediction processing unit (304) may include a palette unit, an intra-block copy unit (which may form part of the motion compensation unit (316)), an affine unit, a linear model (LM) unit, etc. In other examples, the video decoder (300) may include more, fewer, or different functional components.

[0249] The CPB memory (320) may store video data, such as an encoded video bitstream to be decoded by components of the video decoder (300). The video data stored in the CPB memory (320) may be obtained, for example, from a computer-readable medium (110) (Fig. 1). The CPB memory (320) may include a CPB that stores encoded video data (e.g., syntax elements) from the encoded video bitstream. Additionally, the CPB memory (320) may store video data other than syntax elements of the encoded picture, such as transient data representing outputs from various units of the video decoder (300). The DPB (314) generally stores decoded pictures that the video decoder (300) may output and / or use as reference video data when decoding subsequent data or pictures of the encoded video bitstream. The CPB memory (320) and DPB (314) may be formed by any of the various memory devices, such as DRAM, MRAM, RRAM, or other types of memory devices including SDRAM. The CPB memory (320) and DPB (314) may be provided by the same memory device or by separate memory devices. In various examples, the CPB memory (320) may be on-chip with respect to other components of the video decoder (300) or off-chip with respect to those components.

[0250] Additionally or alternatively, in some examples, the video decoder (300) may retrieve coded video data from memory (120) (Fig. 1). That is, the memory (120) may store data as discussed above in CPB memory (320). Likewise, the memory (120) may store instructions to be executed by the video decoder (300) when some or all of the functions of the video decoder (300) are implemented in software to be executed by the processing circuit of the video decoder (300).

[0251] The various units illustrated in FIG. 4 are illustrated to aid in understanding the operations performed by the video decoder (300). The units may be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Similar to FIG. 3, fixed-function circuits refer to circuits that provide specific functionality and are pre-configured for the operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functionality for the operations that can be performed. For example, programmable circuits may execute software or firmware that causes the programmable circuits to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuits may execute software instructions (e.g., to receive parameters or output parameters), but the types of operations performed by the fixed-function circuits are generally invariant. In some examples, one or more of the units may be separate circuit blocks (fixed-function or programmable), and in some examples, one or more of the units may be integrated circuits.

[0252] The video decoder (300) may include ALUs, EFUs, digital circuits, analog circuits, and / or programmable cores formed from programmable circuits. In examples where the operations of the video decoder (300) are performed by software running on the programmable circuits, on-chip or off-chip memory may store instructions (e.g., object code) of the software that the video decoder (300) receives and executes.

[0253] The entropy decoding unit (302) may receive encoded video data from the CPB and entropy decode the video data to reproduce syntax elements. The prediction processing unit (304), the inverse quantization unit (306), the inverse transform processing unit (308), the restoration unit (310), and the filter unit (312) may generate decoded video data based on syntax elements extracted from the bitstream.

[0254] Generally, the video decoder (300) restores the picture on a block-by-block basis. The video decoder (300) may also perform a restoration operation for each block individually (where the block currently being restored, i.e., the block being decoded, may be referred to as the “current block”).

[0255] The entropy decoding unit (302) may entropy decode not only conversion information such as quantization parameters (QP) and / or conversion mode indication(s), but also syntax elements defining the quantized conversion coefficients of the quantized conversion coefficient block. The inverse quantization unit (306) may use the QP associated with the quantized conversion coefficient block to determine the degree of quantization and, similarly, the degree of inverse quantization for the inverse quantization unit (306) to apply. The inverse quantization unit (306) may, for example, perform a bit-by-bit left-shift operation to inversely quantize the quantized conversion coefficients. By doing so, the inverse quantization unit (306) may form a conversion coefficient block containing the conversion coefficients.

[0256] After the inverse quantization unit (306) forms the transform factor block, the inverse transform processing unit (308) may apply one or more inverse transforms to the transform factor block to generate a residual block associated with the current block. For example, the inverse transform processing unit (308) may apply an inverse DCT, an inverse integer transform, an inverse Karhunen-Loeve transform (KLT), an inverse rotation transform, an inverse directional transform, or other inverse transforms to the transform factor block.

[0257] Furthermore, the prediction processing unit (304) generates a prediction block according to the prediction information syntax elements entropy-decoded by the entropy decoding unit (302). For example, if the prediction information syntax elements indicate that the current block is inter-predicted, the motion compensation unit (316) may generate a prediction block. In this case, the prediction information syntax elements may indicate a motion vector identifying the position of the reference block in the reference picture as well as the position of the current block in the current picture, as well as the reference picture in the DPB (314) from which the reference block is to be extracted. The motion compensation unit (316) may generally perform the inter-predict process in a manner substantially similar to that described for the motion compensation unit (224) (Fig. 3).

[0258] As another example, if the prediction information syntax elements indicate that the current block is intra-predicted, the intra-predict unit (318) may generate a prediction block according to the intra-predict mode indicated by the prediction information syntax elements. The intra-predict unit (318) may perform the intra-predict process in a manner substantially similar to that generally described for the intra-predict unit (226) (Fig. 3). The intra-predict unit (318) may extract data of neighboring samples for the current block from the DPB (314).

[0259] The restoration unit (310) may restore the current block using the prediction block and the residual block. For example, the restoration unit (310) may restore the current block by adding samples of the residual block to the corresponding samples of the prediction block.

[0260] The filter unit (312) may perform one or more filter operations on the restored blocks. For example, the filter unit (312) may perform deblocking operations to reduce blocking artifacts along the edges of the restored blocks. The operations of the filter unit (312) are not necessarily performed in all examples.

[0261] The video decoder (300) may store the restored blocks in the DPB (314). For example, in examples where the operations of the filter unit (312) are not performed, the restoration unit (310) may store the restored blocks in the DPB (314). In examples where the operations of the filter unit (312) are performed, the filter unit (312) may store the filtered restored blocks in the DPB (314). As discussed above, the DPB (314) may provide reference information to the prediction processing unit (304), such as samples of previously decoded pictures for subsequent motion compensation and samples of the current picture for intra-prediction. Additionally, the video decoder (300) may output pictures decoded from DPB (314) for subsequent presentation on a display device such as the display device (118) of FIG. 1.

[0262] In this way, the video decoder (300) represents an example of a video decoding device comprising a memory configured to store video data and one or more processor units implemented in circuitry, wherein the one or more processor units are configured to modify a horizontal component of a chroma position pointed to by a motion vector of a current block of a current picture of video data based on chroma phase shifts in the horizontal direction, wherein the chroma position is in a reference picture having a different resolution or a different chroma sampling type from the current picture. One or more processors may also modify a vertical component of a chroma position based on chroma phase shifts in the vertical direction. Additionally, one or more processors may generate chroma samples of a prediction block for the current block based on chroma samples of a reference picture corresponding to the modified chroma position. The modified chroma position is defined by the modified horizontal and vertical components of the chroma position. One or more processors of the video decoder (300) may decode the current block based on a prediction block for the current block.

[0263] In some examples, the video decoder (300) represents an example of a video decoding device comprising a memory configured to store video data and one or more processor units implemented in the circuit, wherein the one or more processor units are configured to modify the horizontal component of the luminance position pointed to by the current block of the current picture based on luminance phase shifts in the horizontal direction. The reference picture has a different resolution or a different chroma sampling type from the current picture. One or more processors may also modify the vertical component of the luminance position based on luminance phase shifts in the vertical direction. Additionally, one or more processors may generate luminance samples of a prediction block for the current block based on luminance samples of the reference picture corresponding to the modified luminance position. The modified luminance position is defined by the modified horizontal and vertical components of the luminance position. One or more processors of the video decoder (300) may decode the current block based on the prediction block for the current block.

[0264] FIG. 5 is a flowchart illustrating an exemplary method of encoding a current block. The current block may include a current CU. Although described in relation to a video encoder (200) (Fig. 1 and Fig. 3), it should be understood that other devices may be configured to perform a method similar to that of FIG. 5.

[0265] In this example, the video encoder (200) initially predicts the current block (350). For example, the video encoder (200) may form a predicted block for the current block. According to one or more techniques of the present disclosure, in examples where the video encoder (200) predicts the current block using inter-prediction from a reference picture having a different resolution or chroma sampling type from the current picture, the video encoder (200) may modify the horizontal component of the chroma position pointed to by the motion vector of the current block based on chroma phase shifts in the horizontal direction. The chroma position in the reference picture may have a different resolution or a different chroma sampling type from the current picture. Additionally, the video encoder (200) may modify the vertical component of the chroma position based on chroma phase shifts in the vertical direction. The video coder (200) may also generate chroma samples of a prediction block for a current block of a current picture based on chroma samples of a reference picture corresponding to a modified chroma position. The modified chroma position is defined by the modified horizontal and vertical components of the chroma position.

[0266] In some examples, the video encoder (200) may modify the horizontal component of the luminance position pointed to by the current block of the current picture based on luminance phase shifts in the horizontal direction. The reference picture may have a different resolution or a different chroma sampling type from the current picture. The video encoder (200) may also modify the vertical component of the luminance position based on luminance phase shifts in the vertical direction. Additionally, the video encoder (200) may generate luminance samples of the prediction block for the current block based on luminance samples of the reference picture corresponding to the modified luminance position. The modified luminance position is defined by the modified horizontal and vertical components of the luminance position.

[0267] The video encoder (200) may then calculate a residual block for the current block (352). To calculate the residual block, the video encoder (200) may calculate the difference between the original unencoded block and the predicted block for the current block. The video encoder (200) may then convert and quantize the transform coefficients of the residual block (354). Next, the video encoder (200) may scan the quantized transform coefficients of the residual block (356). During or after the scan, the video encoder (200) may entropy-encode the transform coefficients (358). For example, the video encoder (200) may entropy-encode the transform coefficients using CAVLC or CABAC. The video encoder (200) may then output the entropy-encoded data of the block (360).

[0268] FIG. 6 is a flowchart illustrating an exemplary method for decoding a current block of video data. The current block may include a current CU. Although described in relation to a video decoder (300) (Fig. 1 and Fig. 4), it should be understood that other devices may be configured to perform a method similar to that of FIG. 6.

[0269] The video decoder (300) may receive entropy-encoded data for the current block, such as entropy-encoded data for the transformation coefficients of the residual block corresponding to the current block and entropy-encoded prediction information (370). The video decoder (300) may entropy-decode the entropy-encoded data to determine prediction information for the current block and reproduce the transformation coefficients of the residual block (372). The video decoder (300) may predict the current block by using an intra- or inter-prediction mode, for example as indicated by the prediction information for the current block, to calculate the prediction block for the current block (374).

[0270] According to one or more techniques of the present disclosure, in examples where a video decoder (300) predicts a current block using inter-prediction from a reference picture that may have a different resolution or chroma sampling type from the current picture, the video decoder (300) may modify the horizontal component of the chroma position pointed to by the motion vector of the current block based on chroma phase shifts in the horizontal direction. Additionally, the video decoder (300) may modify the vertical component of the chroma position based on chroma phase shifts in the vertical direction. The video decoder (300) may also generate chroma samples of the predicted block for the current block of the current picture based on chroma samples of the reference picture corresponding to the modified chroma position. The modified chroma position is defined by the modified horizontal and vertical components of the chroma position.

[0271] In some examples, the video decoder (300) may modify the horizontal component of the luminance position pointed to by the current block of the current picture based on luminance phase shifts in the horizontal direction. The reference picture may have a different resolution or a different chroma sampling type from the current picture. The video decoder (300) may also modify the vertical component of the luminance position based on luminance phase shifts in the vertical direction. Additionally, the video decoder (300) may generate luminance samples of the prediction block for the current block based on the luminance samples of the reference picture corresponding to the modified luminance position. The modified luminance position is defined by the modified horizontal and vertical components of the luminance position.

[0272] The video decoder (300) may backscan the generated transform coefficients to generate a block of quantized transform coefficients (376). The video decoder (300) may then inversely quantize and inversely transform the transform coefficients to generate a residual block (378). The video decoder (300) may decode the current block by combining the prediction block and the residual block (380).

[0273] FIG. 7 is a flowchart illustrating an exemplary method of coding video data according to one or more techniques of the present disclosure. The method of FIG. 7 may be performed by a video coder, such as a video encoder (200) or a video decoder (300).

[0274] In the example of FIG. 7, the video coder may modify the horizontal component of the chroma position pointed to by the motion vector of the current block of the current picture of the video data based on chroma phase shifts in the horizontal direction (400). Additionally, the video coder may modify the vertical component of the chroma position based on chroma phase shifts in the vertical direction (402). In some examples, the chroma phase shifts in the horizontal direction and the chroma phase shifts in the vertical direction are signaled in a parameter set. In some examples, the video coder may modify the horizontal and vertical components of the chroma position during the motion compensation process.

[0275] In this example, the reference picture and the current picture have different resolutions or chroma sampling types / phase shifts. For example, as an example of a reference picture and a current picture having different resolutions, the chroma samples of the reference picture may be formatted according to the 4:2:0 chroma format, and the chroma samples of the current picture may be formatted according to the 4:4:4 chroma format. As an example of a reference picture and a current picture having chroma sampling types, the chroma samples of the reference picture may be type 0, and the chroma samples of the current picture may be type 1. For example, chroma phase shifts in the horizontal direction and chroma phase shifts in the vertical direction are signaled in the parameter set.

[0276] In some examples, to modify the horizontal and vertical components of the motion vector, the video coder may determine the initial horizontal component of the chroma position. In the example in Table 2, the initial horizontal component of the chroma position may be equal to ( ( ( xSb / SubWidthC << 5 ) + mvLX

[0000] ) * hori_scale_fp ). Additionally, the video coder may determine a horizontal additive value (e.g., addX) based on the chroma phase shifts in the horizontal direction. The video coder then obtains the modified horizontal component of the chroma position (e.g., refx) by adding the initial horizontal component of the chroma position and the horizontal additive value. C It may also determine ). In this way, the video coder may determine the horizontal component of the motion vector representing the actual position in the reference picture for generating the prediction block by effectively modifying the horizontal component of the initial motion vector (mvLX) of the current block. Similarly, to modify the vertical component of the chroma position, the video coder may determine the initial vertical component of the chroma position pointed to by the motion vector. In the example in Table 2, the initial vertical component of the chroma position may be equal to ( ( ( ySb / SubHeightC << 5 ) + mvLX

[0001] ) * vert_scale_fp ). Additionally, the video coder may determine a vertical additive value (e.g., addY) based on the chroma phase shifts in the vertical direction. The video coder then obtains the modified vertical component of the chroma position (e.g., refy) by adding the initial vertical component of the chroma position and the vertical additive value. C) may also be determined. In this way, the video coder may determine the vertical component of the motion vector representing the actual position in the reference picture for generating the prediction block by effectively modifying the vertical component of the initial motion vector (mvLX) of the current block. In some examples, to set the horizontal additive value, the video coder may multiply the chroma phase shifts in the horizontal direction by a value equal to the horizontal scaling ratio minus (1 << 14), and setting the vertical additive value involves multiplying the chroma phase shifts in the vertical direction by a value equal to the vertical scaling ratio minus (1 << 14) (e.g., as illustrated in the examples of Tables 3 through 8).

[0277] As described in the previous paragraph, the video coder may also determine a horizontal additive value (e.g., addX) and a vertical additive value (e.g., addY). In some examples, to determine the horizontal additive value, the video coder may determine whether the horizontal scaling ratio is equal to a 1x ratio, whether the vertical scaling ratio is equal to a 1x ratio, whether the chroma format indicator (e.g., chroma_format_idc) indicates that the reference picture is solid, and whether the reference picture is coded using a 4:4:4 chroma format. In these examples, based on the fact that the horizontal scaling ratio is not equal to a 1x ratio and that the chroma format indicator indicates that the reference picture is not solid and is not coded using a 4:4:4 chroma format, the video coder may set the horizontal additive value based on chroma phase shifts in the horizontal direction. Similarly, based on the fact that the vertical scaling ratio is not equal to the 1x ratio and that the chroma format indicator indicates that the reference picture is not solid, the reference picture is not coded using the 4:4:4 chroma format, and the reference picture is not coded using the 4:2:2 chroma format, a vertical additive value is set based on chroma phase shifts in the vertical direction. Thus, chroma phase shift information may be considered only when resampling is applied, that is, hori_scale_fp or vert_scale_fp differs from the 1x ratio, and the techniques of the present disclosure are not applied to vertical filtering for the 4:2:2 format, or to both horizontal and vertical filtering for the 4:4:4 format or the solid format.

[0278] Additionally, the video coder may generate chroma samples of a prediction block for a current block of a current picture based on chroma samples of a reference picture corresponding to a position in the reference picture indicated by a motion vector (404). For example, the video coder may apply an interpolation filter to samples of the reference picture to interpolate samples of the prediction block for a position indicated by a motion vector, and similarly interpolate samples of the prediction block having offsets for samples of the reference picture corresponding to an offset of a position indicated by a motion vector.

[0279] The video coder may code the current block based on the prediction block for the current block (406). For example, in examples where the video coder is a video encoder (e.g., video encoder (200)), coding may refer to encoding, and coding the current block may include generating chroma residual samples for the current block based on the differences between the chroma samples of the prediction block for the current block and the corresponding chroma samples of the current block. In examples where the video coder is a video decoder (e.g., video decoder (300)), coding may refer to decoding, and coding the current block may include restoring the chroma samples of the current block by adding the chroma samples of the prediction block for the current block and the corresponding chroma residual samples of the current block.

[0280] The following is a non-limiting list of embodiments according to one or more techniques of the present disclosure.

[0281] Example 1. A method for coding video data comprises: determining a horizontal component of a motion vector indicating a position in a reference picture based on chroma phase shifts in a horizontal direction, wherein the reference picture and the current picture have different resolutions; determining a vertical component of a motion vector based on chroma phase shifts in a vertical direction; generating chroma samples of a prediction block for a current block of a current picture based on chroma samples of a reference picture corresponding to a position in a reference picture indicated by the motion vector; and coding a current block based on the prediction block for the current block.

[0282] Example 2. In the method of Example 1, chroma phase shifts in the horizontal direction and chroma phase shifts in the vertical direction are signaled in a parameter set.

[0283] Example 3. In the method of any one of Examples 1 to 2, coding includes decoding.

[0284] Example 4. In the method of Example 4, coding the current block includes restoring the chroma samples of the current block by adding the chroma samples of the prediction block for the current block and the corresponding chroma residual samples of the current block.

[0285] Example 5. In the method of any one of Examples 1 to 2, coding includes encoding.

[0286] Example 6. In the method of Example 5, coding the current block includes generating chroma residual samples for the current block based on the differences between the chroma samples of the prediction block for the current block and the corresponding chroma samples of the current block.

[0287] Example 7. A method for coding video data comprises: determining a horizontal component of a motion vector indicating a position in a reference picture based on luminance phase shifts in a horizontal direction, wherein the reference picture and the current picture have different resolutions; determining a vertical component of a motion vector based on luminance phase shifts in a vertical direction; generating luminance samples of a prediction block for a current block of a current picture based on luminance samples of a reference picture corresponding to a position in a reference picture indicated by the motion vector; and coding a current block based on the prediction block for a current block.

[0288] Example 8. In the method of Example 7, the luminance phase shifts in the horizontal direction and the luminance phase shifts in the vertical direction are signaled in the parameter set.

[0289] Example 9. In the method of any of Examples 7 to 8, coding includes decoding.

[0290] Example 10. In the method of Example 9, coding the current block includes restoring the luminance samples of the current block by adding the luminance samples of the prediction block for the current block and the corresponding luminance residual samples of the current block.

[0291] Example 11. In the method of any of Examples 7 to 8, coding includes encoding.

[0292] Example 12. In the method of Example 11, coding the current block includes generating luminous residual samples for the current block based on the differences between the luminous samples of the prediction block for the current block and the corresponding luminous samples of the current block.

[0293] Example 13. A device for coding video data, wherein the device comprises one or more means for performing the method of any one of Examples 1 to 12.

[0294] Example 14. In the device of Example 13, the one or more means include one or more processors implemented in a circuit.

[0295] Example 15. A device of any of Examples 13 and 14 further comprises a memory for storing video data.

[0296] Example 16. A device of any of Examples 13 to 15 further comprises a display configured to display decoded video data.

[0297] Example 17. A device of any of Examples 13 to 16 comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

[0298] Example 18. As a device of any of Examples 13 to 17, the device comprises a video decoder.

[0299] Example 19. As a device of any of Examples 13 to 18, the device comprises a video encoder.

[0300] Example 20. A computer-readable storage medium storing instructions that, when executed, cause one or more processors to perform the method of any of Examples 1 through 12.

[0301] It should be recognized that, depending on the example, any specific operations or events of the techniques described herein may be performed in different sequences and may be added, merged, or removed in their entirety (e.g., not all described operations or events are essential for the implementation of the techniques). Furthermore, in certain examples, acts or events may be performed simultaneously rather than sequentially, for example, through multi-threaded processing, interrupt processing, or multiple processors.

[0302] In one or more examples, the described functions may be implemented in hardware, software, firmware, or any combination thereof. Where implemented in software, the functions may be stored as one or more instructions or code on a computer-readable medium or transmitted therethrough, or executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media corresponding to media of the same type as data storage media, or communication media including any medium that facilitates the transmission of a computer program from one place to another, for example, according to a communication protocol. In this way, computer-readable media may generally correspond to (1) a non-transient type of computer-readable storage media or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to extract instructions, code and / or data structures for the implementation of the technologies described in this disclosure. A computer program product may include computer-readable media.

[0303] As an example, not a limitation, these computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and can be accessed by a computer. Additionally, any connection is appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of the medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but instead relate to non-transient, tangible storage media. As used herein, disks and discs include compact discs (CDs), laser discs, optical discs, digital multi-purpose discs (DVDs), floppy discs, and Blu-ray discs, wherein disks typically reproduce data magnetically, but discs reproduce data optically using lasers. The above combinations should also be included within the range of computer-readable media.

[0304] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Accordingly, as used herein, the terms “processor” and “processing circuit” may refer to any of the structures described above or any other structure suitable for implementing the techniques described herein. Additionally, in some embodiments, the functionality described herein may be provided within dedicated hardware and / or software modules integrated in codecs configured for or combined with encoding and decoding. Furthermore, the techniques may be fully implemented in one or more circuits or logic elements.

[0305] The techniques of the present disclosure may be implemented in a wide range of devices or apparatus, including wireless handsets, integrated circuits (ICs), or sets of ICs (e.g., chip sets). Various components, modules, or units are described in the present disclosure to highlight functional aspects of devices configured to perform the disclosed techniques, but implementation by different hardware units is not required. Rather, as described above, various units may be combined in a codec hardware unit or provided by a set of interoperable hardware units including one or more processors as described above, together with suitable software and / or firmware.

[0306] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

Claim 1 A method for coding video data, comprising the step of determining a modified horizontal component of a chroma position pointed to by a motion vector of a current block of a current picture of the video data based on chroma phase shifts in the horizontal direction, wherein the chroma position is in a reference picture having a different resolution or a different chroma sampling type from the current picture, and the step of determining the modified horizontal component of the chroma position comprises: the step of determining an initial horizontal component of the chroma position; the step of determining a horizontal additive value based on the chroma phase shifts in the horizontal direction; and the step of determining the modified horizontal component of the chroma position by adding the initial horizontal component of the chroma position and the horizontal additive value; and the step of determining a modified vertical component of the chroma position based on chroma phase shifts in the vertical direction, wherein the step of determining the modified vertical component of the chroma position comprises: the step of determining an initial vertical component of the chroma position; and the step of determining a vertical additive value based on the chroma phase shifts in the vertical direction. A method for coding video data, comprising: a step of determining a modified vertical component of a chroma position by adding the initial vertical component of the chroma position and the vertical additive value; a step of generating chroma samples of a prediction block for the current block based on chroma samples of the reference picture corresponding to the modified chroma position, wherein the modified chroma position is defined by the modified horizontal and vertical components of the chroma position; and a step of coding the current block based on the prediction block for the current block. Claim 2 A method for coding video data according to claim 1, wherein the chroma phase shifts in the horizontal direction and the chroma phase shifts in the vertical direction are signaled in a parameter set. Claim 3 delete Claim 4 A method for coding video data according to claim 1, wherein the step of determining the horizontal additive value includes setting the horizontal additive value based on the chroma phase shifts in the horizontal direction based on the fact that the horizontal scaling ratio is not equal to the 1x ratio and that the chroma format indicator indicates that the reference picture is not monochrome and is not coded using the 4:4:4 chroma format, and the step of determining the vertical additive value includes setting the vertical additive value based on the chroma phase shifts in the vertical direction based on the fact that the vertical scaling ratio is not equal to the 1x ratio and that the chroma format indicator indicates that the reference picture is not monochrome and is not coded using the 4:4:4 chroma format and is not coded using the 4:2:2 chroma format. Claim 5 A method for coding video data according to claim 4, wherein the step of setting the horizontal additive value includes the step of multiplying the chroma phase shifts in the horizontal direction by a value equal to the horizontal scaling ratio minus (1 << 14), and the step of setting the vertical additive value includes the step of multiplying the chroma phase shifts in the vertical direction by a value equal to the vertical scaling ratio minus (1 << 14). Claim 6 A method for coding video data according to claim 1, wherein the coding step includes a decoding step. Claim 7 A method for coding video data according to claim 6, wherein the step of coding the current block comprises the step of restoring the chroma samples of the current block by adding the chroma samples of the prediction block for the current block and the corresponding chroma residual samples of the current block. Claim 8 A method for coding video data according to claim 1, wherein the coding step includes an encoding step. Claim 9 A method for coding video data according to claim 8, wherein the step of coding the current block comprises the step of generating chroma residual samples for the current block based on the differences between the chroma samples of the prediction block for the current block and the corresponding chroma samples of the current block. Claim 10 A method for coding video data according to claim 1, wherein the step of determining the modified horizontal component of the chroma position includes the step of determining the modified horizontal component of the chroma position during a motion compensation process, and the step of determining the modified vertical component of the chroma position includes the step of determining the modified vertical component of the chroma position during the motion compensation process. Claim 11 A device for coding video data, comprising: a memory configured to store said video data; and one or more processors implemented in a circuit portion, wherein the one or more processors determine a modified horizontal component of a chroma position pointed to by a motion vector of a current block of a current picture of said video data based on chroma phase shifts in a horizontal direction, wherein the chroma position is in a reference picture and the chroma position is in a reference picture having a different resolution or a different chroma sampling type from said current picture, and wherein the one or more processors determine an initial horizontal component of said chroma position pointed to by the motion vector as part of determining the modified horizontal component of said chroma position; determine a horizontal additive value based on said chroma phase shifts in the horizontal direction; and determine the modified horizontal component of said chroma position by adding the initial horizontal component of said chroma position and the horizontal additive value. Determining a modified vertical component of the chroma position based on chroma phase shifts in the vertical direction, wherein the one or more processors determine an initial vertical component of the chroma position as part of determining the modified vertical component of the chroma position; determine a vertical additive value based on the chroma phase shifts in the vertical direction; and determine the modified vertical component of the chroma position by adding the initial vertical component of the chroma position and the vertical additive value;A device for coding video data, configured to generate chroma samples of a prediction block for a current block based on chroma samples of a reference picture corresponding to a modified chroma position, wherein the modified chroma position generates chroma samples of the prediction block defined by modified horizontal and vertical components of the chroma position; and to code the current block based on the prediction block for the current block. Claim 12 A device for coding video data, wherein the chroma phase shifts in the horizontal direction and the chroma phase shifts in the vertical direction are signaled in a parameter set. Claim 13 delete Claim 14 A device for coding video data, wherein, in part of determining the horizontal additive value, the one or more processors are configured to set the horizontal additive value based on the chroma phase shifts in the horizontal direction, based on the fact that the horizontal scaling ratio is not equal to the 1x ratio and the chroma format indicator indicates that the reference picture is not monochrome and is not coded using the 4:4:4 chroma format; and the one or more processors are configured to set the vertical additive value based on the chroma phase shifts in the vertical direction, based on the fact that the vertical scaling ratio is not equal to the 1x ratio and the chroma format indicator indicates that the reference picture is not monochrome and is not coded using the 4:4:4 chroma format and is not coded using the 4:2:2 chroma format. Claim 15 A device for coding video data, wherein, in part of claim 14, the one or more processors are configured to multiply the chroma phase shifts in the horizontal direction by a value equal to the horizontal scaling ratio minus (1 << 14) as part of setting the horizontal additive value, and the one or more processors are configured to multiply the chroma phase shifts in the vertical direction by a value equal to the vertical scaling ratio minus (1 << 14) as part of setting the vertical additive value. Claim 16 In claim 11, the device is a device for encoding video data, comprising a video decoder. Claim 17 A device for coding video data, wherein the video decoder is configured to restore the chroma samples of the current block by adding the chroma samples of the prediction block for the current block and the corresponding chroma residual samples of the current block as part of coding the current block. Claim 18 In claim 11, the device is a device for coding video data, comprising a video encoder. Claim 19 A device for coding video data, wherein the video encoder is configured to generate chroma residual samples for the current block based on the differences between the chroma samples of the prediction block for the current block and the corresponding chroma samples of the current block as part of coding the current block. Claim 20 A device for coding video data according to claim 11, further comprising a display configured to display decoded video data. Claim 21 In claim 11, the device is a device for encoding video data, comprising one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box. Claim 22 A device for coding video data, wherein, in claim 11, the one or more processors are configured to determine the modified horizontal component of the chroma position during a motion compensation process, and the one or more processors are configured to determine the modified vertical component of the chroma position during a motion compensation process. Claim 23 A non-transient computer-readable storage medium in which instructions are stored, wherein the instructions, when executed, cause one or more processors to determine a modified horizontal component of the chroma position of a motion vector of a current block of a current picture of video data based on chroma phase shifts in the horizontal direction, wherein the chroma position is in a reference picture having a different resolution or a different chroma sampling type from the current picture, and the instructions, when executed, cause one or more processors to determine the modified horizontal component of the chroma position, cause one or more processors to determine an initial horizontal component of the chroma position; and cause a horizontal additive value based on the chroma phase shifts in the horizontal direction; The method comprises instructions for determining the modified horizontal component of the chroma position by adding the initial horizontal component of the chroma position and the horizontal additive value; and for determining the modified vertical component of the motion vector based on chroma phase shifts in the vertical direction, wherein the instructions for causing the one or more processors to determine the modified vertical component of the chroma position, when executed, cause the one or more processors to determine the initial vertical component of the chroma position; determine the vertical additive value based on the chroma phase shifts in the vertical direction; and for determining the modified vertical component of the motion vector by adding the initial vertical component of the chroma position and the vertical additive value;A non-transient computer-readable storage medium that generates chroma samples of a prediction block for a current block of a current picture based on chroma samples of a reference picture corresponding to a position in the reference picture indicated by the motion vector; and codes the current block based on the prediction block for the current block.

Images