Applying filtering to prediction in video coding

Abstract

A video decoder may be configured to determine a reference block of a current block of a current picture of video data, wherein the reference block is in the current picture; determining a prediction block based on the reference block; determining whether to apply filtering to the prediction block; comparing a template of the reference block with a template of the current block based on determining that filtering is to be applied to the prediction block, and filtering the prediction block based on the comparison of the template of the reference block with the template of the current block to determine a filtered prediction block; decoding the current block based on the filtered prediction block to determine a decoded version of the current block; and outputting a decoded picture comprising the video data of the decoded version of the current block.

Description

[0001] This application claims priority to U.S. Patent Application No. 18 / 404,658, filed on January 4, 2024, U.S. Provisional Patent Application No. 63 / 478,657, filed on January 5, 2023, U.S. Provisional Patent Application No. 63 / 496,278, filed on April 14, 2023, U.S. Provisional Patent Application No. 63 / 509,207, filed on June 20, 2023, and U.S. Provisional Patent Application No. 63 / 511,134, filed on June 29, 2023, the entire contents of each of which are incorporated herein by reference. U.S. Patent Application No. 18 / 404,658, filed on January 4, 2024, claims the benefit of U.S. Provisional Patent Application No. 63 / 478,657, filed on January 5, 2023, U.S. Provisional Patent Application No. 63 / 496,278, filed on April 14, 2023, U.S. Provisional Patent Application No. 63 / 509,207, filed on June 20, 2023, and U.S. Provisional Patent Application No. 63 / 511,134, filed on June 29, 2023. Technical Field

[0002] The present disclosure relates to video encoding and video decoding. Background Art

[0003] Digital video capabilities may be incorporated into a wide range of devices, including digital televisions, digital live broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video gaming devices, video game consoles, cellular or satellite radio telephones, so-called "smartphones," video teleconferencing devices, video streaming devices, and the like. Digital video devices implement video coding techniques, such as those described in the standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 Part 10 Advanced Video Coding (AVC), ITU-T H.265 / High Efficiency Video Coding (HEVC), ITU-T H.266 / Versatile Video Coding (VVC), and extensions of such standards, as well as proprietary video codecs / formats such as AOMedia Video 1 (AV1) developed by the Alliance for Open Media. Video devices may more efficiently send, receive, encode, decode, and / or store digital video information by implementing such video coding techniques.

[0004] Video coding techniques include spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or remove redundancy inherent in video sequences. For block-based video coding, a video slice (e.g., a video frame or a portion of a video frame) may be partitioned into video blocks, which may also be referred to as coding tree units (CTUs), coding units (CUs), and / or coding nodes. Video blocks in an intra-frame coded (I) slice of a picture are encoded using spatial prediction of reference samples in neighboring blocks in the same picture. Video blocks in an inter-frame coded (P or B) slice of a picture may use spatial prediction of reference samples in neighboring blocks in the same picture or temporal prediction of reference samples in other reference pictures. A picture may be referred to as a frame, and a reference picture may be referred to as a reference frame. Summary of the Invention

[0005] The techniques of the present disclosure relate to prediction, including inter-frame prediction, intra-frame prediction, and intra-frame block copy (IBC) modes, and more specifically, to techniques for using filtering to improve the quality of prediction blocks. Unlike deblocking filtering, sample adaptive offset filtering, and adaptive loop filtering, which occur after reconstruction, the filtering techniques of the present disclosure can be applied to prediction blocks before reconstruction. By comparing a template of a reference block with a template of a current block and filtering the prediction block based on the comparison of the template of the reference block with the template of the current block to determine a filtered prediction block, the techniques of the present disclosure can produce more accurate predictions, which can result in an improved rate-distortion trade-off. For example, by using the filtering techniques of the present disclosure to produce a prediction block that more accurately matches the original block, the number of bits required to send residual data can be reduced. In addition, because the filtering is template-based, filter coefficients that may require a large amount of bit overhead to signal do not need to be included in the bitstream.

[0006] This disclosure also describes techniques for configuring a video decoder to determine whether to apply filtering to a prediction block based on the coding scenario and signaling overhead. These signaling techniques can minimize signaling overhead, for example, by making filtering conditional on block size, slice type, or other such characteristics of the block.

[0007] According to an example of the present disclosure, a method for decoding video data includes: determining a reference block for a current block of a current picture of the video data, wherein the reference block is in the current picture; determining a prediction block based on the reference block; determining whether to apply filtering to the prediction block; based on determining that filtering will be applied to the prediction block: comparing a template of the reference block with a template of the current block; filtering the prediction block based on the comparison of the template of the reference block with the template of the current block to determine a filtered prediction block; decoding the current block based on the filtered prediction block to determine a decoded version of the current block; and outputting a decoded picture of the video data including the decoded version of the current block.

[0008] According to an example of the present disclosure, a device for decoding video data includes: a memory configured to store video data; one or more processors implemented in a circuit and configured to: determine a reference block for a current block of a current picture of the video data, wherein the reference block is in the current picture; determine a prediction block based on the reference block; determine whether to apply filtering to the prediction block; based on determining that filtering will be applied to the prediction block: compare a template of the reference block with a template of the current block; filter the prediction block based on the comparison of the template of the reference block with the template of the current block to determine a filtered prediction block; decode the current block based on the filtered prediction block to determine a decoded version of the current block; and output a decoded picture of the video data including the decoded version of the current block.

[0009] A computer-readable storage medium stores instructions that, when executed by one or more processors, cause the one or more processors to: determine a reference block for a current block of a current picture of video data, wherein the reference block is in the current picture; determine a prediction block based on the reference block; determine whether to apply filtering to the prediction block; based on determining that filtering is to be applied to the prediction block: compare a template of the reference block with a template of the current block; filter the prediction block based on the comparison of the template of the reference block with the template of the current block to determine a filtered prediction block; decode the current block based on the filtered prediction block to determine a decoded version of the current block; and output a decoded picture of the video data including the decoded version of the current block.

[0010] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a block diagram illustrating an example video encoding and decoding system that may perform the techniques of this disclosure.

[0012] Figures 2A to 2D Illustration of the reference area used in the intra block copy (IBC) mode.

[0013] Figure 3 An example of a reference region used for coding a coding tree unit is shown.

[0014] Figure 4A and Figure 4B Examples of BV adjustments are shown for horizontal flipping and vertical flipping, respectively.

[0015] Figure 5A and Figure 5B Shown are examples of templates that may be used in conjunction with the techniques of this disclosure.

[0016] Figure 6is a conceptual diagram illustrating the spatial portion of the filter.

[0017] Figure 7 A conceptual diagram illustrating intra-template matching prediction (intra-TMP) filtering.

[0018] Figure 8 is a conceptual diagram illustrating a reference template and a current template for an IBC-coded block according to the techniques of this disclosure.

[0019] Figure 9 is a conceptual diagram illustrating reference templates, current templates, and evaluation templates for an IBC-coded block, according to the techniques of this disclosure.

[0020] Figure 10 is a block diagram illustrating an example video encoder that may perform the techniques of this disclosure.

[0021] Figure 11 is a block diagram illustrating an example video decoder that may perform the techniques of this disclosure.

[0022] Figure 12 is a flowchart illustrating an example process for encoding a current block according to techniques of this disclosure.

[0023] Figure 13 is a flowchart illustrating an example process for decoding a current block according to techniques of this disclosure.

[0024] Figure 14 is a flowchart illustrating an example process for decoding a current block according to techniques of this disclosure. DETAILED DESCRIPTION

[0025] Video coding (e.g., video encoding and / or video decoding) typically involves predicting a block of video data from a previously coded block of video data in the same picture (e.g., intra-frame prediction or intra-block copying (IBC)) or a previously coded block of video data in a different picture (e.g., inter-frame prediction). In some cases, the video encoder also calculates residual data by comparing the predicted block to the original block. Thus, the residual data represents the difference between the predicted block and the original block. To reduce the number of bits required to signal the residual data, the video encoder transforms and quantizes the residual data and signals the transformed and quantized residual data in the coded bitstream. The compression achieved by the transform and quantization process can be lossy, meaning that the transform and quantization process can introduce distortion into the decoded video data.

[0026] The video decoder decodes the residual data and adds the residual data to the prediction block to produce a reconstructed video block that more closely matches the original video block than the prediction block alone. Due to losses introduced by the transformation and quantization of the residual data, the first reconstructed block may have distortion or artifacts. A common type of artifact or distortion is called blocking, in which the boundaries of the blocks used to decode the video data are visible.

[0027] To further improve the quality of the decoded video, the video decoder may perform one or more filtering operations on the reconstructed video blocks. Examples of these filtering operations include deblocking filtering, sample adaptive offset (SAO) filtering, and adaptive loop filtering (ALF). The parameters for these filtering operations may be determined by the video encoder and explicitly signaled in the coded video bitstream, or may be implicitly determined by the video decoder without explicitly signaling the parameters in the coded video bitstream.

[0028] The technology of the present disclosure relates to prediction, including inter-frame prediction, intra-frame prediction, and intra-frame block copy (IBC) modes, and more specifically, to technology for using filtering to improve the quality of prediction blocks. Unlike the deblocking filtering, SAO filtering, and ALF described above, which occur after reconstruction, this filtering can be applied to the prediction block before reconstruction. By comparing the template of a reference block with the template of the current block and filtering the prediction block based on the comparison of the template of the reference block with the template of the current block to determine the filtered prediction block, the technology of the present disclosure can produce more accurate predictions, which can lead to an improved rate-distortion trade-off. For example, by using the filtering of the present disclosure to produce a prediction block that more accurately matches the original block, the amount of bits required to send residual data can be reduced. In addition, because the filtering is template-based, the filter coefficients that may require a large amount of bit overhead to signal do not need to be included in the bitstream.

[0029] This disclosure also describes techniques for configuring a video decoder to determine whether to apply filtering to a prediction block based on the coding scenario and signaling overhead. These signaling techniques can minimize signaling overhead, for example, by making filtering conditional on block size, slice type, or other such characteristics of the block.

[0030] Figure 1 is a block diagram illustrating an example video encoding and decoding system 100 that can implement the techniques of this disclosure. The techniques of this disclosure generally relate to transcoding (encoding and / or decoding) video data. Generally speaking, video data includes any data used to process video. Thus, video data can include raw, unencoded video, encoded video, decoded (e.g., reconstructed) video, and video metadata, such as signaling data.

[0031] like Figure 1As shown in , in this example, system 100 includes a source device 102 that provides encoded video data to be decoded and displayed by a destination device 116. Specifically, source device 102 provides the video data to destination device 116 via computer-readable medium 110. Source device 102 and destination device 116 can be or include any of a wide range of devices, such as a desktop computer, a notebook (i.e., laptop) computer, a mobile device, a tablet computer, a set-top box, a telephone handset such as a smartphone, a television, a camera, a display device, a digital media player, a video game console, a video streaming device, a broadcast receiver device, or the like. In some cases, source device 102 and destination device 116 can be equipped for wireless communication and, therefore, can be referred to as wireless communication devices.

[0032] exist Figure 1 In the example of FIG, source device 102 includes a video source 104, a memory 106, a video encoder 200, and an output interface 108. Destination device 116 includes an input interface 122, a video decoder 300, the memory 120, and a display device 118. According to the present disclosure, the video encoder 200 of source device 102 and the video decoder 300 of destination device 116 can be configured to apply the techniques for intra-block copying described herein. Thus, source device 102 represents an example of a video encoding device, while destination device 116 represents an example of a video decoding device. In other examples, the source device and destination device may include other components or arrangements. For example, source device 102 may receive video data from an external video source (e.g., an external camera). Similarly, destination device 116 may interface with an external display device rather than including an integrated display device.

[0033] like Figure 1 The illustrated system 100 is merely an example. In general, any digital video encoding and / or decoding device can implement the techniques for intra-block copying described herein. Source device 102 and destination device 116 are merely examples of such decoding devices that generate coded video data for transmission to destination device 116. This disclosure refers to a "coding" device as a device that performs decoding (encoding and / or decoding) of data. Thus, video encoder 200 and video decoder 300 represent examples of decoding devices, specifically, a video encoder and a video decoder, respectively. In some examples, source device 102 and destination device 116 can operate in a substantially symmetrical manner, such that each includes video encoding and decoding components. Thus, system 100 can support one-way or two-way video transmission between source device 102 and destination device 116, for example, for video streaming, video playback, video broadcasting, or video telephony.

[0034] Generally, video source 104 represents a source of video data (i.e., raw, unencoded video data) and provides a series of consecutive frames of video data (also referred to as "frames") to video encoder 200, which encodes the frame data. Video source 104 of source device 102 may include a video capture device (e.g., a camera), a video archive containing previously captured raw video, and / or a video feed interface for receiving video from a video content provider. Alternatively, video source 104 may generate computer graphics-based data as the source video, or a combination of live video, archived video, and computer-generated video. In each case, video encoder 200 encodes captured, pre-captured, or computer-generated video data. Video encoder 200 may rearrange the frames from the order in which they were received (sometimes referred to as "display order") into a decoding order for decoding. Video encoder 200 may generate a bitstream comprising the encoded video data. Source device 102 may then output the encoded video data to computer-readable medium 110 via output interface 108 for receipt and / or retrieval by, for example, input interface 122 of destination device 116.

[0035] Memory 106 of source device 102 and memory 120 of destination device 116 represent general purpose memory. In some examples, memories 106 and 120 can store raw video data, e.g., raw video from video source 104 and raw decoded video data from video decoder 300. Additionally or alternatively, memories 106 and 120 can store software instructions executable by, for example, video encoder 200 and video decoder 300, respectively. Although memory 106 and memory 120 are shown separately from video encoder 200 and video decoder 300 in this example, it should be understood that video encoder 200 and video decoder 300 can also include internal memory for functionally similar or equivalent purposes. Furthermore, memories 106 and 120 can store encoded video data, e.g., encoded video data output from video encoder 200 and input to video decoder 300. In some examples, portions of memories 106 and 120 can be allocated as one or more video buffers, e.g., to store raw, decoded, and / or encoded video data.

[0036] Computer-readable medium 110 may represent any type of medium or device capable of transmitting encoded video data from source device 102 to destination device 116. In one example, computer-readable medium 110 represents a communication medium that enables source device 102 to transmit encoded video data directly to destination device 116 in real time, for example, via a radio frequency network or a computer-based network. Output interface 108 may modulate a transmit signal comprising encoded video data, and input interface 122 may demodulate a received transmit signal, in accordance with a communication standard (e.g., a wireless communication protocol). The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other device that can be used to facilitate communication from source device 102 to destination device 116.

[0037] In some examples, source device 102 may output the encoded data from output interface 108 to storage device 112. Similarly, destination device 116 may access the encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray disc, DVD, CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0038] In some examples, source device 102 may output the encoded video data to file server 114B or another intermediate storage device that may store the encoded video data generated by source device 102. Destination device 116 may access the stored video data via streaming or downloading to file server 114.

[0039] The file server 114 may be any type of server device capable of storing encoded video data and sending the encoded video data to the destination device 116. The file server 114 may represent a web server (e.g., for a website), a server configured to provide file transfer protocol services (e.g., File Transfer Protocol (FTP) or File Delivery over Unidirectional Transport (FLUTE)), a content delivery network (CDN) device, a hypertext transfer protocol (HTTP) server, a Multimedia Broadcast Multicast Service (MBMS) or enhanced MBMS (eMBMS) server, and / or a network attached storage (NAS) device. The file server 114 may additionally or alternatively implement one or more HTTP streaming protocols, such as HTTP Dynamic Adaptive Streaming (DASH), HTTP Live Streaming (HLS), Real Time Streaming Protocol (RTSP), HTTP Dynamic Streaming, or the like.

[0040] Destination device 116 may access the encoded video data from file server 114 via any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both, suitable for accessing the encoded video data stored on file server 114. Input interface 122 may be configured to operate according to any one or more of the various protocols discussed above for retrieving or receiving media data from file server 114, or other such protocols for retrieving media data.

[0041] Output interface 108 and input interface 122 may represent wireless transmitters / receivers, modems, wired networking components (e.g., Ethernet cards), wireless communication components operating according to any of the various IEEE 802.11 standards, or other physical components. In examples where output interface 108 and input interface 122 include wireless components, output interface 108 and input interface 122 may be configured to transmit data (e.g., encoded video data) according to a cellular communication standard (e.g., 4G, 4G-LTE (Long Term Evolution), LTE Advanced, 5G, or the like). In some examples where output interface 108 includes a wireless transmitter, output interface 108 and input interface 122 may be configured to transmit data (e.g., encoded video data) according to other wireless standards (e.g., IEEE 802.11 specifications, IEEE 802.15 specifications (e.g., ZigBee™), Bluetooth™ standards, or the like). In some examples, source device 102 and / or destination device 116 may include corresponding system-on-chip (SoC) devices. For example, source device 102 may include a SoC device for performing the functions attributed to video encoder 200 and / or output interface 108 , and destination device 116 may include a SoC device for performing the functions attributed to video decoder 300 and / or input interface 122 .

[0042] The techniques of this disclosure may be applied to support video decoding for any of a variety of multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, Internet streaming video transmission (e.g., Dynamic Adaptive Streaming over HTTP (DASH)), digital video encoded onto a data storage medium, decoding of digital video stored on a data storage medium, or other applications.

[0043] The input interface 122 of the destination device 116 receives an encoded video bitstream from the computer-readable medium 110 (e.g., a communication medium, storage device 112, file server 114, or the like). The encoded video bitstream may include signaling information defined by the video encoder 200 and used by the video decoder 300, such as syntax elements with values describing characteristics and / or processing of video blocks or other coded units (e.g., slices, pictures, groups of pictures, sequences, etc.). The display device 118 displays decoded pictures of the decoded video data to a user. The display device 118 may represent any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0044] although Figure 1Not shown, but in some examples, the video encoder 200 and the video decoder 300 can each be integrated with an audio encoder and / or audio decoder, and can include appropriate MUX-DEMUX units or other hardware and / or software to handle multiplexed streams including both audio and video in a common data stream.

[0045] The video encoder 200 and the video decoder 300 can each be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the techniques are partially implemented in software, a device may store instructions for the software on a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the techniques of this disclosure. Each of the video encoder 200 and the video decoder 300 can be included in one or more encoders or decoders, either of which can be integrated as part of a combined encoder / decoder (CODEC) in the corresponding device. A device that includes the video encoder 200 and / or the video decoder 300 can implement the video encoder 200 and / or the video decoder 300 in processing circuitry such as an integrated circuit and / or a microprocessor. Such a device can be a wireless communication device, such as a cellular phone, or any other type of device described herein.

[0046] The video encoder 200 and the video decoder 300 may operate in accordance with a video coding standard such as ITU-T H.265, also known as High Efficiency Video Coding (HEVC), or extensions thereof, such as multi-view and / or scalable video coding extensions. Alternatively, the video encoder 200 and the video decoder 300 may operate in accordance with other proprietary or industry standards such as ITU-T H.266, also known as Versatile Video Coding (VVC). In other examples, the video encoder 200 and the video decoder 300 may operate in accordance with a proprietary video codec / format such as AOMedia Video 1 (AV1), extensions of AV1, and / or successors to AV1 (e.g., AV2). JVET has begun exploring techniques to further improve the decoding performance of VVC. Test model reference software is also under development and is described in J. Chen, Y. Ye, S. Kim, "Algorithm description of Enhanced Compression Model 7 (ECM 7)," 28th JVET Meeting, Mainz, DE, Oct. 2022, JVET-AB2025. In other examples, the video encoder 200 and the video decoder 300 can operate according to other proprietary formats or industry standards. However, the techniques of the present disclosure are not limited to any particular decoding standard or format. In general, the video encoder 200 and the video decoder 300 can be configured to perform the techniques of the present disclosure in conjunction with any video coding technique that uses intra-block copying.

[0047] Generally, the video encoder 200 and video decoder 300 can perform block-based decoding of a picture. The term "block" generally refers to a structure containing data to be processed (e.g., encoded, decoded, or otherwise used in the encoding and / or decoding process). For example, a block may include a two-dimensional matrix of samples of luma and / or chroma data. Generally, the video encoder 200 and video decoder 300 can decode video data represented in a YUV (e.g., Y, Cb, Cr) format. That is, rather than decoding the red, green, and blue (RGB) data of the samples of a picture, the video encoder 200 and video decoder 300 can decode luma and chroma components, where the chroma components may include both red-toned and blue-toned chroma components. In some examples, the video encoder 200 converts received RGB-formatted data to a YUV representation before encoding, and the video decoder 300 converts the YUV representation to an RGB format. Alternatively, pre-processing and post-processing units (not shown) can perform these conversions.

[0048] The present disclosure may generally refer to the decoding (e.g., encoding and decoding) of a picture, including the process of encoding or decoding the data of the picture. Similarly, the present disclosure may refer to the decoding of a block of a picture, including the process of encoding or decoding the data of the block, such as prediction and / or residual coding. A coded video bitstream typically includes a series of values for syntax elements used to represent decoding decisions (e.g., coding mode) and the partitioning of the picture into blocks. Therefore, references to decoding a picture or block should generally be understood to refer to decoding the values of the syntax elements that form the picture or block.

[0049] HEVC defines various blocks, including coding units (CUs), prediction units (PUs), and transform units (TUs). According to HEVC, a video encoder (such as video encoder 200) divides a coding tree unit (CTU) into CUs according to a quadtree structure. That is, the video coder partitions the CTU and CU into four equal non-overlapping squares, and each node of the quadtree has zero or four child nodes. A node without child nodes may be referred to as a "leaf node", and the CU of such a leaf node may include one or more PUs and / or one or more TUs. The video encoder may further partition the PUs and TUs. For example, in HEVC, the residual quadtree (RQT) represents the partitioning of the TU. In HEVC, a PU represents inter-frame prediction data, and a TU represents residual data. An intra-predicted CU includes intra-frame prediction information, such as an intra-frame mode indication.

[0050] As another example, the video encoder 200 and the video decoder 300 can be configured to operate according to VVC. According to VVC, a video encoder (e.g., the video encoder 200) partitions a picture into multiple CTUs. The video encoder 200 can partition the CTUs according to a tree structure (e.g., a quadtree-binary tree (QTBT) structure or a multi-type tree (MTT) structure). The QTBT structure removes the concept of multiple partition types, such as the separation between CU, PU, and TU of HEVC. The QTBT structure includes two levels: a first level partitioned according to quadtree partitioning and a second level partitioned according to binary tree partitioning. The root node of the QTBT structure corresponds to the CTU. The leaf nodes of the binary tree correspond to the CU.

[0051] In the MTT partitioning structure, blocks can be partitioned using quadtree (QT) partitioning, binary tree (BT) partitioning, and one or more types of ternary tree (TT) (also known as ternary tree (TT)) partitioning. A ternary tree or ternary tree partitioning is a partitioning method in which a block is split into three sub-blocks. In some examples, a ternary tree or ternary tree partitioning divides a block into three sub-blocks without dividing the original block through the center. Partition types in MTT (e.g., QT, BT, and TT) can be symmetric or asymmetric.

[0052] When operating in accordance with the AV1 codec, the video encoder 200 and video decoder 300 can be configured to decode video data in blocks. In AV1, the largest coded block that can be processed is called a superblock. In AV1, a superblock can be 128×128 luma samples or 64×64 luma samples. However, in subsequent video coding formats (e.g., AV2), superblocks may be defined by different (e.g., larger) luma sample sizes. In some examples, a superblock is the top level of a block quadtree. The video encoder 200 can also partition a superblock into smaller coded blocks. The video encoder 200 can partition superblocks and other coding blocks into smaller blocks using square or non-square partitioning. Non-square blocks can include N / 2×N, N×N / 2, N / 4×N, and N×N / 4 blocks. The video encoder 200 and video decoder 300 can perform separate prediction and transform processing on each coded block.

[0053] AV1 also defines tiles of video data. A tile is a rectangular array of superblocks that can be decoded independently of other tiles. That is, the video encoder 200 and video decoder 300 can encode and decode coded blocks within a tile, respectively, without using video data from other tiles. However, the video encoder 200 and video decoder 300 can perform filtering across tile boundaries. Tiles can be uniform or non-uniform in size. Tile-based decoding can enable parallel processing and / or multithreading for encoder and decoder implementations.

[0054] In some examples, the video encoder 200 and the video decoder 300 may use a single QTBT or MTT structure to represent each of the luma component and the chroma components, while in other examples, the video encoder 200 and the video decoder 300 may use two or more QTBT or MTT structures, such as one QTBT / MTT structure for the luma component and another QTBT / MTT structure for the two chroma components (or two QTBT / MTT structures for the respective chroma components).

[0055] The video encoder 200 and the video decoder 300 may be configured to use quadtree partitioning, QTBT partitioning, MTT partitioning, superblock partitioning, or other partitioning structures.

[0056] In some examples, a CTU includes a coding tree block (CTB) of luma samples, two corresponding CTBs of chroma samples for a picture having three sample arrays, or a CTB of samples for a monochrome picture or a picture coded using three separate color planes and syntax structures for coding samples. A CTB can be an N×N block of samples with some value of N, such that the division of a component into a CTB is a partition. A component is an array or a single sample from one of the three arrays comprising a picture in 4:2:0, 4:2:2, or 4:4:4 color format, or an array or a single sample of such an array comprising a picture in a monochrome format. In some examples, a coding block is an M×N block of samples with some values of M and N, such that the division of a CTB into a coding block is a partition.

[0057] Blocks (e.g., CTUs or CUs) can be grouped in a picture in various ways. As an example, a brick can refer to a rectangular area of a CTU row within a particular tile in a picture. A tile can be a rectangular area of a CTU within a particular tile column and tile row in a picture. A tile column refers to a rectangular area of a CTU with a height equal to the height of the picture and a width specified by a syntax element (e.g., in a picture parameter set). A tile row refers to a rectangular area of a CTU with a height specified by a syntax element (e.g., in a picture parameter set) and a width equal to the picture width.

[0058] In some examples, a tile may be partitioned into multiple bricks, each of which may include one or more CTU rows within the tile. A tile that is not partitioned into multiple bricks may also be referred to as a brick. However, a brick that is a true subset of a tile may not be referred to as a tile. The tiles in a picture may also be arranged in slices. A slice may be an integer number of tiles of a picture that may be exclusively included in a single network abstraction layer (NAL) unit. In some examples, a slice includes a contiguous sequence of several complete tiles or just complete bricks of one tile.

[0059] This disclosure may use "NxN" and "N by N" interchangeably to refer to the sample size of a block (e.g., a CU or other video block) in terms of vertical and horizontal dimensions, for example, 16x16 samples or 16 by 16 samples. Generally speaking, a 16x16 CU will have 16 samples in the vertical direction (y=16) and 16 samples in the horizontal direction (x=16). Similarly, an NxN CU typically has N samples in the vertical direction and N samples in the horizontal direction, where N represents a non-negative integer value. The samples in a CU may be arranged in rows and columns. Furthermore, a CU does not necessarily need to have the same number of samples in the horizontal direction as in the vertical direction. For example, a CU may include NxM samples, where M is not necessarily equal to N.

[0060] The video encoder 200 encodes the video data of the CU representing prediction and / or residual information and other information. The prediction information indicates how to predict the CU to form the prediction block of the CU. The residual information generally represents the sample-by-sample difference between the samples of the CU before encoding and the prediction block.

[0061] To predict a CU, the video encoder 200 may typically form a prediction block for the CU through inter-frame prediction or intra-frame prediction. Inter-frame prediction typically refers to predicting a CU from data of a previously decoded picture, while intra-frame prediction typically refers to predicting a CU from previously decoded data of the same picture. To perform inter-frame prediction, the video encoder 200 may use one or more motion vectors to generate a prediction block. The video encoder 200 may typically perform a motion search to identify a reference block that closely matches the CU, for example, in terms of the difference between the CU and the reference block. The video encoder 200 may calculate a difference metric using the sum of absolute differences (SAD), the sum of squared differences (SSD), the mean absolute difference (MAD), the mean squared difference (MSD), or other such difference calculations to determine whether the reference block closely matches the current CU. In some examples, the video encoder 200 may use unidirectional prediction or bidirectional prediction to predict the current CU.

[0062] Some examples of VVC also provide an affine motion compensation mode, which can be considered an inter-prediction mode. In affine motion compensation mode, the video encoder 200 can determine two or more motion vectors representing non-translational motion (such as zooming in or out, rotation, perspective motion, or other irregular motion types).

[0063] To perform intra prediction, the video encoder 200 may select an intra prediction mode to generate a prediction block. Some examples of VVC provide sixty-seven intra prediction modes, including various directional modes, as well as a planar mode and a DC mode. Typically, the video encoder 200 selects an intra prediction mode that describes neighboring samples of a current block that are samples from which the current block (e.g., a block of a CU) is predicted. Assuming that the video encoder 200 decodes CTUs and CUs in raster scan order (left to right, top to bottom), such samples may typically be above, above and to the left, or to the left of the current block in the same picture as the current block.

[0064] The video encoder 200 encodes data indicating a prediction mode for the current block. For example, for inter-frame prediction mode, the video encoder 200 may encode data indicating which of various available inter-frame prediction modes to use, as well as motion information for the corresponding mode. For example, for unidirectional or bidirectional inter-frame prediction, the video encoder 200 may encode motion vectors using Advanced Motion Vector Prediction (AMVP) or merge mode. The video encoder 200 may use a similar mode to encode motion vectors for affine motion compensation mode.

[0065] AV1 includes two general techniques for encoding and decoding coded blocks of video data. The two general techniques are intra-frame prediction (e.g., intra prediction or spatial prediction) and inter-frame prediction (e.g., inter-frame prediction or temporal prediction). In the context of AV1, when an intra-frame prediction mode is used to predict a block of a current frame of video data, the video encoder 200 and video decoder 300 do not use video data from other frames of video data. For most intra-frame prediction modes, the video encoder 200 encodes the block of the current frame based on the difference between sample values in the current block and prediction values generated from reference samples in the same frame. The video encoder 200 determines the prediction values generated from the reference samples based on the intra-frame prediction mode.

[0066] After prediction (such as intra-frame prediction or inter-frame prediction of a block), the video encoder 200 can calculate residual data for the block. The residual data (e.g., a residual block) represents the sample-by-sample difference between the block and a predicted block of the block formed using the corresponding prediction mode. The video encoder 200 can apply one or more transforms to the residual block to generate transform data in a transform domain rather than a sample domain. For example, the video encoder 200 can apply a discrete cosine transform (DCT), an integer transform, a wavelet transform, or a conceptually similar transform to the residual video data. In addition, the video encoder 200 can apply a secondary transform after the first transform, such as a mode-dependent non-separable secondary transform (MDNSST), a signal-dependent transform, a Karhunen-Loeve transform (KLT), etc. The video encoder 200 generates transform coefficients after applying one or more transforms.

[0067] As described above, after any transform that produces transform coefficients, the video encoder 200 can perform quantization of the transform coefficients. Quantization generally refers to the process by which transform coefficients are quantized to potentially reduce the amount of data used to represent the transform coefficients, thereby providing further compression. By performing this quantization process, the video encoder 200 can reduce the bit depth associated with some or all transform coefficients. For example, the video encoder 200 can round down an n-bit value to an m-bit value during quantization, where n is greater than m. In some examples, to perform quantization, the video encoder 200 can perform a bit-by-bit right shift of the value to be quantized.

[0068] After quantization, the video encoder 200 may scan the transform coefficients to produce a one-dimensional vector from the two-dimensional matrix comprising the quantized transform coefficients. The scan may be designed to place higher energy (and therefore lower frequency) transform coefficients at the front of the vector and lower energy (and therefore higher frequency) transform coefficients at the back of the vector. In some examples, the video encoder 200 may scan the quantized transform coefficients using a predefined scan order to produce a serialized vector and then entropy encode the quantized transform coefficients of the vector. In other examples, the video encoder 200 may perform adaptive scanning. After scanning the quantized transform coefficients to form a one-dimensional vector, the video encoder 200 may entropy encode the one-dimensional vector, for example, according to context-adaptive binary arithmetic coding (CABAC). The video encoder 200 may also entropy encode the values of syntax elements that describe metadata associated with the encoded video data for use by the video decoder 300 when decoding the video data.

[0069] To perform CABAC, the video encoder 200 may assign context within a context model to a symbol to be transmitted. The context may relate to, for example, whether the symbol's neighboring values are zero. The probability determination may be based on the context assigned to the symbol.

[0070] The video encoder 200 may also generate syntax data, such as block-based syntax data, picture-based syntax data, and sequence-based syntax data, to the video decoder 300, for example, in a picture header, a block header, a slice header, or other syntax data, such as a sequence parameter set (SPS), a picture parameter set (PPS), or a video parameter set (VPS). The video decoder 300 may also decode such syntax data to determine how to decode the corresponding video data.

[0071] In this way, the video encoder 200 can generate a bitstream including the encoded video data, for example, syntax elements describing the partitioning of a picture into blocks (e.g., CUs) and prediction and / or residual information for the blocks. Ultimately, the video decoder 300 can receive the bitstream and decode the encoded video data.

[0072] Generally, video decoder 300 performs processing that is inverse to that performed by video encoder 200 to decode the encoded video data of the bitstream. For example, video decoder 300 may use CABAC to decode the values of syntax elements of the bitstream in a manner substantially similar to, but inverse of, the CABAC encoding process of video encoder 200. The syntax elements may define partitioning information for partitioning a picture into CTUs and partitioning each CTU according to a corresponding partitioning structure (e.g., a QTBT structure) to define CUs of the CTU. The syntax elements may further define prediction and residual information for a block of video data (e.g., a CU).

[0073] The residual information may be represented, for example, by quantized transform coefficients. The video decoder 300 may inverse quantize and inverse transform the quantized transform coefficients of a block to regenerate a residual block for the block. The video decoder 300 uses the signaled prediction mode (intra-frame or inter-frame prediction) and related prediction information (e.g., motion information for inter-frame prediction) to form a prediction block for the block. The video decoder 300 may then combine the prediction block and the residual block (on a sample-by-sample basis) to regenerate the original block. The video decoder 300 may also perform additional processing, such as a deblocking process to reduce visual artifacts along block boundaries.

[0074] This disclosure generally relates to "signaling" certain information, such as syntax elements. The term "signaling" generally refers to the communication of values for syntax elements and / or other data used to decode encoded video data. That is, video encoder 200 can signal the values of syntax elements in a bitstream. Generally speaking, signaling refers to generating values in the bitstream. As mentioned above, source device 102 can transmit the bitstream to destination device 116 in substantially real time or in non-real time (e.g., when storing syntax elements on storage device 112 for later retrieval by destination device 116).

[0075] VVC includes an intra block copy (IBC) mode. IBC mode significantly improves the coding efficiency of screen content material. IBC mode is implemented as a block-level coding mode, and therefore block matching (BM) is performed at the encoder to find the optimal block vector (or motion vector) for each CU. Here, a block vector indicates the displacement from the current block to a reference block that has been reconstructed within the current picture. The luma block vectors of IBC-coded CUs are in integer precision. Chroma block vectors are also rounded to integer precision. When combined with adaptive motion vector resolution (AMVR), IBC mode can switch between 1-pixel and 4-pixel motion vector precision. CUs encoded with IBC are treated as a third prediction mode, distinct from intra or inter prediction modes. In VVC, IBC mode is used for CUs with a width and height of less than or equal to 64 luma samples. Multi-transform selection (MTS) and low-frequency, non-separable transform (LFNST) are disabled for IBC-coded blocks.

[0076] At the CU level, the IBC mode is signaled with a flag and can be signaled as either IBC AMVP mode or IBC skip / merge mode. In IBC skip / merge mode, the merge candidate index is used to indicate which block vectors in the list of neighboring candidate IBC-coded blocks are used to predict the current block. The merge list includes spatial, history-based motion vector prediction (HMVP) and paired candidates. In IBC AMVP mode, block vector differences are coded in the same way as motion vector differences. The block vector prediction process uses two candidates as predictors, one from the left neighbor and one from the upper neighbor (if IBC-coded). When either neighbor is not available, a default block vector can be used as the predictor. A flag is signaled to indicate the block vector predictor index.

[0077] To reduce memory consumption and decoder complexity, the IBC mode in VVC only allows the reconstructed portion of a predefined area, including the region of the current CTU and a certain region of the left CTU.

[0078] Figures 2A to 2D An example of a reference area for IBC mode is shown, where each block represents a 64×64 luma sample unit. Figures 2A to 2D In the example of , current block 130 represents a block currently being encoded. Block 132 represents an already encoded block that can be used to predict block 130 using IBC. Block 134 represents an already encoded block that is not available for predicting block 130. Block 136 represents a not yet encoded block that is not available for predicting block 130.

[0079] Depending on the location of the current coded CU within the current CTU, different blocks can be used for IBC. If the current block falls into the top left 64×64 block of the current CTU, such as Figure 2A As shown, in addition to the reconstructed samples in the current CTU, the current picture reference (CPR) mode can be used to predict the current CTU using the reference samples in the lower right 64×64 block of the left CTU. The CPR mode can also be used to predict the current block using the reference samples in the lower left 64×64 block of the left CTU and the reference samples in the upper right 64×64 block of the left CTU.

[0080] If the current block falls into the upper right 64×64 block of the current CTU, such as Figure 2B As shown, in addition to the reconstructed samples in the current CTU, if the luma position (0, 64) relative to the current CTU has not been reconstructed, the CPR mode can also be used to predict the current block using the reference samples in the lower left 64×64 block and the lower right 64×64 block of the left CTU. Otherwise, the reference samples in the lower right 64×64 block of the left CTU can be used to predict the current block.

[0081] If the current block falls into the lower left 64×64 block of the current CTU, such as Figure 2C As shown, in addition to the samples that have been reconstructed in the current CTU, if the luma position (64, 0) relative to the current CTU has not been reconstructed, the CPR mode can be used to use the reference samples in the upper right 64×64 block and the lower right 64×64 block of the left CTU to predict the current block. Otherwise, the CPR mode can also be used to use the reference samples in the lower right 64×64 block of the left CTU to predict the current block. If the current block falls into the lower right 64×64 block of the current CTU, as shown in Figure 2D As shown, the CPR mode can be used to predict the current block using only the reconstructed samples in the current CTU.

[0082] The current IBC tools in ECM have several improvements over VVC, primarily targeting screen content. For example, ECM includes IBC with Template Matching (TM-IBC). Template matching is used in both IBC Merge Mode and IBC AMVP Mode.

[0083] Compared to the IBC-TM merge list used by the regular IBC merge mode, the IBC-TM merge list is modified so that candidates are selected according to a pruning process that utilizes the motion distance between candidates, as in the regular TM merge mode. The ending zero motion implementation is replaced by the motion vectors of the left (-W, 0), top (0, -H), and top-left (-W, -H), where W is the width of the current CU and H is the height of the current CU.

[0084] In IBC-TM merge mode, the selected candidates are refined with a template matching process before rate-distortion optimization (RDO) or decoding process.IBC-TM merge mode has been made competitive with conventional IBC merge mode and signals the TM-merge flag.

[0085] In IBC-TM AMVP mode, up to three candidates are selected from the IBC-TM merge list. A template matching process is used to refine each of the three selected candidates and sort them according to the template matching cost they produce. In some examples, after sorting, only the top two are typically considered in the motion estimation process.

[0086] In the template matching refinement for both IBC-TM merge mode and AMVP mode, the IBC motion vectors are constrained to be (i) integers and (ii) within the reference region, e.g. Figures 2A to 2D. Thus, in IBC-TM Merge mode, some or all refinements can be performed at integer precision, and in IBC-TM AMVP mode, refinements can be performed at integer or 4-pixel precision, depending on the AMVR value. This refinement only accesses samples without interpolation. In both cases, the refined motion vectors and the template used in each refinement step must obey the constraints of the reference region.

[0087] The IBC in ECM includes an increase in the reference area relative to the VVC. The reference area of the IBC extends to the upper two CTU rows. Figure 3 The reference region for encoding a CTU 140 located at (m, n) is shown. Specifically, for the CTU 140 to be encoded, the reference region includes CTUs 142 with indices (m-2, n-2)…(W, n-2), (0, n-1)…(W, n-1), (0, n)…(m, n), where W represents the maximum horizontal index within the current tile, slice, or picture. When the CTU size is 256, the reference region is limited to one CTU row above. This setting ensures that IBC does not require additional memory in current ETM platforms for CTU sizes of 128B or 256. The per-sample block vector search (or local search) range is limited horizontally to [-(C<1), C>>2] and vertically to [-C, C>>2] to accommodate reference region expansion, where C represents the CTU size.

[0088] ECM also includes reordered reconstruction with inter-block coding (RR-IBC). RR-IBC mode is allowed for blocks coded with IBC. When RR-IBC is applied, samples in the reconstructed block are flipped according to the flip type of the current block. On the encoder side, the original block is flipped before motion search and residual calculation, while the predicted block is derived without flipping. On the decoder side, the reconstructed block is flipped back to restore the original block.

[0089] For RR-IBC coded blocks, two flipping processes (horizontal and vertical) are supported. A syntax flag is initially signaled for IBC AMVP coded blocks, indicating whether the reconstruction is flipped, and if so, another flag specifying the flip type is further signaled. For IBC merging, the flip type is inherited from the neighboring blocks without syntax signaling. The current block and the reference block are typically aligned horizontally or vertically, taking into account horizontal or vertical symmetry. Therefore, when horizontal flipping is applied, the vertical component of BV is not signaled and is inferred to be equal to 0. Similarly, when vertical flipping is applied, the horizontal component of BV is not signaled and is inferred to be equal to 0.

[0090] In order to better utilize the symmetry property, a flip-aware BV adjustment method is applied to refine the block vector candidates. Figure 4A An example of horizontal flipping is shown, and Figure 4BAn example of vertical flipping is shown. Figure 4A and 4B In the example, (x nbr ,y nbr ) represents the coordinates of the center sample 150 of the neighboring block 152, and (x cur ,y cur ) represent the coordinates of the center sample 154 of the current block 156. Respectively, BV nbr represents the BV 158 of the neighboring block 152, and the BV cur 156. Instead of inheriting BV 158 directly from neighboring block 152, the neighboring block 152 is decoded with horizontal flip (ie, BV cur h =2(x nbr -x cur ) BV nbr h ) by adding motion shift to BV nbr The horizontal component (denoted as BV nbr h ) to calculate BV cur Similarly, when decoding adjacent blocks with vertical flip (i.e., BV cur v =2(y nbr -y cur )+BV nbr v ) by adding motion shift to BV nbr The vertical component (denoted as BV nbr v ) to calculate BV cur The vertical component of .

[0091] It should be noted that in one example of ECM, when IBC is enabled, there is no way to selectively disable RR-IBC or Template Matching IBC, ie, these tools are automatically enabled when IBC is enabled from SPS.

[0092] The video encoder 200 and the video decoder 300 may be configured to perform local illumination compensation (LIC). In ECM-7.0, LIC is an inter-frame prediction technique that models the local illumination variation between a current block and its predicted block as a function of the local illumination variation between the current block template and a reference block template. The linear function that LIC applies to a reference block can be described as:

[0093] predVal=α*RefSample+β,

[0094] where 'predVal' is the output of the LIC operation and α and β are two parameters derived by the encoder and decoder from the templates of the current block and the inter-frame reference block.

[0095] LIC has also been proposed for IBC, which is an extension of LIC for conventional inter prediction. LIC uses a scale factor and an offset to compensate for differences in local illumination. The scale factor and offset are estimated from the surrounding templates of the current block and the surrounding templates of the reference block, so these parameters may not require additional signaling.

[0096] The video encoder 200 and video decoder 300 can be configured to perform intra-frame template matching (IntraTMP). IntraTMP is a special intra-frame prediction mode that copies the best prediction block from the reconstructed portion of the current frame. A block vector (BV) is derived from the current block to a reference block by performing template matching within a predefined search range. The block with the template most similar to the current block's template is selected. The same template matching search operation is performed on both the encoder and decoder sides, eliminating the need to signal the block vector (BV).

[0097] There has also been some research on using fractional pixel interpolation for IBC, which has been reported to improve IBC performance. Sub-pixel accuracy of 1 / 2 pixel and 1 / 4 pixel is added to utilize fractional pixel block vectors. Interpolation filters with 12 or 8 taps can be used for interpolation. When the interpolation process requires pixels outside the IBC reference region, the reference region can be filled using only the available pixels inside the reference region.

[0098] Since IBC uses the reconstructed portion of the current picture to find the block predictor for the current block, the similarity between the current block and the predicted block may be lower than in the case of inter prediction. Applying filtering to the predicted block can improve the quality of the predictor.

[0099] Meanwhile, the template matching-based technique can use the similarity between the current block and the template in the reconstructed region to determine the optimal configuration of the filter.

[0100] This disclosure describes techniques for applying filtering to predictor blocks in IBC mode, inter prediction, or intra prediction. Examples of inter prediction modes are merge mode, AMVP mode, affine mode, and the like.

[0101] As an example, the video encoder 200 and the video decoder 300 may be configured to derive the filtering process from a template associated with a current block and a reference block that may be located in the same picture or other reference pictures.

[0102] Figure 5AExamples of templates 170 and 172 are shown, which have an L shape and are adjacent to a current block 174 and a reference block 176, respectively. The templates 170 and 172 are located in a "decoding area" such that the values of the samples in the templates are available when decoding the current block 174. Figure 5A As shown in , for IBC, the reference block 176 is located in the same picture as the current block 174.

[0103] Figure 5B Examples of templates 180 and 182 are shown, which have an L shape and are adjacent to a current block 184 and a reference block 186, respectively. Template 180 is located in the 'decoding area' of the current picture 188 so that the values of the samples of template 180 are available when decoding the current block 184. Figure 5B As shown, for inter prediction, the reference block 186 is located in the reference picture 190 rather than the current picture 188 .

[0104] For intra-frame coding, there are no reference samples, as only reconstructed neighboring samples exist. Derived samples can be used as reference block samples in this description. In one example, the derived samples can be samples derived by applying intra-frame prediction to neighboring samples. A filter model can then be derived by comparing the proximity of the reconstructed neighboring samples to the derived samples. In one example, the model derivation can minimize the difference between the reconstructed and derived samples.

[0105] In some examples, the filtering process (e.g., the number of parameters N, the set of samples / values used as input for each target sample) may be predefined or derived from syntax elements signaled in the bitstream. The number of parameters or the filter length may depend on one or more block characteristics, including the degree of sample diversity (e.g., variance) in the block, the block size, the block or motion vector magnitude, etc.

[0106] In some examples, a 6-tap filter with 5 samples and a constant bias as input may be applied to the predictor:

[0107] predVal = c0C + c1N + c2S + c3W + c4E + c5b,

[0108] Where predVal is the output of the filtering process for the sample, and C corresponds to the input sample concatenated with the target output sample (referred to as the "concatenated sample"). N, S, W, and E represent the north (upper), south (lower), west (left), and east (right) neighbors of the concatenated sample. 'b' is a constant, which in some examples can be a sample value in the middle of the sample value range (e.g., 512 for a 10-bit range [0-1023]).

[0109] In some examples, a second-order term may be added to make predVal a 7-tap filter, as follows:

[0110] PredVal = c -1 C 2 + c0C + c1N + c2S + c3W + c4E + c5b.

[0111] In some examples, the parameters used in the filtering process are determined by minimizing the difference between the template of the current block and the result of applying the filtering process to the template area of the reference block. In one example, the minimization criterion can be the mean square error (MSE).

[0112] In some examples, certain constraints can be imposed on the function of predVal to reduce the solution space. In some examples, the coefficients of predVal can be optimized according to c1=c2=c3=c4=0. In some examples, the coefficients can be optimized according to c1=c2=0. In some examples, the coefficients can be optimized according to c3=c4=0.

[0113] In some examples, an "L-shaped" template of the current block located in a reconstruction region having M rows and N columns (e.g., M=N=4) of samples is used to derive the filtering parameters. The parameters of the filtering process are derived by minimizing the MSE between the L-shaped template of the current block and the filtered L-shaped template of the IBC predictor block.

[0114] An example of a template shape could be to use only the left template, or only the upper template or another other template defined for an adjacent area.

[0115] In some examples, the turning on / off of the proposed filtering process for prediction may be controlled by a block level flag. As another example, the turning on / off of the proposed filtering process for prediction may be derived based on information related to the current block (e.g., block size, block shape, error between the template of the current block and the template of the reference block before and / or after the filtering process). In some examples, there is no need to signal the block level flag, and the block level flag may be determined based on the TM cost of the filtered template (the error between the template of the current block and the template of the reference block / reference block candidate) and the TM cost of the unfiltered template. The flag value is set to 1 only if the filtered template achieves a lower TM cost; otherwise, the flag value is set to 0. In yet another example, the flag value is set to 1 only if the TM cost of the filtered template plus a positive delta term is still lower than the TM cost of the unfiltered template; otherwise, the flag value is set to 0. The delta term is, for example, set to 2 bitdepth–2Multiply by the number of samples on the template of the current block. For example, the number of samples is N and the bit depth of the template samples is BD, and the increment term is set equal to N*2 BD-2 .

[0116] As an example of predictive filtering, in one example of predicting a block coded in intra TMP mode, LIC can be applied to the predicted block. All examples of deriving filter parameters in this document can be used to derive LIC parameters. The application of LIC to intra TMP predicted blocks can be controlled by block-level flags. In some examples, the application of LIC to intra TMP predicted blocks can also be determined implicitly (without explicit signaling) based on predicted block characteristics and / or template characteristics or block size or shape.

[0117] In another example, various filtering modes applied to the prediction may be introduced (e.g., derived with different numbers of parameters or different processes, e.g., LIC)), and the mode selection may be signaled or derived implicitly, which in one example may be accomplished by selecting the mode that produces the smallest difference to the template.

[0118] Now let's discuss linear model filtering for intra-template matching (TMP) prediction blocks. Intra-TMP is an intra-coding tool implemented in ECM7. It performs a template-based search in already decodable (causal) regions and determines the best template that minimizes the difference between the reference template and the current template (the template adjacent to the current block) to determine a displacement vector (or block vector). This block vector is then used to retrieve the corresponding reference block to be used to predict the current block.

[0119] The video encoder 200 and the video decoder 300 can be configured to apply a linear filtering model to the prediction block to further improve prediction accuracy. The model can be derived using a reference template and a current template so that no further signaling of model parameters is required. The 6-tap filter can include a 5-tap plus a sign-shaped spatial component and an offset term.

[0120] Figure 6 is a conceptual diagram illustrating the spatial portion of the filter 131. The input to the spatial 5-tap component of the filter 131 includes the center (C) sample in the reference block located at the corresponding position of the sample in the current block to be predicted, and the upper / north (N), lower / south (S), left / west (W), and right / east (E) neighboring blocks, as shown in FIG. Figure 6 The bias term B represents a scalar offset between the input and output and is set to the mid-luma value (512 for 10-bit content). The output of the filter can be calculated as follows:

[0121] predLumaVal = c0C + c1N + c2S + c3E + c4W + c5B.

[0122] Figure 7 The video encoder 200 and the video decoder 300 may be configured to calculate the filter coefficients ci by minimizing the mean square error (MSE) between the reference template 135 and the current template 137, as shown in FIG. Figure 7 The extension of the area shown in the solid grey shaded region is required to support the “side samples” of the positive shape spatial filter and can be filled in when located in unavailable areas.

[0123] The video encoder 200 and the video decoder 300 may perform MSE minimization by calculating an autocorrelation matrix of a reference template input and a current template output. The autocorrelation matrix may be subjected to LDL decomposition, and back-substitution may be used to calculate the final filter coefficients.

[0124] Therefore, the video encoder 200B and the video decoder 300 can determine the filter coefficients (ci) so that when a filter based on the filter coefficients is applied to each sample of the reference template, the MSE of the filtered reference template and the current template is minimized. Then, the video encoder 200B or the video decoder 300 can apply the filter to the samples of the reference block.

[0125] A CU-level flag can be used to signal the use of intra template matching prediction with filtered linear model (intra TMP-FLM) mode. Intra TMP-FLM can be considered a sub-mode of intra TMP. That is, in some examples, the intra TMP-FLM flag can be signaled only if the intra TMP flag is true (meaning intra TMP is enabled).

[0126] Similar kinds of filtering for IBCs, which currently do not exist in ECMs, may also be beneficial.The examples of this disclosure regarding filtering for IBCs may be applied independently or in combination.

[0127] In some examples, the filtering mode for IBC may be used as an additional mode. In some examples, the filtering mode for IBC may not be applied together with IBC-LIC and / or IBC-CIIP (combined intra-inter prediction mode for IBC), i.e., if the filtering mode for IBC is enabled, LIC and / or CIIP are disabled. The signaling structure may be as follows:

[0128] If(isIBC)

[0129] {

[0130] Parse isIBCFiltered

[0131] If (!isIBCFiltered)

[0132] {

[0133] Parse IBCLIC flag

[0134] }

[0135] If (!isIBCFiltered &&!IBCLIC)

[0136] {

[0137] Parse IBCCIIP flag

[0138] }

[0139] }

[0140] In some examples, the filtering mode for IBC may also replace the IBC-LIC mode.

[0141] In some examples, the filtering mode for IBC may be applied only to certain block sizes. For example, video encoder 200 and video decoder 300 may be configured not to apply the filtering mode for IBC to block sizes (regions) smaller than a threshold. For example, the filtering mode for IBC may not be applied to blocks with width * height < 32.

[0142] In some examples, video encoder 200 and video decoder 300 may be configured not to apply the filtering mode for IBC when neither the left template nor the top template is fully available for the current block. For example, if the position of the CU is (0, 0), signaling may be avoided. If the template size is T, then for the CU position (x, y), if x < T or y < T, signaling for the filtering mode for IBC may be avoided. Alternatively, a predetermined fixed filter model may be applied to those blocks.

[0143] In some examples, the reference block and the reference template may be included in the IBC reference area. Figure 8A conceptual diagram illustrating a reference template 135 (which is a combination of 135A and 135B) and a current template 137 (which is a combination of 137A and 137B) for an IBC-coded block, according to techniques of this disclosure. If the reference template points outside the reference region, only a portion of the template (which is inside the reference region) can be used to derive model parameters. For example, in some cases, if the left template (e.g., template 135B) is outside the reference region but the upper template (e.g., template 135A) is inside the reference region, only the upper template 135A can be used for model generation. Alternatively, if the reference template points outside the reference region, padding from the nearest sample inside the reference region can be used to generate a reference template outside the reference region instead.

[0144] In some examples, for fractional block vectors, the video encoder 200 and video decoder 300 can be configured to generate a reference template using only the integer portion of the block vector to generate a filter model without any associated interpolation (still generating the reference block using fractional pixel motion compensation by interpolation).

[0145] In some examples, this process may be applied only when the block vector is at integer pixels (but not for fractional pixel block vectors).

[0146] In some examples, video encoder 200 and video decoder 300 can be configured to apply different filter models (including different filter shapes) for filtering. These different models can be applied as multiple candidates with additional signaling. Alternatively, the appropriate filter (among those multiple filters) can be implicitly selected based on block size or spatial characteristics.

[0147] In some examples, the video encoder 200 and the video decoder 300 can be configured to reorder different filter models and / or multiple candidates based on calculating the difference between the filtered prediction and the actual reconstruction on the evaluation template. The evaluation template can be selected to include samples that have already been decoded so that the reconstruction is available. The filtered prediction is generated by first performing IBC prediction using the current block vector but for the evaluation template, and then applying the corresponding filter model. Alternatively, only the filter model with the smallest difference can be used for the current block without additional signaling. Figure 9 is a conceptual diagram illustrating reference template 141 , current template 143 , and evaluation template 145 for an IBC-coded block according to the techniques of this disclosure.

[0148] In some examples, the video encoder 200 and the video decoder 300 can be configured to signal the filtering mode for IBC only for non-merge mode. For merge mode, the filtering mode for IBC can be inherited. For example, if the corresponding merge candidate has the filtering mode for IBC enabled, the filtering mode for IBC can also be enabled for the current block because the filter mode for enabling / disabling IBC is inherited. In another example, inheritance can be disabled for certain IBC merge modes (IBC-CIIP, IBC-GPM).

[0149] In some examples, the video encoder 200 and the video decoder 300 may be configured to inherit the IBC filtering mode for the current block only if the merge candidate is a composite candidate (consisting of two or more individual merge candidates) and all merge candidates have the filtering mode enabled (or at least one in some examples).

[0150] In some examples, video encoder 200 and video decoder 300 may be configured to signal filtering modes for IBC only for I slices.

[0151] In some examples, the video encoder 200 and video decoder 300 can be configured to generate filter models using different templates (left only, top only, or L-shaped), which can be considered as multiple candidates, which can then be selected based on signaling (with / without reordering) or implicitly (based on size, spatial characteristics, or based on differences in evaluated templates).

[0152] In some examples, video encoder 200 and video decoder 300 may be configured to apply additional filters to both the current template and the reference template (to remove high frequency artifacts / noise) prior to deriving the model generation.

[0153] In some examples, additional constraints on filters being symmetric may also be used for filter model generation.

[0154] In some examples, video encoder 200 and video decoder 300 may be configured to apply a set of offline trained fixed model filters in addition to the template-generated filter models.

[0155] In some examples, for single-tree scenarios (e.g., when luma and chroma blocks are coded together in a CU), this filtering part may be limited to the luma component only, or in some examples, the chroma part may also have independent model generation and filtering.

[0156] In some examples, when the block vector is in a fractional pixel and the extended reference area (i.e., the reference block area, the associated template with or without padding, and the additional extended area to account for the application of the interpolation filter) is partially outside the available IBC reference area, the external area can be filled from the adjacent available area.

[0157] In some examples, the video encoder 200 and the video decoder 300 can be configured to sort the different fractional pixel positions based on template distortion for the fractional pixel block vector position, i.e., calculate the difference between the current template and the reference template, and instead signal the index of the position of the specified fractional pixel candidate in the sorted list. In addition, not all fractional pixel positions may be signaled. Instead, the first few fractional pixel candidates (a subset) are signaled using an index.

[0158] In some examples, video encoder 200 and video decoder 300 can be configured to select the context of the filtering mode based on the coding mode of the neighboring block. Several additional examples may include whether the neighboring block uses IBC filtering, whether the neighboring block uses IBC-LIC or IBC filtering, or whether the neighboring block uses IBC-LIC or IBC filtering or intra-frame TMP filtering.

[0159] In some examples, when the reference template (and / or current template) is fully or partially unavailable, the video encoder 200 and the video decoder 300 can be configured to use fixed filtering (e.g., fixed parameter model filtering) rather than deriving a model based on a template. Furthermore, this fixed parameter model filtering can be based on the dominant direction of the IBC block vector prediction block or on the current template gradient direction. Furthermore, the filtering can also depend on the gradient magnitude (smoothness) of the current template or block vector prediction block. Alternatively or additionally, fixed parameter filtering can be applied if the reference template and the current template are very different from each other.

[0160] In some examples, an advanced flag may enable / disable this coding tool.For example, video encoder 200B or video decoder 300 may encode or decode a syntax element indicating whether coding tools that determine IBC reference blocks, calculate coefficients, and apply filters are enabled or disabled.

[0161] In some examples, the video encoder 200 and the video decoder 300 can be configured to signal an SPS-level flag to indicate whether this tool is enabled / disabled at the sequence level. This flag can be conditionally signaled based on the IBC flag, i.e., when only IBC is enabled, this flag is signaled. Additionally, this flag can be conditionally signaled based on the intra-frame TMP flag. For example, this flag can be signaled when the intra-frame TMP flag is disabled. If this flag is not signaled, then it can be inferred that the value of this flag is zero.

[0162] In some examples, the video encoder 200 and the video decoder 300 may be configured to enable / disable the tool at the slice level.The signaling may also depend on whether IBC is enabled for a particular slice, ie, the signaling may depend on a slice-level IBC enable flag.

[0163] Figure 10 is a block diagram illustrating an example video encoder 200 that may perform the techniques of this disclosure. Figure 10 This is provided for purposes of explanation and should not be construed as limiting the techniques as broadly illustrated and described in this disclosure. For purposes of explanation, this disclosure describes a video encoder 200 based on techniques of VVC and HEVC. However, the techniques of this disclosure may be performed by video encoding devices configured to other video coding standards and video coding formats (e.g., AV1 and successors of the AV1 video coding format).

[0164] exist Figure 10 In the example of FIG, the video encoder 200 includes a video data memory 230, a mode selection unit 202, a residual generation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a filter unit 216, a decoded picture buffer (DPB) 218, and an entropy coding unit 220. Any or all of the video data memory 230, the mode selection unit 202, the residual generation unit 204, the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the filter unit 216, the DPB 218, and the entropy coding unit 220 can be implemented in one or more processors or in processing circuitry. For example, the units of the video encoder 200 can be implemented as one or more circuits or logic elements, as part of a hardware circuit, or as part of a processor, ASIC, or FPGA. Furthermore, the video encoder 200 can include additional or alternative processors or processing circuitry to perform these and other functions.

[0165] The video data memory 230 may store video data to be encoded by the components of the video encoder 200. The video encoder 200 may receive video data from, for example, the video source 104 ( Figure 1) receives video data stored in the video data memory 230. The DPB 218 can act as a reference picture memory that stores reference video data used by the video encoder 200 to predict subsequent video data. The video data memory 230 and the DPB 218 can be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. The video data memory 230 and the DPB 218 can be provided by the same memory device or separate memory devices. In various examples, the video data memory 230 can be on-chip with other components of the video encoder 200, as shown, or off-chip relative to those components.

[0166] In this disclosure, references to the video data memory 230 should not be construed as limited to memory internal to the video encoder 200 (unless specifically described as such) or memory external to the video encoder 200 (unless specifically described as such). Rather, references to the video data memory 230 should be understood as a reference memory that stores video data received by the video encoder 200 for encoding (e.g., video data of a current block to be encoded). Figure 1 The memory 106 may also provide temporary storage of outputs from the various units of the video encoder 200 .

[0167] Figure 10 Various units are shown to aid in understanding the operations performed by the video encoder 200. The units may be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Fixed-function circuits refer to circuits that provide specific functionality and are preset in the operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functionality in the operations that can be performed. For example, a programmable circuit may execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuits may execute software instructions (e.g., to receive parameters or output parameters), but the type of operations performed by the fixed-function circuits is generally immutable. In some examples, one or more of the units may be different circuit blocks (fixed-function or programmable), and in some examples, one or more of the units may be integrated circuits.

[0168] The video encoder 200 may include an arithmetic logic unit (ALU), an elementary function unit (EFU), a digital circuit, an analog circuit, and / or a programmable core formed by a programmable circuit. In an example where the operation of the video encoder 200 is performed using software executed by a programmable circuit, the memory 106 ( Figure 1) may store instructions (eg, object code) for software received and executed by the video encoder 200, or another memory (not shown) within the video encoder 200 may store such instructions.

[0169] The video data memory 230 is configured to store received video data. The video encoder 200 may retrieve a frame of video data from the video data memory 230 and provide the video data to the residual generation unit 204 and the mode selection unit 202. The video data in the video data memory 230 may be original video data to be encoded.

[0170] Mode selection unit 202 includes a motion estimation unit 222, a motion compensation unit 224, and an intra prediction unit 226. Mode selection unit 202 may include additional functional units for performing video prediction according to other prediction modes. By way of example, mode selection unit 202 may include a palette unit, an intra block copy unit (which may be part of motion estimation unit 222 and / or motion compensation unit 224), an affine unit, a linear model (LM) unit, or the like.

[0171] The mode selection unit 202 typically coordinates multiple coding passes to test combinations of coding parameters and the resulting rate-distortion values for these combinations. The coding parameters may include the CTU to CU partitioning, the CU prediction mode, the transform type of the CU's residual data, the quantization parameter of the CU's residual data, etc. The mode selection unit 202 may ultimately select a combination of coding parameters that has a better rate-distortion value than other tested combinations.

[0172] The video encoder 200 may partition a picture retrieved from the video data memory 230 into a series of CTUs and encapsulate one or more CTUs into a slice. The mode selection unit 202 may partition the CTUs of the picture according to a tree structure (e.g., the MTT structure, QTBT structure, super block structure, or quadtree structure described above). As described above, the video encoder 200 may form one or more CUs by partitioning the CTUs according to the tree structure. Such CUs may also be generally referred to as "video blocks" or "blocks."

[0173] Generally speaking, mode select unit 202 also controls its components (e.g., motion estimation unit 222, motion compensation unit 224, and intra prediction unit 226) to generate a prediction block for a current block (e.g., the current CU, or, in HEVC, the overlapping portions of a PU and TU). For inter prediction of the current block, motion estimation unit 222 may perform a motion search to identify one or more closely matching reference blocks in one or more reference pictures (e.g., one or more previously encoded pictures stored in DPB 218). Specifically, motion estimation unit 222 may calculate a value indicating how similar the potential reference blocks are to the current block, such as based on sum of absolute differences (SAD), sum of squared differences (SSD), mean absolute difference (MAD), mean squared difference (MSD), or the like. Motion estimation unit 222 may typically perform these calculations using the sample-by-sample differences between the current block and the reference block under consideration. Motion estimation unit 222 may identify the reference block with the lowest value resulting from these calculations, indicating the reference block that most closely matches the current block.

[0174] Motion estimation unit 222 may form one or more motion vectors (MVs) that define the position of a reference block in a reference picture relative to the position of the current block in the current picture. Motion estimation unit 222 may then provide the motion vectors to motion compensation unit 224. For example, for unidirectional inter prediction, motion estimation unit 222 may provide a single motion vector, while for bidirectional inter prediction, motion estimation unit 222 may provide two motion vectors. Motion compensation unit 224 may then use the motion vectors to generate a prediction block. For example, motion compensation unit 224 may use the motion vectors to retrieve data for the reference block. As another example, if the motion vectors have fractional sample precision, motion compensation unit 224 may interpolate the values of the prediction block according to one or more interpolation filters. Furthermore, for bidirectional inter prediction, motion compensation unit 224 may retrieve data for the two reference blocks identified by the respective motion vectors and combine the retrieved data, for example, by sample-by-sample averaging or weighted averaging.

[0175] When operating according to the AV1 video coding format, the motion estimation unit 222 and the motion compensation unit 224 may be configured to encode coding blocks of video data (e.g., both luma and chroma coding blocks) using translational motion compensation, affine motion compensation, overlapped block motion compensation (OBMC), and / or compound inter-intra prediction.

[0176] As another example, for intra prediction or intra prediction encoding, the intra prediction unit 226 may generate a prediction block from samples adjacent to the current block. For example, for directional mode, the intra prediction unit 226 may generally mathematically combine the values of the adjacent samples and pad these calculated values in a defined direction on the current block to produce a prediction block. As another example, for DC mode, the intra prediction unit 226 may calculate the average of the adjacent samples of the current block and generate a prediction block to include the obtained average for each sample of the prediction block.

[0177] When operating according to the AV1 video coding format, the intra prediction unit 226 can be configured to encode coding blocks of video data (e.g., luma and chroma coding blocks) using directional intra prediction, non-directional intra prediction, recursive filter intra prediction, luma-chroma (CFL) prediction, intra block copy (IBC), and / or palette mode. The mode selection unit 202 can include additional functional units for performing video prediction according to other prediction modes.

[0178] Mode selection unit 202 provides the prediction block to residual generation unit 204. Residual generation unit 204 receives the original, unencoded version of the current block from video data memory 230 and the prediction block from mode selection unit 202. Residual generation unit 204 calculates the sample-by-sample difference between the current block and the prediction block. The resulting sample-by-sample difference defines a residual block for the current block. In some examples, residual generation unit 204 may also determine the difference between sample values in the residual block to generate the residual block using residual differential pulse code modulation (RDPCM). In some examples, residual generation unit 204 may be formed using one or more subtractor circuits that perform binary subtraction.

[0179] In the example where the mode select unit 202 partitions the CU into PUs, each PU may be associated with a luma prediction unit and a corresponding chroma prediction unit. The video encoder 200 and the video decoder 300 may support PUs of various sizes. As indicated above, the size of a CU may refer to the size of the luma coding block of the CU, and the size of a PU may refer to the size of the luma prediction unit of the PU. Assuming a particular CU size of 2N×2N, the video encoder 200 may support PU sizes of 2N×2N or N×N for intra prediction, and symmetric PU sizes of 2N×2N, 2N×N, N×2N, N×N, or similar sizes for inter prediction. The video encoder 200 and the video decoder 300 may also support asymmetric partitioning of PU sizes of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter prediction.

[0180] In an example where mode select unit 202 does not further partition a CU into PUs, each CU may be associated with a luma coding block and a corresponding chroma coding block. As described above, the size of a CU may refer to the size of the luma coding block of the CU. Video encoder 200 and video decoder 300 may support CU sizes of 2N×2N, 2N×N, or N×2N.

[0181] For other video coding techniques, such as intra block copy mode coding, affine mode coding, and LM mode coding, as some examples, mode selection unit 202 generates a prediction block for the current block being encoded via the corresponding unit associated with the coding technique. In some examples, such as palette mode coding, mode selection unit 202 may not generate a prediction block, but instead generate syntax elements indicating how to reconstruct the block based on the selected palette. In such modes, mode selection unit 202 may provide these syntax elements to entropy coding unit 220 for encoding.

[0182] Mode selection unit 202 also includes a prediction filter (PF) unit 203, which can perform the prediction filtering techniques of the present disclosure. As part of generating a prediction block, PF unit 203 can determine whether to apply filtering to the prediction block and, based on determining that filtering is to be applied to the prediction block, compare a template of a reference block with a template of a current block and, based on the comparison of the template of the reference block with the template of the current block, filter the prediction block to determine a filtered prediction block. PF unit 203 can, for example, determine a filter that minimizes the difference between sample values of the template of the reference block and sample values of the template of the current block and filter the prediction block using the determined filter. Application of the filter can modify sample values of the template of the current block to reduce the mean squared error between sample values of the template of the current block and sample values of the template of the reference block. Thus, when PF unit 203 applies the filter to the prediction block, the filter can also reduce the mean squared error between the prediction block and the corresponding original block of video data.

[0183] As described above, the residual generation unit 204 receives video data of a current block and a corresponding prediction block. The residual generation unit 204 then generates a residual block for the current block. To generate the residual block, the residual generation unit 204 calculates the sample-by-sample difference between the prediction block and the current block.

[0184] The transform processing unit 206 applies one or more transforms to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). The transform processing unit 206 may apply various transforms to the residual block to form the transform coefficient block. For example, the transform processing unit 206 may apply a discrete cosine transform (DCT), a directional transform, a Karhunen-Loeve transform (KLT), or a conceptually similar transform to the residual block. In some examples, the transform processing unit 206 may perform multiple transforms on the residual block, such as a primary transform and a secondary transform, such as a rotation transform. In some examples, the transform processing unit 206 does not apply a transform to the residual block.

[0185] When operating in accordance with AV1, the transform processing unit 206 may apply one or more transforms to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). The transform processing unit 206 may apply various transforms to the residual block to form the transform coefficient block. For example, the transform processing unit 206 may apply a combination of horizontal / vertical transforms, which may include a discrete cosine transform (DCT), an asymmetric discrete sine transform (ADST), a flipped ADST (e.g., an ADST in reverse order), and an identity transform (IDTX). When the identity transform is used, the transform is skipped in one of the vertical or horizontal directions. In some examples, the transform processing may be skipped.

[0186] Quantization unit 208 may quantize the transform coefficients in a transform coefficient block to generate a quantized transform coefficient block. Quantization unit 208 may quantize the transform coefficients of the transform coefficient block according to a quantization parameter (QP) value associated with the current block. Video encoder 200 (e.g., via mode select unit 202) may adjust the degree of quantization applied to the transform coefficient block associated with the current block by adjusting the QP value associated with the CU. Quantization may introduce a loss of information, and therefore, the quantized transform coefficients may have lower precision than the original transform coefficients generated by transform processing unit 206.

[0187] The inverse quantization unit 210 and the inverse transform processing unit 212 may apply inverse quantization and inverse transform, respectively, to the quantized transform coefficient block to reconstruct a residual block from the transform coefficient block. The reconstruction unit 214 may generate a reconstructed block corresponding to the current block (although possibly with some degree of distortion) based on the reconstructed residual block and the prediction block generated by the mode selection unit 202. For example, the reconstruction unit 214 may add samples of the reconstructed residual block to corresponding samples from the prediction block generated by the mode selection unit 202 to generate the reconstructed block.

[0188] Filter unit 216 may perform one or more filter operations on the reconstructed block. For example, filter unit 216 may perform a deblocking operation to reduce blocking artifacts along the edges of the CU. In some examples, the operations of filter unit 216 may be skipped.

[0189] When operating in accordance with AV1, filter unit 216 may perform one or more filter operations on the reconstructed blocks. For example, filter unit 216 may perform a deblocking operation to reduce blocking artifacts along the edges of the CU. In other examples, filter unit 216 may apply a constrained directional enhancement filter (CDEF), which may be applied after deblocking and may include applying a non-separable nonlinear low-pass directional filter based on the estimated edge direction. Filter unit 216 may also include a loop restoration filter applied after CDEF and may include a separable symmetric normalized Wiener filter or a dual self-guided filter.

[0190] The video encoder 200 stores the reconstructed block in the DPB 218. For example, in an example where the operation of the filter unit 216 is not performed, the reconstruction unit 214 may store the reconstructed block in the DPB 218. In an example where the operation of the filter unit 216 is performed, the filter unit 216 may store the filtered reconstructed block in the DPB 218. The motion estimation unit 222 and the motion compensation unit 224 may retrieve a reference picture formed by the reconstructed (and potentially filtered) block from the DPB 218 to perform inter-frame prediction on blocks of subsequently encoded pictures. In addition, the intra-frame prediction unit 226 may use the reconstructed block in the DPB 218 of the current picture to perform intra-frame prediction on other blocks in the current picture.

[0191] In general, entropy coding unit 220 may entropy encode syntax elements received from other functional components of video encoder 200. For example, entropy coding unit 220 may entropy encode blocks of quantized transform coefficients from quantization unit 208. As another example, entropy coding unit 220 may entropy encode prediction syntax elements (e.g., motion information for inter-frame prediction or intra-frame mode information for intra-frame prediction) from mode selection unit 202. Entropy coding unit 220 may perform one or more entropy encoding operations on the syntax elements (which is another example of video data) to produce entropy-encoded data. For example, entropy coding unit 220 may perform a context-adaptive variable length coding (CAVLC) operation, a CABAC operation, a variable-to-variable (V2V) length coding operation, a syntax-based context-adaptive binary arithmetic coding (SBAC) operation, a probability interval partitioning entropy (PIPE) coding operation, an exponential Golomb coding operation, or another type of entropy coding operation on the data. In some examples, entropy coding unit 220 may operate in a bypass mode, in which the syntax elements are not entropy encoded.

[0192] The video encoder 200 may output a bitstream including entropy-encoded syntax elements required to reconstruct blocks of a slice or picture. Specifically, the entropy encoding unit 220 may output a bitstream.

[0193] According to AV1, entropy coding unit 220 may be configured as a symbol-by-symbol adaptive multi-symbol arithmetic coder. Syntax elements in AV1 include an alphabet of N elements, and context (e.g., a probability model) includes a set of N probabilities. Entropy coding unit 220 may store the probabilities as an n-bit (e.g., 15-bit) cumulative distribution function (CDF). Entropy coding unit 220 may perform recursive scaling to update the context using an update factor based on the alphabet size.

[0194] The operations described above are described with respect to blocks. Such descriptions should be understood as operations for luma coding blocks and / or chroma coding blocks. As described above, in some examples, the luma coding blocks and chroma coding blocks are the luma components and chroma components of a CU. In some examples, the luma coding blocks and chroma coding blocks are the luma components and chroma components of a PU.

[0195] In some examples, the operations performed for the luma coding blocks do not need to be repeated for the chroma coding blocks. As an example, the operations for identifying the motion vector (MV) and reference picture for the luma coding blocks do not need to be repeated to identify the MV and reference picture for the chroma blocks. In fact, the MV of the luma coding blocks can be scaled to determine the MV for the chroma blocks, and the reference picture can be the same. As another example, the intra prediction process can be the same for the luma coding blocks and the chroma coding blocks.

[0196] Video encoder 200 represents an example of a device configured to encode video data, the device including a memory configured to store the video data, and one or more processing units implemented in circuitry and configured to: determine a prediction block for a current block of a current picture of the video data; compare a template of the prediction block with a template of the current block; filter the prediction block based on the comparison of the template of the prediction block with the template of the current block to determine a filtered prediction block; and encode the current block based on the filtered prediction block.

[0197] Figure 11 is a block diagram illustrating an example video decoder 300 that may perform the techniques of this disclosure. Figure 11 It is provided for the purpose of explanation and does not limit the techniques as broadly illustrated and described in this disclosure. For the purpose of explanation, this disclosure describes a video decoder 300 according to the techniques of VVC and HEVC. However, the techniques of this disclosure may be performed by video coding devices configured for other video coding standards.

[0198] exist Figure 11In the example of FIG, video decoder 300 includes a coded picture buffer (CPB) memory 320, an entropy decoding unit 302, a prediction processing unit 304, an inverse quantization unit 306, an inverse transform processing unit 308, a reconstruction unit 310, a filter unit 312, and a DPB 314. Any or all of CPB memory 320, entropy decoding unit 302, prediction processing unit 304, inverse quantization unit 306, inverse transform processing unit 308, reconstruction unit 310, filter unit 312, and DPB 314 may be implemented in one or more processors or in processing circuitry. For example, the units of video decoder 300 may be implemented as one or more circuits or logic elements as part of a hardware circuit, or as part of a processor, ASIC, or FPGA. Furthermore, video decoder 300 may include additional or alternative processors or processing circuitry to perform these and other functions.

[0199] The prediction processing unit 304 includes a motion compensation unit 316 and an intra prediction unit 318. The prediction processing unit 304 may include additional units that perform prediction according to other prediction modes. As examples, the prediction processing unit 304 may include a palette unit, an intra block copy unit (which may form part of the motion compensation unit 316), an affine unit, an LM unit, or the like. In other examples, the video decoder 300 may include more, fewer, or different functional components.

[0200] When operating in accordance with AV1, the motion compensation unit 316 can be configured to decode coding blocks of video data (e.g., both luma and chroma coding blocks) using translational motion compensation, affine motion compensation, OBMC, and / or composite inter-intra prediction, as described above. The intra prediction unit 318 can be configured to decode coding blocks of video data (e.g., both luma and chroma coding blocks) using directional intra prediction, non-directional intra prediction, recursive filter intra prediction, CFL, IBC, and / or color palette mode, as described above.

[0201] CPB memory 320 may store video data, such as an encoded video bitstream, to be decoded by components of video decoder 300. The video data may be obtained, for example, from computer readable medium 110 ( Figure 1) obtains video data stored in CPB memory 320. CPB memory 320 may include a CPB that stores coded video data (e.g., syntax elements) from a coded video bitstream. Furthermore, CPB memory 320 may store video data other than syntax elements for decoded pictures, such as temporary data representing outputs from various units of video decoder 300. DPB 314 typically stores decoded pictures, which can be output and / or used by video decoder 300 as reference video data when decoding subsequent data or pictures in the coded video bitstream. CPB memory 320 and DPB 314 may be implemented by any of a variety of memory devices, such as DRAM, including SDRAM, MRAM, RRAM, or other types of memory devices. CPB memory 320 and DPB 314 may be provided by the same memory device or separate memory devices. In various examples, CPB memory 320 may be on-chip with other components of video decoder 300, or off-chip relative to those components.

[0202] Additionally or alternatively, in some examples, video decoder 300 may retrieve the video from memory 120 ( Figure 1 ) retrieves the decoded video data. That is, memory 120 may store data as discussed above with respect to CPB memory 320. Similarly, when some or all of the functionality of video decoder 300 is implemented in software to be executed by processing circuitry of video decoder 300, memory 120 may store instructions to be executed by video decoder 300.

[0203] Figure 11 The various units shown in FIG are shown to aid in understanding the operations performed by the video decoder 300. The units may be implemented as fixed function circuits, programmable circuits, or a combination thereof. Figure 10 Fixed-function circuits are circuits that provide specific functionality and are pre-defined in the operations they can perform. Programmable circuits are circuits that can be programmed to perform a variety of tasks and provide flexible functionality in the operations they can perform. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by the software or firmware's instructions. Fixed-function circuits can execute software instructions (e.g., to receive parameters or output parameters), but the type of operations performed by the fixed-function circuits is generally immutable. In some examples, one or more of the units may be distinct circuit blocks (fixed-function or programmable), and in some examples, one or more of the units may be integrated circuits.

[0204] The video decoder 300 may include an ALU, an EFU, digital circuits, analog circuits, and / or a programmable core formed by programmable circuits. In examples where the operation of the video decoder 300 is performed by software executed on the programmable circuits, on-chip or off-chip memory may store instructions (e.g., object code) of the software received and executed by the video decoder 300.

[0205] Entropy decoding unit 302 may receive encoded video data from the CPB and entropy decode the video data to reproduce syntax elements. Prediction processing unit 304, inverse quantization unit 306, inverse transform processing unit 308, reconstruction unit 310, and filter unit 312 may generate decoded video data based on syntax elements extracted from the bitstream.

[0206] Typically, the video decoder 300 reconstructs a picture on a block-by-block basis. The video decoder 300 may perform a reconstruction operation on each block individually (where the block currently being reconstructed (ie, decoded) may be referred to as the "current block").

[0207] The entropy decoding unit 302 may entropy decode syntax elements that define the quantized transform coefficients of the quantized transform coefficient block, as well as transform information, such as a quantization parameter (QP) and / or a transform mode indication. The inverse quantization unit 306 may use the QP associated with the quantized transform coefficient block to determine a degree of quantization and, similarly, to determine a degree of inverse quantization to be applied by the inverse quantization unit 306. The inverse quantization unit 306 may, for example, perform a bitwise left shift operation to inverse quantize the quantized transform coefficients. The inverse quantization unit 306 may thereby form a transform coefficient block comprising the transform coefficients.

[0208] After inverse quantization unit 306 forms a transform coefficient block, inverse transform processing unit 308 may apply one or more inverse transforms to the transform coefficient block to generate a residual block associated with the current block. For example, inverse transform processing unit 308 may apply an inverse DCT, an inverse integer transform, an inverse Karhunen-Loeve transform (KLT), an inverse rotational transform, an inverse directional transform, or another inverse transform to the transform coefficient block.

[0209] In addition, prediction processing unit 304 generates a prediction block based on the prediction information syntax element entropy decoded by entropy decoding unit 302. For example, if the prediction information syntax element indicates that the current block is inter-predicted, then motion compensation unit 316 may generate the prediction block. In this case, the prediction information syntax element may indicate a reference picture in DPB 314 from which the reference block is retrieved, and a motion vector identifying the position of the reference block in the reference picture relative to the position of the current block in the current picture. Motion compensation unit 316 may generally generate a prediction block in a manner substantially similar to that described with respect to motion compensation unit 224 ( Figure 10 ) performs the inter-frame prediction process in the manner described.

[0210] As another example, if the prediction information syntax element indicates that the current block is intra-predicted, then the intra-prediction unit 318 may generate a prediction block according to the intra-prediction mode indicated by the prediction information syntax element. Again, the intra-prediction unit 318 may generally generate a prediction block in a manner substantially similar to that described with respect to the intra-prediction unit 226 ( Figure 10 The intra prediction process is performed in the manner described in the preceding claims. The intra prediction unit 318 may retrieve data of neighboring samples of the current block from the DPB 314.

[0211] The prediction processing unit 304 also includes a PF unit 305, which can perform the prediction filtering techniques of the present disclosure. As part of generating a prediction block, the PF unit 305 can determine whether to apply filtering to the prediction block and, based on determining that filtering is to be applied to the prediction block, compare the template of the reference block with the template of the current block and, based on the comparison of the template of the reference block with the template of the current block, filter the prediction block to determine a filtered prediction block. The PF unit 305 can, for example, determine a filter that minimizes the difference between sample values of the template of the reference block and sample values of the template of the current block and filter the prediction block using the determined filter. Application of the filter can modify sample values of the template of the current block to reduce the mean squared error between sample values of the template of the current block and sample values of the template of the reference block. Therefore, when the PF unit 305 applies the filter to the prediction block, the filter can also reduce the mean squared error between the prediction block and the corresponding original block of video data.

[0212] The reconstruction unit 310 may reconstruct the current block using the prediction block and the residual block. For example, the reconstruction unit 310 may add samples of the residual block to corresponding samples of the prediction block to reconstruct the current block.

[0213] The filter unit 312 may perform one or more filter operations on the reconstructed block. For example, the filter unit 312 may perform a deblocking operation to reduce blocking artifacts along the edges of the reconstructed block. The operations of the filter unit 312 may not necessarily be performed in all examples.

[0214] The video decoder 300 may store the reconstructed block in the DPB 314. For example, in examples where the operations of the filter unit 312 are not performed, the reconstruction unit 310 may store the reconstructed block to the DPB 314. In examples where the operations of the filter unit 312 are performed, the filter unit 312 may store the filtered reconstructed block to the DPB 314. As discussed above, the DPB 314 may provide reference information (e.g., samples of the current picture for intra-frame prediction and samples of previously decoded pictures for subsequent motion compensation) to the prediction processing unit 304. In addition, the video decoder 300 may output decoded pictures (e.g., decoded video) from the DPB 314 for subsequent display on a display device such as a video player. Figure 1 118).

[0215] In this manner, the video decoder 300 represents an example of a video decoding device that includes a memory configured to store video data, and one or more processing units implemented in circuitry and configured to determine a prediction block for a current block of a current picture of the video data; compare a template of the prediction block with a template of the current block; filter the prediction block based on the comparison of the template of the prediction block with the template of the current block to determine a filtered prediction block; and decode the current block based on the filtered prediction block.

[0216] Figure 12 1 is a flowchart illustrating an example process for encoding a current block according to the techniques of the present disclosure. The current block may be or include a current CU. Although with respect to the video encoder 200 ( Figure 1 and Figure 10 ), but it will be appreciated that other devices may be configured to perform the same Figure 12 The process is similar to the process.

[0217] In this example, the video encoder 200 initially predicts a current block (350). The prediction may include the filtering techniques described herein. The video encoder 200 may then calculate a residual block for the current block (352). To calculate the residual block, the video encoder 200 may calculate the difference between the original, unencoded block and the predicted block for the current block. The video encoder 200 may then transform the residual block and quantize the transform coefficients of the residual block (354). Next, the video encoder 200 may scan the quantized transform coefficients of the residual block (356). During or after the scan, the video encoder 200 may entropy encode the transform coefficients (358). For example, the video encoder 200 may encode the transform coefficients using CAVLC or CABAC. The video encoder 200 may then output the entropy encoded data for the block (360).

[0218] Figure 13 1 is a flowchart illustrating an example process for decoding a current block of video data according to the techniques of this disclosure. The current block may be or include a current CU. Although with respect to the video decoder 300 ( Figure 1 and Figure 11 ), but it will be appreciated that other devices may be configured to perform the same Figure 13 The process is similar to the process.

[0219] The video decoder 300 may receive entropy-encoded data for a current block, such as entropy-encoded prediction information corresponding to the current block and entropy-encoded data of transform coefficients for a residual block (370). The video decoder 300 may entropy decode the entropy-encoded data to determine the prediction information for the current block and regenerate the transform coefficients for the residual block (372). The video decoder 300 may predict the current block (374), for example, using an IBC, intra-frame, or inter-frame prediction mode indicated by the prediction information for the current block to calculate a prediction block for the current block. The prediction may include filtering techniques described herein. The video decoder 300 may then reverse scan the regenerated transform coefficients (376) to create a block of quantized transform coefficients. The video decoder 300 may then inverse quantize the transform coefficients and apply an inverse transform to the transform coefficients to produce a residual block (378). The video decoder 300 may ultimately decode the current block (380) by combining the prediction block and the residual block.

[0220] Figure 14 1 is a flowchart illustrating an example process for decoding a current block of video data according to the techniques of this disclosure. The current block may be or include a current CU. Although with respect to the video decoder 300 ( Figure 1 and Figure 11 ), but it will be appreciated that other devices may be configured to perform the same Figure 14 The process is similar to the process of

[0221] exist Figure 14 In the example of FIG. 3 , the video decoder 300 determines a prediction block for a current block of the current picture based on a reference block in the current picture of the video data ( 382 ). The prediction block can be, for example, a copy of the reference block in the current picture. The video decoder 300 can determine the prediction block for the current block of the current picture, for example, by locating the reference block in the same picture as the current block using a block vector.

[0222] The video decoder 300 determines whether to apply filtering to the prediction block (384). Based on the determination that filtering is to be applied to the prediction block (384, yes), the video decoder 300 compares the template of the reference block with the template of the current block (386), and filters the prediction block based on the comparison of the template of the reference block with the template of the current block to determine a filtered prediction block (388). Based on the determination that filtering is not to be applied to the prediction block (384, no), the video decoder 300 decodes the current block without filtering the prediction block.

[0223] The template of the reference block may be an L-shaped sample group including samples to the left of the reference block and samples above the reference block for determining the prediction block, and the template of the current block may be an L-shaped sample group including samples to the left of the current block and samples above the current block. In some examples, the template of the reference block may include padded samples.

[0224] To filter the prediction block based on a comparison of a template of a reference block with a template of a current block to determine a filtered prediction block, the video decoder 300 may be configured to determine a filter that minimizes the difference between sample values of the template of the reference block and sample values of the template of the current block, and filter the prediction block using the determined filter. To determine the filter that minimizes the difference between sample values of the template of the reference block and sample values of the template of the current block, the video decoder 300 may determine a filter that modifies sample values of the template of the current block to reduce the mean squared error between the sample values of the template of the current block and the sample values of the template of the reference block. In some examples, the video decoder 300 may determine not to filter the prediction block based on a comparison of the template of the reference block with the template of the current block, so that the filtered prediction block is equal to the prediction block.

[0225] To determine whether to apply filtering to a prediction block, the video decoder 300 may, for example, receive a flag, wherein a first value of the flag indicates that intra block copying with filtering is enabled, and a second value of the flag indicates that intra block copying with filtering is disabled. To determine whether to apply filtering to the prediction block, the video decoder 300 may additionally or alternatively determine the size of the current block and determine that filtering is applied to the prediction block by determining that the size is greater than a threshold size. To determine whether to apply filtering to the prediction block, the video decoder 300 may determine whether to apply filtering to the prediction block based on whether the slice type of the current block is an intra slice. For example, receiving the flag may be conditional on the size being greater than the threshold size and / or the slice of the current block being an I slice. To determine a prediction block for a current block in a current picture of video data, the video decoder 300 may determine a block vector for the current block based on a candidate selected from a merge list. To determine whether to apply filtering to the prediction block, the video decoder 300 may determine whether a prediction block corresponding to the candidate is determined through filtering.

[0226] The video decoder 300 decodes the current block based on the filtered prediction block to determine a decoded version of the current block (390). To decode the current block, the video decoder 300 may, for example, add residual data to the filtered prediction block to determine a reconstructed block and apply one or more filtering operations to the reconstructed block.

[0227] The video decoder 300 outputs a decoded picture of the video data that includes a decoded version of the current block (392). To output the decoded picture, the video decoder 300 may, for example, display the decoded picture, store or send a copy of the decoded picture for later display, or store a copy of the decoded picture for use in encoding or decoding other pictures of the video data.

[0228] The following numbered clauses illustrate one or more aspects of the devices and techniques described in this disclosure.

[0229] Item 1A: A method of decoding video data, the method comprising: determining a prediction block for a current block of a current picture of the video data; comparing a template of the prediction block with a template of the current block; filtering the prediction block based on the comparison of the template of the prediction block with the template of the current block to determine a filtered prediction block; and decoding the current block based on the filtered prediction block.

[0230] Clause 2A: The method of clause 1A, wherein filtering the prediction block based on the comparison of the template of the prediction block with the template of the current block to determine a filtered prediction block comprises: determining a filter that minimizes the difference between the template of the prediction block and the template of the current block; and filtering the prediction block using the determined filter.

[0231] Clause 3A: The method of clause 1A, wherein filtering the prediction block based on the comparison of the template of the prediction block with the template of the current block to determine a filtered prediction block comprises: determining not to filter the prediction block based on the comparison of the template of the prediction block with the template of the current block, such that the filtered prediction block is equal to the prediction block.

[0232] Clause 4A: The method of clause 1A, further comprising: receiving a flag; and filtering the prediction block based on a value of the flag.

[0233] Clause 5A: An apparatus for decoding video data, the apparatus comprising one or more apparatus for performing the method of any of clauses 1A-4A.

[0234] Clause 6A: The apparatus of clause 5A, wherein the one or more devices comprise one or more processors implemented in circuitry.

[0235] Clause 7A: The apparatus of any of clauses 5A and 6A, further comprising a memory for storing video data.

[0236] Clause 8A: The apparatus of any of clauses 5A-7A, further comprising a display configured to display the decoded video data.

[0237] Clause 9A: The device of any of clauses 5A-8A, wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

[0238] Clause 10A: The apparatus of any of clauses 5A-9A, wherein the apparatus comprises a video decoder.

[0239] Clause 11A: The apparatus of any of clauses 5A-10A, wherein the apparatus comprises a video encoder.

[0240] Clause 12A: A computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to perform the method of any of clauses 1A-4A.

[0241] Clause 1B: A method of decoding video data, the method comprising: determining a prediction block for a current block of a current picture of the video data; comparing a template of the prediction block to a template of the current block; filtering the prediction block based on the comparison of the template of the prediction block to the template of the current block to determine a filtered prediction block; decoding the current block based on the filtered prediction block to determine a decoded version of the current block; and outputting a decoded picture of the video data including the decoded version of the current block.

[0242] Clause 2B: The method of clause 1B, wherein filtering the prediction block based on the comparison of the template of the prediction block with the template of the current block to determine a filtered prediction block comprises: determining a filter that minimizes a difference between sample values of the template of the prediction block and sample values of the template of the current block; and filtering the prediction block using the determined filter.

[0243] Clause 3B: A method according to clause 2B, wherein determining the filter that minimizes the difference between the sample values of the template of the prediction block and the sample values of the template of the current block includes determining a filter that modifies the sample values of the template of the current block to reduce the mean squared error between the sample values of the template of the current block and the sample values of the template of the prediction block.

[0244] Clause 4B: The method of clause 1B, wherein filtering the prediction block based on the comparison of the template of the prediction block with the template of the current block to determine a filtered prediction block comprises: determining not to filter the prediction block based on the comparison of the template of the prediction block with the template of the current block, such that the filtered prediction block is equal to the prediction block.

[0245] Clause 5B: A method according to clause 1B, wherein the template of the prediction block includes an L-shaped sample group, wherein the L-shaped sample group includes samples to the left of a reference block used to determine the prediction block and samples above the reference block, and the template of the current block includes an L-shaped sample group, wherein the L-shaped sample group includes samples to the left of the current block and samples above the current block.

[0246] Clause 6B: The method of clause 1B, wherein determining the prediction block for the current block of the current picture of video data comprises locating a reference block in the same picture as the current block using a block vector.

[0247] Clause 7B: The method of clause 1B, wherein determining the prediction block for the current block of the current picture of video data comprises locating a reference block in a reference picture using a motion vector.

[0248] Clause 8B: The method of clause 1B, wherein decoding the current block based on the filtered prediction block comprises: adding residual data to the filtered prediction block to determine a reconstructed block; and applying one or more filter operations to the reconstructed block.

[0249] Clause 9B: The method of clause 1B, wherein the decoding method is performed as part of a process of encoding the current block of video data.

[0250] Item 10B: A device for decoding video data, the device comprising: a memory configured to store video data; one or more processors implemented in circuitry and configured to: determine a prediction block for a current block of a current picture of the video data; compare a template of the prediction block with a template of the current block; filter the prediction block based on the comparison of the template of the prediction block with the template of the current block to determine a filtered prediction block; decode the current block based on the filtered prediction block to determine a decoded version of the current block; and output a decoded picture of the video data including the decoded version of the current block.

[0251] Clause 11B: The apparatus of clause 10B, wherein to filter the prediction block based on the comparison of the template of the prediction block with the template of the current block to determine a filtered prediction block, the one or more processors are further configured to: determine a filter that minimizes a difference between sample values of the template of the prediction block and sample values of the template of the current block; and filter the prediction block using the determined filter.

[0252] Clause 12B: An apparatus according to clause 11B, wherein, in order to determine the filter that minimizes the difference between the sample values of the template of the prediction block and the sample values of the template of the current block, the one or more processors are further configured to determine a filter that modifies the sample values of the template of the current block to reduce a mean square error between the sample values of the template of the current block and the sample values of the template of the prediction block.

[0253] Clause 13B: The apparatus of clause 11B, wherein, in order to filter the prediction block based on the comparison of the template of the prediction block with the template of the current block to determine a filtered prediction block, the one or more processors are further configured to: determine not to filter the prediction block based on the comparison of the template of the prediction block with the template of the current block, so that the filtered prediction block is equal to the prediction block.

[0254] Clause 14B: An apparatus according to clause 11B, wherein the template of the prediction block includes an L-shaped sample group, wherein the L-shaped sample group includes samples to the left of a reference block used to determine the prediction block and samples above the reference block, and the template of the current block includes an L-shaped sample group, wherein the L-shaped sample group includes samples to the left of the current block and samples above the current block.

[0255] Clause 15B: The apparatus of clause 11B, wherein to determine the prediction block for the current block of the current picture of video data, the one or more processors are further configured to locate a reference block in the same picture as the current block using a block vector.

[0256] Clause 16B: The apparatus of clause 11B, wherein to determine the prediction block for the current block of the current picture of video data, the one or more processors are further configured to locate a reference block in a reference picture using a motion vector.

[0257] Clause 17B: The apparatus of clause 11B, wherein to decode the current block based on the filtered prediction block, the one or more processors are further configured to: add residual data to the filtered prediction block to determine a reconstructed block; and apply one or more filter operations to the reconstructed block.

[0258] Clause 18B: The apparatus of clause 10B, further comprising: a display configured to output decoded pictures of the video data.

[0259] Clause 19B: The apparatus of clause 10B, further comprising: a camera configured to capture unencoded video data; and wherein the one or more processors are further configured to encode the unencoded video data.

[0260] Item 20B: A computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to: determine a prediction block for a current block of a current picture of video data; compare a template of the prediction block with a template of the current block; filter the prediction block based on the comparison of the template of the prediction block with the template of the current block to determine a filtered prediction block; decode the current block based on the filtered prediction block to determine a decoded version of the current block; and output a decoded picture of video data including the decoded version of the current block.

[0261] Item 1C: A method of encoding or decoding video data, the method comprising: determining, for a current block within a current picture, an intra-block copy (IBC) reference block within the current picture, wherein a block vector of the current block indicates a displacement from the current block to the IBC reference block; based on enabling a filtering mode for the current block: calculating coefficients of a filter based on a reference template for the IBC reference block and a current template for the current block; and applying the filter to the IBC reference block; and after applying the filter to the IBC reference block, encoding or decoding the current block using the IBC reference block.

[0262] Clause 2C: The method of clause 1C, wherein applying the filter to the IBC reference block comprises disabling an IBC local illumination compensation (LIC) mode for the current block and disabling an IBC combined intra-inter prediction mode for the current block based on enabling the filtering mode for the current block.

[0263] Clause 3C: The method of any of clauses 1C to 2C, further comprising determining whether to apply the filter to the IBC reference block based on a size of the current block.

[0264] Clause 4C: The method of any of clauses 1C to 3C, further comprising determining whether to apply the filter to the IBC reference block based on whether a left template or an above template is fully available.

[0265] Clause 5C: A method according to any one of clauses 1C to 4C, wherein calculating the coefficients includes: based on only a portion of the reference template being within the IBC reference area of the current block or a portion of the current template being within the IBC reference area of the current block, calculating the coefficients of the filter using only the portion of the reference template being within the IBC reference area of the current block or the portion of the current template being within the IBC reference area of the current block.

[0266] Clause 6C: A method according to any one of clauses 1C to 4C, wherein calculating the coefficient includes: calculating the coefficient using padded samples and samples of the reference template or current template within the IBC reference area based on the fact that a portion of the reference template is not within the IBC reference area of the current block or a portion of the current template is not within the IBC reference area.

[0267] Clause 7C: The method of any of clauses 1C-6C, wherein, based on the block vector being a fraction, only an integer portion of the block vector is used to generate the reference template.

[0268] Clause 8C: The method of any of clauses 1C to 6C, further comprising determining to enable the filtering mode for the current block only when a block vector is at an integer pixel level.

[0269] Clause 9C: The method of any of clauses 1C to 8C, further comprising determining a filter model from a plurality of available filter models having different filter shapes, wherein the filter applied to the IBC reference block is the determined filter model.

[0270] Clause 10C: The method of Clause 9C, further comprising reordering the list of available filter models based on a difference between the filtered prediction and a reconstruction of the evaluation template.

[0271] Clause 11C: A method as set forth in any of clauses 1C to 10C, wherein the one or more syntax elements indicating application of filters to IBC reference blocks are signaled only when non-merge mode is used to signal block vectors.

[0272] Clause 12C: A method according to any one of clauses 1C-11C, wherein: the method further comprises generating a merge candidate list comprising intra block copy (IBC) candidates, wherein each of the IBC candidates indicates a corresponding block vector; determining the IBC reference block comprises determining the block vector of the current block from the block vectors indicated by the IBC candidates in the merge candidate list, and signaling a merge candidate index in the bitstream indicating the position of the block vector of the current block within the merge candidate list.

[0273] Clause 13C: The method of clause 12C, wherein: the selected IBC candidate indicates a block vector of the current block, and the method further comprises, based on the selected IBC candidate being a composite candidate, determining to enable filtering mode for the current block based on whether filtering mode is enabled for the composite candidate.

[0274] Clause 14C: The method of any of clauses 1C to 13C, wherein the filtering mode is enabled only for blocks in an I slice.

[0275] Clause 15C: A method according to any of clauses 1C-14C, wherein: a plurality of filter models are generated using a plurality of different sets of current templates and reference templates; and a filter model defining the filter is determined from the filter models.

[0276] Clause 16C: The method of any of clauses 1C to 15C, further comprising applying an additional filter to the current template and the reference template prior to calculating the coefficients.

[0277] Clause 17C: The method of any of clauses 1C-16C, wherein the constraint requires the filter to be symmetric.

[0278] Clause 18C: The method of any of clauses 1C to 17C, further comprising applying a set of offline trained fixed model filters to the IBC reference block.

[0279] Clause 19C: The method of any of clauses 1C to 18C, wherein luma and chroma samples of the current block are encoded together, and the filter is applied only to luma samples of the IBC reference block.

[0280] Clause 20C: The method of clause 19C, further comprising: computing coefficients of a second filter based on chroma samples of the reference template and chroma samples of the current template; and applying the second filter to the chroma samples of the IBC reference block.

[0281] Clause 21C: A method according to any one of clauses 1C to 11B or 14C to 20C, wherein: the method further includes generating an Advanced Motion Vector Prediction (AMVP) candidate list comprising AMVP candidates, wherein each of the AMVP candidates indicates a corresponding block vector; the block vector of the current block is defined by the block vector and motion vector difference (MVD) of a selected AMVP candidate in the AMVP candidate list; and the AMVP candidate index and MVD are signaled in the bitstream.

[0282] Clause 22C: The method of any one of clauses 1C to 21C, further comprising: filling an area of the extended reference area from an adjacent area of the available IBC reference area based on the block vector of the current block having fractional pixel accuracy and the extended reference area being partially outside the available IBC reference area.

[0283] Clause 23C: A method according to any one of clauses 1C-22C, wherein: the reference template is a selected reference template, and the method further comprises: generating an ordered list of fractional pixel candidates based on the block vector with fractional pixel accuracy, wherein each fractional pixel candidate corresponds to a corresponding reference template, and the fractional pixel candidates within the ordered list are ordered based on the difference between the current template and the corresponding reference template, wherein an index signaled in the bitstream indicates the position of the selected fractional pixel candidate in the ordered list.

[0284] Clause 24C: The method of any of clauses 1C to 23C, further comprising selecting the context of the filtering mode based on encoding modes of neighboring blocks.

[0285] Clause 25C: A method according to any one of clauses 1C to 24C, wherein the current block is a first block, the IBC reference block is a first IBC reference block, and the method further comprises: determining a second IBC reference block for a second block within the current picture, wherein a block vector of the second block indicates a displacement between the second block and the second IBC reference block; applying the fixed filter to the second IBC reference block based on at least one of the following: availability of a reference template for the second IBC reference block or similarity of the reference template for the second IBC reference block to a current template for the second block, wherein the fixed filter is based on fixed parameters; and after applying the fixed filter to the second IBC reference block, encoding or decoding the second block using the second IBC reference block.

[0286] Clause 26C: The method of Clause 25C, further comprising determining the fixed parameter based on a principal direction of the block vector of the second block.

[0287] Clause 27C: The method of clause 25C or 26C, further comprising determining the fixed parameter based on a gradient magnitude of the second IBC reference block or the reference template of the second IBC reference block.

[0288] Clause 28C: The method of any of clauses 1C to 27C, further comprising encoding or decoding a syntax element that indicates whether coding tools to determine the IBC reference blocks, calculate the coefficients, and apply the filters are enabled or disabled.

[0289] Clause 29C: The method of clause 28C, wherein the syntax element is conditionally signaled based on an intra block copy (IBC) flag.

[0290] Clause 30C: The method of any of clauses 28C to 29C, wherein the syntax element is a sequence parameter set (SPS) level syntax element.

[0291] Clause 31C: The method of any of clauses 28C to 29C, wherein the syntax element is a slice-level syntax element.

[0292] Clause 32C: An apparatus for encoding video data, the apparatus comprising one or more apparatus for performing the method of any of clauses 1C to 31C.

[0293] Clause 33C: The apparatus of clause 32C, wherein the one or more devices comprise one or more processors implemented in circuitry.

[0294] Clause 34C: The apparatus of any of clauses 32C and 33C, further comprising a memory for storing the video data.

[0295] Clause 35C: The apparatus of any of clauses 32C to 34C, further comprising a display configured to display the decoded video data.

[0296] Clause 36C: The device of any of clauses 32C-35C, wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

[0297] Clause 37C: The apparatus of any of clauses 32C-36C, wherein the apparatus comprises a video decoder.

[0298] Clause 38C: The apparatus of any of clauses 32C to 37C, wherein the apparatus comprises a video encoder.

[0299] Clause 39C: A computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to perform the method of any of clauses 1C-31C.

[0300] Item 1D: A method of decoding video data, the method comprising: determining a reference block for a current block of a current picture of the video data, wherein the reference block is in the current picture; determining a prediction block based on the reference block; determining whether to apply filtering to the prediction block; based on determining that the filtering will be applied to the prediction block: comparing a template of the reference block with a template of the current block; filtering the prediction block based on the comparison of the template of the reference block with the template of the current block to determine a filtered prediction block; decoding the current block based on the filtered prediction block to determine a decoded version of the current block; and outputting a decoded picture of the video data including the decoded version of the current block.

[0301] Clause 2D: A method according to clause 1D, wherein determining whether to apply the filtering to the prediction block includes receiving a flag, wherein a first value of the flag indicates that intra-frame block copying with filtering mode is enabled and a second value of the flag indicates that intra-frame block copying with filtering mode is disabled.

[0302] Clause 3D: The method of clause 1C or 2D, wherein determining whether to apply the filtering to the prediction block comprises determining a size of the current block, and determining to apply the filtering to the prediction block comprises determining that the size is greater than a threshold size.

[0303] Clause 4D: The method of any of clauses 1D-3D, wherein the template of the reference block comprises padded samples.

[0304] Clause 5D: A method according to any one of clauses 1D-4D, wherein: determining a prediction block for a current block of a current picture of video data comprises determining a block vector of the current block based on a candidate selected from a merge list; and determining whether to apply the filtering to the prediction block comprises determining whether a prediction block corresponding to the candidate is determined by the filtering.

[0305] Clause 6D: The method of any of clauses 1D to 6D, wherein determining whether to apply the filtering to the prediction block comprises determining whether to apply the filtering to the prediction block based on a slice type of the current block being an intra slice.

[0306] Clause 7D: A method according to any one of clauses 1D to 6D, wherein filtering the prediction block based on the comparison of the template of the reference block and the template of the current block to determine a filtered prediction block includes: determining a filter that minimizes the difference between sample values of the template of the reference block and sample values of the template of the current block; and filtering the prediction block using the determined filter.

[0307] Clause 8D: A method according to clause 7D, wherein determining the filter that minimizes the difference between the sample values of the template of the reference block and the sample values of the template of the current block includes determining a filter that modifies the sample values of the template of the current block to reduce the mean square error between the sample values of the template of the current block and the sample values of the template of the reference block.

[0308] Clause 9D: A method according to any one of clauses 1D to 8D, wherein filtering the prediction block based on a comparison of a template of a reference block with a template of a current block to determine a filtered prediction block includes: determining not to filter the prediction block based on a comparison of the template of the reference block with a template of the current block, so that the filtered prediction block is equal to the prediction block.

[0309] Clause 10D: A method according to any of clauses 1D-9D, wherein

[0310] The template of the reference block includes an L-shaped sample group including samples to the left of the reference block and samples above the reference block for determining the prediction block, and the template of the current block includes an L-shaped sample group including samples to the left of the current block and samples above the current block.

[0311] Clause 11D: The method of any of clauses 1D to 10D, wherein determining the prediction block for the current block of the current picture of the video data comprises locating a reference block in the same picture as the current block using the block vector.

[0312] Clause 12D: The method of any of clauses 1D-11D, wherein decoding the current block based on the filtered prediction block comprises: adding residual data to the filtered prediction block to determine a reconstructed block; and applying one or more filter operations to the reconstructed block.

[0313] Clause 13D: The method of any of clauses 1D or 3D to 12D, wherein the decoding method is performed as part of a process of encoding the current block of video data.

[0314] Item 14D: A device for decoding video data, the device comprising: a memory configured to store video data; one or more processors implemented in circuitry and configured to: determine a reference block for a current block of a current picture of the video data, wherein the reference block is in the current picture; determine a prediction block based on the reference block; determine whether to apply filtering to the prediction block; based on determining that the filtering will be applied to the prediction block: compare a template of the reference block with a template of the current block; filter the prediction block based on the comparison of the template of the reference block with the template of the current block to determine a filtered prediction block; decode the current block based on the filtered prediction block to determine a decoded version of the current block; and output a decoded picture of the video data including the decoded version of the current block.

[0315] Clause 15D: An apparatus according to clause 14D, wherein, to determine whether to apply the filtering to the prediction block, the one or more processors are further configured to receive a flag, wherein a first value of the flag indicates that intra-frame block copying with filtering mode is enabled and a second value of the flag indicates that intra-frame block copying with filtering mode is disabled.

[0316] Clause 16D: An apparatus according to clause 14D or 15D, wherein: to determine whether to apply the filtering to the prediction block, the one or more processors are further configured to determine a size of the current block; and to determine that the filtering will be applied to the prediction block, the one or more processors are further configured to determine that the size is greater than a threshold size.

[0317] Clause 17D: The apparatus of any of clauses 14D-16D, wherein the template of the reference block comprises a padded sample.

[0318] Clause 18D: An apparatus according to any one of clauses 14D-17D, wherein: in order to determine a prediction block for a current block of a current picture of video data, the one or more processors are further configured to determine a block vector for the current block based on a candidate selected from a merge list; and in order to determine whether to apply the filtering to the prediction block, the one or more processors are further configured to determine whether a prediction block corresponding to the candidate is determined by the filtering.

[0319] Clause 19D: The apparatus of any of clauses 14D-18D, wherein determining whether to apply the filtering to the prediction block comprises the one or more processors being further configured to determine whether to apply the filtering to the prediction block based on a slice type of the current block being an intra slice.

[0320] Clause 20D: An apparatus according to any one of clauses 14D to 19D, wherein in order to filter the prediction block based on the comparison of the template of the reference block and the template of the current block to determine a filtered prediction block, the one or more processors are further configured to: determine a filter that minimizes the difference between the sample values of the template of the reference block and the sample values of the template of the current block; and filter the prediction block using the determined filter.

[0321] Clause 21D: An apparatus according to clause 20D, wherein in order to determine a filter that minimizes the difference between sample values of the template of the reference block and sample values of the template of the current block, the one or more processors are further configured to determine a filter that modifies the sample values of the template of the current block to reduce the mean square error between the sample values of the template of the current block and the sample values of the template of the reference block.

[0322] Clause 22D: An apparatus according to any one of clauses 14D to 21D, wherein in order to filter the prediction block based on the comparison of the template of the reference block with the template of the current block to determine a filtered prediction block, the one or more processors are further configured to: determine not to filter the prediction block based on the comparison of the template of the reference block with the template of the current block, so that the filtered prediction block is equal to the prediction block.

[0323] Clause 23D: The apparatus of any of clauses 14D-22D, wherein

[0324] The template of the reference block includes an L-shaped sample group including samples to the left of the reference block and samples above the reference block for determining the prediction block, and the template of the current block includes an L-shaped sample group including samples to the left of the current block and samples above the current block.

[0325] Clause 24D: Apparatus according to any of clauses 14D to 23D, wherein to determine a prediction block for a current block of a current picture of the video data, the one or more processors are further configured to locate a reference block in the same picture as the current block using the block vector.

[0326] Clause 25D: An apparatus according to any of clauses 14D-24D, wherein, to decode the current block based on the filtered prediction block, the one or more processors are further configured to: add residual data to the filtered prediction block to determine a reconstructed block; and apply one or more filter operations to the reconstructed block.

[0327] Clause 26D: The apparatus of any of clauses 14D-25D, further comprising: a display configured to output decoded pictures of the video data.

[0328] Clause 27D: The apparatus of any of clauses 14D-26D, further comprising: a camera configured to capture unencoded video data; and wherein the one or more processors are further configured to encode the unencoded video data.

[0329] Item 28D: A computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to: determine a reference block for a current block of a current picture of video data, wherein the reference block is in the current picture; determine a prediction block based on the reference block; determine whether to apply filtering to the prediction block; based on determining that the filtering will be applied to the prediction block: compare a template of the reference block with a template of the current block; filter the prediction block based on the comparison of the template of the reference block with the template of the current block to determine a filtered prediction block; decode the current block based on the filtered prediction block to determine a decoded version of the current block; and output a decoded picture of video data including the decoded version of the current block.

[0330] Item 29D: A method of encoding video data, the method comprising: determining a reference block for a current block of a current picture of video data, wherein the reference block is in the current picture; determining a prediction block based on the reference block; determining whether to apply filtering to the prediction block; based on determining that the filtering will be applied to the prediction block: comparing a template of the reference block with a template of the current block; filtering the prediction block based on the comparison of the template of the reference block with the template of the current block to determine a filtered prediction block; decoding the current block based on the filtered prediction block to determine a decoded version of the current block; storing a decoded picture of video data including the decoded version of the current block; and encoding a subsequent block of video data based on the stored decoded picture.

[0331] It should be appreciated that, depending on the example, certain actions or events of any of the techniques described herein may be performed in a different order, may be added, combined, or omitted entirely (e.g., not all described actions or events are required to practice the techniques). Furthermore, in some examples, actions or events may be performed simultaneously rather than sequentially, for example, through multithreading, interrupt processing, or multiple processors.

[0332] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or sent via a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to tangible media such as data storage media, or communication media including any media that facilitates, for example, the transfer of a computer program from one place to another according to a communication protocol. In this manner, computer-readable media may generally correspond to (1) a non-transitory, tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementation of the techniques described in this disclosure. A computer program product may include computer-readable media.

[0333] By way of example, and not limitation, such computer-readable storage media may include one or more of RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Furthermore, any connection is properly termed a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies (such as infrared, radio, and microwave), then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies (such as infrared, radio, and microwave) are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but rather refer to non-transitory, tangible storage media. As used herein, disk and optical disk include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disks typically reproduce data magnetically, while optical discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0334] The instructions may be executed by one or more processors, such as one or more DSPs, general-purpose microprocessors, ASICs, FPGAs, or other equivalent integrated or discrete logic circuits. Thus, as used herein, the terms "processor" and "processing circuitry" may refer to any of the aforementioned structures or any other structure suitable for the implementation techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Furthermore, the techniques may be fully implemented in one or more circuits or logic elements.

[0335] The techniques of this disclosure can be implemented in a wide variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs), or collections of ICs (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but they do not necessarily require implementation by distinct hardware units. Specifically, as described above, the various units can be combined in a codec hardware unit in conjunction with appropriate software and / or firmware, or provided by a collection of interoperating hardware units (including one or more processors as described above).

[0336] Various examples have been described. These and other examples are within the scope of the following clauses.

Claims

1. A method for decoding video data, the method comprising: determining a reference block for a current block of a current picture of video data, wherein the reference block is in the current picture; determining a prediction block based on the reference block; determining whether to apply filtering to the prediction block; Based on determining that the filtering is to be applied to the prediction block: comparing the template of the reference block with the template of the current block; filtering the prediction block based on a comparison of a template of the reference block and a template of the current block to determine a filtered prediction block; decoding the current block based on the filtered prediction block to determine a decoded version of the current block; as well as A decoded picture of the video data including the decoded version of the current block is output.

2. The method according to claim 1, wherein Determining whether to apply the filtering to the prediction block includes receiving a flag, wherein a first value of the flag indicates that intra block copy with filtering mode is enabled and a second value of the flag indicates that the intra block copy with filtering mode is disabled.

3. The method according to claim 1, wherein Determining whether to apply the filtering to the prediction block includes determining a size of the current block, and determining that the filtering is to be applied to the prediction block includes determining that the size is greater than a threshold size.

4. The method according to claim 1, wherein The template of the reference block includes padded samples.

5. The method according to claim 1, wherein: determining a prediction block for the current block of the current picture of video data comprises determining a block vector for the current block based on a candidate selected from a merge list; as well as Determining whether to apply the filtering to the prediction block includes determining whether a prediction block corresponding to the candidate is determined by the filtering.

6. The method according to claim 1, wherein Determining whether to apply the filtering to the prediction block includes determining whether to apply the filtering to the prediction block based on a slice type of the current block being an intra slice.

7. The method according to claim 1, wherein Filtering the prediction block based on a comparison of a template of the reference block and a template of the current block to determine a filtered prediction block includes: determining a filter that minimizes a difference between sample values of the template of the reference block and sample values of the template of the current block; and The prediction block is filtered using the determined filter.

8. The method according to claim 7, wherein: Determining a filter that minimizes a difference between sample values of the template of the reference block and sample values of the template of the current block includes determining a filter that modifies sample values of the template of the current block to reduce a mean square error between sample values of the template of the current block and sample values of the template of the reference block.

9. The method according to claim 1, wherein Filtering the prediction block based on a comparison of a template of the reference block and a template of the current block to determine a filtered prediction block includes: It is determined based on a comparison of a template of the reference block and a template of the current block that the prediction block is not filtered, so that the filtered prediction block is equal to the prediction block.

10. The method according to claim 1, wherein The template of the reference block includes an L-shaped sample group including samples on the left side of the reference block used to determine the prediction block and samples above the reference block, and The template of the current block includes an L-shaped sample group including samples on the left side of the current block and samples above the current block.

11. The method according to claim 1, wherein Determining the prediction block for the current block of the current picture of video data includes locating a reference block in the same picture as the current block using a block vector.

12. The method according to claim 1, wherein Decoding the current block based on the filtered prediction block includes: adding residual data to the filtered prediction block to determine a reconstructed block; and One or more filter operations are applied to the reconstructed block.

13. The method according to claim 1, wherein The method of decoding is performed as part of the process of encoding the current block of video data.

14. A device for decoding video data, the device comprising: a memory configured to store video data; One or more processors implemented in circuitry and configured to: determining a reference block for a current block of a current picture of video data, wherein the reference block is in the current picture; determining a prediction block based on the reference block; determining whether to apply filtering to the prediction block; Based on determining that the filtering is to be applied to the prediction block: comparing the template of the reference block with the template of the current block; filtering the prediction block based on a comparison of a template of the reference block and a template of the current block to determine a filtered prediction block; decoding the current block based on the filtered prediction block to determine a decoded version of the current block; as well as A decoded picture of the video data including the decoded version of the current block is output.

15. The apparatus according to claim 14, wherein To determine whether to apply the filtering to the prediction block, the one or more processors are further configured to receive a flag, wherein a first value of the flag indicates that intra-block copying with filtering mode is enabled, and a second value of the flag indicates that intra-block copying with filtering mode is disabled.

16. The apparatus of claim 14, wherein: To determine whether to apply the filtering to the prediction block, the one or more processors are further configured to determine a size of the current block; and To determine that the filtering is to be applied to the prediction block, the one or more processors are further configured to determine that the size is greater than a threshold size.

17. The apparatus according to claim 14, wherein The template of the reference block includes padded samples.

18. The apparatus of claim 14, wherein: To determine the prediction block for the current block of the current picture of video data, the one or more processors are further configured to determine a block vector for the current block based on a candidate selected from a merge list; as well as In order to determine whether to apply the filtering to the prediction block, the one or more processors are further configured to determine whether a prediction block corresponding to the candidate is determined by the filtering.

19. The apparatus according to claim 14, wherein Determining whether to apply the filtering to the prediction block includes the one or more processors being further configured to determine whether to apply the filtering to the prediction block based on a slice type of the current block being an intra slice.

20. The apparatus of claim 14, wherein To filter the prediction block based on a comparison of the template of the reference block and the template of the current block to determine a filtered prediction block, the one or more processors are further configured to: determining a filter that minimizes a difference between sample values of the template of the reference block and sample values of the template of the current block; and The prediction block is filtered using the determined filter.

21. The apparatus according to claim 20, wherein In order to determine a filter that minimizes the difference between the sample values of the template of the reference block and the sample values of the template of the current block, the one or more processors are further configured to determine a filter that modifies the sample values of the template of the current block to reduce the mean square error between the sample values of the template of the current block and the sample values of the template of the reference block.

22. The apparatus of claim 14, wherein: To filter the prediction block based on a comparison of the template of the reference block and the template of the current block to determine a filtered prediction block, the one or more processors are further configured to: It is determined based on a comparison of a template of the reference block and a template of the current block that the prediction block is not filtered, so that the filtered prediction block is equal to the prediction block.

23. The apparatus of claim 14, wherein: The template of the reference block includes an L-shaped sample group including samples on the left side of the reference block used to determine the prediction block and samples above the reference block, and The template of the current block includes an L-shaped sample group including samples on the left side of the current block and samples above the current block.

24. The apparatus of claim 14, wherein: To determine the prediction block for the current block of the current picture of video data, the one or more processors are further configured to locate a reference block in the same picture as the current block using a block vector.

25. The apparatus of claim 14, wherein: To decode the current block based on the filtered prediction block, the one or more processors are further configured to: adding residual data to the filtered prediction block to determine a reconstructed block; and One or more filter operations are applied to the reconstructed block.

26. The apparatus of claim 14, further comprising: A display is configured to output the decoded pictures of the video data.

27. The apparatus of claim 14, further comprising: a camera configured to capture unencoded video data; and The one or more processors are further configured to encode the unencoded video data.

28. A computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to: determining a reference block for a current block of a current picture of video data, wherein the reference block is in the current picture; determining a prediction block based on the reference block; determining whether to apply filtering to the prediction block; Based on determining that the filtering is to be applied to the prediction block: comparing the template of the reference block with the template of the current block; filtering the prediction block based on a comparison of a template of the reference block and a template of the current block to determine a filtered prediction block; decoding the current block based on the filtered prediction block to determine a decoded version of the current block; as well as A decoded picture of the video data including the decoded version of the current block is output.

Images