Method, apparatus and computer program for video coding using boundary filtering

Abstract

Aspects of the present disclosure include methods and apparatuses for video coding. One of the apparatuses includes a processing circuit that receives a coded video bitstream including a current picture having a block. The processing circuit determines a predictive block for the block using one of an intra block copy (IBC) mode and an intra template matching (IntraTMP) mode. If boundary filtering is to be applied to the block, the processing circuit applies boundary filtering to a predictive sample pred(x',y') located at a position (x',y') in the predictive block by determining a parameter W for boundary filtering based on coding information of the block, determining a weight to be used in the boundary filtering by right-shifting the parameter W according to a position (x',y'), and generating a filtered predictive sample based on a linear combination of the reference sample and the predictive sample according to the determined weight.

Description

[Technical Field]

[0001] [Incorporated by reference] This application claims benefit of priority to U.S. Patent Application No. 18 / 380,011, entitled "BOUNDARY FILTERING ON INTRABC AND INTRATMP CODED BLOCKS," filed October 13, 2023, which claims benefit of priority to U.S. Provisional Patent Application No. 63 / 416,905, entitled "PDPC on IntraBC and IntraTMP coded blocks," filed October 17, 2022. The disclosures of the prior applications are incorporated herein by reference in their entireties.

[0002] [Technical field] This disclosure describes aspects generally related to video coding. [Background technology]

[0003] The background discussion provided herein is intended to generally present the context for the present disclosure. The work of the presently named inventors, to the extent that that work is described in this background discussion, along with aspects of the description that would not normally be considered prior art at the time of filing, is not admitted explicitly or implicitly as prior art to the present disclosure.

[0004] Image / video compression helps transmit image / video data across different devices, storage, and networks with minimal quality degradation. In some examples, video codec techniques can compress video based on spatial and temporal redundancy. In one example, a video codec can use a technique called intra-prediction, which can compress images based on spatial redundancy. For example, intra-prediction can use reference data from the current picture being reconstructed for sample prediction. In another example, a video codec can use a technique called inter-prediction, which can compress images based on temporal redundancy. For example, inter-prediction can predict samples in a current picture from a previously reconstructed picture using motion compensation. Motion compensation can be indicated by a motion vector (MV). Summary of the Invention

[0005] Aspects of the present disclosure include methods and apparatuses for video encoding / decoding. In some examples, the apparatus for video decoding includes a processing circuit. The processing circuit receives a coded video bitstream including a current picture having a block coded in one of an intra block copy (IBC) mode and an intra template matching (IntraTMP) mode. The processing circuit determines a predictive block for the block using the one of the IBC mode and the IntraTMP mode. In response to boundary filtering being applied to the block, the processing circuit applies boundary filtering to a predictive sample pred(x',y') located at a position (x',y') in the predictive block corresponding to a sample in the block by determining a boundary filtering parameter W based on coding information of the block, determining a weight to be used in the boundary filtering by right-shifting the parameter W according to the position (x',y'), and generating a filtered predictive sample based on a linear combination of the reference sample and the predictive sample according to the determined weight. In one example, the coding information of a block includes a coding block size, a coding block aspect ratio, whether the block is a luma or chroma component, neighboring reconstructed samples of the block, boundary predicted samples within the predicted block, differences between neighboring reconstructed samples of the block and boundary predicted samples within the predicted block, or the color format of the block.

[0006] In one embodiment, the weights are determined by the left reference sample R -1,y’ The weight w associated with L and the above reference sample R x’,-1 The weight w associated with T and the weight (64-w L -w T ) and weight w L is equal to W>>((x'<<1)>>0), and the weight w Tis equal to W>>((y'<<1)>>0).

[0007] In one aspect, the boundary filtering is performed by a position-dependent predictor combination (PDPC) filter, and the processing circuitry filters the filtered prediction samples as Clip(0,(1< <BitDepth)-1,(w L ×R -1,y’ +w T ×R x’,-1 +(64-w L -w T )×pred(x',y')+32)>>6), where BitDepth indicates the bit depth.

[0008] In one example, the parameter W is determined as 8, 4, 16 or 2.

[0009] In one example, the processing circuitry determines whether boundary filtering is applied to the block from adjacent reconstructed samples.

[0010] In one example, the processing circuit detects a content type of an adjacent reconstructed sample, does not apply boundary filtering to the block in response to the content type of the adjacent reconstructed sample being screen content, and applies boundary filtering to the block in response to the content type of the adjacent reconstructed sample not being screen content.

[0011] In one example, the processing circuit checks the number of color values ​​of adjacent reconstructed samples, and in response to the number of color values ​​being less than a color value threshold, the processing circuit detects the content type as screen content.

[0012] In one example, the adjacent reconstructed samples are of a particular color component, and the color value comprises a value of the particular color component.

[0013] In one example, adjacent reconstructed samples are associated with multiple color components, and the color value comprises a combination of the respective values ​​of the multiple color components.

[0014] In one example, boundary filtering is applied only when the color component of the block is the luma component.

[0015] In one example, boundary filtering is applied to each color component associated with a block.

[0016] In one example, boundary filtering is applied only when the current slice containing the block is an intra-slice.

[0017] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions that, when executed by a computer, cause the computer to perform a method for video decoding / encoding. [Brief explanation of the drawings]

[0018] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings.

[0019] [Figure 1] FIG. 1 is a schematic diagram of an exemplary block diagram of a video processing system (100).

[0020] [Figure 2] FIG. 2 is a schematic diagram of an exemplary block diagram of a decoder.

[0021] [Figure 3] FIG. 2 is a schematic diagram of an exemplary block diagram of an encoder.

[0022] [Figure 4] FIG. 10 illustrates an example of an intra-template matching prediction (IntraTMP) mode according to aspects of the present disclosure.

[0023] [Figure 5A]FIG. 10 illustrates examples of reference samples for position-dependent predictor combining (PDPC) modes applied to various prediction modes. [Figure 5B] FIG. 10 illustrates examples of reference samples for position-dependent predictor combining (PDPC) modes applied to various prediction modes. [Figure 5C] FIG. 10 illustrates examples of reference samples for position-dependent predictor combining (PDPC) modes applied to various prediction modes. [Figure 5D] FIG. 10 illustrates examples of reference samples for position-dependent predictor combining (PDPC) modes applied to various prediction modes.

[0024] [Figure 6] FIG. 10 illustrates an example of boundary filtering using a two tap filter according to an embodiment of the present invention.

[0025] [Figure 7] FIG. 10 illustrates an example of boundary filtering according to aspects of the present disclosure.

[0026] [Figure 8] FIG. 1 shows a flowchart outlining a process according to certain aspects of the present disclosure.

[0027] [Figure 9] FIG. 10 shows a flowchart outlining another process according to an aspect of the present disclosure.

[0028] [Figure 10] FIG. 1 is a schematic diagram of a computer system according to one aspect. DETAILED DESCRIPTION OF THE INVENTION

[0029] 1 illustrates a block diagram of a video processing system 100 in accordance with some examples. The video processing system 100 is an example of an application of the disclosed subject matter, a video encoder and decoder in a streaming environment. The disclosed subject matter may be equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, streaming services, and storage of compressed video on digital media, including CDs, DVDs, memory sticks, etc.

[0030] The video processing system (100) includes a capture subsystem (113), which may include a video source (101), such as a digital camera, that creates a stream of uncompressed video pictures (102). In one example, the stream of video pictures (102) includes samples captured by the digital camera. The stream of video pictures (102) is depicted as a thick line to emphasize its high data volume compared to the encoded video data (104) (or coded video bitstream) and may be processed by an electronic device (120) that includes a video encoder (103) coupled to the video source (101). The video encoder (103) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video data (104) (or coded video bitstream), depicted as a thin line to emphasize its low data volume compared to the stream of video pictures (102), may be stored on a streaming server (105) for future use. One or more streaming client subsystems, such as the client subsystems (106) and (108) of Figure 1, can access the streaming server (105) to retrieve copies (107) and (109) of the encoded video data (104). The client subsystem (106) can include a video decoder (110), for example, within an electronic device (130). The video decoder (110) decodes an input copy (107) of the encoded video data and creates an output stream (111) of video pictures that can be rendered on a display (112) (e.g., a display screen) or other rendering device (not shown). In some streaming systems, the encoded video data (104), (107), and (109) (e.g., a video bitstream) can be encoded according to a particular video coding / compression standard. Examples of these standards include ITU-T Recommendation H.265.In one example, a video coding standard under development is informally known as Versatile Video Coding (VVC), and the disclosed subject matter may be used in the context of VVC.

[0031] It should be noted that electronic devices 120 and 130 may include other components (not shown). For example, electronic device 120 may include a video decoder (not shown), and similarly, electronic device 130 may also include a video encoder (not shown).

[0032] 2 shows an exemplary block diagram of a video decoder (210). The video decoder (210) can be included in an electronic device (230). The electronic device (230) can include a receiver (231) (e.g., a receiving circuit). The video decoder (210) can be used in place of the video decoder (110) in the example of FIG. 1.

[0033] The receiver (231) may receive one or more coded video sequences to be decoded by the video decoder (210), for example, contained in a bitstream. In one aspect, it receives one coded video sequence at a time, where the decoding of each coded video sequence is independent of the decoding of the other coded video sequences. The coded video sequences may be received from a channel (201), which may be a hardware / software link to a storage device that stores the coded video data. The receiver (231) may receive the coded video data along with other data, such as coded audio data and / or auxiliary data streams, which may be forwarded to their respective using entities (not shown). The receiver (231) may separate the coded video sequences from other data. To combat network jitter, a buffer memory (215) may be coupled between the receiver (231) and the entropy decoder / parser (220) (hereinafter, "parser (220)"). In certain applications, the buffer memory (215) is part of the video decoder (210). In other cases, it may be external to the video decoder 210 (not shown). In still other cases, there may be a buffer memory (not shown) external to the video decoder 210, for example, to combat network jitter, and there may be another buffer memory 215 internal to the video decoder 210, for example, to handle playback timing. When the receiver 231 is receiving data from a store-and-forward device with sufficient bandwidth and controllability or from an isosynchronous network, the buffer memory 215 may not be needed or may be small.For use in best-effort packet networks such as the Internet, a buffer memory (215) may be required, the size of which may be relatively large, advantageously adaptively sized, and may be implemented, at least in part, in an operating system or similar element (not shown) external to the video decoder (210).

[0034] The video decoder (210) may include a parser (220) to reconstruct symbols (221) from the coded video sequence. These symbol categories include information used to manage the operation of the video decoder (210) and, potentially, information to control a rendering device, such as a render device (212) (e.g., a display screen) that is not an integral part of the electronic device (230) as shown in FIG. 2 but may be coupled to the electronic device (230). The rendering device control information may be in the form of a Supplemental Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (220) may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may follow a video coding technique or standard and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (220) may extract from the coded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on at least one parameter corresponding to the group. The subgroups may include groups of pictures (GOPs), pictures, tiles, slices, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. The parser (220) may also extract from the coded video sequence information such as transform coefficients, quantization parameter values, motion vectors, etc.

[0035] The parser (220) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (215) to create symbols (221).

[0036] The reconstruction of the symbols (221) can involve several different units, depending on the type of coded video picture or portion thereof (e.g., inter- and intra-picture, inter- and intra-block) and other factors. Which units are involved and how can be controlled by subgroup control information parsed by the parser (220) from the coded video sequence. The flow of such subgroup control information between the parser (220) and the units below is not shown for clarity.

[0037] In addition to the functional blocks already mentioned, the video decoder (210) may be conceptually subdivided into multiple functional units, as described below. In a practical implementation operating under commercial constraints, many of these units will interact closely with each other and may be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, a conceptual subdivision into functional units is appropriate below.

[0038] The first unit may be a scalar / inverse transform unit (251), which receives quantized transform coefficients as symbols (221) from the parser (220), as well as control information including which transform to use, block size, quantization coefficients, quantization scaling matrices, etc. The scalar / inverse transform unit (251) may output blocks containing sample values ​​that can be input to an aggregator (255).

[0039] In some cases, the output samples of the scaler / inverse transform unit (251) may relate to intra-coded blocks. Intra-coded blocks are blocks that do not use prediction information from a previously reconstructed picture but can use prediction information from a previously reconstructed portion of the current picture. Such prediction information can be provided by the intra-picture prediction unit (252). In some cases, the intra-picture prediction unit (252) generates blocks of the same size and shape as the block being reconstructed using surrounding, already reconstructed information fetched from the current picture buffer (258). The current picture buffer (258) may be, for example, a partially reconstructed and / or fully reconstructed current picture. The aggregator (255) optionally adds, on a sample-by-sample basis, the prediction information generated by the intra-prediction unit (252) to the output sample information provided by the scaler / inverse transform unit (251).

[0040] In other cases, the output samples of the scalar / inverse transform unit (251) may relate to a block that is inter-coded and potentially motion-compensated. In such cases, the motion-compensated prediction unit (253) can access the reference picture memory (257) to fetch samples used for prediction. After motion-compensating the fetched samples according to the symbols (221) associated with the block, these samples can be added by the aggregator (255) to the output of the scalar / inverse transform unit (251) (in this case, referred to as residual samples or residual signals) to generate output sample information. The addresses in the reference picture memory (257) from which the motion-compensated prediction unit (253) fetches the prediction samples can be controlled by a motion vector, which is available to the motion-compensated prediction unit (253) in the form of a symbol (221) that can have, for example, X, Y, and reference picture components. Motion compensation can also include interpolation of sample values ​​fetched from the reference picture memory (257) when sub-sample accurate motion vectors are used, motion vector prediction mechanisms, etc.

[0041] The output samples of the aggregator (255) may be subjected to various loop filtering techniques in a loop filter unit (256). Video compression techniques may include in-loop filtering techniques controlled by parameters contained in the coded video sequence (also called a coded video bitstream) and made available to the loop filter unit (256) as symbols (221) from the parser (220). Video compression may also be responsive to meta-information obtained during decoding of previous portions (in decoding order) of the coded picture or coded video sequence, as well as to previously reconstructed loop-filtered sample values.

[0042] The output of the loop filter unit (256) can be a sample stream that can be output to a render device such as a display (212) and can be stored in a reference picture memory (257) for use in future inter-picture prediction.

[0043] Once a particular coded picture is fully reconstructed, it can be used as a reference picture for future prediction. For example, once the coded picture corresponding to the current picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the parser (220)), the current picture buffer (258) can become part of the reference picture memory (257), and the fresh current picture buffer can be reallocated before beginning reconstruction of a subsequent coded picture.

[0044] The video decoder (210) may perform decoding operations according to a given video compression technology or standard, such as ITU-T Rec. H.265. A coded video sequence may conform to the syntax specified by the video compression technology or standard being used, in the sense that the coded video sequence conforms to both the syntax of the video compression technology or standard and a profile documented in the video compression technology or standard. Specifically, a profile may select certain tools from all tools available in the video compression technology or standard as the only tools available for use under that profile. Compliance also requires that the complexity of the coded video sequence be within a range defined by the level of the video compression technology or standard. In some cases, the level may constrain the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference pixel size, etc. The limits set by the level may be further constrained, in some cases, through a hypothetical reference decoder (HRD) specification and metadata for HRD buffer management signaled with the coded video sequence.

[0045] In one aspect, the receiver (231) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the coded video sequence. The additional data may be used by the video decoder (210) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0046] 3 shows an exemplary block diagram of a video encoder (303). The video encoder (303) is included in an electronic device (320). The electronic device (320) includes a transmitter (340) (e.g., a transmission circuit). The video encoder (303) can be used in place of the video encoder (103) in the example of FIG. 1.

[0047] The video encoder (303) may receive video samples from a video source (301) (which, in the example of FIG. 3, is not part of the electronic device (320)) that may capture video images to be coded by the video encoder (303). In another example, the video source (301) is part of the electronic device (320).

[0048] The video source (301) may provide a source video sequence to be coded by the encoder (303) in the form of a digital video sample stream, which may be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media delivery system, the video source (301) may be a storage device that stores pre-prepared video. In a video conferencing system, the video source (301) may be a camera that captures local image information as a video sequence. The video data may be provided as multiple individual pictures that convey motion when viewed in sequence. The pictures themselves may be organized as a spatial array of pixels, where each pixel may comprise one or more samples depending on the sampling structure, color space, etc., in use.

[0049] According to one aspect, the video encoder (303) can code and compress pictures of a source video sequence into a coded video sequence (343) in real time or under any other required time constraints. Enforcing the appropriate coding rate is one function of the controller (350). In some aspects, the controller (350) can also control and be functionally coupled to other functional units, as described below. This coupling is not shown for clarity. Parameters set by the controller (350) can include rate control-related parameters (picture skip, quantizer, lambda value for rate-distortion optimization techniques, ...), picture size, group-of-picture (GOP) layout, maximum motion vector search range, etc. The controller (350) can be configured with other appropriate functions that may be associated with an optimized video encoder (303) for a particular system design.

[0050] In some aspects, the video encoder is configured to operate within a coding loop. As an oversimplified explanation, in one example, the coding loop can include a source coder (330) (responsible for creating symbols, such as a symbol stream, based on an input picture to be coded and a reference picture, for example) and a (local) decoder (333) embedded in the video encoder (303). The decoder (333) reconstructs the symbols to create sample data in a manner similar to that which a (remote) decoder would also create. The reconstructed sample stream (sample data) can be input to a reference picture memory (334). Because decoding of the symbol stream yields bit-exact results independent of the decoder location (local or remote), the contents in the reference picture memory (334) are also bit-exact between the local and remote encoders. In other words, the predictive portion of the encoder "sees" the exact same sample values ​​as the decoder "sees" when using prediction during decoding. This basic principle of reference picture synchronicity (and the resulting drift when synchronicity cannot be maintained due to, for example, channel errors) is used in several related fields as well.

[0051] The operation of the "local" decoder (333) may be the same as a "remote" decoder, such as the video decoder (210), already described above in connection with Figure 2. However, briefly referring also to Figure 2, because symbols are available and the encoding / decoding of symbols into a coded video sequence by the entropy coder (345) and parser (220) may be lossless, the entropy decoding portion of the video decoder (210), including the buffer memory (215) and parser (220), may not be fully implemented in the local decoder (333).

[0052] In one aspect, decoder techniques other than analysis / entropy decoding present in a decoder are present in the same or substantially the same functional form in the corresponding encoder. Therefore, the disclosed subject matter focuses on the operation of the decoder. A description of the encoder techniques can be omitted, as opposed to the decoder techniques, which are exhaustively described. Only in certain areas will more detailed descriptions be provided below.

[0053] During operation, in some examples, the source coder (330) may perform motion-compensated predictive coding, which predictively codes an input picture with respect to one or more previously coded pictures from a video sequence designated as “reference pictures.” In this manner, the coding engine (332) codes differences between pixel blocks of the input picture and pixel blocks of reference pictures that may be selected as predictive references for the input picture.

[0054] The local video decoder (333) may decode coded video data of pictures that may be designated as reference pictures based on symbols created by the source coder (330). The operation of the coding engine (332) may advantageously be a lossy process. When the coded video data is decoded by a video decoder (not shown in FIG. 3), the reconstructed video sequence may typically be a replica of the source video sequence with some errors. The local video decoder (333) may replicate the decoding process that may be performed by the video decoder on the reference pictures and store the reconstructed reference pictures in the reference picture memory (334). In this way, the video encoder (303) may locally store copies of reconstructed reference pictures with common content as reconstructed reference pictures to be obtained by the far-end video decoder (without transmission errors).

[0055] The predictor (335) may perform a predictive search for the coding engine (332). That is, for a new picture to be coded, the predictor (435) may search the reference picture memory (434) for sample data (as candidate reference pixel blocks) or specific metadata, such as reference picture motion vectors, block shapes, etc., which may serve as appropriate prediction references for the new picture. The predictor (335) may operate on a sample block-by-pixel block basis to find appropriate prediction references. In some cases, as determined by the search results obtained by the predictor (335), the input picture may have prediction references drawn from multiple reference pictures stored in the reference picture memory (334).

[0056] The controller (350) may manage the coding operations of the source coder (330), including, for example, setting the parameters and subgroup parameters used to encode the video data.

[0057] The outputs of all of the aforementioned functional units may be subject to entropy coding in an entropy coder (345), which converts the symbols produced by the various functional units into a coded video sequence by applying lossless compression to the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc.

[0058] The transmitter (340) may buffer the coded video sequence produced by the entropy coder (345) and prepare it for transmission over a communication channel (360), which may be a hardware / software link to a storage device that stores the encoded video data. The transmitter (340) may merge the coded video data from the video encoder (303) with other data to be transmitted, such as coded audio data and / or auxiliary data streams (sources not shown).

[0059] The controller (350) may manage the operation of the video encoder (303). During coding, the controller (350) may assign each coded picture a particular coded picture type, which may affect the coding that may be applied to the respective picture. For example, pictures may often be assigned as one of the following picture types:

[0060] An intra picture (I picture) can be coded and decoded without using any other picture in the sequence as a source of prediction. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh (IDR) pictures.

[0061] Predictive pictures (P pictures) can be coded and decoded using intra- or inter-prediction, using motion vectors and reference indices to predict the sample values ​​of each block.

[0062] Bidirectionally predictive pictures (B-pictures) can be coded and decoded using intra- or inter-prediction, using two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple-predictive pictures can use more than two reference pictures and associated metadata for the reconstruction of a single block.

[0063] A source picture is typically spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and may be coded block by block. Blocks may be predictively coded relative to other (already coded) blocks, as determined by the coding assignment applied to the block's respective picture. For example, blocks of an I-picture may be non-predictively coded, or they may be predictively coded relative to already coded blocks of the same picture (spatial prediction or intra-prediction). Pixel blocks of a P-picture may be predictively coded via spatial prediction or via temporal prediction relative to one previously coded reference picture. Blocks of a B-picture may be predictively coded via spatial prediction or via temporal prediction relative to one or two previously coded reference pictures.

[0064] The video encoder (303) may perform coding operations according to a predetermined video coding technique or standard, such as ITU-T Rec. H.265. In doing so, the video encoder (303) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence. The coded video data may therefore conform to a syntax specified by the video coding technique or standard being used.

[0065] In one aspect, the transmitter (340) may transmit additional data along with the encoded video. The source coder (330) may include such data as part of the coded video sequence. The additional data may include other types of redundant data, such as temporal / spatial / SNR enhancement layers, redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0066] Video may be captured as multiple source pictures (video pictures) in a time sequence. Intra-picture prediction (often abbreviated as intra-prediction) exploits spatial correlation within a given picture, while inter-picture prediction exploits correlation (temporal or otherwise) between pictures. In one example, a particular picture being encoded / decoded is called the current picture and is partitioned into blocks. When a block in the current picture is similar to a reference block in a previously coded and still buffered reference picture in the video, the block in the current picture can be coded by a vector called a motion vector. A motion vector points to a reference block within the reference picture and may have a third dimension that identifies the reference picture if multiple reference pictures are used.

[0067] In some aspects, bi-prediction techniques can be used for inter-picture prediction. According to bi-prediction techniques, two reference pictures, such as a first reference picture and a second reference picture, are used that are located before a current picture in a video in decoding order (but are located past and future, respectively, in display order). A block in the current picture can be coded by a first motion vector that points to a first reference block in the first reference picture and a second motion vector that points to a second reference block in the second reference picture. A block can be predicted by a combination of the first and second reference blocks.

[0068] Furthermore, merge mode techniques can be used in inter-picture prediction to improve coding efficiency.

[0069] According to some aspects of the present disclosure, prediction, such as inter-picture prediction and intra-picture prediction, is performed on a block-by-block basis. For example, according to the HEVC standard, pictures in a sequence of video pictures are partitioned into coding tree units (CTUs) for compression, and the CTUs within a picture have the same size, such as 64x64 pixels, 32x32 pixels, or 16x16 pixels. Generally, a CTU includes three coding tree blocks (CTBs), one luma CTB and two chroma CTBs. Each CTU can be recursively quadtree partitioned into one or more coding units (CUs). For example, a 64x64 pixel CTU can be partitioned into one CU of 64x64 pixels, four CUs of 32x32 pixels, or 16 CUs of 16x16 pixels. In one example, each CU is analyzed to determine the prediction type of the CU, such as an inter-prediction type or an intra-prediction type. A CU is divided into one or more prediction units (PUs) according to temporal and / or spatial predictability. Generally, each PU includes a luma prediction block (PB) and two chroma PBs. In one aspect, prediction operations in coding (encoding / decoding) are performed in units of prediction blocks. Using a luma prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values ​​(e.g., luma values), such as 8x8 pixels, 16x16 pixels, 8x16 pixels, 16x8 pixels, etc.

[0070] It should be noted that the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using any suitable technology. In one aspect, the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using one or more integrated circuits. In another aspect, the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using one or more processors executing software instructions.

[0071] The Intra Block Copy (IBC) mode can be used in image and / or video coding, such as Versatile Video Coding (VVC). The IBC concept was previously incorporated into the High Efficiency Video Coding (HEVC) standard. However, some related technologies used in HEVC required implementation costs to be reduced due to the overall size of the already reconstructed area of ​​the current picture. A drawback of the IBC concept in some implementations, such as HEVC, is the requirement for additional memory in the decoded picture buffer (DPB), which hardware implementations may use external memory. The additional external memory accesses entail increased memory bandwidth. Some implementations, such as VVC, use fixed memory, which can realize the IBC mode by using on-chip memory, significantly reducing memory bandwidth requirements and hardware complexity. A reference sample memory (RSM) can be used to store samples of a single CTU. A special feature of the RSM includes a continuous update mechanism that replaces the reconstructed samples of the left neighboring CTU with the reconstructed samples of the current CTU.

[0072] Furthermore, block vector (BV) coding for IBC mode can use the merge list concept used for inter prediction. The IBC list construction process can consider two spatially neighboring BVs and five history-based BVs (HBVPs). In one example, only the first HBVP is compared with spatial candidates when adding it to the candidate list. While regular inter prediction uses two different candidate lists, one for merge mode and one for regular mode, the IBC mode candidate list is used in both cases (e.g., IBC merge mode and IBC regular mode). A merge mode (e.g., IBC merge mode) may use up to six candidates in the list, while a regular mode (e.g., IBC regular mode) uses only the first two candidates. Block vector differential (BVD) coding can use a motion vector differential (MVD) process, resulting in a final BV of any size. The reconstructed BV may point to an area outside the reference sample area and in some instances require correction by removing the absolute offset for each direction using modulo arithmetic with the width and height of the RSM.

[0073] 4 illustrates an example of an intra-template matching prediction (IntraTMP) mode according to an embodiment of the present invention. In embodiments such as Enhanced Compression Model (ECM) software, IntraTMP is a special intra-prediction mode that can copy a best predicted block (e.g., a matching block (421)) from a reconstructed portion of a current frame (or current picture), where the best predicted block's template (e.g., an L-shaped template) (420) can be matched with the current template (410) of the current block (411). For a given search range, the encoder can search for a template most similar to the current template in the reconstructed portion of the current frame and use the corresponding block as the predicted block. The encoder can signal the use of IntraTMP mode, and the same prediction operation can be performed on the decoder side.

[0074] The prediction signal can be generated by matching a current template (410), such as an L-shaped causal neighbor of the current block (411), with a template of another block within a predetermined search area. The exemplary search area shown in Figure 4 can include multiple CTUs (or superblocks). Referring to Figure 4, the search area can include the current CTU R1 (e.g., a portion of the current CTU R1), the upper-left CTU R2, the upper CTU R3, and the left CTU R4. The cost function can include any suitable cost function, such as the sum of absolute differences (SAD).

[0075] Within each region, the decoder can search for the template with the smallest cost (e.g., smallest SAD) relative to the current template, and can use the block associated with the template with the smallest cost as the predicted block.

[0076] The dimensions of the region indicated by (SearchRange_w, SearchRange_h) can be set to be proportional to the block dimensions (BlkW, BlkH) and have a fixed number of SAD comparisons per pixel. SearchRange_w=a×BlkW Formula (a) SearchRange_h=a×BlkH Equation (b)

[0077] The parameter "a" can be a constant that controls the tradeoff between gain and complexity. In one example, "a" is 5.

[0078] The intra template matching tool can be enabled for CUs with specific sizes, such as width and height sizes less than or equal to 64. The maximum CU size for IntraTMP mode can be configurable. IntraTMP mode can be signaled at the CU level through a dedicated flag, for example, when decoder-side intra mode derivation (DIMD) is not used for the current CU.

[0079] Boundary filtering can include applying adjustments (or filtering processes) to prediction samples in a predictive block of a current block, such as prediction samples at block boundaries, using nearby reconstructed samples from previously coded areas. In one example, boundary filtering includes applying adjustments (or filtering processes) to prediction samples at block boundaries using nearby reconstructed samples from previously coded areas, where the prediction samples at block boundaries are located within the prediction block. Boundary filtering using Position-Dependent Predictor Combinations (PDPC) modes can be applied to image and / or video coding. In VVC, the results of intra prediction for DC mode, planar mode, and some angle modes can be further modified by a PDPC method. The PDPC mode is an intra prediction method that invokes a combination of boundary reference samples and HEVC-style intra prediction using filtered boundary reference samples. The PDPC mode can be applied to the following intra modes without signaling: planar mode, DC mode, angular mode with an intra angle equal to or less than horizontal (e.g., the angle corresponding to horizontal mode), and angular mode with an intra angle equal to or greater than vertical (e.g., the angle corresponding to vertical mode) and equal to or less than 80°. The PDPC mode is not applied if the current block is coded in block-based delta pulse code modulation (BDPCM) mode or if the multiple reference line (MRL) index is greater than 0.

[0080] The prediction sample pred(x',y') is predicted using the intra prediction mode (DC, planar, angular) and a linear combination of reference samples according to the following equation (1):

number

[0081] R x,-1 , R -1,y may represent the reference samples located at the top and left boundaries of the current sample (x', y'), respectively.

number

[0082] 5A-5D show reference samples (R x,-1 and R -1,y ) used in the diagonal-upper-right PDPC mode. x,-1 and R -1,y ) is shown in Figure 5B. The reference sample (R x,-1 and R -1,y ) used for the adjacent diagonal upper right mode PDPC mode. x,-1 and R -1,y ) is shown. Figure 5D shows the reference sample (R) used for the PDPC mode in the adjacent diagonal lower left mode. x,-1 and R -1,y ) where the predicted sample pred(x',y') can be located at (x',y') in the prediction block. For example, for diagonal mode, the reference sample R x,-1 The coordinate x of the reference sample R is given by x=x'+y'+1. -1,y Similarly, the coordinate y of is given by y = x' + y' + 1. For other angular modes, the reference sample R x,-1 and R -1,y can be located at a fractional sample position, in which case the sample value at the nearest integer sample position is used.

[0083] In some examples (e.g., PDPC modes applied to DC, planar, horizontal intra mode, and vertical intra mode), Eq. (1) becomes:

number

[0084] When PDPC is applied to DC, planar, horizontal, and vertical intra modes, no additional boundary filters are needed, as are needed for the HEVC DC mode boundary filter or horizontal / vertical mode edge filter. The PDPC process for DC mode and planar mode can be identical. For angular modes, if the current angular mode is HOR_IDX or VER_IDX, then the left or top reference sample is not used, respectively. The PDPC weights and scaling factors may depend on the prediction mode and block size. PDPC modes can be applied to blocks whose width and height are both equal to or greater than a threshold, such as 4.

[0085] In some examples, a PDPC mode can be applied to a specific prediction block, such as an IntraBC and / or IntraTMP prediction block. In the original PDPC design, a PDPC mode designed for an intra prediction mode, such as a DC mode, a planar mode, or an angular mode, may not be optimized for a specific prediction mode, such as an IntraBC mode and / or an IntraTMP prediction mode, and in some examples, there is coding loss when the original PDPC design is directly applied on top of an IntraBC and / or an IntraTMP prediction mode. Each of the IBC and IntraTMP prediction modes can predict a current block using a reference block, where the reference block and the current block are in the same picture and the reference block is indicated by a block vector pointing from the current block to the reference block. In some aspects, a PDPC mode can be applied to one or more other prediction modes that use a reference block in the same picture. In some examples, the IBC or IntraTMP prediction mode may be considered intra prediction (e.g., an intra prediction mode) in the sense that a reference block in the same picture as the current block is used to predict the current block. In some examples, IBC mode or IntraTMP prediction mode may be considered a separate mode different from intra prediction and inter prediction. This disclosure includes methods for applying boundary filtering (such as PDPC mode) to a prediction block coded by a particular prediction mode based on block / template matching, such as IntraBC mode or IntraTMP mode. In one example, boundary filtering applies adjustments to prediction samples at block boundaries using nearby reconstructed samples from neighboring coded blocks.

[0086] The IBC mode may also be referred to as the IntraBC mode. The IBC mode may include different modes, such as the IBC merge mode and the IBC regular mode. Boundary filtering may be applied to a prediction block coded using the IBC mode or the IntraTMP mode. The boundary filtering may apply adjustments to prediction samples within a prediction block, such as prediction samples at block boundaries, using nearby reconstructed samples from previously coded areas. In one aspect, the boundary filtering is the same as the PDPC mode (also referred to as the PDPC filter) applied to other intra prediction modes, such as the DC mode and the planar mode, as described by equations (2)-(4).

[0087] In one aspect, the boundary filtering is based on the PDPC mode of other intra prediction modes, such as DC mode and planar mode, with some adjustments.

[0088] In one example, a different value of the parameter s is used when comparing the PDPC modes used for other intra-prediction modes.

[0089] FIG. 6 illustrates an example of boundary filtering using a two-tap filter according to an embodiment of the present invention. In one embodiment, for a left (top) boundary predicted sample, the boundary filtering is a weighted average of the left (top) neighboring reconstructed sample and the left (top) boundary predicted sample. An example of boundary filtering using a two-tap filter for the row above the boundary predicted sample and the column to the left of the boundary predicted sample is illustrated in FIG. 6. The left neighboring reconstructed sample (613) and the upper neighboring reconstructed sample (611) may be neighbors of the current block (601). The number of rows above and columns to the left of the predicted samples in the current block (601) that are filtered using the boundary filter may depend on the block size.

[0090] In one aspect, a block-level and / or high-level syntax (HLS)-level flag is signaled to indicate whether the PDPC mode applies to an IntraBC and / or IntraTMP predicted block. The HLS may be a flag in a video parameter set (VPS), a picture parameter set (PPS), a sequence parameter set (SPS), an adaptation parameter set (APS), a slice header, a frame header, a tile header, or a CTU header.

[0091] In one aspect, the template matching (TM) cost of the current IBC or IntraTMP block (e.g., similar to that used for the IntraTMP mode shown in FIG. 4) may be used to determine whether and how to apply a boundary filter.

[0092] In one example, for a block coded with the IBC model, the TM cost is calculated based on the template area indicated by the BV of the current block. When the TM cost is less than or equal to a threshold T1, for example, T1 is equal to 0, boundary filtering can be disabled.

[0093] In another example, for a block coded in IntraTMP mode, the TM cost of the IntraTMP mode may be used to check against a threshold T2. When the TM cost is less than or equal to the threshold T2, boundary filtering may be disabled.

[0094] In one embodiment, the values ​​of T1 and T2 can be different.

[0095] In one aspect, the PDPC parameters, such as the parameter s, depend on the template matching cost, as described above.

[0096] In one aspect, the boundary filter is not applied when the current coded block is coded in IBC merge mode.

[0097] In another aspect, when the current block is coded in IBC merge mode and the BVP is derived from the adjacent upper or upper right spatial candidate, only the reconstructed samples of the left neighbor are used for the boundary filter. In another aspect, when the current block is coded in IBC merge mode and the BVP is derived from the adjacent left or lower left spatial candidate, only the reconstructed samples of the upper neighbor are used for the boundary filter.

[0098] In one aspect, the residuals of blocks coded by IntraTMP and IntraBC modes are used to determine whether and how to apply a boundary filter.

[0099] In one example, when the energy of the residual is greater than a threshold T1', a boundary filter is applied.

[0100] In one example, when the energy of the residual is below the threshold T2', the boundary filter is not applied.

[0101] In one example, the energy of the residual is measured by the SAD, SSE, SATD, and MSE of the residual block.

[0102] In one example, the values ​​of the thresholds T1' and T2' may be different for blocks coded by IntraTMP mode and IntraBC mode.

[0103] In one example, the PDPC parameters, such as the parameter s, depend on the energy of the residual.

[0104] In one aspect, the boundary filtering for an IBC prediction block (also referred to as an IntraBC prediction block) and / or an IntraTMP prediction block is a PDPC filter, but the parameters used in the PDPC mode applied to the IBC prediction block or the IntraTMP prediction block may be different from the parameters used in the PDPC mode applied to other intra-prediction modes (such as planar mode, DC mode). In one example, the PDPC mode applied to the IBC prediction block and / or the IntraTMP prediction block is described using equation (4).

[0105] FIG. 7 shows an example of boundary filtering (also referred to as a boundary filter) according to an embodiment of the present invention. A current block can be coded in one of IBC mode (or IntraBC mode) and IntraTMP mode. For example, the current block can be predicted in one of IBC mode and IntraTMP mode, and a predicted block (701) of the current block can be determined (e.g., generated) using one of the IBC mode and IntraTMP mode. In one example, the predicted block (701) is an IBC predicted block (also referred to as an IBC predicted block) predicted using IBC mode. In one example, the predicted block (701) is an IntraTMP predicted block (also referred to as an IntraTMP predicted block) predicted using IntraTMP mode. Boundary filtering can be applied to a predicted block (701) obtained using IBC mode or IntraTMP mode. Therefore, boundary filtering can be applied to predicted samples pred(x',y') (710) (marked with an X in Figure 7) located at position (x',y') in the predicted block (701) that corresponds to a sample in the current block.

[0106] In one aspect, the parameters of the boundary filtering may depend on the coding information of the current block. The parameters of the boundary filtering (e.g., including the parameter W) may be determined based on the coding information of the current block. The coding information may include: (i) size information of the current block, such as the current block size and the current block aspect ratio; (ii) color component information, such as whether the current block is a luma component or a chroma component; (iii) neighboring reconstructed samples (also referred to as reference samples) of the current block or the prediction block (701); (iv) prediction samples (e.g., boundary prediction samples) within the prediction block (701); (v) differences between neighboring reconstructed samples of the current block and the boundary prediction samples within the prediction block (701); and / or (vi) the color format of the current block, such as YUV 4:2:0, YUV 4:2:2, YUV 4:4:4, RGB, etc. Referring to FIG. 7, the prediction block (701) includes prediction samples predicted using IBC mode or IntraTMP mode. In one example, boundary filtering is applied to prediction samples within the prediction block (701). In one example, boundary filtering is applied to boundary prediction samples within the prediction block (701). The boundary prediction samples may include one or more lines of prediction samples near the boundary of the prediction block (701).

[0107] The weights used for boundary filtering can be determined by right-shifting the parameter W according to the position (x',y'). The filtered predicted sample can be generated based on a linear combination of the reference sample and the predicted sample pred(x',y') (710) according to the respective determined weights. The linear combination can be a weighted average of the reference sample and the predicted sample pred(x',y') (710) according to the respective determined weights. Boundary filtering can be performed on the predicted sample pred(x',y') (710) by calculating a weighted average of the reference sample and the predicted sample pred(x',y') (710) according to the respective weights. The reference sample is a left reference sample R -1,y’ (e.g., the adjacent reconstructed sample on the left) and the reference sample R above x’,-1 The weights may include neighboring reconstructed samples of the prediction block (701), such as (e.g., the neighboring reconstructed samples above). The weights may also be referred to as filter coefficients. When the boundary filtering is performed using a PDPC mode (e.g., the boundary filtering is a PDPC mode), the weights may be referred to as PDPC weights or PDPC filter coefficients.

[0108] In one example, the weighted average is (w L ×R -1,y’ +w T ×R x’,-1 +w pred × pred(x', y')). The weights of the boundary filtering are the left reference sample R -1,y’ The weight w associated with L and the above reference sample R x’,-1 The weight w associated with T and the weights w associated with the predicted sample pred(x',y') pred In one example, the sum of the weights is predefined or constant (e.g., 64), so that w predOne weight can be determined from two other weights, such as ∑(64) = (64 - wL - wT). In one example, the weighted average is clipped using a clipping function. In one example, boundary filtering is performed by a PDPC filter (or PDPC mode), e.g., filtered prediction samples (or boundary filtered prediction samples) (710) can be generated using equation (4), where the reference sample R -1,y’ and R x’,-1 The predicted samples (710) are shown in Figure 7. The parameter BitDepth in equation (4) may indicate the bit depth.

[0109] The boundary filtering performed by PDPC (e.g., using equation (4)) on an IBC predicted block or an IntraTMP predicted block may be different from the PDPC (e.g., as described using equations (1)-(3) or (2)-(4)) performed on an intra predicted block obtained using, for example, an intra prediction mode (e.g., DC mode, planar mode).

[0110] In one aspect, the parameters used in the PDPC mode for an IBC predicted block or an IntraTMP predicted block may be different from the parameters in the PDPC mode for an intra prediction mode (e.g., DC mode, planar mode) or for an intra prediction block (e.g., an intra block predicted using an intra prediction mode such as DC mode, planar mode, etc.).

[0111] In one example, when a PDPC mode is applied to an IntraBC and / or IntraTMP prediction block (e.g., prediction block (701)), the following equations (Equations (5)-(6)) are used to derive PDPC filter coefficients (weights). x' and y' may represent the position of the processed sample (e.g., pred(x', y') (602)) within the prediction block (701). Exemplary values ​​of W include, but are not limited to, 8, 4, 16, and 2. In one example, the parameter W is determined to be 8, 4, 16, or 2. In one example, W is not 32. The following equations (Equations (5)-(6)) may be used to derive weights (PDPC filter coefficients) in a PDPC mode applied to an IntraBC prediction block and / or an IntraTMP prediction block.

number

[0112] On the other hand, as mentioned above, the weights (e.g., w) in the PDPC mode for the intra prediction mode (such as the planar mode and DC mode) L and w T ) can be obtained using equations (2)-(3) based on the parameter s and a constant value which is 32.

[0113] Comparing Equations (2)-(3) with Equations (5)-(6), the weights w obtained using Equations (5)-(6) in the PDPC mode for IntraBC and / or IntraTMP prediction blocks are L and w T can be obtained based on the parameter W without using the parameter s (instead, a constant value of "0" is used in equations (5)-(6)). The weights w obtained using equations (2)-(3) in the PDPC mode for intra-predicted blocks are L and w T can be obtained based on the parameter s without using the parameter W (for example, a constant value of "32" is used). The parameter W in equations (5)-(6) can vary, while the value "32" in equations (2)-(3) is fixed. W can also be different from 32. In equations (2)-(3), the parameter s can vary, but in equations (5)-(6), s is replaced with the value "0". In one example, equations (5)-(6) can be obtained based on the parameter w L =W>>((x'<<1)) and w T =W>>((y'<<1)), and the weights w obtained using equations (5)-(6) for the IntraBC and / or IntraTMP prediction blocks. L and w T can be obtained based on the parameter W and is independent of the parameter s.

[0114] With reference to equations (2)-(3), parameters in the PDPC mode for intra-predicted blocks may include parameter s. With reference to equations (5)-(6), parameters in the PDPC mode for IntraBC and / or IntraTMP predicted blocks may include parameter W. Comparing equations (2)-(3) with equations (5)-(6), parameters in the PDPC mode for IntraBC and / or IntraTMP predicted blocks (e.g., including parameter W) may differ from parameters in the PDPC mode for intra-predicted blocks (e.g., including parameter s).

[0115] In one aspect, parameters of a PDPC filter (e.g., a PDPC filter for an IntraBC and / or IntraTMP prediction block) depend on coding information, including, but not limited to, a coding block size, a coding block aspect ratio, whether the current block is a luma component or a chroma component, neighboring reconstructed samples, boundary prediction samples within the prediction block (e.g., boundary IBC prediction samples), a difference between neighboring reconstructed samples and boundary prediction samples, and a color format (e.g., YUV 4:2:0, YUV 4:2:2, YUV 4:4:4, or RGB). In one example, there may be a relationship between parameters of a PDPC filter (e.g., a parameter W) (e.g., for an IntraBC and / or IntraTMP prediction block) and the coding information. In one example, the parameter W increases with the coding block size. In one example, W decreases with the coding block size. Thus, parameters of a PDPC filter (e.g., a PDPC filter for an IntraBC and / or IntraTMP prediction block) may be derived based on the above coding information. PDPC filter weights (e.g., w L and w T ) can depend on the coding information, for example due to the dependence of the weights on the parameter W as shown in equations (5)-(6).

[0116] In one example, parameters of a PDPC filter (eg, a PDPC filter for an IntraBC and / or IntraTMP prediction block) may be derived separately for each color component or a subset of color components.

[0117] In one aspect, whether boundary filtering (e.g., PDPC mode) is applied to a current block or to an IBC or IntraTMP predicted prediction block (701) can be determined from neighboring reconstructed samples. Whether a PDPC mode is applied can be adaptively determined based on neighboring reconstructed samples (also referred to as neighboring reconstructed samples), and whether a PDPC mode such as that described in equations (4)-(6) is applied to the prediction block (701) can be adaptively determined based on the neighboring reconstructed samples.

[0118] In one example, a content type detection process is applied to neighboring reconstructed samples. If the neighboring reconstruction is determined to be screen content (e.g., non-camera-captured content), a PDPC mode is not applied to the IntraBC and / or IntraTMP predicted block (e.g., IBC predicted block and / or IntraTMP predicted block). Otherwise, a PDPC mode is applied to the IntraBC and / or IntraTMP predicted block (e.g., IBC predicted block and / or IntraTMP predicted block). For example, the content type of the neighboring reconstructed sample is detected. If the content type of the neighboring reconstructed sample is screen content, boundary filtering (e.g., PDPC mode) is not applied to the block (e.g., IBC predicted block and / or IntraTMP predicted block such as prediction block 701). If the content type of the neighboring reconstructed sample is not screen content (e.g., the content type is natural camera-captured content), boundary filtering (e.g., PDPC mode) is applied to the block (e.g., IBC predicted block and / or IntraTMP predicted block such as prediction block 701).

[0119] In one example, the content type detection process includes checking whether there are distinct color values ​​in adjacent blocks. If there are color values ​​less than a given threshold, the adjacent reconstructed samples can be determined to have screen content. For example, the number of color values ​​(e.g., the number of distinct color values) of the adjacent reconstructed samples is checked. If the number of color values ​​is less than the color value threshold, the content type of the adjacent reconstructed samples is detected as screen content.

[0120] In one example, the adjacent reconstructed samples are of a particular color component, and the color value includes a value of the particular color component. In one example, the color value means the value of one particular color component, for example, luma or a luma component.

[0121] In one example, adjacent reconstructed samples are associated with multiple color components, and the color value includes a combination of the respective values ​​of the multiple color components. In one example, the color value refers to a combination of the values ​​of multiple color components, such as a combination of Y, Cb, and Cr or a combination of R, G, and B. In one example, Y represents the luma component, and Cb and Cr represent the chroma components.

[0122] In one example, boundary filtering (e.g., PDPC mode) is applied only to a predefined color component, for example, when the color component of the current block (or predicted block (701)) (e.g., an IBC-coded block or an IntraTMP-coded block) is a luma component. Boundary filtering can be applied to each color component associated with the block. In one aspect, PDPC mode is applied only to IntraBC- and / or IntraTMP-coded blocks when the current color component is luma (a luma component). Alternatively, PDPC mode is applied only to IntraBC- and / or IntraTMP-coded blocks for all color components (e.g., all of Y, Cb, and Cr).

[0123] In one example, boundary filtering (eg, PDPC mode) is applied to the prediction block (701) only when the current slice containing the current block is of a predefined slice type, such as an intra slice.

[0124] In one aspect, the PDPC mode is applied only to IntraBC and / or IntraTMP coded blocks (e.g., IBC coded blocks or IntraTMP coded blocks) when the current slice is an intra slice. The current slice includes IntraBC and / or IntraTMP coded blocks. In one example, whether the PDPC mode is applied to an intra slice and / or an inter slice is coded in a high-level syntax (HLS). The HLS may be a flag in a VPS, PPS, SPS, APS, slice header, frame header, tile header, or CTU header.

[0125] 8 is a flowchart outlining a process (800) according to an embodiment of the present disclosure. The process (800) may be used in a video decoder. In various embodiments, the process (800) is performed by a processing circuit, such as a processing circuit that performs the functions of the video decoder (110) or a processing circuit that performs the functions of the video decoder (210). In some embodiments, the process (800) is implemented with software instructions, and thus, the processing circuit performs the process (800) when it executes the software instructions. Processing begins at (S801) and proceeds to (S810).

[0126] At (S810), a coded video bitstream can be received, including a current picture having a block (e.g., the current block shown in FIG. 7) coded in one of an intra block copy (IBC) mode and an intra template matching (IntraTMP) mode.

[0127] In (S820), the one of the IBC mode and the IntraTMP mode can be used to determine a prediction block of the block (for example, the prediction block (701) shown in FIG. 7).

[0128] At (S830), in response to boundary filtering being applied to the block, boundary filtering may be applied to prediction samples pred(x',y') located at positions (x',y') in the prediction block that correspond to samples in the block, as described with reference to Figure 7. The prediction samples pred(x',y') may be filtered by boundary filtering as follows: The weights used for boundary filtering can be determined, for example, by right-shifting a parameter W according to the position (x', y') in the prediction block. The filtered prediction samples can be generated based on a linear combination of the reference samples and the prediction samples according to the determined weights.

[0129] In one example, the parameter W of the boundary filtering is determined (e.g., derived) based on coding information of the block. The coding information may include a coding block size, a coding block aspect ratio, whether the block is a luma or chroma component, neighboring reconstructed samples of the block, boundary predicted samples within the predictive block, differences between neighboring reconstructed samples of the block and boundary predicted samples within the predictive block, or a color format of the block. In one example, the parameter W is determined to be 8, 4, 16, or 2.

[0130] The reference sample is shown in Figure 7. -1,y’ and the above reference sample R x’,-1 The weights can be calculated by dividing the left reference sample R -1,y’ The weight w associated with L and the above reference sample R x’,-1 associated with w Tand the weight (64-w L -w T ) and weight w L A, w L = W>>((x'<<1)>>0), and the weight w T A, w T =W>>((y'<<1)>>0).

[0131] In one example, the boundary filtering is performed by a Position Dependent Predictor Combination (PDPC) filter. The filtered prediction samples are denoted by Clip(0,(1< <BitDepth)-1,(w L ×R -1,y’ +w T ×R x’,-1 +(64-w L -w T )×pred(x',y')+32)>>6) where the parameter BitDepth indicates the bit depth.

[0132] The process then proceeds to (S899) and ends.

[0133] The process (800) may be adapted as appropriate. Steps of the process (800) may be modified and / or omitted. Additional steps may be added. Any suitable order of implementation may be used. In one example, whether boundary filtering is applied to a block is determined from neighboring reconstructed samples of the block or a predicted block.

[0134] 9 is a flowchart outlining a process (900) according to an embodiment of the present disclosure. The process (900) can be used in a video encoder. In various embodiments, the process (900) is performed by a processing circuit, such as a processing circuit that performs the functions of the video encoder (103) or a processing circuit that performs the functions of the video encoder (303). In some embodiments, the process (900) is implemented with software instructions, and thus, the processing circuit performs the process (900) when it executes the software instructions. Processing begins at (S901) and proceeds to (S910).

[0135] At (S910), a prediction block for the block may be determined using one of an intra block copy (IBC) mode and an intra template matching (IntraTMP) mode.

[0136] At (S920), in response to boundary filtering being applied to the block, boundary filtering may be applied to prediction samples pred(x',y') located at positions (x',y') in the prediction block that correspond to samples in the block, as described with reference to Figure 7. The prediction samples pred(x',y') may be filtered by boundary filtering as follows: The weights used for boundary filtering can be determined, for example, by shifting a parameter W to the right according to the position (x', y') in the prediction block. The filtered prediction samples can be generated based on a linear combination of the reference samples and the prediction samples according to the determined weights.

[0137] In one example, the parameter W of the boundary filtering is determined (derived) based on the block size of the block, the block aspect ratio of the block, whether the block is a luma component or a chroma component, neighboring reconstructed samples of the block, boundary predicted samples within the predicted block, differences between neighboring reconstructed samples of the block and boundary predicted samples within the predicted block, or the color format of the block. In one example, the parameter W is determined to be 8, 4, 16, or 2.

[0138] The reference sample is shown in Figure 7. -1,y’ and the above reference sample R x’,-1 The weights can be calculated by dividing the left reference sample R -1,y’ The weight w associated with L and the above reference sample R x’,-1 associated with w T and the weight (64-w L -w T ) and weight w L A, w L = W>>((x'<<1)>>0), and the weight w T A, w T =W>>((y'<<1)>>0).

[0139] In one example, the boundary filtering is performed by a Position Dependent Predictor Combination (PDPC) filter. The filtered prediction samples are denoted by Clip(0,(1< <BitDepth)-1,(w L ×R -1,y’ +w T ×R x’,-1 +(64-w L -w T )×pred(x',y')+32)>>6) where the parameter BitDepth indicates the bit depth.

[0140] The process then proceeds to (S999) and ends.

[0141] The process 900 may be adapted as appropriate. Steps of the process 900 may be modified and / or omitted. Additional steps may be added. Any suitable order of implementation may be used.

[0142] The aspects, embodiments, and / or examples of the present disclosure may be used separately or combined in any order. Each of the methods (or aspects), encoders, and decoders may be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored in a non-transitory computer-readable medium.

[0143] The techniques described above can be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, Figure 10 illustrates a computer system (1000) suitable for implementing aspects of the disclosed subject matter.

[0144] Computer software may be coded using any suitable machine code or computer language that may be subject to assembly, compilation, linking, or similar mechanisms to create code that includes instructions that may be executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., either directly or through interpretation, microcode execution, etc.

[0145] The instructions may be executed in various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0146] 10 for computer system 1000 are exemplary in nature and are not intended to suggest any limitation regarding the scope of use or functionality of the computer software implementing aspects of the present disclosure, nor should the arrangement of components be interpreted as having any dependency or requirement regarding any one or combination of components illustrated in the exemplary embodiment of computer system 1000.

[0147] The computer system 1000 may include certain human interface input devices that may respond to input by one or more human users, for example, through tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, clapping), visual input (e.g., gestures), or olfactory input (not shown). The human interface input devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still image camera), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic vision).

[0148] The human interface input devices may include one or more of a keyboard (1001), a mouse (1002), a trackpad (1003), a touch screen (1010), a data glove (not shown), a joystick (1005), a microphone (1006), a scanner (1007), and a camera (1008) (only one of each is shown).

[0149] The computer system (1000) may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses, for example, through tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touchscreen (1010), data gloves (not shown), or joystick (1005), although haptic feedback devices that do not function as input devices may also exist), audio output devices (e.g., speakers (1010), headphones (not shown), etc.), visual output devices (e.g., screens (1010), including CRT, LCD, plasma, and OLED screens, each with or without touchscreen input capability and each with or without haptic feedback capability, some of which may provide two-dimensional visual output or greater than three-dimensional output through means such as stereoscopic output, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0150] The computer system (1000) may also include human-accessible storage devices and their associated media, such as optical media or similar media (1021), including CD / DVD ROM / RW (1020) with CDs / DVDs, thumb drives (1022), removable hard drives or solid-state drives (1023), legacy magnetic media such as tape and floppy disks (not shown), and special ROM / ASIC / PLD-based devices such as security dongles (not shown).

[0151] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not encompass transmission media, carrier waves, or other transitory signals.

[0152] The computer system 1000 may also include an interface 1054 to one or more communication networks 1055. The networks may be, for example, wireless, wired, or optical. The networks may further be local, wide-area, metropolitan, vehicular, industrial, real-time, delay-tolerant, etc. Examples of networks include local area networks such as Ethernet and WLAN; cellular networks including GSM, 3G, 4G, 5G, LTE, etc.; TV wired or wireless wide-area digital networks including cable TV, satellite TV, and terrestrial TV; and vehicular and industrial networks including CANBus, etc. Certain networks generally require an external network interface adapter (e.g., a USB port on the computer system 1000) attached to a particular general-purpose data port or peripheral bus 1049; others are generally integrated into the core of the computer system 1000 by attachment to a system bus (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system), as described below. Using any of these networks, the computer system (1000) can communicate with other entities. Such communications can be unidirectional receive-only (e.g., broadcast TV), unidirectional transmit-only (e.g., from a particular CANbus to a particular CANbus device), or bidirectional to other computer systems, using, for example, local or wide-area digital networks. As noted above, specific protocols and protocol stacks can be used in each of these networks and network interfaces.

[0153] The aforementioned human interface devices, human-accessible storage devices, and network interfaces can be attached to the core (1040) of the computer system (1000).

[0154] The core (1040) may include one or more central processing units (CPUs) (1041), graphics processing units (GPUs) (1042), dedicated programmable processing units in the form of field programmable gate arrays (FPGAs) (1043), task-specific hardware accelerators (1044), graphics adapters (1050), etc. These devices may be connected through a system bus (1048), along with read-only memory (ROM) (1045), random access memory (RAM) (1046), and internal mass storage (1047), such as an internal non-user-accessible hard drive or SSD. In some computer systems, the system bus (1048) is accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be attached directly to the core's system bus (1048) or via a peripheral bus (1049). In one example, a screen (1010) may be connected to the graphics adapter (1050). Peripheral bus architectures include PCI, USB, and the like.

[0155] The CPU (1041), GPU (1042), FPGA (1043), and accelerator (1044) can execute specific instructions, which, in combination, can constitute the aforementioned computer code. The computer code can be stored in ROM (1045) or RAM (1046). Temporary data can also be stored in RAM (1046), while permanent data can be stored, for example, in internal mass storage (1047). Rapid storage and retrieval from any of the memory devices can be enabled through the use of cache memory, which can be closely associated with one or more of the CPU (1041), GPU (1042), mass storage (1047), ROM (1045), RAM (1046), etc.

[0156] The computer-readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0157] By way of example and not limitation, a computer system (1000) having an architecture, and in particular a core (1040), can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be user-accessible mass storage, as introduced above, as well as media associated with specific storage of the core (1040) that is non-transitory in nature, such as the core's internal mass storage (1047) or ROM (1045). Software implementing various aspects of the present disclosure can be stored on such devices and executed by the core (1040). The computer-readable media can include one or more memory devices or chips according to particular needs. The software can cause the core (1040) and in particular the processor therein (including a CPU, GPU, FPGA, etc.) to perform particular processes or portions of particular processes described herein, including defining data structures stored in RAM (1046) and modifying such data structures according to processes defined by the software. Additionally or alternatively, the computer system may provide functionality as a result of logic hardwired or otherwise embodied in circuitry (e.g., accelerator (1044)), which may operate in place of or in conjunction with software to perform particular processes or portions of particular processes described herein. References to software include logic, and vice versa, where appropriate. References to computer-readable media may encompass circuitry (such as an integrated circuit (IC)) that stores software for execution, circuitry embodying logic for execution, or both, as appropriate. The present disclosure encompasses any suitable combination of hardware and software.

[0158] The use of "at least one of" or "one of" in this disclosure is intended to include any one or combination of the listed elements. For example, reference to at least one of A, B, or C; at least one of A, B, and C; at least one of A, B, and / or C; and at least one of A through C is intended to include A only, B only, C only, or any combination thereof. Reference to one of A or B, and one of A and B is intended to include A or B or (A and B). The use of "one of" does not exclude any combination of the listed elements, when applicable, such as when the elements are not mutually exclusive.

[0159] While this disclosure has described several exemplary embodiments, there are alterations, permutations, and various substitute equivalents that fall within the scope of this disclosure. It will thus be appreciated that those skilled in the art will be able to devise various systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within the spirit and scope of the present disclosure.

Claims

1. 1. A processor-implemented method of video decoding, comprising: receiving a coded video bitstream including a current picture having blocks coded in one of an intra block copy (IBC) mode and an intra template matching (IntraTMP) mode; determining a prediction block for the block using the one of the IBC mode and the IntraTMP mode; responsive to boundary filtering having been applied to the block, applying the boundary filtering to prediction samples pred(x',y') located at positions (x',y') in the prediction block that correspond to samples in the block, determining a parameter W of the boundary filtering based on coding information of the block; determining a weight to be used in said boundary filtering by right-shifting said parameter W according to said position (x', y'); generating a filtered predicted sample based on a linear combination of the reference sample and the predicted sample according to the determined weights; applying the boundary filtering by A method comprising:

2. The weights are the left reference sample R －１，ｙ’ The weight w associated with Ｌ and the above reference sample R ｘ’，－１ The weight w associated with Ｔ and the weight (64-w Ｌ -w Ｔ ) and The weight w Ｌ is equal to W>>((x'<<1)>>0), and the weight w Ｔ is equal to W>>((y'<<1)>>0), The method of claim 1.

3. the boundary filtering is performed by a Position Dependent Predictor Combination (PDPC) filter; The generating of the filtered predicted samples may be performed by filtering the filtered predicted samples using Clip(0, (1<<BitDepth)-1, (w Ｌ ×R －１，ｙ’ +w Ｔ ×R ｘ’，－１ + (64-w Ｌ -w Ｔ )×pred(x′,y′)+32)>>6), where BitDepth indicates the bit depth. The method of claim 2.

4. The coding information of the block includes a coding block size, a coding block aspect ratio, whether the block is a luma component or a chroma component, neighboring reconstructed samples of the block, boundary predicted samples within the predicted block, differences between neighboring reconstructed samples of the block and boundary predicted samples within the predicted block, or a color format of the block. The method of claim 1.

5. The parameter W is determined as 8, 4, 16 or 2; The method of claim 1.

6. determining from neighboring reconstructed samples whether the boundary filtering is applied to the block; The method of claim 1 further comprising:

7. The step of determining whether boundary filtering is applied comprises: detecting the content type of the adjacent reconstructed samples; in response to the content type of the neighboring reconstructed sample being screen content, not applying the boundary filtering to the block; applying the boundary filtering to the block in response to the content type of the neighboring reconstructed sample not being the screen content; The method of claim 6, comprising:

8. The step of detecting a content type includes: checking the number of color values ​​of said adjacent reconstructed samples; detecting the content type as the screen content in response to the number of color values ​​being less than a color value threshold; The method of claim 7, comprising:

9. the adjacent reconstructed samples are of a particular color component, and the color values ​​include values ​​of the particular color component. The method of claim 8.

10. the adjacent reconstructed samples are associated with a plurality of color components, and the color value comprises a combination of values ​​of each of the plurality of color components. The method of claim 8.

11. the boundary filtering is applied only when the color component of the block is a luma component; The method of claim 1.

12. the boundary filtering is applied to each color component associated with the block; The method of claim 1.

13. The boundary filtering is applied only when the current slice containing the block is an intra-slice. The method of claim 1.

14. Apparatus for video decoding, comprising a processing circuit configured to perform the method of any one of claims 1 to 13.

15. A computer program which, when executed by at least one processor, causes the at least one processor to perform the method of any one of claims 1 to 13.

Images