Filtered intrablock copy using gradient and position

Abstract

Aspects of this disclosure include a video decoding and encoding method and a video decoding and encoding apparatus and a method for processing video media data. A video decoding apparatus including a processing circuit configured to receive coded information indicating that the current block in the current picture is predicted using filtered intrablock copy (FIBC) mode, determine a linear predicted value of the current sample in the current block by applying a linear filter to the predicted predicted sample using either IBC mode or intratemplate matching (IntraTMP) mode, determine a gradient value associated with the current sample using at least one gradient filter, and determine a predicted value of the current sample based on the sum of the linear predicted value and at least one modified value including the gradient value. The FIBC filter in FIBC mode includes a linear filter and at least one gradient filter.

Description

[Technical Field]

[0001] Related applications This application claims priority to U.S. Provisional Application No. 63 / 462,235, “Gradient and Location Based Filtered Intra Block Copy,” filed on 26 April 2023, which is incorporated herein by reference in its entirety.

[0002] This disclosure generally describes aspects related to video coding. [Background technology]

[0003] The background information provided below is intended to give a general overview of the background to this disclosure. Within the scope of the background information provided herein, the actions taken by the inventors whose names are listed in this application, and any other descriptions that do not qualify as prior art at the time of filing, are not explicitly or implicitly acknowledged as prior art to this disclosure.

[0004] Image / video compression makes it easier to transmit image / video data across different devices, storage, and networks while minimizing quality degradation. In some cases, video codec technology can compress video based on spatial and temporal redundancy. In one example, a video codec can use a technique called intra-prediction, which can compress images based on spatial redundancy. For example, intra-prediction can use reference data obtained from the currently reconstructed picture for sample prediction. In another example, a video codec can use a technique called inter-prediction, which can compress images based on temporal redundancy. For example, inter-prediction can use motion compensation to predict samples in the current picture from previously reconstructed pictures. Motion compensation can be represented by motion vectors (MV). [Overview of the Initiative] [Problems that the invention aims to solve]

[0005] Aspects of this disclosure include video encoding / decoding methods and video encoding / decoding devices. [Means for solving the problem]

[0006] In one embodiment, a method for processing video media data includes the step of processing a bitstream of video media data according to a format rule. The bitstream includes syntax elements indicating that the current block in the current picture is predicted using filtered intrablock copy (FIBC) mode. The format rule specifies that the linear predicted value of the current sample in the current block is determined by applying a linear filter to the predicted sample using either intrablock copy (IBC) mode or intratemplate matching (IntraTMP) mode. The format rule specifies that the gradient value associated with the current sample in the current block is determined using at least one gradient filter. The format rule specifies that the nonlinear value associated with the current sample is determined from at least one of the current sample and its neighboring samples using a nonlinear value and a nonlinear relationship between the values ​​of the current sample and at least one of the neighboring samples. The format rule specifies that the position value is based on the position of the central sample at the center of the linear filter. The formatting rules specify that the FIBC filter in FIBC mode includes a linear filter, at least one gradient filter, coefficients for the nonlinear value, and coefficients for the position, based on the sum of the current sample's predicted value and at least one modified value including gradient, nonlinear, and position values. The formatting rules specify that the current sample is processed from the current sample's predicted value.

[0007] For example, a linear filter includes a bias term.

[0008] In one example, a linear filter is configured to add the mean value of the current block, and the mean value of the current block is subtracted from each of the samples predicted using either IBC mode or IntraTMP mode.

[0009] In one embodiment, the video coding method includes the steps of: determining a linear predicted value of the current sample in the current block by applying a linear filter to a predicted sample predicted using either an intrablock copy (IBC) mode or an intratemplate matching (IntraTMP) mode, wherein the current block is predicted using filtered IBC (FIBC) mode; determining a gradient value associated with the current sample in the current block using at least one gradient filter; determining a nonlinear value associated with the current sample from at least one of the current sample and neighboring samples using a nonlinear value and a nonlinear relationship between the values ​​of the current sample and at least one of the neighboring samples; determining a predicted value of the current sample based on the sum of a linear predicted value and at least one modified value including the gradient value and the nonlinear value, wherein the FIBC filter in FIBC mode includes a linear filter, at least one gradient filter, and coefficients of nonlinear values; and coding the current sample from the predicted value of the current sample.

[0010] In one example, a video coding method further includes the steps of: determining a position value using the position of a central sample at the center of a linear filter; and determining a predicted value of the current sample based on the sum of a linear predicted value and at least one modified value including a gradient value, a nonlinear value and a position value, wherein the FIBC filter in FIBC mode includes a linear filter, at least one gradient filter, a coefficient for the nonlinear value and a coefficient for the position.

[0011] In one example, the linear filter may include a bias term, or it may be configured to add the mean of the current block and subtract the mean of the current block from each of the samples predicted using either the IBC mode or the IntraTMP mode.

[0012] In one aspect of the present disclosure, the video decoding device includes a processing circuit. The processing circuit is configured to: receive coded information indicating that the current block in the current picture is predicted using filtered intrablock copy (FIBC) mode; determine a linear predicted value of the current sample in the current block by applying a linear filter to the predicted predicted sample using either IBC mode or intratemplate matching (IntraTMP) mode; determine a gradient value associated with the current sample in the current block using at least one gradient filter; determine a predicted value of the current sample based on the sum of the linear predicted value and at least one modified value including the gradient value, wherein the FIBC filter in FIBC mode includes a linear filter and at least one gradient filter; and reconstruct the current sample from the predicted value of the current sample.

[0013] In one example, the processing circuit is configured to determine a position value using the position of the central sample at the center of a linear filter, and to determine a predicted value for the current sample based on the sum of a linear predicted value and at least one corrected value including a gradient value and a position value, wherein the FIBC filter in FIBC mode includes a linear filter, at least one gradient filter, and a position coefficient.

[0014] In one example, the processing circuit is configured to determine a nonlinear value related to the current sample from at least one of the current sample and neighboring samples using a nonlinear value and a nonlinear relationship between the current sample and at least one of the neighboring samples, and to determine a predicted value of the current sample based on the sum of a linear predicted value and at least one corrected value including a gradient value and a nonlinear value, wherein the FIBC filter in FIBC mode includes a linear filter, at least one gradient filter and coefficients for the nonlinear value.

[0015] For example, a linear filter includes a bias term.

[0016] In one example, the mean values ​​of the current block are added using a linear filter, and the mean values ​​of the current block are removed from each of the samples predicted using either the IBC mode or the IntraTMP mode.

[0017] In one example, the processing circuit is configured to clip the predicted value of the current sample.

[0018] In one example, the processing circuit is configured to determine the coefficients of the FIBC filter in FIBC mode from the current template of the current block and the reference template of the reference block indicated by the block vector of the current block.

[0019] In one example, the processing circuit is configured to determine the coefficients of the FIBC filter in FIBC mode using LDL decomposition.

[0020] For example, a linear filter may consist of five samples: (i) a central sample of the linear filter with an offset of (0,0), a northern sample N with an offset of (0,-1), a southern sample S with an offset of (0,1), an eastern sample E with an offset of (1,0), and a western sample W with an offset of (-1,0), where the offsets of the five samples in the linear filter are relative to the central sample; or (ii) a linear filter with an offset of (0,0) The linear filter comprises nine samples: a central sample, two northern samples with offsets of (0,-1) and (0,-2) respectively, two southern samples with offsets of (0,1) and (0,2) respectively, two eastern samples with offsets of (1,0) and (2,0) respectively, and two western samples with offsets of (-1,0) and (-2,0) respectively, wherein the offsets of the nine samples in the linear filter form a cross shape including the nine samples, with the central sample as the reference.

[0021] For example, if a linear filter has 5 samples, the current sample is located at one of the 5 positions, which are the positions of each of the 5 samples. If a linear filter has 9 samples, the current sample is located at one of the 9 positions, which are the positions of each of the 9 samples.

[0022] In one example, the shape of a linear filter is predetermined, and the shape of at least one gradient filter is predetermined.

[0023] In one example, one sample in a linear filter is spatially separated from all the other samples in the linear filter.

[0024] In one example, if at least one gradient filter consists of a horizontal gradient filter, the gradient value is the horizontal gradient value, and the number of first input samples and the position of the first input samples in the horizontal gradient filter are set independently of the linear filter; if at least one gradient filter consists of a vertical gradient filter, the gradient value is the vertical gradient value, and the number of second input samples and the position of the second input samples in the vertical gradient filter are set independently of the linear filter; and if at least one gradient filter includes both a horizontal and a vertical gradient filter, the gradient value is the sum of the horizontal and vertical gradient values, and the number of first input samples and the position of the first input samples in the horizontal gradient filter and the number of second input samples and the position of the second input samples in the vertical gradient filter are set independently of each other and independently of the linear filter.

[0025] In one example, at least one gradient filter includes a horizontal gradient filter, the gradient value includes a horizontal gradient value which is the sum of the horizontal gradients of each first input sample in the horizontal gradient filter, and the processing circuit is configured to determine each horizontal gradient of each first input sample based on one of the following: (i) the difference between each first input sample and the left neighbor sample of each first input sample, (ii) the difference between the left neighbor sample of the first input sample and the right neighbor sample of each first input sample, and (iii) the difference between a first value and a second value, where the first value is the sum based on the upper left neighbor sample, the left neighbor sample and the lower left neighbor sample of the first input sample, and the second value is the sum based on the upper right neighbor sample, the right neighbor sample and the lower right neighbor sample of the first input sample.

[0026] In one example, the processing circuit is configured to determine, based on the position of each first input sample, which difference to use to calculate the horizontal gradient of each first input sample.

[0027] In one example, at least one gradient filter includes a vertical gradient filter, and the gradient value includes a vertical gradient value which is the sum of the vertical gradients of each second input sample in the vertical gradient filter. The processing circuit is configured to determine each vertical gradient of each second input sample based on one of the following: (i) the difference between each second input sample and the upper neighbor sample of each second input sample; (ii) the difference between the upper neighbor sample of each second input sample and the lower neighbor sample of each second input sample; and (iii) the difference between a first value and a second value, where the first value is the sum based on the upper-left neighbor sample, the upper neighbor sample and the upper-right neighbor sample of the second input sample, and the second value is the sum based on the lower-left neighbor sample, the lower neighbor sample and the lower-right neighbor sample of each second input sample.

[0028] In one example, the processing circuit is configured to determine, based on the position of each second input sample, which difference is used to calculate the vertical gradient of each second input sample.

[0029] Aspects of this disclosure also provide a video encoding device. The video encoding device includes a processing circuit configured to perform one of the video encoding methods described.

[0030] Aspects of this disclosure also provide video decoding methods. These methods include any method performed by a video decoding device.

[0031] Aspects of this disclosure also provide a non-temporary computer-readable medium that stores instructions causing a computer to perform one of the described video decoding / encoding methods when executed by the computer.

[0032] Further characteristics, properties, and various effects of the disclosed protected subject matter will become clearer from the detailed description below and the accompanying drawings described below. [Brief explanation of the drawing]

[0033] [Figure 1] This is a schematic diagram of an example block diagram of a communication system (100). [Figure 2] This is a schematic diagram of an example of a decoder block diagram. [Figure 3] This is a schematic diagram of an example block diagram for an encoder. [Figure 4] An example of a convolutional filter according to one aspect of this disclosure is shown. [Figure 5] An example of a reference region used to derive a filter coefficient according to one aspect of this disclosure is shown. [Figure 6] An example of a spatial sample used in a gradient and position-based convolutional component model (GL-CCCM) according to one aspect of this disclosure is shown. [Figure 7A] An example of an IntraTemplate Matching Prediction (IntraTMP) mode according to one aspect of this disclosure is shown. [Figure 7B] An example of a modification of the filtered intrablock copy (FIBC) model according to one aspect of this disclosure is shown. [Figure 8] An example of a filter usable in FIBC mode according to one aspect of this disclosure is shown. [Figure 9] An example of a filter usable in FIBC mode according to one aspect of this disclosure is shown. [Figure 10] An example of a filter usable in FIBC mode according to one aspect of this disclosure is shown. [Figure 11] An example of a neighboring sample of input sample C used to calculate the gradient according to one aspect of this disclosure is shown. [Figure 12] An example of selecting a gradient calculation method according to one aspect of this disclosure is shown. [Figure 13] Examples of the positions of Sample C, A, L, and AL according to one aspect of this disclosure are shown. [Figure 14] A flowchart illustrating the decoding process in some aspects of this disclosure is shown. [Figure 15] A flowchart illustrating the coding process relating to several aspects of this disclosure is shown. [Figure 16] This is a schematic diagram of a computer system relating to one embodiment. [Modes for carrying out the invention]

[0034] Figure 1 shows block diagrams of several examples of video processing systems (100). Video processing systems (100) are examples of applications of the disclosed protected subject, a video encoder and video decoder in a streaming environment. The disclosed protected subject may also be similarly applicable to other applications involving video, such as video conferencing, digital TV, streaming services, and storing compressed video on digital media such as CDs, DVDs, and memory sticks.

[0035] The video processing system (100) includes an imaging subsystem (113) which may include a video source (101), such as a digital camera, that creates a stream (102) of uncompressed video pictures. In one example, the stream (102) of video pictures includes samples taken by the digital camera. The stream (102) of video pictures is shown in bold to emphasize its large data size compared to encoded video data (104) (i.e., a coded video bitstream), and this stream (102) of video pictures can be processed by an electronic device (120) which includes a video encoder (103) connected to the video source (101). The video encoder (103) may include hardware, software, or a combination thereof that enables or implements the disclosed protected aspects, which are described in detail below. To emphasize the smaller data size compared to the video picture stream (102), the encoded video data (104) (i.e., the encoded video bitstream) is shown as a thin line, which can be stored in the streaming server (105) for later use. One or more streaming client subsystems, such as the client subsystems (106) and (108) in Figure 1, can access the streaming server (105) to obtain copies (107) and (109) of the encoded video data (104). The client subsystem (106) may include, for example, a video decoder (110) located in an electronic device (130). The video decoder (110) takes the copy (107) of the encoded video data as input, decodes it, and creates an output stream (111) of a video picture that can be drawn to a display (112) (e.g., a display screen) or other drawing device (not shown). In some streaming systems, encoded video data (104), (107), and (109) (e.g., video bitstream) can be encoded according to specific video coding / compression standards. An example of such a standard is ITU-T Recommendation H.265.For example, a video coding standard under development is informally known as Versatile Video Coding (VVC). The disclosed protected content may be used under the assumption of VVC.

[0036] It should be noted that the electronic devices (120) and (130) may include other components (not shown). For example, electronic device (120) may include a video decoder (not shown), and similarly, electronic device (130) may include a video encoder (not shown).

[0037] Figure 2 shows an example block diagram of a video decoder (210). The video decoder (210) can be included in an electronic device (230). The electronic device (230) can include a receiver (231) (for example, a receiving circuit). The video decoder (210) can be used in place of the video decoder (110) in the example of Figure 1.

[0038] The receiver (231) may receive one or more coded video sequences (e.g., included in a bitstream) which will be decoded by the video decoder (210). In one embodiment, one coded video sequence is received at a time, and the decoding of each coded video sequence is independent of the decoding of other coded video sequences. The coded video sequences may be received from a channel (201), which may be hardware / software connected to a storage device that stores coded video data. The receiver (231) may receive coded video data together with other data, such as coded audio data and / or an associated data stream, and may forward such other data to the corresponding required device (not shown). The receiver (231) may isolate the coded video sequences from other data. To address network jitter, a buffer memory (215) may be connected between the receiver (231) and the entropy decoder / parser (220) (hereinafter referred to as "parser (220)"). In some applications, the buffer memory (215) is part of the video decoder (210). In other applications, the buffer memory (215) can be located outside the video decoder (210) (not shown). In yet another application, a buffer memory (not shown) can be located outside the video decoder (210), for example, to handle network jitter, and in addition, another buffer memory (215) can be located inside the video decoder (210), for example, to handle playout timing. If the receiver (231) receives data from a storage / transfer device or isochronous network with sufficient bandwidth and controllability, the buffer memory (215) may not be required, or it may be small in size.When used in best-effort packet networks such as the Internet, a buffer memory (215) may be required. It is beneficial that the buffer memory (215) can be relatively large and also be of an optimal size. The buffer memory (215) may be implemented at least partially by an operating system or similar element (not shown) outside the video decoder (210).

[0039] The video decoder (210) may include a parser (220) that reconstructs symbols (221) from the coded video sequence. Such a category of symbols may include information used to manage the operation of the video decoder (210) and may include information controlling drawing devices, such as a drawing device (212) (e.g., a display screen), which is not an integral part of the electronic device (230) but can be connected to the electronic device (230), as shown in Figure 2. Control information for one or more drawing devices may take the form of Supplemental Enhancement Information (SEI) messages or Video Usability Information (VUI) parameter set fragments (not shown). The parser (220) can parse / entropy decode the received coded video sequence. The coding of the coded video sequence can conform to video coding techniques and standards and can follow various principles, including variable-length coding, Huffman coding, and arithmetic coding with or without context dependency. The parser (220) may extract from the coded video sequence a set of subgroup parameters for at least one subgroup of pixels in the video decoder, based on at least one parameter corresponding to the subgroup. The subgroups may include Group of Picture (GOP), picture, tile, slice, macroblock, Coding Unit (CU), block, Transform Unit (TU), Prediction Unit (PU), etc. The parser (220) may also extract from coded video sequence information such as transformation coefficients, quantizer parameter values, and motion vectors.

[0040] The parser (220) may create a symbol (221) by performing an entropy decoding / parser operation on the video sequence received from the buffer memory (215).

[0041] The reconstruction of the symbol (221) may involve multiple different regions depending on the type of coded video picture or its parts (e.g., between and within pictures, between and within blocks) and other factors. Which regions are involved and how they are involved can be controlled by subgroup control information parsed from the coded video sequence by the parser (220). For clarity, the flow of such subgroup control information between the parser (220) and the multiple regions described below is not shown.

[0042] To elaborate on the functional blocks already described, the video decoder (210) can be conceptually subdivided into multiple functional parts, as described below. In actual implementations operating under commercial constraints, many of these parts interact closely with each other and can be at least partially integrated. On the other hand, conceptually subdividing into the functional parts described below is suitable for explaining the disclosed protected object.

[0043] The first part is the scaler / inverse transformer (251). The scaler / inverse transformer (251) receives from the parser (220) as one or more symbols (221) containing quantized transformation coefficients and control information including the transformation to be used, block size, quantization factor, and quantization scaling matrix. The scaler / inverse transformer (251) can output a block containing sample values ​​that can be input to the aggregater (255).

[0044] In some examples, the output samples from the scaler / inverse transformer (251) may be suitable for intracoded blocks. While intracoded blocks do not use predictive information from previously reconstructed pictures, they can use predictive information from previously reconstructed portions of the current picture. Such predictive information can be provided by the in-picture predictor (252). In some examples, the in-picture predictor (252) generates a block of the same size and shape as the block being reconstructed, using surrounding, already reconstructed information obtained from the current picture buffer (258). The current picture buffer (258) buffers, for example, partially reconstructed current pictures and / or fully reconstructed current pictures. In some examples, the aggregater (255) adds, sample by sample, the predictive information generated by the intra-predictor (252) to the output sample information provided by the scaler / inverse transformer (251).

[0045] In other examples, the output samples of the scaler / inverse transformer (251) may be suitable for an intercoded block, and in some cases, for a motion-compensated block. In such examples, the motion-compensated prediction unit (253) can access the reference picture memory (257) to obtain samples to be used for prediction. Depending on the symbols (221) related to the block, the obtained samples can be motion-compensated, and then these samples can be added by an aggregater (255) to the output of the scaler / inverse transformer (251) (referred to in this example as residual samples or residual signals) to generate output sample information. The address in the reference picture memory (257) from which the motion-compensated prediction unit (253) obtains prediction samples can be controlled by a motion vector, which is available to the motion-compensated prediction unit (253) in the form of a symbol (221) that may have, for example, X, Y, and reference picture components. Motion compensation may also include interpolation of sample values ​​obtained from reference picture memory (257), motion vector prediction mechanisms, etc., when accurate motion vectors of subsamples are used.

[0046] The output samples of the aggregater (255) can be subjected to various loop filtering techniques in the loop filter section (256). The video compression technique may include an in-loop filtering technique, which is controlled by parameters as symbols (221) from the parser (220) that are included in the coded video sequence (also called the coded video bitstream) and made available to the loop filter section (256). The video compression may also correspond to metadata obtained when decoding earlier (decoding order) parts of the coded picture or coded video sequence, and may also correspond to previously reconstructed and loop-filtered sample values.

[0047] The output of the loop filter section (256) can be a sample stream that can be output to the drawing device (212) as well as stored in the reference picture memory (257) for use in subsequent picture prediction.

[0048] Some coded pictures, once fully reconstructed, can be used as reference pictures for later prediction. For example, once the coded picture corresponding to the current picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the parser (220)), the current picture buffer (258) can become part of the reference picture memory (257), and a new current picture buffer can be reallocated before starting the reconstruction of the next coded picture.

[0049] The video decoder (210) may perform decoding operations according to a default video compression technique or a standard such as ITU-T Recommendation H.265. A coded video sequence can be said to conform to the syntax defined by the video compression technique and standard being used, in the sense that the coded video sequence conforms to both the syntax of the video compression technique and standard and the profile described in the documentation of the video compression technique and standard. In particular, with respect to the profile, a predetermined tool can be selected from all the tools available in the video compression technique and standard as the only tool that can be used in accordance with that profile. It can also be said that the complexity of the coded video sequence must be within the limits defined by the level of the video compression technique and standard for compliance to be necessary. In some cases, depending on the level, there may be limitations on the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured at 1 million samples per second), maximum reference picture size, etc. The limitations imposed by the level may, in some cases, be further restricted by the Hypothetical Reference Decoder (HRD) specification and the metadata for HRD buffer management signaled in the coded video sequence.

[0050] In one embodiment, the receiver (231) may receive accompanying (redundant) data along with the encoded video. The accompanying data may be included as part of the coded video sequence. The accompanying data may be used by the video decoder (210) to appropriately decode the data and / or more accurately reconstruct the original video data. For example, the accompanying data may take the form of a temporal, spatial, or signal-to-noise ratio (SNR) enhancement layer, a redundant slice, a redundant picture, or a forward error correction code.

[0051] Figure 3 shows an example block diagram of a video encoder (303). The video encoder (303) is included in the electronic device (320). The electronic device (320) includes a transmitter (340) (e.g., a transmitting circuit). The video encoder (303) can be used in place of the video encoder (103) in the example of Figure 1.

[0052] The video encoder (303) may receive video samples from a video source (301) (not part of the electronic device (320) in the example of Figure 3) that can capture one or more video images to be coded by the video encoder (303). In another example, the video source (301) is part of the electronic device (320).

[0053] The video source (301) may provide a source video sequence that will be coded by the video encoder (303) as a digital video sample stream capable of any appropriate bit depth (e.g., 8 bits, 10 bits, 12 bits, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any appropriate sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media delivery system, the video source (301) may be a storage device that stores pre-prepared video. In a video conferencing system, the video source (301) may be a camera that captures local image information as a video sequence. The video data may be provided as a series of individual pictures that, when viewed sequentially, create motion. The pictures may be organized as a spatial array of pixels, each pixel containing one or more samples depending on the sampling structure, color space, etc., used. The following description will focus on samples.

[0054] In one embodiment, the video encoder (303) may encode and compress the pictures of the source video sequence into a coded video sequence (343) in real time, or it may encode and compress under any other time constraints as needed. Maintaining an appropriate coding speed is one of the functions of the controller (350). In some embodiments, the controller (350) controls and is functionally connected to other functional components described below. For clarity, the connections are not shown. Parameters set by the controller (350) may include rate control-related parameters (picture skip, quantizer, λ value of rate-distortion optimization method, ...), picture size, group of picture (GOP) layout, search range for the longest motion vector, etc. The controller (350) can be configured to have other appropriate functions for the video encoder (303) optimized for a particular system design.

[0055] In some embodiments, the video encoder (303) is configured to operate in a coding loop. In the simplest possible explanation, one example may include a source coder (330) (responsible for creating symbols, such as a symbol stream, based on the input picture to be coded and one or more reference pictures) and a decoder (333) built into the video encoder (303) (located here and not elsewhere). The decoder (333) reconstructs the symbols to create sample data as if it were also created by a decoder (located elsewhere). The reconstructed sample stream (sample data) is input to the reference picture memory (334). Decoding the symbol stream yields accurate bit results regardless of the decoder's location (whether it is here or elsewhere), so that the contents of the reference picture memory (334) are also bit-accurate when comparing the encoder at this location with the encoder at another location. In other words, the predictive part of the encoder accurately "sees" the reference picture samples as if the decoder "sees" the same sample values ​​when using prediction during decoding. The fundamental principles of simultaneity in reference pictures (and even the resulting drift when simultaneity cannot be maintained due to channel errors, for example) are also used in several related technologies.

[0056] The operation of the "local" decoder (333) can be the same as that of the "remote" video decoder (210), which has already been described in detail above with reference to Figure 2. However, as also referring somewhat to Figure 2, since the symbols are valid and the decoding of the symbols into the encoded / coded video sequence by the entropicorder (345) and parser (220) is reversible, the entropy decoding portion of the video decoder (210), which includes the buffer memory (215) and parser (220), may not be fully performed by the local decoder (333).

[0057] In one embodiment, decoder techniques other than parse / entropy decoding present in the decoder are present in the corresponding encoder in the same or substantially the same functional form. Therefore, the protected subject matter of the disclosure focuses on the operation of the decoder. The description of the encoder technique can be omitted as it is the inverse of the decoder technique described as a whole. A more detailed description of specific parts is provided below.

[0058] In operation, in some examples, the source coder (330) may perform motion-compensated predictive coding, which predicts and codes the input picture by referencing one or more previously coded pictures obtained from a video sequence designated as a “reference picture”. In this way, the coding engine (332) codes the difference between the pixel blocks of the input picture and the pixel blocks of one or more reference pictures that can be selected as one or more predictive references for the input picture.

[0059] The video decoder (333) located at this location may decode the coded video data of a picture that can be designated as a reference picture based on symbols created by the source coder (330). It may be beneficial for the operation of the coding engine (332) to be a lossy process. If the coded video data is decoded by some video decoder (not shown in Figure 3), the reconstructed video sequence may generally be a copy of the source video sequence with some error. The video decoder (333) located at this location may repeat the decoding process that may be performed by the video decoder on the reference picture so that the reconstructed reference picture is stored in the reference picture memory (334). In this way, the video encoder (303) can store at this location a copy of the reconstructed reference picture that has content common to the reconstructed reference picture that will be obtained by the video decoder at the far end (without transmission error).

[0060] The predictor (335) may perform a predictive search for the coding engine (332). That is, for a new picture to be coded, the predictor (335) may search the reference picture memory (334) to obtain sample data (sample data as candidate reference pixel blocks) or specific metadata such as reference picture motion vectors and block shapes that can carry out appropriate predictive references for the new picture. The predictor (335) may operate by using one sample block for each pixel block to find an appropriate predictive reference. In some examples, when determined by the search results obtained by the predictor (335), the input picture may include predictive references obtained from multiple reference pictures stored in the reference picture memory (334).

[0061] For example, the controller (350) may manage the coding operations of the source coder (330), including setting parameters and subgroup parameters used to encode video data.

[0062] The outputs of all the functional parts described above may undergo entropic coding by the entropicorder (345). The entropicorder (345) converts the symbols generated by the various functional parts into a coded video sequence by applying lossless compression to the symbols according to techniques such as Huffman coding, variable-length coding, and arithmetic coding.

[0063] The transmitter (340) may buffer one or more coded video sequences created by the entropicorder (345) in preparation for transmission over the communication channel (360), and the transmitter (340) may be hardware / software connected to a storage device that stores coded video data. The transmitter (340) may integrate other data to be transmitted, such as coded audio data and / or accompanying data streams (source not shown), with the coded video data obtained from the video encoder (303).

[0064] The controller (350) may manage the operation of the video encoder (303). During coding, the controller (350) may assign each coded picture to a specific coded picture type that can influence the coding method that can be applied to each picture. For example, in many cases, a picture can be assigned as one of the following picture types:

[0065] An Intra Picture (I-Picture) allows coding and decoding to be performed without using any other pictures in the sequence as a basis for prediction. Several video codecs support various types of intra pictures, including, for example, Independent Decoder Refresh ("IDR") pictures.

[0066] Predictive picture (P-picture) can use intra-prediction or inter-prediction to predict the sample values ​​of each block, and can perform coding and decoding using motion vectors and reference indices.

[0067] A bidirectionally predictive picture (B-picture) can use intra-prediction or inter-prediction to code and decode using motion vectors and reference indices to predict the sample values ​​of each block. Similarly, a multiple-predictive picture can use three or more reference pictures and associated metadata to reconstruct a single block.

[0068] Generally, a source picture can be spatially subdivided into multiple sample blocks (for example, blocks of 4x4, 8x8, 4x8, or 16x16 samples), and each block can be coded individually. Blocks may be coded by referencing other (already coded) blocks, as determined by the coding assignment applied to the picture corresponding to each block. For example, blocks of picture I may be coded without prediction, or they may be coded with prediction by referencing already coded blocks of the same picture (spatial prediction or intra prediction). Pixel blocks of picture P may be coded with prediction, either by spatial prediction or by referencing one previously coded reference picture with temporal prediction. Blocks of picture B may be coded with prediction, either by spatial prediction or by referencing one or two previously coded reference pictures with temporal prediction.

[0069] The video encoder (303) may perform coding operations according to a default video coding technique or a standard such as ITU-T Recommendation H.265. In such operations, the video encoder (303) may perform various compression operations, including predictive coding operations that utilize the temporal and spatial redundancy of the input video sequence. Therefore, the coded video data may conform to the syntax defined by the video coding technique or standard being used.

[0070] In one embodiment, the transmitter (340) may transmit accompanying data along with the encoded video. The source coder (330) may include such data as part of the coded video sequence. The accompanying data may include a temporal / spatial / SNR enhancement layer, other forms of redundant data such as a redundant picture or redundant slice, SEI messages, VUI parameter set fragments, and the like.

[0071] The video may be captured in chronological order as multiple source pictures (video pictures). Intra-picture prediction (often abbreviated as intra-prediction) utilizes spatial correlations within a given picture, while inter-picture prediction utilizes correlations (temporal correlations or other correlations) between pictures. In one example, a specific picture being encoded / decoded, called the current picture, is divided into blocks. If a block in the current picture resembles a reference block in a previously coded and still buffered reference picture in the video, the block in the current picture can be coded by a vector called a motion vector. The motion vector points towards the reference block in the reference picture, and if multiple reference pictures are used, a third dimension can be used to identify the reference picture.

[0072] In some embodiments, a bi-prediction technique can be used for picture-to-picture prediction. The bi-prediction technique uses two reference pictures, such as a first and second reference picture, which are both ahead of the current picture in the video in the decoding order (or, conversely, ahead of and ahead of the current picture in the display order). The block in the current picture can be coded by a first motion vector pointing towards the first reference block in the first reference picture and a second motion vector pointing towards the second reference block in the second reference picture. The block can be predicted by the combination of the first and second reference blocks.

[0073] Furthermore, coding efficiency can be improved by using the merge mode method for picture-to-picture prediction.

[0074] In some aspects of this disclosure, predictions such as inter-picture prediction and intra-picture prediction are performed in block units. For example, according to the HEVC standard, for compression, pictures in a series of video pictures are divided into coding tree units (CTUs), and the CTUs in a picture have the same size (e.g., 64x64 pixels, 32x32 pixels, or 16x16 pixels). Generally, a CTU contains three coding tree blocks (CTBs), and the three CTBs are one luma CTB and two chroma CTBs. Each CTU can be recursively quadtree-partitioned into one or more coding units (CUs). For example, a CTU with 64x64 pixels can be divided into one CU with 64x64 pixels, four CUs with 32x32 pixels, or sixteen CUs with 16x16 pixels. In one example, each CU is analyzed to determine the prediction type of the CU, such as an inter-prediction type or an intra-prediction type. Depending on temporal and / or spatial predictability, the CU is divided into one or more prediction units (PUs). Generally, each PU contains one luma prediction block (PB) and two chroma PBs. In one embodiment, the coding prediction operation (encoding / decoding) is performed in units of prediction blocks. When a luma prediction block is used as an example of a prediction block, the prediction block contains a matrix of values ​​(e.g., luma values) of pixels such as 8x8 pixels, 16x16 pixels, 8x16 pixels, and 16x8 pixels.

[0075] It should be noted that the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using any appropriate method. In one embodiment, the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using one or more integrated circuits. In another embodiment, the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using one or more processors that execute software instructions.

[0076] Intra-convolutional prediction models are sometimes used in ECM and other applications. Convolutional cross-component models (CCCMs) can be applied to predict chroma samples from reconstructed luma samples, similar to how it is currently done by CCLM modes. Similar to CCLM, when chroma subsampling is used, the reconstructed luma samples can be downsampled to fit a lower-resolution chroma grid. Similar to CCLM, the top reference sample, left reference sample, or top-left reference sample can be used as a template for model derivation.

[0077] Similar to CCLM, you can choose to use a single CCCM model or a variant of CCCM with multiple models. In the variant with multiple models, two models can be used: one model derived for samples exceeding the average luma reference value, and the other model for the remaining samples (following the intent of the CCLM design). The multi-model CCCM mode can be selected, for example, for PUs where at least 128 reference samples are available.

[0078] Convolutional filters, such as a 7-tap convolutional filter, may include a 5-tap plus-shaped spatial component, a nonlinear term, and a bias term (for example, these may be the only components). The input to the 5-tap spatial component of the filter may include a central (C) luma sample placed together with the predicted chroma sample, as shown in Figure 4, and neighboring samples above (i.e., north, north, south, left, west, and right, east, e).

[0079] The nonlinear term NP may be expressed as a power of 2 of the central luma sample C, or it may be scaled to the sample value range of the content as described in Equation 1. NP=(C2 +midVal)>>bitDepth Equation 1

[0080] In other words, if the content is 10 bits, the nonlinear term NP may be calculated using Equation 2. NP=(C 2 +512)>>10 Formula 2

[0081] The middle value (midVal) is 2 10 / 2 is 512.

[0082] The bias term B can be said to represent a scalar offset between the input and output (similar to the offset term in CCLM, for example), and may be set to the middle chroma value (for example, 512 if the content is 10 bits).

[0083] The output of the filter is the filter coefficient c. i It can be calculated as a convolution between the input value and the sample, or it can be clipped using Equation 3 to obtain the range of valid chroma samples. predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6B Formula 3

[0084] Filter coefficient c i This may be calculated, for example, by minimizing the mean squared error (MSE) between the reconstructed chroma sample in the reference region and the predicted chroma sample. Figure 5 shows an example of a reference region (with padding) used to derive the filter coefficient according to one aspect of the present disclosure. The reference region may include chroma samples from the six lines above and to the left of the PU (for example, consisting of these). The reference region may be extended to the right of the PU boundary by the width of the PU and below by the height of the PU. The region may be adjusted to include only available samples. The extended portion of the region may be used to support "side samples" of a plus-shaped spatial filter, which are padded where they are in an unavailable region.

[0085] Minimizing the MSE may be performed by calculating the autocorrelation matrix of the luma input and the cross-correlation vector between the luma input and the chroma output. Alternatively, LDL decomposition may be performed on the autocorrelation matrix, and the final filter coefficients may be calculated using back substitution. This process closely follows the calculation of ALF filter coefficients as used in ECM, but LDL decomposition was chosen instead of Cholesky decomposition to avoid the use of square root operations.

[0086] The autocorrelation matrix may be calculated using the reconstructed values ​​of the luma and chroma samples. The luma and chroma samples may be in the maximum range (for example, between 0 and 1023 if the content is 10 bits), but this will result in a relatively large value for the autocorrelation matrix. This may lead to the use of operations with a large bit depth when calculating model parameters. Offsets determined from the luma and chroma samples of each PU may be removed for each model. This can reduce the size of the values ​​used in model creation and allow for lower precision in fixed-point operations. As a result, in some examples, 16-bit decimal precision may be used instead of the 22-bit precision of the original CCCM implementation.

[0087] In some examples, for simplicity, a reference sample value located just outside the upper-left corner of the PU may be used as an offset (offsetLuma, offsetCb, and offsetCr). The sample values ​​used for both model creation and final prediction (e.g., the luma and chroma of the reference region and the luma of the current PU) may be reduced by a predetermined value as follows: C'=C-offsetLuma, N'=N-offsetLuma, S'=S-offsetLuma, E'=E-offsetLuma, W'=W-offsetLuma, P'=nonLinear(C'), B=midValue=1<<(bitDepth-1), and the chroma value is predicted using Equation 4. Here, offsetChroma is equal to offsetCr and offsetCb used for the Cr component and Cb component, respectively. predChromaVal=c0C'+c1N'+c2S'+c3E'+c4W'+c5P'+c6B+offsetChroma Formula 4

[0088] For example, to avoid further sample-level calculations, the luma offset is removed during the interpolation of luma reference samples. This can be done, for instance, by replacing the rounding term used in the interpolation of luma reference samples with an updated offset that includes both the rounding term and offsetLuma. The chroma offset can also be removed by directly subtracting the chroma offset from the reference chroma samples. Alternatively, the same result can be obtained by removing the effect of the chroma offset from the intercomponent vector. The chroma offset may also be added to the bias term of the convolution model in order to add it back to the output of the convolution prediction operation.

[0089] Division operations may be used in the process of calculating CCCM model parameters. In some cases, performing division operations may not be easy. Division operations may be replaced with multiplication (using a scale factor) and shift operations, and the scale factor and shift number may be calculated based on the denominator, for example, in the same way as used in the calculation of CCLM parameters.

[0090] Using a filter that uses an input including one spatial luma sample, two gradient values, two position information, a non-linear term, and a bias term (for example, consisting of these) by a gradient and location based convolutional cross-component model (GL-CCCM), the luma value may be made to correspond to the chroma value. In the GL-CCCM method, gradient information and position information can be used instead of four samples that are spatially neighboring used in the CCCM filter. The GL-CCCM filter used for prediction can be described using Equation 5. predChromaVal=c0C+c1G y +c2G x +c3Y+c4X+c5P+c6B Equation 5

[0091] FIG. 6 shows an example of spatial samples used for GL-CCCM according to one aspect of the present disclosure. Gy and Gx are the vertical gradient and the horizontal gradient respectively, and are calculated using Equation 6. G y =(2N+NW+NE)-(2S+SW+SE) G x =(2W+NW+SW)-(2E+NE+SE) Equation 6

[0092] Y and X are the spatial coordinates of the central luma sample.

[0093] The remaining parameters may be the same as those of the CCCM tool. The reference region used for parameter calculation may be the same as that of the CCCM method.

[0094] The use of the GL-CCCM mode may be signaled by a flag such as a CABAC-coded PU-level flag. The GL-CCCM mode can be regarded as a sub-mode of CCCM in terms of signaling. For example, the GL-CCCM flag is signaled only when the original CCCM flag is true.

[0095] Similar to CCCM, in some examples, the GL-CCCM tool has six modes used to calculate parameters: single-model GL-CCCM from the top and left templates, single-model GL-CCCM from the top template, single-model GL-CCCM from the left template, multi-model GL-CCCM from the top and left templates, multi-model GL-CCCM from the top template, and multi-model GL-CCCM from the left template. The encoder may also explore the six GL-CCCM modes (for example, a sum of absolute transformed differences (SATD) search) along with the existing CCCM modes to find the best candidate for a complete rate-distortion (RD) test.

[0096] To predict the blocks currently in the picture, for example, from reference blocks in the current picture (for example, reconstructed blocks), you may use intra-block copy (IBC) mode or intra-template matching prediction (intraTMP) mode.

[0097] In one embodiment, IBC mode is a tool used when extending HEVC in screen content coding (SCC). In some examples, IBC mode can significantly improve the coding efficiency of screen content materials. Since IBC mode can be implemented as a block-level coding mode, the encoder can perform block matching (BM) to find the optimal block vector (BV) for each CU. In IBC mode, the BV may be used to indicate the displacement from the current block to the reference block, which has already been reconstructed in the current picture. In one example, the BV can be considered a motion vector when the reference picture is the current picture. The luma BV of an IBC-coded CU may be an integer-precision BV. The chroma BV may be rounded to integer precision. When combined with adaptive motion vector resolution (AMVR) mode, the precision of IBC mode can be switched between 1-pixel precision and 4-pixel precision (for example, also referred to as 1-pel and 4-pel motion vector precisions). In addition to the intra-prediction mode and inter-prediction mode, the IBC mode, which can code IBC-coded CUs, may be treated as a third prediction mode. The IBC mode may be applicable to CUs where both the width and height are 64 luma samples or less.

[0098] On the encoder side, motion estimation using hashing may be performed in IBC mode. The encoder may perform rate-distortion (RD) checks on blocks where either the width or height is less than 16 luman samples. In non-merge mode, block vector search may be performed first using hashing. If no valid candidates are returned by hashing, local search using block matching may be performed.

[0099] In hash-based searches, the hash key matching (32-bit CRC) between the current block and the reference block may be extended to all allowed block sizes. Hash key calculations for all positions in the current picture may be performed based on 4x4 subblocks. If the current block is large, a hash key may be considered a match with the reference block's hash key when all hash keys in all 4x4 subblocks match the hash keys of the corresponding reference positions. If it is found that the hash keys of multiple reference blocks match the hash key of the current block, the block vector cost of each matching reference may be calculated, and the one with the smallest cost may be selected.

[0100] In block matching searches, the search range may be set to include both the previous CTU and the current CTU (for example, the previously reconstructed CTU and the current CTU).

[0101] At the CU level, IBC modes may be signaled using flags, or they may be signaled as IBC adaptive motion vector prediction (AMVP) mode or IBC skip / merge mode, as shown below.

[0102] In IBC skip / merge mode: A merge candidate index may be used to indicate which block vector from the merge list obtained from IBC-coded neighboring block candidates is currently used to predict the block. The merge list may contain one or more spatial candidates, one or more HMVP candidates, and one or more pairwise candidates. In one example, the merge list consists of one or more spatial candidates, one or more HMVP candidates, and one or more pairwise candidates.

[0103] In IBC AMVP mode: Block vector differences (BVDs) may be coded in the same way as motion vector differences. The block vector prediction method may use two candidate predictors: one obtained from the left neighbor (if IBC coded) and one obtained from the upper neighbor (if IBC coded). If neither neighbor is available, the initial BV may be used as the predictor. A flag indicating the block vector predictor index may be signaled.

[0104] Figure 7A shows an example of an Intra-Template Matching Prediction (IntraTMP) mode according to one aspect of the present disclosure. In one aspect, like Enhanced Compression Model (ECM) software, IntraTMP is a special intra-prediction mode that can replicate the best-prediction block (e.g., a matching block (721)) from a reconfigured portion of the current frame (i.e., the current picture). In this mode, the template (e.g., an L-shaped template) (720) of the best-prediction block can match the current template (730) of the current block (731) (e.g., current PU or current CU). Within a predetermined search range, the encoder can search for the template most similar to the current template in the reconfigured portion of the current frame, and the corresponding block can be used as the prediction block. The encoder can signal the use of IntraTMP mode, and the decoder can perform the same prediction operation.

[0105] A prediction signal can be generated by matching the current template (730), for example, an L-shaped portion of the current block (731) with the template of another block within a predetermined search region. The example search region shown in Figure 7A can include multiple CTUs (i.e., multiple superblocks). Referring to Figure 7A, the search region can include the current CTU R1 (for example, a part of the current CTU R1), the upper left CTU R2, the upper CTU R3, and the left CTU R4. The cost function can be any appropriate cost function, such as the sum of absolute differences (SAD).

[0106] Within each domain, the decoder can search for the template with the lowest cost (for example, the one with the lowest SAD) relative to the current template, and the blocks associated with the template with the lowest cost can be used as prediction blocks.

[0107] The dimensions of the region indicated by (SearchRange_w, SearchRange_h) can be set to be proportional to the block dimensions (BlkW, BlkH) to keep the number of SAD comparisons per pixel constant. Therefore, SearchRange_w = a × BlkW and SearchRange_h = a × BlkH.

[0108] Parameter 'a' can be a constant that adjusts the trade-off between gain and complexity. In one example, 'a' is 5.

[0109] In one example, to speed up the template matching process, the search range (e.g., the search range of the entire search area) is subsampled by a factor of two, which reduces the template matching search by a factor of four. After the best match (i.e., the first best match) is found, an improvement process can be performed. Improvement is done by narrowing the range and performing a second template matching search around the best match (i.e., the first best match mentioned above). The narrowed range is defined as min(BlkW,BlkH) / 2.

[0110] The intra-template matching tool can be enabled for CUs whose width and height are 64 or less. The maximum CU size for intra-template matching (e.g., 64) can be set to a size that is configurable.

[0111] For example, a filtered intrablock copy (FIBC) model using IBC mode or IntraTMP mode may be used. In one embodiment, the predicted samples of IBC mode or IntraTMP mode may be enhanced by applying a linear filter. Returning to Figure 4, in one example, the linear filter consists of five spatial terms and a bias term, as shown in Equation 7. The five spatial terms consist of the central (C) position, the upper / north neighbor (N), the lower / south neighbor (S), the left / west neighbor (W), and the right / east neighbor (E). predVal=α0·C+α1·N+α2·S+α3·W+α4·E+α5·β Equation 7

[0112] α iα is a coefficient (for example, i is between 0 and 5), and β is an offset related to the bias term. Filter coefficients (for example, including α0 to α5) may be derived by applying up to 4 rows / columns of samples above and to the left of the current CU. Filter coefficients may also be derived based on minimizing the difference between a template sample and a corresponding reference sample using a regression-based minimization method, such as a similar regression-based minimization method of ECM used in other tools such as CCCM described in this disclosure.

[0113] For signaling purposes, an additional notification flag (also called the FIBC flag) may be introduced in FIBC mode, and this additional notification flag may be signaled after the IBC-local illumination compensation (LIC) flag. For example, if the IBC-LIC flag is true, the FIBC flag may be signaled to indicate whether or not FIBC mode is applied to the current block.

[0114] In the FIBC mode described above, a filtered IBC model may be used, in which the predicted samples of the IBC mode are enhanced by applying a linear filter. Currently, the filter coefficients may be derived by applying up to 4 rows / columns of samples above and to the left of the CU. In various examples, the design of the FIBC filter as described in Equation 7 may not be sufficiently accurate because the filter consists only of linear terms of sample values ​​and does not use other information such as gradient information or positional information. Furthermore, the filter in the FIBC mode described above does not include nonlinear terms.

[0115] Aspects of this disclosure provide a method for improving FIBC modes by including at least one of gradient information, position information, nonlinear terms and / or similar when deriving the filter coefficients of the FIBC modes.

[0116] The methods, embodiments, and examples described herein may be used separately or in any order. The term “IBC mode” may refer to the IBC mode described herein or a variation thereof. The term “IntraTMP mode” may refer to the IntraTMP mode described herein or a variation thereof. Furthermore, the methods, embodiments, and examples may be implemented by processing circuits (for example, one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-temporary computer-readable medium.

[0117] Figure 7B shows an example of a modification of the FIBC mode according to one aspect of the present disclosure. The current block (701) in the current picture (700) may be predicted using either the IBC mode or the IntraTMP mode. In one example, BV(702) is determined using either the IBC mode or the IntraTMP mode. BV(702) may represent a reference block (703) of the current block (701). The reference block (703) is in the current picture (700) and may have already been reconstructed. The reference samples in the reference block (703) may have already been reconstructed and be used to predict the current block (701). Thus, the reference samples in the reference block (703) may be referred to as IBC prediction samples. According to one aspect of the present disclosure, in the FIBC mode (for example, an updated FIBC mode modified from the FIBC mode described in Equation 7), the IBC prediction samples may be filtered by an FIBC filter. In this way, a filtered reference block (703) can be generated, which may contain IBC prediction samples filtered by the FIBC filter. The current sample in the current block (701) can be predicted using IBC predicted samples filtered by the FIBC filter.

[0118] Referring to Figure 7B, in one example, the current sample (710) in the current block (701) is predicted using an updated FIBC mode based on further information (e.g., gradient information, position information, and / or nonlinear terms). The current sample (710) corresponds to the reference sample (711) in the reference block (703), where, for example, BV(702) indicates the displacement between the reference sample (711) and the current sample (710). The current sample (710) may also be predicted using an IBC predicted sample (711) filtered by an FIBC filter. After being filtered by the FIBC filter, the reference sample (711) may be referred to as the FIBC filtered predicted sample (711). For example, the predicted value of the current sample (710) is the value of the FIBC filtered predicted sample (711). The value of the FIBC filtered predicted sample may also be referred to as the predicted value, because, for example, this value may be used directly as the predicted value for the current sample (710).

[0119] In one embodiment, in addition to a linear filter that may be similar to or identical to the filter of Equation 7, the FIBC filter may further include (i) at least one gradient filter (e.g., showing gradient information related to each IBC prediction sample), (ii) a position filter (e.g., showing position information related to the linear filter), and (iii) one or more nonlinear terms (also referred to as nonlinear values) and / or similar related to each IBC prediction sample.

[0120] The coefficients of an FIBC filter (also called filter coefficients) may include the linear coefficients of a linear filter and one or more of the following: (i) the gradient coefficient of at least one gradient filter, (ii) the position coefficient of a position filter, or (iii) one or more nonlinear coefficients of a nonlinear term. If the FIBC filter contains other information, the filter coefficients of the FIBC filter may include yet another coefficient.

[0121] In one embodiment, the coefficients of the FIBC filter may be determined from the current template (704) of the current block (701) and the reference template (705) of the reference block (703).

[0122] The details of the method, form, and examples of FIBC mode using an FIBC filter are described below.

[0123] In one embodiment, the gradient of the reconstructed neighboring sample (G x ,G y IBC prediction samples may be filtered by utilizing gradient information including ). The predicted value pred0(x,y) at (x,y) may be determined (for example, defined) using equations 8 to 12. pred0(x,y)=P0+GX+GY+B Equation 8

number

number

number

number

[0124] pred0(x,y) can be said to be the predicted value at (x,y). The four sets of parameters related to P0, GX, GY, and B may correspond to the input sample value, horizontal gradient information, vertical gradient information, and bias, respectively. P0 can be said to be the linear predicted value obtained using a linear filter as described in Equation 9. GX can be said to be the horizontal gradient value obtained using a horizontal gradient filter as described in Equation 10. GY can be said to be the vertical gradient value obtained using a vertical gradient filter as described in Equation 11. B can be said to be the bias term as described in Equation 12. In one example, the linear filter may include a bias term. In the example shown in Equation 8, at least one gradient filter may include both a horizontal gradient filter and a vertical gradient filter.

[0125] In one example, the current sample (710) is located at position (x,y) in the current block (701). The reference sample, i.e., the IBC predicted sample (711), is located at the same position (x,y) in the reference block (703). The predicted value pred0(x,y) at position (x,y) may be determined by applying an FIBC filter, as described in equations 8-12, to the IBC predicted sample (711) at (x,y).

[0126] Reference block (703) position (x-xoffset 0,i ,y-yoffset 0,i The reconstructed sample located at (x-xoffset) in reference block (703) may be referred to as the first input sample or one of the N0 input samples (where i is 1 to N0). 1,j ,y-yoffset 1,j The reconstructed sample located at (x-x offset) may be referred to as the second input sample or one of the N1 input samples (where j is 1 to N1). 2,k ,y-yoffset 2,k The reconstructed sample located at ) may be referred to as the third input sample or N2 input samples (where k is 1 to N2). Predicted values ​​pred0(x,y) may be obtained using N0 input samples, N1 input samples, and N2 input samples. The N0 input samples, N1 input samples, and N2 input samples may include (i) reconstructed neighboring samples of the reference sample (711), such as samples N, S, W, and E in the reference block (703) as shown in Figure 7B, and (ii) reconstructed samples and / or similar that are not adjacent to the reference sample (711). After the predicted values ​​pred0(x,y) have been determined, for example, for the reference sample (711), the current sample (710) may be predicted based on the predicted values ​​pred0(x,y). For example, the predicted value of the current sample (710) is equal to pred0(x,y). The current sample (710) may be reconstructed based on the predicted value of the current sample (710).

[0127] The first set (i.e., the first set of parameters) may relate to sample values ​​(for example, N1 input sample values), and these sample values ​​may relate to parameters (for example, coefficient c) of N0 input samples. 0,i (x-xoffset) may be included. 0,i ,y-yoffset 0,i ) is the position of the i-th input sample, and c 0,i is the coefficient of the i-th input sample, and t(x-xoffset 0,i ,y-yoffset 0,i ) is the value of the i-th input sample.

[0128] The linear filter used in the FIBC filter has coefficients c for N0 input samples. 0,i And may be based on position. The linear coefficient is the coefficient {c 0,i} may be included. The shape of the linear filter is an offset with respect to a reference position such as (x,y) {(xoffset 0,i ,yoffset 0,i )} may also be used as a basis.

[0129] Referring to Figure 7B, if N0 input samples include samples N, S, W, and E in reference block (703), then N0 is 4. If N0 input samples include samples N, S, W, E, and (711) in reference block (703), then N0 is 5. xoffset of N samples 0,i is 0, and yoffset 0,i It is -1.

[0130] The second set (i.e., the second set of parameters) may relate to horizontal gradient information, which may include the parameters of N1 input samples. 1,j The coefficient of the j-th input sample is G x (x-xoffset 1,j ,y-yoffset 1,j ) is the horizontal gradient G of the j-th input sample xThis is the value. In one example, N1 input samples may be identical to N0 input samples. In another example, N1 input samples may be different from N0 input samples.

[0131] The horizontal gradient filter used in the FIBC filter has coefficients c for N1 input samples. 1,j And may be based on position. The horizontal gradient coefficient is the coefficient {c 1,j} may be included. The shape of the horizontal gradient filter is an offset with respect to a reference position such as (x,y) {(x-xoffset 1,j ,y-yoffset 1,j )} may also be used as a basis.

[0132] The third set (i.e., the third set of parameters) may relate to vertical gradient information, which may include the parameters of N2 input samples. 2,k The coefficient of the k-th input sample is G y (x-xoffset 2,k ,y-yoffset 2,k ) is the vertical gradient G of the k-th input sample y This is a value. In one example, N2 input samples may be identical to N0 input samples. In another example, N2 input samples may be different from N0 input samples. In another example, N2 input samples may be identical to N1 input samples. In another example, N2 input samples may be different from N1 input samples.

[0133] The vertical gradient filter has coefficients c for N2 input samples. 2,k And may be based on position. The vertical gradient coefficient is the coefficient {c 2,k} may be included. The shape of the vertical gradient filter is an offset with respect to a reference position such as (x,y) {(x-xoffset 2,k ,y-yoffset 2,k )} may also be used as a basis.

[0134] A fourth set (i.e., a third set of parameters) may also relate to bias B, and therefore each bias bl The parameter b may also be included. l This is the l-th bias. 3,l The lth bias b l These are the coefficients. In one example, N3 can be zero, and the predicted value pred0(x,y) does not contain bias. In another example, the linear filter may contain bias B.

[0135] In one aspect of this disclosure, the linear predicted value of the current sample in the current block may be determined by applying a linear filter to the predicted sample predicted using either the IBC mode or the intra-template matching (IntraTMP) mode, as described in Equation 9. The gradient value associated with the current sample in the current block may be determined using at least one gradient filter, such as a horizontal gradient filter and / or a vertical gradient filter, as described in Equations 10-11. The predicted value of the current sample may be determined based on the sum of the linear predicted value and at least one modified value including the gradient value. The gradient value is based on at least one of GX and GY. The FIBC filter in FIBC mode may include a linear filter and at least one gradient filter. The current sample may be reconstructed from the predicted value of the current sample.

[0136] In one embodiment, the filters available in FIBC mode may have any suitable shape and / or size. In one example, a linear filter has a cross shape including five samples, as shown in Figure 8: a central sample of the linear filter with an offset of (0,0), a northern sample N with an offset of (0,-1), a southern sample S with an offset of (0,1), an eastern sample E with an offset of (1,0), and a western sample W with an offset of (-1,0). The offsets of the five samples of the linear filter are relative to the central sample. If the linear filter has five samples, the current sample may be located at one of the five positions, which are the positions of each of the five samples.

[0137] In one example, a linear filter has a cross shape containing nine samples, as shown in Figure 9: a central sample of the linear filter with an offset of (0,0), two northern samples with offsets of (0,-1) and (0,-2) respectively, two southern samples with offsets of (0,1) and (0,2) respectively, two eastern samples with offsets of (1,0) and (2,0) respectively, and two western samples with offsets of (-1,0) and (-2,0) respectively. The offsets of the nine samples of the linear filter are relative to the central sample. If the linear filter has nine samples, the current sample is located at one of the nine positions, which is the position of each of the nine samples.

[0138] Figures 8 to 10 show examples of filters available in FIBC mode according to one aspect of the present disclosure. In one example, the available filters include the cross-shaped filters (801) to (802) shown in Figures 8 to 9. In the example shown in Figure 8, if filter (801) is applied to N0 input samples, then N0 is 5, and 5 samples (for example, 5 input samples) are: c 0,i x is the coefficient of C, and x offset 0,i = 0 and yoffset 0,y The central sample (C) is equal to 0 (for example, i=1), c 0,i x is the coefficient of N, and x offset 0,i = 0 and yoffset 0,i For example, a sample (N) from the north where = -1 (i=2), c 0,i x is the coefficient of S, and x offset 0,i = 0 and yoffset 0,i A sample from the south (S) where =1 (for example, i=3), c 0,i x is the coefficient of E, and x offset 0,i = 1 and yoffset 0,i For example, sample (E) from the east where = 0 (i=4), c 0,i x is the coefficient of W, and x offset0,i = -1 and yoffset 0,i The western sample (W) is equal to 0 (for example, i=5) and Includes.

[0139] The filter shape (802) shown in Figure 9 may be interpreted similarly to the filter shape (801). For example, when the filter shape (802) is applied to N0 input samples, N0 is 9, and the N0 input samples include C, N, S, E, and W, which are identical to the samples described in Figure 8 (e.g., C, N, S, E, and W). The N0 input samples in Figure 9 further include samples NN, SS, EE, and WW. In one example, the sample offset for NN is (0, -2).

[0140] In the examples above, such as Figures 8 and 9, the current sample's position and the corresponding reference sample's position may be located at the central sample of the filter (e.g., (801) or (802)), and therefore the offset of the central sample of the filter (e.g., (801) or (802)) is (0,0) with respect to (x,y). The current sample's position and the corresponding reference sample's position are not limited to the positions shown in the examples (e.g., the central position), and other input samples can be used as the reference sample or the current sample. Referring to filter (801) in Figure 8, the current sample or the corresponding reference sample may be located at one of five positions, which are the positions of five input samples (e.g., C, N, S, E, or W). In one example, the reference sample is located at N, and therefore the offset of N is (0,0) with respect to (x,y), the offset of C is (0,1), and the offsets of the other samples are shifted accordingly. Referring to filter (802) in Figure 9, the current sample or the corresponding reference sample may be located at one of nine positions, which are the positions of the nine input samples. In one example, the reference sample corresponding to the current sample is located at W, and therefore the offset of W is (0,0) and the offset of C is (1,0), and the offsets of the other samples are shifted accordingly.

[0141] In one embodiment, the allowable offset (e.g., indicated by xoffset and yoffset) is defined for each sample i (e.g., each of N0 input samples) of the first set of parameters related to P0, GX, and GY, the second set of parameters, and the third set of parameters, respectively, and each G x Element j and each G y The element k may be predetermined. In one example, the positions of N0 input samples are predetermined, and therefore the shape of the linear filter is predetermined. At least one shape of one gradient filter is predetermined. In one example, the positions of N1 input samples are predetermined, and therefore the shape of the horizontal gradient filter is predetermined. In one example, the positions of N2 input samples are predetermined, and therefore the shape of the vertical gradient filter is predetermined.

[0142] In one embodiment, the distribution of the current sample and its neighboring samples, determined by (x-xoffset, y-yoffset), does not need to be contiguous. As described above, the position of the current sample in the current block and the position of the corresponding reference sample in the reference block are (x, y), and the distribution of the reference sample and its neighboring samples is determined by (x-xoffset, y-yoffset), and does not need to be contiguous. In one example, one input sample in a linear filter, horizontal gradient filter, and vertical gradient filter is spatially separated from all the remaining input samples in one of the linear filter, horizontal gradient filter, and vertical gradient filter. For example, one sample in a linear filter is spatially separated from all the remaining samples in the linear filter.

[0143] In one example, none of the input samples in one of the linear, horizontal, and vertical gradient filters are spatially adjacent to any other input sample in the same filter, as shown in Figure 10.

[0144] Figure 10 shows an example of a filter shape (803) without consecutive samples according to one aspect of the present disclosure. The filter shape, i.e., filter (803), is C, +2 x , -2 x , +2 y and -2 y It can include 5 input samples located at position 4 (for example, +2). x , -2 x , +2 y and -2 y Each of these is shifted by 2 samples from C, and therefore the offset is 2, with x / y indicating the direction. For example, position +2 x The offset from C is (+2,0). P0, GX and / or GY may be derived using the filter shape (803).

[0145] In one embodiment, several variations of the linear prediction value P0 may be used. For example, when an input (e.g., N0 input samples) is given to the FIBC input reference samples, an operation to remove the mean can be applied to P0. P1 is a variation of the linear prediction value and can be said to be P0 to which the operation to remove the mean has been applied, as described in Equation 13. The operation to remove the mean may be the same as the method used in CCCM. Subsequently, the prediction value pred1(x,y) of (x,y) can be generated using Equations 13 to 14.

number

[0146] In one example, the mean of the current block (e.g., the mean in Equation 13) is added by a linear filter (as described in Equation 13), and the predicted sample (e.g., IBC predicted sample t(x-x offset)) is calculated using either the IBC mode or the IntraTMP mode. 0,i ,y-yoffset 0,i Remove the current block's average value from each of the following.

[0147] For example, a clipping operation can be applied to generate the final predicted value pred2(x,y). pred2(x,y)=clip(min,max,predi(x,y)) Equation 15

[0148] The min and max values ​​can be said to be the minimum and maximum values ​​of the samples in the template, respectively. predi(x,y) can be said to be pred0(x,y), pred1(x,y), etc. In one example, the predicted value predi(x,y) of the current sample may be cut out as shown in Equation 15.

[0149] Further details regarding gradient information are explained below.

[0150] The number and position of different sets of parameters can be set independently. For example, N0 and the position of N0 input samples can be set independently. For example, N1 and the position of N1 input samples can be set independently. For example, N2 and the position of N2 input samples can be set independently. For example, a linear filter, a horizontal gradient filter, and a vertical gradient filter can be set independently.

[0151] In one embodiment, both GX and GY can be used as inputs to a filter in FIBC mode, where N1 is equal to N2, and the same sample position is used as input to both sets. In one example, the horizontal gradient filter and the vertical gradient filter have the same size and shape.

[0152] In one example, if at least one gradient filter includes a horizontal gradient filter and a vertical gradient filter, the gradient value is the sum of the horizontal gradient value (e.g., GX) and the vertical gradient value (e.g., GY), and the number of first input samples (e.g., N1 input samples) in the horizontal gradient filter and the positions of the first input samples, and the number of second input samples (e.g., N2 input samples) in the vertical gradient filter and the positions of the second input samples, are set independently of each other and independently of the linear filter (e.g., based on N0 and N0 input samples).

[0153] In one embodiment, both GX and GY can be used as inputs to a filter in FIBC mode, where N1 is equal to N2, but different sample positions are used as inputs to GX and GY, respectively. In one example, the positions of the first input samples in the horizontal gradient filter (e.g., N1 input samples) are different from the positions of the second input samples in the vertical gradient filter (e.g., N2 input samples). For example, the horizontal gradient filter and the vertical gradient filter may have the same size but different shapes.

[0154] If at least one gradient filter consists of a horizontal gradient filter, the gradient value is the horizontal gradient value, and the number N1 of first input samples in the horizontal gradient filter and the position of the first input samples are set independently of the linear filter.

[0155] In one embodiment, only GX can be used as an input to a filter in FIBC mode. For example, if at least one gradient filter consists of a horizontal gradient filter, the gradient value is the horizontal gradient value (e.g., GX), and the number of first input samples (e.g., N1 input samples) in the horizontal gradient filter, N1, and the position of the first input samples are set independently of the linear filter (e.g., based on N0 and N0 input samples). For example, pred0(x,y)=P0+GX or pred1(x,y)=P1+GX.

[0156] If at least one gradient filter consists of a vertical gradient filter, the gradient value is the vertical gradient value, and the number of second input samples and the positions of the second input samples in the vertical gradient filter are set independently of the linear filter.

[0157] In one embodiment, only GY can be used as the input to a filter in FIBC mode. For example, if at least one gradient filter consists of a vertical gradient filter, the gradient value is the vertical gradient value (e.g., GY), and the number of second input samples (e.g., N2 input samples) in the vertical gradient filter, N2, and the positions of the second input samples are set independently of the linear filter (e.g., based on N0 and N0 input samples). For example, pred0(x,y)=P0+GY or pred1(x,y)=P1+GY.

[0158] The gradient value is the horizontal gradient (G) of each of the first input samples (for example, N1 input samples), as shown in Equations 8, 10, and 11. x The horizontal gradient value (GX) determined based on ), or the vertical gradient G for each of the second input samples (e.g., N2 input samples). y It may be determined based on at least one of the vertical slope values ​​(GY) determined based on the above.

[0159] Horizontal gradient G x or vertical slope G y The gradients associated with the input samples, such as those shown, may be calculated using any appropriate method. Several examples are described below.

[0160] Figure 11 shows an example of a neighboring sample of input sample C used to calculate a gradient according to one aspect of the present disclosure. Given an input sample C and neighboring samples of input sample C (for example, those shown using "N", "S", "E", "W", "NW", "NE", "SW", and "SE" as described above in Figures 8-9), G x Ya G y The individual gradients of these can be calculated as follows.

[0161] In one aspect, G x and G y are calculated based on samples C, W, and N. G y =(C - N) G x =(C - W) Equation 16

[0162] In one aspect, G x and G y may be calculated based on samples N, S, W, and E. G y =(N - S) G x =(W - E) Equation 17

[0163] In one aspect, G x and G y may be calculated based on N, S, W, E, NW, SW, NE, and SE. G y =(2N + NW + NE)-(2S + SW + SE) G x =(2W + NW + SW)-(2E + NE + SE) Equation 18

[0164] In one aspect, at least one gradient filter includes a horizontal gradient filter. The gradient value includes a horizontal gradient value (e.g., GX) that is the sum of the horizontal gradients G x for each of the first input samples in the horizontal gradient filter (e.g., Equation 10). Each horizontal gradient of each of the first input samples is (i) the difference between each of the first input samples and the samples in the vicinity to the left of each of the first input samples (e.g., Equation 16), and (ii) the difference between the samples in the vicinity to the left of the first input sample and the samples in the vicinity to the right of each of the first input samples (e.g., Equation 17), (iii) the difference between a first value (2W + NW + SW) and a second value (2E + NE + SE) as shown in Equation 18 The value may be determined based on one of the following: The first value is a weighted sum based on the top-left neighbor sample, the left neighbor sample, and the bottom-left neighbor sample of the first input sample. The second value is a weighted sum based on the top-right neighbor sample, the right neighbor sample, and the bottom-right neighbor sample of the first input sample.

[0165] The difference used to calculate the horizontal gradient of each first input sample may be determined based on the position of each first input sample.

[0166] In one example, at least one gradient filter includes a vertical gradient filter. The gradient value includes a vertical gradient value which is the sum of the vertical gradients of each second input sample in the vertical gradient filter (e.g., Equation 11). Each vertical gradient of each second input sample may be determined based on one of the following: (i) the difference between each second input sample and the upper neighbor sample of each second input sample; (ii) the difference between the upper neighbor sample of each second input sample and the lower neighbor sample of each second input sample; and (iii) the difference between a first value and a second value as shown in Equation 18. The first value is a weighted sum based on the upper-left, upper-right, and upper-right neighbor samples of the second input sample. The second value is a weighted sum based on the lower-left, lower-right, and lower-right neighbor samples of each second input sample.

[0167] The difference used to calculate the vertical gradient for each second input sample is determined based on the position of each second input sample.

[0168] In one embodiment, G x and G yThe gradient {G} may be calculated according to the position of the input sample C. Figure 12 shows an example according to one aspect of this disclosure in which a gradient calculation method (for example, one of the methods described using Equations 16 to 18) is selected based on the position of the input sample. The gradient {G} of the input samples (for example, input samples (1201) to (1209)) is calculated using different gradient calculation methods depending on the position of the input sample. x ,G y You may calculate}.

[0169] Input samples (1201) to (1205) are in the Type 1 position. For each of the input samples (1201) to (1205) in the Type 1 position, by using the samples C, W, and N of the respective input samples, for example, using Equation 16, the G of the input sample can be calculated. x and G y You can also calculate this. For example, for input sample (1202), samples C, W, and N can be said to be samples (1202), (1203), and (1206), respectively. For example, for input sample (1204), samples C, W, and N can be said to be samples (1204), (1206), and (1205), respectively.

[0170] Input sample (1206) is located in the type 2 position. Using samples N, S, W, and E, for example, equation 17 can be used to determine the G of input sample (1206) located in the type 2 position. x and G y You may also calculate this.

[0171] Input samples (1207) to (1209) are in the Type 3 position. For each of the input samples (1207) to (1209) in the Type 3 position, the corresponding samples N, S, W, E, NW, SW, NE, and SE of each input sample can be used, for example, with Equation 18 to determine the G of the input sample. x and G y You may also calculate this.

[0172] The position of the central luma sample may be used to filter the IBC prediction samples. In one example, the central luma sample is within filters such as filter (801), filter (802), and filter (803). For example, the central luma sample is within linear filters such as filter (801), filter (802), and filter (803). The position of the central luma sample may move as filters are applied to each input sample. Referring to Figure 8, the central luma sample may be at position C.

[0173] In one embodiment, the predicted values ​​include, but are not limited to, pred0(x,y) or pred1(x,y). The predicted values ​​may be further improved by adding coordinate information of the central luma sample using Equation 19.

number

[0174] predi(x,y) may include (but is not limited to) pred0(x,y), pred1(x,y), etc., as well as uncorrected predictions (for example, predictions without location information). LX and LY can be said to be the location information of the central luma sample in the horizontal and vertical directions, respectively. LX and LY may be obtained (for example, defined) using Equation 20. LX = c4X LY=c5Y Equation 20

[0175] c4 and c5 are position coefficients, and the X and Y parameters are the spatial coordinates of the central luma sample.

[0176] As explained in Equation 20, the position value (for example, LX+LY) may be determined using the position of the central sample at the center of the filter (for example, a linear filter).

[0177] Based on Equation 19, if predi(x,y)=pred0(x,y),

number

number

number

[0178] Based on Equation 19, if predi(x,y) = pred1(x,y),

number

number

[0179] Therefore, the current sample's predicted value

number

[0180] IBC prediction samples may be filtered using nonlinear terms derived from reconstructed neighboring samples.

[0181] In one embodiment, the predicted value predi(x,y) can be further improved by adding a nonlinear term (also called a nonlinear value) NP using Equation 21.

number

[0182] predi(x,y) is the uncorrected prediction (e.g., a prediction without location information) which includes (but is not limited to) pred0(x,y), pred1(x,y), etc. In one example, the improved prediction is calculated using Equation 21 based on the predicted value predi(x,y) (e.g., pred0(x,y), pred1(x,y), etc.) and a nonlinear term.

number

[0183] In one example, the nonlinear term is generated from the uncorrected predicted values ​​of the currently predicted samples using Equation 22. NP=(C×C+midVal)>>bitDepth Formula 22

[0184] The value of C in Equation 22 can be said to be the uncorrected predicted value of the current sample, which is the IBC predicted value of the current sample obtained from either the IBC mode or the IntraTMP mode. Therefore, the value of C in Equation 22 can be said to be the value of the reference sample (IBC predicted sample) without using the FIBC filter.

[0185] For example, if the content is 10 bits, the mid value (midVal) is 2 10 This is / 2, which is 512. Therefore, if the content is 10 bits, the nonlinear term NP may be calculated using equation 23. NP=(C×C+512)>>10 Formula 23

[0186] Subsequently, the final prediction pred2(x,y) can be generated by applying the truncation operation using Equation 24.

number

[0187] In another example, the nonlinear term may be generated from the current sample and a reconstructed value of the most recent currently predicted sample, i.e., a previously predicted value. Figure 13 shows examples of the locations of samples A, L, and AL according to one aspect of this disclosure. Samples A, L, and AL are currently neighboring samples to sample C. M1 may be the mean of the values ​​of samples A, L, AL, and C, such as M1 = mean(A, L, AL, C). M1 may also be the median of the values ​​of samples A, L, AL, and C, such as M1 = median(A, L, AL, C). The nonlinear term NP is determined using Equation 25. NP=(M1×M1+midVal)>>bitDepth Formula 25

[0188] In another example, the nonlinear term may be generated from the reconstructed value of the most recent currently predicted sample, i.e., from a previously predicted value. The nonlinear term NP is defined using Equation 26. NP=(M2×M2+midVal)>>bitDepth Formula 26

[0189] M2 can be the mean of the sample values ​​A, L, and AL, as in M2 = mean(A, L, AL). M2 can also be the median of the sample values ​​A, L, and AL, as in M2 = median(A, L, AL).

[0190] The nonlinear value associated with the current sample, i.e., the nonlinear term NP, may be determined from at least one of the current sample (e.g., C in Figure 13) and neighboring samples (e.g., A, L, AL in Figure 13) using the nonlinear relationship (e.g., Equations 23, 25, and 26) between the nonlinear value NP and at least one of the values ​​of the current sample and neighboring samples (e.g., C, A, L, and AL). Predicted value of the current sample

number

[0191] In one embodiment, the coefficients (e.g., the filter coefficients of the FIBC filter) are derived from the template (e.g., current template) of the coding block (e.g., current block) and the corresponding reference template. Returning to Figure 7B, in one embodiment, the filter coefficients of the FIBC filter may be determined based on the current template (704) of the current block (701) and the reference template (705) of the reference block (703). In one example, the filter coefficients of the FIBC filter may be derived by applying up to 4 rows and 4 columns of samples above and to the left of the current block (701).

[0192] In one embodiment, the filter coefficients of the FIBC filter are the linear filter coefficients {c} of the linear filter. 0,i} and (i) the gradient filter coefficients of the horizontal gradient filter {c 1,j} and / or gradient filter coefficients of the vertical gradient filter {c 2,k}, (ii) position coefficients c4 and c5 of the position filter and (iii) one or more nonlinear coefficients c6 of the nonlinear term. For example, if bias B is included, the filter coefficients are {c 3,l} includes. For example, a linear filter is {c 3,l This includes}. If the FIBC filter contains other information, additional coefficients may be included in the filter coefficients of the FIBC filter.

[0193] The filter coefficients of the FIBC filter may be determined (e.g., derived) based on the current template (704) of the current block (701) and the reference template (705) of the reference block (703). In the example shown in Figure 7B, the current template (704) includes the top four rows of the current block (701) and the left four columns of the current block (701). The samples in the current template (704) have already been reconstructed. Predicted samples for each sample in the current template (704) may be generated by applying an FIBC filter (e.g., one or more of equations 8-15, 19, and 21) to the corresponding samples in the reference template (705). The filter coefficients are determined, for example, by minimizing the error function (e.g., MSE) between the predicted samples in the current template and the reconstructed samples, as used in CCCM.

[0194] In one embodiment, the coefficients are determined (e.g., derived) using the LDL method (also called LDL decomposition) or variance used in CCCM. In LDL decomposition, matrix A is given by A = LDL T It can also be decomposed as follows: L is a lower unit triangular (mono-triangular) matrix, D is a diagonal matrix, and L T is the transpose of L. LDL decomposition can be said to be a closely related variation of Cholesky decomposition. In some cases, LDL decomposition can avoid the need for square roots, and therefore can be more effective than Cholesky decomposition.

[0195] Figure 14 shows a flowchart schematically illustrating a process (1400) according to one embodiment of the present disclosure. Process (1400) can be used in devices such as video decoders. In various embodiments, process (1400) is executed by processing circuits, such as a processing circuit that performs the functions of a video decoder (110) and a processing circuit that performs the functions of a video decoder (210). In some embodiments, process (1400) is carried out by software instructions, and therefore, when a processing circuit executes a software instruction, the processing circuit executes process (1400). The process starts at (S1401) and proceeds to (S1410).

[0196] In (S1410), coded information is received indicating that the current block in the current picture is predicted using filtered intrablock copy (FIBC) mode.

[0197] In (S1420), the linear predicted value of the current sample in the current block is determined by applying a linear filter to the predicted sample predicted using either IBC mode or Intra-Template Matching (IntraTMP) mode.

[0198] For example, a linear filter includes a bias term.

[0199] In one example, the mean values ​​of the current block are added using a linear filter, and the mean values ​​of the current block are removed from each of the samples predicted using either the IBC mode or the IntraTMP mode.

[0200] For example, a linear filter may consist of five samples: (i) a central sample of the linear filter with an offset of (0,0), a northern sample N with an offset of (0,-1), a southern sample S with an offset of (0,1), an eastern sample E with an offset of (1,0), and a western sample W with an offset of (-1,0), where the offsets of the five samples in the linear filter are relative to the central sample; or (ii) a linear filter with an offset of (0,0) The linear filter comprises nine samples: a central sample, two northern samples with offsets of (0,-1) and (0,-2) respectively, two southern samples with offsets of (0,1) and (0,2) respectively, two eastern samples with offsets of (1,0) and (2,0) respectively, and two western samples with offsets of (-1,0) and (-2,0) respectively, wherein the offsets of the nine samples in the linear filter form a cross shape including the nine samples, with the central sample as the reference.

[0201] If a linear filter has 5 samples, the current sample is located at one of the 5 positions, which are the positions of each of the 5 samples. If a linear filter has 9 samples, the current sample is located at one of the 9 positions, which are the positions of each of the 9 samples.

[0202] In (S1430), at least one gradient filter is used to determine the gradient value associated with the current sample in the current block.

[0203] In one example, if at least one gradient filter consists of a horizontal gradient filter, the gradient value is the horizontal gradient value, and the number of first input samples and the positions of the first input samples in the horizontal gradient filter are set independently of the linear filter. If at least one gradient filter consists of a vertical gradient filter, the gradient value is the vertical gradient value, and the number of second input samples and the positions of the second input samples in the vertical gradient filter are set independently of the linear filter. If at least one gradient filter includes both a horizontal and a vertical gradient filter, the gradient value is the sum of the horizontal and vertical gradient values, and the number of first input samples and the positions of the first input samples in the horizontal gradient filter, and the number of second input samples and the positions of the second input samples in the vertical gradient filter are set independently of each other and independently of the linear filter.

[0204] In one example, at least one gradient filter includes a horizontal gradient filter. The gradient value includes a horizontal gradient value which is the sum of the horizontal gradients of each first input sample in the horizontal gradient filter. Each horizontal gradient of each first input sample is determined based on one of the following: (i) the difference between each first input sample and the left neighbor sample of each first input sample; (ii) the difference between the left neighbor sample of the first input sample and the right neighbor sample of each first input sample; and (iii) the difference between a first value and a second value, where the first value is the sum of the upper-left neighbor sample, the left neighbor sample and the lower-left neighbor sample of the first input sample; and the second value is the sum of the upper-right neighbor sample, the right neighbor sample and the lower-right neighbor sample of the first input sample.

[0205] In one example, the difference used to calculate the horizontal gradient of each first input sample is determined based on the position of each first input sample.

[0206] In one example, at least one gradient filter includes a vertical gradient filter. The gradient value includes a vertical gradient value which is the sum of the vertical gradients of each second input sample in the vertical gradient filter. Each vertical gradient of each second input sample is determined based on one of the following: (i) the difference between each second input sample and the upper neighbor sample of each second input sample; (ii) the difference between the upper neighbor sample of each second input sample and the lower neighbor sample of each second input sample; and (iii) the difference between a first value and a second value, where the first value is the sum of the upper-left neighbor sample, the upper neighbor sample and the upper-right neighbor sample of the second input sample; and the second value is the sum of the lower-left neighbor sample, the lower neighbor sample and the lower-right neighbor sample of each second input sample.

[0207] In one example, the difference used to calculate the vertical gradient of each second input sample is determined based on the position of each second input sample.

[0208] (In S1440), the predicted value of the current sample is determined based on the sum of the linear prediction value and at least one correction value including the gradient value, and the FIBC filter in the FIBC mode includes a linear filter and at least one gradient filter.

[0209] In one example, the predicted value of the current sample is clipped.

[0210] (In S1450), the current sample is reconstructed from the predicted value of the current sample.

[0211] Thereafter, the process proceeds to (S1499) and ends.

[0212] The process (1400) can be appropriately adapted. One or more steps of the process (1400) can be modified and / or omitted. One or more additional different steps can be added. Any appropriate order of implementation can be used.

[0213] In one example, the position value is determined using the position of the central sample that is at the center of the linear filter, and the predicted value of the current sample is determined based on the sum of the linear prediction value and at least one correction value including the gradient value and the position value. The FIBC filter in the FIBC mode includes a linear filter, at least one gradient filter, and a position filter (for example, a coefficient of position).

[0214] In one example, a non - linear value related to the current sample is determined from at least one of the current sample and the samples in the vicinity of the current sample using the non - linear relationship between the non - linear value and the value of at least one of the current sample and the samples in the vicinity, and the predicted value of the current sample is determined based on the sum of the linear prediction value and at least one correction value including the gradient value and the non - linear value. The FIBC filter in the FIBC mode includes a linear filter, at least one gradient filter, and a coefficient of the non - linear value.

[0215] In one example, the coefficients of the FIBC filter in FIBC mode are determined from the current template of the current block and the reference template of the reference block indicated by the block vector of the current block.

[0216] In one example, the coefficients of the FIBC filter in FIBC mode are determined using LDL decomposition.

[0217] In one example, the shape of a linear filter is predetermined, and the shape of at least one gradient filter is predetermined.

[0218] In one example, one sample in a linear filter is spatially separated from all the other samples in the linear filter.

[0219] Figure 15 shows a flowchart schematically illustrating a process (1500) according to one aspect of the present disclosure. Process (1500) can be used in a video encoder. In various aspects, process (1500) is executed by processing circuits, such as a processing circuit that performs the functions of a video encoder (103) and a processing circuit that performs the functions of a video encoder (303). In some aspects, process (1500) is carried out by software instructions, and therefore, when a processing circuit executes a software instruction, the processing circuit executes process (1500). The process starts at (S1501) and proceeds to (S1510).

[0220] In (S1510), the linear predicted value of the current sample in the current block is determined by applying a linear filter to the sample predicted using either intrablock copy (IBC) mode or intratemplate matching. The current block is predicted using filtered IBC (FIBC) mode, which includes a linear filter.

[0221] In one example, the linear filter may include a bias term, or the linear filter may be configured to add the mean of the current block and subtract the mean of the current block from each of the samples predicted using either the IBC mode or the IntraTMP mode.

[0222] In (S1520), at least one gradient filter is used to determine the gradient value associated with the current sample in the current block. The FIBC filter includes at least one gradient filter.

[0223] In (S1530), a nonlinear value related to the current sample is determined from the current sample and at least one of the neighboring samples, using a nonlinear relationship between the current sample and at least one of the neighboring samples.

[0224] In (S1540), the predicted value for the current sample is determined based on the sum of a linear predicted value and at least one corrected value including a gradient value and a nonlinear value. The FIBC filter in FIBC mode includes a linear filter, at least one gradient filter, and coefficients for nonlinear values.

[0225] In (S1550), the current sample is encoded from the predicted value of the current sample.

[0226] The process then proceeds to (S1599) and terminates.

[0227] Process (1500) can be appropriately adapted. One or more steps of process (1500) can be modified and / or omitted. One or more further steps can be added. Any appropriate order of implementation can be used.

[0228] In one example, the position value is determined using the position of the central sample located in the center of a linear filter, and the predicted value of the current sample is determined based on the sum of the linear predicted value and at least one modified value including the gradient value, nonlinear value, and position value. The FIBC filter in FIBC mode includes a linear filter, at least one gradient filter, coefficients for nonlinear values, and a position filter (e.g., a coefficient for position).

[0229] In one embodiment, a method for processing video media data is disclosed. The method includes the step of processing a bitstream of video media data in accordance with format rules. The bitstream includes syntax elements that indicate the current block in the current picture is expected to be used with filtered intrablock copy (FIBC) mode. The format rules specify that the linear predicted value of the current sample in the current block is determined by applying a linear filter to the predicted sample using either the intrablock copy IBC mode or the intratemplate matching (IntraTMP) mode; that the gradient value associated with the current sample in the current block is determined using at least one gradient filter; that the nonlinear value associated with the current sample is determined from at least one of the current sample and its neighboring samples using a nonlinear value and a nonlinear relationship between the current sample and at least one of the neighboring samples; that the position value is based on the position of the central sample at the center of the linear filter; that the predicted value of the current sample is based on the sum of the linear predicted value and at least one modified value including the gradient value, nonlinear value and position value; that the FIBC filter in FIBC mode includes a linear filter, at least one gradient filter, coefficients for the nonlinear value and a coefficient for the position; and that the current sample is processed from the predicted value of the current sample.

[0230] For example, a linear filter includes a bias term.

[0231] In one example, the linear filter is configured to add the average value of the current block and subtract the average value of the current block from each of the samples predicted using one of the IBC mode and the IntraTMP mode.

[0232] Aspects and / or examples of the present disclosure may be used separately or combined in any order. For example, some aspects and / or examples executed by a decoder may be executed by an encoder, or vice versa. Each of the method (i.e., aspect), encoder, and decoder may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-transitory computer-readable medium.

[0233] Using computer-readable instructions, the above-described techniques may be implemented as computer software physically stored on one or more computer-readable media. For example, FIG. 16 shows a computer system (1600) suitable for implementing certain aspects of the disclosed subject matter.

[0234] The computer software can be coded using any suitable machine language, i.e., computer language, which may be processed by an assembly, compilation, linking, or similar mechanism for creating code with instructions, and the instructions can be executed directly or through interpretation, microcode execution, etc., by one or more central processing units (CPUs), graphics processing units (GPUs), etc., of a computer.

[0235] The instructions can be executed on various types of computers or computer components, including, for example, personal computers, tablet computers, servers, smartphones, gaming machines, Internet of Things devices, etc.

[0236] The components shown in Figure 16 for the computer system (1600) are examples and are not intended to imply any limitation on the scope of use or functionality of the computer software implementing the aspects of this disclosure. Nor should it be construed that the configuration of the components is dependent on or requires any one or any combination thereof of the components shown in the examples of the aspects of the computer system (1600).

[0237] The computer system (1600) may include certain human interface input devices. Such human interface input devices may respond to input from one or more human users via, for example, tactile input (keystrokes, swipes, data glove movements, etc.), voice input (vocalization, clapping, etc.), visual input (gestures, etc.), or olfactory input (not shown). Human interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as sound (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images taken from a still image camera), or video (e.g., two-dimensional video, three-dimensional video including stereoscopic video, etc.).

[0238] The input human interface device may include one or more of the following (only one of each) (1601, 1602, 1603, 1610, 1601, 1601, 1602, 1602, 1603, 1604, 1605, 1606, 1607, 1608).

[0239] The computer system (1600) may also include some kind of human interface output device. Such a human interface output device may stimulate the senses of one or more human users, for example, through haptic output, sound, light and smell / taste. Such a human interface output device may include tactile output devices (e.g., touch screens (1610), tactile feedback via data gloves (not shown) and joysticks (1605), although tactile feedback devices that do not serve as input devices may also exist), audio output devices (speakers (1609), headphones (not shown), etc.), visual output devices (screens (1610), including CRT screens, LCD screens, plasma screens, and OLED screens, each with or without touch screen input functionality, each with or without tactile feedback functionality—some of these screens may have the functionality to output two-dimensional visual output or three-dimensional or more output by means such as stereoscopic image output devices, virtual reality glasses (not shown), holographic displays and smoke tanks (not shown)), and printers (not shown).

[0240] The computer system (1600) may also include human-handled storage devices and related media, such as optical media including CD / DVD ROM / RW (1620) with CD / DVD or similar media (1621), thumb drives (1622), removable hard drives and solid-state drives (1623), legacy magnetic media such as tapes and floppy disks (not shown), and dedicated ROM / ASIC / PLD-based devices such as security dongles (not shown).

[0241] Those skilled in the art will also understand that the term “computer-readable medium” as used in relation to the subject matter protected by this disclosure does not include a transmission medium, carrier wave, or other transient signal.

[0242] A computer system (1600) may also include an interface (1654) to one or more communication networks (1655). The networks may be, for example, wireless, wired, or optical. Networks can further be local, wide-area, metropolitan, automotive, and industrial, real-time, latency-tolerant, etc. Examples of networks include local area networks such as Ethernet, cellular networks including wireless LAN, GSM, 3G, 4G, 5G, LTE, etc., wired or wireless wide-area digital networks for television including cable television, satellite television, and terrestrial television, and automotive and industrial networks including CANBus. Some networks generally require an external network interface adapter attached to some general-purpose data port or peripheral bus (1649) (for example, a USB port on a computer system (1600)), while others are generally integrated into the core of the computer system (1600) by being attached to a system bus, as described below (for example, an Ethernet interface integrated into a PC computer system or a cellular network interface integrated into a smartphone computer system). Using any of these networks, the computer system (1600) can communicate with others. Such communication can be one-way, receive-only (e.g., television broadcasting), one-way transmit-only (e.g., from a CANbus to a specific CANbus device), or bidirectional, for example, bidirectional communication to other computer systems using local or wide-area digital networks. Several protocols and protocol stacks can be used for each of the above-mentioned networks and network interfaces.

[0243] The above-mentioned human interface device, human-operable memory device, and network interface can be mounted in the central part (1640) of the computer system (1600).

[0244] The central unit (1640) may include one or more dedicated programmable processing units in the form of a central processing unit (CPU) (1641), a graphics processing unit (GPU) (1642), a field-programmable gate area (FPGA) (1643), a hardware accelerator (1644) used for specific tasks, a graphics adapter (1650), and the like. These devices may be connected via a system bus (1648) along with read-only memory (ROM) (1645), random access memory (1646), and built-in, user-inaccessible mass storage such as a hard disk or SSD (1647). In some computer systems, the system bus (1648) may take the form of one or more physical plugs that allow expansion by adding CPUs, GPUs, etc., and can be manually operated. Peripheral devices can be connected directly to the central system bus (1648) or via a peripheral bus (1649). For example, a screen (1610) can be connected to the graphics adapter (1650). Peripheral bus architectures include PCI, USB, and others.

[0245] The CPU (1641), GPU (1642), FPGA (1643), and accelerator (1644) can execute several instructions that, in combination, form the computer code described above. This computer code can be stored in ROM (1645) or RAM (1646). While changing data can be stored in RAM (1646), unchanging data can be stored, for example, in built-in large-capacity storage (1647). By using a cache memory that can work closely with one or more CPUs (1641), GPUs (1642), large-capacity storage (1647), ROMs (1645), RAM (1646), etc., it is possible to enable high-speed storage and retrieval from any of the memory devices.

[0246] A computer-readable medium may contain computer code for performing various computer operations. The medium and computer code may be specifically designed and configured for the purposes of this disclosure, or they may be of a type that is well known and available to those skilled in the computer software technology.

[0247] As an example, and not an limitation, a computer system having architecture (1600), particularly a central unit (1640), can function as a result of a processor (including CPU, GPU, FPGA, accelerator, etc.) executing software implemented in one or more tangible computer-readable media. Such computer-readable media can be media related to user-handled mass storage as described above, as well as specific storage of the central unit (1640) with non-transient properties, such as central unit-integrated mass storage (1647) or ROM (1645). Software implementing various aspects of this disclosure can be stored in such devices and executed by the central unit (1640). The computer-readable media may include one or more memory devices or chips as required by individual needs. The software can cause the central unit (1640), particularly the processor (including CPU, GPU, FPGA, etc.) within the central unit (1640), to execute a specific process or a specific part of a specific process as described in this application (including determining data structures to be stored in RAM (1646) and modifying such data structures according to a process defined by the software). In addition to or instead thereof, the computer system may function as a result of logic through connections or by being implemented in other ways in the circuit (e.g., accelerators (1644)), which may act in place of or in cooperation with software to perform a particular process or a particular part of a particular process as described herein. Where appropriate, “software” may include logic and vice versa. Where appropriate, “computer-readable medium” may include circuits that store the software to be executed (such as integrated circuits (ICs)), circuits that implement the logic to be executed, or both. The disclosure includes any appropriate combination of hardware and software.

[0248] Whereever “at least one of” or “one of” is used in this disclosure, it is intended to include any one of the elements described, or any combination of the elements described. For example, where it says “at least one of A, B, or C,” where it says “at least one of A, B, and C,” where it says “at least one of A, B, and / or C,” and where it says “at least one of A through C,” it is intended to include A only, B only, C only, or any combination thereof. Where it says “one of A or B,” and where it says “one of A and B,” it is intended to include A or B, or (A and B). No combination of the elements described is excluded (where applicable, such as when “one of” is used and the elements are not mutually exclusive).

[0249] While this disclosure has described several examples of embodiments, variations, substitutions, and various alternative equivalents exist and are included within the scope of this disclosure. Therefore, those skilled in the art will find that they can implement the principles of this disclosure and thus recall numerous systems and methods that fall within its spirit and scope, even if they are not explicitly shown or described in this application. [Explanation of Symbols]

[0250] 100 Video processing system, communication system, 101 Video source, 102 Stream, 103 Video encoder, 104 Video data, 105 Streaming server, 106 Client subsystem, 107 Replication, 110 Video decoder, 111 Output stream, 112 Display, 113 Imaging subsystem, 120 Electronic device, 130 Electronic device, 201 Channel, 210 Video decoder, 212 Drawing device, 215 Buffer memory, 220 Parser, 221 Symbol, 230 Electronic device, 231 Receiver, 251 Inverse transform unit, 252 Intra prediction unit, Picture prediction unit, 253 Motion compensation prediction unit, 255 Aggregator, 256 Loop filter unit, 257 Reference picture memory, 258 Current picture buffer, 301 Video source, 303 Video encoder, 320 Electronic device, 330 Source coder, 332 coding engine, 333 video decoder, 334 reference picture memory, 335 predictor, 340 transmitter, 343 video sequence, 345 entropicorder, 350 controller, 360 communication channel, 700 picture, 701 current block, 703 reference block, 704 current template, 705 reference template, 710 current sample, 711 IBC predictor sample, reference sample, 721 matching block, 730 current template, 731 current block, 801 cross-shaped filter, 802 cross-shaped filter, filter shape, 803 filter, filter shape, 1201 input sample, 1202 input sample, 1204 input sample, 1206 input sample, 1207 input sample, 1400 process, 1500 process, 1600 computer system, 1601 keyboard, 1602 mouse, 1603 Trackpad, 1605 Joystick, 1606 Microphone, 1607 Scanner, 1608 Camera, 1609 Speaker, 1610 Touchscreen, 1620 CD / DVD ROM / RW, 1621 Media, 1622 Thumb Drive, 1623 Solid State Drive, 1640 Core, 1641 CPU, 1642 GPU, 1643 FPGA, 1644 Hardware Accelerator, 1645 ROM, 1646 RAM, 1647Built-in high-capacity storage, central built-in high-capacity storage, 1648 system bus, 1649 peripheral bus, 1650 graphics adapter, 1654 interface, 1655 communication network

Claims

1. A method for processing video media data, wherein the method is Steps to process the bitstream of the video media data in accordance with formatting rules. Equipped with, The bitstream includes syntax elements indicating that the current block in the current picture is predicted using filtered intrablock copy (FIBC) mode, The aforementioned formatting rules are: The linear predicted value of the current sample in the current block is determined by applying a linear filter to the predicted sample using either the intrablock copy (IBC) mode or the intratemplate matching (IntraTMP) mode. The gradient value associated with the current sample in the current block is determined using at least one gradient filter, The nonlinear value associated with the current sample is determined from at least one of the current sample and a neighboring sample of the current sample using a nonlinear relationship between the nonlinear value and the value of at least one of the current sample and the neighboring sample. The position value is based on the position of the central sample located in the center of the linear filter, The FIBC filter in FIBC mode includes the linear filter, the at least one gradient filter, the coefficient of the nonlinear value, and the coefficient of the position, based on the sum of the predicted value of the current sample and the linear predicted value and at least one corrected value including the gradient value, the nonlinear value, and the position value. The current sample is processed from the predicted value of the current sample. A method for specifying this.

2. The method according to claim 1, wherein the linear filter includes a bias term.

3. The method according to claim 1 or 2, wherein the linear filter is configured to add the mean value of the current block, and the mean value of the current block is removed from each of the samples predicted using either the IBC mode or the IntraTMP mode.

4. A step of determining the linear predicted value of the current sample in the current block by applying a linear filter to the predicted sample predicted using either intrablock copy (IBC) mode or intratemplate matching (IntraTMP) mode, wherein the current block is predicted using filtered IBC (FIBC) mode. A step of determining the gradient value related to the current sample in the current block using at least one gradient filter, A step of determining a nonlinear value related to the current sample from at least one of the current sample and a neighboring sample of the current sample, using a nonlinear relationship between the nonlinear value and the value of at least one of the current sample and the neighboring sample. A step of determining the predicted value of the current sample based on the sum of the linear predicted value and at least one corrected value including the gradient value and the nonlinear value, wherein the FIBC filter in FIBC mode includes the linear filter, the at least one gradient filter and the coefficient of the nonlinear value, A step of encoding the current sample from the predicted value of the current sample. A video encoding method comprising the following features.

5. The steps include determining the position value using the position of the central sample at the center of the linear filter, A step of determining the predicted value of the current sample based on the sum of the linear predicted value and the at least one corrected value including the gradient value, the nonlinear value and the position value, wherein the FIBC filter in FIBC mode includes the linear filter, the at least one gradient filter, the coefficient of the nonlinear value and the coefficient of the position. The method according to claim 4, further comprising:

6. The linear filter includes a bias term, or The linear filter is configured to add the average value of the current block and remove the average value of the current block from each of the samples predicted using either the IBC mode or the IntraTMP mode. The method according to claim 4 or 5.

7. Using filtered intrablock copy (FIBC) mode, it receives coded information indicating what the current block in the current picture is expected to be, The linear predicted value of the current sample in the current block is determined by applying a linear filter to the predicted sample predicted using either IBC mode or Intra-Template Matching (IntraTMP) mode, Determining the gradient value related to the current sample in the current block using at least one gradient filter, The predicted value of the current sample is determined based on the sum of the linear predicted value and at least one corrected value including the gradient value, wherein the FIBC filter in FIBC mode includes the linear filter and the at least one gradient filter. Reconstructing the current sample from the predicted value of the current sample. A processing circuit configured to perform the following A video decoding device equipped with the following features.

8. The aforementioned processing circuit is The position value is determined using the position of the central sample at the center of the linear filter, The predicted value of the current sample is determined based on the sum of the linear predicted value and the at least one corrected value including the gradient value and the position value, wherein the FIBC filter in FIBC mode includes the linear filter, the at least one gradient filter and the position coefficient. The apparatus according to claim 7, configured to perform the following.

9. The aforementioned processing circuit is Determining a nonlinear value related to the current sample from at least one of the current sample and a neighboring sample of the current sample, using the nonlinear relationship between the nonlinear value and the value of at least one of the current sample and the neighboring sample, The predicted value of the current sample is determined based on the sum of the linear predicted value and the at least one correction value including the gradient value and the nonlinear value, wherein the FIBC filter in FIBC mode includes the linear filter, the at least one gradient filter and the coefficient of the nonlinear value. The apparatus according to claim 7, configured to perform the following.

10. The apparatus according to any one of claims 7 to 9, wherein the linear filter includes a bias term.

11. The apparatus according to any one of claims 7 to 10, wherein the linear filter adds the average values ​​of the current block and removes the average values ​​of the current block from each of the samples predicted using either the IBC mode or the IntraTMP mode.

12. The apparatus according to any one of claims 7 to 11, wherein the processing circuit is configured to cut out the predicted value of the current sample.

13. The apparatus according to any one of claims 7 to 12, wherein the processing circuit is configured to determine the coefficients of the FIBC filter in FIBC mode from the current template of the current block and the reference template of the reference block indicated by the block vector of the current block.

14. The aforementioned linear filter is (i) Five samples including the central sample of the linear filter with an offset of (0,0), the northern sample N with an offset of (0,-1), the southern sample S with an offset of (0,1), the eastern sample E with an offset of (1,0), and the western sample W with an offset of (-1,0), wherein the offsets of the five samples in the linear filter are based on the central sample, or (ii) Nine samples, including the central sample of the linear filter with an offset of (0,0), two northern samples with offsets of (0,-1) and (0,-2), two southern samples with offsets of (0,1) and (0,2), two eastern samples with offsets of (1,0) and (2,0), and two western samples with offsets of (-1,0) and (-2,0), wherein the offsets of the nine samples in the linear filter are relative to the central sample. The apparatus according to any one of claims 7 to 13, having a cross shape including

15. If the linear filter has the five samples, the current sample is located at one of the five positions, which are the positions of each of the five samples. If the linear filter has the nine samples, the current sample is located at one of the nine positions, which are the positions of each of the nine samples. The apparatus according to claim 14.

Images