Encoding method, decoding method, encoder, decoder, and storage medium

Abstract

Provided in embodiments of the present application are an encoding method, a decoding method, an encoder, a decoder, and a storage medium. The decoding method comprises: performing intra template matching prediction on a current block, so as to determine a predicted block and motion information of the current block; on the basis of the predicted block of the current block and a residual block of the current block, determining a reconstructed block of the current block; and storing the motion information of the current block into a motion information buffer corresponding to the current block, wherein the motion information in the motion information buffer is used for constructing a candidate merge set for a target block, and the target block is predicted on the basis of intra block copy.

Description

Coding and decoding method, codec and storage medium Technical Field

[0001] The present application relates to the technical field of video coding and decoding, and in particular to a coding and decoding method, a codec, and a storage medium. Background Art

[0002] Intra block copy (IBC)-based prediction technology is currently widely used in various codec standards. How to improve the prediction performance of IBC blocks is a problem that needs to be solved.

[0003] Summary of the Invention

[0004] The embodiments of the present application provide a coding method, a codec, and a storage medium to improve the prediction performance of IBC blocks. The following describes various aspects of the present application.

[0005] In a first aspect, a decoding method is provided, which is applied to a decoder, including: performing intra-frame template matching prediction on a current block to determine a prediction block and motion information of the current block; determining a reconstructed block of the current block based on the prediction block of the current block and the residual block of the current block; storing the motion information of the current block in a motion information cache corresponding to the current block, the motion information in the motion information cache being used to construct a merge candidate set for a target block, and the target block being predicted based on intra-frame block copying.

[0006] In a second aspect, a coding method is provided, which is applied to an encoder, including: performing intra-frame template matching prediction on a current block to determine the prediction block and motion information of the current block; determining the residual block of the current block based on the prediction block of the current block; storing the motion information of the current block in a motion information cache corresponding to the current block, and using the motion information in the motion information cache to construct a merge candidate set for a target block, and the target block is predicted based on intra-frame block copying.

[0007] According to a third aspect, a decoder is provided, comprising: a first decoding unit, configured to perform intra-frame template matching prediction on a current block, and determine a prediction block and motion information of the current block; a second decoding unit, configured to determine a reconstructed block of the current block based on the prediction block of the current block and the residual block of the current block; and a storage unit, configured to store the motion information of the current block in a motion information cache corresponding to the current block, wherein the motion information in the motion information cache is used to construct a merge candidate set for a target block, and the target block is predicted based on intra-frame block copying.

[0008] According to a fourth aspect, a decoder is provided, comprising: a memory for storing a computer program; and a processor for executing the method according to the first aspect when running the computer program.

[0009] In a fifth aspect, an encoder is provided, comprising: a first encoding unit, configured to perform intra-frame template matching prediction on a current block, and determine the prediction block and motion information of the current block; a second encoding unit, configured to determine the residual block of the current block based on the prediction block of the current block; a storage unit, configured to store the motion information of the current block in a motion information cache corresponding to the current block, the motion information in the motion information cache being used for constructing a merge candidate set for a target block, and the target block being predicted based on intra-frame block copying.

[0010] In a sixth aspect, an encoder is provided, comprising: a memory for storing a computer program; and a processor for executing the method described in the second aspect when running the computer program.

[0011] In a seventh aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, the method as described in the first aspect or the second aspect is implemented.

[0012] In an eighth aspect, a computer program product is provided, comprising a computer program, which implements the method described in the first aspect or the second aspect when executed.

[0013] In a ninth aspect, a recording method is provided for recording a code stream on a recording medium, the recording method comprising a recording step for recording a code stream generated by the encoding method described in the second aspect on a recording medium.

[0014] In a tenth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a code stream generated by the encoding method described in the second aspect.

[0015] The embodiment of the present application constructs a merge set (merge list) of IBC blocks based on the motion information of the current block (i.e., the image block predicted based on intra-frame template matching), thereby enriching the information source of the merge set of IBC blocks and improving the prediction performance of the IBC blocks. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] FIG1 is a schematic structural diagram of a video encoder to which an embodiment of the present application can be applied.

[0017] FIG2 is a schematic structural diagram of a video decoder to which an embodiment of the present application can be applied.

[0018] FIG3 is a diagram illustrating an implementation example of the intra template matching prediction (IntraTMP) technology.

[0019] Figure 4 shows an example diagram of BV and template error.

[0020] FIG5 is a schematic diagram illustrating an implementation of the IntraTMP adaption for camera-captured content technology.

[0021] FIG6 is a schematic diagram of an implementation of the IntraTMP multi-candidate technology.

[0022] FIG7 is a schematic diagram of a filter of the IntraTMP filtering technology.

[0023] FIG8 is a diagram illustrating an example of how filter coefficients are determined in the IntraTMP filtering technology.

[0024] FIG. 9 is a diagram illustrating an example of a division method of an image block in a combined inter and intra prediction (CIIP) technology.

[0025] FIG10 is a schematic diagram of an implementation of the local illumination compensation (LIC) technology.

[0026] FIG11 is an example diagram showing the positional relationship between adjacent and non-adjacent candidates of the current block in Inter-Merge.

[0027] FIG12 is an example diagram of templates of the current block and the reference block.

[0028] FIG13 is a diagram illustrating an example of the transformation process of the template of the reference block.

[0029] FIG14 is a flow chart of the decoding method provided in an embodiment of the present application.

[0030] FIG15 is a flow chart of the encoding method provided in an embodiment of the present application.

[0031] FIG16 is a schematic diagram of the structure of a decoder provided in one embodiment of the present application.

[0032] FIG17 is a schematic diagram of the structure of a decoder provided in another embodiment of the present application.

[0033] FIG18 is a schematic diagram of the structure of an encoder provided in one embodiment of the present application.

[0034] FIG19 is a schematic diagram of the structure of an encoder provided in another embodiment of the present application.

[0035] Figure 20 is a flow chart of the recording method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] FIG1 is a schematic block diagram of a video encoder according to an embodiment of the present application.

[0037] It should be understood that the video encoder 100 can be used to perform lossy compression or lossless compression on an image. The lossless compression can be visually lossless compression or mathematically lossless compression.

[0038] The video encoder 100 can be applied to image data in a luminance and chrominance (YCbCr, YUV) format. For example, the YUV ratio can be 4:2:0, 4:2:2, or 4:4:4, where Y represents brightness (Luma), Cb (U) represents blue chrominance, Cr (V) represents red chrominance, and U and V represent chrominance (Chroma) for describing color and saturation. For example, in terms of color format, 4:2:0 means that every 4 pixels have 4 luminance components and 2 chrominance components (YYYYCbCr), 4:2:2 means that every 4 pixels have 4 luminance components and 4 chrominance components (YYYYCbCrCbCr), and 4:4:4 represents full pixel display (YYYYCbCrCbCrCbCrCbCr).

[0039] For example, the video encoder 100 reads video data, and for each image in the video data, divides the image into a number of coding tree units (CTUs). In some examples, CTUs may be referred to as "tree blocks", "largest coding units" (LCUs) or "coding tree blocks" (CTBs). Each CTU may be associated with a pixel block of equal size within the image. Each pixel may correspond to a luminance (luminance or luma) sample and two chrominance (chroma) samples. Therefore, each CTU may be associated with a luminance sample block and two chroma sample blocks. The size of a CTU is, for example, 128×128, 64×64, 32×32, etc. A CTU may be further divided into a number of coding units (CUs) for encoding. A CU may be a rectangular block or a square block. A CU can be further divided into prediction units (PUs) and transform units (TUs), allowing for separation of coding, prediction, and transform, and greater flexibility in processing. In one example, a CTU is divided into CUs using a quadtree, and a CU is divided into TUs and PUs using a quadtree.

[0040] The video encoder and video decoder can support various PU sizes. Assuming that the size of a particular CU is 2N×2N, the video encoder and video decoder can support PU sizes of 2N×2N or N×N for intra-frame prediction, and support symmetric PUs of 2N×2N, 2N×N, N×2N, N×N, or similar sizes for inter-frame prediction. The video encoder and video decoder can also support asymmetric PUs of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter-frame prediction.

[0041] In some embodiments, as shown in FIG1 , the video encoder 100 may include a prediction unit 110, a residual unit 120, a transform / quantization unit 130, an inverse transform / quantization unit 140, a reconstruction unit 150, a loop filter unit 160, a decoded image buffer 170, and an entropy coding unit 180. It should be noted that the video encoder 100 may include more, fewer, or different functional components.

[0042] Optionally, in this application, the current block may be referred to as the current coding unit (CU) or the current prediction unit (PU), etc. The prediction block may also be referred to as a predicted image block or an image prediction block, and the reconstructed image block may also be referred to as a reconstructed block or an image reconstruction block.

[0043] In some embodiments, the prediction unit 110 includes an inter-frame prediction unit 111 and an intra-frame prediction unit 112. Because there is a strong correlation between adjacent pixels in a video image, intra-frame prediction is used in video coding and decoding technologies to eliminate spatial redundancy between adjacent pixels. Because there is a strong similarity between adjacent images in a video, inter-frame prediction is used in video coding and decoding technologies to eliminate temporal redundancy between adjacent images, thereby improving coding efficiency.

[0044] The inter-frame prediction unit 111 can be used for inter-frame prediction. Inter-frame prediction can include motion estimation and motion compensation. It can refer to image information from different images. Inter-frame prediction uses motion information to find a reference block from the reference image and generate a prediction block based on the reference block to eliminate temporal redundancy. Inter-frame prediction uses motion information to find a reference block from the reference image and generate a prediction block based on the reference block. Motion information includes the reference image list in which the reference image is located, the reference image index, and the motion vector. The motion vector can be integer pixel or fractional pixel. If the motion vector is fractional pixel, interpolation filtering is required to generate the required fractional pixel block in the reference image. Here, the integer pixel or fractional pixel block in the reference image found based on the motion vector is called a reference block. Some technologies directly use the reference block as the prediction block, while others further process the reference block to generate a prediction block. Reprocessing the reference block to generate a prediction block can also be understood as using the reference block as the prediction block and then processing the prediction block to generate a new prediction block.

[0045] The intra-frame prediction unit 112 only refers to information of the same image to predict pixel information within the current code image block to eliminate spatial redundancy.

[0046] Intra-frame prediction has multiple prediction modes. For example, the H-series international digital video coding standard H.264 / AVC has eight angular prediction modes and one non-angular prediction mode. H.265 / HEVC expands this to 33 angular prediction modes and two non-angular prediction modes. HEVC uses planar, DC, and 33 angular modes for a total of 35 intra-frame prediction modes. VVC uses planar, DC, and 65 angular modes for a total of 67 intra-frame prediction modes.

[0047] It should be noted that with the increase of angle modes, intra-frame prediction will be more accurate and more in line with the needs of high-definition and ultra-high-definition digital video development.

[0048] Residual unit 120 may generate a residual block for a CU based on the pixel blocks of the CU and the prediction blocks of the PUs of the CU. For example, residual unit 120 may generate a residual block for the CU such that each sample in the residual block has a value equal to the difference between the sample in the pixel blocks of the CU and the corresponding sample in the prediction blocks of the PUs of the CU.

[0049] The transform / quantization unit 130 may quantize the transform coefficients. The transform / quantization unit 130 may quantize the transform coefficients associated with the TUs of the CU based on a quantization parameter (QP) value associated with the CU. The video encoder 100 may adjust the degree of quantization applied to the transform coefficients associated with the CU by adjusting the QP value associated with the CU.

[0050] The inverse transform / quantization unit 140 may apply inverse quantization and inverse transform, respectively, to the quantized transform coefficients to reconstruct a residual block from the quantized transform coefficients.

[0051] Reconstruction unit 150 may add samples of the reconstructed residual block to corresponding samples of one or more prediction blocks generated by prediction unit 110 to generate a reconstructed image block associated with the TU. By reconstructing the sample blocks of each TU of a CU in this manner, video encoder 100 can reconstruct the pixel blocks of the CU.

[0052] The loop filter unit 160 is used to process the inverse transformed and inverse quantized pixels to compensate for distortion information and provide a better reference for subsequent coded pixels. For example, it can perform a deblocking filtering operation to reduce the blocking effect of pixel blocks associated with the CU.

[0053] In some embodiments, the loop filtering unit 160 includes a deblocking filtering unit and a sample adaptive offset / adaptive loop filtering (SAO / ALF) unit, wherein the deblocking filtering unit is used to remove blocking effects, and the SAO / ALF unit is used to remove ringing effects.

[0054] The decoded image buffer 170 may store the reconstructed pixel blocks. The inter-prediction unit 111 may use a reference image containing the reconstructed pixel blocks to perform inter-prediction on PUs of other images. In addition, the intra-prediction unit 112 may use the reconstructed pixel blocks in the decoded image buffer 170 to perform intra-prediction on other PUs in the same image as the CU.

[0055] The entropy encoding unit 180 may receive the quantized transform coefficients from the transform / quantization unit 130. The entropy encoding unit 180 may perform one or more entropy encoding operations on the quantized transform coefficients to generate entropy-encoded data.

[0056] FIG2 is a schematic block diagram of a video decoder according to an embodiment of the present application.

[0057] 2 , video decoder 200 includes an entropy decoding unit 210, a prediction unit 220, an inverse quantization / transformation unit 230, a reconstruction unit 240, a loop filter unit 250, and a decoded picture buffer 260. It should be noted that video decoder 200 may include more, fewer, or different functional components.

[0058] Video decoder 200 may receive a bitstream. Entropy decoding unit 210 may parse the bitstream to extract syntax elements from the bitstream. As part of parsing the bitstream, entropy decoding unit 210 may parse the entropy-encoded syntax elements in the bitstream. Prediction unit 220, inverse quantization / transform unit 230, reconstruction unit 240, and loop filter unit 250 may decode video data based on the syntax elements extracted from the bitstream, thereby generating decoded video data.

[0059] In some embodiments, the prediction unit 220 includes an intra-frame prediction unit 222 and an inter-frame prediction unit 221 .

[0060] The intra-frame prediction unit 222 may perform intra-frame prediction to generate a prediction block for the PU. The intra-frame prediction unit 222 may use an intra-frame prediction mode to generate a prediction block for the PU based on the pixel blocks of spatially neighboring PUs. The intra-frame prediction unit 222 may also determine the intra-frame prediction mode of the PU based on one or more syntax elements parsed from the codestream.

[0061] The inter-frame prediction unit 221 may construct a first reference picture list (List 0) and a second reference picture list (List 1) based on syntax elements parsed from the codestream. In addition, if a PU is encoded using inter-frame prediction, the entropy decoding unit 210 may parse the motion information of the PU. The inter-frame prediction unit 221 may determine one or more reference blocks of the PU based on the motion information of the PU. The inter-frame prediction unit 221 may generate a prediction block for the PU based on the one or more reference blocks of the PU.

[0062] The inverse quantization / transform unit 230 may inversely quantize (ie, dequantize) the transform coefficients associated with the TU. The inverse quantization / transform unit 230 may use the QP value associated with the CU of the TU to determine the degree of quantization.

[0063] After inverse quantizing the transform coefficients, the inverse quantization / transform unit 230 may apply one or more inverse transforms to the inverse quantized transform coefficients in order to generate a residual block associated with the TU.

[0064] Reconstruction unit 240 uses the residual block associated with the TU of the CU and the prediction block of the PU of the CU to reconstruct the pixel block of the CU. For example, reconstruction unit 240 can add samples of the residual block to corresponding samples of the prediction block to reconstruct the pixel block of the CU to obtain a reconstructed image block.

[0065] The loop filtering unit 250 may perform a deblocking filtering operation to reduce blocking artifacts of pixel blocks associated with a CU.

[0066] The video decoder 200 may store the reconstructed image of the CU in the decoded image buffer 260. The video decoder 200 may use the reconstructed image in the decoded image buffer 260 as a reference image for subsequent prediction, or transmit the reconstructed image to a display device for presentation.

[0067] The basic process of video encoding and decoding is as follows: At the encoder end, an image is divided into blocks. For the current block, the prediction unit 110 uses intra-frame prediction or inter-frame prediction to generate a prediction block for the current block. The residual unit 120 calculates a residual block based on the predicted block and the original block of the current block. This residual block is the difference between the predicted block and the original block of the current block. This residual block can also be referred to as residual information. This residual block undergoes transformation and quantization by the transform / quantization unit 130, removing information that is insensitive to the human eye and eliminating visual redundancy. Optionally, the residual block before transformation and quantization by the transform / quantization unit 130 can be referred to as a time-domain residual block, and the time-domain residual block after transformation and quantization by the transform / quantization unit 130 can be referred to as a frequency residual block or a frequency-domain residual block. The entropy coding unit 180 receives the quantized change coefficients output by the transform and quantization unit 130, performs entropy coding on these quantized change coefficients, and outputs a bitstream. For example, the entropy coding unit 180 can eliminate character redundancy based on the target context model and probability information of the binary bitstream.

[0068] At the decoding end, the entropy decoding unit 210 can parse the code stream to obtain the prediction information, quantization coefficient matrix, etc. of the current block. The prediction unit 220 uses intra-frame prediction or inter-frame prediction on the current block based on the prediction information to generate a prediction block for the current block. The inverse quantization / transformation unit 230 uses the quantization coefficient matrix obtained from the code stream to inverse quantize and inverse transform the quantization coefficient matrix to obtain a residual block. The reconstruction unit 240 adds the prediction block and the residual block to obtain a reconstructed block. The reconstructed blocks constitute a reconstructed image, and the loop filtering unit 250 performs loop filtering on the reconstructed image based on the image or block to obtain a decoded image. The encoding end also requires similar operations as the decoding end to obtain a decoded image. The decoded image can also be called a reconstructed image, and the reconstructed image can be used as a reference image for inter-frame prediction of subsequent images.

[0069] It should be noted that the block division information determined by the encoder, as well as mode information or parameter information such as prediction, transform, quantization, entropy coding, and loop filtering, etc., are carried in the bitstream when necessary. The decoder parses the bitstream and analyzes the existing information to determine the same block division information, prediction, transform, quantization, entropy coding, loop filtering, etc. mode information or parameter information as the encoder, thereby ensuring that the decoded image obtained by the encoder and the decoder are identical.

[0070] The above is the basic process of the video codec under the block-based hybrid coding framework. With the development of technology, some modules or steps of the framework or process may be optimized. This application is applicable to the basic process of the video codec under the block-based hybrid coding framework, but is not limited to the framework and process.

[0071] The preceding text describes in detail the codec framework provided by the embodiments of the present application. This application primarily relates to the intra-frame prediction unit in the aforementioned codec. The following describes in detail the relevant concepts involved in the embodiments of the present application.

[0072] IntraTMP Technology

[0073] IntraTMP is a special intra-frame prediction coding method primarily used for encoding screen content. IntraTMP can be implemented using the process described below. First, reconstructed pixels in the L-shaped portion adjacent to the current block are selected as a template, and the most similar template is searched for within the reconstructed area of ​​the current frame. The reconstructed block corresponding to the most similar template is then used as a reference block (or matching block) for predicting the current block. As shown in Figure 3, R1 to R4 are the available IntraTMP search areas. Next, a reference block is searched point by point within these areas using a raster scan sequence.

[0074] As shown in Figure 4, the reference block can be represented by the block vector (BV) pointing from the current block to the reference block. The similarity between templates can be represented by the size of the template error. The smaller the template error, the higher the similarity between the two templates. As an example, the template error can be calculated using the sum of absolute differences (SAD). The smaller the SAD between two templates, the more similar the two templates are.

[0075] The codec can indicate whether the current block is encoded using the IntraTMP mode through a flag (such as intra_tmp_flag). If the current block is encoded using the IntraTMP mode, the same template matching process can be performed on the decoding end to obtain the same predicted block on the decoding end without the need to additionally encode the BV from the current block to the reference block.

[0076] IntraTMP adaption for camera-captured content

[0077] The IntraTMP adaptation technology for camera-captured content builds on the existing IntraTMP technology by proposing template matching with a step size S (S>1) (as shown in Figure 5(a)). Rather than searching for reference blocks point by point in a raster scan sequence, this technology searches every other point in the search area horizontally and vertically with a step size of S. For example, if the current BV for template matching is (X0, Y0), the next BV for template matching is (X0+S, Y0). The vertical coordinate of the BV for template matching in the next row should be Y0+S. After template matching is complete, the optimal reference block can be refined within a certain range. For example, as shown in Figure 5(b), during the refinement process, template matching can be performed with a smaller step size S' to optimize the matching results. This technology effectively reduces the complexity of the IntraTMP mode while maintaining good coding efficiency.

[0078] IntraTMP multiple candidate technology

[0079] IntraTMP multi-candidate technology can obtain N candidate reference blocks in the search area through the template matching process, or build a candidate reference block list of length N. The candidate reference blocks in the list can be sorted according to the size of the template error between them and the current block. A candidate block in the list is selected as the final prediction block by the index. For the coding block using IntraTMP multi-candidate technology, as shown in the code below, after decoding the IntraTMP flag intra_tmp_flag to true (true), the syntax element intra_tmp_idx can be decoded. This syntax element can represent the index of the candidate reference block selected by the encoder.

[0080] An example of a template matching process for constructing a candidate reference block list is given below.

[0081] In the first step, a first search is performed with a certain step size. For example, setting both the horizontal step size and the vertical step size to 4 can obtain the best N candidate reference blocks with a certain spacing (such as the first N candidate reference blocks with the smallest template error).

[0082] In the second step, a second search is performed in the neighboring areas of the N candidate reference blocks obtained in the first step. These neighboring areas can be set to multiple non-overlapping areas based on the step size in the first step. The optimal M candidate reference blocks are obtained from these areas (which may include the candidate reference blocks obtained in the first step).

[0083] The same construction process is used at the encoding and decoding ends to obtain a consistent list of candidate reference blocks.

[0084] Intra_tmp_idx can use fixed-length encoding or variable-length encoding. For example, Intra_tmp_idx can use truncated binary encoding.

[0085] A possible variable-length encoding method of Intra_tmp_idx is given below.

[0086] The smaller the index (the smaller the intra_tmp_idx value), the smaller the template error of the corresponding candidate block, and the greater the probability of being selected statistically. You can set a shorter codeword for a smaller intra_tmp_idx. For example:

[0087] If the maximum value N of intra_tmp_idx is large, codewords of the same length can be assigned to larger intra_tmp_idx values. For example, as shown in the following table, if N is 15, codewords of the same length can be assigned to indexes 3 to 6 or 7 to 14.

[0088] In the above table, x can be obtained by truncating the binary code.

[0089] IntraTMP fusion prediction technology

[0090] Through intra-frame template matching, the template error between the reconstructed block and the current block at different positions can be obtained. These reconstructed blocks can be represented by the BV pointed to by the current block to the reconstructed block. A candidate BV list can be constructed to record the BV with the smaller template error during the template matching process. Then, one or more BVs can be selected from the candidate BV list based on conditions such as the spacing between BVs and the template error, and the reconstructed block pointed to by the selected BV is used as the reference block of the current block. Then, a weight value can be determined for each reference block, and these reference blocks are weightedly fused according to their weight values ​​to obtain the final prediction block. The above process describes the process of IntraTMP fusion prediction, and the process of IntraTMP fusion prediction can also be seen in Figure 6.

[0091] The number of reference blocks to be fused can be a fixed value or determined based on the size of the template errors of each reference block. For example, for the N available reference blocks, a threshold Threshold = minSAD << 1 can be set, where minSAD is the minimum template error among these reference blocks. Only reference blocks with a template error less than or equal to this threshold are used in the fusion process. This method can determine the reference blocks used for fusion.

[0092] After the reference blocks for fusion are determined, the weight of each reference block can be determined based on a preset fixed value, a template error, or derivation based on the template.

[0093] IntraTMP Filtering

[0094] The reference block obtained by intra-frame template matching can be used directly as the prediction block for the current block. Alternatively, the prediction block can be filtered to improve the prediction effect. In implementation, a block-level flag can be used to indicate whether the current block has used filtering for the prediction block.

[0095] There are many forms of filters. One possible filter form is as follows: PredC = c0C + c1N + c2S + c3E + c4W + c5B;

[0096] Where C is the pixel to be filtered, N is the pixel above it, S is the pixel below it, W is the pixel to its left, and E is the pixel to its right, as shown in Figure 7. B (Bias) is a fixed value, for example, B is the median of the pixel range. c0 to c5 are the filter coefficients.

[0097] One method for determining filter coefficients is to train the filter coefficients using a reference block template and the current block template. For example, referring to Figure 8, the template area is the reconstruction area four rows above and four columns to the left of the current block. For the reference block, an additional row of area above, below, and to the left of the template area is also required as a reference. If some of this additional area is not fully encoded, the template area can be copied to obtain the additional area.

[0098] One way to train the filter coefficients is to calculate a set of coefficients such that the minimum mean square error (MSE) between the filtered reference block template and the current block template is minimized.

[0099] If the current block uses IntraTMP filtering, the reference block can be filtered. One filtering method is to filter each pixel in the reference block in order from left to right and from top to bottom. The filtered value can then be used as the prediction value.

[0100] Template-derived IntraTMP fusion

[0101] IntraTMP fusion prediction can obtain multiple reference blocks through the intra-frame template matching process and perform weighted fusion on these reference blocks. The weight value is usually a predefined fixed value or calculated based on the template error of each reference block. For example, the template errors of the reference blocks are SAD1 to SADn, and one way to calculate the weight is: SADi = (SADi == 0)? 1: SADi Wi=(SUM-SADi) / ((n-1)*SUM)

[0102] Where n is the total number of reference blocks, and Wi is the weight value corresponding to the reference block with template error SADi. The form of the prediction block after weighted fusion can be as follows:

[0103] The template-derived IntraTMP fusion method uses a method similar to filter coefficient training, and obtains the weights for fusion prediction based on the training of each reference block template and the current block template. For example, 5 reference blocks can be weighted fused to determine the prediction block, as follows:

[0104] One way to calculate the weight is to calculate a set of coefficients so that the MSE of the reference block template after fusion with the current block template is minimized.

[0105] Template-Based Intra Mode Derivation (TIMD) technology

[0106] TIMD technology uses the reconstructed pixels of the L-shaped portion adjacent to the current block as a template. By traversing the most probable mode (MPM) list, it calculates the predicted pixels of the template area under different intra-frame prediction modes, thereby obtaining the template error between the predicted pixels and the reconstructed pixels under different intra-frame prediction modes. This template error can be represented by the sum of absolute transformed differences (SATD). Then, the optimal intra-frame prediction mode can be selected based on the template error. At the decoding end, the intra-frame prediction mode is obtained through the same derivation method, thereby reducing the coding bits of the mode information.

[0107] CIIP technology

[0108] CIIP technology combines intra-frame prediction and inter-frame prediction, and uses a weighted combination of intra-frame prediction blocks and inter-frame prediction blocks to obtain the prediction block of the current block. In ECM, CIIP is combined with template-based prediction technology, and different weight values ​​are assigned to different areas, further improving the accuracy of the prediction. For example, the intra-frame prediction block pred_intra can be obtained by the TIMD mode, and the inter-frame prediction block pred_inter can be obtained by the template-based Merge mode. According to the derived intra-frame prediction mode and the position of the pixel to be predicted, the weight values ​​wIntra and wInter can be determined. The final prediction block Pred can be calculated using the following formula: Pred = (wIntra*pred_intra+wInter*pred_inter+4)>>3

[0109] Among them, wIntra and wInter can be determined according to the intra prediction mode intra_dir derived from TIMD. There are 65 intra-frame angular prediction modes in ECM (2≤intra_dir<=66). When 2≤intra_dir<34, the current block is divided into four equal parts vertically; when 34<=intra_dir<=66, the current block is divided into four equal parts horizontally. The weight values ​​of wIntra and wInter for each region are:

[0110] The index of each area when divided into four equal parts vertically or horizontally is shown in Figure 9.

[0111] When intra_dir is equal to 0 or 1, the sub-region is not divided, and wIntra and wInter are selected from (3, 1), (2, 2), and (1, 3) according to the coding type (intra-frame or inter-frame) of the two coding blocks located on the left and above.

[0112] IBC

[0113] IBC is an intra-frame prediction technique that uses block matching to obtain predicted pixels. Similar to inter-frame prediction, IBC achieves prediction by using a block-valued (BV) that points from the current block to a reference block. The difference is that the reference block for inter-frame prediction comes from an already coded reconstructed frame, while the reference block for IBC comes from the reconstructed portion of the current frame. BVs must be transmitted in the bitstream, so similar to intra-frame prediction, IBC technology has two modes: IBC-AMVP and IBC-Merge.

[0114] IBC-AMVP mode: The predicted BV is obtained by constructing a merge candidate list. The reference block of the current block and the corresponding final BV are obtained through hash search, full search, and other processes. The final BV is encoded based on the predicted BV, thereby improving coding efficiency.

[0115] IBC-Merge mode: Prediction is performed using a constructed list of merge candidates. The best merge candidate in the list is selected through coding processes such as SATD and rate distortion optimization (RDO). The prediction is then completed by inheriting the BV of the merge candidate to obtain the reference block. IBC-Merge mode encodes the index of the merge candidate in the list rather than the BV itself, which can improve coding efficiency.

[0116] The merge candidate list can be composed of coding information such as adjacent and non-adjacent coded blocks, historical coded blocks, temporal coded blocks, and the average of candidate BVs. After constructing the merge candidate list, the list can be reordered based on the template error of each candidate in the list. The template error can be obtained by the error value (such as the SAD value) between the reference block template of each candidate and the current block template.

[0117] The merge candidate may include information such as a LIC flag. If the LIC flag of a candidate is true, when the current block selects the candidate, the IBC-LIC technology is used for prediction (for an introduction to the LIC technology, see below).

[0118] Based on the IBC-Merge mode, related technologies also introduce the IBC-MBVD mode. After the BV is obtained based on the merge candidate, the IBC-MBVD mode determines the BV error (block vector difference, BVD) by a preset offset (offset) and direction, thereby correcting the original BV. For example, if the original BV is BV0 = (x0, y0), for BV0, a ​​BVD with an upward direction and an offset size of k can be applied. Then, the new BV0' after applying BVD can be expressed as: BV0' = (x0 + k, y).

[0119] LIC technology

[0120] As shown in Figure 10, LIC technology is a block-level linear transformation technology. LIC technology assumes that there is a linear relationship between the current block and the reference block, which can be expressed as: Pred = α·ref + β

[0121] Among them, α and β can be derived from the pixels of the reconstructed area adjacent to the current block and the pixels of the reconstructed area adjacent to the reference block, and obtained by methods such as the least squares method.

[0122] In addition, the threshold can be set, and different linear parameters can be selected according to the size relationship between the reference block pixels and the threshold, which can be expressed as:

[0123] Non-adjacent candidates for Inter-Merge

[0124] When constructing the merge list for inter-frame coded blocks in Merge mode, non-adjacent candidates can be added. For example, non-adjacent points can be selected based on the current block size and used to construct merge candidates. The positions of non-adjacent points and the current block are shown in Figure 11. Candidates numbered 1-5 in Figure 11 are called adjacent candidates, while candidates at other positions are called non-adjacent candidates.

[0125] Multimodal IBC-LIC technology

[0126] The multi-modal IBC-LIC technique is further divided into multiple sub-methods based on the template area used to derive linear parameters and the number of linear transformation models. One example is to divide IBC-LIC into the following four sub-methods:

[0127] 1. Use L-shaped template area, single model

[0128] 2. Use the left template area, single model

[0129] 3. Use the template area on the right, a single model

[0130] 4. Use L-shaped template area and two models (select different models for linear transformation based on threshold)

[0131] A specific IBC-LIC sub-method can be selected by means of a code index.

[0132] List-based IBC transformation prediction technology

[0133] List-based IBC transform prediction technology builds a candidate list of available transform models for the current block by using the transform models of adjacent and non-adjacent coded blocks, historical transform models, temporal transform models, and default transform models. It then selects a specific transform model from the list based on the index written into the bitstream. This transform model is used to transform the reference block of the current block to obtain a predicted block.

[0134] For coding blocks encoded using Inter, IBC, and IntraTMP, the reference block can be transformed to obtain the prediction block. The reference block transformation methods can include the following:

[0135] LIC transform: The relationship between the predicted block pixel Pred(x,y) and the reference block pixel ref(x,y) is as follows: Pred(x,y)=α·ref(x,y)+β

[0136] Multi-model LIC transform: The relationship between the predicted block pixel Pred(x,y) and the reference block pixel ref(x,y) is as follows:

[0137] Wherein, threshold is the reference pixel threshold, and different parameters are used to perform linear transformation on reference block pixels that exceed the threshold and those that do not exceed the threshold.

[0138] Filter transformation: The relationship between the predicted block pixels Pred(x,y) and the reference block pixels is as follows: Pred(x,y)=c0C+c1N+c2S+c3E+c4W+c5B

[0139] Where C is the reference block pixel at (x, y), N is the pixel above it, S is the pixel below it, W is the pixel to its left, and E is the pixel to its right. B (Bias) is a fixed value, for example, the midpoint of the pixel range. c0 to c5 are the filter coefficients.

[0140] Accordingly, according to the above-mentioned various transformation forms, the parameter information required for the transformation (such as α i ,β i ,c i , threshold, etc.) as the transformation model corresponding to the coding block position. These transformation models can be used to construct the transformation model candidate list for subsequent IBC coding blocks.

[0141] Before applying the transformation model in the list, you can calculate the offset value based on the pixel values ​​of the current block and the reference block template area, and adjust the predicted value based on the offset, for example:

[0142] Assuming the template contains N pixels, the template prediction Pred of each pixel can be calculated based on the reference block template, thereby calculating the total difference totalDiff and offset between the template prediction and the current block template pixel Rec as follows: totalDiff = ∑(Rec-Pred); offset = totalDiff / N.

[0143] The final predicted value can be the transformed predicted value plus the offset. Taking LIC transformation as an example, the final predicted value can be calculated using the following formula: Pred(x,y)=α·ref(x,y)+β+offset

[0144] You can determine whether the transformation models are the same as follows:

[0145] a. If the transformation type is different, the transformation model is different;

[0146] b. If the LIC transformation α coefficient is different, the transformation model is different;

[0147] c. If it is a multi-model LIC transformation, any α i If they are different, the transformation models are different;

[0148] d. If it is a filter transformation, any c i If they are different, the transformation models are different;

[0149] e. If the thresholds of multiple models are different, the transformation models will be different.

[0150] For example, a coding block adjacent to the current block is encoded using the IBC-LIC method, which derives the model parameters α1 and β1 from the template region. This transformation model (α0, β0) is saved. When encoding the current block, this transformation model is added as a candidate to the list of available transformation model candidates. Then, the transformation model is selected based on the index, and the predicted block can be obtained through the following steps:

[0151] Step 1: Calculate the offset of the model based on the current block, reference block template, and the selected transformation model (α0, β0);

[0152] Step 2: Adjust the parameters according to the offset: β1 = β1 + offset

[0153] Step 3: Calculate the predicted block: Pred = α1·ref + β1

[0154] The block-level flag cu_ibc_model_merge_flag can be set for the above method. cu_ibc_model_merge_flag can be used to indicate whether the method is used, and the index cu_ibc_model_merge_idx is used to indicate the index of the selected transformation model in the list. One example is that the method explicitly transmits the above syntax elements under both the IBC-AMVP and IBC-Merge models. In addition, when the coding block in the IBC-Merge mode inherits the coding information of the encoded IBC-Model-Merge block, it only inherits the type information of the transformation model and re-derives a new transformation model based on the template for prediction.

[0155] Adaptive IBC-MBVD list construction technology

[0156] This technology proposes an IBC-MBVD list construction technology with adaptive offset size. An MBVD list containing K candidates can be obtained by the following steps:

[0157] Step 1. Set the maximum offset value to N pixels, for example, N is 256. Set the number of possible directions D, for example, D is 4 (up, down, left, right). Set the starting interval of the search to M pixels, for example, M is 8. Set the size of the MBVD list K, for example, K is 8;

[0158] Step 2: Determine the reference block template error at various offset sizes along each direction, with intervals of M, where the offset does not exceed N. The K candidates with the smallest template error are saved in the MBVD list. A candidate is a combination of an offset value and a direction, and a specific reference block position can be determined based on the candidate.

[0159] Step 3: For each candidate in the list, determine the template error of the candidate's new position with an offset of +-M / 2 along the direction. Save the K candidates with the smallest template errors in the MBVD list.

[0160] Step 4. Repeat step 3, dividing M by 2 each time until the value of M is 1 pixel.

[0161] IBC's historical MV prediction technology

[0162] This technology maintains a historical BV list with a maximum length of N, recording the BVs of encoded IBC blocks. When a block is encoded using IBC mode, the historical BV list can be updated. If the current BV is already in the historical BV list, it is moved to the end of the list. Otherwise, the historical BV list is first checked to see if it is full. If so, the BV at the beginning of the list is removed. The BV is then added to the end of the historical BV list. This historical block vector list can be used to construct the IBC Merge list.

[0163] IBC is currently widely used in various codec standards. Improving IBC's prediction performance, and thus further improving coding efficiency, is a challenge that needs to be addressed.

[0164] Based on this, an embodiment of the present application proposes a coding method, including: performing intra-frame template matching prediction on a current block to determine the prediction block and motion information of the current block; determining the residual block of the current block based on the prediction block of the current block; storing the motion information of the current block in a motion information cache corresponding to the current block, and using the motion information in the motion information cache to construct a merge candidate set for a target block, and the target block is predicted based on intra-frame block copying.

[0165] In addition, an embodiment of the present application also proposes a decoding method, including: performing intra-frame template matching prediction on the current block to determine the prediction block and motion information of the current block; determining the reconstructed block of the current block based on the prediction block of the current block and the residual block of the current block; storing the motion information of the current block in a motion information cache corresponding to the current block, and the motion information in the motion information cache is used to construct a merge candidate set for the target block, and the target block is predicted based on intra-frame block copying.

[0166] From the above content, it can be seen that the embodiment of the present application constructs a merge set (merge list) of IBC blocks based on the motion information of the IntraTMP block, thereby enriching the information source of the merge set of IBC blocks and improving the encoding performance of the IBC blocks.

[0167] Figure 14 is a flowchart of a decoding method provided by an embodiment of the present application. The method of Figure 14 can be applied to a decoder.

[0168] 14 , in step S1410 , intra template matching prediction is performed on the current block to determine a prediction block and motion information of the current block.

[0169] In some implementations, the current block refers to a current block to be decoded or a current decoding block. Since the current block uses intra template matching prediction, the current block may sometimes be referred to as an IntraTMP decoding block or an IntraTMP block.

[0170] In some implementations, the current block refers to a current block to be predicted or a current prediction block.

[0171] The prediction mode of the current block is intra-frame template matching prediction. Therefore, the prediction process of the current block may include: determining the template of the current block; searching for a target template that matches the template of the current block in the reconstructed area based on the template of the current block (for example, the template error between the templates can be determined, and the template with the smallest template error can be used as the target template); then, the reconstructed block corresponding to the target template can be determined as the reference block of the current block; and the prediction block of the current block can be determined based on the reference block.

[0172] In some implementations, the reference block may be directly determined as the prediction block.

[0173] In some implementations, the reference block may be filtered, and the filtered reference block may be determined as the prediction block.

[0174] In some implementations, the motion information of the current block may include one or more of the following: one or more BVs corresponding to the current block (indicating the positional relationship between the current block and the reference block); and the number of reference blocks of the current block.

[0175] In step S1420 , a reconstructed block of the current block is determined according to the prediction block of the current block and the residual block of the current block.

[0176] In some implementations, the sum of the prediction block and the residual block may be determined as the reconstructed block of the current block.

[0177] The residual block of the current block can be obtained by parsing the bitstream. For example, the bitstream can be parsed to determine the quantized residual block; the quantized residual block is then dequantized to determine the residual block of the current block.

[0178] In step S1430, the motion information of the current block is stored in the motion information buffer corresponding to the current block. The motion information in the motion information buffer is used to construct a merging candidate set for the target block.

[0179] The target block is an image block or decoded block predicted based on intra-block copying, i.e., an IBC block. The target block is decoded after the current block, i.e., it is a subsequent decoded block of the current block. It should be understood that "current" is a relative term; when the target block is decoded, it becomes the current block. However, for ease of description, the current block mentioned in this embodiment of the application primarily refers to the IntraTMP decoded block mentioned in step S1410.

[0180] The present embodiment fully utilizes the decoding information (motion information) of the IntraTMP decoding block, enriching the information source of the IBC merge candidate set, thereby improving the prediction performance of the IBC block. In addition, the present embodiment combines the two decoding tools with the motion information cache as the link, which basically does not increase the decoding complexity.

[0181] The motion information buffer corresponding to the current block may be used to store motion information of image blocks related to the spatial position of the target block (eg, spatially adjacent blocks or spatially non-adjacent blocks of the target block).

[0182] In some implementations, the current block is spatially related to the target block, for example, the current block is a spatially adjacent block or a spatially non-adjacent block of the target block.

[0183] In some implementations, the motion information of the current block may include first identification information. The first identification information may be used to indicate whether the current block uses a filter-based prediction mode. In other words, the first identification information may be stored as motion information so that subsequent IBC blocks to be decoded can inherit the first identification information, thereby improving IBC block decoding performance.

[0184] In some implementations, the first identification information may be a flag bit. If the value of the flag bit is 1 or true, it indicates that the current block uses a filter-based prediction mode; if the value of the flag bit is 0 or false, it indicates that the current block does not use a filter-based prediction mode.

[0185] In some implementations, the first identification information may be referred to as an IBC filter flag.

[0186] In some implementations, if the current block uses a filter-based prediction mode and the target block is predicted based on motion information of the current block, the target block uses the filter-based prediction mode.

[0187] For example, when decoding an IBC block (corresponding to the target block mentioned above), a Merge list can be constructed for the IBC block first. When constructing the Merge list, motion information can be obtained from spatially adjacent and / or non-adjacent positions of the IBC block. When the block at the corresponding position is predicted using the IBC mode or the IntraTMP mode (the block predicted using the IntraTMP mode corresponds to the current block mentioned above), the motion information at the corresponding position can be used as an available Merge candidate and added to the Merge list of the IBC block. For a specific Merge candidate, if the IBC filter flag in the motion information of the candidate is true, the IBC block can be predicted using the filtering mode. For example, the reference block template of the IBC block can be filtered to calculate the template error. For another example, the reference block of the IBC block can be filtered as the final prediction block.

[0188] In some implementations, if the current block uses a filter-based prediction mode and the target block is predicted based on motion information of the current block, the target block determines whether to use the filter-based prediction mode based on a template error between the target block and the current block.

[0189] For example, when decoding an IBC block (corresponding to the target block mentioned above), a Merge list can be constructed for the IBC block first. When constructing the Merge list, motion information can be obtained from spatially adjacent and / or non-adjacent positions of the IBC block. When the block at the corresponding position is predicted using the IBC mode or the IntraTMP mode (the block predicted using the IntraTMP mode corresponds to the current block mentioned above), the motion information at the corresponding position can be used as an available Merge candidate and added to the Merge list of the IBC block. For a specific Merge candidate, if the IBC filter flag in the motion information of the candidate is true, the IBC block can determine the template error between the template of the IBC block and the template of the block corresponding to the specific Merge candidate. If the template error is greater than or equal to the preset threshold, the IBC block can use the filter-based prediction mode; otherwise, the IBC block may not use the filter-based prediction mode.

[0190] In some implementations, if the current block uses a filter-based prediction mode, the target block uses a prediction mode based on a transform model (such as an LIC model). For example, the target block can inherit the first identification information of the current block. The first identification information indicates that the current block uses a filter-based prediction mode. After inheriting the first identification information, the target block does not use a filter-based prediction mode, but instead uses a transform-based prediction mode. Alternatively, after inheriting the first identification information, the target block can choose between a filter-based prediction mode and a transform-based prediction mode. The above implementation can further enhance the flexibility of IBC block decoding.

[0191] In some implementations, the bitstream can be parsed to determine the second identification information. The second identification information is used to indicate whether the motion information of the current block includes the first identification information. In other words, a flag can be introduced to indicate whether the IBC block is allowed to inherit the filtering operations of the IntraTMP block. This allows the functions provided by the embodiments of the present application to be flexibly enabled or disabled based on actual conditions.

[0192] In some implementations, the second identification information may correspond to the first image frame (the frame in which the current block is located) or the first image frame sequence (the frame sequence in which the current block is located). In other words, frame-level or sequence-level identification information may be introduced to control the activation or deactivation of the functions provided in the embodiments of the present application, thereby reducing encoding overhead.

[0193] In some implementations, if the size of the current block satisfies a preset condition, the motion information of the current block includes the first identification information. For example, if the width of the current block is greater than or equal to a first threshold, and / or the height of the current block is greater than or equal to a second threshold, the first identification information is saved in the motion information of the current block; otherwise, the first identification information is not saved in the motion information of the current block.

[0194] In some implementations, the current block performs intra template matching prediction based on multiple reference blocks. For example, the current block performs fusion prediction based on multiple reference blocks. In this case, the motion information of the current block may include third identification information. The third identification information is used to indicate that the prediction direction of the current block is bidirectional. In other words, if the current block performs intra template matching prediction based on multiple reference blocks, the current block can be treated as an IBC block that has performed bidirectional intra block copy mode, and the bidirectional prediction information can be stored as motion information, providing richer motion information for subsequent IBC blocks.

[0195] In some implementations, if the target block is predicted based on the motion information of the current block, the target block is predicted using a bidirectional intra block copy mode. That is, if the target block is predicted based on the motion information of the current block, the target block inherits the third identification information.

[0196] In some implementations, if intra template matching prediction is performed on a current block based on multiple reference blocks, the current block can determine a first block vector and a second block vector from the block vectors corresponding to the multiple reference blocks. The first block vector can serve as the forward block vector corresponding to the current block, and the second block vector can serve as the backward block vector corresponding to the current block. The first block vector and the second block vector can be used for bidirectional intra block copy mode prediction by subsequent IBC blocks.

[0197] The first block vector and the second block vector can be any two block vectors from among the block vectors corresponding to multiple reference blocks. In some implementations, the first block vector is a block vector corresponding to a first reference block from among the multiple reference blocks, and the second block vector is a block vector corresponding to a second reference block from among the multiple reference blocks. The first reference block and the second reference block are the two reference blocks from among the multiple reference blocks that have the smallest template error with the current block. Storing the two block vectors with the smallest corresponding template error in the motion information can improve the prediction performance of subsequent IBC blocks.

[0198] In some implementations, if the current block is predicted by intra template matching based on multiple reference blocks, the motion information of the current block is not stored in the motion information buffer.

[0199] In some implementations, the code stream can be parsed to determine the fourth identification information. The fourth identification information is used to indicate whether the motion information of the current block includes the third identification information. In other words, a flag can be introduced to indicate whether the function provided by the embodiment of the present application is enabled (i.e., for IntraTMP blocks predicted based on multiple reference blocks, the identification information of the bidirectional intra-frame block copy mode is stored in the corresponding motion information). This allows the function provided by the embodiment of the present application to be flexibly enabled or disabled according to actual conditions.

[0200] In some implementations, the fourth identification information may correspond to the first image frame (the frame in which the current block is located) or the first image frame sequence (the frame sequence in which the current block is located). In other words, frame-level or sequence-level identification information may be introduced to control the activation or deactivation of the functions provided in the embodiments of the present application, thereby reducing encoding overhead.

[0201] In some implementations, if the size of the current block satisfies a preset condition, the motion information of the current block includes the third identification information. For example, if the width of the current block is greater than or equal to a first threshold, and / or the height of the current block is greater than or equal to a second threshold, the third identification information is saved in the motion information of the current block; otherwise, the third identification information is not saved in the motion information of the current block.

[0202] In some implementations, the current block is predicted by intra-frame template matching based on multiple reference blocks, and the motion information of the current block includes a third block vector. The value of the third block vector is determined based on the values ​​of the block vectors corresponding to the multiple reference blocks. For example, the value of the third block vector is the average of the values ​​of the block vectors corresponding to the multiple reference blocks. In another example, the value of the third block vector is the weighted sum of the values ​​of the block vectors corresponding to the multiple reference blocks. The third block vector takes into account the values ​​of multiple block vectors, and saving the third block vector as motion information may improve the prediction performance of subsequent IBC blocks.

[0203] In some implementations, intra-frame template matching prediction is performed on the current block based on multiple reference blocks. The motion information of the current block includes a block vector corresponding to a third reference block. The third reference block is determined from the multiple reference blocks based on the error between a reconstructed block of the current block and the multiple reference blocks. For example, the third reference block is the reference block with the smallest error with the reconstructed block, or the reference block with an error value less than a certain threshold. By considering the error between the reference block and the reconstructed block, the block vector stored in the motion information can be a block vector with better prediction performance, which helps improve the prediction performance of the IBC block.

[0204] The decoding method provided by the embodiment of the present application is described in detail above in conjunction with Figure 14. The encoding method provided by the embodiment of the present application is described in detail below in conjunction with Figure 15.

[0205] Figure 15 is a flow chart of an encoding method according to an embodiment of the present application. The method of Figure 14 can be applied to an encoder.

[0206] 15 , in step S1510 , intra template matching prediction is performed on the current block to determine the prediction block and motion information of the current block.

[0207] In some implementations, the current block refers to a current block to be encoded or a current coding block. Since the current block uses intra template matching prediction, the current block may sometimes be referred to as an IntraTMP coding block or an IntraTMP block.

[0208] In some implementations, the current block refers to a current block to be predicted or a current prediction block.

[0209] The prediction mode of the current block is intra-frame template matching prediction. Therefore, the prediction process of the current block may include: determining the template of the current block; searching for a target template that matches the template of the current block in the reconstructed area based on the template of the current block (for example, the template error between the templates can be determined, and the template with the smallest template error can be used as the target template); then, the reconstructed block corresponding to the target template can be determined as the reference block of the current block; and the prediction block of the current block can be determined based on the reference block.

[0210] In some implementations, the reference block may be directly determined as the prediction block.

[0211] In some implementations, the reference block may be filtered, and the filtered reference block may be determined as the prediction block.

[0212] In some implementations, the motion information of the current block may include one or more of the following: one or more BVs corresponding to the current block (indicating the positional relationship between the current block and the reference block); and the number of reference blocks of the current block.

[0213] In step S1520 , a residual block of the current block is determined according to the prediction block of the current block.

[0214] In some implementations, the difference between the predicted block and the original block may be determined as a residual block of the current block.

[0215] In some implementations, the residual block of the current block may be quantized to determine a quantized residual block; the quantized residual block may be encoded, and the encoded bits may be written into a bitstream.

[0216] In step S1530, the motion information of the current block is stored in the motion information buffer corresponding to the current block. The motion information in the motion information buffer is used to construct a merging candidate set for the target block.

[0217] The target block is an image block or coding block predicted based on intra-block copying, i.e., an IBC block. The coding order of the target block is after the current block, i.e., the target block is the subsequent coding block of the current block. It should be understood that "current" is a relative concept. When encoding the target block, the target block will become the current block. For ease of description, the current block mentioned in the embodiment of the present application mainly refers to the IntraTMP coding block mentioned in step S1510.

[0218] The present embodiment fully utilizes the coding information (motion information) of the IntraTMP coding block, enriching the information source of the IBC merge candidate set, thereby improving the prediction performance of the IBC block. In addition, the present embodiment combines the two coding tools with the motion information cache as a link, which basically does not increase the coding complexity.

[0219] The motion information buffer corresponding to the current block may be used to store motion information of image blocks related to the spatial position of the target block (eg, spatially adjacent blocks or spatially non-adjacent blocks of the target block).

[0220] In some implementations, the current block is spatially related to the target block, for example, the current block is a spatially adjacent block or a spatially non-adjacent block of the target block.

[0221] In some implementations, the motion information of the current block may include first identification information. The first identification information may be used to indicate whether the current block uses a filter-based prediction mode. In other words, the first identification information may be stored as motion information so that subsequent IBC blocks to be encoded can inherit the first identification information, thereby improving the encoding performance of the IBC blocks.

[0222] In some implementations, the first identification information may be a flag bit. If the value of the flag bit is 1 or true, it indicates that the current block uses a filter-based prediction mode; if the value of the flag bit is 0 or false, it indicates that the current block does not use a filter-based prediction mode.

[0223] In some implementations, the first identification information may be referred to as an IBC filter flag.

[0224] In some implementations, if the current block uses a filter-based prediction mode and the target block is predicted based on motion information of the current block, the target block uses the filter-based prediction mode.

[0225] For example, when encoding an IBC block (corresponding to the target block mentioned above), a Merge list can be constructed for the IBC block first. When constructing the Merge list, motion information can be obtained from spatially adjacent and / or non-adjacent positions of the IBC block. When the block at the corresponding position is predicted using the IBC mode or the IntraTMP mode (the block predicted using the IntraTMP mode corresponds to the current block mentioned above), the motion information at the corresponding position can be used as an available Merge candidate and added to the Merge list of the IBC block. For a specific Merge candidate, if the IBC filter flag in the motion information of the candidate is true, the IBC block can be predicted using the filtering mode. For example, the reference block template of the IBC block can be filtered to calculate the template error. For another example, the reference block of the IBC block can be filtered as the final prediction block.

[0226] In some implementations, if the current block uses a filter-based prediction mode and the target block is predicted based on motion information of the current block, the target block determines whether to use the filter-based prediction mode based on a template error between the target block and the current block.

[0227] For example, when encoding an IBC block (corresponding to the target block mentioned above), a Merge list can be constructed for the IBC block first. When constructing the Merge list, motion information can be obtained from spatially adjacent and / or non-adjacent positions of the IBC block. When the block at the corresponding position is predicted using the IBC mode or the IntraTMP mode (the block predicted using the IntraTMP mode corresponds to the current block mentioned above), the motion information of the corresponding position can be used as an available Merge candidate and added to the Merge list of the IBC block. For a specific Merge candidate, if the IBC filter flag in the motion information of the candidate is true, the IBC block can determine the template error between the template of the IBC block and the template of the block corresponding to the specific Merge candidate. If the template error is greater than or equal to the preset threshold, the IBC block can use the filter-based prediction mode; otherwise, the IBC block may not use the filter-based prediction mode.

[0228] In some implementations, if the current block uses a filter-based prediction mode, the target block uses a prediction mode based on a transform model (such as an LIC model). For example, the target block can inherit the first identification information of the current block. The first identification information indicates that the current block uses a filter-based prediction mode. After inheriting the first identification information, the target block does not use a filter-based prediction mode, but instead uses a transform-based prediction mode. Alternatively, after inheriting the first identification information, the target block can choose between a filter-based prediction mode and a transform-based prediction mode. The above implementation can further enhance the flexibility of IBC block coding.

[0229] In some implementations, the second identification information can be encoded and the encoded bits written into the bitstream. The second identification information is used to indicate whether the motion information of the current block includes the first identification information. In other words, a flag can be introduced to indicate whether the IBC block is allowed to inherit the filtering operations of the IntraTMP block. This allows the functions provided by the embodiments of the present application to be flexibly enabled or disabled based on actual conditions.

[0230] In some implementations, the second identification information may correspond to the first image frame (the frame in which the current block is located) or the first image frame sequence (the frame sequence in which the current block is located). In other words, frame-level or sequence-level identification information may be introduced to control the activation or deactivation of the functions provided in the embodiments of the present application, thereby reducing encoding overhead.

[0231] In some implementations, if the size of the current block satisfies a preset condition, the motion information of the current block includes the first identification information. For example, if the width of the current block is greater than or equal to a first threshold, and / or the height of the current block is greater than or equal to a second threshold, the first identification information is saved in the motion information of the current block; otherwise, the first identification information is not saved in the motion information of the current block.

[0232] In some implementations, the current block performs intra template matching prediction based on multiple reference blocks. For example, the current block performs fusion prediction based on multiple reference blocks. In this case, the motion information of the current block may include third identification information. The third identification information is used to indicate that the prediction direction of the current block is bidirectional. In other words, if the current block performs intra template matching prediction based on multiple reference blocks, the current block can be treated as an IBC block that has performed bidirectional intra block copy mode, and the bidirectional prediction information can be stored as motion information, providing richer motion information for subsequent IBC blocks.

[0233] In some implementations, if the target block is predicted based on the motion information of the current block, the target block is predicted using a bidirectional intra block copy mode. That is, if the target block is predicted based on the motion information of the current block, the target block inherits the third identification information.

[0234] In some implementations, if intra template matching prediction is performed on a current block based on multiple reference blocks, the current block can determine a first block vector and a second block vector from the block vectors corresponding to the multiple reference blocks. The first block vector can serve as the forward block vector corresponding to the current block, and the second block vector can serve as the backward block vector corresponding to the current block. The first block vector and the second block vector can be used for bidirectional intra block copy mode prediction by subsequent IBC blocks.

[0235] The first block vector and the second block vector can be any two block vectors from among the block vectors corresponding to multiple reference blocks. In some implementations, the first block vector is a block vector corresponding to a first reference block from among the multiple reference blocks, and the second block vector is a block vector corresponding to a second reference block from among the multiple reference blocks. The first reference block and the second reference block are the two reference blocks from among the multiple reference blocks that have the smallest template error with the current block. Storing the two block vectors with the smallest corresponding template error in the motion information can improve the prediction performance of subsequent IBC blocks.

[0236] In some implementations, if the current block is predicted by intra template matching based on multiple reference blocks, the motion information of the current block is not stored in the motion information buffer.

[0237] In some implementations, the fourth identification information can be encoded and the encoded bits written into the bitstream. The fourth identification information is used to indicate whether the motion information of the current block includes the third identification information. In other words, an identification information can be introduced to indicate whether the function provided by the embodiment of the present application is enabled (i.e., for IntraTMP blocks predicted based on multiple reference blocks, the identification information of the bidirectional intra-frame block copy mode is stored in the corresponding motion information). In this way, the function provided by the embodiment of the present application can be flexibly enabled or disabled according to actual conditions.

[0238] In some implementations, the fourth identification information may correspond to the first image frame (the frame in which the current block is located) or the first image frame sequence (the frame sequence in which the current block is located). In other words, frame-level or sequence-level identification information may be introduced to control the activation or deactivation of the functions provided in the embodiments of the present application, thereby reducing encoding overhead.

[0239] In some implementations, if the size of the current block satisfies a preset condition, the motion information of the current block includes the third identification information. For example, if the width of the current block is greater than or equal to a first threshold, and / or the height of the current block is greater than or equal to a second threshold, the third identification information is saved in the motion information of the current block; otherwise, the third identification information is not saved in the motion information of the current block.

[0240] In some implementations, the current block is predicted by intra-frame template matching based on multiple reference blocks, and the motion information of the current block includes a third block vector. The value of the third block vector is determined based on the values ​​of the block vectors corresponding to the multiple reference blocks. For example, the value of the third block vector is the average of the values ​​of the block vectors corresponding to the multiple reference blocks. In another example, the value of the third block vector is the weighted sum of the values ​​of the block vectors corresponding to the multiple reference blocks. The third block vector takes into account the values ​​of multiple block vectors, and saving the third block vector as motion information may improve the prediction performance of subsequent IBC blocks.

[0241] In some implementations, intra-frame template matching prediction is performed on the current block based on multiple reference blocks. The motion information of the current block includes a block vector corresponding to a third reference block. The third reference block is determined from the multiple reference blocks based on the error between a reconstructed block of the current block and the multiple reference blocks. For example, the third reference block is the reference block with the smallest error with the reconstructed block, or the reference block with an error value less than a certain threshold. By considering the error between the reference block and the reconstructed block, the block vector stored in the motion information can be a block vector with better prediction performance, which helps improve the prediction performance of the IBC block.

[0242] The following examples are used to describe the embodiments of the present application in more detail. It should be noted that the following examples are only intended to help those skilled in the art understand the embodiments of the present application, and are not intended to limit the embodiments of the present application to the specific numerical values ​​or specific scenarios illustrated. It is apparent that those skilled in the art can make various equivalent modifications or changes based on the examples given, and such modifications or changes also fall within the scope of the embodiments of the present application.

[0243] Example 1:

[0244] Both IntraTMP and IBC can filter the prediction block to improve prediction performance. IntraTMP can use a block-level flag to indicate whether the current block uses the filtering process for the prediction block. IBC-Merge inherits the flag from the selected Merge candidate to indicate whether the current block uses the filtering process for the prediction block. After the IntraTMP coding block is encoded, its filtering flag can be saved. When a subsequent IBC-Merge coding block selects the motion information, it can inherit the filtering flag to determine whether to use filtering prediction.

[0245] The decoding process is as follows:

[0246] Step 1: The IntraTMP block completes the prediction and confirms the motion information that needs to be saved. For example, if the IntraTMP block uses a filter-based prediction method, the IBC filter flag can be saved as one of the motion information. In addition, the motion information can include block vectors, the number of reference blocks, etc.

[0247] Step 2: Save the motion information of IntraTMP in step 1 to the motion information buffer corresponding to the current block. This motion information can be used when subsequent encoding blocks use the current block position as a reference.

[0248] Step 3: The IBC block constructs a Merge list, where motion information can be obtained from spatially adjacent and non-adjacent locations. When the block at the corresponding location is predicted based on the IBC mode or IntraTMP mode, the motion information corresponding to the block can be added to the IBC-Merge list as one of the available Merge candidates.

[0249] Step 4: For a specific Merge candidate, if the IBC filter flag in the candidate's motion information is true, it indicates that the current IBC block is predicted using a filtering mode. For example, its reference block template can be filtered to calculate the template error; for example, its reference block is filtered as the final prediction block.

[0250] Example 2:

[0251] Both IntraTMP and IBC can use multiple reference blocks to complete prediction. Among them, IntraTMP can use a block-level flag to indicate whether the current block uses fusion prediction. IBC can indicate whether the current block uses bidirectional IBC mode by using a block-level flag or by inheriting the flag from the selected Merge candidate. When the IntraTMP block uses fusion prediction, its prediction direction can be saved as bidirectional, and the two block vectors with the smallest template error values ​​can be saved as forward block vectors and backward block vectors respectively. When the subsequent IBC-Merge encoding block selects this motion information, it can use this motion information as the motion information of the bidirectional IBC mode.

[0252] The decoding process is as follows:

[0253] Step 1: The IntraTMP block completes the prediction and confirms the motion information to be saved. For example, if the IntraTMP block uses a fusion method for prediction, the prediction direction can be saved as bidirectional, and the two block vectors with the smallest template error values ​​can be saved as the forward block vector and the backward block vector, respectively.

[0254] Step 2: Save the motion information of the IntraTMP block in step 1 to the motion information buffer corresponding to the current block. Subsequent IBC blocks can use this motion information when using the current block position as a reference.

[0255] Step 3: The IBC block constructs the IBC-Merge list. The IBC block can obtain motion information from spatially adjacent, non-adjacent, and other positions. When the block at the corresponding position is encoded in IBC mode or IntraTMP mode, the motion information corresponding to the block can be added to the IBC-Merge list as one of the available Merge candidates. If the motion information of the IntraTMP block is bidirectionally predicted, the forward block vector and the backward block vector can be added to the Merge list as two different Merge candidates. For example, first add the forward block vector as Merge candidate 1 to the Merge list. Then add the backward block vector as Merge candidate 2 to the Merge list.

[0256] This method uses the encoding information of IntraTMP encoding blocks to enrich the encoding information when IBC constructs the Merge list, thereby improving encoding efficiency. It combines two existing encoding tools without introducing additional complexity.

[0257] The method embodiment of the present application is described in detail above in conjunction with Figures 1 to 15 . The device embodiment of the present application is described in detail below in conjunction with Figures 16 to 19 . It should be understood that the description of the method embodiment corresponds to the description of the device embodiment. Therefore, for portions not described in detail, reference can be made to the above method embodiment.

[0258] FIG16 is a schematic diagram of the structure of a decoder provided by an embodiment of the present application. As shown in FIG16 , the decoder 1600 includes: a first decoding unit 1610 , a second decoding unit 1620 , and a storage unit 1630 .

[0259] The first decoding unit 1610 is configured to perform intra template matching prediction on the current block to determine a prediction block and motion information of the current block.

[0260] The second decoding unit 1620 is configured to determine a reconstructed block of the current block according to the prediction block of the current block and the residual block of the current block.

[0261] The storage unit 1630 is configured to store the motion information of the current block in a motion information cache corresponding to the current block, and the motion information in the motion information cache is used to construct a merge candidate set for a target block, and the target block is predicted based on intra block copy.

[0262] In some implementations, the motion information of the current block includes first identification information, where the first identification information is used to indicate whether the current block uses a filtering-based prediction mode.

[0263] In some implementations, if the current block uses a filter-based prediction mode and the target block is predicted based on motion information of the current block, the target block uses the filter-based prediction mode; or, if the current block uses a filter-based prediction mode and the target block is predicted based on motion information of the current block, the target block determines whether to use the filter-based prediction mode based on a template error between the target block and the current block.

[0264] In some implementations, if the current block uses a filtering-based prediction mode, the target block uses a transform model-based prediction mode.

[0265] In some implementations, the transformation model is a local illumination compensation model.

[0266] In some implementations, the decoder 1600 further includes: a third decoding unit configured to parse the code stream and determine second identification information, where the second identification information is used to indicate whether the motion information of the current block includes the first identification information.

[0267] In some implementations, the current block belongs to a first image frame or a first image frame sequence, and the second identification information corresponds to the first image frame or the first image frame sequence.

[0268] In some implementations, if the size of the current block meets a preset condition, the motion information of the current block includes the first identification information.

[0269] In some implementations, the preset condition includes: a width of the current block is greater than or equal to a first threshold; and / or a height of the current block is greater than or equal to a second threshold.

[0270] In some implementations, the current block is predicted by intra-frame template matching based on multiple reference blocks, and the motion information of the current block includes third identification information, where the third identification information is used to indicate that the prediction direction of the current block is bidirectional.

[0271] In some implementations, if the target block is predicted based on motion information of the current block, the target block is predicted using a bidirectional intra block copy mode.

[0272] In some implementations, the motion information of the current block includes a first block vector and a second block vector, where the first block vector is a forward block vector corresponding to the current block, and the second block vector is a backward block vector corresponding to the current block, and the first block vector and the second block vector are determined based on the block vectors corresponding to the multiple reference blocks.

[0273] In some implementations, the first block vector is a block vector corresponding to a first reference block among the multiple reference blocks, the second block vector is a block vector corresponding to a second reference block among the multiple reference blocks, and the first reference block and the second reference block are two reference blocks among the multiple reference blocks with the smallest template error with the current block.

[0274] In some implementations, the decoder 1600 further includes: a fourth decoding unit configured to parse the code stream and determine fourth identification information, where the fourth identification information is used to indicate whether the motion information of the current block includes the third identification information.

[0275] In some implementations, the current block belongs to a first image frame or a first image frame sequence, and the fourth identification information corresponds to the first image frame or the first image frame sequence.

[0276] In some implementations, if the size of the current block meets a preset condition, the motion information of the current block includes the third identification information.

[0277] In some implementations, the preset condition includes: a width of the current block is greater than or equal to a first threshold; and / or a height of the current block is greater than or equal to a second threshold.

[0278] In some implementations, the current block is predicted by intra-frame template matching based on multiple reference blocks, and the motion information of the current block includes a third block vector, and the value of the third block vector is determined based on the values ​​of the block vectors corresponding to the multiple reference blocks.

[0279] In some implementations, the value of the third block vector is an average of the values ​​of the block vectors corresponding to the multiple reference blocks.

[0280] In some implementations, the current block is predicted by intra-frame template matching based on multiple reference blocks, and the motion information of the current block includes a block vector corresponding to a third reference block, and the third reference block is determined from the multiple reference blocks based on an error between a reconstructed block of the current block and the multiple reference blocks.

[0281] In some implementations, the third reference block is a reference block among the multiple reference blocks that has the smallest error with the reconstructed block.

[0282] In some implementations, the motion information cache is used to cache motion information of an image block associated with a spatial position of the target block.

[0283] It is understandable that in the embodiments of the present application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the various components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional modules.

[0284] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0285] Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the decoder 1600. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the decoding method in the first embodiment.

[0286] Based on the composition of the above-mentioned decoder 1600 and the computer-readable storage medium, refer to Figure 17, which shows a specific hardware structure diagram of the decoder 1600 provided in an embodiment of the present application. As shown in Figure 17, the decoder 1700 may include: a communication interface 1710, a memory 1720 and a processor 1730; each component is coupled together through a bus system 1740. It can be understood that the bus system 1740 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 1740 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as bus system 1740 in Figure 17. Among them,

[0287] Communication interface 1710, used for sending and receiving signals when sending and receiving information with other external network elements;

[0288] Memory 1720, for storing computer programs;

[0289] The processor 1730 is configured to, when running the computer program, execute:

[0290] Performing intra-frame template matching prediction on the current block to determine a predicted block and motion information of the current block;

[0291] Determining a reconstructed block of the current block according to the prediction block of the current block and the residual block of the current block;

[0292] The motion information of the current block is stored in a motion information cache corresponding to the current block. The motion information in the motion information cache is used to construct a merge candidate set for a target block. The target block is predicted based on intra block copying.

[0293] It is understood that the memory 1720 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDRSDRAM), enhanced synchronous DRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 1720 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0294] The processor 1730 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the processor 1730. The above-mentioned processor 1730 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory 1720 , and the processor 1730 reads the information in the memory 1720 and completes the steps of the above method in combination with its hardware.

[0295] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or its combination.For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field-Programmable Gate Array, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing functions described in the present application or its combination.For software implementation, the technology described in the present application can be realized by the module (such as process, function etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0296] Optionally, as another embodiment, the processor 1730 is further configured to execute the decoding method described in the above embodiment when running the computer program.

[0297] FIG18 is a schematic diagram of the structure of an encoder provided by an embodiment of the present application. As shown in FIG18 , the encoder 1800 includes: a first encoding unit 1810 , a second encoding unit 1820 , and a storage unit 1830 .

[0298] The first encoding unit 1810 is configured to perform intra template matching prediction on the current block to determine a prediction block and motion information of the current block.

[0299] The second encoding unit 1820 is configured to determine a residual block of the current block according to the prediction block of the current block.

[0300] The storage unit 1830 is configured to store the motion information of the current block in a motion information cache corresponding to the current block, and the motion information in the motion information cache is used to construct a merge candidate set for a target block, and the target block is predicted based on intra block copy.

[0301] In some implementations, the motion information of the current block includes first identification information, where the first identification information is used to indicate whether the current block uses a filtering-based prediction mode.

[0302] In some implementations, if the current block uses a filter-based prediction mode and the target block is predicted based on the motion information of the current block, the target block uses the filter-based prediction mode; or, if the current block uses a filter-based prediction mode and the target block is predicted based on the motion information of the current block, the target block determines whether to use the filter-based prediction mode based on a template error between the target block and the current block.

[0303] In some implementations, if the current block uses a filtering-based prediction mode, the target block uses a transform model-based prediction mode.

[0304] In some implementations, the transformation model is a local illumination compensation model.

[0305] In some implementations, the encoder 1800 further includes: a third encoding unit configured to encode second identification information and write the encoded bits into the bitstream, where the second identification information is used to indicate whether the motion information of the current block includes the first identification information.

[0306] In some implementations, the current block belongs to a first image frame or a first image frame sequence, and the second identification information corresponds to the first image frame or the first image frame sequence.

[0307] In some implementations, if the size of the current block meets a preset condition, the motion information of the current block includes the first identification information.

[0308] In some implementations, the preset condition includes: a width of the current block is greater than or equal to a first threshold; and / or a height of the current block is greater than or equal to a second threshold.

[0309] In some implementations, the current block is predicted by intra-frame template matching based on multiple reference blocks, and the motion information of the current block includes third identification information, where the third identification information is used to indicate that the prediction direction of the current block is bidirectional.

[0310] In some implementations, if the target block is predicted based on motion information of the current block, the target block is predicted using a bidirectional intra block copy mode.

[0311] In some implementations, the motion information of the current block includes a first block vector and a second block vector, the first block vector is a forward block vector corresponding to the current block, the second block vector is a backward block vector corresponding to the current block, and the first block vector and the second block vector are determined based on the block vectors corresponding to the multiple reference blocks.

[0312] In some implementations, the first block vector is a block vector corresponding to a first reference block among the multiple reference blocks, the second block vector is a block vector corresponding to a second reference block among the multiple reference blocks, and the first reference block and the second reference block are two reference blocks among the multiple reference blocks with the smallest template error with the current block.

[0313] In some implementations, the encoder 1800 further includes: a fourth encoding unit configured to encode fourth identification information and write the encoded bits into the bitstream, wherein the fourth identification information is used to indicate whether the motion information of the current block includes the third identification information.

[0314] In some implementations, the current block belongs to a first image frame or a first image frame sequence, and the fourth identification information corresponds to the first image frame or the first image frame sequence.

[0315] In some implementations, if the size of the current block meets a preset condition, the motion information of the current block includes the third identification information.

[0316] In some implementations, the preset condition includes: a width of the current block is greater than or equal to a first threshold; and / or a height of the current block is greater than or equal to a second threshold.

[0317] In some implementations, the current block is predicted by intra-frame template matching based on multiple reference blocks, and the motion information of the current block includes a third block vector, and the value of the third block vector is determined based on the values ​​of the block vectors corresponding to the multiple reference blocks.

[0318] In some implementations, the value of the third block vector is an average of the values ​​of the block vectors corresponding to the multiple reference blocks.

[0319] In some implementations, the current block is predicted by intra-frame template matching based on multiple reference blocks, and the motion information of the current block includes a block vector corresponding to a third reference block, and the third reference block is determined from the multiple reference blocks based on an error between a reconstructed block of the current block and the multiple reference blocks.

[0320] In some implementations, the third reference block is a reference block among the multiple reference blocks that has the smallest error with the reconstructed block.

[0321] In some implementations, the motion information cache is used to cache motion information of an image block associated with a spatial position of the target block.

[0322] It is understandable that in the embodiments of the present application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the various components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional modules.

[0323] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0324] Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the encoder 1800. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the decoding method described in any one of the aforementioned embodiments.

[0325] Based on the composition of the above-mentioned encoder 1800 and the computer-readable storage medium, refer to Figure 19, which shows a specific hardware structure diagram of the encoder 1800 provided in an embodiment of the present application. As shown in Figure 19, the encoder 1900 may include: a communication interface 1910, a memory 1920 and a processor 1930; each component is coupled together through a bus system 1940. It can be understood that the bus system 1940 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 1940 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as bus system 1940 in Figure 19. Among them,

[0326] Communication interface 1910, used for sending and receiving signals when sending and receiving information with other external network elements;

[0327] Memory 1920, for storing computer programs;

[0328] Processor 1930 is configured to, when running the computer program, execute:

[0329] Performing intra-frame template matching prediction on the current block to determine a predicted block and motion information of the current block;

[0330] Determining a residual block of the current block according to the prediction block of the current block;

[0331] The motion information of the current block is stored in a motion information cache corresponding to the current block. The motion information in the motion information cache is used to construct a merge candidate set for a target block. The target block is predicted based on intra block copying.

[0332] It is understood that the memory 1920 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDRSDRAM), enhanced synchronous DRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 1920 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0333] Processor 1930 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in processor 1930. The above-mentioned processor 1930 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory 1920, and the processor 1930 reads the information in the memory 1920 and completes the steps of the above method in combination with its hardware.

[0334] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or its combination.For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field-Programmable Gate Array, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing functions described in the present application or its combination.For software implementation, the technology described in the present application can be realized by the module (such as process, function etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0335] Optionally, as another embodiment, the processor 1930 is further configured to execute the encoding method in the aforementioned embodiment when running the computer program.

[0336] As shown in Figure 20, the embodiment of the present application further provides a recording method for recording a code stream to a recording medium, comprising a recording step 2010 of recording a code stream generated by the encoding method described in any of the above embodiments to the recording medium.

[0337] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a code stream generated by the encoding method described in any of the above embodiments.

[0338] It should be noted that, in this application, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0339] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0340] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0341] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0342] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0343] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A decoding method, applied to a decoder, comprising: Performing intra-frame template matching prediction on the current block to determine a prediction block and motion information of the current block; Determine a reconstructed block of the current block according to the prediction block of the current block and the residual block of the current block; The motion information of the current block is stored in a motion information cache corresponding to the current block, the motion information in the motion information cache is used to construct a merge candidate set for a target block, and the target block is predicted based on intra block copying.

2. The method according to claim 1, wherein: The motion information of the current block includes first identification information, where the first identification information is used to indicate whether the current block uses a filtering-based prediction mode.

3. The method according to claim 2, wherein: If the current block uses a filter-based prediction mode, and the target block is predicted based on motion information of the current block, the target block uses a filter-based prediction mode; or, If the current block uses a filter-based prediction mode and the target block is predicted based on motion information of the current block, the target block determines whether to use a filter-based prediction mode based on a template error between the target block and the current block.

4. The method according to claim 2, wherein: If the current block uses a prediction mode based on filtering, the target block uses a prediction mode based on a transform model.

5. The method according to claim 4, wherein: The transformation model is a local illumination compensation model.

6. The method according to any one of claims 2 to 5, wherein: The method further comprises: Parse the bitstream to determine second identification information, where the second identification information is used to indicate whether the motion information of the current block includes the first identification information.

7. The method according to claim 6, wherein: The current block belongs to a first image frame or a first image frame sequence, and the second identification information corresponds to the first image frame or the first image frame sequence.

8. The method according to any one of claims 2 to 5, wherein: If the size of the current block meets a preset condition, the motion information of the current block includes the first identification information.

9. The method according to claim 8, wherein: The preset conditions include: The width of the current block is greater than or equal to a first threshold; and / or The height of the current block is greater than or equal to a second threshold.

10. The method according to claim 1, wherein: The current block is predicted by intra-frame template matching based on multiple reference blocks, and the motion information of the current block includes third identification information, where the third identification information is used to indicate that the prediction direction of the current block is bidirectional.

11. The method according to claim 10, wherein: If the target block is predicted based on the motion information of the current block, the target block is predicted using a bidirectional intra block copy mode.

12. The method according to claim 10, wherein: The motion information of the current block includes a first block vector and a second block vector, the first block vector is a forward block vector corresponding to the current block, the second block vector is a backward block vector corresponding to the current block, and the first block vector and the second block vector are determined based on the block vectors corresponding to the multiple reference blocks.

13. The method according to claim 12, wherein: The first block vector is a block vector corresponding to a first reference block among the multiple reference blocks, the second block vector is a block vector corresponding to a second reference block among the multiple reference blocks, and the first reference block and the second reference block are two reference blocks among the multiple reference blocks having the smallest template error with the current block.

14. The method according to any one of claims 10 to 13, wherein: The method further comprises: Parse the bitstream to determine fourth identification information, where the fourth identification information is used to indicate whether the motion information of the current block includes the third identification information.

15. The method according to claim 14, wherein: The current block belongs to a first image frame or a first image frame sequence, and the fourth identification information corresponds to the first image frame or the first image frame sequence.

16. The method according to any one of claims 10 to 13, wherein: If the size of the current block meets a preset condition, the motion information of the current block includes the third identification information.

17. The method according to claim 16, wherein: The preset conditions include: The width of the current block is greater than or equal to a first threshold; and / or The height of the current block is greater than or equal to a second threshold.

18. The method according to claim 1, wherein: The current block performs intra-frame template matching prediction based on multiple reference blocks, and the motion information of the current block includes a third block vector. The value of the third block vector is determined based on the values ​​of block vectors corresponding to the multiple reference blocks.

19. The method according to claim 18, wherein: The value of the third block vector is the average value of the block vectors corresponding to the multiple reference blocks.

20. The method according to claim 1, wherein: The current block performs intra-frame template matching prediction based on multiple reference blocks, and the motion information of the current block includes a block vector corresponding to a third reference block. The third reference block is determined from the multiple reference blocks based on an error between a reconstructed block of the current block and the multiple reference blocks.

21. The method according to claim 20, wherein: The third reference block is a reference block having the smallest error with the reconstructed block among the multiple reference blocks.

22. The method according to any one of claims 1 to 21, wherein: The motion information cache is used to cache motion information of an image block related to a spatial position of the target block.

23. A coding method, applied to an encoder, comprising: Performing intra-frame template matching prediction on the current block to determine a prediction block and motion information of the current block; Determine a residual block of the current block according to the prediction block of the current block; The motion information of the current block is stored in a motion information cache corresponding to the current block, the motion information in the motion information cache is used to construct a merge candidate set for a target block, and the target block is predicted based on intra block copying.

24. The method according to claim 23, wherein: The motion information of the current block includes first identification information, where the first identification information is used to indicate whether the current block uses a filtering-based prediction mode.

25. The method of claim 24, wherein: If the current block uses a filter-based prediction mode, and the target block is predicted based on motion information of the current block, the target block uses a filter-based prediction mode; or, If the current block uses a filter-based prediction mode and the target block is predicted based on motion information of the current block, the target block determines whether to use a filter-based prediction mode based on a template error between the target block and the current block.

26. The method according to claim 24, wherein: If the current block uses a prediction mode based on filtering, the target block uses a prediction mode based on a transform model.

27. The method according to claim 26, wherein: The transformation model is a local illumination compensation model.

28. The method according to any one of claims 24 to 27, wherein: The method further comprises: The second identification information is encoded and the encoded bits are written into the bitstream, where the second identification information is used to indicate whether the motion information of the current block includes the first identification information.

29. The method according to claim 28, wherein: The current block belongs to a first image frame or a first image frame sequence, and the second identification information corresponds to the first image frame or the first image frame sequence.

30. The method according to any one of claims 24 to 27, wherein: If the size of the current block meets a preset condition, the motion information of the current block includes the first identification information.

31. The method according to claim 30, wherein: The preset conditions include: The width of the current block is greater than or equal to a first threshold; and / or The height of the current block is greater than or equal to a second threshold.

32. The method of claim 23, wherein: The current block is predicted by intra-frame template matching based on multiple reference blocks, and the motion information of the current block includes third identification information, where the third identification information is used to indicate that the prediction direction of the current block is bidirectional.

33. The method of claim 32, wherein: If the target block is predicted based on the motion information of the current block, the target block is predicted using a bidirectional intra block copy mode.

34. The method of claim 32, wherein: The motion information of the current block includes a first block vector and a second block vector, the first block vector is a forward block vector corresponding to the current block, the second block vector is a backward block vector corresponding to the current block, and the first block vector and the second block vector are determined based on the block vectors corresponding to the multiple reference blocks.

35. The method of claim 34, wherein: The first block vector is a block vector corresponding to a first reference block among the multiple reference blocks, the second block vector is a block vector corresponding to a second reference block among the multiple reference blocks, and the first reference block and the second reference block are two reference blocks among the multiple reference blocks having the smallest template error with the current block.

36. The method according to any one of claims 32 to 35, wherein: The method further comprises: The fourth identification information is encoded, and the encoding bits are written into the bitstream, wherein the fourth identification information is used to indicate whether the motion information of the current block includes the third identification information.

37. The method of claim 36, wherein: The current block belongs to a first image frame or a first image frame sequence, and the fourth identification information corresponds to the first image frame or the first image frame sequence.

38. The method according to any one of claims 32 to 35, wherein: If the size of the current block meets a preset condition, the motion information of the current block includes the third identification information.

39. The method of claim 38, wherein: The preset conditions include: The width of the current block is greater than or equal to a first threshold; and / or The height of the current block is greater than or equal to a second threshold.

40. The method of claim 23, wherein: The current block performs intra-frame template matching prediction based on multiple reference blocks, and the motion information of the current block includes a third block vector. The value of the third block vector is determined based on the values ​​of block vectors corresponding to the multiple reference blocks.

41. The method of claim 40, wherein: The value of the third block vector is the average value of the block vectors corresponding to the multiple reference blocks.

42. The method of claim 23, wherein: The current block performs intra-frame template matching prediction based on multiple reference blocks, and the motion information of the current block includes a block vector corresponding to a third reference block. The third reference block is determined from the multiple reference blocks based on an error between a reconstructed block of the current block and the multiple reference blocks.

43. The method of claim 42, wherein: The third reference block is a reference block having the smallest error with the reconstructed block among the multiple reference blocks.

44. The method according to any one of claims 23 to 43, wherein: The motion information cache is used to cache motion information of an image block related to a spatial position of the target block.

45. A decoder comprising: A first decoding unit, configured to perform intra-frame template matching prediction on a current block to determine a prediction block and motion information of the current block; A second decoding unit, configured to determine a reconstructed block of the current block according to a prediction block of the current block and a residual block of the current block; The storage unit is configured to store the motion information of the current block in a motion information cache corresponding to the current block, the motion information in the motion information cache is used to construct a merge candidate set for a target block, and the target block is predicted based on intra-block copying.

46. ​​A decoder, the decoder comprising: Memory for storing computer programs; A processor, configured to execute the method according to any one of claims 1 to 22 when running the computer program.

47. An encoder comprising: A first encoding unit, configured to perform intra-frame template matching prediction on a current block to determine a prediction block and motion information of the current block; A second encoding unit, configured to determine a residual block of the current block according to a prediction block of the current block; The storage unit is configured to store the motion information of the current block in a motion information cache corresponding to the current block, the motion information in the motion information cache is used to construct a merge candidate set for a target block, and the target block is predicted based on intra-block copying.

48. An encoder, comprising: Memory for storing computer programs; A processor, configured to execute the method according to any one of claims 23 to 44 when running the computer program.

49. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 22 or the method according to any one of claims 23 to 44 is implemented.

50. A recording method for recording a code stream on a recording medium, the recording method comprising a recording step of recording the code stream generated by the encoding method according to any one of claims 23 to 44 on the recording medium.

51. A computer-readable storage medium storing a code stream generated by the encoding method according to any one of claims 23 to 44.

Images