Encoding / decoding method, bitstream, encoding device, decoding device, and storage medium

Abstract

The embodiments of the present application provide an encoding / decoding method, a bitstream, an encoding device, a decoding device, and a storage medium, which include the steps of: an encoding device determining a first template corresponding to a current block, determining one or more block vectors corresponding to the current block based on the first template, determining one or more reference blocks for the current block based on the one or more block vectors, determining a predicted value of the current block based on the one or more reference blocks, and determining a reconstructed value of the current block based on the predicted value of the current block; and a decoding device determining a first template corresponding to the current block, determining one or more block vectors corresponding to the current block based on the first template, determining one or more reference blocks for the current block based on the one or more block vectors, and determining a predicted value of the current block based on the one or more reference blocks.

Description

[Technical Field]

[0001] The embodiments of the present application relate to the technical field of video encoding and decoding, and in particular to an encoding and decoding method, a bitstream, an encoding device, a decoding device, and a storage medium. [Background technology]

[0002] The intra template matching prediction (IntraTMP) technology uses the template of the coding block to search for a matching template that minimizes the cost according to a predetermined cost function within a predefined search range in the current image, and determines the optimal matching reconstruction block corresponding to the matching template as the prediction block of the current coding block.

[0003] However, in the actual encoding process, related technologies usually use the reconstructed samples of the best-matching reconstructed block as the predicted samples of the current encoding block, but due to insufficient consideration, there will be large deviations in some scenes, resulting in low prediction accuracy. Summary of the Invention [Problem to be solved by the invention]

[0004] The embodiments of the present application provide an encoding / decoding method, a bitstream, an encoding device, a decoding device, and a storage medium, which can improve prediction accuracy and achieve optimal prediction results. [Means for solving the problem]

[0005] The technical solutions in the embodiments of the present application can be realized as follows:

[0006] According to a first aspect, an embodiment of the present application provides a decoding method for use in a decoding device, the decoding method comprising: determining a first template corresponding to the current block; determining one or more block vectors corresponding to the current block based on the first template; determining one or more reference blocks for a current block based on the one or more block vectors, and determining a predicted value for the current block based on the one or more reference blocks; determining a reconstructed value for the current block based on the predicted value for the current block.

[0007] According to a second aspect, an embodiment of the present application provides an encoding method for use in an encoding device, the encoding method comprising: determining a first template corresponding to the current block; determining one or more block vectors corresponding to the current block based on the first template; determining one or more reference blocks for the current block based on the one or more block vectors, and determining a predicted value for the current block based on the one or more reference blocks.

[0008] According to a third aspect, an embodiment of the present application provides a bitstream, the bit stream is generated by bit encoding based on encoding target information, The encoding target information includes at least one of a prediction difference value of the current block, a predetermined number N, and one or more block vectors.

[0009] According to a fourth aspect, an embodiment of the present application provides an encoding device, including a first determining unit, The first determination unit is configured to determine a first template corresponding to a current block, determine one or more block vectors corresponding to the current block based on the first template, determine one or more reference blocks for the current block based on the one or more block vectors, and determine a predicted value of the current block based on the one or more reference blocks.

[0010] According to a fifth aspect, an embodiment of the present application provides an encoding device, including a first memory and a first processor; the first memory is configured to store a computer program executable by the first processor; The first processor is configured to execute the computer program to perform the encoding method according to the second aspect.

[0011] According to a sixth aspect, an embodiment of the present application provides a decoding device, including a second determining unit; The second determination unit is configured to determine a first template corresponding to the current block, determine one or more block vectors corresponding to the current block based on the first template, determine one or more reference blocks for the current block based on the one or more block vectors, determine a predicted value of the current block based on the one or more reference blocks, and determine a reconstructed value of the current block based on the predicted value of the current block.

[0012] According to a seventh aspect, an embodiment of the present application provides a decoding device, including a second memory and a second processor; the second memory is configured to store a computer program executable by the second processor; The second processor is configured to execute the computer program to perform the decoding method according to the first aspect.

[0013] According to an eighth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored therein, the computer program being capable of implementing the decoding method according to the first aspect or the encoding method according to the second aspect when executed. [Effects of the Invention]

[0014] The present application provides an encoding / decoding method, an encoding device, a decoding device, and a storage medium. The encoding device determines a first template corresponding to a current block, determines one or more block vectors corresponding to the current block based on the first template, determines one or more reference blocks for the current block based on the one or more block vectors, determines a predicted value of the current block based on the one or more reference blocks, and determines a reconstructed value of the current block based on the predicted value of the current block. The decoding device determines a first template corresponding to the current block, determines one or more block vectors corresponding to the current block based on the first template, determines one or more reference blocks for the current block based on the one or more block vectors, and determines a predicted value of the current block based on the one or more reference blocks. Thus, the present application provides an IntraTMP Fusion prediction method, which can select and determine at least one block vector for the current block and further obtain a predicted value of the current block using at least one reference block corresponding to the at least one block vector. In other words, the encoding / decoding method according to the embodiment of the present application fully considers the difference in importance of the reconstruction block information corresponding to different matching templates in the search process to the prediction of the current block, thereby improving the accuracy of prediction and achieving optimal prediction results. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a predicted schematic diagram of IntraTMP. [Figure 2] FIG. 1 is a schematic diagram of the prediction process based on the IntraTMP technology. [Figure 3] Schematic diagram of template type of IntraTMP technology. [Figure 4] 1 is a schematic diagram of a search process based on IntraTMP technology. [Figure 5] FIG. 1 is a schematic diagram of parameter definitions for a current block and its template. [Figure 6] FIG. 2 is a schematic diagram of a template search area. [Figure 7] FIG. 10 is a schematic diagram of division of a search area into different sub-areas. [Figure 8] This is one schematic diagram of the process of determining the search area. [Figure 9] FIG. 1 is a schematic diagram of the search process. [Figure 10] 1A and 1B are block diagrams illustrating the configuration of an encoding device. [Figure 11] Schematic diagram of the network architecture of the encoding / decoding system. [Figure 12] 1 is a flowchart of a decoding method according to an embodiment of the present application; [Figure 13] This is the second schematic diagram of the process of determining the search area. [Figure 14] This is the third schematic diagram of the process of determining the search area. [Figure 15] 1 is a flowchart of an encoding method according to an embodiment of the present application; [Figure 16] FIG. 1 is a schematic diagram illustrating the configuration of an encoding device. [Figure 17] FIG. 2 is a schematic diagram of the hardware structure of the encoding device. [Figure 18] FIG. 2 is a schematic diagram illustrating the configuration of a decoding device. [Figure 19] FIG. 2 is a schematic diagram of the hardware structure of a decoding device. [Figure 20] FIG. 1 is a schematic diagram of the configuration of an encoding / decoding system. DETAILED DESCRIPTION OF THE INVENTION

[0016] In order to explain the configuration and technical content of the embodiments of the present application in more detail, the implementation of the embodiments of the present application will be described in detail below with reference to the drawings, and the accompanying drawings are for reference and explanation purposes only and do not limit the embodiments of the present application.

[0017] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by those skilled in the art. The terms used in this specification are only for the purpose of describing the examples of this application and are not intended to limit this application.

[0018] In the following description, "some embodiments" refers to a subset of all possible embodiments, but "some embodiments" may refer to the same or different subsets of all possible embodiments and may be combined with each other if not inconsistent. The terms "first / second / third" used in the embodiments of the present application are merely used to distinguish between similar objects and do not represent a specific order for the objects. The terms "first / second / third" may be interchanged to indicate a specific order or context, depending on the circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0019] Before describing the embodiments of the present application in more detail, the nouns and terms related to the embodiments of the present application will be explained first. The nouns and terms related to the embodiments of the present application will be explained as follows.

[0020] Coding Block, CB Block Matching:Block Matching, BM Coding Unit: Coding Unit (CU) Block Vector, BV Sum of Absolute Difference (SAD) Sum of Absolute Transformed Difference (SATD) Mean Square Error, MSE Sum of Squared Differences, SSD Mean Absolute Deviation (MAD) Mean Square Differences, MSD Normalized Correlation Coefficient: NCC H.266 / Various video coding: VVC VVC's reference software test platform: VVC Test Model, VTM Intra Template Matching Prediction, IntraTMP Beyond VVC Reference Software Test Platform: Enhanced Compression Model, ECM

[0021] In a video image, a coding block is generally represented by a first color component, a second color component, and a third color component, where these three color components are one luma component, one blue chroma component, and one red chroma component, respectively. Specifically, the luma component is usually represented by the symbol Y, the blue chroma component is usually represented by the symbol Cb or U, and the red chroma component is usually represented by the symbol Cr or V. Thus, a video image may be represented in either a YCbCr format or a YUV format.

[0022] IntraTMP is a special intra prediction mode, in which both the encoding device and the decoding device use a coding block template (T) to search for a matching template (T_BEST) that minimizes the cost within a predefined search range in the current image according to a predetermined cost function. Here, the offset of the best matching template relative to the current coding block template is the best block vector (BEST Block Vector, BV_BEST), and the reconstruction block (Ref Block) corresponding to the matching template is set as the prediction block of the current coding block (Cur Block). The coding block template usually selects a neighboring reconstruction region of the current coding block.

[0023] Taking the adjacent reconstruction area of ​​the current block as an example, FIG. 1 is a schematic diagram of IntraTMP prediction. As shown in FIG. 1, the dark-shaded area indicates the reconstruction area, the grid-shaded block is the current block, and the adjacent area of ​​the current block is the first template (T). The shaded block is the reference block, and the adjacent area of ​​the reference block is the second template (i.e., the best matching template, T_BEST). Here, the offset of the second template relative to the first template is the best block vector (BV_BEST). In this case, the reference block can be block-copied to be the predicted block of the current block.

[0024] In the embodiments of the present application, the predetermined cost function may be the sum of absolute differences SAD, the sum of absolute transformed differences SATD, the mean squared error MSE, the sum of squared residuals SSD, the mean absolute deviation MAD, the mean sum of squared residuals MSD, the normalized correlation coefficient NCC, etc., and is not specifically limited here.

[0025] For example, taking the sum of absolute errors SAD as an example, the cost function at this time is as follows:

number

[0026] where T i is the template in the search process, and M represents the number of samples in the template.

[0027] The prediction process of the IntraTMP technology in the related art will be described in detail below.

[0028] The inputs of IntraTMP are the position of the current block (xTbCmp, yTbCmp), the width of the current block nTbW, and the height of the current block nTbH.

[0029] The output of the IntraTMP is the predictions of the current block, predSamples[x][y], where x=0...nTbW-1, y=0...nTbH-1.

[0030] Specifically, the prediction process of the IntraTMP technique can be divided into four steps: determining the current template type, acquiring the current template reconstruction sample, determining the block vector within a predefined search range, and generating a predicted value. Thus, the predicted value of the current block can be obtained through these steps. The IntraTMP technique may be used to predict either the luma component or the chroma component, and is not specifically limited here.

[0031] 2 is a schematic diagram of a prediction process based on IntraTMP technology. As shown in FIG. 2, the process may include the following steps:

[0032] In S201, the current template type is determined.

[0033] The IntraTMP technique searches for a matching template within a predefined search area using neighboring reconstructed samples of the current block as templates. Here, the neighboring reconstructed samples may be the upper reference sample, upper-left reference sample, upper-right reference sample, left reference sample, and lower-left reference sample of the current block. Therefore, depending on whether neighboring reconstructed samples are available, the template type can be classified and the corresponding template type can be determined.

[0034] The template type may be represented by refTemplateType. Figure 3 shows a schematic diagram of the template types of the IntraTMP technology. As shown in Figure 3, the block filled with a grid is the current block, and the adjacent area of ​​the current block is the template T. Here, six types of template types are shown.

[0035] Illustratively, the six template types are:

[0036] If the upper left reference sample, the upper reference sample, and the left reference sample are all available, the value of refTemplateType is 1 and the template shape is as shown in FIG. 3(a).

[0037] If only the left reference sample is available, the value of refTemplateType is 2 and the template shape is shown in Figure 3(b).

[0038] If only the upper reference sample is available, the value of refTemplateType is 3 and the template shape is as shown in FIG. 3(c).

[0039] If only the left and upper left reference samples are available, the value of refTemplateType is 4 and the template shape is shown in FIG. 3(d).

[0040] If only the left and bottom-left reference samples are available, the value of refTemplateType is 5 and the template shape is shown in FIG. 3(e).

[0041] If only the upper and upper right reference samples are available, the value of refTemplateType is 6 and the template shape is shown in FIG. 3(f).

[0042] In S202, the current template sample is obtained.

[0043] The template for the IntraTMP technique may be composed of reconstructed samples from one or more regions of the upper, upper right, left, lower left, and upper left sides of the current block. The template size may be preset. For example, when acquiring a left template, the template width (templateW_size) may be set to 4, and when acquiring an upper template, the template height (templateH_size) may be set to 4.

[0044] Note that the portion of the reconstructed samples to be acquired may be determined based on the value of refTemplateType. For example, if the value of refTemplateType is 1, reconstructed samples on the left, upper left, and upper sides of the current block are acquired; if the value of refTemplateType is 2, only reconstructed samples in the four columns on the left side of the current block are acquired; or if the value of refTemplateType is 3, only reconstructed samples in the four rows on the top side of the current block are acquired.

[0045] In S203, a block vector is determined within a predefined search range.

[0046] The search process of the IntraTMP technique is mainly divided into an initialization process, specifying the search area within the current frame of the template, and searching and determining the optimal block vector in the search area.

[0047] When searching for an optimal matching template in a search area, a search strategy of performing a coarse search followed by a fine search may be used, or only a fine search may be performed, or only a coarse search may be performed, and there are no particular limitations here.

[0048] In the embodiment of the present application, the coarse search here may specifically determine the optimal coarse matching template in the search area with a first predetermined step width (e.g., 2), or may determine the optimal coarse matching template in the search area with a downsampled template (e.g., a downsampling factor of 2).

[0049] In the embodiment of the present application, the precise search here may specifically determine the optimal precise matching template in the search area with a second predetermined step width (e.g., 1), or may perform a coarse search and then determine the optimal precise matching template in the vicinity of the optimal coarse matching template.

[0050] 4 is a schematic diagram of a search process based on IntraTMP technology according to an embodiment of the present application. As shown in FIG. 4, the process may include the following steps:

[0051] In S401, parameters are initialized.

[0052] Note that uiPatchWidth is initialized to nTbW + templateW_size, and uiPatchHeight is initialized to nTbH + templateH_size. Here, templateW_size and templateH_size may be fixed constants or may be dynamically adjusted according to the size of the current block. Also, templateW_size and templateH_size may or may not be equal. For example, templateW_size = 4, templateH_size, or if the width of the current block is greater than 8, set templateW_size = 4, if the width of the current block is 8 or less, set templateW_size = 2, if the height of the current block is greater than 8, set templateH_size = 4, and if the height of the current block is 8 or less, set templateH_size = 2.

[0053] For example, Figure 5 is a schematic diagram of parameter definitions for the current block and its template. As shown in Figure 5, the specific meanings of the parameters are as follows: nTbW and nTbH represent the size of the current block, templateW_size and templateH_size represent the template size, and uiPatchWidth and uiPatchHeight represent the block sizes of the current block and its template.

[0054] Furthermore, the cost threshold between the initialization templates is denoted by diffThreshold. For example, if the cost function is SAD, the threshold is diffThreshold=((1<<bitDepth)> >2)×(uiPatchHeight×uiPatchWidth−nTbH×nTbW), where diffThreshold indicates that the maximum distortion of each sample point in the template region is 256 when the image bit depth bitDepth is 10.

[0055] Furthermore, the position of the coding tree block CTB where the current block CB is located is initialized to be ctbRsX, ctbRsY.

[0056] Furthermore, the position offsets of the current block CB within the current CTB are initialized so that offsetLCBY=yTbCmp-ctbRsY and offsetLCBX=xTbCmp-ctbRsX.

[0057] Furthermore, initialization is performed so that iTemplateSizeH=templateH_size and iTemplateSizeW=templateW_size.

[0058] Furthermore, iBvShift is initialized. iBvShift is the precision of the block vector BV. For example, the precision of BV may be full sample precision, in which case iBvShift is 0, and the precision of BV may be sub-sample precision. For example, when iBvShift is 1, it indicates 1 / 2 sample precision, and when iBvShift is 2, it indicates 1 / 4 sample precision, but this is not specifically limited here.

[0059] Furthermore, a predetermined search range of the template is initialized. The predetermined search range of the template may be set to a fixed size, or the search range may be dynamically adjusted according to the coding block size. For example, searchRangeWidth=TMP_SEARCH_RANGE_MULT_FACTOR×nTbW, searchRangeHeight=TMP_SEARCH_RANGE_MULT_FACTOR×nTbH. Here, the value of TMP_SEARCH_RANGE_MULT_FACTOR may be a predetermined value, for example, 5.

[0060] In S402, a search area within the current frame of the template is determined.

[0061] The search area of ​​the IntraTMP technique is the reconstructed portion of the current image and is limited by the size of the search range. Figure 6 is a schematic diagram of the template search area. As shown in Figure 6, the background area filled in with dark color is the reconstructed area, the background block filled in with black is the current block, and the dotted frame is the search range window. Therefore, the search area of ​​the IntraTMP technique does not exceed the overlapping area between the reconstructed area shown with the dark background and the area marked with the dotted frame.

[0062] In this way, the search area of ​​the current block template can be the reconstructed part of the CTB where the current block is located, or it can be another reconstructed CTB area. The search area here is actually the set of all search points. Because the shape of the area cannot usually be represented by a single rectangular area, in a specific implementation, searches can be performed in multiple rectangular areas, and the search results of different areas can be combined to obtain the final best-matching block and best-matching block vector.

[0063] For example, refer to FIG. 7. FIG. 7 is a schematic diagram of dividing a search area into different sub-areas. As shown in FIG. 7, eight different sub-area division methods are shown. Here, the background block filled in black is the current block, and the five division methods (a), (b), (c), (d), and (f) all divide the search area into four sub-search areas, and the three division methods (e), (g), and (h) all divide the search area into three sub-search areas. Here, different fill patterns represent one sub-search area.

[0064] In Figure 7, (a), (b), (c), and (d) consider all available search ranges, while (e), (f), (g), and (h) do not search the areas directly above or directly to the left.

[0065] For example, when different sub-search areas are represented by regionId, the template sample of the current block needs to be obtained from the image reconstruction area, and the reconstruction block sample corresponding to the template also needs to be obtained from the reconstruction area. Therefore, it is necessary to further determine searchable positions within the sub-search areas represented by different regionIds based on the position of the current block (xTbCmp, yTbCmp), the size of the current block (nTbW, nTbH), the size of the current image (picWidth, picHeight), the size of the CTB in which the current block is located (CtbSizeW, CtbSizeH), the specified search range of the template (searchRangeWidth, searchRangeHeight), and the position offset of the current block within the current CTB (offsetLCBY, offsetLCBX), and to determine the block vector BV. Specifically, iVerMin represents the absolute coordinate position that can be searched at least in the vertical direction, iVerMax represents the absolute coordinate position that can be searched at most in the vertical direction, iHorMin represents the absolute coordinate position that can be searched at least in the horizontal direction, and iHorMax represents the absolute coordinate position that can be searched at most in the horizontal direction. The values ​​of iVerMin, iVerMax, iHorMin, and iHorMax are different in search regions represented by different regionIds.

[0066] Taking Fig. 7(f) as an example, the search area is divided into four sub-search areas, and the realization method is as follows.

[0067] If regionId is equal to 0, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0068] iHorMax0=min((xTbCmp+searchRangeWidth)< <iBvShift,((picWidth-nTbW)<<iBvShift)) iHorMin0=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iVerMax0=(yTbCmp-nTbH-offsetLCBY)< <iBvShift iVerMin0=max(((iTemplateSizeH)< <iBvShift),((yTbCmp-searchRangeHeight)<<iBvShift)) If regionId is equal to 1, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0069] iHorMin1=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iHorMax1=(xTbCmp-offsetLCBX-nTbW)< <iBvShift iVerMin1=(yTbCmp+1)< <iBvShift iVerMax1=min(picHeight-nTbH,(yTbCmp-offsetLCBY+CtbSizeH-nTbH)< <iBvShift) If regionId is equal to 2, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0070] iHorMax2=(xTbCmp-offsetLCBX-nTbW)< <iBvShift iHorMin2=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iVerMin2=max((iTemplateSizeH)< <iBvShift,(yTbCmp-nTbH-offsetLCBY)<<iBvShift) iVerMax2=(yTbCmp)< <iBvShift If regionId is equal to 3, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0071] iHorMin3=max((iTemplateSizeW)< <iBvShift,(xTbCmp-offsetLCBX-nTbW+1)<<iBvShift) iHorMax3=(xTbCmp-nTbW)< <iBvShift iVerMin3=max(((iTemplateSizeH)< <iBvShift),(yTbCmp-offsetLCBY-nTbH+1)<<iBvShift) iVerMax3=(yTbCmp-nTbH)< <iBvShift In practical application, iHorMin regionId , iHorMax regionId , iVerMin regionId , iVerMax regionId represent the left, right, top and bottom edges of different sub-search areas, respectively.

[0072] To intuitively explain the different sub-search areas corresponding to different regionIds, please refer to Figure 8. Figure 8 is a schematic diagram of a specific search area determination process. As shown in Figure 8, R1, R2, R3, and R4 represent four different sub-search areas. Note that Figure 8 represents the sample range that can align the upper left corner sample of the block.

[0073] In S403, the optimum block vector BV is searched for and determined within the search region.

[0074] The minimum and maximum offsets of the block vector in the horizontal direction are represented by bvXMins and bvXMaxs, respectively, and the minimum and maximum offsets of the block vector in the vertical direction are represented by bvYMins and bvYMaxs, respectively.

[0075] bvXMins regionId , bvXMaxs regionId, bvYMins regionId , bvYMaxs regionId is the iVerMin determined by step S402 as follows: regionId , iVerMax regionId , iHorMin regionId , iHorMax regionId It can be calculated based on:

[0076] bvXMins regionId =iHorMin regionId -xTbCmp bvXMaxs regionId =iHorMax regionId -xTbCmp bvYMins regionId =iVerMin regionId -yTbCmp bvYMaxs regionId =iVerMax regionId -yTbCmp where bvXMins regionId , bvXMaxs regionId , bvYMins regionId and bvYMaxs regionId determines the range of the block vector BV, which is the range of horizontal and vertical shift amounts of the search point relative to the current block.

[0077] Note that the search point (iPosHor, iPoxVer) in each search area, i.e., each block vector BV (consisting of horizontal and vertical components (pX, pY), where pX = iPosHor-xTbCmp and pY = iPosVer-yTbCmp, where pX is between bvXMins and bvXMaxs, and pY is between bvYMins and bvYMaxs), is used to search for a matching reconstruction block of one current block in the reconstruction area, and the adjacent reconstruction sample of the matching reconstruction block is the matching template (i.e., the second template described above). This allows us to calculate the matching cost between the adjacent template of the current block and the adjacent template of the matching reconstruction block, which is denoted as pDiff.

[0078] Furthermore, all search points in all search ranges (regionId=0,1,2,3) are traversed, and the search point with the smallest matching cost pDiff is compared and obtained, and the corresponding matching cost is noted as pDiff_BEST, the corresponding block vector BV is noted as the optimal block vector BV_BEST(pX_BEST, pY_BEST), and the corresponding matching template is the optimal matching template T_BEST.

[0079] In one possible implementation, if the search strategy is to perform only a coarse search, this is specifically performed as follows.

[0080] Within each region, pX is bvXMins regionId and bvXMaxs regionId and pY is between bvYMins regionId and bvYMaxs regionId Within the search range between pX_BEST and pY_BEST, a coarse search is performed with a step size greater than 1, for example, a step size of 2, and the optimal matching cost obtained by template matching is denoted as pDiff_BEST, and the corresponding block vector BV is denoted as optimal block vector BV_BEST(pX_BEST, pY_BEST).

[0081] In another possible embodiment, when the search strategy is to perform only a precision search, it is specifically performed as follows.

[0082] Within each region, pX is bvXMins regionId and bvXMaxs regionId and pY is between bvYMins regionId and bvYMaxs regionId Within the search range between these, a precise search is performed with a step size of 1, for example, and the optimal matching cost obtained by template matching is denoted as pDiff_BEST, and the corresponding block vector BV is denoted as optimal block vector BV_BEST(pX_BEST, pY_BEST).

[0083] In yet another possible embodiment, when the search strategy is to perform a coarse search followed by a fine search, the specific process is as follows, as shown in Figure 9. Figure 9 is a schematic diagram of the search process.

[0084] In S901, the step size is set to 2 and the optimum coarse matching template is determined in the search area.

[0085] In S902, the step size is set to 1, and the optimum fine matching template is determined in the vicinity of the optimum coarse matching template.

[0086] In step S901, in the rough search stage, Within each region, pX is bvXMins regionId and bvXMaxs regionId and pY is between bvYMins regionId and bvYMaxs regionId Within the search range between pX1_BEST and pY1_BEST, a coarse search is performed with a step size greater than 1. For example, a coarse search is performed with a step size of 2, and the optimal matching cost obtained by template matching is recorded as pDiff1_BEST, the corresponding block vector BV is recorded as the optimal block vector BV1_BEST(pX1_BEST, pY1_BEST), and the search region in which the optimal matching search point is located is bestRegionId.

[0087] In addition, step S902 is a precise search stage. Further search is performed in the vicinity of the best block vector BV1_BEST obtained by the coarse search. Specifically, first, a refinement search range TmpRefineRange is determined. The refinement search range may be a fixed size or may be related to the current block size, for example, min(nTbW, nTbH) / 2. Then, the position of the best-matching reconstruction block obtained by the coarse search may be calculated as the reference position of the refinement search area so that BestPosX=xTbCmp+pX1_BEST and BestPosY=yTbCmp+pY1_BEST.

[0088] First, based on the value of bestRegionId, bestRegionId , iVerMax bestRegionId , iHorMin bestRegionId , iHorMax bestRegionId The value of iVerMin is then obtained, and a new search range iVerMin is created based on the optimal matching block position obtained by the coarse search. refine , iVerMax refine , iHorMin refine , iHorMax refine The acquisition method is as follows:

[0089] iHorMin refine =max(iHorMin bestRegionId ,BestPosX-TmpRefineRange) iHorMax refine =min(iHorMax bestRegionId ,BestPosX+TmpRefineRange) iVerMin refine =max(iVerMin bestRegionId ,BestPosY-TmpRefineRange) iVerMax refine =min(iVerMax bestRegionId ,BestPosY+TmpRefineRange) And bvXMins, bvXMaxs, bvYMins, bvYMaxs of the adjusted block vector BV are iVMin refine , iVerMaxrefine , iHorMin refine , iHorMax refine It can be calculated based on:

[0090] bvXMins=iHorMin refine -xTbCmp bvXMaxs=iHorMax refine -xTbCmp bvYMins=iVerMin refine -yTbCmp bvYMaxs=iVerMax refine -yTbCmp Thus, the precision search is performed by determining whether pX is bvXMins refine and bvXMaxs refine and pY is between bvYMins refine and bvYMaxs refine For example, a search is performed with a step size of 1, and the optimal matching cost obtained by template matching is denoted as pDiff_BEST, and the corresponding block vector BV is denoted as the optimal block vector BV_BEST(pX_BEST, pY_BEST).

[0091] After completing the above operations, the best block vector BV_BEST(pX_BEST, pY_BEST) can be obtained, where pX_BEST and pY_BEST are the horizontal and vertical offset amounts of the best matching template relative to the current block template, and are also the horizontal and vertical offset amounts of the best matching reconstruction block relative to the current block.

[0092] In S204, a predicted value is generated.

[0093] Here, this can be achieved by a simple translation copy. The specific operations are as follows:

[0094] For x=0...nTbW-1, y=0...nTbH-1, predSamples[x][y]=recSamples[x+pX_BEST][y+pY_BEST] (2)

[0095] Here, recSamples represents the reconstructed samples of the current frame.

[0096] That is, in the related art, the IntraTMP technology searches for a matching template with the minimum cost within a predefined search range in the current image based on the template of the current block according to a predetermined cost function, and selects the optimal matching reconstruction block (Ref Block) corresponding to the matching template as the prediction block of the current block (Cur Block). The template of the current block can usually select an adjacent reconstruction area of ​​the current block.

[0097] However, in an actual encoding process, the method of directly using the reconstructed samples of the optimal matching reconstructed block as predicted samples of the current block, as in the related art, is not optimal in some cases, and when the light intensity of different regions within a frame differs or the sample noise distribution differs, the difference between the content of the copied reconstructed block and the content of the current region encoding block is large, resulting in reduced prediction accuracy.If the template matching cost function cannot faithfully reflect the difference between the current encoding block template and the matching template in the search process, when the matching template with the lowest cost obtained is used as the optimal matching template, the finally obtained optimal candidate block may not actually be the optimal candidate block, and effective reference information may be limited, ultimately reducing prediction accuracy.

[0098] As described above, general encoding and decoding methods can have problems such as large deviations in predicted values, resulting in low prediction accuracy and failure to achieve optimal prediction results.

[0099] To solve the above problems, embodiments of the present application provide an encoding / decoding method, an encoding device, a decoding device, and a storage medium. The encoding device determines a first template corresponding to a current block, determines one or more block vectors corresponding to the current block based on the first template, determines one or more reference blocks for the current block based on the one or more block vectors, determines a predicted value of the current block based on the one or more reference blocks, and determines a reconstructed value of the current block based on the predicted value of the current block. The decoding device determines a first template corresponding to the current block, determines one or more block vectors corresponding to the current block based on the first template, determines one or more reference blocks for the current block based on the one or more block vectors, and determines a predicted value of the current block based on the one or more reference blocks. Thus, embodiments of the present application provide a prediction method for IntraTMP Fusion, which can select and determine at least one block vector for the current block and further obtain a predicted value of the current block using at least one reference block corresponding to the at least one block vector. In other words, the encoding / decoding method according to the embodiment of the present application fully considers the difference in importance of the reconstruction block information corresponding to different matching templates in the search process to the prediction of the current block, thereby improving the accuracy of prediction and achieving optimal prediction results.

[0100] Hereinafter, each embodiment of the present application will be described in detail with reference to the drawings.

[0101] 10(A) is a block diagram of an encoding device according to an embodiment of the present application. As shown in FIG. 10(A), the encoding device (specifically, a "video encoding device") 100 may include a transform and quantization unit 101, an intra estimation unit 102, an intra prediction unit 103, a motion compensation unit 104, a motion estimation unit 105, an inverse transform and inverse quantization unit 106, a filter control analysis unit 107, a filtering unit 108, an encoding unit 109, and a decoded image buffer unit 110. Here, the filtering unit 108 can realize deblocking filtering and Sample Adaptive Offset (SAO) filtering, and the encoding unit 109 can realize header information coding and Context-based Adaptive Binary Arithmetic Coding (CABAC). The video coding method includes: obtaining a video coding block by dividing a coding tree unit (CTU) for an input original video signal; and then transforming the video coding block by a transform and quantization unit 101 based on residual sample information obtained by intra or inter prediction, specifically, transforming the residual information from the sample domain to the transform domain, and quantizing the resulting transform coefficients to further reduce the bit rate. The intra estimation unit 102 and the intra prediction unit 103 are configured to perform intra prediction on the video coding block. The intra estimation unit 102 and the intra prediction unit 103 are configured to determine an intra prediction mode for encoding the video coding block. The motion compensation unit 104 and the motion estimation unit 105 are configured to perform inter prediction encoding of the received video coding block with respect to one or more blocks in one or more reference frames to provide temporal prediction information.The motion estimation performed by the motion estimation unit 105 is a process of generating a motion vector, based on which the motion of the video coding block can be estimated, and then the motion compensation unit 104 performs motion compensation based on the motion vector determined by the motion estimation unit 105. After determining the intra prediction mode, the intra prediction unit 103 is further configured to provide the selected intra prediction data to the coding unit 109, and the motion estimation unit 105 also sends the calculated motion vector data to the coding unit 109. The inverse transform and inverse quantization unit 106 is also configured to reconstruct the video coding block, reconstructing a residual block in the sample domain, and the reconstructed residual block is subjected to blocking artifact removal by the filter control analysis unit 107 and the filtering unit 108, after which the reconstructed residual block is added to a prediction block in a frame of the decoded image buffer unit 110 to generate a reconstructed video coding block. The encoding unit 109 is configured to encode various coding parameters and quantized transform coefficients, and in a CABAC-based encoding algorithm, the context content may be based on neighboring coding blocks, and may be configured to encode information indicating a determined intra-prediction mode and output a bitstream of the video signal. The decoded picture buffer unit 110 is configured to store reconstructed video coding blocks for reference and prediction. As video image encoding progresses, new reconstructed video coding blocks are constantly generated, and all of these reconstructed video coding blocks are stored in the decoded picture buffer unit 110.

[0102] FIG. 10(B) is a block diagram of a decoding device according to an embodiment of the present application. As shown in FIG. 10(B), a decoding device (specifically, a "video decoding device") 200 includes a decoding unit 201, an inverse transform and inverse quantization unit 202, an intra prediction unit 203, a motion compensation unit 204, a filtering unit 205, and a decoded image buffer unit 206. Here, the decoding unit 201 can decode header information and CABAC decoding, and the filtering unit 205 can deblocking filtering and SAO filtering. After an input video signal undergoes the encoding process of FIG. 10(A), a bitstream of the video signal is output. The bitstream is input to the decoding device 200 and first used to obtain transform coefficients decoded by the decoding unit 201. The transform coefficients are then processed by the inverse transform and inverse quantization unit 202 to generate residual blocks in the sample domain. The intra prediction unit 203 is configured to generate prediction data for the current video decoded block based on the determined intra prediction mode and data from a previously decoded block in the current frame or picture. The motion compensation unit 204 is configured to determine prediction information for the video decoded block by analyzing the motion vectors and other related syntax elements, and generate a prediction block for the video decoded block being decoded based on the prediction information. A decoded video block is formed by adding the residual block from the inverse transform and inverse quantization unit 202 to the corresponding prediction block generated by the intra prediction unit 203 or the motion compensation unit 204. The decoded video signal can be filtered by the filtering unit 205 to remove blocking artifacts, thereby improving video quality. The decoded video block is then stored in the decoded image buffer unit 206, which stores reference images for subsequent intra prediction or motion compensation, and is also used to output a video signal, i.e., obtain a reconstructed original video signal.

[0103] Furthermore, the embodiments of the present application further provide a network architecture of an encoding / decoding system including an encoding device and a decoding device. FIG. 11 is a schematic diagram of the network architecture of the encoding / decoding system according to the embodiments of the present application. As shown in FIG. 11, the network architecture includes one or more electronic devices 13-1N and a communication network 01. Here, the electronic devices 13-1N can perform video interaction via the communication network 01. The electronic devices may be various types of devices having video encoding / decoding functions during the implementation process. For example, the electronic devices may include smartphones, tablet computers, personal computers, personal digital assistants, navigators, digital telephones, videophones, televisions, sensing devices, servers, etc., and the embodiments of the present application are not specifically limited. Here, the decoding device or encoding device according to the embodiments of the present application may be the above-mentioned electronic devices.

[0104] Note that the method according to the embodiment of the present application is mainly applied to the intra prediction unit 103 portion shown in Fig. 10(A) and the intra prediction unit 203 portion shown in Fig. 10(B). That is, the embodiment of the present application may be applied to an encoding device, a decoding device, or both an encoding device and a decoding device, but the embodiment of the present application is not specifically limited.

[0105] When applied to the intra prediction unit 103 portion, the term "current block" specifically refers to the coded block that is currently being intra predicted, and when applied to the intra prediction unit 203 portion, the term "current block" specifically refers to the decoded block that is currently being intra predicted.

[0106] An embodiment of the present application provides a decoding method applied to a decoding device. Figure 12 is a flowchart of the decoding method according to the embodiment of the present application. As shown in Figure 12, the method by which the decoding device performs decoding processing may include the following steps:

[0107] In step 101, a first template corresponding to the current block is determined.

[0108] In an embodiment of the present application, a first template corresponding to the current block may be determined first, where, when obtaining the first template, a template type corresponding to the current block may be determined first, and then the first template corresponding to the current block may be further determined based on the template type.

[0109] In addition, a decoding method according to an embodiment of the present application is applied to a decoding device. The decoding method may include an intra prediction method, more specifically, a color component prediction method. Here, a video image may be divided into multiple decoding blocks, each of which may include a first color component, a second color component, and a third color component. The current block in the embodiment of the present application refers to a decoding block currently performing intra prediction in the video image.

[0110] Here, when a first color component needs to be predicted, the component to be predicted is the first color component; when a second color component needs to be predicted, the component to be predicted is the second color component; and when a third color component needs to be predicted, the component to be predicted is the third color component. Furthermore, if the current block predicts the first color component and the first color component is a luma component, i.e., the component to be predicted is a luma component, the current block may be called a luma block. Alternatively, if the current block predicts the second color component and the second color component is a chroma component, i.e., the component to be predicted is a chroma component, the current block may be called a chroma block.

[0111] In the embodiments of the present application, the reference sample of the current block may refer to a reference sample point adjacent to the current block. Here, adjacent may be, but is not limited to, spatial adjacent. For example, adjacent may be temporal adjacent or spatial-temporal adjacent. Furthermore, the reference sample of the current block may be a reference sample obtained by performing some processing on a spatially adjacent, temporally adjacent, or spatial-temporal adjacent reference sample point, and the embodiments of the present application are not limited thereto.

[0112] Furthermore, in an embodiment of the present application, the template type of the current block can be determined based on the reference samples of the current block, where the reference samples of the current block include at least one of the following: a left-side neighboring reference sample of the current block, an upper-side neighboring reference sample of the current block, an upper-left neighboring reference sample of the current block, a lower-left neighboring reference sample of the current block, and an upper-right neighboring reference sample of the current block.

[0113] It should be noted that in the embodiments of the present application, the reference samples of the current block may include adjacent reconstructed samples of the current block, i.e., the adjacent reconstructed samples of the current block may be used as templates to search for a matching template within a predefined search area.

[0114] It should be noted that in the embodiments of the present application, the reference samples of the current block, that is, the adjacent reconstructed samples of the current block, may include the upper reference sample, the upper left reference sample, the upper right reference sample, the left reference sample, and the lower left reference sample of the current block.

[0115] In addition, in the embodiment of the present application, when the template type of the current block is determined using the reference samples of the current block, the template type can be determined by classifying the template type depending on whether adjacent reference samples are available.

[0116] Furthermore, in an embodiment of the present application, when determining the template type of the current block based on the reference samples of the current block, if the left adjacent reference sample of the current block, the upper adjacent reference sample of the current block, and the upper left adjacent reference sample of the current block are all available, the template type of the current block is determined as a first value. If the left adjacent reference sample of the current block is available, the template type of the current block is determined as a second value. If the upper adjacent reference sample of the current block is available, the template type of the current block is determined as a third value. If the left adjacent reference sample of the current block and the upper left adjacent reference sample of the current block are all available, the template type of the current block is determined as a fourth value. If the left adjacent reference sample of the current block and the lower left adjacent reference sample of the current block are all available, the template type of the current block is determined as a fifth value. If the upper adjacent reference sample of the current block and the upper right adjacent reference sample of the current block are all available, the template type of the current block is determined as a sixth value.

[0117] In the examples of the present application, the first, second, third, fourth, fifth, and sixth values ​​may be any numerical values, and the present application does not specifically limit them. For example, the first, second, third, fourth, fifth, and sixth values ​​are 1, 2, 3, 4, 5, and 6, respectively.

[0118] For example, in the embodiment of the present application, the template type can be represented by refTemplateType, and accordingly, as shown in Figure 3 above, the grid-filled block is the current block, and the adjacent area of ​​the current block is template T, where six types of template types are shown.

[0119] Exemplarily, these six template types are as follows: If the upper-left reference sample, upper reference sample, and left reference sample are all available, the value of refTemplateType is 1, and the template shape is shown in FIG. 3(a). If only the left reference sample is available, the value of refTemplateType is 2, and the template shape is shown in FIG. 3(b). If only the upper reference sample is available, the value of refTemplateType is 3, and the template shape is shown in FIG. 3(c). If only the left reference sample and the upper-left reference sample are available, the value of refTemplateType is 4, and the template shape is shown in FIG. 3(d). If only the left reference sample and the lower-left reference sample are available, the value of refTemplateType is 5, and the template shape is shown in FIG. 3(e). If only the upper reference sample and the upper-right reference sample are available, the value of refTemplateType is 6, and the template shape is shown in FIG. 3(f).

[0120] Furthermore, in an embodiment of the present application, when determining a first template corresponding to a current block based on a template type, a template reference sample of the current block may be determined first based on the template type and a template size corresponding to the template type, and then the first template of the current block may be determined based on the template reference sample.

[0121] In addition, in the embodiment of the present application, the first template of the current block may include a template reference sample of the current block, where the template reference sample of the current block may be determined according to the template type of the current block and the template size corresponding to the template type.

[0122] In addition, in the embodiments of the present application, the first template of the current block may be composed of reconstructed samples of one or more regions of the top, top right, left, bottom left, and top left of the current block, i.e., it may be composed of the reference samples of the current block.

[0123] In the embodiments of the present application, the template size corresponding to the template type may be set in advance. For example, when acquiring a left template, the template width (templateW_size) may be set to 4, and when acquiring a top template, the template height (templateH_size) may be set to 4.

[0124] Correspondingly, in the embodiments of the present application, by combining the value of the template type refTemplateType of the current block and the template size corresponding to this refTemplateType, it is possible to determine which part of the reconstructed sample should be taken as the template reference sample of the current block, and further to determine the corresponding first template.

[0125] For example, in the embodiment of the present application, if the value of refTemplateType is 1, the reconstructed samples on the left, top-left, and top of the current block are selected and acquired; if the value of refTemplateType is 2, only the reconstructed samples in the left four columns of the current block are acquired; if the value of refTemplateType is 3, only the reconstructed samples in the top four rows of the current coding block are acquired.

[0126] Of course, the preset template size value is not limited to 4, and may be any integer greater than 0, and the present application does not specifically limit it.

[0127] It should be noted that in the embodiment of the present application, the template reference sample of the current block determined from the reference sample of the current block based on the template type of the current block and the corresponding template size can be the first template corresponding to the current block.

[0128] In step 102, one or more block vectors corresponding to the current block are determined based on the first template.

[0129] In an embodiment of the present application, after determining the first template corresponding to the current block, one or more block vectors corresponding to the current block may be further determined based on the first template.

[0130] In addition, in the embodiment of the present application, the block vector search process includes an initialization process, determining a search area within the current frame of the first template, and searching for and determining one or more optimal block vectors in the search area. Therefore, when performing the search process, the initialization operation must first be completed.

[0131] Exemplarily, as shown in FIG. 5 above, nTbW and nTbH represent the size of the current block, templateW_size and templateH_size represent the template size, and uiPatchWidth and uiPatchHeight represent the block size including the current block and its template.

[0132] Correspondingly, in the initialization process, uiPatchWidth can be initialized to nTbW+templateW_size, and uiPatchHeight can be initialized to nTbH+templateH_size. Here, templateW_size and templateH_size may be fixed constants or may be dynamically adjusted based on the coding block size, and templateW_size and templateH_size may or may not be equal. For example, templateW_size=4, templateH_size=4, or templateW_size=4 is set when the width of the coding block is greater than 8, templateW_size=2 is set when the width of the coding block is 8 or less, templateH_size=4 is set when the height of the coding block is greater than 8, and templateH_size=2 is set when the height of the coding block is 8 or less.

[0133] Furthermore, the cost threshold between the initialization templates is denoted by diffThreshold. For example, if the cost function is SAD, the threshold is diffThreshold=((1<<bitDepth)> >2)×(uiPatchHeight×uiPatchWidth−nTbH×nTbW), where diffThreshold indicates that the maximum distortion of each sample point in the template region is 256 when the image bit depth bitDepth is 10.

[0134] Furthermore, the position of the coding tree block CTB where the current block CB is located is initialized to be ctbRsX, ctbRsY.

[0135] Furthermore, the position offsets of the current block CB within the current CTB are initialized so that offsetLCBY=yTbCmp-ctbRsY and offsetLCBX=xTbCmp-ctbRsX.

[0136] Furthermore, initialization is performed so that iTemplateSizeH=templateH_size and iTemplateSizeW=templateW_size.

[0137] Furthermore, iBvShift is initialized. iBvShift is the precision of the block vector BV. For example, the precision of BV may be full sample precision, in which case iBvShift is 0, or the precision of BV may be sub-sample precision, for example, when iBvShift is 1, it indicates 1 / 2 sample precision, and when iBvShift is 2, it indicates 1 / 4 sample precision, but this is not specifically limited here.

[0138] Furthermore, a predetermined search range of the template is initialized. The predetermined search range of the template may be set to a fixed size, or the search range may be dynamically adjusted according to the coding block size. For example, searchRangeWidth=TMP_SEARCH_RANGE_MULT_FACTOR×nTbW, searchRangeHeight=TMP_SEARCH_RANGE_MULT_FACTOR×nTbH. Here, the value of TMP_SEARCH_RANGE_MULT_FACTOR may be a predetermined value, for example, 5.

[0139] Furthermore, in an embodiment of the present application, when determining one or more block vectors corresponding to a current block based on a first template, a predetermined search area may be determined based on the first template first, and then one or more block vectors may be determined by searching within the predetermined search area.

[0140] In the embodiment of the present application, the search area is the reconstructed portion of the current image and is limited by the size of the search range. As shown in Figure 6, the background area filled with dark color is the reconstructed area, the background block filled with black is the current block, and the dotted frame is the search range window. Therefore, the search area of ​​the IntraTMP technique does not exceed the overlapping area between the reconstructed area shown with the dark background and the area marked with the dotted frame.

[0141] In this way, the search area of ​​the current block template can be the reconstructed part of the CTB where the current block is located, or it can be another reconstructed CTB area. The search area here is actually the set of all search points. Because the shape of the area cannot usually be represented by a single rectangular area, in a specific implementation, searches can be performed in multiple rectangular areas, and the search results of different areas can be combined to obtain the final best-matching block and best-matching block vector.

[0142] For example, refer to Figure 7, which shows eight different sub-area division methods. Here, the black background block is the current block. Five division methods (a), (b), (c), (d), and (f) all divide the search area into four sub-search areas, and three division methods (e), (g), and (h) all divide the search area into three sub-search areas. Here, different fill patterns represent one sub-search area.

[0143] In Figure 7, (a), (b), (c), and (d) consider all available search ranges, while (e), (f), (g), and (h) do not search the areas directly above or directly to the left.

[0144] For example, when different sub-search areas are represented by regionId, the template sample of the current coding block needs to be obtained from the image reconstruction area, and the reconstruction block sample corresponding to the template also needs to be obtained from the reconstruction area. Therefore, it is necessary to further determine searchable positions within the sub-search areas represented by different regionIds based on the position of the current coding block (xTbCmp, yTbCmp), the size of the current coding block (nTbW, nTbH), the size of the current image (picWidth, picHeight), the size of the CTB in which the current block is located (CtbSizeW, CtbSizeH), the specified search range of the template (searchRangeWidth, searchRangeHeight), and the position offset of the current coding block within the current CTB (offsetLCBY, offsetLCBX), and to determine the block vector BV. Specifically, iVerMin represents the absolute coordinate position that can be searched at least in the vertical direction, iVerMax represents the absolute coordinate position that can be searched at most in the vertical direction, iHorMin represents the absolute coordinate position that can be searched at least in the horizontal direction, and iHorMax represents the absolute coordinate position that can be searched at most in the horizontal direction. The values ​​of iVerMin, iVerMax, iHorMin, and iHorMax are different in search regions represented by different regionIds.

[0145] In some embodiments, taking FIG. 7(f) as an example, the search area is divided into four sub-search areas, which are implemented as follows:

[0146] If regionId is equal to 0, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0147] iHorMax0=min((xTbCmp+searchRangeWidth)< <iBvShift,((picWidth-nTbW)<<iBvShift)) iHorMin0=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iVerMax0=(yTbCmp-nTbH-offsetLCBY)< <iBvShift iVerMin0=max(((iTemplateSizeH)< <iBvShift),((yTbCmp-searchRangeHeight)<<iBvShift)) If regionId is equal to 1, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0148] iHorMin1=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iHorMax1=(xTbCmp-offsetLCBX-nTbW)< <iBvShift iVerMin1=(yTbCmp+1)< <iBvShift iVerMax1=min(picHeight-nTbH,(yTbCmp-offsetLCBY+CtbSizeH-nTbH)< <iBvShift) If regionId is equal to 2, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0149] iHorMax2=(xTbCmp-offsetLCBX-nTbW)< <iBvShift iHorMin2=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iVerMin2=max((iTemplateSizeH)< <iBvShift,(yTbCmp-nTbH-offsetLCBY)<<iBvShift) iVerMax2=(yTbCmp)< <iBvShift If regionId is equal to 3, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0150] iHorMin3=max((iTemplateSizeW)< <iBvShift,(xTbCmp-offsetLCBX-nTbW+1)<<iBvShift) iHorMax3=(xTbCmp-nTbW)< <iBvShift iVerMin3=max(((iTemplateSizeH)< <iBvShift),(yTbCmp-offsetLCBY-nTbH+1)<<iBvShift) iVerMax3=(yTbCmp-nTbH)< <iBvShift In practical application, iHorMin regionId , iHorMax regionId , iVerMin regionId , iVerMax regionId represent the left, right, top and bottom edges of different sub-search areas, respectively.

[0151] To intuitively explain the different sub-search areas corresponding to different regionIds, R1, R2, R3, and R4 represent four different sub-search areas as shown in Figure 8. Note that Figure 8 represents the sample range in which the upper left corner sample of the block can be aligned.

[0152] In some embodiments, taking FIG. 7(a) as an example, the search area is divided into four sub-search areas, and the implementation is as follows:

[0153] If regionId is equal to 0, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0154] iHorMax0=min((xTbCmp+searchRangeWidth)< <iBvShift,((picWidth-nTbW)<<iBvShift)) iHorMin0=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iVerMax0=(yTbCmp-nTbH-offsetLCBY)< <iBvShift iVerMin0=max(((iTemplateSizeH)< <iBvShift),((yTbCmp-searchRangeHeight)<<iBvShift)) If regionId is equal to 1, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0155] iHorMin1=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iHorMax1=(xTbCmp-offsetLCBX-nTbW)< <iBvShift iVerMin1=(yTbCmp+1)< <iBvShift iVerMax1=min(picHeight-nTbH,(yTbCmp-offsetLCBY+CtbSizeH-nTbH)< <iBvShift) If regionId is equal to 2, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0156] iHorMax2=(xTbCmp-nTbW)< <iBvShift iHorMin2=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iVerMin2=max((iTemplateSizeH)< <iBvShift,(yTbCmp-nTbH)<<iBvShift) iVerMax2=(yTbCmp)< <iBvShift If regionId is equal to 3, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0157] iHorMin3=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iHorMax3=(xTbCmp)< <iBvShift iVerMin3=max(((iTemplateSizeH)< <iBvShift),(yTbCmp-offsetLCBY-nTbH+1)<<iBvShift) iVerMax3=(yTbCmp-nTbH)< <iBvShift In practical application, iHorMin regionId , iHorMax regionId , iVerMin regionId , iVerMax regionId represent the left, right, top and bottom edges of different sub-search areas, respectively.

[0158] To intuitively explain the different sub-search areas corresponding to different regionIds, R1, R2, R3, and R4 denote four different sub-search areas as shown in Figure 8. Note that Figure 8 represents the sample range in which the upper left corner sample of the block can be aligned.

[0159] In some embodiments, taking FIG. 7(b) as an example, the search area is divided into four sub-search areas, which are implemented as follows:

[0160] If regionId is equal to 0, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0161] iHorMax0=min((xTbCmp+searchRangeWidth)< <iBvShift,((picWidth-nTbW)<<iBvShift)) iHorMin0=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iVerMax0=(yTbCmp-nTbH-offsetLCBY)< <iBvShift iVerMin0=max(((iTemplateSizeH)< <iBvShift),((yTbCmp-searchRangeHeight)<<iBvShift)) If regionId is equal to 1, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0162] iHorMin1=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iHorMax1=(xTbCmp-offsetLCBX-nTbW)< <iBvShift iVerMin1=(yTbCmp+1)< <iBvShift iVerMax1=min(picHeight-nTbH,(yTbCmp-offsetLCBY+CtbSizeH-nTbH)< <iBvShift) If regionId is equal to 2, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0163] iHorMax2=(xTbCmp-nTbW)< <iBvShift iHorMin2=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iVerMin2=max((iTemplateSizeH)< <iBvShift,(yTbCmp-nTbH-offsetLCBY)<<iBvShift) iVerMax2=(yTbCmp)< <iBvShift If regionId is equal to 3, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0164] iHorMin3=max((iTemplateSizeW)< <iBvShift,(xTbCmp-nTbW+1)<<iBvShift) iHorMax3=(xTbCmp)< <iBvShift iVerMin3=max(((iTemplateSizeH)< <iBvShift),(yTbCmp-offsetLCBY-nTbH+1)<<iBvShift) iVerMax3=(yTbCmp-nTbH)< <iBvShift In practical application, iHorMin regionId , iHorMax regionId , iVerMin regionId , iVerMax regionId represent the left, right, top and bottom edges of different sub-search areas, respectively.

[0165] To intuitively explain the different sub-search areas corresponding to different regionIds, please refer to Figure 13. Figure 13 shows a schematic diagram of a specific search area determination process. As shown in Figure 13, R1, R2, R3, and R4 represent four different sub-search areas. Note that Figure 13 represents the sample range that can align the upper left corner sample of the block.

[0166] Furthermore, in the embodiments of the present application, when searching within a predetermined search area to determine one or more block vectors, the search points in the predetermined search area may be traversed, and a matching cost between the matching template corresponding to the search points in the predetermined search area and the first template may be determined based on a predetermined matching criterion, and one or more block vectors and one or more candidate templates corresponding to the one or more block vectors may be determined based on the matching cost.

[0167] It should be noted that in the embodiments of the present application, the number of block vectors determined by the search process may be one or more, for example, N block vectors of the current block may be determined, where N is an integer greater than 0.

[0168] Accordingly, in the embodiments of the present application, when searching within a predetermined search area to determine one or more block vectors, a predetermined number N corresponding to candidate templates may be first determined, and then the search points in the predetermined search area may be traversed, and the matching cost between the matching template corresponding to the search points in the predetermined search area and the first template may be determined based on a predetermined matching criterion; and finally, N block vectors and N candidate templates corresponding to the N block vectors may be determined based on the matching cost.

[0169] That is, in an embodiment of the present application, when searching for and determining N block vectors corresponding to N matching templates in a predetermined search area, i.e., the process of searching for and determining block vectors BV corresponding to N matching templates in a search area (predetermined search area), may include determining the value of the number N of candidate templates, determining a matching template comparison criterion, and recording N block vectors BV corresponding to the N matching templates (selected N candidate templates).

[0170] In addition, in the embodiments of the present application, when determining the predetermined number N corresponding to the candidate templates, N may be determined by decoding the bitstream, N may be determined based on a first predetermined value, or N may be determined based on a predetermined value range.

[0171] That is, in the embodiment of the present application, the value of N must first be determined. Here, N may be artificially preset to a certain constant, for example, N=4. N may be within a predetermined numerical range, for example, N may be any integer in [2, 8]. The range of N may be preset, and the encoding side may determine N according to a predetermined strategy criterion. For example, the optimal N value may be determined using a method such as coarse selection based on cost 1, coarse selection based on cost 2, coarse selection based on cost 3, or fine selection based on cost 4, and the optimal N value may be transmitted to the decoding side as a bitstream. The predetermined strategy criterion, which includes each of costs 1, 2, 3, and 4, may be one of cost functions for evaluating modes, such as SAD, SATD, MSE, MAD, and RDO. The present application does not particularly limit the method for determining N.

[0172] It should be noted that in the embodiment of the present application, the predetermined matching criterion includes any one of cost functions for evaluating modes, such as sum of absolute differences SAD, sum of absolute transformed differences SATD, sum of squared errors SSE, mean absolute deviation MAD, mean absolute error MAE, mean squared error MSE, normalized correlation coefficient NCC, etc.

[0173] Furthermore, in the embodiments of the present application, when determining N block vectors and N candidate templates corresponding to the N block vectors based on matching costs, i.e., when determining one or more block vectors and one or more candidate templates corresponding to one or more block vectors, N minimum matching costs may be first determined from the matching costs between the matching templates corresponding to search points in a predetermined search area and the first template, and then N block vectors and N candidate templates corresponding to the N minimum matching costs may be determined.

[0174] In the embodiments of the present application, for any one of the N block vectors, bvXMins and bvXMaxs may represent the minimum and maximum horizontal shifts of the block vector, respectively, and bvYMins and bvYMaxs may represent the minimum and maximum vertical shifts of the block vector, respectively.

[0175] bvXMinsregionId, bvXMaxsregionId, bvYMinsregionId, bvYMaxsregionId can be calculated based on the determined iVerMinregionId, iVerMaxregionId, iHorMinregionId, iHorMaxregionId as follows:

[0176] bvXMinsregionId=iHorMinregionId-xTbCmp bvXMaxsregionId=iHorMaxregionId-xTbCmp bvYMinsregionId=iVerMinregionId-yTbCmp bvYMaxsregionId=iVerMaxregionId-yTbCmp bvXMinsregionId, bvXMaxsregionId, bvYMinsregionId, and bvYMaxsregionId determine the range of the block vector BV, which is the range of horizontal and vertical shift amounts for the search point of the current block.

[0177] Note that the search points (iPosHor, iPoxVer) in each search area, i.e., each block vector BV (consisting of horizontal and vertical components (pX, pY), where pX = iPosHor-xTbCmp and pY = iPosVer-yTbCmp, where pX is between bvXMins and bvXMaxs, and pY is between bvYMins and bvYMaxs), can be used to search for one or more matching reconstruction blocks of the current block in the reconstruction area, and the adjacent reconstruction samples of the one or more matching reconstruction blocks are matching templates. This allows the matching costs between the current block adjacent templates and one or more reconstruction block adjacent templates to be calculated, which is denoted as pDiff.

[0178] Furthermore, traverse all search points in all search ranges (regionId=0,1,2,3), compare and obtain one or more search points with the smallest matching cost pDiff, denote the corresponding matching cost as pDiff_BEST, denote one or more corresponding block vectors BV as optimal block vectors BV_BEST(pX_BEST, pY_BEST), and obtain one or more corresponding matching templates as optimal matching templates T_BEST, i.e., obtain one or more candidate templates.

[0179] In the embodiment of the present application, after determining the value of the number N of candidate templates, it is necessary to select N block vectors BV corresponding to the N candidate templates that are relatively matched according to a certain comparison criterion. That is, compared with the related art that searches for a general matching template and records the block vectors BV, the difference of the present application is that multiple block vectors BV n Select and record the block vector BV n , i.e., the template shift amount pX n and pY nThe method involves obtaining a matching template based on

[0000] , calculating a template matching cost, and recording N BVs corresponding to the N matching templates with the smallest costs. Here, the N matching templates are referred to as N candidate templates. Here, the template matching cost (predetermined matching criterion) may be one of cost functions for evaluating modes, such as SAD, SATD, MSE, MAD, RDO, correlation coefficient, etc.

[0180] For example, in the embodiment of the present application, if the matching cost comparison criterion (predetermined matching criterion) is Mean Absolute Difference (MAD), the calculation formula is as follows:

number

[0181] where refT is the matching template in the search process, curT is the current coding block template (the first template of the current block), M is the number of samples in the current coding block template, and MAD(refT) is the mean absolute deviation between the current coding block template curT and the searched matching template.

[0182] Accordingly, in the embodiment of the present application, the selection criterion by MAD is to compare and record the N block vectors BV corresponding to the matching templates with the N smallest MAD costs.

[0183] Exemplarily, for N candidate templates, the MAD between the n-th candidate template and the current coding block template is:

number

[0184] where refT nis the nth candidate template, and MAD(refT n ) is the mean absolute deviation between the current coding block template curT and the n-th candidate template, where n=0,...,N-1.

[0185] Illustratively, in an embodiment of the present application, when the matching cost comparison criterion (predetermined matching criterion) is SAD, the calculation formula is as follows:

number

[0186] Here, SAD(refT) is the sum of absolute differences between the current coding block template (first template of the current block) curT and the searched matching template.

[0187] Correspondingly, in the embodiment of the present application, the selection criterion by SAD is to compare and record N block vectors BV corresponding to N matching templates with relatively small SAD costs.

[0188] Exemplarily, for N candidate templates, the SAD between the n-th candidate template and the current coding block template is:

number

[0189] where SAD(refT n ) is the sum of absolute differences between the current coding block template and the n-th candidate template.

[0190] Illustratively, in the embodiment of the present application, the NCC normalized correlation coefficient can be compared as the template matching criterion, and the calculation formula is as follows:

number

[0191] where refT is the matching template in the search process, curT is the current coding block template (the first template of the current block), M is the number of samples in the current coding block template, and refT Avg is the sample mean value of the searched matching template, and curT Avg is the sample mean value of the current coding block template, and R(refT) is the correlation coefficient between the current coding block template and the searched matching template.

[0192] Accordingly, in the embodiment of the present application, the comparison criterion by NCC is to sort and record the N block vectors BV corresponding to the matching templates with the N largest correlation coefficients R.

[0193] For N candidate templates, the correlation coefficient between the nth candidate template and the current coding block template is:

number

[0194] where refT nAvg is the sample mean of the nth candidate template, and R(refT n ) is the correlation coefficient between the current coding block template and the n-th candidate template. The range of the NCC normalized correlation coefficient R is [-1, 1], and the larger R indicates the stronger the correlation.

[0195] In the embodiment of the present application, when performing a search process, available search strategies may include, but are not limited to, a search method based on different search step sizes, such as a coarse search based on a first search step size and / or a precise search based on a second search step size, where the first search step size is larger than the second search step size.

[0196] Furthermore, in the embodiment of the present application, the block vectors and candidate templates may be determined by traversing the search points in the predetermined search area according to a first search step width, or by traversing the search points in the predetermined search area according to a second search step width.

[0197] Furthermore, in the embodiment of the present application, first, the search points in the predetermined search area are traversed according to a first search step width to determine an initial block vector and an initial matching template corresponding to the initial block vector; then, a first search area is determined based on the initial matching template, where the first search area is smaller than the predetermined search area; finally, the search points in the first search area are traversed according to a second search step width to determine a block vector and a candidate template, where the first search step width is larger than the second search step width.

[0198] That is, in the embodiments of the present application, when searching for the optimum matching template in the search area, a search strategy of performing a coarse search followed by a fine search may be adopted, or only a fine search may be performed, or only a coarse search may be performed.

[0199] Exemplarily, in the embodiments of the present application, the coarse search may specifically include determining an optimal coarse matching template in the search area according to a first predetermined step width (i.e., a first search step width, for example, 2), i.e., obtaining a final candidate template, or determining an optimal coarse matching template in the search area by a down-sampled template (for example, the down-sampling factor is 2), i.e., obtaining a final candidate template.

[0200] Illustratively, in the embodiments of the present application, the precise search may specifically include determining an optimal precise matching template in the search area according to a second predetermined step width (i.e., a second search step width, for example, 1), i.e., obtaining a final candidate template, or performing a coarse search and then determining an optimal precise matching template in the vicinity of the optimal coarse matching template, i.e., obtaining a final candidate template.

[0201] For example, in an embodiment of the present application, if the search strategy is to perform only a coarse search, a coarse search is performed within the search range in each region where pX is between bvXMinsregionId and bvXMaxsregionId and pY is between bvYMinsregionId and bvYMaxsregionId, with a step size greater than 1, for example, with a step size of 2 (i.e., the first search step size is 2), and the optimal matching cost or costs obtained by template matching are recorded as pDiff_BEST, and the corresponding block vector or vectors BV are noted as optimal block vectors BV_BEST(pX_BEST, pY_BEST). That is, by performing a search process in a predetermined search region, one or more block vectors corresponding to the current block are obtained, and the matching template or templates corresponding to the optimal matching cost or costs may be the final candidate template or templates.

[0202] For example, in an embodiment of the present application, if the search strategy is to perform only a precise search, a precise search is performed within the search range in each region where pX is between bvXMinsregionId and bvXMaxsregionId and pY is between bvYMinsregionId and bvYMaxsregionId, for example, with a step size of 1 (i.e., the second search step size is 1), and one or more optimal matching costs obtained by template matching are recorded as pDiff_BEST, and one or more corresponding block vectors BV are noted as optimal block vectors BV_BEST(pX_BEST, pY_BEST). That is, by performing a search process in a predetermined search region, one or more block vectors corresponding to the current block are obtained, and one or more matching templates corresponding to the one or more optimal matching costs may be the final one or more candidate templates.

[0203] For example, in an embodiment of the present application, when the search strategy is to perform a coarse search followed by a fine search, as shown in FIG. 10, first, a coarse search is performed with a step size of 2 (i.e., the first search step size is 2), and an optimal coarse matching template (initial matching template) obtained by template matching is obtained; then, an optimal fine matching template is determined in the vicinity of the optimal coarse matching template with a step size of 1 (i.e., the second search step size is 1), and thus a final candidate template can be obtained.

[0204] Here, in the coarse search stage, within the search range in each region where pX is between bvXMinsregionId and bvXMaxsregionId and pY is between bvYMinsregionId and bvYMaxsregionId, a coarse search is performed with a step size greater than 1. For example, a coarse search is performed with a step size of 2, and the optimal matching cost obtained by template matching is recorded as pDiff1_BEST. The corresponding block vector BV is recorded as an initial block vector, which is the optimal block vector BV1_BEST(pX1_BEST, pY1_BEST), and the corresponding matching template is the initial matching template. In this case, the search region where the optimal matching search point is located is bestRegionId.

[0205] Next, in the fine search stage, a search may be performed near the optimal block vector BV1_BEST (initial block vector) obtained by the coarse search, i.e., a search may be performed in the first search area. Therefore, it is first necessary to determine a refined search range TmpRefineRange, i.e., a first search area TmpRefineRange. Here, the refined search range (first search area TmpRefineRange) may be a fixed size or may be related to the current block size, for example, set to min(nTbW, nTbH) / 2. Furthermore, the position of the optimal matching reconstruction block obtained by the coarse search may be calculated as the reference position of the fine search area based on BestPosX = xTbCmp + pX1_BEST and BestPosY = yTbCmp + pY1_BEST.

[0206] In one embodiment, based on the value of bestRegionId, first obtain the calculated values ​​of iVerMinbestRegionId, iVerMaxbestRegionId, iHorMinbestRegionId, and iHorMaxbestRegionId, and then obtain new search ranges iVerMinrefine, iVerMaxrefine, iHorMinrefine, and iHorMaxrefine based on the best matching block position obtained by coarse search. The obtaining method is as follows:

[0207] iHorMinrefine=max(iHorMinbestRegionId,BestPosX-TmpRefineRange) iHorMaxrefine=min(iHorMaxbestRegionId,BestPosX+TmpRefineRange) iVerMinrefine=max(iVerMinbestRegionId,BestPosY-TmpRefineRange) iVerMaxrefine=min(iVerMaxbestRegionId,BestPosY+TmpRefineRange) Then, bvXMins, bvXMaxs, bvYMins, and bvYMaxs of the adjusted block vector BV can be calculated based on iVerMinrefine, iVerMaxrefine, iHorMinrefine, and iHorMaxrefine as follows:

[0208] bvXMins=iHorMinrefine-xTbCmp bvXMaxs=iHorMaxrefine-xTbCmp bvYMins=iVerMinrefine-yTbCmp bvYMaxs=iVerMaxrefine-yTbCmp The refined search is performed within the block vector range where pX is between bvXMinsrefine and bvXMaxsrefine and pY is between bvYMinsrefine and bvYMaxsrefine. For example, the search is performed with a step size of 1, and the optimal matching cost obtained by template matching is recorded as pDiff_BEST. The corresponding block vector BV is recorded as the optimal block vector BV_BEST(pX_BEST, pY_BEST), i.e., the block vector of the finally determined current block, and the corresponding matching template is the candidate template for the current block.

[0209] After completing the above operations, the best block vector BV_BEST(pX_BEST, pY_BEST) is obtained, where pX_BEST and pY_BEST are the horizontal and vertical offset amounts of the best matching template relative to the current coding block template, respectively, that is, the horizontal and vertical offset amounts of the best matching reconstruction block relative to the current coding block.

[0210] Thus, in the embodiment of the present application, the process of searching and determining one or more block vectors in the search area may include a process of performing only a coarse search, a process of performing only a fine search, or a process of performing a fine search after the coarse search. Accordingly, if the search strategy is to perform only a coarse search, N block vectors BV corresponding to N matching templates are determined only in the coarse search process, and the corresponding N matching templates are set as N candidate templates. If the search strategy is to perform only a fine search, N block vectors BV corresponding to N matching templates are determined only in the fine search process, and the corresponding N matching templates are set as N candidate templates. If the search strategy is to perform a fine search after the coarse search, K block vectors BV corresponding to K (K is an integer greater than or equal to N) preliminary matching templates (initial matching templates) may be first determined in the coarse search process. After performing a fine search process based on the K preliminary matching templates, N block vectors BV corresponding to the final N matching templates are determined, and the corresponding N matching templates are set as N candidate templates.

[0211] Furthermore, in an embodiment of the present application, when performing a search process, the search points in a sub-search area of ​​a predetermined search area may be traversed according to a first search step width to determine an initial vector corresponding to the sub-search area and a second search area corresponding to an initial block vector, multiple target sub-search areas may be determined in the sub-search area based on the initial block vector and the second search area, and the search points in the multiple target sub-search areas may be traversed according to the second search step width to determine block vectors and candidate templates, wherein the first search step width is greater than the second search step width.

[0212] In addition, in the embodiment of the present application, when determining multiple target sub-search areas in a sub-search area based on an initial block vector and a second search area, area determination can be performed based on the initial block vector, which is the optimal block vector of each sub-search area, and the corresponding second search area, to determine multiple target sub-search areas to be used in the subsequent search process.

[0213] That is, in the embodiment of the present application, in the search strategy of performing a coarse search followed by a fine search, a fine search may be performed across different search regions, and after completing the coarse search process, a method relating to each region boundary may be performed in the fine search process.

[0214] For example, in the embodiment of the present application, taking FIG. 7(a) as an example, the search area is divided into four sub-search areas, and the implementation method is as follows:

[0215] If regionId is equal to 0, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0216] iHorMax0=min((xTbCmp+searchRangeWidth)< <iBvShift,((picWidth-nTbW)<<iBvShift)) iHorMin0=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iVerMax0=(yTbCmp-nTbH-offsetLCBY)< <iBvShift iVerMin0=max(((iTemplateSizeH)< <iBvShift),((yTbCmp-searchRangeHeight)<<iBvShift)) If regionId is equal to 1, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0217] iHorMin1=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iHorMax1=(xTbCmp-offsetLCBX-nTbW)< <iBvShift iVerMin1=(yTbCmp+1)< <iBvShift iVerMax1=min(picHeight-nTbH,(yTbCmp-offsetLCBY+CtbSizeH-nTbH)< <iBvShift) If regionId is equal to 2, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0218] iHorMax2=(xTbCmp-nTbW)< <iBvShift iHorMin2=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iVerMin2=max((iTemplateSizeH)< <iBvShift,(yTbCmp-nTbH)<<iBvShift) iVerMax2=(yTbCmp)< <iBvShift If regionId is equal to 3, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0219] iHorMin3=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iHorMax3=(xTbCmp)< <iBvShift iVerMin3=max(((iTemplateSizeH)< <iBvShift),(yTbCmp-offsetLCBY-nTbH+1)<<iBvShift) iVerMax3=(yTbCmp-nTbH)< <iBvShift In practical application, iHorMin regionId , iHorMax regionId , iVerMin regionId , iVerMax regionId represent the left, right, top and bottom edges of different sub-search areas, respectively.

[0220] To intuitively explain the different sub-search areas corresponding to different regionIds, please refer to Figure 14. Figure 14 shows a schematic diagram of a specific search area determination process. As shown in Figure 14, R1, R2, R3, and R4 represent four different sub-search areas. Note that Figure 14 represents the sample range that can align the upper left corner sample of the block.

[0221] In the refinement stage, the neighborhood of each block vector obtained by the coarse search may be searched, and further, the neighborhood of the optimal block vector BV1_BESTk obtained by the coarse search may be searched. Specifically, first, a refinement search range TmpRefineRange is determined. The refinement search range may be a fixed size or may be related to the size of the current block, for example, set to min(nTbW, nTbH) / 2. Then, the position of the optimal matching reconstruction block obtained by the coarse search may be calculated as the reference position of the refinement search area so that BestPosXk=xTbCmp+pX1_BESTk and BestPosYk=yTbCmp+pY1_BESTk.

[0222] In some embodiments, the refined search performed based on each BestPosXk may span multiple regions, i.e., may span different search regions. Here, first, the value of bestRegionId involved in the calculation is determined, and then the calculation is performed for regionId=0, 1, 2, 3 as follows:

[0223] BestPosYk-TmpRefineRange>=iVerMinregionId and BestPosYk-TmpRefineRange<=iVerMaxregionId, or BestPosYk+TmpRefineRange>=iVerMinregionId and BestPosYk+TmpRefineRange<=iVerMaxregionId, or If BestPosYk-TmpRefineRange<=iVerMinregionId and BestPosYk+TmpRefineRange>=iVerMaxregionId, For that regionId, set the corresponding bestSearchFlagregionId as 1.

[0224] For each search region whose bestSearchFlagregionId value is 1, bestRegionId is set as the regionId of the region in turn, and the following refined search is further performed: First, the calculated values ​​of iVerMinbestRegionId, iVerMaxbestRegionId, iHorMinbestRegionId, and iHorMaxbestRegionId are obtained based on the value of bestRegionId, and then new search ranges iVerMinrefine, iVerMaxrefine, iHorMinrefine, and iHorMaxrefine are obtained based on the optimal matching block position obtained by the coarse search. The method of obtaining these values ​​is as follows:

[0225] iHorMinrefine=max(iHorMinbestRegionId,BestPosX-TmpRefineRange) iHorMaxrefine=min(iHorMaxbestRegionId,BestPosX+TmpRefineRange) iVerMinrefine=max(iVerMinbestRegionId,BestPosY-TmpRefineRange) iVerMaxrefine=min(iVerMaxbestRegionId,BestPosY+TmpRefineRange).

[0226] Then, bvXMins, bvXMaxs, bvYMins, and bvYMaxs of the adjusted block vector BV can be calculated based on iVerMinrefine, iVerMaxrefine, iHorMinrefine, and iHorMaxrefine as follows:

[0227] bvXMins=iHorMinrefine-xTbCmp bvXMaxs=iHorMaxrefine-xTbCmp bvYMins=iVerMinrefine-yTbCmp bvYMaxs=iVerMaxrefine-yTbCmp.

[0228] In this way, the refined search is performed within the block vector range where pX is between bvXMinsrefine and bvXMaxsrefine and pY is between bvYMinsrefine and bvYMaxsrefine. For example, a search is performed with a step size of 1, and the optimal matching cost obtained by template matching is recorded as pDiff_BEST, and the corresponding block vector BV is written as the optimal block vector BV_BEST(pX_BEST, pY_BEST).

[0229] Obtain the sum of the optimal block vectors for the multiple regions involved in the search.

[0230] After completing the above search operation, the best block vector BV_BEST(pX_BEST, pY_BEST) can be obtained, where pX_BEST and pY_BEST are respectively the horizontal offset and vertical offset of the best matching template relative to the current coding block template, and are also the horizontal offset and vertical offset of the best matching reconstruction block relative to the current coding block.

[0231] In step 103, one or more reference blocks of the current block are determined based on the one or more block vectors, and a predicted value of the current block is determined based on the one or more reference blocks.

[0232] In an embodiment of the present application, after determining one or more block vectors corresponding to a current block based on a first template, one or more reference blocks for the current block may be determined based on the one or more block vectors, and then a predicted value of the current block may be determined based on the one or more reference blocks.

[0233] It should be noted that in the embodiments of the present application, one or more reference blocks of the current block include a first reference block and / or a second reference block, where the first reference block and the current block both belong to the current image, and the second reference block belongs to a reference image of the current image corresponding to the current block.

[0234] That is, in an embodiment of the present application, one or more reference blocks of the current block may be a first reference block in the current image obtained by intra prediction, or may be a second reference block in a reference image of the current image obtained by inter prediction, or may include a first reference block in the current image obtained by intra prediction and a second reference block in a reference image of the current image obtained by inter prediction.

[0235] Accordingly, in an embodiment of the present application, when obtaining a second reference block, one or more block vectors may be determined by decoding the bitstream, and then a search may be performed in a reference image of the current image to determine the second reference block corresponding to the one or more block vectors.

[0236] That is, in the embodiment of the present application, a template matching search may be performed in the current image using an intra template matching method to determine one or more reference blocks corresponding to the current block, which are first reference blocks. A template matching search may be performed in one or more inter reference images of the current image using an inter template matching method to determine one or more reference blocks corresponding to the current block, which are second reference blocks. The intra template matching method and the inter template matching method may be performed in the current image and one or more inter reference images, respectively, to determine one or more reference blocks corresponding to the current block, where the one or more reference blocks include a first reference block and a second reference block.

[0237] Furthermore, in the embodiments of the present application, when determining one or more reference blocks corresponding to the current block based on one or more block vectors, one or more initial reconstructed blocks corresponding to the current block may be determined first based on one or more block vectors, and then a correction process may be performed on the one or more initial reconstructed blocks to determine one or more reference blocks.

[0238] That is, in the embodiments of the present application, when obtaining N candidate reconstruction blocks (reference blocks), other methods may be adopted, for example, first correcting the initial reconstruction block corresponding to the obtained candidate template, and then determining the corresponding reference block.

[0239] Accordingly, N candidate reconstructed blocks (reference blocks) may be copied and obtained using N BV-compatible matching reconstructed blocks (initial reconstructed blocks), and then these N candidate reconstructed blocks may be weighted as they are to obtain a predicted value of the current block; alternatively, N BV-compatible matching reconstructed blocks (initial reconstructed blocks) may be corrected to obtain candidate reconstructed blocks (reference blocks), and then weighted to obtain a predicted value of the current block.

[0240] Furthermore, in the embodiments of the present application, when a correction process is performed on one or more initial reconstruction blocks to determine one or more reference blocks, a filtering process may be performed on one or more initial reconstruction blocks to determine one or more reference blocks.

[0241] Furthermore, in an embodiment of the present application, when performing a correction process on one or more initial reconstruction blocks to determine one or more reference blocks, one or more correction parameter vectors may be first determined based on one or more candidate templates corresponding to one or more block vectors, and then a correction process may be performed on one or more initial reconstruction blocks based on the one or more correction parameter vectors to determine one or more reference blocks.

[0242] Furthermore, in an embodiment of the present application, when determining one or more correction parameter vectors based on one or more candidate templates corresponding to one or more block vectors, an autocorrelation matrix corresponding to the candidate templates may be determined based on sample values ​​in the candidate templates, a cross-correlation vector may be determined based on sample values ​​in the first template and sample values ​​in the candidate templates, and a correction parameter vector may be determined based on the autocorrelation matrix and the cross-correlation vector.

[0243] In addition, in the embodiments of the present application, the method for correcting the initial reconstruction block may include a step of performing a filtering process on the initial reconstruction block directly, where the filtering method used when performing the process may be a conventional filtering method such as bilateral filtering, averaging filtering, etc., or may be a neural network-based filter enhancement.

[0244] It should be noted that in the embodiment of the present application, the method for correcting the initial reconstructed block may further include correcting the matching reconstructed block (initial reconstructed block) using matching template information (candidate template).

[0245] For example, in the embodiment of the present application, when the matching template information is used to perform the correction process on the matching reconstruction block, each candidate template refT n and the corresponding candidate reconstruction block (initial reconstruction block) RefBlock n For candidate template refT n and the current coding block template (first template of the current block) curT is used to calculate the correction parameter vector C n Calculate the correction parameter vector C n candidate reconstruction block RefBlock n and weighted synthesis to form the final corrected reconstructed block RefBlock' n That is, the reference block of the current block finally obtained is RefBlock' n is.

[0246] In some embodiments, the correction parameter vector C n is the candidate template refT n and the minimized MSE of the template sample values ​​to be predicted.

[0247] In the embodiment of the present application, the correction parameter vector C n may be viewed as one L-tap filter.

[0248] In some embodiments, the correction parameter vector C n When calculating each candidate template refT n For (n=0, 1..., N-1), the process of minimizing MSE inputs the autocorrelation matrix of the candidate template sample refT and the cross-correlation vector between the candidate template sample refT and the adjacent template sample curT of the current coding block, and outputs the weight of the candidate reconstruction block corresponding to the current candidate template.

[0249] Illustratively, in the examples of the present application, MSE is calculated as follows:

number

[0250] To make it easier to show the formula for calculating MSE, we will use E to represent the mean square error MSE as follows:

number

[0251] where K is the number of samples in the template.

[0252] In some embodiments, the weight c of the candidate reconstruction block corresponding to the candidate template refT by minimizing MSE l The process of deriving may include the following steps:

[0253] (1) First, c l Calculate the partial derivative with respect to and set it to 0.

number

[0254] After rearranging, the following formula was obtained:

number

[0255] (2) Candidate template region refT n After determining, the equation obtained in step (1) is expanded into matrix form as follows:

number

[0256] (3) The autocorrelation matrix and cross-correlation vector in step (2) are both known quantities. By solving the linear equations in (2), the filter weighting coefficients c0,...,c L-1 The correction parameter vector C n The filtering coefficients of a candidate reconstruction block in can be obtained.

[0257] Accordingly, if x=0...nTbW-1, y=0...nTbH-1, the corrected candidate reconstruction block RefBlock' n is as follows:

number

[0258] In the embodiment of the present application, after determining N block vectors BV corresponding to N candidate templates of the current block, N candidate reconstructed blocks (i.e., N reference blocks) are obtained using the N BVs, and then weighted synthesis is performed on the N candidate reconstructed blocks to obtain a predicted block of the current block (i.e., a predicted value of the current block).Here, when generating a final predicted value, first obtain the N candidate reconstructed blocks (N reference blocks), then determine the corresponding weighted synthesis weights (weight values), and then generate a predicted value of the current block through weighted synthesis processing.

[0259] In the embodiment of the present application, when determining one or more reference blocks for a current block based on one or more block vectors, N block vectors BV corresponding to the obtained N candidate templates are used. n On the other hand, BV n N candidate reconstruction blocks (i.e., reference blocks) RefBlock obtained from the current image and / or the reference image based on n You can select where BV n The horizontal offset is pX n and the vertical offset is pY nwhere n=0, 1…, N-1.

[0260] For example, in the embodiment of the present application, determining one or more reference blocks of the current block may be achieved by simple translational copying. Specifically, when x=0...nTbW-1, y=0...nTbH-1, the reconstructed samples of the current frame (i.e., the reference blocks of the current block) are determined by the following formula: RefBlock n [x][y]=recSamples[x+pX n ][y+pY n ] (15)

[0261] Furthermore, in the embodiments of the present application, when determining a predicted value of a current block based on one or more reference blocks, one or more weight values ​​corresponding to the one or more reference blocks are first determined, and then a weighted synthesis process is performed on the one or more reference blocks based on the one or more weight values, and finally a predicted value of the current block is determined.

[0262] In the embodiment of the present application, after obtaining N candidate reconstruction blocks RefBlock (i.e., reference blocks of the current block), it is necessary to calculate the weights W of the weighted combination of the N candidate reconstruction blocks. Here, the weight values ​​corresponding to the reference blocks can be determined in a number of ways. For example, the weights can be predefined values ​​(e.g., second predetermined values), or can be self-adaptively calculated values ​​using costs, sample values, etc.

[0263] In one embodiment, one or more weight values ​​may be determined based on a second predetermined value, where the second predetermined value may include N arbitrary values ​​greater than 0, and the corresponding weight values ​​for different reference blocks among the N reference blocks may be the same or different, and the present application does not specifically limit the second predetermined value.

[0264] In one embodiment, one or more weight values ​​may be determined based on one or more candidate templates corresponding to one or more block vectors, where determining one or more weight values ​​based on one or more candidate templates corresponding to one or more block vectors may involve first determining an autocorrelation matrix corresponding to the candidate templates based on sample values ​​in the candidate templates, then determining a cross-correlation vector based on sample values ​​in the first template and sample values ​​in the candidate templates, and then determining the weight values ​​based on the autocorrelation matrix and the cross-correlation vector.

[0265] In some embodiments, the candidate template refT n and the template to be predicted (first template) refpredT n The weighted combining weights (weight values) can be derived by using the minimized MSE of the sample values ​​of the vectors.

[0266] In order to make the weight derivation process more flexible, a nonlinear term and a bias term may be introduced in the process of deriving the weighted synthesis weights.

[0267] For example, in the embodiment of the present application, when deriving the weighted combining weights, the nonlinear terms are constructed based on the candidate templates, where one candidate template numbered 0 can be selected from N candidate templates for construction. Accordingly, when m=0,...,M-1, the following equation exists: NonLinearTerm_T n,m =(refT n,m *refT n,m +MidVal)≫bitDepth (16)

[0268] Here, n is 0, 1, ..., N-1, representing any template among the N candidate templates, MidVal is 1<<(bitDepth-1), and bitDepth is the image bit depth.

[0269] Exemplarily, in the embodiments of the present application, when applying weighted composite weights to each candidate reconstruction block (reference block) corresponding to N candidate templates, the non-linear term NonLinearTerm_Block is constructed based on the candidate reconstruction blocks. Here, the candidate reconstruction block corresponding to the candidate template with the number 0 can be selected. Accordingly, when x = 0, 1…nTbW - 1 and y = 0,…nTbH - 1, there is the following formula. NonLinearTerm_Block n,x,y =(refBlock n,x,y *refBlock n,x,y +MidVal)≫bitDepth (17)

[0270] Here, n is 0, 1,…, N - 1, and represents the candidate reconstruction block (reference block) corresponding to any one of the N candidate templates.

[0271] In the process of deriving weights and the process of using weights, the bias value Bias may be any one constant in the image sample range [0, (1<<bitDepth) - 1]. For example, Bias may be set to 1<<(bitDepth - 1).

[0272] Since BiasTerm is a constant, in the actual calculation process, it needs to be extended into a matrix and represented as BiasTerm.

[0273] For each of the N candidate templates, when m = 0,…, M - 1, BiasTerm m =Bias (18) For each candidate reconstruction block corresponding to the N candidate templates, when x = 0, 1…nTbW - 1 and y = 0…nTbH - 1, BiasTerm x,y =Bias (19)

[0274] In the embodiment of the present application, after adding the nonlinear term and the bias term, it is necessary to derive N+2 weighting coefficients (weight values). For convenience of explanation, the number of weighting coefficients is recorded by a variable P (P=N+2), and the matching template sample / nonlinear term sample and the bias term sample are collectively referred to as the matching reference sample refT. N and refT N+1 This allows us to collectively refer to the reference quantities involved in all operations as refT p Similarly, the reconstructed block corresponding to the matching template, the reconstructed block corresponding to the matching template related to the nonlinear term, and the bias term are collectively expressed as the candidate reconstructed sample refBlock p where p = 0,...,P-1.

[0275] For example, in the embodiment of the present application, the process of minimizing MSE inputs the autocorrelation matrix of the previous P matching reference samples refT, the cross-correlation vector between the previous P matching reference samples refT and the current coding block adjacent template samples curT, and outputs the weights of the reconstruction block corresponding to each matching reference term.

[0276] Here, the MSE calculation formula is as follows:

[0277] For each matching reference template, i.e., m=0, 1…, M-1, for the co-located samples,

number

[0278] To make it easier to show the formula for calculating MSE, we will use E to represent the mean square error MSE as follows:

number

[0279] The weights w of the reconstruction blocks corresponding to each matching reference sample are calculated using the minimized MSE. p The specific steps to derive are as follows:

[0280] (1) First, w p Calculate the partial derivative with respect to and set it to 0.

number

[0281] After rearranging, the following formula was obtained:

number

[0282] (2) Matching reference sample regions refT0, refT1, ..., refT P-1 After determining, the equation obtained in step (1) is expanded into matrix form as follows:

number

[0283] (3) The autocorrelation matrix and cross-correlation vector in step (2) are both known quantities. By solving the linear equation in (2), the weights w0,...,w of the reconstructed samples (reference blocks) corresponding to each matching reference term are obtained. P-1 can be calculated.

[0284] In one embodiment, when determining one or more weight values ​​based on one or more candidate templates corresponding to one or more block vectors, a matching cost between the first template and the candidate templates may be determined first, and then the weight values ​​may be determined based on the matching cost.

[0285] Furthermore, in the embodiments of the present application, the weighting coefficients may be calculated in other ways, for example, by using a nonlinear weighting model to calculate the weighting coefficients for each candidate reconstruction block (reference block) RefBlock based on the cost of N candidate matching templates. n Appropriate weight for n can be assigned.

[0286] It should be noted that in the embodiments of the present application, the weight model may include, but is not limited to, a non-linear normalization function, a non-linear exponential normalization function, and the like.

[0287] For example, in an embodiment of the present application, the weight of each candidate reconstruction block (reference block) may be calculated by the following nonlinear function, where the input of the weight model is the current coding block template curT and the candidate template refT: n The matching cost is the matching cost between the current coding block template curT and the candidate template refT n SAD(refT n ), MAD(refT n ), correlation coefficient R(refT n ) and the like, but are not limited to these.

[0288] In some embodiments, the formula corresponding to the weight model is:

number

[0289] Here, offset is a predetermined value, for example, offset is 1.

[0290] In some embodiments, the matching cost is calculated by the normalized correlation coefficient R(refT n ), the corresponding formula for the weight model is:

number

[0291] In some embodiments, a Softmax function may be used as the weight model, and the calculation formula is as follows:

number

[0292] Here, S is a model control parameter, and under a predetermined condition, the weight model can be adjusted by adjusting the parameter S. For example, the parameter S may be related to the current block size or the template type.

[0293] In some embodiments, instead of the above nonlinear weighting model, the weighting coefficients of each candidate reconstruction block may be directly averaged as follows, for example.

number

[0294] In some embodiments, the candidate template refT n and the template to be predicted (first template) refpredT n In the process of deriving the weighted combining weights (weight values) by using the minimized MSE of the sample values ​​of the vectors, it is not necessary to add a nonlinear term and a bias term.

[0295] For example, in the embodiment of the present application, if no nonlinear term and bias term are added, only N weight coefficients (weight values) need to be derived, where the process of minimizing the MSE inputs the autocorrelation matrix of the previous N candidate template samples refT and the cross-correlation vector between the previous P candidate template samples refT and the current coding block adjacent template samples curT, and outputs the weights of the candidate reconstruction blocks corresponding to each candidate template.

[0296] In some embodiments, for the co-located samples of each candidate template, ie, m=0, 1 . . . , M-1, the MSE calculation formula is:

number

[0297] To make it easier to show the formula for calculating MSE, we will use E to represent the mean square error MSE as follows:

number

[0298] The weights w of the candidate reconstruction blocks corresponding to each candidate template are calculated by minimizing the MSE. n The specific steps to obtain are as follows:

[0299] (1) First, w n Calculate the partial derivative with respect to and set it to 0.

number

[0300] After rearranging, the following formula was obtained:

number

[0301] (2) Candidate template regions refT0, refT1, ..., refT N-1 After determining, the equation obtained in (1) is expanded into matrix form as follows:

number

[0302] (3) The autocorrelation matrix and cross-correlation vector in step (2) are both known quantities. By solving the linear equations in (2), the weights w0,...w of the reconstructed samples corresponding to each candidate template are obtained. N-1can be calculated.

[0303] In some embodiments, the candidate template refT n and the template to be predicted (first template) refpredT n In the process of deriving the weighted combining weights (weight values) by using the minimized MSE of the sample values ​​of the above, only the bias term may be added, and no nonlinear term may be added.

[0304] For example, in the embodiment of the present application, if only the bias term is added and no nonlinear term is added, N+1 weighting coefficients (weight values) need to be derived. For convenience of explanation, the number of weighting coefficients is recorded by a variable P (P=N+1), and the candidate template sample / bias term sample is collectively referred to as the matching reference sample refT. N and refT N+1 This allows us to collectively refer to the reference quantities involved in all operations as refT p Similarly, the candidate reconstructed block corresponding to the candidate template and the bias term can be collectively expressed as the candidate reconstructed sample refBlock p where p=0, 1..., P-1.

[0305] In some embodiments, the process of minimizing MSE inputs the autocorrelation matrix of the previous P matching reference samples refT, the cross-correlation vector between the previous P matching reference samples refT and the current coding block neighboring template samples curT, and outputs the weights of the reconstruction block corresponding to each matching reference term.

[0306] In some embodiments, for the co-located samples of each candidate template, ie, m=0, 1 . . . , M-1, the MSE calculation formula is:

number

[0307] To make it easier to show the formula for calculating MSE, we will use E to represent the mean square error MSE as follows:

number

[0308] The weights w of the reconstruction blocks corresponding to each matching reference sample are calculated using the minimized MSE. p The specific steps to derive are as follows:

[0309] (1) First, w p Calculate the partial derivative with respect to and set it to 0.

number

[0310] After rearranging, the following formula was obtained:

number

[0311] (2) Matching reference sample regions refT0, refT1, …, refT P-1 After determining, the equation obtained in step (1) is expanded into matrix form as follows:

number

[0312] (3) The autocorrelation matrix and cross-correlation vector in step (2) are both known quantities. By solving the linear equation in (2), the weights w0,...w of the reconstructed samples corresponding to each matching reference term are obtained. P-1 can be calculated.

[0313] In step 104, a reconstructed value for the current block is determined based on the predicted value for the current block.

[0314] In an embodiment of the present application, one or more reference blocks of a current block may be determined based on one or more block vectors, a predicted value of the current block may be determined based on the one or more reference blocks, and then a reconstructed value of the current block may be determined based on the predicted value of the current block.

[0315] It should be noted that in the embodiments of the present application, the bitstream may be decoded first to determine a prediction residual value corresponding to the current block, and then a reconstructed value of the current block may be determined based on the prediction residual value and the prediction value.

[0316] As described above, the decoding method proposed in steps 101 to 104 is improved and optimized based on the general IntraTMP technology, and a weighted synthesis method is adopted to propose an IntraTMP Fusion prediction method. Here, in the process of searching and determining a block vector BV in the search area, at least one block vector of the current block can be determined, that is, multiple block vectors can be determined. In addition, in the configuration for generating a predicted value, a weighted synthesis process is performed using at least one reference block corresponding to at least one block vector to obtain a predicted value of the current block, and then a reconstructed value of the current block is determined.

[0317] That is, the embodiment of the present application proposes IntraTMP Fusion technology, in which the current coding block template (the first template of the current block) searches for N block vectors BV corresponding to N candidate matching templates with the smallest matching cost within a predefined range, and then finds N candidate reconstruction blocks (N reference blocks) corresponding to these N candidate matching templates (N candidate templates) using the N BVs, and then performs weighted synthesis with a predetermined weight to obtain a predicted block (predicted value) of the current block.

[0318] The IntraTMP Fusion method proposed in the embodiments of the present application can improve the accuracy of prediction values. Here, the current coding block template (first template of the current block) is determined by searching for N block vectors BV corresponding to N candidate matching templates with the minimum matching cost within a predefined range, and then using the N BVs to find N candidate reconstructed blocks (N reference blocks) corresponding to these N candidate matching templates (N candidate templates). Then, weighted synthesis is performed with predetermined weights to determine the predicted block of the current block. Rather than simply considering the reconstructed block corresponding to the template with the minimum matching cost, the method fully considers information about reconstructed blocks corresponding to different matching templates during the search process. Furthermore, weights are self-adaptively assigned to the candidate reconstructed blocks according to the matching template information, fully considering the difference in importance of different reconstructed block information for the prediction of the current block.

[0319] In addition, the IntraTMP Fusion method proposed in the embodiments of the present application can fully utilize candidate reconstruction block information corresponding to different matching templates in the process of searching for a matching template. Rather than simply considering reconstruction block information corresponding to the template with the smallest matching cost, the method fully utilizes reconstruction block information corresponding to different matching templates in the search process. In addition, the matching template information is fully utilized to self-adaptively assign weights to candidate reconstruction blocks, fully considering the difference in importance of different reconstruction block information for predicting the current block.

[0320] In this way, the IntraTMP Fusion method according to the embodiment of the present application can avoid to some extent the degradation of prediction accuracy caused by inaccurate template matching criteria or by directly copying a reconstructed block as a predicted block.

[0321] Compared to common encoding / decoding techniques, the IntraTMP Fusion method according to the embodiment of this application can achieve BD-rate changes (i.e., average bitrate changes at equivalent psnr) of -0.29%, -0.30%, and -0.39% for Y, Cb, and Cr, respectively, when tested under All Intra conditions at 24-frame intervals.

[0322] According to a decoding method of an embodiment of the present application, a decoding device determines a first template corresponding to a current block, determines one or more block vectors corresponding to the current block based on the first template, determines one or more reference blocks for the current block based on the one or more block vectors, determines a predicted value of the current block based on the one or more reference blocks, and determines a reconstructed value of the current block based on the predicted value of the current block. Thus, the embodiment of the present application provides an IntraTMP Fusion prediction method, which can select and determine at least one block vector of the current block, and further obtain a predicted value of the current block using at least one reference block corresponding to the at least one block vector. In other words, the encoding / decoding method of an embodiment of the present application fully considers the difference in importance of reconstructed block information corresponding to different matching templates in the search process to predict the current block, thereby improving prediction accuracy and achieving optimal prediction results.

[0323] An embodiment of the present application provides an encoding method for use in an encoding device. Figure 15 is a flowchart of the encoding method according to an embodiment of the present application. As shown in Figure 15, the method for the encoding device to perform encoding processing may include the following steps:

[0324] In step 201, a first template corresponding to the current block is determined.

[0325] In an embodiment of the present application, a first template corresponding to the current block may be determined first, where, when obtaining the first template, a template type corresponding to the current block may be determined first, and then the first template corresponding to the current block may be further determined based on the template type.

[0326] In addition, the encoding method of the embodiment of the present application is applied to an encoding device. The encoding method may also include an intra prediction method, more specifically, a color component prediction method. Here, a video image may be divided into multiple coding blocks, each of which may include a first color component, a second color component, and a third color component. The current block in the embodiment of the present application refers to the coding block in the video image that currently performs intra prediction.

[0327] Here, when a first color component needs to be predicted, the component to be predicted is the first color component; when a second color component needs to be predicted, the component to be predicted is the second color component; and when a third color component needs to be predicted, the component to be predicted is the third color component. Furthermore, if the current block predicts the first color component and the first color component is a luma component, i.e., the component to be predicted is a luma component, the current block may be called a luma block. Alternatively, if the current block predicts the second color component and the second color component is a chroma component, i.e., the component to be predicted is a chroma component, the current block may be called a chroma block.

[0328] In the embodiments of the present application, the reference sample of the current block may refer to a reference sample point adjacent to the current block. Here, adjacent may be, but is not limited to, spatial adjacent. For example, adjacent may be temporal adjacent or spatial-temporal adjacent. Furthermore, the reference sample of the current block may be a reference sample obtained by performing some processing on a spatially adjacent, temporally adjacent, or spatial-temporal adjacent reference sample point, and the embodiments of the present application are not limited thereto.

[0329] Furthermore, in an embodiment of the present application, the template type of the current block can be determined based on the reference samples of the current block, where the reference samples of the current block include at least one of the following: a left-side neighboring reference sample of the current block, an upper-side neighboring reference sample of the current block, an upper-left neighboring reference sample of the current block, a lower-left neighboring reference sample of the current block, and an upper-right neighboring reference sample of the current block.

[0330] It should be noted that in the embodiments of the present application, the reference samples of the current block may include adjacent reconstructed samples of the current block, i.e., the adjacent reconstructed samples of the current block may be used as templates to search for a matching template within a predefined search area.

[0331] It should be noted that in the embodiments of the present application, the reference samples of the current block, that is, the adjacent reconstructed samples of the current block, may include the upper reference sample, the upper left reference sample, the upper right reference sample, the left reference sample, and the lower left reference sample of the current block.

[0332] In addition, in the embodiment of the present application, when the template type of the current block is determined by the reference samples of the current block, the template type can be classified and determined according to whether adjacent reference samples are available.

[0333] Furthermore, in an embodiment of the present application, when determining the template type of the current block based on the reference samples of the current block, if the left adjacent reference sample of the current block, the upper adjacent reference sample of the current block, and the upper left adjacent reference sample of the current block are all available, the template type of the current block is determined as a first value. If the left adjacent reference sample of the current block is available, the template type of the current block is determined as a second value. If the upper adjacent reference sample of the current block is available, the template type of the current block is determined as a third value. If the left adjacent reference sample of the current block and the upper left adjacent reference sample of the current block are all available, the template type of the current block is determined as a fourth value. If the left adjacent reference sample of the current block and the lower left adjacent reference sample of the current block are all available, the template type of the current block is determined as a fifth value. If the upper adjacent reference sample of the current block and the upper right adjacent reference sample of the current block are all available, the template type of the current block is determined as a sixth value.

[0334] In the examples of the present application, the first, second, third, fourth, fifth, and sixth values ​​may be any numerical values, and the present application does not specifically limit them. For example, the first, second, third, fourth, fifth, and sixth values ​​are 1, 2, 3, 4, 5, and 6, respectively.

[0335] For example, in the embodiment of the present application, the template type can be represented by refTemplateType, and accordingly, as shown in Figure 3 above, the grid-filled block is the current block, and the adjacent area of ​​the current block is template T, where six types of template types are shown.

[0336] Exemplarily, these six template types are as follows: If the upper-left reference sample, upper reference sample, and left reference sample are all available, the value of refTemplateType is 1, and the template shape is shown in FIG. 3(a). If only the left reference sample is available, the value of refTemplateType is 2, and the template shape is shown in FIG. 3(b). If only the upper reference sample is available, the value of refTemplateType is 3, and the template shape is shown in FIG. 3(c). If only the left reference sample and the upper-left reference sample are available, the value of refTemplateType is 4, and the template shape is shown in FIG. 3(d). If only the left reference sample and the lower-left reference sample are available, the value of refTemplateType is 5, and the template shape is shown in FIG. 3(e). If only the upper reference sample and the upper-right reference sample are available, the value of refTemplateType is 6, and the template shape is shown in FIG. 3(f).

[0337] Furthermore, in an embodiment of the present application, when determining a first template corresponding to a current block based on a template type, a template reference sample of the current block may be determined first based on the template type and a template size corresponding to the template type, and then the first template of the current block may be determined based on the template reference sample.

[0338] In addition, in the embodiment of the present application, the first template of the current block may include a template reference sample of the current block, where the template reference sample of the current block may be determined according to the template type of the current block and the template size corresponding to the template type.

[0339] In addition, in the embodiments of the present application, the first template of the current block may be composed of reconstructed samples of one or more regions of the top, top right, left, bottom left, and top left of the current block, i.e., it may be composed of the reference samples of the current block.

[0340] In the embodiments of the present application, the template size corresponding to the template type may be set in advance. For example, when acquiring a left template, the template width (templateW_size) may be set to 4, and when acquiring a top template, the template height (templateH_size) may be set to 4.

[0341] Correspondingly, in the embodiments of the present application, by combining the value of the template type refTemplateType of the current block and the template size corresponding to this refTemplateType, it is possible to determine which part of the reconstructed sample should be taken as the template reference sample of the current block, and further to determine the corresponding first template.

[0342] For example, in the embodiment of the present application, if the value of refTemplateType is 1, the reconstructed samples on the left, top-left, and top of the current block are selected and acquired; if the value of refTemplateType is 2, only the reconstructed samples in the left four columns of the current block are acquired; if the value of refTemplateType is 3, only the reconstructed samples in the top four rows of the current coding block are acquired.

[0343] Of course, the preset template size value is not limited to 4, and may be any integer greater than 0, and the present application does not specifically limit it.

[0344] It should be noted that in the embodiment of the present application, the template reference sample of the current block determined from the reference sample of the current block based on the template type of the current block and the corresponding template size can be the first template corresponding to the current block.

[0345] In step 202, one or more block vectors corresponding to the current block are determined based on a first template.

[0346] In an embodiment of the present application, after determining the first template corresponding to the current block, one or more block vectors corresponding to the current block may be further determined based on the first template.

[0347] In addition, in the embodiment of the present application, the block vector search process includes an initialization process, determining a search area within the current frame of the first template, and searching for and determining one or more optimal block vectors in the search area. Therefore, when performing the search process, the initialization operation must first be completed.

[0348] Exemplarily, as shown in FIG. 5 above, nTbW and nTbH represent the size of the current block, templateW_size and templateH_size represent the template size, and uiPatchWidth and uiPatchHeight represent the block size including the current block and its template.

[0349] Accordingly, in the initialization process, uiPatchWidth can be initialized to nTbW+templateW_size, and uiPatchHeight can be initialized to nTbH+templateH_size. Here, templateW_size and templateH_size may be fixed constants or may be dynamically adjusted based on the coding block size, and templateW_size and templateH_size may or may not be equal. For example, templateW_size=4, templateH_size=4, or templateW_size=4 is set when the width of the coding block is greater than 8, templateW_size=2 is set when the width of the coding block is 8 or less, templateH_size=4 is set when the height of the coding block is greater than 8, and templateH_size=2 is set when the height of the coding block is 8 or less.

[0350] Furthermore, the cost threshold between the initialization templates is denoted by diffThreshold. For example, if the cost function is SAD, the threshold is diffThreshold=((1<<bitDepth)> >2)×(uiPatchHeight×uiPatchWidth−nTbH×nTbW), where diffThreshold indicates that the maximum distortion of each sample point in the template region is 256 when the image bit depth bitDepth is 10.

[0351] Furthermore, the position of the coding tree block CTB where the current block CB is located is initialized to ctbRsX, ctbRsY.

[0352] Furthermore, the position offsets of the current block CB within the current CTB are initialized so that offsetLCBY=yTbCmp-ctbRsY and offsetLCBX=xTbCmp-ctbRsX.

[0353] Furthermore, initialization is performed so that iTemplateSizeH=templateH_size and iTemplateSizeW=templateW_size.

[0354] Furthermore, iBvShift is initialized. iBvShift is the precision of the block vector BV. For example, the precision of BV may be full sample precision, in which case iBvShift is 0, or the precision of BV may be sub-sample precision, for example, when iBvShift is 1, it indicates 1 / 2 sample precision, and when iBvShift is 2, it indicates 1 / 4 sample precision, but this is not specifically limited here.

[0355] Furthermore, a predetermined search range of the template is initialized. The predetermined search range of the template may be set to a fixed size, or the search range may be dynamically adjusted according to the coding block size. For example, searchRangeWidth=TMP_SEARCH_RANGE_MULT_FACTOR×nTbW, searchRangeHeight=TMP_SEARCH_RANGE_MULT_FACTOR×nTbH. Here, the value of TMP_SEARCH_RANGE_MULT_FACTOR may be a predetermined value, for example, 5.

[0356] Furthermore, in an embodiment of the present application, when determining one or more block vectors corresponding to a current block based on a first template, a predetermined search area may be determined based on the first template first, and then one or more block vectors may be determined by searching within the predetermined search area.

[0357] In the embodiment of the present application, the search area is the reconstructed portion of the current image and is limited by the size of the search range. As shown in Figure 6, the background area filled with dark color is the reconstructed area, the background block filled with black is the current block, and the dotted frame is the search range window. Therefore, the search area of ​​the IntraTMP technique does not exceed the overlapping area between the reconstructed area shown with the dark background and the area marked with the dotted frame.

[0358] In this way, the search area of ​​the current block template can be the reconstructed part of the CTB where the current block is located, or it can be another reconstructed CTB area. The search area here is actually the set of all search points. Because the shape of the area cannot usually be represented by a single rectangular area, in a specific implementation, searches can be performed in multiple rectangular areas, and the search results of different areas can be combined to obtain the final best-matching block and best-matching block vector.

[0359] For example, refer to Figure 7, which shows eight different sub-area division methods. Here, the black background block is the current block. Five division methods (a), (b), (c), (d), and (f) all divide the search area into four sub-search areas, and three division methods (e), (g), and (h) all divide the search area into three sub-search areas. Here, different fill patterns represent one sub-search area.

[0360] In Figure 7, (a), (b), (c), and (d) consider all available search ranges, while (e), (f), (g), and (h) do not search the area directly above or directly to the left.

[0361] For example, when different sub-search areas are represented by regionId, the template sample of the current coding block needs to be obtained from the image reconstruction area, and the reconstruction block sample corresponding to the template also needs to be obtained from the reconstruction area. Therefore, it is necessary to further determine searchable positions within the sub-search areas represented by different regionIds based on the position of the current coding block (xTbCmp, yTbCmp), the size of the current coding block (nTbW, nTbH), the size of the current image (picWidth, picHeight), the size of the CTB in which the current coding block is located (CtbSizeW, CtbSizeH), the specified search range of the template (searchRangeWidth, searchRangeHeight), and the position offset of the current coding block within the current CTB (offsetLCBY, offsetLCBX), to determine the block vector BV. Specifically, iVerMin represents the absolute coordinate position that can be searched at least in the vertical direction, iVerMax represents the relative coordinate position that can be searched at most in the vertical direction, iHorMin represents the absolute coordinate position that can be searched at least in the horizontal direction, and iHorMax represents the absolute coordinate position that can be searched at most in the horizontal direction. The values ​​of iVerMin, iVerMax, iHorMin, and iHorMax are different in search regions represented by different regionIds.

[0362] In some embodiments, taking FIG. 7(f) as an example, the search area is divided into four sub-search areas, which are implemented as follows:

[0363] If regionId is equal to 0, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0364] iHorMax0=min((xTbCmp+searchRangeWidth)< <iBvShift,((picWidth-nTbW)<<iBvShift)) iHorMin0=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iVerMax0=(yTbCmp-nTbH-offsetLCBY)< <iBvShift iVerMin0=max(((iTemplateSizeH)< <iBvShift),((yTbCmp-searchRangeHeight)<<iBvShift)) If regionId is equal to 1, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0365] iHorMin1=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iHorMax1=(xTbCmp-offsetLCBX-nTbW)< <iBvShift iVerMin1=(yTbCmp+1)< <iBvShift iVerMax1=min(picHeight-nTbH,(yTbCmp-offsetLCBY+CtbSizeH-nTbH)< <iBvShift) If regionId is equal to 2, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0366] iHorMax2=(xTbCmp-offsetLCBX-nTbW)< <iBvShift iHorMin2=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iVerMin2=max((iTemplateSizeH)< <iBvShift,(yTbCmp-nTbH-offsetLCBY)<<iBvShift) iVerMax2=(yTbCmp)< <iBvShift。

[0367] If regionId is equal to 3, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows: iHorMin3=max((iTemplateSizeW)< <iBvShift,(xTbCmp-offsetLCBX-nTbW+1)<<iBvShift) iHorMax3=(xTbCmp-nTbW)< <iBvShift iVerMin3=max(((iTemplateSizeH)< <iBvShift),(yTbCmp-offsetLCBY-nTbH+1)<<iBvShift) iVerMax3=(yTbCmp-nTbH)< <iBvShift In practical application, iHorMin regionId , iHorMax regionId , iVerMin regionId , iVerMax regionId represent the left, right, top and bottom edges of different sub-search areas, respectively.

[0368] To intuitively explain the different sub-search areas corresponding to different regionIds, R1, R2, R3, and R4 represent four different sub-search areas as shown in Figure 8. Note that Figure 8 represents the sample range in which the upper left corner sample of the block can be aligned.

[0369] In some embodiments, taking FIG. 7(a) as an example, the search area is divided into four sub-search areas, and the implementation is as follows:

[0370] If regionId is equal to 0, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0371] iHorMax0=min((xTbCmp+searchRangeWidth)< <iBvShift,((picWidth-nTbW)<<iBvShift)) iHorMin0=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iVerMax0=(yTbCmp-nTbH-offsetLCBY)< <iBvShift iVerMin0=max(((iTemplateSizeH)< <iBvShift),((yTbCmp-searchRangeHeight)<<iBvShift)) If regionId is equal to 1, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0372] iHorMin1=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iHorMax1=(xTbCmp-offsetLCBX-nTbW)< <iBvShift iVerMin1=(yTbCmp+1)< <iBvShift iVerMax1=min(picHeight-nTbH,(yTbCmp-offsetLCBY+CtbSizeH-nTbH)< <iBvShift) If regionId is equal to 2, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0373] iHorMax2=(xTbCmp-nTbW)< <iBvShift iHorMin2=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iVerMin2=max((iTemplateSizeH)< <iBvShift,(yTbCmp-nTbH)<<iBvShift) iVerMax2=(yTbCmp)< <iBvShift If regionId is equal to 3, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0374] iHorMin3=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iHorMax3=(xTbCmp)< <iBvShift iVerMin3=max(((iTemplateSizeH)< <iBvShift),(yTbCmp-offsetLCBY-nTbH+1)<<iBvShift) iVerMax3=(yTbCmp-nTbH)< <iBvShift In practical application, iHorMin regionId , iHorMax regionId , iVerMin regionId , iVerMax regionId represent the left, right, top and bottom edges of different sub-search areas, respectively.

[0375] To intuitively explain the different sub-search areas corresponding to different regionIds, R1, R2, R3, and R4 denote four different sub-search areas as shown in Figure 8. Note that Figure 8 represents the sample range in which the upper left corner sample of the block can be aligned.

[0376] In some embodiments, taking FIG. 7(b) as an example, the search area is divided into four sub-search areas, which are implemented as follows:

[0377] If regionId is equal to 0, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0378] iHorMax0=min((xTbCmp+searchRangeWidth)< <iBvShift,((picWidth-nTbW)<<iBvShift)) iHorMin0=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iVerMax0=(yTbCmp-nTbH-offsetLCBY)< <iBvShift iVerMin0=max(((iTemplateSizeH)< <iBvShift),((yTbCmp-searchRangeHeight)<<iBvShift)) If regionId is equal to 1, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0379] iHorMin1=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iHorMax1=(xTbCmp-offsetLCBX-nTbW)< <iBvShift iVerMin1=(yTbCmp+1)< <iBvShift iVerMax1=min(picHeight-nTbH,(yTbCmp-offsetLCBY+CtbSizeH-nTbH)< <iBvShift) If regionId is equal to 2, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0380] iHorMax2=(xTbCmp-nTbW)< <iBvShift iHorMin2=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iVerMin2=max((iTemplateSizeH)< <iBvShift,(yTbCmp-nTbH-offsetLCBY)<<iBvShift) iVerMax2=(yTbCmp)< <iBvShift If regionId is equal to 3, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0381] iHorMin3=max((iTemplateSizeW)< <iBvShift,(xTbCmp-nTbW+1)<<iBvShift) iHorMax3=(xTbCmp)< <iBvShift iVerMin3=max(((iTemplateSizeH)< <iBvShift),(yTbCmp-offsetLCBY-nTbH+1)<<iBvShift) iVerMax3=(yTbCmp-nTbH)< <iBvShift In practical application, iHorMin regionId , iHorMax regionId , iVerMin regionId , iVerMax regionId represent the left, right, top and bottom edges of different sub-search areas, respectively.

[0382] To intuitively explain the different sub-search areas corresponding to different regionIds, R1, R2, R3, and R4 denote four different sub-search areas as shown in Figure 13. Note that Figure 13 represents the sample range in which the upper left corner sample of the block can be aligned.

[0383] Furthermore, in the embodiments of the present application, when searching within a predetermined search area to determine one or more block vectors, the search points in the predetermined search area may be traversed, and a matching cost between the matching template corresponding to the search points in the predetermined search area and the first template may be determined based on a predetermined matching criterion, and one or more block vectors and one or more candidate templates corresponding to the one or more block vectors may be determined based on the matching cost.

[0384] It should be noted that in the embodiments of the present application, the number of block vectors determined by the search process may be one or more, for example, N block vectors of the current block may be determined, where N is an integer greater than 0.

[0385] Accordingly, in the embodiments of the present application, when searching within a predetermined search area to determine one or more block vectors, a predetermined number N corresponding to candidate templates may be first determined, then the search points in the predetermined search area may be traversed, and the matching cost between the matching template corresponding to the search points in the predetermined search area and the first template may be determined based on a predetermined matching criterion; and finally, N block vectors and N candidate templates corresponding to the N block vectors may be determined based on the matching cost.

[0386] That is, in an embodiment of the present application, when searching for and determining N block vectors corresponding to N matching templates in a predetermined search area, i.e., the process of searching for and determining block vectors BV corresponding to N matching templates in a search area (predetermined search area), may include determining the value of the number N of candidate templates, determining a matching template comparison criterion, and recording N block vectors BV corresponding to the N matching templates (selected N candidate templates).

[0387] In addition, in the embodiments of the present application, when determining the predetermined number N corresponding to the candidate templates, N may be determined by decoding the bitstream, N may be determined based on a first predetermined value, or N may be determined based on a predetermined value range.

[0388] That is, in the embodiment of the present application, the value of N must first be determined. Here, N may be artificially preset to a certain constant, for example, N=4. N may be within a predetermined numerical range, for example, N may be any integer in [2, 8]. The range of N may be preset, and the encoding side may determine N according to a predetermined strategy criterion. For example, the optimal N value may be determined using a method such as coarse selection based on cost 1, coarse selection based on cost 2, coarse selection based on cost 3, or fine selection based on cost 4, and the optimal N value may be transmitted to the decoding side as a bitstream. The predetermined strategy criterion, which includes each of costs 1, 2, 3, and 4, may be one of cost functions for evaluating modes, such as SAD, SATD, MSE, MAD, and RDO. The present application does not particularly limit the method for determining N.

[0389] It should be noted that in the embodiment of the present application, the predetermined matching criterion includes any one of cost functions for evaluating modes, such as sum of absolute differences SAD, sum of absolute transformed differences SATD, sum of squared errors SSE, mean absolute deviation MAD, mean absolute error MAE, mean squared error MSE, normalized correlation coefficient NCC, etc.

[0390] Furthermore, in the embodiments of the present application, when determining N block vectors and N candidate templates corresponding to the N block vectors based on matching costs, i.e., when determining one or more block vectors and one or more candidate templates corresponding to one or more block vectors, N minimum matching costs may be first determined from the matching costs between the matching templates corresponding to search points in a predetermined search area and the first template, and then N block vectors and N candidate templates corresponding to the N minimum matching costs may be determined.

[0391] In the embodiments of the present application, for any one of the N block vectors, bvXMins and bvXMaxs may represent the minimum and maximum horizontal shifts of the block vector, respectively, and bvYMins and bvYMaxs may represent the minimum and maximum vertical shifts of the block vector, respectively.

[0392] bvXMinsregionId, bvXMaxsregionId, bvYMinsregionId, bvYMaxsregionId can be calculated based on the determined iVerMinregionId, iVerMaxregionId, iHorMinregionId, iHorMaxregionId as follows:

[0393] bvXMinsregionId=iHorMinregionId-xTbCmp bvXMaxsregionId=iHorMaxregionId-xTbCmp bvYMinsregionId=iVerMinregionId-yTbCmp bvYMaxsregionId=iVerMaxregionId-yTbCmp bvXMinsregionId, bvXMaxsregionId, bvYMinsregionId, and bvYMaxsregionId determine the range of the block vector BV, which is the range of horizontal and vertical shift amounts for the search point of the current block.

[0394] Note that the search points (iPosHor, iPoxVer) in each search area, i.e., each block vector BV (consisting of horizontal and vertical components (pX, pY), where pX = iPosHor-xTbCmp and pY = iPosVer-yTbCmp, where pX is between bvXMins and bvXMaxs, and pY is between bvYMins and bvYMaxs), can be used to search for one or more matching reconstruction blocks of the current block in the reconstruction area, and the adjacent reconstruction samples of the one or more matching reconstruction blocks are matching templates. This allows the matching costs between the current block adjacent templates and one or more reconstruction block adjacent templates to be calculated, which is denoted as pDiff.

[0395] Furthermore, traverse all search points in all search ranges (regionId=0,1,2,3), compare and obtain one or more search points with the smallest matching cost pDiff, denote the corresponding matching cost as pDiff_BEST, denote one or more corresponding block vectors BV as optimal block vectors BV_BEST(pX_BEST, pY_BEST), and obtain one or more corresponding matching templates as optimal matching templates T_BEST, i.e., obtain one or more candidate templates.

[0396] In the embodiment of the present application, after determining the value of the number N of candidate templates, it is necessary to select N block vectors BV corresponding to the N candidate templates that are relatively matched according to a certain comparison criterion. That is, compared with the related art that searches for a general matching template and records the block vectors BV, the difference of the present application is that multiple block vectors BV n Select and record the block vector BV n , i.e., the template shift amount pX n and pY nThe method involves obtaining a matching template based on

[0000] , calculating a template matching cost, and recording N BVs corresponding to the N matching templates with the smallest costs. Here, the N matching templates are referred to as N candidate templates. Here, the template matching cost (predetermined matching criterion) may be one of cost functions for evaluating modes, such as SAD, SATD, MSE, MAD, RDO, correlation coefficient, etc.

[0397] For example, in the embodiment of the present application, if the matching cost comparison criterion (predetermined matching criterion) is Mean Absolute Difference (MAD), the calculation formula is as follows:

number

[0398] where refT is the matching template in the search process, curT is the current coding block template (the first template of the current block), M is the number of samples in the current coding block template, and MAD(refT) is the mean absolute deviation between the current coding block template curT and the searched matching template.

[0399] Accordingly, in the embodiment of the present application, the selection criterion by MAD is to compare and record the N block vectors BV corresponding to the matching templates with the N smallest MAD costs.

[0400] Exemplarily, for N candidate templates, the MAD between the n-th candidate template and the current coding block template is:

number

[0401] where refT nis the nth candidate template, and MAD(refT n ) is the mean absolute deviation between the current coding block template curT and the n-th candidate template, where n=0,...,N-1.

[0402] Illustratively, in an embodiment of the present application, when the matching cost comparison criterion (predetermined matching criterion) is SAD, the calculation formula is as follows:

number

[0403] Here, SAD(refT) is the sum of absolute differences between the current coding block template (first template of the current block) curT and the searched matching template.

[0404] Correspondingly, in the embodiment of the present application, the selection criterion by SAD is to compare and record N block vectors BV corresponding to N matching templates with relatively small SAD costs.

[0405] Exemplarily, for N candidate templates, the SAD between the n-th candidate template and the current coding block template is:

number

[0406] where SAD(refT n ) is the sum of absolute differences between the current coding block template and the n-th candidate template.

[0407] Illustratively, in the embodiment of the present application, the NCC normalized correlation coefficient can be compared as the template matching criterion, and the calculation formula is as follows:

number

[0408] where refT is the matching template in the search process, curT is the current coding block template (the first template of the current block), M is the number of samples in the current coding block template, and refT Avg is the sample mean value of the searched matching template, and curT Avg is the sample mean value of the current coding block template, and R(refT) is the correlation coefficient between the current coding block template and the searched matching template.

[0409] Accordingly, in the embodiment of the present application, the comparison criterion by NCC is to sort and record the N block vectors BV corresponding to the matching templates with the N largest correlation coefficients R.

[0410] For N candidate templates, the correlation coefficient between the nth candidate template and the current coding block template is:

[0411]

number

[0412] In the embodiment of the present application, when performing a search process, available search strategies may include, but are not limited to, a search method based on different search step sizes, such as a coarse search based on a first search step size and / or a precise search based on a second search step size, where the first search step size is larger than the second search step size.

[0413] Furthermore, in the embodiment of the present application, the block vectors and candidate templates may be determined by traversing the search points in the predetermined search area according to a first search step width, or by traversing the search points in the predetermined search area according to a second search step width.

[0414] Furthermore, in the embodiment of the present application, first, the search points in the predetermined search area are traversed according to a first search step width to determine an initial block vector and an initial matching template corresponding to the initial block vector; then, a first search area is determined based on the initial matching template, where the first search area is smaller than the predetermined search area; finally, the search points in the first search area are traversed according to a second search step width to determine a block vector and a candidate template, where the first search step width is larger than the second search step width.

[0415] That is, in the embodiments of the present application, when searching for the optimum matching template in the search area, a search strategy of performing a coarse search followed by a fine search may be adopted, or only a fine search may be performed, or only a coarse search may be performed.

[0416] Exemplarily, in the embodiments of the present application, the coarse search may specifically include determining an optimal coarse matching template in the search area according to a first predetermined step width (i.e., a first search step width, for example, 2), i.e., obtaining a final candidate template, or determining an optimal coarse matching template in the search area by a down-sampled template (for example, the down-sampling factor is 2), i.e., obtaining a final candidate template.

[0417] Illustratively, in the embodiments of the present application, the precise search may specifically include determining an optimal precise matching template in the search area according to a second predetermined step width (i.e., a second search step width, for example, 1), i.e., obtaining a final candidate template, or performing a coarse search and then determining an optimal precise matching template in the vicinity of the optimal coarse matching template, i.e., obtaining a final candidate template.

[0418] For example, in an embodiment of the present application, if the search strategy is to perform only a coarse search, a coarse search is performed within the search range in each region where pX is between bvXMinsregionId and bvXMaxsregionId and pY is between bvYMinsregionId and bvYMaxsregionId, with a step size greater than 1, for example, with a step size of 2 (i.e., the first search step size is 2), and the optimal matching cost or costs obtained by template matching are recorded as pDiff_BEST, and the corresponding block vector or vectors BV are noted as optimal block vectors BV_BEST(pX_BEST, pY_BEST). That is, by performing a search process in a predetermined search region, one or more block vectors corresponding to the current block are obtained, and the matching template or templates corresponding to the optimal matching cost or costs may be the final candidate template or templates.

[0419] For example, in an embodiment of the present application, if the search strategy is to perform only a precise search, a precise search is performed within the search range in each region where pX is between bvXMinsregionId and bvXMaxsregionId and pY is between bvYMinsregionId and bvYMaxsregionId, for example, with a step size of 1 (i.e., the second search step size is 1), and one or more optimal matching costs obtained by template matching are recorded as pDiff_BEST, and one or more corresponding block vectors BV are noted as optimal block vectors BV_BEST(pX_BEST, pY_BEST). That is, by performing a search process in a predetermined search region, one or more block vectors corresponding to the current block are obtained, and one or more matching templates corresponding to the one or more optimal matching costs may be the final one or more candidate templates.

[0420] For example, in an embodiment of the present application, when the search strategy is to perform a coarse search followed by a fine search, as shown in FIG. 10, first, a coarse search is performed with a step size of 2 (i.e., the first search step size is 2), and an optimal coarse matching template (initial matching template) obtained by template matching is obtained; then, an optimal fine matching template is determined in the vicinity of the optimal coarse matching template with a step size of 1 (i.e., the second search step size is 1), and thus a final candidate template can be obtained.

[0421] Here, in the coarse search stage, within the search range in each region where pX is between bvXMinsregionId and bvXMaxsregionId and pY is between bvYMinsregionId and bvYMaxsregionId, a coarse search is performed with a step size greater than 1. For example, a coarse search is performed with a step size of 2, and the optimal matching cost obtained by template matching is recorded as pDiff1_BEST. The corresponding block vector BV is recorded as an initial block vector, which is the optimal block vector BV1_BEST(pX1_BEST, pY1_BEST), and the corresponding matching template is the initial matching template. In this case, the search region where the optimal matching search point is located is bestRegionId.

[0422] Next, in the fine search stage, a search may be performed near the optimal block vector BV1_BEST (initial block vector) obtained by the coarse search, i.e., a search may be performed in the first search area. Therefore, it is first necessary to determine a refined search range TmpRefineRange, i.e., a first search area TmpRefineRange. Here, the refined search range (first search area TmpRefineRange) may be a fixed size or may be related to the current block size, for example, set to min(nTbW, nTbH) / 2. Furthermore, the position of the optimal matching reconstruction block obtained by the coarse search may be calculated as the reference position of the fine search area so that BestPosX = xTbCmp + pX1_BEST and BestPosY = yTbCmp + pY1_BEST.

[0423] In one embodiment, based on the value of bestRegionId, first obtain the calculated values ​​of iVerMinbestRegionId, iVerMaxbestRegionId, iHorMinbestRegionId, and iHorMaxbestRegionId, and then obtain new search ranges iVerMinrefine, iVerMaxrefine, iHorMinrefine, and iHorMaxrefine based on the best matching block position obtained by coarse search. The obtaining method is as follows:

[0424] iHorMinrefine=max(iHorMinbestRegionId,BestPosX-TmpRefineRange) iHorMaxrefine=min(iHorMaxbestRegionId,BestPosX+TmpRefineRange) iVerMinrefine=max(iVerMinbestRegionId,BestPosY-TmpRefineRange) iVerMaxrefine=min(iVerMaxbestRegionId,BestPosY+TmpRefineRange) Then, bvXMins, bvXMaxs, bvYMins, and bvYMaxs of the adjusted block vector BV can be calculated based on iVerMinrefine, iVerMaxrefine, iHorMinrefine, and iHorMaxrefine as follows:

[0425] bvXMins=iHorMinrefine-xTbCmp bvXMaxs=iHorMaxrefine-xTbCmp bvYMins=iVerMinrefine-yTbCmp bvYMaxs=iVerMaxrefine-yTbCmp The refined search is performed within the block vector range where pX is between bvXMinsrefine and bvXMaxsrefine and pY is between bvYMinsrefine and bvYMaxsrefine. For example, the search is performed with a step size of 1, and the optimal matching cost obtained by template matching is recorded as pDiff_BEST. The corresponding block vector BV is recorded as the optimal block vector BV_BEST(pX_BEST, pY_BEST), i.e., the block vector of the finally determined current block, and the corresponding matching template is the candidate template for the current block.

[0426] After completing the above operations, the best block vector BV_BEST(pX_BEST, pY_BEST) is obtained, where pX_BEST and pY_BEST are the horizontal and vertical offset amounts of the best matching template relative to the current coding block template, respectively, that is, the horizontal and vertical offset amounts of the best matching reconstruction block relative to the current coding block.

[0427] Thus, in the embodiment of the present application, the process of searching and determining one or more block vectors in the search area may include a process of performing only a coarse search, a process of performing only a fine search, and a process of performing a fine search after the coarse search. Accordingly, if the search strategy is to perform only a coarse search, N block vectors BV corresponding to N matching templates are determined only in the coarse search process, and the corresponding N matching templates are set as N candidate templates. If the search strategy is to perform a fine search, N block vectors BV corresponding to N matching templates are determined only in the fine search process, and the corresponding N matching templates are set as N candidate templates. If the search strategy is to perform a fine search after the coarse search, K block vectors BV corresponding to K (K is an integer greater than or equal to N) preliminary matching templates (initial matching templates) are first determined in the coarse search process, and after performing a fine search process based on the K preliminary matching templates, N block vectors BV corresponding to the final N matching templates are determined, and the corresponding N matching templates are set as N candidate templates.

[0428] Furthermore, in an embodiment of the present application, when performing a search process, the search points in a sub-search area of ​​a predetermined search area may be traversed according to a first search step width to determine an initial vector corresponding to the sub-search area and a second search area corresponding to an initial block vector, multiple target sub-search areas may be determined in the sub-search area based on the initial block vector and the second search area, and the search points in the multiple target sub-search areas may be traversed according to the second search step width to determine block vectors and candidate templates, wherein the first search step width is greater than the second search step width.

[0429] In addition, in the embodiment of the present application, when determining multiple target sub-search areas in a sub-search area based on an initial block vector and a second search area, area determination can be performed based on the initial block vector, which is the optimal block vector of each sub-search area, and the corresponding second search area, to determine multiple target sub-search areas to be used in the subsequent search process.

[0430] That is, in the embodiment of the present application, in the search strategy of performing a coarse search followed by a fine search, a fine search may be performed across different search regions, and after completing the coarse search process, a method relating to each region boundary may be performed in the fine search process.

[0431] For example, in the embodiment of the present application, taking FIG. 7(a) as an example, the search area is divided into four sub-search areas, and the implementation method is as follows:

[0432] If regionId is equal to 0, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0433] iHorMax0=min((xTbCmp+searchRangeWidth)< <iBvShift,((picWidth-nTbW)<<iBvShift)) iHorMin0=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iVerMax0=(yTbCmp-nTbH-offsetLCBY)< <iBvShift iVerMin0=max(((iTemplateSizeH)< <iBvShift),((yTbCmp-searchRangeHeight)<<iBvShift)) If regionId is equal to 1, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0434] iHorMin1=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iHorMax1=(xTbCmp-offsetLCBX-nTbW)< <iBvShift iVerMin1=(yTbCmp+1)< <iBvShift iVerMax1=min(picHeight-nTbH,(yTbCmp-offsetLCBY+CtbSizeH-nTbH)< <iBvShift) If regionId is equal to 2, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0435] iHorMax2=(xTbCmp-nTbW)< <iBvShift iHorMin2=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iVerMin2=max((iTemplateSizeH)< <iBvShift,(yTbCmp-nTbH)<<iBvShift) iVerMax2=(yTbCmp)< <iBvShift If regionId is equal to 3, iVerMin, iVerMax, iHorMin, iHorMax can be calculated as follows:

[0436] iHorMin3=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift) iHorMax3=(xTbCmp)< <iBvShift iVerMin3=max(((iTemplateSizeH)< <iBvShift),(yTbCmp-offsetLCBY-nTbH+1)<<iBvShift) iVerMax3=(yTbCmp-nTbH)< <iBvShift In practical application, iHorMin regionId , iHorMax regionId , iVerMin regionId , iVerMax regionId represent the left, right, top and bottom edges of different sub-search areas, respectively.

[0437] To intuitively explain the different sub-search areas corresponding to different regionIds, R1, R2, R3, and R4 denote four different sub-search areas as shown in Figure 14. Note that Figure 14 represents the sample range in which the upper left corner sample of the block can be aligned.

[0438] In the refinement stage, the neighborhood of each block vector obtained by the coarse search may be searched, and further, the neighborhood of the optimal block vector BV1_BESTk obtained by the coarse search may be searched. Specifically, first, a refinement search range TmpRefineRange is determined. The refinement search range may be a fixed size or may be related to the size of the current block, for example, set to min(nTbW, nTbH) / 2. Then, the position of the optimal matching reconstruction block obtained by the coarse search may be calculated as the reference position of the refinement search area so that BestPosXk=xTbCmp+pX1_BESTk and BestPosYk=yTbCmp+pY1_BESTk.

[0439] In some embodiments, the refined search performed based on each BestPosXk may span multiple regions, i.e., may span different search regions. Here, first, the value of bestRegionId involved in the calculation is determined, and then the calculation is performed for regionId=0, 1, 2, 3 as follows:

[0440] BestPosYk-TmpRefineRange>=iVerMinregionId and BestPosYk-TmpRefineRange<=iVerMaxregionId, or BestPosYk+TmpRefineRange>=iVerMinregionId and BestPosYk+TmpRefineRange<=iVerMaxregionId, or If BestPosYk-TmpRefineRange<=iVerMinregionId and BestPosYk+TmpRefineRange>=iVerMaxregionId, For that regionId, set the corresponding bestSearchFlagregionId to 1.

[0441] For each search region whose bestSearchFlagregionId value is 1, bestRegionId is set as the regionId of the region in turn, and the following refined search is further performed: First, the calculated values ​​of iVerMinbestRegionId, iVerMaxbestRegionId, iHorMinbestRegionId, and iHorMaxbestRegionId are obtained based on the value of bestRegionId, and then new search ranges iVerMinrefine, iVerMaxrefine, iHorMinrefine, and iHorMaxrefine are obtained based on the optimal matching block position obtained by the coarse search. The method of obtaining these values ​​is as follows:

[0442] iHorMinrefine=max(iHorMinbestRegionId,BestPosX-TmpRefineRange) iHorMaxrefine=min(iHorMaxbestRegionId,BestPosX+TmpRefineRange) iVerMinrefine=max(iVerMinbestRegionId,BestPosY-TmpRefineRange) iVerMaxrefine=min(iVerMaxbestRegionId,BestPosY+TmpRefineRange) Then, bvXMins, bvXMaxs, bvYMins, and bvYMaxs of the adjusted block vector BV can be calculated based on iVerMinrefine, iVerMaxrefine, iHorMinrefine, and iHorMaxrefine as follows:

[0443] bvXMins=iHorMinrefine-xTbCmp bvXMaxs=iHorMaxrefine-xTbCmp bvYMins=iVerMinrefine-yTbCmp bvYMaxs=iVerMaxrefine-yTbCmp In this way, the refined search is performed within the block vector range where pX is between bvXMinsrefine and bvXMaxsrefine and pY is between bvYMinsrefine and bvYMaxsrefine. For example, a search is performed with a step size of 1, and the optimal matching cost obtained by template matching is recorded as pDiff_BEST, and the corresponding block vector BV is written as the optimal block vector BV_BEST(pX_BEST, pY_BEST).

[0444] Obtain the sum of the optimal block vectors for the multiple regions involved in the search.

[0445] After completing the above search operation, the best block vector BV_BEST(pX_BEST, pY_BEST) can be obtained, where pX_BEST and pY_BEST are respectively the horizontal offset and vertical offset of the best matching template relative to the current coding block template, and are also the horizontal offset and vertical offset of the best matching reconstruction block relative to the current coding block.

[0446] In step 203, one or more reference blocks for the current block are determined based on the one or more block vectors, and a predicted value for the current block is determined based on the one or more reference blocks.

[0447] In an embodiment of the present application, after determining one or more block vectors corresponding to a current block based on a first template, one or more reference blocks for the current block may be determined based on the one or more block vectors, and then a predicted value of the current block may be determined based on the one or more reference blocks.

[0448] It should be noted that in the embodiments of the present application, one or more reference blocks of the current block include a first reference block and / or a second reference block, where the first reference block and the current block both belong to the current image, and the second reference block belongs to a reference image of the current image corresponding to the current block.

[0449] That is, in an embodiment of the present application, one or more reference blocks of the current block may be a first reference block in the current image obtained by intra prediction, or may be a second reference block in a reference image of the current image obtained by inter prediction, or may include a first reference block in the current image obtained by intra prediction and a second reference block in a reference image of the current image obtained by inter prediction.

[0450] Accordingly, in an embodiment of the present application, when obtaining a second reference block, one or more block vectors may be determined, and then a search may be performed in a reference image of the current image to determine a second reference block corresponding to the one or more block vectors.

[0451] That is, in the embodiment of the present application, a template matching search may be performed in the current image using an intra template matching method to determine one or more reference blocks corresponding to the current block, which are first reference blocks. A template matching search may be performed in one or more inter reference images of the current image using an inter template matching method to determine one or more reference blocks corresponding to the current block, which are second reference blocks. The intra template matching method and the inter template matching method may be performed in the current image and one or more inter reference images, respectively, to determine one or more reference blocks corresponding to the current block, where the one or more reference blocks include a first reference block and a second reference block.

[0452] Furthermore, in the embodiments of the present application, when determining one or more reference blocks corresponding to the current block based on one or more block vectors, one or more initial reconstructed blocks corresponding to the current block may be determined first based on one or more block vectors, and then a correction process may be performed on the one or more initial reconstructed blocks to determine one or more reference blocks.

[0453] That is, in the embodiments of the present application, when obtaining N candidate reconstruction blocks (reference blocks), other methods may be adopted, for example, first correcting the initial reconstruction block corresponding to the obtained candidate template, and then determining the corresponding reference block.

[0454] Accordingly, N candidate reconstructed blocks (reference blocks) may be copied and obtained using N BV-compatible matching reconstructed blocks (initial reconstructed blocks), and then these N candidate reconstructed blocks may be weighted as they are to obtain a predicted value of the current block; alternatively, N BV-compatible matching reconstructed blocks (initial reconstructed blocks) may be corrected to obtain candidate reconstructed blocks (reference blocks), and then weighted to obtain a predicted value of the current block.

[0455] Furthermore, in the embodiments of the present application, when a correction process is performed on one or more initial reconstruction blocks to determine one or more reference blocks, a filtering process may be performed on one or more initial reconstruction blocks to determine one or more reference blocks.

[0456] Furthermore, in an embodiment of the present application, when performing a correction process on one or more initial reconstruction blocks to determine one or more reference blocks, one or more correction parameter vectors may be first determined based on one or more candidate templates corresponding to one or more block vectors, and then a correction process may be performed on one or more initial reconstruction blocks based on the one or more correction parameter vectors to determine one or more reference blocks.

[0457] Furthermore, in an embodiment of the present application, when determining one or more correction parameter vectors based on one or more candidate templates corresponding to one or more block vectors, an autocorrelation matrix corresponding to the candidate templates may be determined based on sample values ​​in the candidate templates, a cross-correlation vector may be determined based on sample values ​​in the first template and sample values ​​in the candidate templates, and a correction parameter vector may be determined based on the autocorrelation matrix and the cross-correlation vector.

[0458] In addition, in the embodiments of the present application, the method for correcting the initial reconstruction block may include a step of performing a filtering process on the initial reconstruction block directly, where the filtering method used when performing the process may be a conventional filtering method such as bilateral filtering, averaging filtering, etc., or may be a neural network-based filter enhancement.

[0459] It should be noted that in the embodiment of the present application, the method for correcting the initial reconstructed block may further include correcting the matching reconstructed block (initial reconstructed block) using matching template information (candidate template).

[0460] For example, in the embodiment of the present application, when the matching template information is used to perform the correction process on the matching reconstruction block, each candidate template refT n and the corresponding candidate reconstruction block (initial reconstruction block) RefBlock n For candidate template refT n and the current coding block template (first template of the current block) curT is used to calculate the correction parameter vector C n Calculate the correction parameter vector C n candidate reconstruction block RefBlock n and weighted synthesis to form the final corrected reconstructed block RefBlock' n That is, the reference block of the current block finally obtained is RefBlock' n is.

[0461] In some embodiments, the correction parameter vector C n is the candidate template refT n and the minimized MSE of the template sample values ​​to be predicted.

[0462] In the embodiment of the present application, the correction parameter vector C n may be viewed as one L-tap filter.

[0463] In some embodiments, the correction parameter vector C n When calculating each candidate template refT n For (n=0, 1..., N-1), the process of minimizing MSE inputs the autocorrelation matrix of the candidate template sample refT and the cross-correlation vector between the candidate template sample refT and the current coding block adjacent template sample curT, and outputs the weight of the candidate reconstruction block corresponding to the current candidate template.

[0464] Illustratively, in the examples of the present application, MSE is calculated as follows:

number

[0465] To make it easier to show the formula for calculating MSE, we will use E to represent the mean square error MSE as follows:

number

[0466] where K is the number of samples in the template.

[0467] In some embodiments, the weight c of the candidate reconstruction block corresponding to the candidate template refT by minimizing MSE l The process of deriving may include the following steps:

[0468] (1) First, c l Calculate the partial derivative with respect to and set it to 0.

number

[0469] After rearranging, the following formula was obtained:

number

[0470] (2) Candidate template region refT n After determining, the equation obtained in step (1) is expanded into matrix form as follows:

number

[0471] (3) The autocorrelation matrix and cross-correlation vector in step (2) are both known quantities. By solving the linear equations in (2), the filter weighting coefficients c0,...,c L-1 The correction parameter vector C n The filtering coefficients of a candidate reconstruction block in can be obtained.

[0472] Accordingly, if x=0...nTbW-1, y=0...nTbH-1, the corrected candidate reconstruction block RefBlock' n is as follows:

number

[0473] In the embodiment of the present application, after determining N block vectors BV corresponding to N candidate templates of the current block, N candidate reconstructed blocks (i.e., N reference blocks) are obtained using the N BVs, and then weighted synthesis is performed on the N candidate reconstructed blocks to obtain a predicted block of the current block (i.e., a predicted value of the current block).Here, when generating a final predicted value, first obtain the N candidate reconstructed blocks (N reference blocks), then determine the corresponding weighted synthesis weights (weight values), and then generate a predicted value of the current block through weighted synthesis processing.

[0474] In the embodiment of the present application, when determining one or more reference blocks for a current block based on one or more block vectors, N block vectors BV corresponding to the obtained N candidate templates are used. n On the other hand, BV n N candidate reconstruction blocks (i.e., reference blocks) RefBlock obtained from the current image and / or the reference image based on n You can select where BV n The horizontal offset is pX n and the vertical offset is pY nwhere n=0, 1…, N-1.

[0475] For example, in the embodiment of the present application, determining one or more reference blocks of the current block may be achieved by simple translational copying. Specifically, when x=0...nTbW-1, y=0...nTbH-1, the reconstructed samples of the current frame (i.e., the reference blocks of the current block) are determined by the following formula: RefBlock n [x][y]=recSamples[x+pX n ][y+pY n ] (15)

[0476] Furthermore, in the embodiments of the present application, when determining a predicted value of a current block based on one or more reference blocks, one or more weight values ​​corresponding to the one or more reference blocks are first determined, and then a weighted synthesis process is performed on the one or more reference blocks based on the one or more weight values, and finally a predicted value of the current block is determined.

[0477] In the embodiment of the present application, after obtaining N candidate reconstruction blocks RefBlock (i.e., reference blocks of the current block), it is necessary to calculate the weights W of the weighted combination of the N candidate reconstruction blocks. Here, the weight values ​​corresponding to the reference blocks can be determined in a number of ways. For example, the weights can be predefined values ​​(e.g., second predetermined values), or can be self-adaptively calculated values ​​using costs, sample values, etc.

[0478] In one embodiment, one or more weight values ​​may be determined based on a second predetermined value, where the second predetermined value may include N arbitrary values ​​greater than 0, and the corresponding weight values ​​for different reference blocks among the N reference blocks may be the same or different, and the present application does not specifically limit the second predetermined value.

[0479] In one embodiment, one or more weight values ​​may be determined based on one or more candidate templates corresponding to one or more block vectors, where determining one or more weight values ​​based on one or more candidate templates corresponding to one or more block vectors may involve first determining an autocorrelation matrix corresponding to the candidate templates based on sample values ​​in the candidate templates, then determining a cross-correlation vector based on sample values ​​in the first template and sample values ​​in the candidate templates, and then determining the weight values ​​based on the autocorrelation matrix and the cross-correlation vector.

[0480] In some embodiments, the candidate template refT n and the template to be predicted (first template) refpredT n The weighted combining weights (weight values) can be derived by using the minimized MSE of the sample values ​​of the vectors.

[0481] In order to make the weight derivation process more flexible, a nonlinear term and a bias term may be introduced in the process of deriving the weighted synthesis weights.

[0482] For example, in the embodiment of the present application, when deriving the weighted combining weights, the nonlinear terms are constructed based on the candidate templates, where one candidate template numbered 0 can be selected from N candidate templates for construction. Accordingly, when m=0,...,M-1, the following equation exists: NonLinearTerm_T n,m =(refT n,m *refT n,m +MidVal)≫bitDepth (16)

[0483] Here, n is 0, 1, ..., N-1, representing any template among the N candidate templates, MidVal is 1<<(bitDepth-1), and bitDepth is the image bit depth.

[0484] Exemplarily, in the embodiments of the present application, when applying weighted combined weights to each candidate reconstruction block (reference block) corresponding to N candidate templates, the non-linear term NonLinearTerm_Block is constructed based on the candidate reconstruction block. Here, the candidate reconstruction block corresponding to the candidate template numbered 0 can be selected. Accordingly, when x = 0, 1... nTbW-1 and y = 0,... nTbH-1, there is the following formula. NonLinearTerm_Block n,x,y =(refBlock n,x,y *refBlock n,x,y +MidVal)≫bitDepth (17)

[0485] Here, n is 0, 1,..., N-1, and represents the candidate reconstruction block (reference block) corresponding to any one of the N candidate templates.

[0486] ​​​​​​​​​​​​​​​​​​​​​​​​​​​In the embodiment of the present application, after adding the nonlinear term and the bias term, it is necessary to derive N+2 weighting coefficients (weight values). For convenience of explanation, the number of weighting coefficients is recorded by a variable P (P=N+2), and the matching template sample / nonlinear term sample and the bias term sample are collectively referred to as the matching reference sample refT. N and refT N+1 This allows us to collectively refer to the reference quantities involved in all operations as refT p Similarly, the reconstructed block corresponding to the matching template, the reconstructed block corresponding to the matching template related to the nonlinear term, and the bias term are collectively expressed as the candidate reconstructed sample refBlock p where p = 0,...,P-1.

[0490] For example, in the embodiment of the present application, the process of minimizing MSE inputs the autocorrelation matrix of the previous P matching reference samples refT, the cross-correlation vector between the previous P matching reference samples refT and the current coding block adjacent template samples curT, and outputs the weights of the reconstruction block corresponding to each matching reference term.

[0491] Here, the MSE calculation formula is as follows:

[0492] For each matching reference template, i.e., m=0, 1…, M-1, for the co-located samples,

number

[0493] To make it easier to show the formula for calculating MSE, we will use E to represent the mean square error MSE as follows:

number

[0494] The weights w of the reconstruction blocks corresponding to each matching reference sample are calculated using the minimized MSE. p The specific steps to derive are as follows:

[0495] (1) First, w p Calculate the partial derivative with respect to and set it to 0.

number

[0496] After rearranging, the following formula was obtained:

number

[0497] (2) Matching reference sample regions refT0, refT1, ..., refT P-1 After determining, the equation obtained in step (1) is expanded into matrix form as follows:

number

[0498] (3) The autocorrelation matrix and cross-correlation vector in step (2) are both known quantities. By solving the linear equation in (2), the weights w0,...,w of the reconstructed samples (reference blocks) corresponding to each matching reference term are obtained. P-1 can be calculated.

[0499] In one embodiment, when determining one or more weight values ​​based on one or more candidate templates corresponding to one or more block vectors, a matching cost between the first template and the candidate templates may be determined first, and then the weight values ​​may be determined based on the matching cost.

[0500] Furthermore, in the embodiments of the present application, the weighting coefficients may be calculated in other ways, for example, by using a nonlinear weighting model to calculate the weighting coefficients for each candidate reconstruction block (reference block) RefBlock based on the cost of N candidate matching templates. n Appropriate weight for n can be assigned.

[0501] It should be noted that in the embodiments of the present application, the weight model may include, but is not limited to, a non-linear normalization function, a non-linear exponential normalization function, and the like.

[0502] For example, in an embodiment of the present application, the weight of each candidate reconstruction block (reference block) may be calculated by the following nonlinear function, where the input of the weight model is the current coding block template curT and the candidate template refT: n The matching cost is the matching cost between the current coding block template curT and the candidate template refT n SAD(refT n ), MAD(refT n ), correlation coefficient R(refT n ) and the like, but are not limited to these.

number

[0503] Here, offset is a predetermined value, for example, offset is 1.

[0504] In some embodiments, the matching cost is calculated by the normalized correlation coefficient R(refT n ), the corresponding formula for the weight model is:

number

[0505] In some embodiments, a Softmax function may be used as the weight model, and the calculation formula is as follows:

number

[0506] Here, S is a model control parameter, and under a predetermined condition, the weight model can be adjusted by adjusting the parameter S. For example, the parameter S may be related to the current block size or the template type.

[0507] In some embodiments, instead of the above nonlinear weighting model, the weighting coefficients of each candidate reconstruction block may be directly averaged as follows, for example.

number

[0508] In some embodiments, the candidate template refT n and the template to be predicted (first template) refpredT n In the process of deriving the weighted combining weights (weight values) by using the minimized MSE of the sample values ​​of the vectors, it is not necessary to add a nonlinear term and a bias term.

[0509] For example, in the embodiment of the present application, if no nonlinear term and bias term are added, only N weight coefficients (weight values) need to be derived, where the process of minimizing the MSE inputs the autocorrelation matrix of the previous N candidate template samples refT and the cross-correlation vector between the previous P candidate template samples refT and the current coding block adjacent template samples curT, and outputs the weights of the candidate reconstruction blocks corresponding to each candidate template.

[0510] In some embodiments, for co-located samples of each candidate template, ie, m=0, 1 . . . , M-1, the MSE calculation formula is:

number

[0511] To make it easier to show the formula for calculating MSE, we will use E to represent the mean square error MSE as follows:

number

[0512] The weights w of the candidate reconstruction blocks corresponding to each candidate template are calculated using the minimized MSE. n The specific steps to obtain are as follows:

[0513] (1) First, w n Calculate the partial derivative with respect to and set it to 0.

number

[0514] After rearranging, the following formula was obtained:

number

[0515] (2) Candidate template regions refT0, refT1, ..., refT N-1 After determining, the equation obtained in (1) is expanded into matrix form as follows:

number

[0516] (3) The autocorrelation matrix and cross-correlation vector in step (2) are both known quantities. By solving the linear equations in (2), the weights w0,...w of the reconstructed samples corresponding to each candidate template are obtained.N-1 can be calculated.

[0517] In some embodiments, the candidate template refT n and the template to be predicted (first template) refpredT n In the process of deriving the weighted combining weights (weight values) by using the minimized MSE of the sample values ​​of the above, only the bias term may be added, and no nonlinear term may be added.

[0518] For example, in the embodiment of the present application, if only the bias term is added and no nonlinear term is added, N+1 weighting coefficients (weight values) need to be derived. For convenience of explanation, the number of weighting coefficients is recorded by a variable P (P=N+1), and the candidate template sample / bias term sample is collectively referred to as the matching reference sample refT. N and refT N+1 This allows us to collectively refer to the reference quantities involved in all operations as refT p Similarly, the candidate reconstructed block corresponding to the candidate template and the bias term can be collectively expressed as the candidate reconstructed sample refBlock p where p=0, 1..., P-1.

[0519] In some embodiments, the process of minimizing MSE inputs the autocorrelation matrix of the previous P matching reference samples refT, the cross-correlation vector between the previous P matching reference samples refT and the current coding block neighboring template samples curT, and outputs the weights of the reconstruction block corresponding to each matching reference term.

[0520] In some embodiments, for co-located samples of each candidate template, ie, m=0, 1 . . . , M-1, the MSE calculation formula is:

number

[0521] To make it easier to show the formula for calculating MSE, we will use E to represent the mean square error MSE as follows:

number

[0522] The weights w of the reconstruction blocks corresponding to each matching reference sample are calculated using the minimized MSE. p The specific steps to derive are as follows:

[0523] (1) First, w p Calculate the partial derivative with respect to and set it to 0.

number

[0524] After rearranging, the following formula was obtained:

number

[0525] (2) Matching reference sample regions refT0, refT1, …, refT P-1 After determining, the equation obtained in step (1) is expanded into matrix form as follows:

number

[0526] (3) The autocorrelation matrix and cross-correlation vector in step (2) are both known quantities. By solving the linear equations in (2), the weights w0,...w of the reconstructed samples corresponding to each matching reference term are obtained. P-1 can be calculated.

[0527] In an embodiment of the present application, one or more reference blocks of a current block are determined based on one or more block vectors, a predicted value of the current block is determined based on the one or more reference blocks, and then a predicted residual value of the current block is determined based on the predicted value of the current block, and the predicted residual value is written into a bitstream.Therefore, a decoding device can determine a predicted residual value corresponding to the current block by decoding the bitstream, and then determine a reconstructed value of the current block based on the predicted residual value and the predicted value.

[0528] As described above, the coding method proposed in steps 201 to 204 is improved and optimized based on the general IntraTMP technology, and a weighted synthesis method is adopted to propose an IntraTMP Fusion prediction method. Here, in the process of searching and determining a block vector BV in the search area, at least one block vector of the current block can be determined, that is, multiple block vectors can be determined. In addition, in the configuration for generating a predicted value, a weighted synthesis process is performed using at least one reference block corresponding to at least one block vector to obtain a predicted value of the current block, and then a reconstructed value of the current block is determined.

[0529] That is, the embodiment of the present application proposes IntraTMP Fusion technology, in which the current coding block template (the first template of the current block) searches for N block vectors BV corresponding to N candidate matching templates with the smallest matching cost within a predefined range, and then finds N candidate reconstruction blocks (N reference blocks) corresponding to these N candidate matching templates (N candidate templates) using the N BVs, and then performs weighted synthesis with a predetermined weight to obtain a predicted block (predicted value) of the current block.

[0530] The IntraTMP Fusion method proposed in the embodiments of the present application can improve the accuracy of prediction values. Here, the current coding block template (first template of the current block) is determined by searching for N block vectors BV corresponding to N candidate matching templates with the minimum matching cost within a predefined range, and then using the N BVs to find N candidate reconstructed blocks (N reference blocks) corresponding to these N candidate matching templates (N candidate templates). Then, weighted synthesis is performed with predetermined weights to determine the predicted block of the current block. Rather than simply considering the reconstructed block corresponding to the template with the minimum matching cost, the method fully considers information about reconstructed blocks corresponding to different matching templates during the search process. Furthermore, weights are self-adaptively assigned to the candidate reconstructed blocks according to the matching template information, fully considering the difference in importance of different reconstructed block information for the prediction of the current block.

[0531] In addition, the IntraTMP Fusion method proposed in the embodiments of the present application can fully utilize candidate reconstruction block information corresponding to different matching templates in the process of searching for a matching template. Rather than simply considering reconstruction block information corresponding to the template with the smallest matching cost, the method fully utilizes reconstruction block information corresponding to different matching templates in the search process. In addition, the matching template information is fully utilized to self-adaptively assign weights to candidate reconstruction blocks, fully considering the difference in importance of different reconstruction block information for predicting the current block.

[0532] In this way, the IntraTMP Fusion method according to the embodiment of the present application can avoid to some extent the degradation of prediction accuracy caused by inaccurate template matching criteria or by directly copying a reconstructed block as a predicted block.

[0533] Compared to common encoding / decoding techniques, the IntraTMP Fusion method according to the embodiment of this application can achieve BD-rate changes (i.e., average bitrate changes at equivalent psnr) of -0.29%, -0.30%, and -0.39% for Y, Cb, and Cr, respectively, when tested under All Intra conditions at 24-frame intervals.

[0534] According to an encoding method of an embodiment of the present application, an encoding device determines a first template corresponding to a current block, determines one or more block vectors corresponding to the current block based on the first template, determines one or more reference blocks for the current block based on the one or more block vectors, and determines a predicted value of the current block based on the one or more reference blocks. Thus, the embodiment of the present application provides an IntraTMP Fusion prediction method, which can select and determine at least one block vector for the current block, and can obtain a predicted value of the current block using at least one reference block corresponding to the at least one block vector. In other words, the encoding / decoding method of an embodiment of the present application fully considers the difference in importance of reconstructed block information corresponding to different matching templates in the search process to predict the current block, thereby improving prediction accuracy and achieving optimal prediction results.

[0535] For another embodiment of the present application, please refer to Figure 16. Figure 16 is a structural diagram of an encoding device according to an embodiment of the present application. As shown in Figure 16, the encoding device 180 includes a first determination unit 1801.

[0536] The first determination unit 1801 is configured to determine a first template corresponding to a current block, determine one or more block vectors corresponding to the current block based on the first template, determine one or more reference blocks for the current block based on the one or more block vectors, and determine a predicted value of the current block based on the one or more reference blocks.

[0537] In the embodiments of the present application, a "unit" may be a part of a circuit, a part of a processor, a part of a program or software, etc., and may of course be a module or may be non-modular. Furthermore, each component in the present embodiment may be integrated into a single processing unit, each unit may exist physically independently, or two or more units may be integrated into a single unit. The integrated unit may be implemented as hardware or as a software functional module.

[0538] When the integrated unit is implemented as a software functional module rather than being sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on this, a substantial part of the technical solution of the present embodiment, in other words, a part that contributes to the prior art, may be implemented as a software product, or the technical solution may be implemented entirely or partially as a software product. The computer software product is stored in a storage medium and includes instructions for causing a computer device (such as a personal computer, a server, or a network device) or a processor to execute all or part of the steps of the method of the present embodiment. The storage medium includes various media capable of storing program code, such as a USB disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0539] Therefore, an embodiment of the present application provides a computer-readable storage medium applied to the encoding device 180, the computer-readable storage medium storing a computer program, which, when executed by the first processor, realizes the method described in any one of the above embodiments.

[0540] For the configuration of the encoding device 180 and a computer-readable storage medium, please refer to FIG. 17. FIG. 17 is a schematic diagram of a specific hardware structure of the encoding device 180 according to an embodiment of the present application. As shown in FIG. 17, the encoding device 180 may include a first communication interface 1901, a first memory 1902, and a first processor 1903. Each component is coupled via a first bus system 1904. The first bus system 1904 is configured to realize communication between these components. The first bus system 1904 may further include a power bus, a control bus, and a status signal bus in addition to a data bus. However, for clarity, all various buses are shown as the first bus system 1904 in FIG. 17.

[0541] The first communication interface 1901 is configured to receive and transmit signals in the process of transmitting and receiving information to and from other external network entities.

[0542] The first memory 1902 is configured to store a computer program executable by the first processor 1903 .

[0543] The first processor 1903 executes the computer program. determining a first template corresponding to the current block; determining one or more block vectors corresponding to the current block based on the first template; and determining one or more reference blocks for the current block based on the one or more block vectors; and determining a prediction value for the current block based on the one or more reference blocks.

[0544] It should be noted that first memory 1902 in the present embodiment may be volatile memory, nonvolatile memory, or both volatile and nonvolatile memory. Nonvolatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM) used as an external cache. By way of example, various RAMs may be used, including, but not limited to, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (Synchlink DRAM, SLDRAM), and direct Rambus random access memory (DRRAM). The first memory 1902 of the present systems and methods may include, but is not limited to, these and any other suitable types of memory.

[0545] The first processor 1903 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by a hardware integrated logic circuit or software instructions in the first processor 1903. The first processor 1903 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate, a transistor logic device, or a discrete hardware component, and can implement or execute each method, step, and logic block diagram described in the embodiments of the present application. The general-purpose processor may be a microprocessor or any general processor. The steps based on the method described in the embodiments of the present application may be performed directly by a hardware decoding processor, or may be performed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium well-established in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, or a register. The storage medium is located in a first memory 1902, and a first processor 1903 reads information in the first memory 1902 and completes the steps of the above method in combination with its hardware.

[0546] It should be noted that the embodiments described herein may be implemented by hardware, software, firmware, middleware, microcode, or a combination thereof. For a hardware implementation, the processing unit may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processing devices (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units, or a combination thereof, for performing the functions described herein. For a software implementation, the techniques described herein may be implemented by modules (e.g., processes, functions, etc.) that perform the functions described herein. The software code may be stored in a memory and executed by a processor. The memory may be implemented within the processor or external to the processor.

[0547] Optionally, as another embodiment, the first processor 1903 is further configured to execute the method according to any one of the above embodiments when executing a computer program.

[0548] For another embodiment of the present application, please refer to Fig. 18. Fig. 18 is a structural diagram of a decoding device according to an embodiment of the present application. As shown in Fig. 18, the decoding device 200 may include a second determination unit 2001.

[0549] The second determination unit 2001 is configured to determine a first template corresponding to a current block, determine one or more block vectors corresponding to the current block based on the first template, determine one or more reference blocks for the current block based on the one or more block vectors, determine a predicted value of the current block based on the one or more reference blocks, and determine a reconstructed value of the current block based on the predicted value of the current block.

[0550] In the embodiments of the present application, a "unit" may be a part of a circuit, a part of a processor, a part of a program or software, etc., and may of course be a module or may be non-modular. Furthermore, each component in the present embodiment may be integrated into a single processing unit, each unit may exist physically independently, or two or more units may be integrated into a single unit. The integrated unit may be implemented as hardware or as a software functional module.

[0551] When the above-mentioned integrated unit is realized as a software functional module rather than being sold or used as an independent product, it may be stored in one computer-readable storage medium. On this basis, this embodiment further provides a computer-readable storage medium applied to the decoding device 200, the computer-readable storage medium storing a computer program, which, when executed by the second processor, realizes the method described in any one of the above embodiments.

[0552] The configuration of the decoding device 200 and a computer-readable storage medium are shown in FIG. 19. FIG. 19 is a schematic diagram of a specific hardware structure of the decoding device 200 according to an embodiment of the present application. As shown in FIG. 19, the decoding device 200 may include a second communication interface 2201, a second memory 2202, and a second processor 2203. Each component is coupled via a second bus system 2204. The second bus system 2204 is configured to realize communication between these components. The second bus system 2204 may further include a power bus, a control bus, and a status signal bus in addition to a data bus. However, for clarity, all of the various buses are shown as the second bus system 2204 in FIG. 19.

[0553] The second communication interface 2201 is configured to receive and transmit signals in the process of transmitting and receiving information to and from other external network entities.

[0554] The second memory 2202 is configured to store a computer program executable by the second processor 2203 .

[0555] When the second processor 2203 executes the computer program, determining a first template corresponding to the current block; determining one or more block vectors corresponding to the current block based on the first template; determining one or more reference blocks for the current block based on the one or more block vectors, and determining a prediction value for the current block based on the one or more reference blocks; and determining a reconstructed value of the current block based on the predicted value of the current block.

[0556] Optionally, as another embodiment, the second processor 2203 is further configured to, when executing a computer program, perform the method according to any one of the above embodiments.

[0557] The second memory 2202 has similar hardware functions to the first memory 1902, and the second processor 2203 has similar hardware functions to the first processor 1903, so detailed description thereof will be omitted here.

[0558] This embodiment provides an encoding / decoding device. The encoding device determines a first template corresponding to a current block, determines one or more block vectors corresponding to the current block based on the first template, determines one or more reference blocks for the current block based on the one or more block vectors, determines a predicted value of the current block based on the one or more reference blocks, and determines a reconstructed value of the current block based on the predicted value of the current block. The decoding device determines a first template corresponding to the current block, determines one or more block vectors corresponding to the current block based on the first template, determines one or more reference blocks for the current block based on the one or more block vectors, and determines a predicted value of the current block based on the one or more reference blocks. Thus, this embodiment provides a prediction method for IntraTMP Fusion, which can select and determine at least one block vector for the current block and further obtain a predicted value of the current block using at least one reference block corresponding to the at least one block vector. In other words, the encoding / decoding method according to the embodiment of the present application fully considers the difference in importance of the reconstruction block information corresponding to different matching templates in the search process to the prediction of the current block, thereby improving the accuracy of prediction and achieving optimal prediction results.

[0559] For another embodiment of the present application, please refer to Fig. 20. Fig. 20 is a schematic diagram of the configuration structure of an encoding / decoding system according to an embodiment of the present application. As shown in Fig. 20, the encoding / decoding system 230 may include an encoding device 2301 and a decoding device 2302.

[0560] In the embodiments of the present application, the encoding device 2301 may be the encoding device described in any one of the above embodiments, and the decoding device 2302 may be the decoding device described in any one of the above embodiments.

[0561] In addition, an embodiment of the present application further provides a bitstream, where the bitstream is generated by bit encoding based on encoding target information, and the encoding target information includes at least one of a predicted difference value of a current block, a predetermined number N, and one or more block vectors.

[0562] It should be noted that in this application, terms such as "comprises," "having," or any other variation thereof are intended to cover a non-exclusive inclusion, whereby a process, method, article, or apparatus comprising a set of elements further includes those elements as well as other elements not expressly listed or inherent in such process, method, article, or apparatus. Unless further limited, for elements defined by phrases such as "comprises a," additional identical elements may be present in the process, method, article, or apparatus that comprises the element.

[0563] The numbers of the examples in the present application above are for illustrative purposes only and do not indicate the superiority or inferiority of the examples.

[0564] The methods disclosed in the various method embodiments of this application may be combined in any manner consistent with one another to arrive at new method embodiments.

[0565] The features disclosed in the various article embodiments of the present application may be combined in any manner consistent with each other to obtain new article embodiments.

[0566] The features disclosed in any method or apparatus embodiment of the present application may be combined in any manner consistent with each other to produce a new method or apparatus embodiment.

[0567] The above are only specific embodiments of the present application, and the scope of protection of the present application is not limited thereto. Any modifications or replacements that can be easily conceived by those skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application is subject to the scope of protection of the claims. [Industrial Applicability]

[0568] The present application provides an encoding / decoding method, an encoding device, a decoding device, and a storage medium. The encoding device determines a first template corresponding to a current block, determines one or more block vectors corresponding to the current block based on the first template, determines one or more reference blocks for the current block based on the one or more block vectors, determines a predicted value of the current block based on the one or more reference blocks, and determines a reconstructed value of the current block based on the predicted value of the current block. The decoding device determines a first template corresponding to the current block, determines one or more block vectors corresponding to the current block based on the first template, determines one or more reference blocks for the current block based on the one or more block vectors, and determines a predicted value of the current block based on the one or more reference blocks. Thus, the present application provides an IntraTMP Fusion prediction method, which can select and determine at least one block vector for the current block and further obtain a predicted value of the current block using at least one reference block corresponding to the at least one block vector. In other words, the encoding / decoding method according to the embodiment of the present application fully considers the difference in importance of the reconstruction block information corresponding to different matching templates in the search process to the prediction of the current block, thereby improving the accuracy of prediction and achieving optimal prediction results.

Claims

1. A decoding method applied to a decoding device, comprising: determining a first template corresponding to the current block; determining one or more block vectors corresponding to the current block based on the first template; determining one or more reference blocks for the current block based on the one or more block vectors, and determining a predicted value for the current block based on the one or more reference blocks; and determining a reconstructed value of the current block based on a predicted value of the current block; Decryption method.

2. The step of determining one or more block vectors corresponding to the current block based on the first template includes: determining a predetermined search area based on the first template; and performing a search within the predetermined search area to determine the one or more block vectors; The decoding method of claim 1 .

3. The step of searching within the predetermined search area to determine the one or more block vectors comprises: traversing search points within the predetermined search area and determining a matching cost between the first template and a matching template corresponding to the search points within the predetermined search area based on a predetermined matching criterion; and determining the one or more block vectors and one or more candidate templates corresponding to the one or more block vectors based on the matching costs; The decoding method according to claim 2 .

4. The decoding method comprises: determining a predetermined number N corresponding to the candidate templates, where N is an integer greater than 0; traversing search points within the predetermined search area and determining a matching cost between the first template and a matching template corresponding to the search points within the predetermined search area based on the predetermined matching criteria; and determining N block vectors and N candidate templates corresponding to the N block vectors based on the matching costs; The decoding method according to claim 3.

5. The step of determining a predetermined number N corresponding to the candidate templates comprises: decoding the bitstream to determine N; The decoding method according to claim 4.

6. The step of determining a predetermined number N corresponding to the candidate templates comprises: determining N based on a first predetermined value or range of values; The decoding method according to claim 4.

7. The step of determining N block vectors and N candidate templates corresponding to the N block vectors based on the matching costs includes: determining N minimum matching costs from matching costs between the first template and matching templates corresponding to search points within the predetermined search area; and determining the N block vectors and the N candidate templates corresponding to the N minimum matching costs; The decoding method according to claim 4.

8. the predetermined matching criterion comprises any one of the following: sum of absolute differences (SAD), sum of absolute transformed differences (SATD), sum of squared residuals (SSE), mean absolute deviation (MAD), mean absolute error (MAE), mean squared error (MSE), normalized correlation coefficient (NCC); The decoding method according to claim 4.

9. The decoding method comprises: traversing search points within the predetermined search area according to a first search step width to determine the block vectors and the candidate templates; or traversing search points within the predetermined search area according to a second search step width to determine the block vector and the candidate template; The first search step width is greater than the second search step width.

5. The decoding method according to claim 3 or 4.

10. The decoding method comprises: traversing search points within the predetermined search area according to a first search step width to determine an initial block vector and an initial matching template corresponding to the initial block vector; determining a first search area based on the initial matching template, the first search area being smaller than the predetermined search area; and traversing search points within the first search area according to a second search step width to determine the block vectors and the candidate templates; The first search step width is greater than the second search step width.

5. The decoding method according to claim 3 or 4.

11. The decoding method comprises: traversing search points within a sub-search area of ​​the predetermined search area according to a first search step width to determine an initial vector corresponding to the sub-search area and a second search area corresponding to the initial vector block; determining a plurality of target sub-search areas within the sub-search area based on the initial vector block and the second search area; and traversing search points within the plurality of target sub-search areas according to a second search step width to determine the block vectors and the candidate templates; The first search step width is greater than the second search step width.

6. The decoding method according to claim 4 or 5.

12. the one or more reference blocks include a first reference block and / or a second reference block; The first reference block belongs to the current image together with the current block; the second reference block belongs to a reference image of the current image corresponding to the current block; The decoding method of claim 1 .

13. The decoding method comprises: decoding the bitstream to determine the one or more block vectors; and determining the second reference block corresponding to the one or more block vectors by searching in a reference image of the current image; The decoding method according to claim 12.

14. The step of determining one or more reference blocks corresponding to the current block based on the one or more block vectors includes: determining one or more initial reconstructed blocks corresponding to the current block based on the one or more block vectors; and performing a correction process on the one or more initial reconstruction blocks to determine the one or more reference blocks; The decoding method of claim 1 .

15. The step of determining the one or more reference blocks by performing a correction process on the one or more initial reconstruction blocks includes: performing a filtering process on the one or more initial reconstruction blocks to determine the one or more reference blocks; 15. The decoding method of claim 14.

16. The step of determining the one or more reference blocks by performing a correction process on the one or more initial reconstruction blocks includes: determining one or more correction parameter vectors based on one or more candidate templates corresponding to the one or more block vectors; and performing a correction process on the one or more initial reconstruction blocks based on the one or more correction parameter vectors to determine the one or more reference blocks; 15. The decoding method of claim 14.

17. determining one or more correction parameter vectors based on one or more candidate templates corresponding to the one or more block vectors, determining an autocorrelation matrix corresponding to the candidate template based on sample values ​​in the candidate template; determining a cross-correlation vector based on sample values ​​in the first template and sample values ​​in the candidate template; and determining the correction parameter vector based on the autocorrelation matrix and the cross-correlation vector; 17. The decoding method of claim 16.

18. The step of determining a predicted value of the current block based on the one or more reference blocks comprises: determining one or more weight values ​​corresponding to the one or more reference blocks; and performing a weighted synthesis process on the one or more reference blocks based on the one or more weight values ​​to determine a predicted value of the current block; The decoding method of claim 1 .

19. The step of determining one or more weight values ​​corresponding to the one or more reference blocks includes: determining the one or more weight values ​​based on a second predetermined value; 20. The decoding method of claim 18.

20. The step of determining one or more weight values ​​corresponding to the one or more reference blocks includes: determining the one or more weight values ​​based on one or more candidate templates corresponding to the one or more block vectors; 20. The decoding method of claim 18.

21. determining the one or more weight values ​​based on one or more candidate templates corresponding to the one or more block vectors, determining an autocorrelation matrix corresponding to the candidate template based on sample values ​​in the candidate template; determining a cross-correlation vector based on sample values ​​in the first template and sample values ​​in the candidate template; and determining the weight values ​​based on the autocorrelation matrix and the cross-correlation vector; 21. The decoding method of claim 20.

22. determining the one or more weight values ​​based on one or more candidate templates corresponding to the one or more block vectors, determining a matching cost between the first template and the candidate template; and determining the weight value based on the matching cost; 21. The decoding method of claim 20.

23. The step of determining a first template corresponding to the current block includes: determining a template type corresponding to the current block; and determining a first template corresponding to the current block based on the template type; The decoding method of claim 1 .

24. The step of determining a template type corresponding to the current block includes: determining a template type of the current block based on a reference sample of the current block; the reference samples of the current block include at least one of a left neighboring reference sample of the current block, an upper neighboring reference sample of the current block, an upper left neighboring reference sample of the current block, a lower left neighboring reference sample of the current block, and an upper right neighboring reference sample of the current block; 24. The decoding method of claim 23.

25. The step of determining a template type of the current block based on a reference sample of the current block includes: determining a template type of the current block as a first value when the left neighboring reference sample of the current block, the upper neighboring reference sample of the current block, and the upper left neighboring reference sample of the current block are all available; determining a template type of the current block as a second value if a left-neighboring reference sample of the current block is available; determining a template type of the current block as a third value if the upper neighboring reference sample of the current block is available; determining a template type of the current block as a fourth value if the left neighboring reference sample of the current block and the upper-left neighboring reference sample of the current block are all available; determining a template type of the current block as a fifth value if the left neighboring reference sample of the current block and the lower-left neighboring reference sample of the current block are all available; and if the upper neighboring reference sample of the current block and the upper right neighboring reference sample of the current block are all available, determining the template type of the current block as a sixth value; 24. The decoding method of claim 23.

26. The step of determining a first template corresponding to the current block based on the template type includes: determining a template reference sample for the current block based on the template type and a template size corresponding to the template type; and determining a first template for the current block based on the template reference samples; The decoding method of claim 1 .

27. The step of determining a reconstructed value of the current block based on a predicted value of the current block comprises: decoding a bitstream to determine a prediction difference value corresponding to the current block; and determining a reconstructed value for the current block based on the predicted difference value and the predicted value; The decoding method of claim 1 .

28. An encoding method used in an encoding device, comprising: determining a first template corresponding to the current block; determining one or more block vectors corresponding to the current block based on the first template; and determining one or more reference blocks for the current block based on the one or more block vectors, and determining a predicted value for the current block based on the one or more reference blocks; Encoding method.

29. The step of determining one or more block vectors corresponding to the current block based on the first template includes: determining a predetermined search area based on the first template; and performing a search within the predetermined search area to determine the one or more block vectors; 29. The encoding method of claim 28.

30. The step of searching within the predetermined search area to determine the one or more block vectors comprises: traversing search points within the predetermined search area and determining a matching cost between the first template and a matching template corresponding to the search points within the predetermined search area based on a predetermined matching criterion; and determining the one or more block vectors and one or more candidate templates corresponding to the one or more block vectors based on the matching costs; 29. The encoding method of claim 28.

31. The encoding method comprises: determining a predetermined number N corresponding to the candidate templates, where N is an integer greater than 0; traversing search points within the predetermined search area and determining a matching cost between the first template and a matching template corresponding to the search points within the predetermined search area based on the predetermined matching criteria; and determining N block vectors and N candidate templates corresponding to the N block vectors based on the matching costs; 31. The encoding method of claim 30.

32. The step of determining a predetermined number N corresponding to the candidate templates comprises: determining N according to predetermined criteria; 32. The encoding method of claim 31.

33. The step of determining a predetermined number N corresponding to the candidate templates comprises: determining N based on a first predetermined value or range of values; The encoding method according to claim 4.

34. The step of determining N block vectors and N candidate templates corresponding to the N block vectors based on the matching costs includes: determining N minimum matching costs from matching costs between the first template and matching templates corresponding to search points within the predetermined search area; and determining the N block vectors and the N candidate templates corresponding to the N minimum matching costs; The encoding method according to claim 4.

35. the predetermined matching criterion comprises any one of the following: sum of absolute differences (SAD), sum of absolute transformed differences (SATD), sum of squared residuals (SSE), mean absolute deviation (MAD), mean absolute error (MAE), mean squared error (MSE), normalized correlation coefficient (NCC); The encoding method according to claim 4.

36. The encoding method comprises: traversing search points within the predetermined search area according to a first search step width to determine the block vectors and the candidate templates; or traversing search points within the predetermined search area according to a second search step width to determine the block vector and the candidate template; The first search step width is greater than the second search step width.

5. The encoding method according to claim 3 or 4.

37. The encoding method comprises: traversing search points within the predetermined search area according to a first search step width to determine an initial block vector and an initial matching template corresponding to the initial block vector; determining a first search area based on the initial matching template, the first search area being smaller than the predetermined search area; and traversing search points within the first search area according to a second search step width to determine the block vectors and the candidate templates; The first search step width is greater than the second search step width.

5. The encoding method according to claim 3 or 4.

38. The encoding method comprises: traversing search points within a sub-search area of ​​the predetermined search area according to a first search step width to determine an initial vector corresponding to the sub-search area and a second search area corresponding to the initial vector block; determining a plurality of target sub-search areas within the sub-search area based on the initial vector block and the second search area; and traversing search points within the plurality of target sub-search areas according to a second search step width to determine the block vectors and the candidate templates; The first search step width is greater than the second search step width.

6. The encoding method according to claim 4 or 5.

39. the one or more reference blocks include a first reference block and / or a second reference block; The first reference block belongs to the current image together with the current block; the second reference block belongs to a reference image of the current image corresponding to the current block; The encoding method of claim 1 .

40. The encoding method comprises: determining the one or more block vectors; and performing a search within a reference image of the current image to determine the second reference block corresponding to the one or more block vectors; The encoding method of claim 12.

41. The step of determining one or more reference blocks corresponding to the current block based on the one or more block vectors includes: determining one or more initial reconstructed blocks corresponding to the current block based on the one or more block vectors; and performing a correction process on the one or more initial reconstruction blocks to determine the one or more reference blocks; The encoding method of claim 1 .

42. The step of determining the one or more reference blocks by performing a correction process on the one or more initial reconstruction blocks includes: performing a filtering process on the one or more initial reconstruction blocks to determine the one or more reference blocks; 15. The encoding method of claim 14.

43. The step of determining the one or more reference blocks by performing a correction process on the one or more initial reconstruction blocks includes: determining one or more correction parameter vectors based on one or more candidate templates corresponding to the one or more block vectors; and performing a correction process on the one or more initial reconstruction blocks based on the one or more correction parameter vectors to determine the one or more reference blocks; 15. The encoding method of claim 14.

44. determining one or more correction parameter vectors based on one or more candidate templates corresponding to the one or more block vectors, determining an autocorrelation matrix corresponding to the candidate template based on sample values ​​in the candidate template; determining a cross-correlation vector based on sample values ​​in the first template and sample values ​​in the candidate template; and determining the correction parameter vector based on the autocorrelation matrix and the cross-correlation vector; 17. The encoding method of claim 16.

45. The step of determining a predicted value of the current block based on the one or more reference blocks comprises: determining one or more weight values ​​corresponding to the one or more reference blocks; and performing a weighted synthesis process on the one or more reference blocks based on the one or more weight values ​​to determine a predicted value of the current block; The encoding method of claim 1 .

46. The step of determining one or more weight values ​​corresponding to the one or more reference blocks includes: determining the one or more weight values ​​based on a second predetermined value; 20. The encoding method of claim 18.

47. The step of determining one or more weight values ​​corresponding to the one or more reference blocks includes: determining the one or more weight values ​​based on one or more candidate templates corresponding to the one or more block vectors; 20. The encoding method of claim 18.

48. determining the one or more weight values ​​based on one or more candidate templates corresponding to the one or more block vectors, determining an autocorrelation matrix corresponding to the candidate template based on sample values ​​in the candidate template; determining a cross-correlation vector based on sample values ​​in the first template and sample values ​​in the candidate template; and determining the weight values ​​based on the autocorrelation matrix and the cross-correlation vector; 21. The encoding method of claim 20.

49. determining the one or more weight values ​​based on one or more candidate templates corresponding to the one or more block vectors, determining a matching cost between the first template and the candidate template; and determining the weight value based on the matching cost; 21. The encoding method of claim 20.

50. The step of determining a first template corresponding to the current block includes: determining a template type corresponding to the current block; and determining a first template corresponding to the current block based on the template type; The encoding method of claim 1 .

51. The step of determining a template type corresponding to the current block includes: determining a template type of the current block based on a reference sample of the current block; the reference samples of the current block include at least one of a left neighboring reference sample of the current block, an upper neighboring reference sample of the current block, an upper left neighboring reference sample of the current block, a lower left neighboring reference sample of the current block, and an upper right neighboring reference sample of the current block; 24. The encoding method of claim 23.

52. The step of determining a template type of the current block based on a reference sample of the current block includes: determining a template type of the current block as a first value when the left neighboring reference sample of the current block, the upper neighboring reference sample of the current block, and the upper left neighboring reference sample of the current block are all available; determining a template type of the current block as a second value if a left-neighboring reference sample of the current block is available; determining a template type of the current block as a third value if the upper neighboring reference sample of the current block is available; determining a template type of the current block as a fourth value if the left neighboring reference sample of the current block and the upper-left neighboring reference sample of the current block are all available; determining a template type of the current block as a fifth value if the left neighboring reference sample of the current block and the lower-left neighboring reference sample of the current block are all available; and if the upper neighboring reference sample of the current block and the upper right neighboring reference sample of the current block are all available, determining the template type of the current block as a sixth value; 24. The encoding method of claim 23.

53. The step of determining a first template corresponding to the current block based on the template type includes: determining a template reference sample for the current block based on the template type and a template size corresponding to the template type; and determining a first template for the current block based on the template reference samples; The encoding method of claim 1 .

54. The encoding method comprises: determining a prediction difference value of the current block based on the prediction value of the current block, and writing the prediction difference value to a bitstream; The encoding method of claim 1 .

55. A bitstream comprising: the bit stream is generated by bit coding of information to be coded; The encoding target information includes at least one of a prediction difference value of a current block, a predetermined number N, and one or more block vectors. Bitstream.

56. 1. An encoding device, comprising: a first determination unit; The first determination unit is configured to determine a first template corresponding to a current block, determine one or more block vectors corresponding to the current block based on the first template, determine one or more reference blocks for the current block based on the one or more block vectors, and determine a predicted value of the current block based on the one or more reference blocks. Encoding device.

57. 1. An encoding device, comprising: a first memory and a first processor; the first memory is configured to store a computer program executable by the first processor; The first processor is configured to perform the encoding method according to any one of claims 28 to 54 when executing the computer program. Encoding device.

58. A decoding device, comprising: a second determination unit; the second determination unit is configured to determine a first template corresponding to a current block, determine one or more block vectors corresponding to the current block based on the first template, determine one or more reference blocks for the current block based on the one or more block vectors, determine a predicted value of the current block based on the one or more reference blocks, and determine a reconstructed value of the current block based on the predicted value of the current block. Decryption device.

59. A decoding device, comprising: a second memory and a second processor; the second memory is configured to store a computer program executable by the second processor; The second processor is configured to execute the decoding method according to any one of claims 1 to 27 when executing the computer program. Decryption device.

60. 1. A computer-readable storage medium, comprising: The computer-readable storage medium stores a computer program; The computer program, when executed, implements the decoding method according to any one of claims 1 to 27 or the encoding method according to any one of claims 28 to 54. A computer-readable storage medium.

Images