Encoding and decoding method, code stream, encoder, decoder and storage medium

Abstract

The embodiment of the invention discloses an encoding and decoding method, a code stream, an encoder, a decoder and a storage medium. The method comprises the steps that the encoder determines a first template corresponding to a current block; determining one or more block vectors corresponding to the current block according to the first template; determining one or more reference blocks of the current block according to the one or more block vectors, and determining a predicted value of the current block according to the one or more reference blocks; and determining a reconstruction value of the current block according to the prediction value of the current block. The decoder determines a first template corresponding to the current block; determining one or more block vectors corresponding to the current block according to the first template; and determining one or more reference blocks of the current block according to the one or more block vectors, and determining a prediction value of the current block according to the one or more reference blocks.

Description

Coding and decoding method, code stream, encoder, decoder and storage medium Technical Field

[0001] The embodiments of the present application relate to the field of video coding and decoding technology, and in particular to a coding and decoding method, a bit stream, an encoder, a decoder, and a storage medium. Background Art

[0002] Intra Template Matching Prediction (Intra TMP) technology uses the template of the coding block to search for the matching template with the minimum cost according to a preset cost function within a predefined search range in the current image, and uses the best matching reconstructed block corresponding to the matching template as the prediction block of the current coding block.

[0003] However, in the actual encoding process, related technologies usually directly use the reconstructed pixels of the best matching reconstructed block as the predicted pixels of the current encoding block. However, due to incomplete consideration, there will be large deviations in some scenarios, resulting in low prediction accuracy.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a coding and decoding method, a code stream, an encoder, a decoder, and a storage medium, which can improve the accuracy of prediction and thus obtain the optimal prediction effect.

[0006] The technical solution of the embodiment of the present application can be implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides a decoding method, applied to a decoder, the method comprising:

[0008] Determine a first template corresponding to the current block;

[0009] determining one or more block vectors corresponding to the current block according to the first template;

[0010] determining one or more reference blocks for a current block according to the one or more block vectors, and determining a prediction value for the current block according to the one or more reference blocks;

[0011] A reconstructed value of the current block is determined according to the predicted value of the current block.

[0012] In a second aspect, an embodiment of the present application provides an encoding method, applied to an encoder, the method comprising:

[0013] Determine a first template corresponding to the current block;

[0014] determining one or more block vectors corresponding to the current block according to the first template;

[0015] One or more reference blocks of the current block are determined according to the one or more block vectors, and a prediction value of the current block is determined according to the one or more reference blocks.

[0016] In a third aspect, an embodiment of the present application provides a code stream, wherein the code stream is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least one of the following:

[0017] The prediction difference of the current block, a preset number N, or one or more block vectors.

[0018] In a fourth aspect, an embodiment of the present application provides an encoder, comprising a first determining unit; wherein,

[0019] The first determination unit is configured to determine a first template corresponding to the current block; determine one or more block vectors corresponding to the current block based on the first template; determine one or more reference blocks of the current block based on the one or more block vectors, and determine a prediction value of the current block based on the one or more reference blocks.

[0020] In a fifth aspect, an embodiment of the present application provides an encoder, comprising a first memory and a first processor; wherein,

[0021] The first memory is used to store a computer program that can be run on the first processor;

[0022] The first processor is configured to execute the method according to the second aspect when running the computer program.

[0023] In a sixth aspect, an embodiment of the present application provides a decoder, comprising a second determining unit; wherein,

[0024] The second determination unit is configured to determine a first template corresponding to the current block; determine one or more block vectors corresponding to the current block based on the first template; determine one or more reference blocks of the current block based on the one or more block vectors, and determine a prediction value of the current block based on the one or more reference blocks; and determine a reconstructed value of the current block based on the prediction value of the current block.

[0025] In a seventh aspect, an embodiment of the present application provides a decoder, comprising a second memory and a second processor; wherein,

[0026] The second memory is used to store a computer program that can be run on the second processor;

[0027] The second processor is configured to execute the method according to the first aspect when running the computer program.

[0028] In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, the method as described in the first aspect or the method as described in the second aspect is implemented.

[0029] The embodiment of the present application provides a coding and decoding method, an encoder, a decoder and a storage medium, wherein the encoder determines a first template corresponding to the current block; determines one or more block vectors corresponding to the current block based on the first template; determines one or more reference blocks of the current block based on the one or more block vectors, and determines a predicted value of the current block based on the one or more reference blocks; and determines a reconstructed value of the current block based on the predicted value of the current block. The decoder determines a first template corresponding to the current block; determines one or more block vectors corresponding to the current block based on the first template; determines one or more reference blocks of the current block based on the one or more block vectors, and determines a predicted value of the current block based on the one or more reference blocks. It can be seen that in the embodiment of the present application, an Intra TMP Fusion prediction method is proposed, wherein at least one block vector of the current block can be selected and determined, and then at least one reference block corresponding to at least one block vector can be used to obtain a predicted value of the current block. In other words, the coding and decoding method proposed in the embodiment of the present application fully takes into account the different importance of the reconstructed block information corresponding to different matching templates in the search process to the prediction of the current block, and can improve the accuracy of the prediction, thereby obtaining the best prediction effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a schematic diagram of the prediction of Intra TMP;

[0031] FIG2 is a schematic diagram of the prediction process based on the IntraTMP technology;

[0032] FIG3 is a schematic diagram of template types for the Intra TMP technology;

[0033] FIG4 is a schematic diagram of a search process based on the IntraTMP technology;

[0034] FIG5 is a schematic diagram of parameter definition of the current block and its template;

[0035] FIG6 is a schematic diagram of a template search area;

[0036] FIG7 is a schematic diagram showing the division of the search area into different sub-areas;

[0037] FIG8 is a schematic diagram of the search area determination process 1;

[0038] FIG9 is a schematic diagram of the search process;

[0039] FIG10A is a schematic block diagram of the encoder;

[0040] FIG10B is a schematic block diagram of the encoder;

[0041] FIG11 is a schematic diagram of the network architecture of the encoding and decoding system;

[0042] FIG12 is a schematic diagram of a flow chart of a decoding method proposed in an embodiment of the present application;

[0043] FIG13 is a second schematic diagram of the process of determining the search area;

[0044] FIG14 is a third schematic diagram of the process of determining the search area;

[0045] FIG15 is a schematic diagram of a flow chart of an encoding method proposed in an embodiment of the present application;

[0046] FIG16 is a schematic diagram of the composition structure of the encoder;

[0047] Figure 17 is a schematic diagram of the encoder hardware structure;

[0048] FIG18 is a schematic diagram of the composition structure of a decoder;

[0049] FIG19 is a schematic diagram of the decoder hardware structure;

[0050] FIG20 is a schematic diagram showing the composition structure of the encoding and decoding system. DETAILED DESCRIPTION

[0051] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below with reference to the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0053] In the following description, reference is made to "some embodiments," which describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. It should also be noted that the terms "first, second, and third" in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that "first, second, and third" may be interchanged in a specific order or sequential order where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0054] Before further explaining the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained first. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations:

[0055] Coding Block (CB);

[0056] Block Maching (BM);

[0057] Coding Unit (CU);

[0058] Block Vector (BV);

[0059] Sum of Absolute Difference (SAD);

[0060] Sum of Absolute Transformed Difference (SATD);

[0061] Mean Square Error (MSE);

[0062] Sum of Squared Differences (SSD);

[0063] Mean Absolute Deviation (MAD);

[0064] Mean Square Differences (MSD);

[0065] Normalized Correlation Coefficient (NCC);

[0066] H.266 / Versatile Video Coding (VVC);

[0067] VVC Test Model (VTM), a reference software testing platform for VVC;

[0068] Intra Template Matching Prediction (Intra TMP);

[0069] Beyond VVC's reference software testing platform (Enhanced Compression Model, ECM).

[0070] It can be understood that in a video image, a first color component, a second color component, and a third color component are generally used to represent a coding block. These three color components are a luminance component, a blue chrominance component, and a red chrominance component. Specifically, the luminance component is usually represented by the symbol Y, the blue chrominance component is usually represented by the symbol Cb or U, and the red chrominance component is usually represented by the symbol Cr or V. Thus, a video image can be represented in either the YCbCr format or the YUV format.

[0071] It can also be understood that Intra TMP is a special intra-frame prediction mode. Both the encoder and decoder use the template (T) of the coding block to search for the matching template (T_BEST) with the minimum cost within a predefined search range in the current image according to a preset cost function. The offset of the best matching template relative to the current coding block template is the best block vector (BEST Block Vector, BV_BEST). The reconstructed block (Ref Block) corresponding to the matching template is then used as the prediction block for the current coding block (Cur Block). The template of the coding block is usually selected from the adjacent reconstructed area of ​​the current coding block.

[0072] For example, taking the adjacent reconstructed area of ​​the current block as an example, FIG1 is a prediction diagram of Intra TMP. As shown in FIG1 , the area filled with dark color represents the reconstructed area, the block filled with grid is the current block, and the adjacent area of ​​the current block is the first template (T); the block filled with slashes is the reference block, and the adjacent area of ​​the reference block is the second template (i.e., the best matching template, T_BEST); wherein, the offset of the second template relative to the first template is the best block vector (BV_BEST). At this time, the reference block can be block-copied as the prediction block of the current block.

[0073] In an embodiment of the present application, the preset cost function can be the sum of absolute error SAD, the sum of absolute transformation difference SATD, the mean square error MSE, the sum of squared errors SSD, the mean absolute difference MAD, the mean sum of squared errors MSD, the normalized correlation coefficient NCC, etc., which is not specifically limited here.

[0074] For example, taking absolute error and SAD as examples, the cost function at this time is as follows:

[0075]

[0076] Among them, T i is the template in the search process, and M represents the number of pixels in the template.

[0077] The following is a detailed introduction to the prediction process of the Intra TMP technology in the related art.

[0078] Input of IntraTMP: the position of the current block (xTbCmp, yTbCmp), the width of the current block nTbW, and the height of the current block nTbH.

[0079] Output of IntraTMP: predicted value predSamples[x][y] of the current block, where x = 0..nTbW-1, y = 0..nTbH–1.

[0080] Specifically, the IntraTMP prediction process can be divided into four steps: determining the current template type, obtaining the reconstructed pixels of the current template, determining the block vector within a predefined search range, and generating the predicted value. This process yields the predicted value for the current block. It's important to note that the IntraTMP technique can be used to predict both luma and chroma components, and this is not specifically limited here.

[0081] See Figure 2, which shows a schematic diagram of a prediction process based on the IntraTMP technology. As shown in Figure 2, the process may include:

[0082] S201: Determine the current template type.

[0083] It should be noted that the Intra TMP technology uses the adjacent reconstructed pixels of the current block as templates to search for matching templates within a predefined search area, where the adjacent reconstructed pixels can be the upper reference pixel, upper left reference pixel, upper right reference pixel, left reference pixel, lower left reference pixel, etc. of the current block. Therefore, the template type can be classified and the corresponding template type can be determined based on the availability of the adjacent reconstructed pixels.

[0084] It should also be noted that refTemplateType can be used to represent a template type. Figure 3 shows a schematic diagram of template types for the Intra TMP technology. As shown in Figure 3, the block filled with a grid is the current block, and the adjacent area of ​​the current block is the template T. Six template types are shown here.

[0085] For example, the six template types are as follows:

[0086] When the upper left reference pixel, the upper reference pixel, and the left reference pixel are all available, the value of refTemplateType is 1, and the template shape is as shown in (a) of Figure 3;

[0087] When only the left reference pixel is available, the value of refTemplateType is 2, and the template shape is shown in (b) in Figure 3;

[0088] When only the upper reference pixel is available, the value of refTemplateType is 3, and the template shape is shown in (c) in Figure 3;

[0089] When only the left reference pixel and the upper left reference pixel are available, the value of refTemplateType is 4, and the template shape is shown in (d) in Figure 3;

[0090] When only the left reference pixel and the lower left reference pixel are available, the value of refTemplateType is 5, and the template shape is shown in (e) of Figure 3;

[0091] When only the upper reference pixel and the upper right reference pixel are available, the value of refTemplateType is 6, and the template shape is shown in (f) in Figure 3.

[0092] S202: Obtain current template pixels.

[0093] It should be noted that the Intra TMP template can be composed of reconstructed pixels from one or more of the following regions: the top, upper right, left, lower left, or upper left of the current block. Furthermore, the template size can be pre-set. For example, when obtaining the left template, the template width (templateW_size) can be set to 4, and when obtaining the top template, the template height (templateH_size) can be set to 4.

[0094] It should also be noted that the portion of the reconstructed pixels to be obtained can be determined by the value of refTemplateType. For example, when the value of refTemplateType is 1, the reconstructed pixels of the left, upper-left, and upper sides of the current block are obtained; or, when the value of refTemplateType is 2, only the reconstructed pixels of the four left columns of the current block are obtained; or, when the value of refTemplateType is 3, only the reconstructed pixels of the four upper rows of the current block are obtained.

[0095] S203: Determine a block vector within a predefined search range.

[0096] It should be noted that the search process of the Intra TMP technology is mainly divided into an initialization process, determining a search area of ​​the template in the current frame, and searching and determining the best block vector in the search area.

[0097] It should also be noted that when searching for the best matching template in the search area, a search strategy of coarse search first and then fine search may be adopted, or only fine search or only coarse search may be performed, which is not specifically limited here.

[0098] In an embodiment of the present application, the coarse search here can specifically be: determining the best coarse matching template in the search area with a first preset step size (for example, 2), or using a downsampled template (for example, a downsampling factor of 2) to determine the best coarse matching template in the search area.

[0099] In the embodiment of the present application, the fine search here may specifically be: determining the best fine matching template in the search area with a second preset step size (eg, 1), or determining the best fine matching template near the best coarse matching template after completing the coarse search.

[0100] See Figure 4, which shows a schematic diagram of a search process based on the IntraTMP technology provided by an embodiment of the present application. As shown in Figure 4, the process may include:

[0101] S401: Initialize parameters.

[0102] It should be noted that uiPatchWidth is initialized to nTbW + templateW_size, and uiPatchHeight is initialized to nTbH + templateH_size. TemplateW_size and templateH_size can be fixed constants or dynamically adjusted according to the size of the current block. In addition, templateW_size and templateH_size can be equal or unequal. For example, templateW_size = 4, templateH_size = 4; or, when the width of the current block is greater than 8, set templateW_size = 4; when the width of the current block is less than or equal to 8, set templateW_size = 2; when the height of the current block is greater than 8, set templateH_size = 4; when the height of the current block is less than or equal to 8, set templateH_size = 2.

[0103] For example, Figure 5 shows a schematic diagram of the parameter definition of the current block and its template. As shown in Figure 5, the specific meanings of the parameters are: nTbW and nTbH represent the size of the current block, templateW_size and templateH_size represent the template size, and uiPatchWidth and uiPatchHeight represent the size of the block containing the current block and its template.

[0104] Furthermore, the cost threshold between the initialization templates is represented by diffThreshold. For example, when the cost function is SAD, the threshold can be: diffThreshold = ((1 <<bitDepth)> >2)×(uiPatchHeight×uiPatchWidth-nTbH×nTbW). When the image bit depth bitDepth is 10, diffThreshold indicates that the maximum distortion of each pixel in the template area is 256.

[0105] Furthermore, the position of the coding tree block CTB where the current block CB is located is initialized: ctbRsX, ctbRsY.

[0106] Furthermore, the position offset of the current block CB in the current CTB is initialized: offsetLCBY=yTbCmp−ctbRsY, offsetLCBX=xTbCmp−ctbRsX.

[0107] Furthermore, initialize iTemplateSizeH=templateH_size and iTemplateSizeW=templateW_size.

[0108] Furthermore, iBvShift is initialized, where iBvShift is the precision of the block vector BV. For example, the precision of BV can be integer pixel precision, in which case iBvShift is 0; the precision of BV can also be sub-pixel precision, for example, iBvShift of 1 indicates 1 / 2 pixel precision, and iBvShift of 2 indicates 1 / 4 pixel precision. This is not specifically limited here.

[0109] Furthermore, the preset search range of the template is initialized. The preset search range of the template can be set to a fixed size or dynamically adjusted according to the encoding block size. For example, searchRangeWidth = TMP_SEARCH_RANGE_MULT_FACTOR × nTbW, searchRangeHeight = TMP_SEARCH_RANGE_MULT_FACTOR × nTbH; where the value of TMP_SEARCH_RANGE_MULT_FACTOR can be a preset value, such as 5.

[0110] S402: Determine a search area of ​​the template in the current frame.

[0111] It should be noted that the Intra TMP technique's search area is the reconstructed portion of the current image and is limited by the search range. Figure 6 shows a schematic diagram of the template search area. As shown in Figure 6, the dark-filled background area represents the reconstructed area, the black-filled background block represents the current block, and the dashed box represents the search range window. Therefore, the IntraTMP technique's search area is no larger than the overlap between the reconstructed area represented by the dark background and the area indicated by the dashed box.

[0112] As can be seen, the search area for the current block template can be the reconstructed portion of the CTB where the current block resides, or it can be another reconstructed CTB area. The search area here is actually the set of all search points. Because the shape of this area is often not representable by a single rectangular region, in practice, multiple rectangular regions can be searched, and the search results from these different regions can be combined to obtain the final best matching block and best block vector.

[0113] For example, see Figure 7, which shows a schematic diagram of different sub-region divisions of a search area. As shown in Figure 7, eight different sub-region division methods are shown. The black-filled background block is the current block; the five division methods (a), (b), (c), (d), and (f) all divide the search area into four sub-search areas, and the three division methods (e), (g), and (h) all divide the search area into three sub-search areas. Here, different fill patterns represent a sub-search area.

[0114] In Figure 7, (a), (b), (c), and (d) consider all available search ranges, and (e), (f), (g), and (h) do not search the area directly above and to the left.

[0115] For example, assuming that different sub-search regions are represented by regionId, and considering that the template pixels of the current block need to be retrieved in the image reconstruction region and the pixels of the reconstructed block corresponding to the template also need to be retrieved in the reconstruction region, it is necessary to further determine the positions that can be searched within the sub-search regions represented by different regionIds based on the current block position (xTbCmp, yTbCmp), the current block size (nTbW, nTbH), the current image size (picWidth, picHeight), the size of the CTB in which the current block is located (CtbSizeW, CtbSizeH), the preset search range of the template (searchRangeWidth, searchRangeHeight), and the position offset of the current block within the current CTB (offsetLCBY, offsetLCBX), thereby determining the block vector BV. Specifically, iVerMin and iVerMax represent the absolute coordinate positions that can be searched at least and at most in the vertical direction, respectively, and iHorMin and iHorMax represent the absolute coordinate positions that can be searched at least and at most in the horizontal direction, respectively. The values ​​of iVerMin, iVerMax, iHorMin, and iHorMax are different in the search areas represented by different regionIds.

[0116] Taking (f) in Figure 7 as an example, the search area is divided into four sub-search areas, which are implemented as follows:

[0117] When regionId is equal to 0, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0118] iHorMax0=min((xTbCmp+searchRangeWidth)< <iBvShift,((picWidth-nTbW)<<iBvShift));

[0119] iHorMin0=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift);

[0120] iVerMax0=(yTbCmp-nTbH-offsetLCBY)< <iBvShift;

[0121] iVerMin0=max(((iTemplateSizeH)< <iBvShift),((yTbCmp-searchRangeHeight)<<iBvShift))。

[0122] When regionId is equal to 1, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0123] iHorMin1=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift);

[0124] iHorMax1=(xTbCmp-offsetLCBX-nTbW)< <iBvShift;

[0125] iVerMin1=(yTbCmp+1)< <iBvShift;

[0126] iVerMax1=min(picHeight-nTbH,(yTbCmp-offsetLCBY+CtbSizeH-nTbH)< <iBvShift)。

[0127] When regionId is equal to 2, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0128] iHorMax2=(xTbCmp-offsetLCBX-nTbW)< <iBvShift;

[0129] iHorMin2=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift);

[0130] iVerMin2=max((iTemplateSizeH)< <iBvShift,(yTbCmp-nTbH-offsetLCBY)<<iBvShift);

[0131] iVerMax2=(yTbCmp)< <iBvShift。

[0132] When regionId is equal to 3, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0133] iHorMin3=max((iTemplateSizeW)< <iBvShift,(xTbCmp-offsetLCBX-nTbW+1)<<iBvShift);

[0134] iHorMax3=(xTbCmp-nTbW)< <iBvShift;

[0135] iVerMin3=max(((iTemplateSizeH)< <iBvShift),(yTbCmp-offsetLCBY-nTbH+1)<<iBvShift);

[0136] iVerMax3=(yTbCmp-nTbH)< <iBvShift。

[0137] In practical applications, the iHorMin regionId 、iHorMax regionId 、iVerMin regionId 、iVerMax regionId Represent the left edge, right edge, top edge and bottom edge of different sub-search areas respectively.

[0138] To intuitively illustrate the different sub-search regions corresponding to different regionIds, see Figure 8, which shows a schematic diagram of the specific search region determination process. As shown in Figure 8, R1, R2, R3, and R4 represent four different sub-search regions. Note that Figure 8 represents the pixel range aligned with the top-left corner pixel of the block.

[0139] S403: Search and determine the best block vector BV in the search area.

[0140] It should be noted that bvXMins and bvXMaxs represent the minimum offset and maximum offset of the block vector in the horizontal direction respectively; bvYMins and bvYMaxs represent the minimum offset and maximum offset of the block vector in the vertical direction respectively.

[0141] bvXMins regionId 、bvXMaxs regionId 、bvYMins regionId 、bvYMaxs regionId iVerMin determined in step S402 regionId 、iVerMax regionId 、iHorMin regionId 、iHorMax regionId Calculation yields:

[0142] bvXMins regionId =iHorMin regionId –xTbCmp;

[0143] bvXMaxs regionId =iHorMax regionId–xTbCmp;

[0144] bvYMins regionId =iVerMin regionId –yTbCmp;

[0145] bvYMaxs regionId =iVerMax regionId –yTbCmp;

[0146] Among them, bvXMins regionId 、bvXMaxs regionId 、bvYMins regionId and bvYMaxs regionId The horizontal and vertical offset ranges of the search point relative to the current block, ie, the range of the block vector BV, are determined.

[0147] It should also be noted that, through the search points (iPosHor, iPoxVer) in each search area, that is, each block vector BV (composed of horizontal and vertical components: (pX, pY), where pX = iPosHor-xTbCmp, pY = iPosVer-yTbCmp, then pX is between bvXMins and bvXMaxs, and pY is between bvYMins and bvYMaxs; in this way, a matching reconstructed block of the current block can be found in the reconstructed area, and the adjacent reconstructed pixels of the matching reconstructed block are the matching template (that is, the second template mentioned above). Therefore, the matching cost value of the adjacent template of the current block and the adjacent template of the matching reconstructed block can be calculated, which is recorded as pDiff.

[0148] Furthermore, all search points in all search ranges (regionId = 0, 1, 2, 3) are traversed, and the search point with the smallest matching cost value pDiff is obtained by comparison. Its corresponding matching cost value is recorded as pDiff_BEST, its corresponding block vector BV is recorded as the best block vector BV_BEST (pX_BEST, pY_BEST), and its corresponding matching template is the best matching template T_BEST.

[0149] In one possible implementation, if the search strategy is to perform only a coarse search, the specific implementation is as follows:

[0150] In each region at pX within bvXMins regionId with bvXMaxs regionId Between, pY is in bvYMins regionId with bvYMaxs regionIdWithin the search range between , a coarse search is performed with a step size greater than 1, for example, a coarse search is performed with a step size of 2. The best matching cost value obtained by template matching is recorded as pDiff_BEST, and its corresponding block vector BV is recorded as the best block vector BV_BEST (pX_BEST, pY_BEST).

[0151] In another possible implementation, if the search strategy is to perform only detailed search, the specific implementation is as follows:

[0152] In each region at pX within bvXMins regionId with bvXMaxs regionId Between, pY is in bvYMins regionId with bvYMaxs regionId Within the search range between, for example, a fine search is performed with a step size of 1 to record the optimal matching cost value obtained by template matching, pDiff_BEST, and its corresponding block vector BV is recorded as the best block vector BV_BEST (pX_BEST, pY_BEST).

[0153] In another possible implementation, if the search strategy is: perform a coarse search first and then a fine search; FIG9 is a schematic diagram of the search process, as shown in FIG9. The specific process is as follows:

[0154] S901: Determine the best coarse matching template in the search area with a step size of 2.

[0155] S902: Determine the best fine matching template near the best coarse matching template with a step size of 1.

[0156] It should be noted that, for step S901, in the coarse search phase:

[0157] In each region at pX within bvXMins regionId with bvXMaxs regionId Between, pY is in bvYMins regionId with bvYMaxs regionId Within the search range between , a coarse search is performed with a step size greater than 1. For example, the best matching cost obtained by performing a coarse search with a step size of 2 is recorded as pDiff1_BEST, and its corresponding block vector BV is recorded as the best block vector BV1_BEST(pX1_BEST, pY1_BEST), and the search region where the best matching search point is located is bestRegionId.

[0158] It should also be noted that, for step S902, in the detailed search phase:

[0159] A further search is performed near the best block vector BV1_BEST obtained from the coarse search. Specifically, a refined search range TmpRefineRange is first determined. The refined search range can be a fixed size or related to the current block size, for example, min(nTbW, nTbH) / 2. The position of the best matching reconstructed block obtained from the coarse search is then calculated as the reference position of the fine search area: BestPosX = xTbCmp + pX1_BEST, BestPosY = yTbCmp + pY1_BEST.

[0160] According to the value of bestRegionId, first obtain the calculated values ​​of iVerMinbestRegionId, iVerMaxbestRegionId, iHorMinbestRegionId, and iHorMaxbestRegionId, and then obtain the new search range iVerMin according to the optimal matching block position obtained by the rough search. refine 、iVerMax refine 、iHorMin refine 、iHorMax refine , the acquisition method is as follows:

[0161] iHorMin refine =max(iHorMinbestRegionId,BestPosX-TmpRefineRange);

[0162] iHorMax refine =min(iHorMaxbestRegionId,BestPosX+TmpRefineRange);

[0163] iVerMin refine =max(iVerMinbestRegionId,BestPosY-TmpRefineRange);

[0164] iVerMax refine =min(iVerMaxbestRegionId,BestPosY+TmpRefineRange);

[0165] Then, the adjusted block vectors BVbvXMins, bvXMaxs, bvYMins, bvYMaxs can be obtained by iVerMin refine 、iVerMax refine 、iHorMin refine 、iHorMax refine Calculation yields:

[0166] bvXMins=iHorMin refine –xTbCmp;

[0167] bvXMaxs=iHorMax refine –xTbCmp;

[0168] bvYMins=iVerMin refine –yTbCmp;

[0169] bvYMaxs=iVerMax refine –yTbCmp;

[0170] Thus, the fine search is performed at pX within bvXMins refine with bvXMaxs refine Between, pY is in bvYMins refine with bvYMaxs refine For example, the search is performed with a step size of 1, and the best matching cost obtained by template matching is recorded as pDiff_BEST, and its corresponding block vector BV is recorded as the best block vector BV_BEST (pX_BEST, pY_BEST).

[0171] After completing the above operations, the best block vector BV_BEST (pX_BEST, pY_BEST) can be obtained, where pX_BEST and pY_BEST are the horizontal offset and vertical offset of the best matching template relative to the current block template, respectively, and are also the horizontal offset and vertical offset of the best matching reconstructed block relative to the current block.

[0172] S204: Generate a predicted value.

[0173] Here, we can use simple translation copy to achieve it. The specific operation is:

[0174] For x = 0 ... nTbW-1, y = 0 ... nTbH-1;

[0175] predSamples[x][y]=recSamples[x+pX_BEST][y+pY_BEST] (2)

[0176] Among them, recSamples represents the reconstructed pixels of the current frame.

[0177] In simple terms, the Intra TMP technique, as used in related technologies, uses the template of the current block to search for the matching template with the lowest cost within a predefined search range in the current image according to a preset cost function. The best matching reconstructed block (Ref Block) corresponding to the matching template is used as the prediction block for the current block (Cur Block). The template of the current block can usually be selected from the reconstructed area adjacent to the current block.

[0178] However, in actual encoding, directly using the reconstructed pixels of the best-matching reconstructed block as the predicted pixels for the current block is not optimal in some cases. If different regions within a frame have varying illumination intensities or pixel noise distributions, the content of the copied reconstructed block and the current region's coded block will differ significantly, leading to decreased prediction accuracy. If the template matching cost function does not actually reflect the difference between the current coded block template and the matching template used in the search process, then using the matching template with the minimum cost as the best matching template will result in the final best candidate block not being the optimal candidate block, resulting in limited effective reference information and ultimately reduced prediction accuracy.

[0179] In summary, common encoding and decoding methods may have the problem of large deviations in prediction values, resulting in low prediction accuracy and failure to achieve optimal prediction results.

[0180] In order to solve the above problems, the embodiments of the present application provide a coding and decoding method, an encoder, a decoder and a storage medium, wherein the encoder determines a first template corresponding to the current block; determines one or more block vectors corresponding to the current block based on the first template; determines one or more reference blocks of the current block based on the one or more block vectors, and determines a predicted value of the current block based on the one or more reference blocks; and determines a reconstructed value of the current block based on the predicted value of the current block. The decoder determines a first template corresponding to the current block; determines one or more block vectors corresponding to the current block based on the first template; determines one or more reference blocks of the current block based on the one or more block vectors, and determines a predicted value of the current block based on the one or more reference blocks. It can be seen that in the embodiments of the present application, an Intra TMP Fusion prediction method is proposed, wherein at least one block vector of the current block can be selected and determined, and then at least one reference block corresponding to at least one block vector can be used to obtain a predicted value of the current block. In other words, the coding and decoding method proposed in the embodiments of the present application fully takes into account the different importance of the reconstructed block information corresponding to different matching templates in the search process for predicting the current block, and can improve the accuracy of the prediction, thereby obtaining the best prediction effect.

[0181] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0182] Referring to Figure 10A, which shows a schematic block diagram of the composition of an encoder provided in an embodiment of the present application. As shown in Figure 10A, the encoder (specifically, a "video encoder") 100 may include a transform and quantization unit 101, an intra-frame estimation unit 102, an intra-frame prediction unit 103, a motion compensation unit 104, a motion estimation unit 105, an inverse transform and inverse quantization unit 106, a filter control analysis unit 107, a filtering unit 108, an encoding unit 109, and a decoded image cache unit 110, etc., wherein the filtering unit 108 can implement deblocking filtering and sample adaptive offset (SAO) filtering, and the encoding unit 109 can implement header information encoding and context-based adaptive binary arithmetic coding (CABAC).For the input original video signal, a video coding block can be obtained by dividing the coding tree unit (CTU). Then, the residual pixel information obtained after intra-frame or inter-frame prediction is transformed by the transformation and quantization unit 101, including transforming the residual information from the pixel domain to the transform domain and quantizing the obtained transform coefficients to further reduce the bit rate; the intra-frame estimation unit 102 and the intra-frame prediction unit 103 are used to perform intra-frame prediction on the video coding block. Specifically, the intra-frame estimation unit 102 and the intra-frame prediction unit 103 are used to determine the intra-frame prediction mode to be used to encode the video coding block; the motion compensation unit 104 and the motion estimation unit 105 are used to perform inter-frame prediction coding on the received video coding block relative to one or more blocks in one or more reference frames to provide temporal prediction information; the motion estimation performed by the motion estimation unit 105 is the process of generating a motion vector, which can estimate the motion of the video coding block. The motion compensation unit 104 then calculates the motion vector based on the motion vector determined by the motion estimation unit 105. After determining the intra-frame prediction mode, the intra-frame prediction unit 103 is further configured to provide the selected intra-frame prediction data to the encoding unit 109, and the motion estimation unit 105 also sends the calculated motion vector data to the encoding unit 109. In addition, the inverse transform and inverse quantization unit 106 is configured to reconstruct the video coding block and reconstruct a residual block in the pixel domain. The reconstructed residual block is subjected to the filter control analysis unit 107 and the filtering unit 108 to remove the block effect artifacts. The reconstructed residual block is then added to a predictive block in the frame of the decoded image buffer unit 110 to generate a reconstructed video coding block. The encoding unit 109 is configured to encode various coding parameters and quantized transform coefficients. In the CABAC-based coding algorithm, the context content can be based on adjacent coding blocks and can be used to encode information indicating the determined intra-frame prediction mode, and output the code stream of the video signal. The decoded image buffer unit 110 is configured to store the reconstructed video coding block for prediction reference. As the video image encoding proceeds, new reconstructed video encoding blocks are continuously generated, and these reconstructed video encoding blocks are stored in the decoded image buffer unit 110 .

[0183] Referring to FIG10B , which shows a block diagram of a decoder provided in an embodiment of the present application, as shown in FIG10B , the decoder (specifically, a "video decoder") 200 includes a decoding unit 201, an inverse transform and inverse quantization unit 202, an intra-frame prediction unit 203, a motion compensation unit 204, a filtering unit 205, and a decoded image buffer unit 206. The decoding unit 201 can implement header information decoding and CABAC decoding, and the filtering unit 205 can implement deblocking filtering and SAO filtering. After the input video signal is encoded as shown in FIG10A, a code stream of the video signal is output; the code stream is input to the decoder 200 and first passes through the decoding unit 201 to obtain decoded transform coefficients; the transform coefficients are processed by the inverse transform and inverse quantization unit 202 to generate a residual block in the pixel domain; the intra-frame prediction unit 203 can be used to generate prediction data for the current video decoding block based on the determined intra-frame prediction mode and data from the previously decoded block of the current frame or picture; the motion compensation unit 204 determines the prediction information for the video decoding block by analyzing the motion vector and other associated syntax elements, and uses The prediction information is used to generate a predictive block for the video decoding block being decoded; a decoded video block is formed by summing the residual block from the inverse transform and inverse quantization unit 202 with the corresponding predictive block generated by the intra-frame prediction unit 203 or the motion compensation unit 204; the decoded video signal passes through the filtering unit 205 to remove blocking artifacts, thereby improving video quality; the decoded video block is then stored in the decoded image buffer unit 206, which stores reference images used for subsequent intra-frame prediction or motion compensation, and is also used for outputting the video signal, thereby obtaining the restored original video signal.

[0184] Furthermore, the embodiment of the present application also provides a network architecture of a coding and decoding system including an encoder and a decoder, wherein Figure 11 shows a schematic diagram of the network architecture of a coding and decoding system provided by the embodiment of the present application. As shown in Figure 11, the network architecture includes one or more electronic devices 13 to 1N and a communication network 01, wherein the electronic devices 13 to 1N can perform video interaction through the communication network 01. During implementation, the electronic device can be various types of devices with video coding and decoding functions. For example, the electronic device can include a smart phone, a tablet computer, a personal computer, a personal digital assistant, a navigator, a digital phone, a video phone, a television, a sensing device, a server, etc., and the embodiment of the present application is not specifically limited. Here, the decoder or encoder described in the embodiment of the present application can be the above-mentioned electronic device.

[0185] It should be noted that the method of the embodiment of the present application is mainly applied to the intra-frame prediction unit 103 shown in Figure 10A and the intra-frame prediction unit 203 shown in Figure 10B. In other words, the embodiment of the present application can be applied to both the encoder and the decoder, or even to both the encoder and the decoder at the same time, but the embodiment of the present application is not specifically limited thereto.

[0186] It should also be noted that, when applied to the intra-frame prediction unit 103, the "current block" specifically refers to the coding block currently to be intra-frame predicted; when applied to the intra-frame prediction unit 203, the "current block" specifically refers to the decoding block currently to be intra-frame predicted.

[0187] An embodiment of the present application provides a decoding method, which is applied to a decoder. FIG12 is a flowchart of the decoding method provided in the embodiment of the present application. As shown in FIG12 , the decoding method performed by the decoder may include the following steps:

[0188] Step 101: Determine a first template corresponding to the current block.

[0189] In an embodiment of the present application, the first template corresponding to the current block may be determined first. When acquiring the first template, the template type corresponding to the current block may be determined first, and then the first template corresponding to the current block may be further determined based on the template type.

[0190] It should be noted that the decoding method in the embodiment of the present application is applied to a decoder. Furthermore, the decoding method may include an intra-frame prediction method, more specifically, a color component prediction method. A video image may be divided into multiple decoding blocks, each of which may include a first color component, a second color component, and a third color component. The current block in the embodiment of the present application refers to a decoding block in the video image that is currently to be intra-frame predicted.

[0191] Here, when the first color component needs to be predicted, the component to be predicted is the first color component; when the second color component needs to be predicted, the component to be predicted is the second color component; when the third color component needs to be predicted, the component to be predicted is the third color component. In addition, assuming that the current block predicts the first color component and the first color component is the luminance component, that is, the component to be predicted is the luminance component, then the current block can also be called a luminance block; or assuming that the current block predicts the second color component and the second color component is the chrominance component, that is, the component to be predicted is the chrominance component, then the current block can also be called a chrominance block.

[0192] It should also be noted that, in the embodiments of the present application, the reference pixel (Reference Sample) of the current block may refer to a reference pixel point adjacent to the current block; wherein, adjacent here may be spatially adjacent, but is not limited to this. For example, adjacent may also mean temporal adjacent, spatially and temporally adjacent, or even the reference pixel of the current block may be a reference pixel obtained by performing some processing on spatially adjacent, temporally adjacent, or spatially and temporally adjacent reference pixels, etc., and the embodiments of the present application do not impose any limitation thereto.

[0193] Furthermore, in an embodiment of the present application, the template type of the current block can be determined based on the reference pixels of the current block, wherein the reference pixels of the current block include at least one of the following: the left adjacent reference pixels of the current block, the upper adjacent reference pixels of the current block, the upper left adjacent reference pixels of the current block, the lower left adjacent reference pixels of the current block, and the upper right adjacent reference pixels of the current block.

[0194] It is understandable that in the embodiment of the present application, the reference pixels of the current block may include the adjacent reconstructed pixels of the current block, that is, the adjacent reconstructed pixels of the current block may be selected as a template to search for a matching template in a predefined search area.

[0195] It should be noted that, in an embodiment of the present application, the reference pixels of the current block, that is, the adjacent reconstructed pixels of the current block may include the upper reference pixel, the upper left reference pixel, the upper right reference pixel, the left reference pixel and the lower left reference pixel of the current block.

[0196] It can be understood that in the embodiment of the present application, when the template type of the current block is determined using the reference pixels of the current block, the template type can be classified and determined according to whether the adjacent reference pixels are available.

[0197] Further, in an embodiment of the present application, when determining the template type of the current block based on the reference pixels of the current block, if the left adjacent reference pixels of the current block, the upper adjacent reference pixels of the current block and the upper-left adjacent reference pixels of the current block are all available, the template type of the current block is determined to be a first value; if the left adjacent reference pixels of the current block are available, the template type of the current block is determined to be a second value; if the upper adjacent reference pixels of the current block are available, the template type of the current block is determined to be a third value; if the left adjacent reference pixels of the current block and the upper-left adjacent reference pixels of the current block are both available, the template type of the current block is determined to be a fourth value; if the left adjacent reference pixels of the current block and the lower-left adjacent reference pixels of the current block are both available, the template type of the current block is determined to be a fifth value; if the upper adjacent reference pixels of the current block and the upper-right adjacent reference pixels of the current block are both available, the template type of the current block is determined to be a sixth value.

[0198] It should be noted that in the embodiments of the present application, the first value, the second value, the third value, the fourth value, the fifth value, and the sixth value can be any values ​​and are not specifically limited in this application. For example, the first value, the second value, the third value, the fourth value, the fifth value, and the sixth value can be 1, 2, 3, 4, 5, and 6, respectively.

[0199] For example, in an embodiment of the present application, refTemplateType can be used to represent the template type. Accordingly, as shown in FIG3 above, the block filled with the grid is the current block, and the adjacent area of ​​the current block is the template T. Six template types are shown here.

[0200] Exemplarily, these six template types are as follows: when the upper left reference pixel, the upper reference pixel and the left reference pixel are all available, the value of refTemplateType is 1, and the template shape is shown in (a) in Figure 3; when only the left reference pixel is available, the value of refTemplateType is 2, and the template shape is shown in (b) in Figure 3; when only the upper reference pixel is available, the value of refTemplateType is 3, and the template shape is shown in (c) in Figure 3; when only the left reference pixel and the upper left reference pixel are available, the value of refTemplateType is 4, and the template shape is shown in (d) in Figure 3; when only the left reference pixel and the lower left reference pixel are available, the value of refTemplateType is 5, and the template shape is shown in (e) in Figure 3; when only the upper reference pixel and the upper right reference pixel are available, the value of refTemplateType is 6, and the template shape is shown in (f) in Figure 3.

[0201] Furthermore, in an embodiment of the present application, when determining the first template corresponding to the current block based on the template type, the template reference pixels of the current block can be determined first based on the template type and the template size corresponding to the template type, and then the first template of the current block can be determined based on the template reference pixels.

[0202] It should be noted that, in an embodiment of the present application, the first template of the current block may include template reference pixels of the current block, wherein the template reference pixels of the current block may be determined by the template type of the current block and the template size corresponding to the template type.

[0203] It should be noted that, in an embodiment of the present application, the first template of the current block can be composed of reconstructed pixels of one or more areas above, upper right, left, lower left, and upper left of the current block, that is, it can be composed of reference pixels of the current block.

[0204] It should be noted that in the embodiments of the present application, the template size corresponding to the template type can be pre-set. For example, when obtaining the left template, the template width templateW_size can be set to 4, and when obtaining the upper template, the template height templateH_size can be set to 4.

[0205] Accordingly, in an embodiment of the present application, by combining the value of the template type refTemplateType of the current block and the template size corresponding to the refTemplateType, it is possible to determine which part of the reconstructed pixels to obtain as the template reference pixels of the current block, and then determine the corresponding first template.

[0206] Exemplarily, in an embodiment of the present application, when the value of refTemplateType is 1, you can choose to obtain the left, upper left and upper reconstructed pixels of the current block; when the value of refTemplateType is 2, only the left 4 columns of reconstructed pixels of the current block are obtained; when the value of refTemplateType is 3, only the upper 4 rows of reconstructed pixels of the current coding block are obtained.

[0207] Of course, the value of the preset template size can be any integer greater than 0, and is not limited to 4, and this application does not make any specific limitation on this.

[0208] It can be understood that in the embodiment of the present application, in combination with the template type and the corresponding template size of the current block, the template reference pixel of the current block determined from the reference pixels of the current block can be the first template corresponding to the current block.

[0209] Step 102: Determine one or more block vectors corresponding to the current block according to the first template.

[0210] In an embodiment of the present application, after determining the first template corresponding to the current block, one or more block vectors corresponding to the current block may be further determined based on the first template.

[0211] It should be noted that, in an embodiment of the present application, the block vector search process may include an initialization process, determining a search area of ​​the first template in the current frame, searching and determining one or a pair of optimal block vectors in the search area. Therefore, when performing the search process, the initialization operation needs to be completed first.

[0212] Exemplarily, as shown in FIG5 , nTbW and nTbH represent the size of the current block, templateW_size and templateH_size represent the size of the template, and uiPatchWidth and uiPatchHeight represent the size of the block including the current block and its template.

[0213] Accordingly, during initial processing, uiPatchWidth can be initialized to nTbW + templateW_size, and uiPatchHeight can be initialized to nTbH + templateH_size, where templateW_size and templateH_size can be fixed constants or dynamically adjusted according to the size of the coding block. TemplateW_size and templateH_size can be equal or unequal. For example, templateW_size = 4, templateH_size = 4; or when the width of the coding block is greater than 8, set templateW_size = 4, when the width of the coding block is less than or equal to 8, set templateW_size = 2, when the height of the coding block is greater than 8, set templateH_size = 4, when the height of the coding block is less than or equal to 8, set templateH_size = 2.

[0214] Furthermore, the cost threshold between the initialization templates is represented by diffThreshold. For example, when the cost function is SAD, the threshold can be: diffThreshold = ((1 <<bitDepth)> >2)×(uiPatchHeight×uiPatchWidth-nTbH×nTbW). When the image bit depth bitDepth is 10, diffThreshold indicates that the maximum distortion of each pixel in the template area is 256.

[0215] Furthermore, the position of the coding tree block CTB where the current block CB is located is initialized: ctbRsX, ctbRsY.

[0216] Furthermore, the position offset of the current block CB in the current CTB is initialized: offsetLCBY=yTbCmp−ctbRsY, offsetLCBX=xTbCmp−ctbRsX.

[0217] Furthermore, initialize iTemplateSizeH=templateH_size and iTemplateSizeW=templateW_size.

[0218] Furthermore, iBvShift is initialized, where iBvShift is the precision of the block vector BV. For example, the precision of BV can be integer pixel precision, in which case iBvShift is 0; the precision of BV can also be sub-pixel precision, for example, iBvShift of 1 indicates 1 / 2 pixel precision, and iBvShift of 2 indicates 1 / 4 pixel precision. This is not specifically limited here.

[0219] Furthermore, the preset search range of the template is initialized. The preset search range of the template can be set to a fixed size or dynamically adjusted according to the encoding block size. For example, searchRangeWidth = TMP_SEARCH_RANGE_MULT_FACTOR × nTbW, searchRangeHeight = TMP_SEARCH_RANGE_MULT_FACTOR × nTbH; where the value of TMP_SEARCH_RANGE_MULT_FACTOR can be a preset value, such as 5.

[0220] Furthermore, in an embodiment of the present application, when determining one or more block vectors corresponding to the current block according to the first template, a preset search area can be first determined according to the first template; then a search can be performed within the preset search area to determine one or more block vectors.

[0221] It should be noted that in the embodiments of this application, the search area is the reconstructed portion of the current image and is limited by the size of the search range. As shown in Figure 6 above, the dark-filled background area is the reconstructed area, the black-filled background block is the current block, and the dashed box is the search range window. Therefore, the search area of ​​the IntraTMP technology is no larger than the overlap between the reconstructed area represented by the dark background and the area indicated by the dashed box.

[0222] As can be seen, the search area for the current block template can be the reconstructed portion of the CTB where the current block resides, or it can be another reconstructed CTB area. The search area here is actually the set of all search points. Because the shape of this area is often not representable by a single rectangular region, in practice, multiple rectangular regions can be searched, and the search results from these different regions can be combined to obtain the final best matching block and best block vector.

[0223] For example, see Figure 7 above, which shows eight different sub-region division methods. The black-filled background block is the current block; methods (a), (b), (c), (d), and (f) all divide the search area into four sub-search areas, and methods (e), (g), and (h) all divide the search area into three sub-search areas. Here, different fill patterns represent a sub-search area.

[0224] In Figure 7, (a), (b), (c), and (d) consider all available search ranges, and (e), (f), (g), and (h) do not search the area directly above and to the left.

[0225] For example, assuming that different search regions are represented by regionId, considering that the current coding block template pixels need to be obtained in the image reconstruction region and the reconstructed block pixels corresponding to the template also need to be obtained in the reconstruction region, it is necessary to further determine the positions that can be searched within the search regions represented by different regionIds based on the position of the current coding block (xTbCmp, yTbCmp), the size of the current coding block (nTbW, nTbH), the size of the current image (picWidth, picHeight), the size of the CTB where the current coding block is located (CtbSizeW, CtbSizeH), the preset search range of the template (searchRangeWidth, searchRangeHeight), and the position offset of the current coding block within the current CTB (offsetLCBY, offsetLCBX), thereby determining the block vector BV. Specifically, iVerMin and iVerMax represent the absolute coordinate positions that can be searched at least and at most in the vertical direction, respectively, and iHorMin and iHorMax represent the absolute coordinate positions that can be searched at least and at most in the horizontal direction. The values ​​of iVerMin, iVerMax, iHorMin, and iHorMax are different in the search areas represented by different regionIds.

[0226] In some embodiments, taking (f) in FIG. 7 as an example, the search area is divided into four sub-search areas, which are implemented as follows:

[0227] When regionId is equal to 0, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0228] iHorMax0=min((xTbCmp+searchRangeWidth)< <iBvShift,((picWidth-nTbW)<<iBvShift));

[0229] iHorMin0=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift);

[0230] iVerMax0=(yTbCmp-nTbH-offsetLCBY)< <iBvShift;

[0231] iVerMin0=max(((iTemplateSizeH)< <iBvShift),((yTbCmp-searchRangeHeight)<<iBvShift))。

[0232] When regionId is equal to 1, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0233] iHorMin1=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift);

[0234] iHorMax1=(xTbCmp-offsetLCBX-nTbW)< <iBvShift;

[0235] iVerMin1=(yTbCmp+1)< <iBvShift;

[0236] iVerMax1=min(picHeight-nTbH,(yTbCmp-offsetLCBY+CtbSizeH-nTbH)< <iBvShift)。

[0237] When regionId is equal to 2, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0238] iHorMax2=(xTbCmp-offsetLCBX-nTbW)< <iBvShift;

[0239] iHorMin2=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift);

[0240] iVerMin2=max((iTemplateSizeH)< <iBvShift,(yTbCmp-nTbH-offsetLCBY)<<iBvShift);

[0241] iVerMax2=(yTbCmp)< <iBvShift。

[0242] When regionId is equal to 3, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0243] iHorMin3=max((iTemplateSizeW)< <iBvShift,(xTbCmp-offsetLCBX-nTbW+1)<<iBvShift);

[0244] iHorMax3=(xTbCmp-nTbW)< <iBvShift;

[0245] iVerMin3=max(((iTemplateSizeH)< <iBvShift),(yTbCmp-offsetLCBY-nTbH+1)<<iBvShift);

[0246] iVerMax3=(yTbCmp-nTbH)< <iBvShift。

[0247] In practical applications, the iHorMin regionId 、iHorMax regionId 、iVerMin regionId 、iVerMax regionId Represent the left edge, right edge, top edge and bottom edge of different sub-search areas respectively.

[0248] To intuitively describe the different sub-search regions corresponding to different regionIds, as shown in Figure 8, R1, R2, R3, and R4 represent four different sub-search regions. It should be noted that Figure 8 is represented by the pixel range that can be aligned with the upper left corner pixel of the block.

[0249] In some embodiments, taking (a) in FIG. 7 as an example, the search area is divided into four sub-search areas, which are implemented as follows:

[0250] When regionId is equal to 0, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0251] iHorMax0=min((xTbCmp+searchRangeWidth)< <iBvShift,((picWidth-nTbW)<<iBvShift))

[0252] iHorMin0=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift)

[0253] iVerMax0=(yTbCmp-nTbH-offsetLCBY)< <iBvShift

[0254] iVerMin0=max(((iTemplateSizeH)< <iBvShift),((yTbCmp-searchRangeHeight)<<iBvShift))。

[0255] When regionId is equal to 1, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0256] iHorMin1=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift)

[0257] iHorMax1=(xTbCmp-offsetLCBX-nTbW)< <iBvShift;

[0258] iVerMin1=(yTbCmp+1)< <iBvShift;

[0259] iVerMax1=min(picHeight-nTbH, (yTbCmp-offsetLCBY+CtbSizeH-nTbH)< <iBvShift)。

[0260] When regionId is equal to 2, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0261] iHorMax2=(xTbCmp-nTbW)< <iBvShift;

[0262] iHorMin2=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift);

[0263] iVerMin2=max((iTemplateSizeH)< <iBvShift,(yTbCmp-nTbH)<<iBvShift);

[0264] iVerMax2=(yTbCmp)< <iBvShift。

[0265] When regionId is equal to 3, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows

[0266] iHorMin3=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift);

[0267] iHorMax3=(xTbCmp)< <iBvShift;

[0268] iVerMin3=max(((iTemplateSizeH)< <iBvShift),(yTbCmp-offsetLCBY-nTbH+1)<<iBvShift);

[0269] iVerMax3=(yTbCmp-nTbH)< <iBvShift。

[0270] In practical applications, the iHorMin regionId 、iHorMax regionId 、iVerMin regionId 、iVerMax regionId Represent the left edge, right edge, top edge and bottom edge of different sub-search areas respectively.

[0271] To intuitively describe the different sub-search regions corresponding to different regionIds, R1, R2, R3, and R4 represent four different sub-search regions, as shown in Figure 8. It should be noted that Figure 8 is represented by the pixel range that can be aligned with the upper left corner pixel of the block.

[0272] In some embodiments, taking (b) in FIG. 7 as an example, the search area is divided into four sub-search areas, which are implemented as follows:

[0273] When regionId is equal to 0, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0274] iHorMax0=min((xTbCmp+searchRangeWidth)< <iBvShift,((picWidth-nTbW)<<iBvShift))

[0275] iHorMin0=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift)

[0276] iVerMax0=(yTbCmp-nTbH-offsetLCBY)< <iBvShift

[0277] iVerMin0=max(((iTemplateSizeH)< <iBvShift),((yTbCmp-searchRangeHeight)<<iBvShift))。

[0278] When regionId is equal to 1, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0279] iHorMin1=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift)

[0280] iHorMax1=(xTbCmp-offsetLCBX-nTbW)< <iBvShift;

[0281] iVerMin1=(yTbCmp+1)< <iBvShift;

[0282] iVerMax1=min(picHeight-nTbH, (yTbCmp-offsetLCBY+CtbSizeH-nTbH)< <iBvShift)。

[0283] When regionId is equal to 2, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0284] iHorMax2=(xTbCmp-nTbW)< <iBvShift;

[0285] iHorMin2=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift);

[0286] iVerMin2=max((iTemplateSizeH)< <iBvShift,(yTbCmp-nTbH-offsetLCBY)<<iBvShift);

[0287] iVerMax2=(yTbCmp)< <iBvShift。

[0288] When regionId is equal to 3, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0289] iHorMin3=max((iTemplateSizeW)< <iBvShift,(xTbCmp-nTbW+1)<<iBvShift);

[0290] iHorMax3=(xTbCmp)< <iBvShift;

[0291] iVerMin3=max(((iTemplateSizeH)< <iBvShift),(yTbCmp-offsetLCBY-nTbH+1)<<iBvShift);

[0292] iVerMax3=(yTbCmp-nTbH)< <iBvShift。

[0293] In practical applications, the iHorMin regionId 、iHorMax regionId 、iVerMin regionId 、iVerMax regionId Represent the left edge, right edge, top edge and bottom edge of different sub-search areas respectively.

[0294] To visually illustrate the different sub-search regions corresponding to different regionIds, see Figure 13, which shows a schematic diagram of the specific search region determination process. As shown in Figure 13, R1, R2, R3, and R4 represent four different sub-search regions. Note that Figure 13 represents the pixel range aligned with the top-left corner pixel of the block.

[0295] Furthermore, in an embodiment of the present application, when searching within a preset search area to determine one or more block vectors, it is possible to traverse the search points in the preset search area and determine the matching cost between the matching template corresponding to the search point in the preset search area and the first template based on a preset matching criterion; and then, based on the matching cost, it is possible to determine one or more block vectors and one or more candidate templates corresponding to the one or more block vectors.

[0296] It should be noted that, in the embodiment of the present application, the number of block vectors determined by the search process may be one or more. For example, N block vectors of the current block may be determined, where N is an integer greater than 0.

[0297] Accordingly, in an embodiment of the present application, when searching within a preset search area to determine one or more block vectors, a preset number N corresponding to candidate templates can be first determined; then, the search points in the preset search area are traversed, and matching cost values ​​between matching templates corresponding to the search points in the preset search area and the first template are determined based on a preset matching criterion; finally, N block vectors and N candidate templates corresponding to the N block vectors can be determined based on the matching cost values.

[0298] That is, in an embodiment of the present application, when searching for and determining N block vectors corresponding to N matching templates in a preset search area, that is, the process of searching for and determining the block vectors BV corresponding to N matching templates in the search area (preset search area) may include determining the value of the number N of candidate templates, determining the matching template comparison criteria and recording the N block vectors BV corresponding to the N matching templates (the selected N candidate templates).

[0299] It is understandable that in the embodiment of the present application, when determining the preset number N corresponding to the candidate template, N can be determined by decoding the code stream; N can also be determined according to a first preset value; or N can be determined according to a preset value range.

[0300] That is, in the embodiments of the present application, the value of N must first be determined. Here, N can be preset as a constant, such as 4. N can also be within a certain numerical range, such as any integer in [2, 8]. The range of N is preset, and the optimal N value can be determined at the encoding end by using a rough selection of cost 1, a rough selection of cost 2, or a rough selection of cost 3, a fine selection of cost 4, and the optimal N value is transmitted to the decoding end in the form of a bitstream. Among them, costs 1, 2, 3, and 4 can be one of the cost functions used to measure the mode, such as SAD, SATD, MSE, MAD, and RDO. This application does not specifically limit the method for determining N.

[0301] It should be noted that in the embodiments of the present application, the preset matching criteria include any one of the cost functions used to measure the pattern, such as the sum of absolute error SAD, the sum of transformed absolute error SATD, the sum of squared differences SSE, the mean absolute difference MAD, the mean absolute error MAE, the mean squared error MSE, and the normalized correlation coefficient NCC.

[0302] Furthermore, in an embodiment of the present application, when determining N block vectors and N candidate templates corresponding to the N block vectors based on the matching cost values, that is, when determining one or more block vectors and one or more candidate templates corresponding to the one or more block vectors, N minimum matching cost values ​​can be first determined among the matching cost values ​​between the matching template corresponding to the search point in the preset search area and the first template; and then the N block vectors and N candidate templates corresponding to the N minimum matching cost values ​​are determined.

[0303] It should be noted that, in the embodiment of the present application, for any one of the N block vectors, bvXMins and bvXMaxs can be selected to represent the minimum offset and maximum offset of the block vector in the horizontal direction, respectively; bvYMins and bvYMaxs can be selected to represent the minimum offset and maximum offset of the block vector in the vertical direction, respectively.

[0304] bvXMinsregionId, bvXMaxsregionId, bvYMinsregionId, bvYMaxsregionId can be calculated using the already determined iVerMinregionId, iVerMaxregionId, iHorMinregionId, iHorMaxregionId:

[0305] bvXMinsregionId=iHorMinregionId-xTbCmp;

[0306] bvXMaxsregionId=iHorMaxregionId-xTbCmp;

[0307] bvYMinsregionId=iVerMinregionId-yTbCmp;

[0308] bvYMaxsregionId=iVerMaxregionId-yTbCmp;

[0309] bvXMinsregionId, bvXMaxsregionId, bvYMinsregionId, and bvYMaxsregionId determine the horizontal and vertical offset ranges of the search point relative to the current block, that is, the range of the block vector BV.

[0310] It should also be noted that through the search points (iPosHor, iPoxVer) in each search area, that is, each block vector BV (composed of horizontal and vertical components: (pX, pY), where pX = iPosHor-xTbCmp, pY = iPosVer-yTbCmp, then pX is between bvXMins and bvXMaxs, and pY is between bvYMins and bvYMaxs), one or more matching reconstructed blocks of the current block can be found in the reconstructed area. The adjacent reconstructed pixels of the one or more matching reconstructed blocks are the matching templates. In this way, the matching costs of the adjacent templates of the current block and the adjacent templates of the one or more reconstructed blocks can be calculated respectively, which is recorded as pDiff.

[0311] Furthermore, all search points in all search ranges (regionId = 0, 1, 2, 3) are traversed, and one or more search points with the smallest matching cost pDiff are obtained by comparison. The corresponding matching cost is recorded as pDiff_BEST, and the corresponding one or more block vectors BV are recorded as the best block vector BV_BEST (pX_BEST, pY_BEST). The corresponding one or more matching templates are the best matching templates T_BEST, that is, one or more candidate templates are obtained.

[0312] It is understood that in the embodiment of the present application, after determining the number N of candidate templates, it is necessary to select N block vectors BV corresponding to the N more matching candidate templates based on a certain comparison criterion. That is, compared with the common related technology of searching for matching templates and recording block vectors BV, the difference of the present application is that it can select and record multiple block vectors BV. n , according to the block vector BV n , that is, according to the template offset pX n and pY n After obtaining the matching templates, the template matching cost is calculated and the N BVs corresponding to the N matching templates with the smallest cost are recorded. These N matching templates are referred to as N candidate templates. The template matching cost (preset matching criterion) can be one of the cost functions used to measure patterns, such as SAD, SATD, MSE, MAD, RDO, and correlation coefficient.

[0313] For example, in an embodiment of the present application, when the matching cost comparison criterion (preset matching criterion) is the mean absolute difference (MAD), the calculation formula is as follows:

[0314]

[0315] Where refT is the matching template in the search process, curT is the current coding block template (the first template of the current block), M is the number of pixels of the current coding block template, and MAD(refT) is the mean absolute difference between the current coding block template curT and the searched matching template.

[0316] Accordingly, in the embodiment of the present application, the MAD-based screening criterion is: comparing and recording N block vectors BV corresponding to N matching templates with the smallest MAD costs.

[0317] For example, for N candidate templates, the MAD between the nth candidate template and the current coding block template is as follows:

[0318]

[0319] Among them, refT n is the nth candidate template, MAD(refT n ) is the average absolute difference between the current coding block template curT and the nth candidate template, n=0,…,N-1.

[0320] For example, in an embodiment of the present application, when the matching cost comparison criterion (preset matching criterion) is SAD, the calculation formula is as follows:

[0321]

[0322] Wherein, SAD(refT) is the sum of absolute errors between the current coding block template (the first template of the current block) curT and the searched matching template.

[0323] Accordingly, in the embodiment of the present application, the SAD-based screening criterion is: comparing and recording N block vectors BV corresponding to N matching templates with smaller SAD costs.

[0324] For example, for N candidate templates, the SAD between the nth candidate template and the current coding block template is as follows:

[0325]

[0326] Among them, SAD(refT n ) is the sum of the absolute errors between the current coding block template and the nth candidate template.

[0327] For example, in an embodiment of the present application, the NCC normalized correlation coefficient may be used as a template matching criterion for comparison, and the calculation formula is as follows:

[0328]

[0329] Among them, refT is the matching template in the search process, curT is the current coding block template (the first template of the current block), M is the number of pixels of the current coding block template, refT Avg is the average pixel value of the searched matching template, curT Avg is the average pixel value of the current coding block template, and R(refT) is the correlation coefficient between the current coding block template and the searched matching template.

[0330] Accordingly, in the embodiment of the present application, the NCC comparison criterion is as follows: sort and record N block vectors BV corresponding to N matching templates with larger correlation coefficients R.

[0331] Among them, for N candidate templates, the correlation coefficient between the nth candidate template and the current coding block template is as follows:

[0332]

[0333] in, is the average pixel value of the nth candidate template, R(refT n ) is the correlation coefficient between the current coding block template and the nth candidate template. It is worth mentioning that the range of the NCC normalized correlation coefficient R is [-1, 1]. The larger R is, the stronger the correlation is.

[0334] It should be noted that in an embodiment of the present application, when performing search processing, the search strategies that can be adopted may include but are not limited to search methods based on different search step sizes, for example, a coarse search based on a first search step size and / or a fine search based on a second search step size, wherein the first search step size is larger than the second search step size.

[0335] Furthermore, in an embodiment of the present application, the block vector and the candidate template can be determined by traversing the search points in the preset search area according to a first search step size. Alternatively, the block vector and the candidate template can be determined by traversing the search points in the preset search area according to a second search step size.

[0336] Furthermore, in an embodiment of the present application, the search points in the preset search area can be traversed according to a first search step size to determine an initial block vector and an initial matching template corresponding to the initial block vector; then, a first search area can be determined based on the initial matching template; wherein the first search area is smaller than the preset search area; finally, the search points in the first search area can be traversed according to a second search step size to determine a block vector and a candidate template; wherein the first search step size is larger than the second search step size.

[0337] That is, in the embodiment of the present application, when searching for the best matching template in the search area, a search strategy of coarse search first and then fine search may be adopted, or only fine search may be performed, or only coarse search may be performed.

[0338] Exemplarily, in an embodiment of the present application, the coarse search may specifically include: determining the best coarse matching template in the search area with a first preset step size (i.e., the first search step size, for example, 2), that is, obtaining the final candidate template, or, using a downsampled template (for example, a downsampling factor of 2) to determine the best coarse matching template in the search area, that is, obtaining the final candidate template.

[0339] Exemplarily, in an embodiment of the present application, the fine search may specifically include: determining the best fine matching template in the search area with a second preset step size (i.e., the second search step size, for example, 1), that is, obtaining the final candidate template, or, determining the best fine matching template near the best coarse matching template after completing the coarse search, that is, obtaining the final candidate template.

[0340] Exemplarily, in an embodiment of the present application, if the search strategy is to perform only a coarse search, then it is possible to select within each region a search range where pX is between bvXMinsregionId and bvXMaxsregionId, and pY is between bvYMinsregionId and bvYMaxsregionId, and perform a coarse search with a step size greater than 1. For example, a coarse search is performed with a step size of 2 (i.e., the first search step size is 2) and the optimal one or more matching costs obtained by recording template matching is pDiff_BEST, and its corresponding one or more block vectors BV are recorded as the best block vector BV_BEST (pX_BEST, pY_BEST), that is, one or more block vectors corresponding to the current block are obtained by performing a search process in the preset search area, and the one or more matching templates corresponding to the optimal one or more matching costs can be the final one or more candidate templates.

[0341] Exemplarily, in an embodiment of the present application, if the search strategy is to perform only a fine search, then it can be selected that in each region, pX is between bvXMinsregionId and bvXMaxsregionId, and pY is within the search range between bvYMinsregionId and bvYMaxsregionId. For example, a fine search is performed with a step size of 1 (that is, the second search step size is 1) to record the optimal one or more matching costs obtained by template matching as pDiff_BEST, and its corresponding one or more block vectors BV are recorded as the best block vector BV_BEST (pX_BEST, pY_BEST), that is, one or more block vectors corresponding to the current block are obtained by performing a search process in the preset search area, and the one or more matching templates corresponding to the optimal one or more matching costs can be the final one or more candidate templates.

[0342] For example, in an embodiment of the present application, if the search strategy is to perform a coarse search first and then a fine search, as shown in FIG10 , a coarse search record template matching can be performed first with a step size of 2 (i.e., a first search step size of 2) to obtain the best coarse matching template (initial matching template), and then a step size of 1 (i.e., a second search step size of 1) can be used to determine the best fine matching template near the best coarse matching template, thereby obtaining the final candidate template.

[0343] In the coarse search stage, a coarse search can be performed in each region with pX between bvXMinsregionId and bvXMaxsregionId and pY between bvYMinsregionId and bvYMaxsregionId, with a step size greater than 1. For example, the best matching cost obtained by performing a coarse search with a step size of 2 and recording the template matching is pDiff1_BEST, and its corresponding block vector BV is recorded as the best block vector BV1_BEST (pX1_BEST, pY1_BEST), that is, the initial block vector, and its corresponding matching template is the initial matching template. At this time, the search region where the best matching search point is located is bestRegionId.

[0344] Next, in the fine search stage, you can choose to search near the best block vector BV1_BEST (initial block vector) obtained in the coarse search, that is, search in the first search area. Therefore, you first need to determine the refined search range TmpRefineRange, that is, you need to determine the first search area TmpRefineRange. Among them, the refined search range (first search area TmpRefineRange) can be a fixed size or related to the current block size. For example, it can be set to min(nTbW, nTbH) / 2, and then calculate the position of the best matching reconstructed block obtained in the coarse search as the reference position of the fine search area: BestPosX=xTbCmp+pX1_BEST, BestPosY=yTbCmp+pY1_BEST.

[0345] In one embodiment, the calculated values ​​of iVerMinbestRegionId, iVerMaxbestRegionId, iHorMinbestRegionId, and iHorMaxbestRegionId can be first obtained based on the value of bestRegionId, and then new search ranges iVerMinrefine, iVerMaxrefine, iHorMinrefine, and iHorMaxrefine can be obtained based on the optimal matching block position obtained by the coarse search. The obtaining method is as follows:

[0346] iHorMinrefine=max(iHorMinbestRegionId, BestPosX-TmpRefineRange)

[0347] iHorMaxrefine=min(iHorMaxbestRegionId, BestPosX+TmpRefineRange)

[0348] iVerMinrefine=max(iVerMinbestRegionId, BestPosY-TmpRefineRange)

[0349] iVerMaxrefine=min(iVerMaxbestRegionId, BestPosY+TmpRefineRange)

[0350] Then, the adjusted block vectors BVbvXMins, bvXMaxs, bvYMins, bvYMaxs can be calculated by iVerMinrefine, iVerMaxrefine, iHorMinrefine, iHorMaxrefine:

[0351] bvXMins=iHorMinrefine-xTbCmp;

[0352] bvXMaxs=iHorMaxrefine-xTbCmp;

[0353] bvYMins=iVerMinrefine-yTbCmp;

[0354] bvYMaxs=iVerMaxrefine-yTbCmp;

[0355] The refined search is performed within the block vector range where pX is between bvXMinsrefine and bvXMaxsrefine, and pY is between bvYMinsrefine and bvYMaxsrefine. For example, the search is performed with a step size of 1. The best match cost obtained by template matching is recorded as pDiff_BEST, and its corresponding block vector BV is recorded as the best block vector BV_BEST(pX_BEST, pY_BEST), which is the final block vector of the current block. The corresponding matching template is the candidate template for the current block.

[0356] After completing the above operations, the optimal block vector BV_BEST (pX_BEST, pY_BEST) can be obtained, where pX_BEST and pY_BEST are the horizontal offset and vertical offset of the best matching template relative to the current coding block template, respectively, and are also the horizontal offset and vertical offset of the best matching reconstructed block relative to the current coding block.

[0357] Thus, in the embodiments of the present application, the process of searching and determining one or more block vectors in the search area can include three search strategies: performing only a coarse search, performing only a fine search, and performing a coarse search followed by a fine search. Accordingly, if the search strategy is to perform only a coarse search, then only the N block vectors BV corresponding to the N matching templates are determined during the coarse search process, and the corresponding N matching templates are used as N candidate templates. If the search strategy is to perform only a fine search, then only the N block vectors BV corresponding to the N matching templates are determined during the fine search process, and the corresponding N matching templates are used as N candidate templates. If the search strategy is to perform a coarse search first, then a fine search later, then during the coarse search process, K block vectors BV corresponding to K preliminary matching templates (initial matching templates) can be first determined, where K is an integer greater than or equal to N. Based on these K preliminary matching templates, after the fine search process is executed, N block vectors BV corresponding to the final N matching templates are determined, and the corresponding N matching templates are used as N candidate templates.

[0358] Furthermore, in an embodiment of the present application, when performing a search process, the search points in a sub-search area of ​​a preset search area can be traversed according to a first search step size to determine an initial vector corresponding to the sub-search area and a second search area corresponding to the initial block vector; based on the initial block vector and the second search area, multiple target sub-search areas are determined in the sub-search area; and the search points in the multiple target sub-search areas are traversed according to a second search step size to determine a block vector and a candidate template; wherein the first search step size is greater than the second search step size.

[0359] It can be understood that in the embodiments of the present application, when determining multiple target sub-search areas in the sub-search area based on the initial block vector and the second search area, area judgment can be performed based on the best block vector of each sub-search area, that is, the initial block vector, and the corresponding second search area, so that multiple target sub-search areas used in the subsequent search processing process can be determined.

[0360] That is to say, in the embodiment of the present application, for the search strategy of coarse search first and search later, it is also possible to choose to perform fine search across different search areas. After completing the coarse search process, the method of performing the boundary of each area in the fine search process can be used.

[0361] For example, in an embodiment of the present application, taking (a) in FIG. 7 as an example, the search area is divided into four sub-search areas, which are implemented as follows:

[0362] When regionId is equal to 0, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0363] iHorMax0=min((xTbCmp+searchRangeWidth)< <iBvShift,((picWidth-nTbW)<<iBvShift))

[0364] iHorMin0=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift)

[0365] iVerMax0=(yTbCmp-nTbH-offsetLCBY)< <iBvShift

[0366] iVerMin0=max(((iTemplateSizeH)< <iBvShift),((yTbCmp-searchRangeHeight)<<iBvShift))。

[0367] When regionId is equal to 1, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0368] iHorMin1=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift)

[0369] iHorMax1=(xTbCmp-offsetLCBX-nTbW)< <iBvShift;

[0370] iVerMin1=(yTbCmp+1)< <iBvShift;

[0371] iVerMax1=min(picHeight-nTbH, (yTbCmp-offsetLCBY+CtbSizeH-nTbH)< <iBvShift)。

[0372] When regionId is equal to 2, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0373] iHorMax2=(xTbCmp-nTbW)< <iBvShift;

[0374] iHorMin2=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift);

[0375] iVerMin2=max((iTemplateSizeH)< <iBvShift,(yTbCmp-nTbH)<<iBvShift);

[0376] iVerMax2=(yTbCmp)< <iBvShift。

[0377] When regionId is equal to 3, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows

[0378] iHorMin3=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift);

[0379] iHorMax3=(xTbCmp)< <iBvShift;

[0380] iVerMin3=max(((iTemplateSizeH)< <iBvShift),(yTbCmp-offsetLCBY-nTbH+1)<<iBvShift);

[0381] iVerMax3=(yTbCmp-nTbH)< <iBvShift。

[0382] In practical applications, the iHorMin regionId 、iHorMax regionId 、iVerMin regionId 、iVerMax regionId Represent the left edge, right edge, top edge and bottom edge of different sub-search areas respectively.

[0383] To visually illustrate the different sub-search regions corresponding to different regionIds, see Figure 14, which shows a schematic diagram of the specific search region determination process. As shown in Figure 14, R1, R2, R3, and R4 represent four different sub-search regions. Note that Figure 14 represents the pixel range aligned with the top-left corner pixel of the block.

[0384] In the fine search phase, a search can be performed near each block vector obtained by the coarse search. Furthermore, a search can be performed near the best block vector BV1_BESTk obtained by the coarse search. Specifically, a refinement search range TmpRefineRange is first determined. The refinement search range can be a fixed size or related to the size of the current block. For example, it can be set to min(nTbW, nTbH) / 2. Then, the position of the best matching reconstructed block obtained by the coarse search is calculated as the reference position of the fine search area: BestPosXk=xTbCmp+pX1_BESTk, BestPosYk=yTbCmp+pY1_BESTk.

[0385] In some embodiments, the fine search performed based on each BestPosXk can be performed across multiple regions, that is, the fine search can be performed across different search regions. The value of bestRegionId involved in the operation can be determined first: for regionId = 0, 1, 2, 3, the following is judged: if BestPosYk-TmpRefineRange>=iVerMinregionId, and BestPosYk-TmpRefineRange<=iVerMaxregionId,

[0386] Or BestPosYk+TmpRefineRange>=iVerMinregionId, and BestPosYk+TmpRefineRange<=iVerMaxregionId,

[0387] Or BestPosYk-TmpRefineRange<=iVerMinregionId, and BestPosYk+TmpRefineRange>=iVerMaxregionId,

[0388] Then the regionId sets the corresponding bestSearchFlagregionId to 1.

[0389] For each search region with the value of bestSearchFlagregionId set to 1, set bestRegionId to the regionId of the region and perform the following refined search: first obtain the calculated values ​​of iVerMinbestRegionId, iVerMaxbestRegionId, iHorMinbestRegionId according to the value of bestRegionId, and then obtain the new search ranges iVerMinrefine, iVerMaxrefine, iHorMinrefine, and iHorMaxrefine based on the optimal matching block position obtained in the coarse search. The acquisition method is as follows:

[0390] iHorMinrefine=max(iHorMinbestRegionId, BestPosX-TmpRefineRange)

[0391] iHorMaxrefine=min(iHorMaxbestRegionId, BestPosX+TmpRefineRange)

[0392] iVerMinrefine=max(iVerMinbestRegionId, BestPosY-TmpRefineRange)

[0393] iVerMaxrefine=min(iVerMaxbestRegionId, BestPosY+TmpRefineRange)

[0394] Then, the adjusted block vectors BVbvXMins, bvXMaxs, bvYMins, bvYMaxs can be calculated by iVerMinrefine, iVerMaxrefine, iHorMinrefine, iHorMaxrefine:

[0395] bvXMins=iHorMinrefine-xTbCmp;

[0396] bvXMaxs=iHorMaxrefine-xTbCmp;

[0397] bvYMins=iVerMinrefine-yTbCmp;

[0398] bvYMaxs=iVerMaxrefine-yTbCmp;

[0399] Thus, the refined search is performed within the block vector range where pX is between bvXMinsrefine and bvXMaxsrefine, and pY is between bvYMinsrefine and bvYMaxsrefine. For example, the search is performed with a step size of 1. The best match cost obtained by template matching is recorded as pDiff_BEST, and the corresponding block vector BV is recorded as the best block vector BV_BEST(pX_BEST, pY_BEST).

[0400] Get the total best block vector of multiple search regions.

[0401] After completing the above search operation, the optimal block vector BV_BEST (pX_BEST, pY_BEST) can be obtained, where pX_BEST and pY_BEST are the horizontal offset and vertical offset of the best matching template relative to the current coding block template, respectively, and are also the horizontal offset and vertical offset of the best matching reconstructed block relative to the current coding block.

[0402] Step 103: Determine one or more reference blocks of the current block according to the one or more block vectors, and determine a prediction value of the current block according to the one or more reference blocks.

[0403] In an embodiment of the present application, after determining one or more block vectors corresponding to the current block according to the first template, one or more reference blocks of the current block can be further determined according to the one or more block vectors, and then the prediction value of the current block can be determined according to the one or more reference blocks.

[0404] It should be noted that, in an embodiment of the present application, one or more reference blocks of the current block include a first reference block and / or a second reference block; wherein the first reference block and the current block both belong to the current image, and the second reference block belongs to the reference image of the current image corresponding to the current block.

[0405] That is to say, in an embodiment of the present application, one or more reference blocks of the current block may be the first reference block in the current image obtained through intra-frame prediction, or the second reference block in the reference image of the current image obtained through inter-frame prediction; or may include both the first reference block in the current image obtained through intra-frame prediction and the second reference block in the reference image of the current image obtained through inter-frame prediction.

[0406] Accordingly, in an embodiment of the present application, when acquiring the second reference block, one or more block vectors can be determined by decoding the code stream; then a search can be performed in the reference image of the current image to determine the second reference block corresponding to the one or more block vectors.

[0407] That is to say, in an embodiment of the present application, it is possible to perform a template matching search within the current image by means of intra-frame template matching to determine one or more reference blocks corresponding to the current block, i.e., the first reference block; it is also possible to perform a template matching search on one or more inter-frame reference images of the current image by means of inter-frame template matching to determine one or more reference blocks corresponding to the current block, i.e., the second reference block; it is also possible to combine the intra-frame template matching method and the inter-frame template matching method to perform template matching searches within the current image and on one or more inter-frame reference images, respectively, to determine one or more reference blocks corresponding to the current block, including the first reference block and the second reference block.

[0408] Furthermore, in an embodiment of the present application, when determining one or more reference blocks corresponding to a current block based on one or more block vectors, one or more initial reconstructed blocks corresponding to the current block can be first determined based on the one or more block vectors; then, the one or more initial reconstructed blocks can be corrected to determine one or more reference blocks.

[0409] That is to say, in the embodiment of the present application, other methods may be used to obtain N candidate reconstructed blocks (reference blocks), for example, first correcting the initial reconstructed block corresponding to the obtained candidate template, and then determining the corresponding reference block.

[0410] Correspondingly, after obtaining N candidate reconstructed blocks (reference blocks) by copying N BV corresponding matching reconstructed blocks (initial reconstructed blocks), these N candidate reconstructed blocks can be directly weighted to obtain the predicted value of the current block; or the candidate reconstructed blocks (reference blocks) obtained after correcting the N BV corresponding matching reconstructed blocks (initial reconstructed blocks) can be weighted again to obtain the predicted value of the current block.

[0411] Furthermore, in an embodiment of the present application, when performing correction processing on one or more initial reconstructed blocks to determine one or more reference blocks, filtering processing may be performed on the one or more initial reconstructed blocks to determine one or more reference blocks.

[0412] Furthermore, in an embodiment of the present application, when performing correction processing on one or more initial reconstructed blocks to determine one or more reference blocks, one or more correction parameter vectors can be first determined based on one or more candidate templates corresponding to one or more block vectors; then, the one or more initial reconstructed blocks can be corrected based on the one or more correction parameter vectors to determine one or more reference blocks.

[0413] Furthermore, in an embodiment of the present application, when determining one or more correction parameter vectors based on one or more candidate templates corresponding to one or more block vectors, the autocorrelation matrix corresponding to the candidate template can be first determined based on the pixel values ​​in the candidate template; then, the cross-correlation vector can be determined based on the pixel values ​​in the first template and the pixel values ​​in the candidate template; and then, the correction parameter vector can be determined based on the autocorrelation matrix and the cross-correlation vector.

[0414] It can be understood that in an embodiment of the present application, the scheme for correcting the initial reconstructed block may include directly filtering the initial reconstructed block, wherein the filtering scheme adopted when performing the processing may be a traditional filtering method, such as bilateral filtering, mean filtering, etc., or it may be a filtering enhancement based on a neural network.

[0415] It is understandable that, in the embodiment of the present application, the solution of correcting the initial reconstructed block may further include correcting the matching reconstructed block (initial reconstructed block) using matching template information (candidate template).

[0416] For example, in the embodiment of the present application, when the matching template information is used to modify the matching reconstructed block, for each candidate template refT n And its corresponding candidate reconstruction block (initial reconstruction block) RefBlock n , using the candidate template refT n and the current coding block template (the first template of the current block) curT to calculate the correction parameter vector C n , the correction parameter vector C n Used to reconstruct the candidate block RefBlock n Perform weighted fusion to obtain the final corrected reconstruction block RefBlock' n , that is, the reference block of the current block is finally obtained as RefBlock' n .

[0417] In some embodiments, the modified parameter vector C n Can be passed through the candidate template refT n The reconstruction value and the template pixel value to be predicted are deduced by minimizing the MSE.

[0418] It can be understood that, in the embodiment of the present application, the correction parameter vector C n It can be regarded as an L-tap filter.

[0419] In some embodiments, when calculating the modified parameter vector C n When, for each candidate template refT n, where n = 0, 1…, N-1, the MSE minimization process takes the autocorrelation matrix of the candidate template sample refT, the cross-correlation vector of the candidate template sample refT and the adjacent template sample curT of the current coding block as input, and outputs the weight of the candidate reconstructed block corresponding to the current candidate template.

[0420] For example, in the embodiments of the present application, the MSE is calculated as follows:

[0421]

[0422] In order to conveniently express the calculation formula of MSE, E is used here to represent the mean square error MSE, that is:

[0423]

[0424] Where K is the number of pixels in the template.

[0425] In some embodiments, the weight c of the candidate template refT corresponding to the candidate reconstructed block is derived by minimizing the MSE. l The process may include the following steps:

[0426] (1) First, l Find the partial derivative and make it equal to 0:

[0427]

[0428]

[0429] Arranged:

[0430]

[0431] (2) After determining the candidate template area refT n Then expand the equation obtained in step (1) into a matrix form:

[0432]

[0433] (3) The autocorrelation matrix and cross-correlation vector in step (2) are both known quantities. Solving the linear equations in (2) can obtain the filter weight coefficients c0, ...c L-1 , that is, the modified parameter vector C n The filter coefficients of a candidate reconstructed block in .

[0434] Correspondingly, for x=0…nTbW-1, y=0…nTbH-1, the modified candidate reconstructed block RefBlock' n for:

[0435]

[0436] It should be noted that, in the embodiments of the present application, after determining the N block vectors BV corresponding to the N candidate templates of the current block, N candidate reconstructed blocks (i.e., N reference blocks) can be obtained through the N BVs, and then the N candidate reconstructed blocks are weightedly fused to obtain the prediction block of the current block (i.e., the prediction value of the current block). When generating the final prediction value, the N candidate reconstructed blocks (N reference blocks) can be first obtained, and then the corresponding weighted fusion weights (weight values) can be determined, and then the prediction value of the current block can be generated through weighted fusion.

[0437] It is understood that, in the embodiment of the present application, when determining one or more reference blocks of the current block according to one or more block vectors, the N block vectors BV corresponding to the obtained N candidate templates are n , you can choose to directly according to BV n N candidate reconstruction blocks (i.e., reference blocks) RefBlock obtained in the current image and / or reference image n , among which, BV n The horizontal offset is pX n , the vertical offset is pY n , where n=0, 1…, N-1.

[0438] For example, in an embodiment of the present application, when determining one or more reference blocks for the current block, a simple translation copy can be used. The specific operation is: for x = 0 ... nTbW-1, y = 0 ... nTbH-1, the reconstructed pixels of the current frame (i.e., the reference blocks of the current block) are determined by the following formula:

[0439] RefBlock n [x][y]=recSamples[x+pX n ][y+pY n ] (15)

[0440] Furthermore, in an embodiment of the present application, when determining the prediction value of the current block based on one or more reference blocks, one or more weight values ​​corresponding to the one or more reference blocks can be determined first; then, one or more reference blocks can be weightedly fused according to the one or more weight values, and finally the prediction value of the current block can be determined.

[0441] It should be noted that in the embodiments of the present application, after obtaining N candidate reconstructed blocks RefBlock (i.e., reference blocks for the current block), it is necessary to calculate the weight W for weighted fusion of the N candidate reconstructed blocks. The weight values ​​corresponding to the reference blocks can be determined in a variety of ways. For example, the weights can be predefined values ​​(e.g., a second preset value) or adaptively calculated using cost values, pixel values, and the like.

[0442] In one embodiment, one or more weight values ​​may be determined according to a second preset value. The second preset value may include N arbitrary values ​​greater than 0, and the corresponding weight values ​​for different reference blocks in the N reference blocks may be the same or different, which is not specifically limited in this application.

[0443] In one embodiment, one or more weight values ​​may be determined based on one or more candidate templates corresponding to one or more block vectors. When determining one or more weight values ​​based on one or more candidate templates corresponding to one or more block vectors, an autocorrelation matrix corresponding to the candidate template may be first determined based on pixel values ​​in the candidate template; a cross-correlation vector may then be determined based on pixel values ​​in the first template and pixel values ​​in the candidate template; and finally, a weight value may be determined based on the autocorrelation matrix and the cross-correlation vector.

[0444] In some embodiments, the candidate template refT n The reconstructed value and the template to be predicted (first template) refpredT n The pixel value of is minimized to minimize the MSE to derive the weighted fusion weight (weight value).

[0445] It should be noted that in order to make the weight derivation process more flexible, nonlinear terms and offset terms can be added in the process of deriving the weighted fusion weights.

[0446] For example, in the embodiment of the present application, when deriving the weighted fusion weight, the nonlinear term NonLinearTerm_T is constructed based on the candidate template. Among them, a candidate template with sequence number 0 can be selected from N candidate templates for construction. Accordingly, for m=0, 1..., M-1, the following formula is obtained:

[0447] NonLinearTerm_T n，m =(refT n，m *refT n，m +MidVal)>>bitDepth (16)

[0448] Among them, n is 0, 1, … or N - 1, representing any one of the N candidate templates; MidVal is 1<<(bitDepth - 1), and bitDepth is the image bit depth.

[0449] Exemplarily, in an embodiment of the present application, for each candidate reconstruction block (reference block) corresponding to the N candidate templates, when applying the weighted fusion weight, the non - linear term NonLinearTerm_Block is constructed based on the candidate reconstruction block. Among them, the candidate reconstruction block corresponding to the candidate template with serial number 0 can be selected. Correspondingly, for x = 0, 1…nTbW - 1, y = 0…nTbH - 1, there is the following formula:

[0450] NonLinearTerm_Block n，x，y =(refBlock n，x，y *refBlock n，x，y +MidVal)>>bitDepth (17)

[0451] Among them, n is 0, 1, … or N - 1, representing the candidate reconstruction block (reference block) corresponding to any one of the N candidate templates.

[0452] The offset value Bias in the process of deriving the weight and applying the weight can be any constant within the image pixel range [0, (1<<bitDepth)-1]. For example, Bias can be set to 1<<(bitDepth - 1).

[0453] Since BiasTerm is a constant, in the actual calculation process, it needs to be expanded into a matrix and represented as BiasTerm. Specifically:

[0454] For each of the N candidate templates:

[0455] For m = 0, 1…, M - 1:

[0456] BiasTerm m =Bias (18)

[0457] For each candidate reconstruction block corresponding to the N candidate templates:

[0458] For x = 0, 1…nTbW - 1, y = 0…nTbH - 1:

[0459] BiasTerm x，y =Bias (19)

[0460] It is understood that in the embodiment of the present application, after adding the nonlinear term and the offset term, N+2 weighted weights (weight values) need to be derived. For the convenience of description, the variable P is used here to record the number of weighted weights, where P = N+2, and the matching template sample / nonlinear term sample and offset term sample are uniformly referred to as matching reference sample refT N and refT N+1 , so all reference quantities involved in the operation can be uniformly expressed as refT p Similarly, the reconstructed block corresponding to the matching template, the reconstructed block corresponding to the matching template involved in the nonlinear term, and the offset term are uniformly referred to as candidate reconstructed samples refBlock p , where p = 0, 1…, P-1.

[0461] Exemplarily, in an embodiment of the present application, the MSE minimization process takes as input the autocorrelation matrix of the first P matching reference samples refT, the cross-correlation vector of the first P matching reference samples refT and the adjacent template samples curT of the current coding block, and outputs the weight of the reconstructed block corresponding to each matching reference item.

[0462] The MSE calculation formula is as follows:

[0463] For each pixel at the same position that matches the reference template, that is, for m=0, 1…, M-1:

[0464]

[0465] In order to conveniently express the calculation formula of MSE, E is used here to represent the mean square error MSE, that is:

[0466]

[0467] Minimize MSE to derive the weight w of each matching reference sample corresponding to the reconstructed block p The specific steps are as follows:

[0468] (1) First, w p Find the partial derivative and make it equal to 0:

[0469]

[0470]

[0471] Arranged:

[0472]

[0473] (2) After determining the matching reference sample area refT0, refT1, ...refT P-1 Then expand the equation obtained in step (1) into a matrix form:

[0474]

[0475] (3) The autocorrelation matrix and cross-correlation vector in step (2) are both known quantities. By solving the linear equations in (2), the weights w0,…w of each matching reference item corresponding to the reconstructed sample (reference block) can be calculated. P-1 .

[0476] In one embodiment, when determining one or more weight values ​​based on one or more candidate templates corresponding to one or more block vectors, a matching cost value between the first template and the candidate template may be determined first; and then the weight value may be determined based on the matching cost value.

[0477] Furthermore, in the embodiments of the present application, other methods may be used to calculate the weighted weights. For example, a nonlinear weight model may be used to calculate the weighted weights for each candidate reconstruction block (reference block) RefBlock according to the cost of N matching candidate templates. n Assign the corresponding weight w n .

[0478] It should be noted that, in the embodiments of the present application, the weight model may include but is not limited to a nonlinear normalization function, a nonlinear exponential normalization function, and the like.

[0479] For example, in an embodiment of the present application, the following nonlinear function can be selected to calculate the weight of each candidate reconstructed block (reference block), wherein the input of the weight model is the current coding block template curT and the candidate template refT n The matching cost between the current coding block template curT and the candidate template refT n SAD between (refT n )、MAD(refT n ), correlation coefficient R(refT n )wait,

[0480] In some embodiments, the calculation formula corresponding to the weight model is as follows:

[0481]

[0482] Here, offset is a preset value, for example, offset is 1.

[0483] In some embodiments, when the matching cost is the normalized correlation coefficient R(refT n ), the calculation formula corresponding to the weight model is as follows:

[0484]

[0485] In some embodiments, a Softmax function may also be used as a weight model, and the calculation formula is as follows:

[0486]

[0487] Here, S is a model control parameter. Under certain conditions, the parameter S can be adjusted to adjust the weight model. For example, the parameter S can be related to the current block size or the template type.

[0488] In some embodiments, in addition to the above nonlinear weight model, the weights of the candidate reconstructed blocks may be directly set to an average value, for example:

[0489]

[0490] In some embodiments, when using the candidate template refT n The reconstructed value and the template to be predicted (first template) refpredT n In the process of deriving weighted fusion weights (weight values) by minimizing the MSE of the pixel values, it is also possible to choose not to add nonlinear terms and offset terms.

[0491] For example, in the embodiments of the application, if the nonlinear term and the offset term are not added, then only N weighted weights (weight values) need to be derived. The MSE minimization process takes as input the autocorrelation matrix of the first N candidate template samples refT and the cross-correlation vector of the first P candidate template samples refT and the adjacent template samples curT of the current coding block, and outputs the weight of each candidate template corresponding to the candidate reconstructed block.

[0492] In some embodiments, for pixels at the same position of each candidate template, that is, for m=0, 1..., M-1, the MSE calculation formula is as follows:

[0493]

[0494] In order to conveniently express the calculation formula of MSE, E is used here to represent the mean square error MSE, that is:

[0495]

[0496] Minimize MSE to derive the weight w of each candidate template corresponding to the candidate reconstruction block n The specific steps are as follows:

[0497] (1) First, w n Find the partial derivative and make it equal to 0:

[0498]

[0499]

[0500] Arranged:

[0501]

[0502] (2) After determining the candidate template area refT0, refT1, ...refT N-1 Then expand the equation obtained in (1) into a matrix form:

[0503]

[0504] (3) The autocorrelation matrix and cross-correlation vector in step (2) are both known quantities. By solving the linear equations in (2), the weights w0,…w of each candidate template corresponding to the reconstructed sample can be calculated. N-1 .

[0505] In some embodiments, when using the candidate template refT n The reconstructed value and the template to be predicted (first template) refpredT n In the process of deriving the weighted fusion weight (weight value) by minimizing the MSE of the pixel value, it is also possible to choose to add only the offset term without adding the nonlinear term.

[0506] For example, in the embodiment of the application, if only the offset term is added without the nonlinear term, then N+1 weighted weights (weight values) need to be derived. For the convenience of description, the variable P is used here to record the number of weighted weights, where P = N+1, and the candidate template sample / offset item sample is uniformly referred to as the matching reference sample refT N and refT N+1 , so all reference quantities involved in the operation can be uniformly expressed as refT p Similarly, the candidate reconstruction blocks and offset items corresponding to the candidate templates are uniformly referred to as candidate reconstruction samples refBlock p , where p = 0, 1…, P-1.

[0507] In some embodiments, the MSE minimization process takes as input the autocorrelation matrix of the first P matching reference samples refT, the cross-correlation vector of the first P matching reference samples refT and the adjacent template samples curT of the current coding block, and outputs the weight of the reconstructed block corresponding to each matching reference item.

[0508] In some embodiments, for pixels at the same position of each candidate template, that is, for m=0, 1..., M-1, the MSE calculation formula is as follows:

[0509]

[0510] In order to conveniently express the calculation formula of MSE, E is used here to represent the mean square error MSE, that is:

[0511]

[0512] Minimize MSE to derive the weight w of each matching reference sample corresponding to the reconstructed block p The specific steps are as follows:

[0513] (1) First, w p Find the partial derivative and make it equal to 0:

[0514]

[0515]

[0516] Arranged:

[0517]

[0518] (2) After determining the matching reference sample area refT0, refT1, ...refT P-1 Then expand the equation obtained in step (1) into a matrix form:

[0519]

[0520] (3) The autocorrelation matrix and cross-correlation vector in step (2) are both known quantities. By solving the linear equations in (2), the weights w0,…w of each matching reference item corresponding to the reconstructed sample can be calculated. P-1 .

[0521] Step 104: Determine a reconstructed value of the current block according to the predicted value of the current block.

[0522] In an embodiment of the present application, after determining one or more reference blocks of the current block based on one or more block vectors and determining a prediction value of the current block based on the one or more reference blocks, a reconstructed value of the current block can be further determined based on the prediction value of the current block.

[0523] It should be noted that, in the embodiment of the present application, the code stream may be decoded first to determine the prediction residual corresponding to the current block; and then the reconstructed value of the current block may be further determined based on the prediction residual and the prediction value.

[0524] In summary, the decoding method proposed in steps 101 to 104 above is improved and optimized based on the common intra TMP technology, employing weighted fusion to propose an intra TMP fusion prediction method. Specifically, in the process of searching for and determining the block vector BV in the search area, at least one block vector for the current block can be selectively determined, i.e., multiple block vectors can be determined. Simultaneously, in generating the predicted value, at least one reference block corresponding to the at least one block vector is weightedly fused to obtain a predicted value for the current block, thereby determining a reconstructed value for the current block.

[0525] That is to say, the embodiment of the present application proposes the Intra TMP Fusion technology. After the current coding block template (the first template of the current block) searches for N block vectors BV corresponding to N candidate matching templates with the smallest matching cost within a predefined range, the N candidate reconstructed blocks (N reference blocks) corresponding to these N candidate matching templates (N candidate templates) are found through the N BVs, and then weighted fusion is performed according to a certain weight, and used as the prediction block (prediction value) of the current block.

[0526] It can be understood that the Intra TMP Fusion method proposed in the embodiment of the present application can improve the accuracy of the prediction value. Among them, after the current coding block template (the first template of the current block) searches for the N block vectors BV corresponding to the N candidate matching templates with the smallest matching cost within a predefined range, the N candidate reconstruction blocks (N reference blocks) corresponding to these N candidate matching templates (N candidate templates) are found through the N BVs, and then weighted fusion is performed according to certain weights as the prediction block of the current block. On the one hand, the reconstructed block information corresponding to different matching templates in the search process is fully considered, rather than simply considering the reconstructed block corresponding to the template with the smallest matching cost; on the other hand, weights are adaptively assigned to the candidate reconstructed blocks through the matching template information, which fully considers the different importance of different reconstructed block information to the prediction of the current block.

[0527] It is understood that the Intra TMP Fusion method proposed in the embodiments of this application can fully utilize the candidate reconstructed block information corresponding to different matching templates during the matching template search process. On the one hand, it fully utilizes the reconstructed block information corresponding to different matching templates during the search process, rather than simply considering the reconstructed block information corresponding to the template with the lowest matching cost. On the other hand, it fully utilizes the matching template information to adaptively assign weights to the candidate reconstructed blocks, fully considering the different importance of different reconstructed block information for predicting the current block.

[0528] It can be seen that the Intra TMP Fusion method proposed in the embodiment of the present application can, to a certain extent, avoid the decrease in prediction accuracy caused by inaccurate template matching basis or directly copying the reconstructed block as the prediction block.

[0529] Compared with common coding and decoding technologies, the Intra TMP Fusion method proposed in the embodiment of the present application is tested under All Intra conditions at a 24-frame interval, and can achieve BD-rate changes of -0.29%, -0.30%, and -0.39% on Y, Cb, and Cr, respectively (i.e., the average bit rate changes under the same PSNR).

[0530] An embodiment of the present application provides a decoding method, wherein the decoder determines a first template corresponding to the current block; determines one or more block vectors corresponding to the current block based on the first template; determines one or more reference blocks of the current block based on the one or more block vectors, and determines a predicted value of the current block based on the one or more reference blocks; and determines a reconstructed value of the current block based on the predicted value of the current block. It can be seen that in an embodiment of the present application, an Intra TMP Fusion prediction method is proposed, wherein at least one block vector of the current block can be selected and determined, and then at least one reference block corresponding to at least one block vector can be used to obtain the predicted value of the current block. In other words, the encoding and decoding method proposed in an embodiment of the present application fully takes into account the different importance of the reconstructed block information corresponding to different matching templates in the search process for predicting the current block, and can improve the accuracy of the prediction, thereby obtaining the best prediction effect.

[0531] An embodiment of the present application provides an encoding method, which is applied to an encoder. FIG15 is a flow chart of the encoding method provided in the embodiment of the present application. As shown in FIG15 , the encoding method performed by the encoder may include the following steps:

[0532] Step 201: Determine the first template corresponding to the current block.

[0533] In an embodiment of the present application, the first template corresponding to the current block may be determined first. When acquiring the first template, the template type corresponding to the current block may be determined first, and then the first template corresponding to the current block may be further determined based on the template type.

[0534] It should be noted that the encoding method of the embodiment of the present application is applied to an encoder. In addition, the encoding method may include an intra-frame prediction method, more specifically, a color component prediction method. The video image can be divided into multiple coding blocks, each coding block may include a first color component, a second color component, and a third color component, and the current block in the embodiment of the present application refers to the coding block in the video image that is currently to be intra-frame predicted.

[0535] Here, when the first color component needs to be predicted, the component to be predicted is the first color component; when the second color component needs to be predicted, the component to be predicted is the second color component; when the third color component needs to be predicted, the component to be predicted is the third color component. In addition, assuming that the current block predicts the first color component and the first color component is the luminance component, that is, the component to be predicted is the luminance component, then the current block can also be called a luminance block; or assuming that the current block predicts the second color component and the second color component is the chrominance component, that is, the component to be predicted is the chrominance component, then the current block can also be called a chrominance block.

[0536] It should also be noted that, in the embodiments of the present application, the reference pixel (Reference Sample) of the current block may refer to a reference pixel point adjacent to the current block; wherein, adjacent here may be spatially adjacent, but is not limited to this. For example, adjacent may also mean temporal adjacent, spatially and temporally adjacent, or even the reference pixel of the current block may be a reference pixel obtained by performing some processing on spatially adjacent, temporally adjacent, or spatially and temporally adjacent reference pixels, etc., and the embodiments of the present application do not impose any limitation thereto.

[0537] Furthermore, in an embodiment of the present application, the template type of the current block can be determined based on the reference pixels of the current block, wherein the reference pixels of the current block include at least one of the following: the left adjacent reference pixels of the current block, the upper adjacent reference pixels of the current block, the upper left adjacent reference pixels of the current block, the lower left adjacent reference pixels of the current block, and the upper right adjacent reference pixels of the current block.

[0538] It is understandable that in the embodiment of the present application, the reference pixels of the current block may include the adjacent reconstructed pixels of the current block, that is, the adjacent reconstructed pixels of the current block may be selected as a template to search for a matching template in a predefined search area.

[0539] It should be noted that, in an embodiment of the present application, the reference pixels of the current block, that is, the adjacent reconstructed pixels of the current block may include the upper reference pixel, the upper left reference pixel, the upper right reference pixel, the left reference pixel and the lower left reference pixel of the current block.

[0540] It can be understood that in the embodiment of the present application, when the template type of the current block is determined using the reference pixels of the current block, the template type can be classified and determined according to whether the adjacent reference pixels are available.

[0541] Further, in an embodiment of the present application, when determining the template type of the current block based on the reference pixels of the current block, if the left adjacent reference pixels of the current block, the upper adjacent reference pixels of the current block and the upper-left adjacent reference pixels of the current block are all available, the template type of the current block is determined to be a first value; if the left adjacent reference pixels of the current block are available, the template type of the current block is determined to be a second value; if the upper adjacent reference pixels of the current block are available, the template type of the current block is determined to be a third value; if the left adjacent reference pixels of the current block and the upper-left adjacent reference pixels of the current block are both available, the template type of the current block is determined to be a fourth value; if the left adjacent reference pixels of the current block and the lower-left adjacent reference pixels of the current block are both available, the template type of the current block is determined to be a fifth value; if the upper adjacent reference pixels of the current block and the upper-right adjacent reference pixels of the current block are both available, the template type of the current block is determined to be a sixth value.

[0542] It should be noted that in the embodiments of the present application, the first value, the second value, the third value, the fourth value, the fifth value, and the sixth value can be any values ​​and are not specifically limited in this application. For example, the first value, the second value, the third value, the fourth value, the fifth value, and the sixth value can be 1, 2, 3, 4, 5, and 6, respectively.

[0543] For example, in an embodiment of the present application, refTemplateType can be used to represent the template type. Accordingly, as shown in FIG3 above, the block filled with the grid is the current block, and the adjacent area of ​​the current block is the template T. Six template types are shown here.

[0544] Exemplarily, these six template types are as follows: when the upper left reference pixel, the upper reference pixel and the left reference pixel are all available, the value of refTemplateType is 1, and the template shape is shown in (a) in Figure 3; when only the left reference pixel is available, the value of refTemplateType is 2, and the template shape is shown in (b) in Figure 3; when only the upper reference pixel is available, the value of refTemplateType is 3, and the template shape is shown in (c) in Figure 3; when only the left reference pixel and the upper left reference pixel are available, the value of refTemplateType is 4, and the template shape is shown in (d) in Figure 3; when only the left reference pixel and the lower left reference pixel are available, the value of refTemplateType is 5, and the template shape is shown in (e) in Figure 3; when only the upper reference pixel and the upper right reference pixel are available, the value of refTemplateType is 6, and the template shape is shown in (f) in Figure 3.

[0545] Furthermore, in an embodiment of the present application, when determining the first template corresponding to the current block based on the template type, the template reference pixels of the current block can be determined first based on the template type and the template size corresponding to the template type, and then the first template of the current block can be determined based on the template reference pixels.

[0546] It should be noted that, in an embodiment of the present application, the first template of the current block may include template reference pixels of the current block, wherein the template reference pixels of the current block may be determined by the template type of the current block and the template size corresponding to the template type.

[0547] It should be noted that, in an embodiment of the present application, the first template of the current block can be composed of reconstructed pixels of one or more areas above, upper right, left, lower left, and upper left of the current block, that is, it can be composed of reference pixels of the current block.

[0548] It should be noted that in the embodiments of the present application, the template size corresponding to the template type can be pre-set. For example, when obtaining the left template, the template width templateW_size can be set to 4, and when obtaining the upper template, the template height templateH_size can be set to 4.

[0549] Accordingly, in an embodiment of the present application, by combining the value of the template type refTemplateType of the current block and the template size corresponding to the refTemplateType, it is possible to determine which part of the reconstructed pixels to obtain as the template reference pixels of the current block, and then determine the corresponding first template.

[0550] Exemplarily, in an embodiment of the present application, when the value of refTemplateType is 1, you can choose to obtain the left, upper left and upper reconstructed pixels of the current block; when the value of refTemplateType is 2, only the left 4 columns of reconstructed pixels of the current block are obtained; when the value of refTemplateType is 3, only the upper 4 rows of reconstructed pixels of the current coding block are obtained.

[0551] Of course, the value of the preset template size can be any integer greater than 0, and is not limited to 4, and this application does not make any specific limitation on this.

[0552] It can be understood that in the embodiment of the present application, in combination with the template type and the corresponding template size of the current block, the template reference pixel of the current block determined from the reference pixels of the current block can be the first template corresponding to the current block.

[0553] Step 202: Determine one or more block vectors corresponding to the current block according to the first template.

[0554] In an embodiment of the present application, after determining the first template corresponding to the current block, one or more block vectors corresponding to the current block may be further determined based on the first template.

[0555] It should be noted that, in an embodiment of the present application, the block vector search process may include an initialization process, determining a search area of ​​the first template in the current frame, searching and determining one or a pair of optimal block vectors in the search area. Therefore, when performing the search process, the initialization operation needs to be completed first.

[0556] Exemplarily, as shown in FIG5 , nTbW and nTbH represent the size of the current block, templateW_size and templateH_size represent the size of the template, and uiPatchWidth and uiPatchHeight represent the size of the block including the current block and its template.

[0557] Accordingly, during initial processing, uiPatchWidth can be initialized to nTbW + templateW_size, and uiPatchHeight can be initialized to nTbH + templateH_size, where templateW_size and templateH_size can be fixed constants or dynamically adjusted according to the size of the coding block. TemplateW_size and templateH_size can be equal or unequal. For example, templateW_size = 4, templateH_size = 4; or when the width of the coding block is greater than 8, set templateW_size = 4, when the width of the coding block is less than or equal to 8, set templateW_size = 2, when the height of the coding block is greater than 8, set templateH_size = 4, when the height of the coding block is less than or equal to 8, set templateH_size = 2.

[0558] Furthermore, the cost threshold between the initialization templates is represented by diffThreshold. For example, when the cost function is SAD, the threshold can be: diffThreshold = ((1 <<bitDepth)> >2)×(uiPatchHeight×uiPatchWidth-nTbH×nTbW). When the image bit depth bitDepth is 10, diffThreshold indicates that the maximum distortion of each pixel in the template area is 256.

[0559] Furthermore, the position of the coding tree block CTB where the current block CB is located is initialized: ctbRsX, ctbRsY.

[0560] Furthermore, the position offset of the current block CB in the current CTB is initialized: offsetLCBY=yTbCmp−ctbRsY, offsetLCBX=xTbCmp−ctbRsX.

[0561] Furthermore, initialize iTemplateSizeH=templateH_size and iTemplateSizeW=templateW_size.

[0562] Furthermore, iBvShift is initialized, where iBvShift is the precision of the block vector BV. For example, the precision of BV can be integer pixel precision, in which case iBvShift is 0; the precision of BV can also be sub-pixel precision, for example, iBvShift of 1 indicates 1 / 2 pixel precision, and iBvShift of 2 indicates 1 / 4 pixel precision. This is not specifically limited here.

[0563] Furthermore, the preset search range of the template is initialized. The preset search range of the template can be set to a fixed size or dynamically adjusted according to the encoding block size. For example, searchRangeWidth = TMP_SEARCH_RANGE_MULT_FACTOR × nTbW, searchRangeHeight = TMP_SEARCH_RANGE_MULT_FACTOR × nTbH; where the value of TMP_SEARCH_RANGE_MULT_FACTOR can be a preset value, such as 5.

[0564] Furthermore, in an embodiment of the present application, when determining one or more block vectors corresponding to the current block according to the first template, a preset search area can be first determined according to the first template; then a search can be performed within the preset search area to determine one or more block vectors.

[0565] It should be noted that in the embodiments of this application, the search area is the reconstructed portion of the current image and is limited by the size of the search range. As shown in Figure 6 above, the dark-filled background area is the reconstructed area, the black-filled background block is the current block, and the dashed box is the search range window. Therefore, the search area of ​​the IntraTMP technology is no larger than the overlap between the reconstructed area represented by the dark background and the area indicated by the dashed box.

[0566] As can be seen, the search area for the current block template can be the reconstructed portion of the CTB where the current block resides, or it can be another reconstructed CTB area. The search area here is actually the set of all search points. Because the shape of this area is often not representable by a single rectangular region, in practice, multiple rectangular regions can be searched, and the search results from these different regions can be combined to obtain the final best matching block and best block vector.

[0567] For example, see Figure 7 above, which shows eight different sub-region division methods. The black-filled background block is the current block; methods (a), (b), (c), (d), and (f) all divide the search area into four sub-search areas, and methods (e), (g), and (h) all divide the search area into three sub-search areas. Here, different fill patterns represent a sub-search area.

[0568] In Figure 7, (a), (b), (c), and (d) consider all available search ranges, and (e), (f), (g), and (h) do not search the area directly above and to the left.

[0569] For example, assuming that different search regions are represented by regionId, considering that the current coding block template pixels need to be retrieved in the image reconstruction region and the reconstructed block pixels corresponding to the template also need to be retrieved in the reconstruction region, it is necessary to further determine the positions that can be searched within the search regions represented by different regionIds based on the position of the current coding block (xTbCmp, yTbCmp), the size of the current coding block (nTbW, nTbH), the size of the current image (picWidth, picHeight), the size of the CTB in which the current coding block is located (CtbSizeW, CtbSizeH), the preset search range of the template (searchRangeWidth, searchRangeHeight), and the position offset of the current coding block within the current CTB (offsetLCBY, offsetLCBX), thereby determining the block vector BV. Specifically, iVerMin and iVerMax represent the absolute coordinate positions that can be searched at least and at most in the vertical direction, respectively, and iHorMin and iHorMax represent the absolute coordinate positions that can be searched at least and at most in the horizontal direction. The values ​​of iVerMin, iVerMax, iHorMin, and iHorMax are different in the search areas represented by different regionIds.

[0570] In some embodiments, taking (f) in FIG. 7 as an example, the search area is divided into four sub-search areas, which are implemented as follows:

[0571] When regionId is equal to 0, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0572] iHorMax0=min((xTbCmp+searchRangeWidth)< <iBvShift,((picWidth-nTbW)<<iBvShift));

[0573] iHorMin0=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift);

[0574] iVerMax0=(yTbCmp-nTbH-offsetLCBY)< <iBvShift;

[0575] iVerMin0=max(((iTemplateSizeH)< <iBvShift),((yTbCmp-searchRangeHeight)<<iBvShift))。

[0576] When regionId is equal to 1, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0577] iHorMin1=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift);

[0578] iHorMax1=(xTbCmp-offsetLCBX-nTbW)< <iBvShift;

[0579] iVerMin1=(yTbCmp+1)< <iBvShift;

[0580] iVerMax1=min(picHeight-nTbH,(yTbCmp-offsetLCBY+CtbSizeH-nTbH)< <iBvShift)。

[0581] When regionId is equal to 2, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0582] iHorMax2=(xTbCmp-offsetLCBX-nTbW)< <iBvShift;

[0583] iHorMin2=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift);

[0584] iVerMin2=max((iTemplateSizeH)< <iBvShift,(yTbCmp-nTbH-offsetLCBY)<<iBvShift);

[0585] iVerMax2=(yTbCmp)< <iBvShift。

[0586] When regionId is equal to 3, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0587] iHorMin3=max((iTemplateSizeW)< <iBvShift,(xTbCmp-offsetLCBX-nTbW+1)<<iBvShift);

[0588] iHorMax3=(xTbCmp-nTbW)< <iBvShift;

[0589] iVerMin3=max(((iTemplateSizeH)< <iBvShift),(yTbCmp-offsetLCBY-nTbH+1)<<iBvShift);

[0590] iVerMax3=(yTbCmp-nTbH)< <iBvShift。

[0591] In practical applications, the iHorMin regionId 、iHorMax regionId 、iVerMin regionId 、iVerMax regionId Represent the left edge, right edge, top edge and bottom edge of different sub-search areas respectively.

[0592] To intuitively describe the different sub-search regions corresponding to different regionIds, as shown in Figure 8, R1, R2, R3, and R4 represent four different sub-search regions. It should be noted that Figure 8 is represented by the pixel range that can be aligned with the upper left corner pixel of the block.

[0593] In some embodiments, taking (a) in FIG. 7 as an example, the search area is divided into four sub-search areas, which are implemented as follows:

[0594] When regionId is equal to 0, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0595] iHorMax0=min((xTbCmp+searchRangeWidth)< <iBvShift,((picWidth-nTbW)<<iBvShift))

[0596] iHorMin0=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift)

[0597] iVerMax0=(yTbCmp-nTbH-offsetLCBY)< <iBvShift

[0598] iVerMin0=max(((iTemplateSizeH)< <iBvShift),((yTbCmp-searchRangeHeight)<<iBvShift))。

[0599] When regionId is equal to 1, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0600] iHorMin1=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift)

[0601] iHorMax1=(xTbCmp-offsetLCBX-nTbW)< <iBvShift;

[0602] iVerMin1=(yTbCmp+1)< <iBvShift;

[0603] iVerMax1=min(picHeight-nTbH, (yTbCmp-offsetLCBY+CtbSizeH-nTbH)< <iBvShift)。

[0604] When regionId is equal to 2, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0605] iHorMax2=(xTbCmp-nTbW)< <iBvShift;

[0606] iHorMin2=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift);

[0607] iVerMin2=max((iTemplateSizeH)< <iBvShift,(yTbCmp-nTbH)<<iBvShift);

[0608] iVerMax2=(yTbCmp)< <iBvShift。

[0609] When regionId is equal to 3, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows

[0610] iHorMin3=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift);

[0611] iHorMax3=(xTbCmp)< <iBvShift;

[0612] iVerMin3=max(((iTemplateSizeH)< <iBvShift),(yTbCmp-offsetLCBY-nTbH+1)<<iBvShift);

[0613] iVerMax3=(yTbCmp-nTbH)< <iBvShift。

[0614] In practical applications, the iHorMin regionId 、iHorMax regionId 、iVerMin regionId 、iVerMax regionId Represent the left edge, right edge, top edge and bottom edge of different sub-search areas respectively.

[0615] To intuitively describe the different sub-search regions corresponding to different regionIds, R1, R2, R3, and R4 represent four different sub-search regions, as shown in Figure 8. It should be noted that Figure 8 is represented by the pixel range that can be aligned with the upper left corner pixel of the block.

[0616] In some embodiments, taking (b) in FIG. 7 as an example, the search area is divided into four sub-search areas, which are implemented as follows:

[0617] When regionId is equal to 0, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0618] iHorMax0=min((xTbCmp+searchRangeWidth)< <iBvShift,((picWidth-nTbW)<<iBvShift))

[0619] iHorMin0=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift)

[0620] iVerMax0=(yTbCmp-nTbH-offsetLCBY)< <iBvShift

[0621] iVerMin0=max(((iTemplateSizeH)< <iBvShift),((yTbCmp-searchRangeHeight)<<iBvShift))。

[0622] When regionId is equal to 1, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0623] iHorMin1=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift)

[0624] iHorMax1=(xTbCmp-offsetLCBX-nTbW)< <iBvShift;

[0625] iVerMin1=(yTbCmp+1)< <iBvShift;

[0626] iVerMax1=min(picHeight-nTbH, (yTbCmp-offsetLCBY+CtbSizeH-nTbH)< <iBvShift)。

[0627] When regionId is equal to 2, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0628] iHorMax2=(xTbCmp-nTbW)< <iBvShift;

[0629] iHorMin2=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift);

[0630] iVerMin2=max((iTemplateSizeH)< <iBvShift,(yTbCmp-nTbH-offsetLCBY)<<iBvShift);

[0631] iVerMax2=(yTbCmp)< <iBvShift。

[0632] When regionId is equal to 3, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0633] iHorMin3=max((iTemplateSizeW)< <iBvShift,(xTbCmp-nTbW+1)<<iBvShift);

[0634] iHorMax3=(xTbCmp)< <iBvShift;

[0635] iVerMin3=max(((iTemplateSizeH)< <iBvShift),(yTbCmp-offsetLCBY-nTbH+1)<<iBvShift);

[0636] iVerMax3=(yTbCmp-nTbH)< <iBvShift。

[0637] In practical applications, the iHorMin regionId 、iHorMax regionId 、iVerMin regionId 、iVerMax regionId Represent the left edge, right edge, top edge and bottom edge of different sub-search areas respectively.

[0638] To intuitively describe the different sub-search regions corresponding to different regionIds, R1, R2, R3, and R4 represent four different sub-search regions, as shown in Figure 13. It should be noted that Figure 13 is expressed in terms of the pixel range that can be aligned with the upper left corner pixel of the block.

[0639] Furthermore, in an embodiment of the present application, when searching within a preset search area to determine one or more block vectors, it is possible to traverse the search points in the preset search area and determine the matching cost between the matching template corresponding to the search point in the preset search area and the first template based on a preset matching criterion; and then, based on the matching cost, it is possible to determine one or more block vectors and one or more candidate templates corresponding to the one or more block vectors.

[0640] It should be noted that, in the embodiment of the present application, the number of block vectors determined by the search process may be one or more. For example, N block vectors of the current block may be determined, where N is an integer greater than 0.

[0641] Accordingly, in an embodiment of the present application, when searching within a preset search area to determine one or more block vectors, a preset number N corresponding to candidate templates can be first determined; then, the search points in the preset search area are traversed, and matching cost values ​​between matching templates corresponding to the search points in the preset search area and the first template are determined based on a preset matching criterion; finally, N block vectors and N candidate templates corresponding to the N block vectors can be determined based on the matching cost values.

[0642] That is, in an embodiment of the present application, when searching for and determining N block vectors corresponding to N matching templates in a preset search area, that is, the process of searching for and determining the block vectors BV corresponding to N matching templates in the search area (preset search area) may include determining the value of the number N of candidate templates, determining the matching template comparison criteria and recording the N block vectors BV corresponding to the N matching templates (the selected N candidate templates).

[0643] It can be understood that in the embodiment of the present application, when determining the preset number N corresponding to the candidate template, you can choose to determine N according to a preset decision criterion; you can also determine N according to a first preset value; you can also determine N according to a preset value range.

[0644] That is, in the embodiments of the present application, the value of N must first be determined. Here, N can be preset as a constant, such as 4. N can also be within a certain numerical range, such as any integer in [2, 8]. The range of N is preset, and N can be determined at the encoding end according to a preset decision criterion. For example, the optimal N value can be determined by using a rough selection of cost 1, a rough selection of cost 2, or a rough selection of cost 3, a fine selection of cost 4, and the optimal N value is transmitted to the decoding end in the form of a bitstream. The preset decision criterion for costs 1, 2, 3, and 4 can be one of the cost functions used to measure the mode, such as SAD, SATD, MSE, MAD, and RDO. This application does not specifically limit the method for determining N.

[0645] It should be noted that in the embodiments of the present application, the preset matching criteria include any one of the cost functions used to measure the pattern, such as the sum of absolute error SAD, the sum of transformed absolute error SATD, the sum of squared differences SSE, the mean absolute difference MAD, the mean absolute error MAE, the mean squared error MSE, and the normalized correlation coefficient NCC.

[0646] Furthermore, in an embodiment of the present application, when determining N block vectors and N candidate templates corresponding to the N block vectors based on the matching cost values, that is, when determining one or more block vectors and one or more candidate templates corresponding to the one or more block vectors, N minimum matching cost values ​​can be first determined among the matching cost values ​​between the matching template corresponding to the search point in the preset search area and the first template; and then the N block vectors and N candidate templates corresponding to the N minimum matching cost values ​​are determined.

[0647] It should be noted that, in the embodiment of the present application, for any one of the N block vectors, bvXMins and bvXMaxs can be selected to represent the minimum offset and maximum offset of the block vector in the horizontal direction, respectively; bvYMins and bvYMaxs can be selected to represent the minimum offset and maximum offset of the block vector in the vertical direction, respectively.

[0648] bvXMinsregionId, bvXMaxsregionId, bvYMinsregionId, bvYMaxsregionId can be calculated using the already determined iVerMinregionId, iVerMaxregionId, iHorMinregionId, iHorMaxregionId:

[0649] bvXMinsregionId=iHorMinregionId-xTbCmp;

[0650] bvXMaxsregionId=iHorMaxregionId-xTbCmp;

[0651] bvYMinsregionId=iVerMinregionId-yTbCmp;

[0652] bvYMaxsregionId=iVerMaxregionId-yTbCmp;

[0653] bvXMinsregionId, bvXMaxsregionId, bvYMinsregionId, and bvYMaxsregionId determine the horizontal and vertical offset ranges of the search point relative to the current block, that is, the range of the block vector BV.

[0654] It should also be noted that through the search points (iPosHor, iPoxVer) in each search area, that is, each block vector BV (composed of horizontal and vertical components: (pX, pY), where pX = iPosHor-xTbCmp, pY = iPosVer-yTbCmp, then pX is between bvXMins and bvXMaxs, and pY is between bvYMins and bvYMaxs), one or more matching reconstructed blocks of the current block can be found in the reconstructed area. The adjacent reconstructed pixels of the one or more matching reconstructed blocks are the matching templates. In this way, the matching costs of the adjacent templates of the current block and the adjacent templates of the one or more reconstructed blocks can be calculated respectively, which is recorded as pDiff.

[0655] Furthermore, all search points in all search ranges (regionId = 0, 1, 2, 3) are traversed, and one or more search points with the smallest matching cost pDiff are obtained by comparison. The corresponding matching cost is recorded as pDiff_BEST, and the corresponding one or more block vectors BV are recorded as the best block vector BV_BEST (pX_BEST, pY_BEST). The corresponding one or more matching templates are the best matching templates T_BEST, that is, one or more candidate templates are obtained.

[0656] It is understood that in the embodiment of the present application, after determining the number N of candidate templates, it is necessary to select N block vectors BV corresponding to the N more matching candidate templates based on a certain comparison criterion. That is, compared with the common related technology of searching for matching templates and recording block vectors BV, the difference of the present application is that it can select and record multiple block vectors BV. n , according to the block vector BV n , that is, according to the template offset pX n and pY n After obtaining the matching templates, the template matching cost is calculated and the N BVs corresponding to the N matching templates with the smallest cost are recorded. These N matching templates are referred to as N candidate templates. The template matching cost (preset matching criterion) can be one of the cost functions used to measure patterns, such as SAD, SATD, MSE, MAD, RDO, and correlation coefficient.

[0657] For example, in an embodiment of the present application, when the matching cost comparison criterion (preset matching criterion) is the mean absolute difference (MAD), the calculation formula is as follows:

[0658]

[0659] Where refT is the matching template in the search process, curT is the current coding block template (the first template of the current block), M is the number of pixels of the current coding block template, and MAD(refT) is the mean absolute difference between the current coding block template curT and the searched matching template.

[0660] Accordingly, in the embodiment of the present application, the MAD-based screening criterion is: comparing and recording N block vectors BV corresponding to N matching templates with the smallest MAD costs.

[0661] For example, for N candidate templates, the MAD between the nth candidate template and the current coding block template is as follows:

[0662]

[0663] Among them, refT n is the nth candidate template, MAD(refT n ) is the average absolute difference between the current coding block template curT and the nth candidate template, n=0,…,N-1.

[0664] For example, in an embodiment of the present application, when the matching cost comparison criterion (preset matching criterion) is SAD, the calculation formula is as follows:

[0665]

[0666] Wherein, SAD(refT) is the sum of absolute errors between the current coding block template (the first template of the current block) curT and the searched matching template.

[0667] Accordingly, in the embodiment of the present application, the SAD-based screening criterion is: comparing and recording N block vectors BV corresponding to N matching templates with smaller SAD costs.

[0668] For example, for N candidate templates, the SAD between the nth candidate template and the current coding block template is as follows:

[0669]

[0670] Among them, SAD(refT n ) is the sum of the absolute errors between the current coding block template and the nth candidate template.

[0671] For example, in an embodiment of the present application, the NCC normalized correlation coefficient may be used as a template matching criterion for comparison, and the calculation formula is as follows:

[0672]

[0673] Among them, refT is the matching template in the search process, curT is the current coding block template (the first template of the current block), M is the number of pixels of the current coding block template, refT Avg is the average pixel value of the searched matching template, curT Avg is the average pixel value of the current coding block template, and R(refT) is the correlation coefficient between the current coding block template and the searched matching template.

[0674] Accordingly, in the embodiment of the present application, the NCC comparison criterion is as follows: sort and record N block vectors BV corresponding to N matching templates with larger correlation coefficients R.

[0675] Among them, for N candidate templates, the correlation coefficient between the nth candidate template and the current coding block template is as follows:

[0676]

[0677] in, is the average pixel value of the nth candidate template, R(refT n ) is the correlation coefficient between the current coding block template and the nth candidate template. It is worth mentioning that the range of the NCC normalized correlation coefficient R is [-1, 1]. The larger R is, the stronger the correlation is.

[0678] It should be noted that in an embodiment of the present application, when performing search processing, the search strategies that can be adopted may include but are not limited to search methods based on different search step sizes, for example, a coarse search based on a first search step size and / or a fine search based on a second search step size, wherein the first search step size is larger than the second search step size.

[0679] Furthermore, in an embodiment of the present application, the block vector and the candidate template can be determined by traversing the search points in the preset search area according to a first search step size. Alternatively, the block vector and the candidate template can be determined by traversing the search points in the preset search area according to a second search step size.

[0680] Furthermore, in an embodiment of the present application, the search points in the preset search area can be traversed according to a first search step size to determine an initial block vector and an initial matching template corresponding to the initial block vector; then, a first search area can be determined based on the initial matching template; wherein the first search area is smaller than the preset search area; finally, the search points in the first search area can be traversed according to a second search step size to determine a block vector and a candidate template; wherein the first search step size is larger than the second search step size.

[0681] That is, in the embodiment of the present application, when searching for the best matching template in the search area, a search strategy of coarse search first and then fine search may be adopted, or only fine search may be performed, or only coarse search may be performed.

[0682] Exemplarily, in an embodiment of the present application, the coarse search may specifically include: determining the best coarse matching template in the search area with a first preset step size (i.e., the first search step size, for example, 2), that is, obtaining the final candidate template, or, using a downsampled template (for example, a downsampling factor of 2) to determine the best coarse matching template in the search area, that is, obtaining the final candidate template.

[0683] Exemplarily, in an embodiment of the present application, the fine search may specifically include: determining the best fine matching template in the search area with a second preset step size (i.e., the second search step size, for example, 1), that is, obtaining the final candidate template, or, determining the best fine matching template near the best coarse matching template after completing the coarse search, that is, obtaining the final candidate template.

[0684] Exemplarily, in an embodiment of the present application, if the search strategy is to perform only a coarse search, then it is possible to select within each region a search range where pX is between bvXMinsregionId and bvXMaxsregionId, and pY is between bvYMinsregionId and bvYMaxsregionId, and perform a coarse search with a step size greater than 1. For example, a coarse search is performed with a step size of 2 (i.e., the first search step size is 2) and the optimal one or more matching costs obtained by recording template matching is pDiff_BEST, and its corresponding one or more block vectors BV are recorded as the best block vector BV_BEST (pX_BEST, pY_BEST), that is, one or more block vectors corresponding to the current block are obtained by performing a search process in the preset search area, and the one or more matching templates corresponding to the optimal one or more matching costs can be the final one or more candidate templates.

[0685] Exemplarily, in an embodiment of the present application, if the search strategy is to perform only a fine search, then it can be selected that in each region, pX is between bvXMinsregionId and bvXMaxsregionId, and pY is within the search range between bvYMinsregionId and bvYMaxsregionId. For example, a fine search is performed with a step size of 1 (that is, the second search step size is 1) to record the optimal one or more matching costs obtained by template matching as pDiff_BEST, and its corresponding one or more block vectors BV are recorded as the best block vector BV_BEST (pX_BEST, pY_BEST), that is, one or more block vectors corresponding to the current block are obtained by performing a search process in the preset search area, and the one or more matching templates corresponding to the optimal one or more matching costs can be the final one or more candidate templates.

[0686] For example, in an embodiment of the present application, if the search strategy is to perform a coarse search first and then a fine search, as shown in FIG10 , a coarse search record template matching can be performed first with a step size of 2 (i.e., a first search step size of 2) to obtain the best coarse matching template (initial matching template), and then a step size of 1 (i.e., a second search step size of 1) can be used to determine the best fine matching template near the best coarse matching template, thereby obtaining the final candidate template.

[0687] In the coarse search stage, a coarse search can be performed in each region with pX between bvXMinsregionId and bvXMaxsregionId and pY between bvYMinsregionId and bvYMaxsregionId, with a step size greater than 1. For example, the best matching cost obtained by performing a coarse search with a step size of 2 and recording the template matching is pDiff1_BEST, and its corresponding block vector BV is recorded as the best block vector BV1_BEST (pX1_BEST, pY1_BEST), that is, the initial block vector, and its corresponding matching template is the initial matching template. At this time, the search region where the best matching search point is located is bestRegionId.

[0688] Next, in the fine search stage, you can choose to search near the best block vector BV1_BEST (initial block vector) obtained in the coarse search, that is, search in the first search area. Therefore, you first need to determine the refined search range TmpRefineRange, that is, you need to determine the first search area TmpRefineRange. Among them, the refined search range (first search area TmpRefineRange) can be a fixed size or related to the current block size. For example, it can be set to min(nTbW, nTbH) / 2, and then calculate the position of the best matching reconstructed block obtained in the coarse search as the reference position of the fine search area: BestPosX=xTbCmp+pX1_BEST, BestPosY=yTbCmp+pY1_BEST.

[0689] In one embodiment, the calculated values ​​of iVerMinbestRegionId, iVerMaxbestRegionId, iHorMinbestRegionId, and iHorMaxbestRegionId can be first obtained based on the value of bestRegionId, and then new search ranges iVerMinrefine, iVerMaxrefine, iHorMinrefine, and iHorMaxrefine can be obtained based on the optimal matching block position obtained by the coarse search. The obtaining method is as follows:

[0690] iHorMinrefine=max(iHorMinbestRegionId, BestPosX-TmpRefineRange)

[0691] iHorMaxrefine=min(iHorMaxbestRegionId, BestPosX+TmpRefineRange)

[0692] iVerMinrefine=max(iVerMinbestRegionId, BestPosY-TmpRefineRange)

[0693] iVerMaxrefine=min(iVerMaxbestRegionId, BestPosY+TmpRefineRange)

[0694] Then, the adjusted block vectors BVbvXMins, bvXMaxs, bvYMins, bvYMaxs can be calculated by iVerMinrefine, iVerMaxrefine, iHorMinrefine, iHorMaxrefine:

[0695] bvXMins=iHorMinrefine-xTbCmp;

[0696] bvXMaxs=iHorMaxrefine-xTbCmp;

[0697] bvYMins=iVerMinrefine-yTbCmp;

[0698] bvYMaxs=iVerMaxrefine-yTbCmp;

[0699] The refined search is performed within the block vector range where pX is between bvXMinsrefine and bvXMaxsrefine, and pY is between bvYMinsrefine and bvYMaxsrefine. For example, the search is performed with a step size of 1. The best match cost obtained by template matching is recorded as pDiff_BEST, and its corresponding block vector BV is recorded as the best block vector BV_BEST(pX_BEST, pY_BEST), which is the final block vector of the current block. The corresponding matching template is the candidate template for the current block.

[0700] After completing the above operations, the optimal block vector BV_BEST (pX_BEST, pY_BEST) can be obtained, where pX_BEST and pY_BEST are the horizontal offset and vertical offset of the best matching template relative to the current coding block template, respectively, and are also the horizontal offset and vertical offset of the best matching reconstructed block relative to the current coding block.

[0701] Thus, in the embodiments of the present application, the process of searching and determining one or more block vectors in the search area can include three search strategies: performing only a coarse search, performing only a fine search, and performing a coarse search followed by a fine search. Accordingly, if the search strategy is to perform only a coarse search, then only the N block vectors BV corresponding to the N matching templates are determined during the coarse search process, and the corresponding N matching templates are used as N candidate templates. If the search strategy is to perform only a fine search, then only the N block vectors BV corresponding to the N matching templates are determined during the fine search process, and the corresponding N matching templates are used as N candidate templates. If the search strategy is to perform a coarse search first, then a fine search later, then during the coarse search process, K block vectors BV corresponding to K preliminary matching templates (initial matching templates) can be first determined, where K is an integer greater than or equal to N. Based on these K preliminary matching templates, after the fine search process is executed, N block vectors BV corresponding to the final N matching templates are determined, and the corresponding N matching templates are used as N candidate templates.

[0702] Furthermore, in an embodiment of the present application, when performing a search process, the search points in a sub-search area of ​​a preset search area can be traversed according to a first search step size to determine an initial vector corresponding to the sub-search area and a second search area corresponding to the initial block vector; based on the initial block vector and the second search area, multiple target sub-search areas are determined in the sub-search area; and the search points in the multiple target sub-search areas are traversed according to a second search step size to determine a block vector and a candidate template; wherein the first search step size is greater than the second search step size.

[0703] It can be understood that in the embodiments of the present application, when determining multiple target sub-search areas in the sub-search area based on the initial block vector and the second search area, area judgment can be performed based on the best block vector of each sub-search area, that is, the initial block vector, and the corresponding second search area, so that multiple target sub-search areas used in the subsequent search processing process can be determined.

[0704] That is to say, in the embodiment of the present application, for the search strategy of coarse search first and search later, it is also possible to choose to perform fine search across different search areas. After completing the coarse search process, the method of performing the boundary of each area in the fine search process can be used.

[0705] For example, in an embodiment of the present application, taking (a) in FIG. 7 as an example, the search area is divided into four sub-search areas, which are implemented as follows:

[0706] When regionId is equal to 0, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0707] iHorMax0=min((xTbCmp+searchRangeWidth)< <iBvShift,((picWidth-nTbW)<<iBvShift))

[0708] iHorMin0=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift)

[0709] iVerMax0=(yTbCmp-nTbH-offsetLCBY)< <iBvShift

[0710] iVerMin0=max(((iTemplateSizeH)< <iBvShift),((yTbCmp-searchRangeHeight)<<iBvShift))。

[0711] When regionId is equal to 1, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0712] iHorMin1=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift)

[0713] iHorMax1=(xTbCmp-offsetLCBX-nTbW)< <iBvShift;

[0714] iVerMin1=(yTbCmp+1)< <iBvShift;

[0715] iVerMax1=min(picHeight-nTbH, (yTbCmp-offsetLCBY+CtbSizeH-nTbH)< <iBvShift)。

[0716] When regionId is equal to 2, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows:

[0717] iHorMax2=(xTbCmp-nTbW)< <iBvShift;

[0718] iHorMin2=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift);

[0719] iVerMin2=max((iTemplateSizeH)< <iBvShift,(yTbCmp-nTbH)<<iBvShift);

[0720] iVerMax2=(yTbCmp)< <iBvShift。

[0721] When regionId is equal to 3, iVerMin, iVerMax, iHorMin, and iHorMax can be calculated as follows

[0722] iHorMin3=max((iTemplateSizeW)< <iBvShift,(xTbCmp-searchRangeWidth)<<iBvShift);

[0723] iHorMax3=(xTbCmp)< <iBvShift;

[0724] iVerMin3=max(((iTemplateSizeH)< <iBvShift),(yTbCmp-offsetLCBY-nTbH+1)<<iBvShift);

[0725] iVerMax3=(yTbCmp-nTbH)< <iBvShift。

[0726] In practical applications, the iHorMin regionId 、iHorMax regionId 、iVerMin regionId 、iVerMax regionId Represent the left edge, right edge, top edge and bottom edge of different sub-search areas respectively.

[0727] To intuitively describe the different sub-search regions corresponding to different regionIds, R1, R2, R3, and R4 represent four different sub-search regions, as shown in Figure 14. It should be noted that Figure 14 is expressed in terms of the pixel range that can be aligned with the upper left corner pixel of the block.

[0728] In the fine search phase, a search can be performed near each block vector obtained by the coarse search. Furthermore, a search can be performed near the best block vector BV1_BESTk obtained by the coarse search. Specifically, a refinement search range TmpRefineRange is first determined. The refinement search range can be a fixed size or related to the size of the current block. For example, it can be set to min(nTbW, nTbH) / 2. Then, the position of the best matching reconstructed block obtained by the coarse search is calculated as the reference position of the fine search area: BestPosXk=xTbCmp+pX1_BESTk, BestPosYk=yTbCmp+pY1_BESTk.

[0729] In some embodiments, the fine search performed based on each BestPosXk can be performed across multiple regions, that is, the fine search can be performed across different search regions. The value of bestRegionId involved in the operation can be determined first: for regionId = 0, 1, 2, 3, the judgment is made: if BestPosYk-TmpRefineRange>=iVerMinregionId, and BestPosYk-TmpRefineRange<=iVerMaxregionId,

[0730] Or BestPosYk+TmpRefineRange>=iVerMinregionId, and BestPosYk+TmpRefineRange<=iVerMaxregionId,

[0731] Or BestPosYk-TmpRefineRange<=iVerMinregionId, and BestPosYk+TmpRefineRange>=iVerMaxregionId,

[0732] Then the regionId sets the corresponding bestSearchFlagregionId to 1.

[0733] For each search region with the value of bestSearchFlagregionId set to 1, set bestRegionId to the regionId of the region and perform the following refined search: first obtain the calculated values ​​of iVerMinbestRegionId, iVerMaxbestRegionId, iHorMinbestRegionId according to the value of bestRegionId, and then obtain the new search ranges iVerMinrefine, iVerMaxrefine, iHorMinrefine, and iHorMaxrefine based on the optimal matching block position obtained in the coarse search. The acquisition method is as follows:

[0734] iHorMinrefine=max(iHorMinbestRegionId, BestPosX-TmpRefineRange)

[0735] iHorMaxrefine=min(iHorMaxbestRegionId, BestPosX+TmpRefineRange)

[0736] iVerMinrefine=max(iVerMinbestRegionId, BestPosY-TmpRefineRange)

[0737] iVerMaxrefine=min(iVerMaxbestRegionId, BestPosY+TmpRefineRange)

[0738] Then, the adjusted block vectors BVbvXMins, bvXMaxs, bvYMins, bvYMaxs can be calculated by iVerMinrefine, iVerMaxrefine, iHorMinrefine, iHorMaxrefine:

[0739] bvXMins=iHorMinrefine-xTbCmp;

[0740] bvXMaxs=iHorMaxrefine-xTbCmp;

[0741] bvYMins=iVerMinrefine-yTbCmp;

[0742] bvYMaxs=iVerMaxrefine-yTbCmp;

[0743] Thus, the refined search is performed within the block vector range where pX is between bvXMinsrefine and bvXMaxsrefine, and pY is between bvYMinsrefine and bvYMaxsrefine. For example, the search is performed with a step size of 1. The best match cost obtained by template matching is recorded as pDiff_BEST, and the corresponding block vector BV is recorded as the best block vector BV_BEST(pX_BEST, pY_BEST).

[0744] Get the total best block vector of multiple search regions.

[0745] After completing the above search operation, the optimal block vector BV_BEST (pX_BEST, pY_BEST) can be obtained, where pX_BEST and pY_BEST are the horizontal offset and vertical offset of the best matching template relative to the current coding block template, respectively, and are also the horizontal offset and vertical offset of the best matching reconstructed block relative to the current coding block.

[0746] Step 203: Determine one or more reference blocks of the current block according to the one or more block vectors, and determine a prediction value of the current block according to the one or more reference blocks.

[0747] In an embodiment of the present application, after determining one or more block vectors corresponding to the current block according to the first template, one or more reference blocks of the current block can be further determined according to the one or more block vectors, and then the prediction value of the current block can be determined according to the one or more reference blocks.

[0748] It should be noted that, in an embodiment of the present application, one or more reference blocks of the current block include a first reference block and / or a second reference block; wherein the first reference block and the current block both belong to the current image, and the second reference block belongs to the reference image of the current image corresponding to the current block.

[0749] That is to say, in an embodiment of the present application, one or more reference blocks of the current block may be the first reference block in the current image obtained through intra-frame prediction, or the second reference block in the reference image of the current image obtained through inter-frame prediction; or may include both the first reference block in the current image obtained through intra-frame prediction and the second reference block in the reference image of the current image obtained through inter-frame prediction.

[0750] Accordingly, in an embodiment of the present application, when acquiring the second reference block, one or more block vectors can be determined; then a search can be performed in the reference image of the current image to determine the second reference block corresponding to the one or more block vectors.

[0751] That is to say, in an embodiment of the present application, it is possible to perform a template matching search within the current image by means of intra-frame template matching to determine one or more reference blocks corresponding to the current block, i.e., the first reference block; it is also possible to perform a template matching search on one or more inter-frame reference images of the current image by means of inter-frame template matching to determine one or more reference blocks corresponding to the current block, i.e., the second reference block; it is also possible to combine the intra-frame template matching method and the inter-frame template matching method to perform template matching searches within the current image and on one or more inter-frame reference images, respectively, to determine one or more reference blocks corresponding to the current block, including the first reference block and the second reference block.

[0752] Furthermore, in an embodiment of the present application, when determining one or more reference blocks corresponding to a current block based on one or more block vectors, one or more initial reconstructed blocks corresponding to the current block can be first determined based on the one or more block vectors; then, the one or more initial reconstructed blocks can be corrected to determine one or more reference blocks.

[0753] That is to say, in the embodiment of the present application, other methods may be used to obtain N candidate reconstructed blocks (reference blocks), for example, first correcting the initial reconstructed block corresponding to the obtained candidate template, and then determining the corresponding reference block.

[0754] Correspondingly, after obtaining N candidate reconstructed blocks (reference blocks) by copying N BV corresponding matching reconstructed blocks (initial reconstructed blocks), these N candidate reconstructed blocks can be directly weighted to obtain the predicted value of the current block; or the candidate reconstructed blocks (reference blocks) obtained after correcting the N BV corresponding matching reconstructed blocks (initial reconstructed blocks) can be weighted again to obtain the predicted value of the current block.

[0755] Furthermore, in an embodiment of the present application, when performing correction processing on one or more initial reconstructed blocks to determine one or more reference blocks, filtering processing may be performed on the one or more initial reconstructed blocks to determine one or more reference blocks.

[0756] Furthermore, in an embodiment of the present application, when performing correction processing on one or more initial reconstructed blocks to determine one or more reference blocks, one or more correction parameter vectors can be first determined based on one or more candidate templates corresponding to one or more block vectors; then, the one or more initial reconstructed blocks can be corrected based on the one or more correction parameter vectors to determine one or more reference blocks.

[0757] Furthermore, in an embodiment of the present application, when determining one or more correction parameter vectors based on one or more candidate templates corresponding to one or more block vectors, the autocorrelation matrix corresponding to the candidate template can be first determined based on the pixel values ​​in the candidate template; then, the cross-correlation vector can be determined based on the pixel values ​​in the first template and the pixel values ​​in the candidate template; and then, the correction parameter vector can be determined based on the autocorrelation matrix and the cross-correlation vector.

[0758] It can be understood that in an embodiment of the present application, the scheme for correcting the initial reconstructed block may include directly filtering the initial reconstructed block, wherein the filtering scheme adopted when performing the processing may be a traditional filtering method, such as bilateral filtering, mean filtering, etc., or it may be a filtering enhancement based on a neural network.

[0759] It is understandable that, in the embodiment of the present application, the solution of correcting the initial reconstructed block may further include correcting the matching reconstructed block (initial reconstructed block) using matching template information (candidate template).

[0760] For example, in the embodiment of the present application, when the matching template information is used to modify the matching reconstructed block, for each candidate template refT n And its corresponding candidate reconstruction block (initial reconstruction block) RefBlock n , using the candidate template refT n and the current coding block template (the first template of the current block) curT to calculate the correction parameter vector C n , the correction parameter vector Cn Used to reconstruct the candidate block RefBlock n Perform weighted fusion to obtain the final corrected reconstruction block RefBlock' n , that is, the reference block of the current block is finally obtained as RefBlock' n .

[0761] In some embodiments, the modified parameter vector C n Can be passed through the candidate template refT n The reconstruction value and the template pixel value to be predicted are deduced by minimizing the MSE.

[0762] It can be understood that, in the embodiment of the present application, the correction parameter vector C n It can be regarded as an L-tap filter.

[0763] In some embodiments, when calculating the modified parameter vector C n When, for each candidate template refT n , where n = 0, 1…, N-1, the MSE minimization process takes the autocorrelation matrix of the candidate template sample refT, the cross-correlation vector of the candidate template sample refT and the adjacent template sample curT of the current coding block as input, and outputs the weight of the candidate reconstructed block corresponding to the current candidate template.

[0764] For example, in the embodiments of the present application, the MSE is calculated as follows:

[0765]

[0766] In order to conveniently express the calculation formula of MSE, E is used here to represent the mean square error MSE, that is:

[0767]

[0768] Where K is the number of pixels in the template.

[0769] In some embodiments, the weight c of the candidate template refT corresponding to the candidate reconstructed block is derived by minimizing the MSE. l The process may include the following steps:

[0770] (1) First, l Find the partial derivative and make it equal to 0:

[0771]

[0772]

[0773] Arranged:

[0774]

[0775] (2) After determining the candidate template area refT n Then expand the equation obtained in step (1) into a matrix form:

[0776]

[0777] (3) The autocorrelation matrix and cross-correlation vector in step (2) are both known quantities. Solving the linear equations in (2) can obtain the filter weight coefficients c0, ...c L-1 , that is, the modified parameter vector C n The filter coefficients of a candidate reconstructed block in .

[0778] Correspondingly, for x=0…nTbW-1, y=0…nTbH-1, the modified candidate reconstructed block RefBlock' n for:

[0779]

[0780] It should be noted that, in the embodiments of the present application, after determining the N block vectors BV corresponding to the N candidate templates of the current block, N candidate reconstructed blocks (i.e., N reference blocks) can be obtained through the N BVs, and then the N candidate reconstructed blocks are weightedly fused to obtain the prediction block of the current block (i.e., the prediction value of the current block). When generating the final prediction value, the N candidate reconstructed blocks (N reference blocks) can be first obtained, and then the corresponding weighted fusion weights (weight values) can be determined, and then the prediction value of the current block can be generated through weighted fusion.

[0781] It is understood that, in the embodiment of the present application, when determining one or more reference blocks of the current block according to one or more block vectors, the N block vectors BV corresponding to the obtained N candidate templates are n , you can choose to directly according to BV n N candidate reconstruction blocks (i.e., reference blocks) RefBlock obtained in the current image and / or reference image n , among which, BV n The horizontal offset is pX n , the vertical offset is pY n , where n=0, 1…, N-1.

[0782] For example, in an embodiment of the present application, when determining one or more reference blocks for the current block, a simple translation copy can be used. The specific operation is: for x = 0 ... nTbW-1, y = 0 ... nTbH-1, the reconstructed pixels of the current frame (i.e., the reference blocks of the current block) are determined by the following formula:

[0783] RefBlock n [x][y]=recSamples[x+pX n ][y+pY n ] (15)

[0784] Furthermore, in an embodiment of the present application, when determining the prediction value of the current block based on one or more reference blocks, one or more weight values ​​corresponding to the one or more reference blocks can be determined first; then, one or more reference blocks can be weightedly fused according to the one or more weight values, and finally the prediction value of the current block can be determined.

[0785] It should be noted that in the embodiments of the present application, after obtaining N candidate reconstructed blocks RefBlock (i.e., reference blocks for the current block), it is necessary to calculate the weight W for weighted fusion of the N candidate reconstructed blocks. The weight values ​​corresponding to the reference blocks can be determined in a variety of ways. For example, the weights can be predefined values ​​(e.g., a second preset value) or adaptively calculated using cost values, pixel values, and the like.

[0786] In one embodiment, one or more weight values ​​may be determined according to a second preset value. The second preset value may include N arbitrary values ​​greater than 0, and the corresponding weight values ​​for different reference blocks in the N reference blocks may be the same or different, which is not specifically limited in this application.

[0787] In one embodiment, one or more weight values ​​may be determined based on one or more candidate templates corresponding to one or more block vectors. When determining one or more weight values ​​based on one or more candidate templates corresponding to one or more block vectors, an autocorrelation matrix corresponding to the candidate template may be first determined based on pixel values ​​in the candidate template; a cross-correlation vector may then be determined based on pixel values ​​in the first template and pixel values ​​in the candidate template; and finally, a weight value may be determined based on the autocorrelation matrix and the cross-correlation vector.

[0788] In some embodiments, the candidate template refT n The reconstructed value and the template to be predicted (first template) refpredT n The pixel value of is minimized to minimize the MSE to derive the weighted fusion weight (weight value).

[0789] It should be noted that in order to make the weight derivation process more flexible, nonlinear terms and offset terms can be added in the process of deriving the weighted fusion weights.

[0790] Exemplarily, in an embodiment of the present application, when deriving the weighted fusion weights, the non-linear term NonLinearTerm_T is constructed based on the candidate templates. Among them, one candidate template with the serial number 0 can be selected from N candidate templates for construction. Correspondingly, for m = 0, 1,..., M - 1, there is the following formula:

[0791] NonLinearTerm_T n，m =(refT n，m *refT n，m +MidVal)>>bitDepth (16)

[0792] Where n is 0, 1,..., or N - 1, representing any one of the N candidate templates; MidVal is 1<<(bitDepth - 1), and bitDepth is the image bit depth.

[0793] Exemplarily, in an embodiment of the present application, for each candidate reconstruction block (reference block) corresponding to the N candidate templates, when applying the weighted fusion weights, the non-linear term NonLinearTerm_Block is constructed based on the candidate reconstruction block. Among them, the candidate reconstruction block corresponding to the candidate template with the serial number 0 can be selected. Correspondingly, for x = 0, 1...nTbW - 1, y = 0...nTbH - 1, there is the following formula:

[0794] NonLinearTerm_Block n，x，y =(refBlock n，x，y *refBlock n，x，y +MidVal)>>bitDepth (17)

[0795] Where n is 0, 1,..., or N - 1, representing the candidate reconstruction block (reference block) corresponding to any one of the N candidate templates.

[0796] The offset value Bias in the process of deriving the weights and applying the weights can be any constant in the image pixel range [0, (1<<bitDepth) - 1]. For example, Bias can be set to 1<<(bitDepth - 1).

[0797] Since BiasTerm is a constant, in the actual calculation process, it needs to be extended to a matrix and represented as BiasTerm. Specifically:

[0798] For each of the N candidate templates:

[0799] For m = 0, 1,..., M - 1:

[0800] BiasTermm =Bias (18)

[0801] For each candidate reconstruction block corresponding to N candidate templates:

[0802] For x=0, 1…nTbW-1, y=0…nTbH-1:

[0803] BiasTerm x，y =Bias (19)

[0804] It is understood that in the embodiment of the present application, after adding the nonlinear term and the offset term, N+2 weighted weights (weight values) need to be derived. For the convenience of description, the variable P is used here to record the number of weighted weights, where P = N+2, and the matching template sample / nonlinear term sample and offset term sample are uniformly referred to as matching reference sample refT N and refT N+1 , so all reference quantities involved in the operation can be uniformly expressed as refT p Similarly, the reconstructed block corresponding to the matching template, the reconstructed block corresponding to the matching template involved in the nonlinear term, and the offset term are uniformly referred to as candidate reconstructed samples refBlock p , where p = 0, 1…, P-1.

[0805] Exemplarily, in an embodiment of the present application, the MSE minimization process takes as input the autocorrelation matrix of the first P matching reference samples refT, the cross-correlation vector of the first P matching reference samples refT and the adjacent template samples curT of the current coding block, and outputs the weight of the reconstructed block corresponding to each matching reference item.

[0806] The MSE calculation formula is as follows:

[0807] For each pixel at the same position that matches the reference template, that is, for m=0, 1…, M-1:

[0808]

[0809] In order to conveniently express the calculation formula of MSE, E is used here to represent the mean square error MSE, that is:

[0810]

[0811] Minimize MSE to derive the weight w of each matching reference sample corresponding to the reconstructed block p The specific steps are as follows:

[0812] (1) First, w p Find the partial derivative and make it equal to 0:

[0813]

[0814]

[0815] Arranged:

[0816]

[0817] (2) After determining the matching reference sample area refT0, refT1, ...refT P-1 Then expand the equation obtained in step (1) into a matrix form:

[0818]

[0819] (3) The autocorrelation matrix and cross-correlation vector in step (2) are both known quantities. By solving the linear equations in (2), the weights w0,…w of each matching reference item corresponding to the reconstructed sample (reference block) can be calculated. P-1 .

[0820] In one embodiment, when determining one or more weight values ​​based on one or more candidate templates corresponding to one or more block vectors, a matching cost value between the first template and the candidate template may be determined first; and then the weight value may be determined based on the matching cost value.

[0821] Furthermore, in the embodiments of the present application, other methods may be used to calculate the weighted weights. For example, a nonlinear weight model may be used to calculate the weighted weights for each candidate reconstruction block (reference block) RefBlock according to the cost of N matching candidate templates. n Assign the corresponding weight w n .

[0822] It should be noted that, in the embodiments of the present application, the weight model may include but is not limited to a nonlinear normalization function, a nonlinear exponential normalization function, and the like.

[0823] For example, in an embodiment of the present application, the following nonlinear function can be selected to calculate the weight of each candidate reconstructed block (reference block), wherein the input of the weight model is the current coding block template curT and the candidate template refT n The matching cost between the current coding block template curT and the candidate template refT n SAD between (refT n )、MAD(refT n ), correlation coefficient R(refT n )wait,

[0824] In some embodiments, the calculation formula corresponding to the weight model is as follows:

[0825]

[0826] Here, offset is a preset value, for example, offset is 1.

[0827] In some embodiments, when the matching cost is the normalized correlation coefficient R(refT n ), the calculation formula corresponding to the weight model is as follows:

[0828]

[0829] In some embodiments, a Softmax function may also be used as a weight model, and the calculation formula is as follows:

[0830]

[0831] Here, S is a model control parameter. Under certain conditions, the parameter S can be adjusted to adjust the weight model. For example, the parameter S can be related to the current block size or the template type.

[0832] In some embodiments, in addition to the above nonlinear weight model, the weights of the candidate reconstructed blocks may be directly set to an average value, for example:

[0833]

[0834] In some embodiments, when using the candidate template refT n The reconstructed value and the template to be predicted (first template) refpredT n In the process of deriving weighted fusion weights (weight values) by minimizing the MSE of the pixel values, it is also possible to choose not to add nonlinear terms and offset terms.

[0835] For example, in the embodiments of the application, if the nonlinear term and the offset term are not added, then only N weighted weights (weight values) need to be derived. The MSE minimization process takes as input the autocorrelation matrix of the first N candidate template samples refT and the cross-correlation vector of the first P candidate template samples refT and the adjacent template samples curT of the current coding block, and outputs the weight of each candidate template corresponding to the candidate reconstructed block.

[0836] In some embodiments, for pixels at the same position of each candidate template, that is, for m=0, 1..., M-1, the MSE calculation formula is as follows:

[0837]

[0838] In order to conveniently express the calculation formula of MSE, E is used here to represent the mean square error MSE, that is:

[0839]

[0840] Minimize MSE to derive the weight w of each candidate template corresponding to the candidate reconstruction block n The specific steps are as follows:

[0841] (1) First, w n Find the partial derivative and make it equal to 0:

[0842]

[0843]

[0844] Arranged:

[0845]

[0846] (2) After determining the candidate template area refT0, refT1, ...refT N-1 Then expand the equation obtained in (1) into a matrix form:

[0847]

[0848] (3) The autocorrelation matrix and cross-correlation vector in step (2) are both known quantities. By solving the linear equations in (2), the weights w0,…w of each candidate template corresponding to the reconstructed sample can be calculated. N-1 .

[0849] In some embodiments, when using the candidate template refT n The reconstructed value and the template to be predicted (first template) refpredT n In the process of deriving the weighted fusion weight (weight value) by minimizing the MSE of the pixel value, it is also possible to choose to add only the offset term without adding the nonlinear term.

[0850] For example, in the embodiment of the application, if only the offset term is added without the nonlinear term, then N+1 weighted weights (weight values) need to be derived. For the convenience of description, the variable P is used here to record the number of weighted weights, where P = N+1, and the candidate template sample / offset item sample is uniformly referred to as the matching reference sample refT N and refT N+1 , so all reference quantities involved in the operation can be uniformly expressed as refT p Similarly, the candidate reconstruction blocks and offset items corresponding to the candidate templates are uniformly referred to as candidate reconstruction samples refBlock p , where p = 0, 1…, P-1.

[0851] In some embodiments, the MSE minimization process takes as input the autocorrelation matrix of the first P matching reference samples refT, the cross-correlation vector of the first P matching reference samples refT and the adjacent template samples curT of the current coding block, and outputs the weight of the reconstructed block corresponding to each matching reference item.

[0852] In some embodiments, for pixels at the same position of each candidate template, that is, for m=0, 1..., M-1, the MSE calculation formula is as follows:

[0853]

[0854] In order to conveniently express the calculation formula of MSE, E is used here to represent the mean square error MSE, that is:

[0855]

[0856] Minimize MSE to derive the weight w of each matching reference sample corresponding to the reconstructed block p The specific steps are as follows:

[0857] (1) First, w p Find the partial derivative and make it equal to 0:

[0858]

[0859]

[0860] Arranged:

[0861]

[0862] (2) After determining the matching reference sample area refT0, refT1, ...refT P-1 Then expand the equation obtained in step (1) into a matrix form:

[0863]

[0864] (3) The autocorrelation matrix and cross-correlation vector in step (2) are both known quantities. By solving the linear equations in (2), the weights w0,…w of each matching reference item corresponding to the reconstructed sample can be calculated. P-1 .

[0865] In an embodiment of the present application, after determining one or more reference blocks for a current block based on one or more block vectors and determining a predicted value for the current block based on the one or more reference blocks, a prediction residual for the current block can be further determined based on the predicted value for the current block and written into the bitstream. This allows a decoder to determine the prediction residual corresponding to the current block by decoding the bitstream; and then, based on the prediction residual and the predicted value, a reconstructed value for the current block can be determined.

[0866] In summary, the encoding method proposed in steps 201 to 204 above is improved and optimized based on the common intra TMP technology, employing weighted fusion to propose an intra TMP fusion prediction method. Specifically, in the process of searching for and determining the block vector BV in the search area, at least one block vector for the current block can be selectively determined, i.e., multiple block vectors can be determined. Simultaneously, in generating the predicted value, a weighted fusion process is performed using at least one reference block corresponding to the at least one block vector to obtain the predicted value for the current block, thereby determining the reconstructed value for the current block.

[0867] That is to say, the embodiment of the present application proposes the Intra TMP Fusion technology. After the current coding block template (the first template of the current block) searches for N block vectors BV corresponding to N candidate matching templates with the smallest matching cost within a predefined range, the N candidate reconstructed blocks (N reference blocks) corresponding to these N candidate matching templates (N candidate templates) are found through the N BVs, and then weighted fusion is performed according to a certain weight, and used as the prediction block (prediction value) of the current block.

[0868] It can be understood that the Intra TMP Fusion method proposed in the embodiment of the present application can improve the accuracy of the prediction value. Among them, after the current coding block template (the first template of the current block) searches for the N block vectors BV corresponding to the N candidate matching templates with the smallest matching cost within a predefined range, the N candidate reconstruction blocks (N reference blocks) corresponding to these N candidate matching templates (N candidate templates) are found through the N BVs, and then weighted fusion is performed according to certain weights as the prediction block of the current block. On the one hand, the reconstructed block information corresponding to different matching templates in the search process is fully considered, rather than simply considering the reconstructed block corresponding to the template with the smallest matching cost; on the other hand, weights are adaptively assigned to the candidate reconstructed blocks through the matching template information, which fully considers the different importance of different reconstructed block information to the prediction of the current block.

[0869] It is understood that the Intra TMP Fusion method proposed in the embodiments of this application can fully utilize the candidate reconstructed block information corresponding to different matching templates during the matching template search process. On the one hand, it fully utilizes the reconstructed block information corresponding to different matching templates during the search process, rather than simply considering the reconstructed block information corresponding to the template with the lowest matching cost. On the other hand, it fully utilizes the matching template information to adaptively assign weights to the candidate reconstructed blocks, fully considering the different importance of different reconstructed block information for predicting the current block.

[0870] It can be seen that the Intra TMP Fusion method proposed in the embodiment of the present application can, to a certain extent, avoid the decrease in prediction accuracy caused by inaccurate template matching basis or directly copying the reconstructed block as the prediction block.

[0871] Compared with common coding and decoding technologies, the Intra TMP Fusion method proposed in the embodiment of the present application is tested under All Intra conditions at a 24-frame interval, and can achieve BD-rate changes of -0.29%, -0.30%, and -0.39% on Y, Cb, and Cr, respectively (i.e., the average bit rate changes under the same PSNR).

[0872] An embodiment of the present application provides an encoding method, in which the encoder determines a first template corresponding to the current block; determines one or more block vectors corresponding to the current block based on the first template; determines one or more reference blocks of the current block based on the one or more block vectors, and determines a prediction value of the current block based on the one or more reference blocks. It can be seen that in an embodiment of the present application, an Intra TMP Fusion prediction method is proposed, in which at least one block vector of the current block can be selected and determined, and then the prediction value of the current block can be obtained by using at least one reference block corresponding to at least one block vector. In other words, the encoding and decoding method proposed in the embodiment of the present application fully takes into account the different importance of the reconstructed block information corresponding to different matching templates in the search process to the prediction of the current block, and can improve the accuracy of the prediction, thereby obtaining the best prediction effect.

[0873] In another embodiment of the present application, see Figure 16, which shows a schematic diagram of the structure of an encoder provided by an embodiment of the present application. As shown in Figure 16, the encoder 180 may include: a first determining unit 1801; wherein,

[0874] The first determination unit 1801 is configured to determine a first template corresponding to a current block; determine one or more block vectors corresponding to the current block based on the first template; determine one or more reference blocks of the current block based on the one or more block vectors, and determine a prediction value of the current block based on the one or more reference blocks.

[0875] It is understandable that in the embodiments of the present application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the various components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional modules.

[0876] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0877] Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the encoder 180. The computer-readable storage medium stores a computer program, and when the computer program is executed by the first processor, it implements the method described in any one of the aforementioned embodiments.

[0878] Based on the composition of the encoder 180 and the computer-readable storage medium, refer to Figure 17, which shows a specific hardware structure diagram of the encoder 180 provided in an embodiment of the present application. As shown in Figure 17, the encoder 180 may include: a first communication interface 1901, a first memory 1902 and a first processor 1903; each component is coupled together through a first bus system 1904. It can be understood that the first bus system 1904 is used to achieve connection and communication between these components. In addition to the data bus, the first bus system 1904 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the first bus system 1904 in Figure 17. Among them,

[0879] The first communication interface 1901 is used to receive and send signals when sending and receiving information with other external network elements;

[0880] A first memory 1902 is used to store computer programs that can be run on the first processor 1903;

[0881] The first processor 1903 is configured to, when running the computer program, execute:

[0882] Determine a first template corresponding to the current block;

[0883] determining one or more block vectors corresponding to the current block according to the first template;

[0884] One or more reference blocks of the current block are determined according to the one or more block vectors, and a prediction value of the current block is determined according to the one or more reference blocks.

[0885] It is understood that the first memory 1902 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The first memory 1902 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0886] The first processor 1903 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the first processor 1903. The above-mentioned first processor 1903 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the first memory 1902 , and the first processor 1903 reads the information in the first memory 1902 and completes the steps of the above method in combination with its hardware.

[0887] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or its combination.For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field-Programmable Gate Array, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing functions described in the present application or its combination.For software implementation, the technology described in the present application can be realized by the module (such as process, function etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0888] Optionally, as another embodiment, the first processor 1903 is further configured to execute any one of the methods described in the foregoing embodiments when running the computer program.

[0889] In another embodiment of the present application, see FIG18 , which shows a schematic diagram of the structure of a decoder provided by an embodiment of the present application. As shown in FIG18 , the decoder 200 may include: a second determining unit 2001; wherein,

[0890] The second determination unit 2001 is configured to determine a first template corresponding to a current block; determine one or more block vectors corresponding to the current block based on the first template; determine one or more reference blocks of the current block based on the one or more block vectors, and determine a prediction value of the current block based on the one or more reference blocks; and determine a reconstructed value of the current block based on the prediction value of the current block.

[0891] It is understood that in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular system. Furthermore, the various components in this embodiment can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The aforementioned integrated units can be implemented in the form of hardware or software functional modules.

[0892] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, this embodiment provides a computer-readable storage medium, which is applied to the decoder 200 and stores a computer program. When the computer program is executed by the second processor, it implements any of the methods in the aforementioned embodiments.

[0893] Based on the composition of the decoder 200 and the computer-readable storage medium, refer to Figure 19, which shows a specific hardware structure diagram of the decoder 200 provided in an embodiment of the present application. As shown in Figure 19, the decoder 200 may include: a second communication interface 2201, a second memory 2202 and a second processor 2203; each component is coupled together through a second bus system 2204. It can be understood that the second bus system 2204 is used to achieve connection and communication between these components. In addition to the data bus, the second bus system 2204 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the second bus system 2204 in Figure 19. Among them,

[0894] The second communication interface 2201 is used to receive and send signals during the process of sending and receiving information between other external network elements;

[0895] The second memory 2202 is used to store computer programs that can be run on the second processor 2203;

[0896] The second processor 2203 is configured to, when running the computer program, execute:

[0897] Determine a first template corresponding to the current block;

[0898] determining one or more block vectors corresponding to the current block according to the first template;

[0899] determining one or more reference blocks for a current block according to the one or more block vectors, and determining a prediction value for the current block according to the one or more reference blocks;

[0900] A reconstructed value of the current block is determined according to the predicted value of the current block.

[0901] Optionally, as another embodiment, the second processor 2203 is further configured to execute any one of the methods described in the foregoing embodiments when running the computer program.

[0902] It can be understood that the hardware functions of the second memory 2202 are similar to those of the first memory 1902, and the hardware functions of the second processor 2203 are similar to those of the first processor 1903; they will not be described in detail here.

[0903] The present embodiment provides a codec, wherein the encoder determines a first template corresponding to the current block; determines one or more block vectors corresponding to the current block based on the first template; determines one or more reference blocks of the current block based on the one or more block vectors, and determines a predicted value of the current block based on the one or more reference blocks; and determines a reconstructed value of the current block based on the predicted value of the current block. The decoder determines a first template corresponding to the current block; determines one or more block vectors corresponding to the current block based on the first template; determines one or more reference blocks of the current block based on the one or more block vectors, and determines a predicted value of the current block based on the one or more reference blocks. It can be seen that in the embodiment of the present application, an Intra TMP Fusion prediction method is proposed, wherein at least one block vector of the current block can be selected and determined, and then at least one reference block corresponding to at least one block vector can be used to obtain a predicted value of the current block. In other words, the coding and decoding method proposed in the embodiment of the present application fully takes into account the different importance of the reconstructed block information corresponding to different matching templates in the search process for predicting the current block, and can improve the accuracy of the prediction, thereby obtaining the best prediction effect.

[0904] In yet another embodiment of the present application, referring to FIG20 , a schematic diagram of the structure of a coding and decoding system provided by an embodiment of the present application is shown. As shown in FIG20 , the coding and decoding system 230 may include an encoder 2301 and a decoder 2302 .

[0905] In the embodiment of the present application, the encoder 2301 may be the encoder described in any one of the aforementioned embodiments, and the decoder 2302 may be the decoder described in any one of the aforementioned embodiments.

[0906] Furthermore, an embodiment of the present application also provides a code stream, wherein the code stream is generated by bit encoding based on the information to be encoded; wherein the information to be encoded includes at least one of the following: a predicted difference value of the current block, a preset number N, and one or more block vectors.

[0907] It should be noted that, in this application, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0908] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0909] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0910] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0911] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0912] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims. Industrial Applicability

[0913] The embodiment of the present application provides a coding and decoding method, an encoder, a decoder and a storage medium, wherein the encoder determines a first template corresponding to the current block; determines one or more block vectors corresponding to the current block based on the first template; determines one or more reference blocks of the current block based on the one or more block vectors, and determines the prediction value of the current block based on the one or more reference blocks; and determines the reconstruction value of the current block based on the prediction value of the current block. The decoder determines the first template corresponding to the current block; determines one or more block vectors corresponding to the current block based on the first template; determines one or more reference blocks of the current block based on the one or more block vectors, and determines the prediction value of the current block based on the one or more reference blocks. It can be seen that in the embodiment of the present application, an Intra TMP Fusion prediction method is proposed, wherein at least one block vector of the current block can be selected and determined, and then at least one reference block corresponding to at least one block vector can be used to obtain the prediction value of the current block. In other words, the coding and decoding method proposed in the embodiment of the present application fully takes into account the different importance of the reconstructed block information corresponding to different matching templates in the search process to the prediction of the current block, and can improve the accuracy of the prediction, thereby obtaining the best prediction effect.

Claims

1. A decoding method, applied to a decoder, the method comprising: Determine a first template corresponding to the current block; Determine one or more block vectors corresponding to the current block according to the first template; Determine one or more reference blocks of a current block according to the one or more block vectors, and determine a prediction value of the current block according to the one or more reference blocks; A reconstructed value of the current block is determined according to the predicted value of the current block.

2. The method according to claim 1, wherein: The determining one or more block vectors corresponding to the current block according to the first template includes: Determine a preset search area according to the first template; A search is performed in the preset search area to determine the one or more block vectors.

3. The method according to claim 2, wherein: The searching within the preset search area to determine the one or more block vectors includes: Traversing the search points in the preset search area, and determining a matching cost between a matching template corresponding to the search point in the preset search area and the first template according to a preset matching criterion; The one or more block vectors and one or more candidate templates corresponding to the one or more block vectors are determined according to the matching cost value.

4. The method according to claim 3, wherein: The method further comprises: Determine a preset number N corresponding to the candidate template; wherein N is an integer greater than 0; Traversing the search points in the preset search area, and determining the matching cost between the matching template corresponding to the search point in the preset search area and the first template according to the preset matching criterion; According to the matching cost value, N block vectors and N candidate templates corresponding to the N block vectors are determined.

5. The method according to claim 4, wherein: The determining the preset number N corresponding to the candidate templates includes: Decode the code stream and determine the N.

6. The method according to claim 4, wherein: The determining the preset number N corresponding to the candidate templates includes: The N is determined according to a first preset value or a preset value range.

7. The method according to claim 4, wherein: The determining, according to the matching cost value, N block vectors and N candidate templates corresponding to the N block vectors includes: Determine N minimum matching cost values ​​among the matching cost values ​​between the matching template corresponding to the search point in the preset search area and the first template; The N block vectors and the N candidate templates corresponding to the N minimum matching cost values ​​are determined.

8. The method according to claim 4, wherein: The preset matching criteria include any one of the sum of absolute error SAD, the sum of transformed absolute error SATD, the sum of squared differences SSE, the mean absolute difference MAD, the mean absolute error MAE, the mean squared error MSE, and the normalized correlation coefficient NCC.

9. The method according to claim 3 or 4, wherein: The method further comprises: Traversing the search points in the preset search area according to a first search step length to determine the block vector and the candidate template; or, The search points in the preset search area are traversed according to a second search step length to determine the block vector and the candidate template; wherein the first search step length is greater than the second search step length.

10. The method according to claim 3 or 4, wherein: The method further comprises: Traversing the search points in the preset search area according to a first search step length to determine an initial block vector and an initial matching template corresponding to the initial vector block; Determine a first search area according to the initial matching template; wherein the first search area is smaller than the preset search area; The search points in the first search area are traversed according to a second search step length to determine the block vector and the candidate template; wherein the first search step length is greater than the second search step length.

11. The method according to claim 4 or 5, wherein: The method further comprises: Traversing the search points in the sub-search area of ​​the preset search area according to the first search step length, and determining the initial vector corresponding to the sub-search area and the second search area corresponding to the initial vector block; Determine a plurality of target sub-search areas in the sub-search area according to the initial vector block and the second search area; The search points in the plurality of target sub-search areas are traversed according to a second search step length to determine the block vector and the candidate template; wherein the first search step length is greater than the second search step length.

12. The method according to claim 1, wherein: The one or more reference blocks include a first reference block and / or a second reference block; wherein the first reference block and the current block both belong to a current image, and the second reference block belongs to a reference image of the current image corresponding to the current block.

13. The method according to claim 12, wherein: The method further comprises: Decoding the code stream to determine the one or more block vectors; Search is performed in the reference image of the current image to determine the second reference block corresponding to the one or more block vectors.

14. The method according to claim 1, wherein: The determining one or more reference blocks corresponding to the current block according to the one or more block vectors includes: Determine one or more initial reconstructed blocks corresponding to the current block according to the one or more block vectors; The one or more initial reconstructed blocks are corrected to determine the one or more reference blocks.

15. The method according to claim 14, wherein: The step of performing correction processing on the one or more initial reconstructed blocks to determine the one or more reference blocks includes: The one or more initial reconstructed blocks are filtered to determine the one or more reference blocks.

16. The method according to claim 14, wherein: The step of performing correction processing on the one or more initial reconstructed blocks to determine the one or more reference blocks includes: Determine one or more correction parameter vectors according to one or more candidate templates corresponding to the one or more block vectors; The one or more initial reconstructed blocks are corrected according to the one or more correction parameter vectors to determine the one or more reference blocks.

17. The method according to claim 16, wherein: The determining of the one or more modified parameter vectors according to the one or more candidate templates corresponding to the one or more block vectors includes: Determine the autocorrelation matrix corresponding to the candidate template according to the pixel values ​​in the candidate template; Determining a cross-correlation vector according to pixel values ​​in the first template and pixel values ​​in the candidate template; The correction parameter vector is determined according to the autocorrelation matrix and the cross-correlation vector.

18. The method according to claim 1, wherein: The determining, according to the one or more reference blocks, a prediction value of the current block comprises: Determining one or more weight values ​​corresponding to the one or more reference blocks; The one or more reference blocks are subjected to weighted fusion processing according to the one or more weight values ​​to determine a prediction value of the current block.

19. The method according to claim 18, wherein: The determining one or more weight values ​​corresponding to the one or more reference blocks includes: The one or more weight values ​​are determined according to a second preset value.

20. The method according to claim 18, wherein: The determining one or more weight values ​​corresponding to the one or more reference blocks includes: The one or more weight values ​​are determined according to one or more candidate templates corresponding to the one or more block vectors.

21. The method according to claim 20, wherein: The determining the one or more weight values ​​according to the one or more candidate templates corresponding to the one or more block vectors includes: Determine the autocorrelation matrix corresponding to the candidate template according to the pixel values ​​in the candidate template; Determining a cross-correlation vector according to pixel values ​​in the first template and pixel values ​​in the candidate template; The weight value is determined according to the autocorrelation matrix and the cross-correlation vector.

22. The method according to claim 20, wherein: The determining the one or more weight values ​​according to the one or more candidate templates corresponding to the one or more block vectors includes: Determining a matching cost between the first template and the candidate template; The weight value is determined according to the matching cost value.

23. The method according to claim 1, wherein: The determining the first template corresponding to the current block includes: A template type corresponding to the current block is determined, and a first template corresponding to the current block is determined according to the template type.

24. The method according to claim 23, wherein: The determining of the template type corresponding to the current block includes: Determining a template type of the current block according to a reference pixel of the current block; Among them, the reference pixels of the current block include at least one of the following: the left adjacent reference pixels of the current block, the upper adjacent reference pixels of the current block, the upper left adjacent reference pixels of the current block, the lower left adjacent reference pixels of the current block, and the upper right adjacent reference pixels of the current block.

25. The method according to claim 23, wherein: The determining, according to the reference pixel of the current block, the template type of the current block comprises: If the left adjacent reference pixel of the current block, the upper adjacent reference pixel of the current block, and the upper left adjacent reference pixel of the current block are all available, determining that the template type of the current block is a first value; If the left adjacent reference pixel of the current block is available, determining the template type of the current block to be a second value; If the upper adjacent reference pixel of the current block is available, determining the template type of the current block to be a third value; If both the left adjacent reference pixel of the current block and the upper left adjacent reference pixel of the current block are available, determining that the template type of the current block is a fourth value; If both the left adjacent reference pixel of the current block and the lower left adjacent reference pixel of the current block are available, determining that the template type of the current block is a fifth value; If both the upper neighboring reference pixels of the current block and the upper right neighboring reference pixels of the current block are available, the template type of the current block is determined to be a sixth value.

26. The method of claim 1, wherein: The determining, according to the template type, a first template corresponding to the current block includes: Determining a template reference pixel of the current block according to the template type and a template size corresponding to the template type; A first template of the current block is determined according to the template reference pixels.

27. The method of claim 1, wherein: The step of determining the reconstruction value of the current block according to the prediction value of the current block includes: Decoding the bitstream to determine a prediction difference corresponding to the current block; A reconstructed value of the current block is determined according to the prediction difference and the prediction value.

28. A coding method, applied to an encoder, the method comprising: Determine a first template corresponding to the current block; Determine one or more block vectors corresponding to the current block according to the first template; One or more reference blocks of the current block are determined according to the one or more block vectors, and a prediction value of the current block is determined according to the one or more reference blocks.

29. The method according to claim 28, wherein: The determining one or more block vectors corresponding to the current block according to the first template includes: Determine a preset search area according to the first template; A search is performed in the preset search area to determine the one or more block vectors.

30. The method of claim 28, wherein: The searching within the preset search area to determine the one or more block vectors includes: Traversing the search points in the preset search area, and determining a matching cost between a matching template corresponding to the search point in the preset search area and the first template according to a preset matching criterion; The one or more block vectors and one or more candidate templates corresponding to the one or more block vectors are determined according to the matching cost value.

31. The method according to claim 30, wherein: The method further comprises: Determine a preset number N corresponding to the candidate template; wherein N is an integer greater than 0; Traversing the search points in the preset search area, and determining the matching cost between the matching template corresponding to the search point in the preset search area and the first template according to the preset matching criterion; According to the matching cost value, N block vectors and N candidate templates corresponding to the N block vectors are determined.

32. The method according to claim 31, wherein: The determining the preset number N corresponding to the candidate templates includes: The N is determined according to a preset decision criterion.

33. The method of claim 4, wherein: The determining the preset number N corresponding to the candidate templates includes: The N is determined according to a first preset value or a preset value range.

34. The method of claim 4, wherein: The determining, according to the matching cost value, N block vectors and N candidate templates corresponding to the N block vectors includes: Determine N minimum matching cost values ​​among the matching cost values ​​between the matching template corresponding to the search point in the preset search area and the first template; The N block vectors and the N candidate templates corresponding to the N minimum matching cost values ​​are determined.

35. The method of claim 4, wherein: The preset matching criteria include any one of the sum of absolute error SAD, the sum of transformed absolute error SATD, the sum of squared differences SSE, the mean absolute difference MAD, the mean absolute error MAE, the mean squared error MSE, and the normalized correlation coefficient NCC.

36. The method according to claim 3 or 4, wherein: The method further comprises: Traversing the search points in the preset search area according to a first search step length to determine the block vector and the candidate template; or, The search points in the preset search area are traversed according to a second search step length to determine the block vector and the candidate template; wherein the first search step length is greater than the second search step length.

37. The method according to claim 3 or 4, wherein: The method further comprises: Traversing the search points in the preset search area according to a first search step length to determine an initial block vector and an initial matching template corresponding to the initial vector block; Determine a first search area according to the initial matching template; wherein the first search area is smaller than the preset search area; The search points in the first search area are traversed according to a second search step length to determine the block vector and the candidate template; wherein the first search step length is greater than the second search step length.

38. The method according to claim 4 or 5, wherein: The method further comprises: Traversing the search points in the sub-search area of ​​the preset search area according to the first search step length, and determining the initial vector corresponding to the sub-search area and the second search area corresponding to the initial vector block; Determine a plurality of target sub-search areas in the sub-search area according to the initial vector block and the second search area; The search points in the plurality of target sub-search areas are traversed according to a second search step length to determine the block vector and the candidate template; wherein the first search step length is greater than the second search step length.

39. The method of claim 1, wherein: The one or more reference blocks include a first reference block and / or a second reference block; wherein the first reference block and the current block both belong to a current image, and the second reference block belongs to a reference image of the current image corresponding to the current block.

40. The method of claim 12, wherein: The method further comprises: determining the one or more block vectors; Search is performed in the reference image of the current image to determine the second reference block corresponding to the one or more block vectors.

41. The method of claim 1, wherein: The determining one or more reference blocks corresponding to the current block according to the one or more block vectors includes: Determine one or more initial reconstructed blocks corresponding to the current block according to the one or more block vectors; The one or more initial reconstructed blocks are corrected to determine the one or more reference blocks.

42. The method of claim 14, wherein: The step of performing correction processing on the one or more initial reconstructed blocks to determine the one or more reference blocks includes: The one or more initial reconstructed blocks are filtered to determine the one or more reference blocks.

43. The method of claim 14, wherein: The step of performing correction processing on the one or more initial reconstructed blocks to determine the one or more reference blocks includes: Determine one or more correction parameter vectors according to one or more candidate templates corresponding to the one or more block vectors; The one or more initial reconstructed blocks are corrected according to the one or more correction parameter vectors to determine the one or more reference blocks.

44. The method of claim 16, wherein: The determining of the one or more modified parameter vectors according to the one or more candidate templates corresponding to the one or more block vectors includes: Determine the autocorrelation matrix corresponding to the candidate template according to the pixel values ​​in the candidate template; Determining a cross-correlation vector according to pixel values ​​in the first template and pixel values ​​in the candidate template; The correction parameter vector is determined according to the autocorrelation matrix and the cross-correlation vector.

45. The method of claim 1, wherein: The determining, according to the one or more reference blocks, a prediction value of the current block comprises: Determining one or more weight values ​​corresponding to the one or more reference blocks; The one or more reference blocks are subjected to weighted fusion processing according to the one or more weight values ​​to determine a prediction value of the current block.

46. ​​The method of claim 18, wherein: The determining one or more weight values ​​corresponding to the one or more reference blocks includes: The one or more weight values ​​are determined according to a second preset value.

47. The method of claim 18, wherein: The determining one or more weight values ​​corresponding to the one or more reference blocks includes: The one or more weight values ​​are determined according to one or more candidate templates corresponding to the one or more block vectors.

48. The method of claim 20, wherein: The determining the one or more weight values ​​according to the one or more candidate templates corresponding to the one or more block vectors includes: Determine the autocorrelation matrix corresponding to the candidate template according to the pixel values ​​in the candidate template; Determining a cross-correlation vector according to pixel values ​​in the first template and pixel values ​​in the candidate template; The weight value is determined according to the autocorrelation matrix and the cross-correlation vector.

49. The method of claim 20, wherein: The determining the one or more weight values ​​according to the one or more candidate templates corresponding to the one or more block vectors includes: Determining a matching cost between the first template and the candidate template; The weight value is determined according to the matching cost value.

50. The method according to claim 1, wherein: The determining the first template corresponding to the current block includes: A template type corresponding to the current block is determined, and a first template corresponding to the current block is determined according to the template type.

51. The method of claim 23, wherein: The determining of the template type corresponding to the current block includes: Determining a template type of the current block according to a reference pixel of the current block; Among them, the reference pixels of the current block include at least one of the following: the left adjacent reference pixels of the current block, the upper adjacent reference pixels of the current block, the upper left adjacent reference pixels of the current block, the lower left adjacent reference pixels of the current block, and the upper right adjacent reference pixels of the current block.

52. The method of claim 23, wherein: The determining, according to the reference pixel of the current block, the template type of the current block comprises: If the left adjacent reference pixel of the current block, the upper adjacent reference pixel of the current block, and the upper left adjacent reference pixel of the current block are all available, determining that the template type of the current block is a first value; If the left adjacent reference pixel of the current block is available, determining the template type of the current block to be a second value; If the upper adjacent reference pixel of the current block is available, determining the template type of the current block to be a third value; If both the left adjacent reference pixel of the current block and the upper left adjacent reference pixel of the current block are available, determining that the template type of the current block is a fourth value; If both the left adjacent reference pixel of the current block and the lower left adjacent reference pixel of the current block are available, determining that the template type of the current block is a fifth value; If both the upper neighboring reference pixels of the current block and the upper right neighboring reference pixels of the current block are available, the template type of the current block is determined to be a sixth value.

53. The method of claim 1, wherein: The determining, according to the template type, a first template corresponding to the current block includes: Determining a template reference pixel of the current block according to the template type and a template size corresponding to the template type; A first template of the current block is determined according to the template reference pixels.

54. The method of claim 1, wherein: The method further comprises: According to the prediction value of the current block, a prediction difference value of the current block is determined, and the prediction difference value is written into a bitstream.

55. A code stream, wherein The code stream is generated by bit encoding according to the information to be encoded; wherein the information to be encoded includes at least one of the following: The prediction difference of the current block, a preset number N, one or more block vectors.

56. An encoder, comprising a first determining unit; wherein, The first determination unit is configured to determine a first template corresponding to the current block; determine one or more block vectors corresponding to the current block according to the first template; determine one or more reference blocks of the current block according to the one or more block vectors, and determine a prediction value of the current block according to the one or more reference blocks.

57. An encoder comprising a first memory and a first processor; wherein, The first memory is used to store a computer program that can be run on the first processor; The first processor is configured to execute the method according to any one of claims 28 to 54 when running the computer program.

58. A decoder comprising a second determination unit; wherein: The second determination unit is configured to determine a first template corresponding to the current block; determine one or more block vectors corresponding to the current block according to the first template; determine one or more reference blocks of the current block according to the one or more block vectors, and determine a prediction value of the current block according to the one or more reference blocks; and determine a reconstruction value of the current block according to the prediction value of the current block.

59. A decoder, comprising a second memory and a second processor; wherein: The second memory is used to store a computer program that can be run on the second processor; The second processor is configured to execute the method according to any one of claims 1 to 27 when running the computer program.

60. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 27 is implemented, or the method according to any one of claims 28 to 54 is implemented.

Images