Intra-frame template matching prediction method, video encoding and decoding method, device and system

Abstract

The invention relates to an intra-frame template matching prediction method, a video encoding and decoding method, a video encoding and decoding device and a video encoding and decoding system, which are characterized in that when intra-frame prediction is carried out on a current block, a reference block template is searched in a reconstruction area of a current image, the difference of the searched reference block template relative to a current block template is calculated, a candidate list of an intraTMP is constructed according to the difference, and the intraTMP is decoded according to the candidate list. And determining N reference blocks in the candidate list and the sequence of the N reference blocks, wherein N is greater than or equal to 2.

Description

Intra-frame template matching prediction method, video encoding and decoding method, device and system Technical Field

[0001] The embodiments of the present disclosure relate to, but are not limited to, video technology, and more specifically, to an intra-frame template matching prediction method, a video encoding and decoding method, device, and system. Background Art

[0002] Digital video compression technology primarily compresses large amounts of digital video data for easier transmission and storage. Currently popular video codec standards, such as H.266 / Versatile Video Coding (VVC), employ a block-based hybrid coding framework. Each video frame is divided into square largest coding units (LCUs) of equal size (e.g., 128x128, 64x64, etc.). Each LCU can be divided into rectangular coding units (CUs) based on a specific rule. Coding units may also be divided into prediction units (PUs) and transform units (TUs). The hybrid coding framework includes modules such as prediction, transform, quantization, entropy coding, and in-loop filtering. The prediction module includes intra-frame prediction and inter-frame prediction, which are used to reduce or remove inherent redundancy in the video. Inter-frame prediction includes motion estimation and motion compensation. Because adjacent pixels within a video frame are strongly correlated, intra-frame prediction is used in video codecs to eliminate spatial redundancy between adjacent pixels. Because adjacent frames within a video are highly similar, inter-frame prediction is used to eliminate temporal redundancy between them, thereby improving coding efficiency. In contrast to the prediction signal, the residual information is transformed, quantized, and entropy-encoded on a block-by-block basis into a bitstream.

[0003] With the surge in Internet videos and people's increasing demand for video clarity, although existing digital video compression standards can save a lot of video data, there is still a need to pursue better digital video compression technology to reduce the bandwidth and traffic pressure of digital video transmission.

[0004] SUMMARY OF THE INVENTION

[0005] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0006] An embodiment of the present disclosure provides a method for constructing a candidate list for intra-frame template matching prediction, comprising:

[0007] Determine a first search range for intra template matching prediction intraTMP for the current block;

[0008] searching for a reference block template according to the first search range, and calculating a difference between the searched reference block template and the current block template, wherein the reference block template corresponds to the reference block in a one-to-one manner;

[0009] A candidate list of intraTMP is constructed according to the difference, and N reference blocks in the candidate list and an order of the N reference blocks are determined, where N≥2.

[0010] An embodiment of the present disclosure further provides a video decoding method, including:

[0011] Decode the intra template matching prediction intraTMP mode usage flag of the current block;

[0012] When determining that the current block uses the intraTMP mode according to the intraTMP mode use flag, continue decoding the intraTMP index of the current block, where the intraTMP index is used to indicate the position of the reference block used by the current block in the intraTMP candidate list;

[0013] A candidate list is constructed, a reference block used by the current block is determined according to the intraTMP index and the candidate list, and intra-frame prediction is performed on the current block according to the reference block used by the current block.

[0014] An embodiment of the present disclosure further provides a video encoding method, including:

[0015] When it is determined that the current block allows the use of a multi-candidate intra-frame template matching prediction intraTMP mode, constructing an intraTMP candidate list according to the intraTMP candidate list construction method as described in any embodiment of the present disclosure, the candidate list including N reference blocks, N≥2;

[0016] Calculating the coding costs when predicting the current block based on the N reference blocks in the candidate list, and using the smallest coding cost as the coding cost of the multi-candidate intraTMP mode to perform rate-distortion optimization on the current block;

[0017] When it is determined through rate-distortion optimization that the current block uses multiple candidate intraTMP modes for intra prediction, syntax elements related to the current block and the multiple candidate intraTMP modes are encoded.

[0018] An embodiment of the present disclosure further provides a device for constructing a candidate list for intra-frame template matching prediction, comprising a processor and a memory storing a computer program, wherein when the processor executes the computer program, it can implement the method for constructing a candidate list for intra-frame template matching prediction as described in any embodiment of the present disclosure.

[0019] An embodiment of the present disclosure further provides a video decoding device, comprising a processor and a memory storing a computer program, wherein the processor can implement the video decoding method as described in any embodiment of the present disclosure when executing the computer program.

[0020] An embodiment of the present disclosure further provides a video encoding device, including a processor and a memory storing a computer program, wherein the processor can implement the video encoding method as described in any embodiment of the present disclosure when executing the computer program.

[0021] An embodiment of the present disclosure further provides a video encoding and decoding system, which includes the video encoding device described in any embodiment of the present disclosure and the video decoding device described in any embodiment of the present disclosure.

[0022] An embodiment of the present disclosure further provides a non-transitory computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program implements the method described in any embodiment of the present disclosure when executed by a processor.

[0023] An embodiment of the present disclosure further provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, it can implement the method described in any embodiment of the present disclosure.

[0024] An embodiment of the present disclosure further provides a method for determining a search range of an intraTMP, including:

[0025] When the intraTMP mode is allowed for the current block, the first search distance in the width direction and the second search distance in the height direction of the first search range are determined in the following manner relative to a base point indicating the position of the current block:

[0026] Calculating the product of the current block width and a first scale factor, and using the larger value of the product and a set minimum search distance in the width direction as the first search distance; calculating the product of the current block height and a second scale factor, and using the larger value of the product and a set minimum search distance in the height direction as the second search distance, wherein the first scale factor and the second scale factor are equal or different; or

[0027] Determine a larger value between a current block width and a set minimum search distance in a width direction, and use the product of the larger value and a first scale factor as the first search distance; determine a larger value between a current block height and a set minimum search distance in a height direction, and use the product of the larger value and a second scale factor as the second search distance, where the first scale factor and the second scale factor are equal or different; or

[0028] The first search distance is obtained by multiplying the current block width by the corresponding first scale factor. There are multiple first scale factors. The larger the first scale factor, the larger the corresponding current block width. The second search distance is obtained by multiplying the current block height by the corresponding second scale factor. There are multiple second scale factors. The larger the second scale factor, the larger the corresponding current block height.

[0029] Still other aspects will become apparent upon reading and understanding the accompanying drawings and detailed description.

[0030] Summary of the Figures

[0031] The accompanying drawings are used to provide an understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the technical solutions of the present disclosure and do not constitute a limitation to the technical solutions of the present disclosure.

[0032] FIG1A is a schematic diagram of a coding and decoding system according to an embodiment of the present disclosure;

[0033] FIG1B is a framework diagram of an encoding end according to an embodiment of the present disclosure;

[0034] FIG1C is a framework diagram of a decoding end according to an embodiment of the present disclosure;

[0035] FIG2A is a schematic diagram of predicting a current block using an intra-frame prediction method;

[0036] FIG2B is a schematic diagram of predicting a current block using a multi-reference row intra prediction method;

[0037] FIG3 is a schematic diagram of a conventional intra mode used in a non-wide angle mode in VVC;

[0038] FIG4 is a schematic diagram of a conventional intra mode used in a wide angle mode in VVC;

[0039] FIG5 is a schematic diagram of a conventional intra-frame mode used by AVS3;

[0040] FIG6 is a schematic diagram of intra-frame prediction based on the IBC mode;

[0041] FIG7 is a schematic diagram of inter-frame prediction based on template matching technology;

[0042] FIG8 is a schematic diagram of intra-frame prediction based on the intraTMP mode;

[0043] FIG9 is a flowchart of a method for constructing a candidate list of intraTMP according to an embodiment of the present disclosure;

[0044] FIG10 is a schematic diagram of setting a search distance according to an embodiment of the present disclosure;

[0045] FIG11 is a flowchart of a video decoding method according to an embodiment of the present disclosure;

[0046] FIG12 is a flowchart of a video encoding method according to an embodiment of the present disclosure;

[0047] FIG13A is a schematic diagram of the position indicated by BV during the first level search according to an embodiment of the present disclosure;

[0048] FIG13B is a schematic diagram of determining a local search range during a second-level search based on the BV retained by the first-level search according to an embodiment of the present disclosure;

[0049] FIG14 is a module diagram of an intra-frame prediction apparatus according to an embodiment of the present disclosure.

[0050] Details

[0051] The present disclosure describes multiple embodiments, but the description is exemplary rather than restrictive, and it is obvious to those skilled in the art that there may be more embodiments and implementations within the scope of the embodiments described in the present disclosure.

[0052] In the description of the present disclosure, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment described as "exemplary" or "for example" in the present disclosure should not be interpreted as being more preferred or advantageous than other embodiments. "And / or" in this article is a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. "Multiple" refers to two or more than two. In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present disclosure, words such as "first" and "second" are used to distinguish between identical or similar items with basically the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0053] In this document, "including any one or more of the following: Option 1, Option 2,..." or "including any one or more of Option 1, Option 2,..." means including any one of the listed options, or any combination of multiple options. For example, "including any one or more of the following: A, B" or "including any one or more of A and B" means including only A, only B, or both A and B. Another example is: "including any one or more of the following: A, B, C" or "including any one or more of A, B, and C" means including only A, only B, only C, both A and B, both A and C, both B and C, or both A, B, and C. The same applies to options with more options.

[0054] When describing representative exemplary embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of the steps described herein, the method or process should not be limited to the steps in the specific order described. As will be understood by those skilled in the art, other sequences of steps are also possible. Therefore, the specific sequence of the steps set forth in the specification should not be interpreted as a limitation to the claims. In addition, the claims for the method and / or process should not be limited to the steps performed in the order written, and those skilled in the art can readily understand that these sequences can vary and still remain within the spirit and scope of the disclosed embodiments.

[0055] The intra-frame prediction method and video coding and decoding method of the embodiments of the present disclosure can be applied to various video coding and decoding standards, such as: H.264 / Advanced Video Coding (AVC), H.265 / High Efficiency Video Coding (HEVC), H.266 / Versatile Video Coding (VVC), AVS (Audio Video Coding Standard), and other standards developed by MPEG (Moving Picture Experts Group), AOM (Alliance for Open Media), JVET (Joint Video Experts Team) and extensions of these standards, or any other customized standards.

[0056] Figure 1A is a block diagram of a video encoding and decoding system that can be used in embodiments of the present disclosure. As shown in the figure, the system is divided into an encoding end 1 and a decoding end 2. The encoding end 1 generates a bitstream. The decoding end 2 can decode the bitstream. The decoding end 2 can receive the bitstream from the encoding end 1 via a link 3. The link 3 includes one or more media or devices capable of moving the bitstream from the encoding end 1 to the decoding end 2. In one example, the link 3 includes one or more communication media that enable the encoding end 1 to send the bitstream directly to the decoding end 2. The encoding end 1 modulates the bitstream according to a communication standard and sends the modulated bitstream to the decoding end 2. The one or more communication media may include wireless and / or wired communication media and may form part of a packet network. In another example, the bitstream can also be output from the output interface 15 to a storage device, and the decoding end 2 can read the stored data from the storage device via streaming or downloading.

[0057] As shown in the figure, encoding end 1 includes a data source 11, a video encoding device 13, and an output interface 15. Data source 11 may include a video capture device (e.g., a camera), an archive containing previously captured data, a feed interface for receiving data from a content provider, a computer graphics system for generating data, or a combination of these sources. Video encoding device 13, also referred to as a video encoding end, encodes the data from data source 11 and outputs it to output interface 15. Output interface 15 may include at least one of a modulator, a modem, and a transmitter. Decoding end 2 includes an input interface 21, a video decoding device 23, and a display device 25. Input interface 21 includes at least one of a receiver and a modem. Input interface 21 may receive a bitstream via link 3 or from a storage device. Video decoding device 23, also referred to as a video decoding end, is used to decode the received bitstream. Display device 25 is used to display the decoded data. Display device 25 may be integrated with other devices in decoding end 2 or provided separately; display device 25 is optional for the decoding end. In other examples, the decoding end may include other devices or equipment for applying the decoded data.

[0058] FIG1B is a block diagram of an exemplary video encoding device that can be used in an embodiment of the present disclosure. As shown in the figure, the video encoding device 10 includes:

[0059] The division unit 101 is configured to cooperate with the prediction unit 100 to divide the received video data into slices, coding tree units (CTUs) or other larger units. The received video data may be a video sequence including video frames such as I frames, P frames or B frames.

[0060] The prediction unit 100 is configured to divide a CTU into coding units (CUs) and perform intra-frame prediction coding or inter-frame prediction coding on the CU. When performing intra-frame prediction and inter-frame prediction on the CU, the CU can be divided into one or more prediction units (PUs).

[0061] The prediction unit 100 includes an inter-frame prediction unit 121 and an intra-frame prediction unit 126 .

[0062] The inter-frame prediction unit 121 is configured to perform inter-frame prediction on the PU and generate prediction data for the PU, wherein the prediction data includes the prediction block of the PU, the motion information of the PU, and various syntax elements. The inter-frame prediction unit 121 may include a motion estimation (ME) unit and a motion compensation (MC) unit. The motion estimation unit can be used to perform motion estimation to generate a motion vector, and the motion compensation unit can be used to obtain or generate a prediction block based on the motion vector.

[0063] The intra prediction unit 126 is configured to perform intra prediction on a PU and generate prediction data for the PU. The prediction data for the PU may include a prediction block and various syntax elements of the PU.

[0064] The residual generating unit 102 (indicated by the circle with a plus sign after the dividing unit 101 in the figure) is configured to subtract the prediction block of the PU into which the CU is divided from the original block of the CU to generate a residual block of the CU.

[0065] The transform processing unit 104 is configured to partition a CU into one or more transform units (TUs). The partitioning of prediction units and transform units may be different. A TU-associated residual block is a sub-block obtained by partitioning the residual block of the CU. A TU-associated coefficient block is generated by applying one or more transforms to the TU-associated residual block.

[0066] The quantization unit 106 is configured to quantize the coefficients in the coefficient block based on a quantization parameter. The quantization degree of the coefficient block can be changed by adjusting the quantization parameter (QP: Quantizer Parameter).

[0067] The inverse quantization unit 108 and the inverse transform unit 110 are configured to apply inverse quantization and inverse transform to the coefficient block, respectively, to obtain a reconstructed residual block associated with the TU.

[0068] The reconstruction unit 112 (represented by the circle with a plus sign after the inverse transform processing unit 110 in the figure) is configured to add the reconstructed residual block and the prediction block generated by the prediction unit 100 to generate a reconstructed image.

[0069] The filter unit 113 is configured to perform loop filtering on the reconstructed image.

[0070] The decoded image buffer 114 is configured to store the reconstructed image after loop filtering. The intra prediction unit 126 can extract reference images of blocks adjacent to the current block from the decoded image buffer 114 to perform intra prediction. The inter prediction unit 121 can use the reference image of the previous frame cached in the decoded image buffer 114 to perform inter prediction on the PU of the current frame image.

[0071] The entropy coding unit 115 is configured to perform entropy coding operations on received data (such as syntax elements, quantized coefficient blocks, motion information, etc.) to generate a video bitstream.

[0072] In other examples, the video encoding apparatus 10 may include more, fewer, or different functional components than those in this example, for example, the transform processing unit 104 and the inverse transform processing unit 110 may be eliminated.

[0073] FIG1C is a block diagram of an exemplary video decoding device that can be used in an embodiment of the present disclosure. As shown in the figure, the video decoding device 15 includes:

[0074] Entropy decoding unit 150 is configured to perform entropy decoding on the received encoded video stream, extracting syntax elements, quantized coefficient blocks, and motion information for PUs. Prediction unit 152, inverse quantization unit 154, inverse transform processing unit 156, reconstruction unit 158, and filter unit 159 may each perform corresponding operations based on the syntax elements extracted from the stream.

[0075] The inverse quantization unit 154 is configured to perform inverse quantization on the coefficient block associated with the quantized TU.

[0076] The inverse transform processing unit 156 is configured to apply one or more inverse transforms to the inverse quantized coefficient block to generate a reconstructed residual block for the TU.

[0077] Prediction unit 152 includes an inter-prediction unit 162 and an intra-prediction unit 164. If the current block is encoded using intra prediction, intra prediction unit 164 determines an intra prediction mode for the PU based on syntax elements decoded from the codestream, and performs intra prediction in conjunction with reconstructed reference information adjacent to the current block obtained from decoded image buffer 160. If the current block is encoded using inter prediction, inter prediction unit 162 determines a reference block for the current block based on motion information of the current block and corresponding syntax elements, and performs inter prediction on the reference block obtained from decoded image buffer 160.

[0078] The reconstruction unit 158 ​​(represented by a circle with a plus sign after the inverse transform processing unit 155 in the figure) is set to perform intra-frame prediction or inter-frame prediction on the current block based on the reconstructed residual block associated with the TU and the prediction unit 152 to obtain a reconstructed image.

[0079] The filter unit 159 is configured to perform loop filtering on the reconstructed image.

[0080] The decoded image buffer 160 is configured to store the reconstructed image after loop filtering as a reference image for subsequent motion compensation, intra-frame prediction, inter-frame prediction, etc. The filtered reconstructed image can also be output as decoded video data for presentation on a display device.

[0081] In other embodiments, the video decoding device 15 may include more, fewer, or different functional components. For example, the inverse transform processing unit 155 may be eliminated in some cases.

[0082] Based on the above-described video encoding and decoding devices, the following basic encoding and decoding process can be performed. On the encoding side, a frame of an image is divided into blocks, or first into multiple slices and then into blocks. Slices within the same image can be processed in parallel. Intra-frame prediction, inter-frame prediction, or other algorithms are performed on the current block to generate a prediction block for the current block. The prediction block is subtracted from the original block of the current block to obtain a residual block. The residual block is transformed and quantized to obtain a quantization coefficient matrix. The quantization coefficient matrix is ​​then entropy encoded to generate a bitstream. On the decoding side, intra-frame prediction or inter-frame prediction is performed on the current block to generate a prediction block for the current block. The quantization coefficient matrix obtained from the decoded bitstream is then inversely quantized and inversely transformed to obtain a residual block. The prediction block and residual block are added together to obtain a reconstructed block. The reconstructed block forms a reconstructed image. The reconstructed image is then subjected to image-based or block-based loop filtering to obtain a decoded image. The encoding side also performs similar operations as the decoding side to obtain a decoded image, also known as a loop-filtered reconstructed image. The loop-filtered reconstructed image can be used as a reference frame for inter-frame prediction of subsequent frames. The block division information, prediction, transform, quantization, entropy coding, loop filtering, and other mode and parameter information determined by the encoder can be written into the bitstream. The decoder determines the block division information, prediction, transform, quantization, entropy coding, loop filtering, and other mode and parameter information used by the encoder by decoding the bitstream or analyzing existing information, thereby ensuring that the decoded image obtained by the encoder and the decoder is the same.

[0083] While the above example is based on a block-based hybrid coding framework, the embodiments of the present disclosure are not limited thereto. With technological advancements, one or more modules in the framework, and one or more steps in the process, may be replaced or optimized. The embodiments of the present disclosure relate to, but are not limited to, the intra-frame prediction units and corresponding intra-frame prediction methods in the encoding and decoding ends described above.

[0084] Herein, the current block may be a coding unit (CU) or a prediction unit (PU) currently being encoded or decoded, or a block-level coding unit such as a sub-block into which a CU or PU is divided.

[0085] Intra-frame prediction

[0086] The intra-frame prediction method uses the reconstructed pixels that have been coded and decoded around the current block as reference pixels to predict the current block. An example is shown in Figure 2A. The 4x4 block in the figure is the current block, and the pixels in the left column and the upper row of the current block are the reference pixels of the current block. Intra-frame prediction uses these reference pixels to predict the current block. These reference pixels may all have been coded and decoded, or some may not be available. For example, if the current block is the leftmost of the entire frame, the reference pixels on the left side of the current block are not available. Or when encoding and decoding the current block, the lower left part of the current block has not been coded and decoded, so the reference pixels on the lower left are also not available. For the case where reference pixels are unavailable, available reference pixels or certain values ​​or methods can be used for filling, or no filling can be performed.

[0087] Multiple reference line (MRL) intra prediction can use more reference pixels to improve coding efficiency. FIG2B shows an example of using four reference rows / columns.

[0088] Traditional intra-frame prediction mode

[0089] Intra-frame prediction has multiple prediction modes, and with technological advancements and the expansion of block sizes, the number of prediction modes is increasing. For example, HEVC uses a planar mode, a mean value mode (DC mode), and 33 angle modes, for a total of 35 prediction modes. VVC uses a planar mode, a DC mode, and 65 angle modes, for a total of 67 prediction modes, as shown in Figure 3. In addition to these 67 modes, VVC also provides wide-angle modes for rectangular blocks with a large difference between length and width. These modes, indicated by the dotted lines in Figure 4, represent the ranges -14 to -1 and 67 to 80, replacing some conventional modes. AVS3 uses a DC mode, a planar mode, a bilinear mode, a pulse code modulation mode (PCM mode), and 62 angle modes, for a total of 66 prediction modes, as shown in Figure 5.

[0090] Inter-frame prediction

[0091] Videos are composed of multiple images. To ensure a smooth video appearance, each second of video contains dozens or even hundreds of frames. Examples include 24 fps, 30 fps, 50 fps, 60 fps, and 120 fps. This results in significant temporal redundancy in the video. In other words, there is a significant amount of temporal correlation. Inter-frame prediction exploits this temporal correlation to improve compression efficiency. Inter-frame prediction often uses motion to exploit this temporal correlation. A very simple motion model is that an object is at a certain position in the image corresponding to a certain moment. After a certain amount of time, it moves to a different position in the image corresponding to the same moment. This is called translational motion in video codecs. Inter-frame prediction uses motion information to represent motion. Basic motion information includes information about the reference frame (or reference picture) and motion vectors (MVs). The codec determines the reference picture based on the reference picture information and the coordinates of the reference block based on the motion vector information and the coordinates of the current block. The reference block is determined by the coordinates of the reference block in the reference picture. Taking the determined reference block as the prediction block is the most basic prediction method of inter-frame prediction.

[0092] Motion in videos isn't always simple. Even motion that can be considered translational can exhibit subtle changes over time, including subtle deformations, brightness variations, and noise variations. For better prediction, more than one reference block can be used to predict the current block. For example, bidirectional prediction, commonly used today, uses two reference blocks to predict the current block. These two reference blocks can be a forward reference block and a backward reference block. Future video codec standards may support prediction using multiple reference blocks. A simple method for generating a prediction block using two reference blocks is to average the pixel values ​​at corresponding positions in the two reference blocks. For even better prediction, a weighted average can be used, such as the BCW (Bi-prediction with CU-level weights) currently used in VVC. VVC's GPM (Geometric Partitioning Mode) can also be considered a special form of bidirectional prediction. Bidirectional prediction requires two reference blocks, which requires information from two reference images and motion vectors.

[0093] Intra Block Copy (IBC)

[0094] Intra Block Copy (IBC) technology can significantly improve the compression efficiency of screen content coding. Therefore, from HEVC to VVC, IBC mode has been used for screen content coding. Screen content is different from camera-captured content. It is generated by computers. Screen content is noise-free and contains text, computer graphics, etc. with clear boundaries. There is also a lot of repeated content in the screen content, as shown in Figure 6.

[0095] Inter-frame prediction uses a reference block in a different image as the prediction block for the current block. The reference block is not the current image. The IBC mode, however, applies the inter-frame prediction method to intra-frame prediction. IBC mode uses a block from the coded (or reconstructed) portion of the current image as the prediction block for the current block. IBC mode is also known as intra picture block compensation (IPBC) or current picture referencing (CPR) mode.

[0096] The IBC mode uses a block vector (BV) to represent the positional difference between the current block and the reference block, similar to the motion vector (MV) for inter-frame prediction. The encoder uses block matching within the search range to determine the best match for the current block and encodes the BV. IBC can be considered a form of intra-frame prediction, or a separate prediction method from both intra-frame and inter-frame prediction.

[0097] Template Matching(TM)

[0098] Template matching (TM) technology was first used in inter-frame prediction. It uses the correlation between adjacent pixels and uses some areas around the current block as templates. When the current block is encoded and decoded, the left and top sides of the current block have already been encoded and decoded according to the encoding order. When implemented on the actual hardware decoding end, it is not necessarily guaranteed that the left and top sides of the current block have been decoded when the current block begins decoding. For example, in HEVC, when generating a prediction block for an inter-frame coded block, the surrounding reconstructed pixels are not required, so the prediction process of the inter-frame block can be carried out in parallel. However, intra-frame coded blocks require reconstructed pixels on the left and top sides as reference pixels. Through corresponding adjustments to the hardware design, the reconstructed pixels on the left and top sides of the current block can be obtained. However, the reconstructed pixels on the right and bottom sides of the current block are not available under the encoding order of existing standards such as VVC.

[0099] As shown in Figure 7, the rectangular areas to the left and above the current block are used as templates. The height of the left template is generally the same as the height of the current block, and the width of the upper template is generally the same as the width of the current block, but can be different. The best matching position of the template is found in the reference frame to determine the motion information, or motion vector, of the current block. This process can be roughly described as starting from a starting position in a reference frame and searching within a certain range around it. Search rules, such as the search range and search step size, can be predefined. At each position, the degree of match between the template corresponding to that position and the templates surrounding the current block is calculated. The degree of match can be measured using differences, such as the sum of absolute difference (SAD), the sum of absolute transformed differences (SATD) after a Hadamard transformation, or the mean-square error (MSE). Smaller values ​​of SAD, SATD, and MSE indicate a higher degree of match. The cost is calculated based on the predicted block of the template corresponding to that position and the reconstructed blocks of the templates surrounding the current block. The motion information of the current block is determined based on the position with the highest degree of template matching. By utilizing the correlation between adjacent pixels, the motion information that is appropriate for the template may also be appropriate for the current block.

[0100] The template matching method may not be applicable to all blocks, so some methods can be used to determine whether the template matching method is used for the current block, such as using a control switch on the current block to indicate whether the template matching method is used. A classic template matching technology is called decoder-side motion vector derivation (DMVD). Both the encoder and decoder can use templates to search to derive motion information, or find better motion information based on the original motion information. It does not require the transmission of specific motion vectors or motion vector differences. Instead, both the encoder and decoder search according to the same rules, thus ensuring consistency in encoding and decoding. The template matching method can improve compression performance, but it also requires "searching" on the decoder side, which brings a certain degree of decoding complexity.

[0101] Intra-frame template matching prediction (intraTMP)

[0102] Intra-template matching prediction (intraTMP) is a technique that combines IBC and TM. Applying TM to interframes can reduce the overhead of encoding MVs. Similarly, applying TM to IBC can reduce the overhead of encoding BVs. For example, instead of encoding BVs, the block with the highest matching score found by TM can be used as the prediction block for the intraTMP mode of the current block, participating in rate-distortion optimization to determine the intra-frame prediction mode for the current block.

[0103] An example of intraTMP is shown in Figure 8. The inverted L-shaped area in the upper left corner of the current block is the template of the current block. The partial area R1 of the current CTU shown in the figure, the CTU area R2 on the upper left side of the current block, the CTU area R3 on the upper side of the current block, and the CTU area R4 on the left side of the current block are the reconstruction areas available for search. The reference block template should be searched within the available reconstruction area, and the actual search range can be smaller than the reconstruction area. However, this is just an example, and the available reconstruction areas in actual applications may be different. The illustrated example finds the best matching block in R2 through the search, that is, the reference block corresponding to the reference block template with the smallest difference (i.e., the highest matching degree) from the current block template. The reference block in the R2 area in the figure is the best matching block, and the area filled with diagonal lines surrounding the left and top sides of the reference block is the template of the reference block, also known as the reference block template corresponding to the reference block.

[0104] As mentioned above, IBC can significantly improve the compression efficiency of screen content encoding. One of the important reasons is that many repeated blocks can be found in screen content, and screen content usually has sharp boundaries. In terms of color, there will be large areas of the same color (brightness and chroma). However, this situation is almost non-existent in content captured by a camera. The content captured by a camera inevitably contains noise. Even if the color of some areas captured by the camera appears to be uniform at first glance, the brightness and chroma will vary to a greater or lesser extent. The content captured by the camera rarely has sharp boundaries. On the other hand, due to reasons such as perspective, it is difficult to find exactly the same blocks in the content captured by the camera, but repeated textures do exist in the content captured by the camera. The content captured by the camera does have nearly repeated blocks, that is, there are only slight changes in noise and brightness between blocks.

[0105] IntraTMP will use the best matching block found through intra-frame template matching as its final prediction block. In other words, when decoding the current block, you can determine whether the current block uses intraTMP by decoding a flag. If the current block uses intraTMP, the decoder will use the intra-frame template matching method to search for a best matching block in the reconstruction area, and use the reconstructed value of the best matching block as the prediction value of the current block. The decoding end does not have the original value of the current block when searching, and can only use the block with the highest template matching degree as the best matching block searched by intraTMP. However, although the template is strongly correlated with the current block, it is not the current block after all. The best matching block found by template matching is not necessarily the actual best matching block of the current block. The coding efficiency in intraTMP mode needs to be further improved.

[0106] To this end, an embodiment of the present disclosure provides an intra-frame prediction method for intraTMP, as shown in FIG9 , including the following steps:

[0107] Step S110, determining a search range for intraTMP prediction of the current block;

[0108] Step S120, searching for a reference block template according to the search range, and calculating a difference between the searched reference block template and the current block template, wherein the reference block template corresponds to the reference block in a one-to-one manner;

[0109] Step S130 : constructing a candidate list of intraTMP according to the difference, and determining N reference blocks in the candidate list and the order of the N reference blocks, where N≥2.

[0110] The reference block template corresponding to a reference block referred to herein is the template of the reference block. As shown in FIG8 , in the R2 region, a reference block and a template of the reference block are shown. The template of the reference block is represented in the figure as an area filled with oblique lines, also referred to as the reference block template corresponding to the reference block. The size and shape of the template of the reference block are the same as the size and shape of the current block template, and the relative positional relationship between the template of the reference block and the reference block is also the same as the relative positional relationship between the current block template and the current block. In the illustrated example, the current block template is an L-shaped area surrounding the left and top sides of the current block, while the template of the reference block is an L-shaped area surrounding the left and top sides of the reference block. The disclosed embodiment does not impose any restrictions on the number of rows and columns included in the reference block template and the current block template. In addition, the template of a block may also extend to the upper right and / or lower left sides of the block.

[0111] The disclosed embodiment constructs a candidate list including multiple reference blocks through difference-based template matching. The multiple reference blocks in the candidate list may be used as reference blocks for intra-frame prediction of the current block in intraTMP mode. The first option in the candidate list is the best matching block determined by template matching, but the coding efficiency of the best matching block when used for current block prediction is not necessarily the best. Using other reference blocks in the candidate list for prediction may result in higher overall coding efficiency. By constructing a candidate list, an index can be used to indicate a reference block with the highest coding efficiency. The encoding end constructs the candidate list in the same way, and finds the reference block indicated by the index according to the index to predict the current block, thereby improving coding efficiency.

[0112] In an exemplary embodiment of the present disclosure, the search range is within the reconstruction area of ​​the current image, and the difference between the reference block template and the current block template is determined based on the SAD, SATD, or MSE between the reconstructed values ​​of the reference block template and the reconstructed values ​​of the current block template. This embodiment uses the SAD, SATD, or MSE between the reconstructed values ​​of the reference block template and the reconstructed values ​​of the current block template to represent the difference between the reference block template and the current block template, thereby reflecting the degree of similarity between the two templates.

[0113] In an exemplary embodiment of the present disclosure, the order of the N reference blocks in the candidate list is determined based on the order of differences in the corresponding reference block templates, from smallest to largest. The template of the reference block has a strong correlation with the reference block. The reference block template with the highest degree of similarity, determined by template matching (i.e., calculating the difference in the templates), has a high probability of corresponding to the reference block with the highest degree of similarity to the current block. Therefore, the embodiment of the present disclosure determines the order of the N reference blocks in the candidate list based on the order of differences in the corresponding reference block templates, from smallest to largest. This ensures that the candidate reference blocks at the front are more likely to be selected, and the codewords at this time are shorter, which can save coding overhead.

[0114] In an exemplary embodiment of the present disclosure, the reference block in the candidate list is identified by a block vector BV of the reference block, and the BV of the reference block is used to indicate the position of the reference block relative to the current block, and the BV corresponding to the reference block template is the BV of the reference block corresponding to the reference block template.

[0115] In this embodiment, the reference block's BV is used to identify the reference block in the candidate list, that is, the BV is actually filled in the candidate list. The current block position can be represented by a specified base point, which can be a pixel point in the current block. The base point in this embodiment is the point (pixel point) in the upper left corner of the current block, but the present disclosure is not limited to this. It can also be the upper right corner of the current block, the center point, or a point adjacent to the center point. In other examples, a point on the current block template can be used as the base point. As long as the relative position of the base point and the current block is fixed and known, it can be used to locate the current block. In this embodiment, assuming that the coordinates of the base point are (50,50) and the coordinates of the upper left corner point of a searched reference block are (120,120), the BV used when searching for the reference block can be represented as (70,70), which is the position offset relative to the base point. Graphically, it can be represented by a vector pointing from the base point to the upper left corner point of the reference block, see Figure 8. The coordinates of the base point plus the position offset represented by the BV are called the position indicated by the BV. In Figure 8, it is the upper left corner point of the reference block. For the sake of convenience, this article refers to the BV of a reference block corresponding to a reference block template as the BV corresponding to the reference block template, and the two are also one-to-one corresponding.

[0116] In an exemplary embodiment of the present disclosure, a reference block template is searched according to the first search range, a difference between the searched reference block template and the current block template is calculated, and the candidate list of intraTMP is constructed according to the difference, including: determining a group of BVs according to the first search step and the first search range, the positions indicated by the group of BVs are within the first search range; searching for the corresponding reference block template according to the group of BVs; and calculating the difference between the searched reference block template and the current block template, and filling the BVs corresponding to N reference block templates with the smallest differences into the candidate list.

[0117] This embodiment searches for reference block templates based on BV. As mentioned above, a BV can indicate the position of a reference block. For example, when the upper left corner of the current block is used as the base point, the BV can indicate the upper left corner of the reference block. Since the size and shape of the reference block are the same as those of the current block, the area where the reference block is located, or the reconstructed pixels included in the reference block, can be determined based on the position indicated by a BV. The relative position of the template of the reference block and the reference block is fixed, so a BV can also determine the area where a reference block template is located. The search for the reference block template can be achieved based on a group of BVs. A reference block can be uniquely determined based on a BV, so the BV of the reference block can be used as the identifier of the reference block to fill in the candidate list.

[0118] In this embodiment, when constructing a candidate list, the N reference blocks in the candidate list and their order are determined based on the difference between the searched reference block template and the current block template. In one example, the candidate list is populated with the first N reference block templates found. Starting with the N+1th reference block template found, the difference between the currently found reference block template and the N reference block templates in the candidate list is compared. If the difference between the reference block template and the N reference block templates in the candidate list is less than the largest difference among the N reference block templates in the candidate list, the candidate list is updated, the BV corresponding to the largest difference is deleted, and the BV corresponding to the reference block template is added to the candidate list. After processing the last reference block template found, the candidate list is constructed. During the construction process, the N BVs in the candidate list are sorted in ascending order by the size of their corresponding differences to facilitate comparison. The differences corresponding to the BVs are the differences between the reference block templates corresponding to the BVs. In another example, after all reference block templates have been searched, the N reference block templates with the smallest differences are added to the candidate list based on the differences between the reference block templates to complete the candidate list construction.

[0119] In this article, the difference of the reference block template refers to the difference between the reference block template and the current block template, which is called the difference of the reference block template for convenience of expression.

[0120] In an exemplary embodiment of the present disclosure, searching for a reference block template according to the first search range, calculating a difference between the searched reference block template and the current block template, and constructing the candidate list of intraTMP according to the difference includes:

[0121] Determining a set of BVs according to the first search step and the first search range, searching according to the set of BVs, calculating differences between the searched reference block templates and the current block template, and filling the BVs corresponding to the N reference block templates with the smallest differences into the candidate list;

[0122] Determine M second search ranges based on the BVs corresponding to the M reference block templates with the smallest differences found in the first search, determine M groups of BVs based on the second search step and the M second search ranges, search for corresponding reference block templates in the M second search ranges based on the M groups of BVs, calculate the difference between the searched reference block template and the current block template, and update the candidate list based on the difference; wherein, the second search step is smaller than the first search step, the second search range is smaller than the first search range, and the second search ranges do not overlap with each other, and M≥N.

[0123] In an example of this embodiment, the BV is represented by a position offset relative to a base point, where the base point is a point (pixel point or sub-pixel point) in the current block; the M second search ranges respectively cover the positions indicated by the BVs corresponding to the M reference block templates, and the positions indicated by the BVs are determined based on the base point and the position offset.

[0124] This embodiment is a hierarchical search method, divided into two levels. The search step size of each level is smaller than that of the previous level. The next level has multiple search ranges, each of which is a portion of the previous level's search range. A first-level search is performed within the first search range using a larger search step size. N BVs are selected based on the differences in the reference block templates found and added to the candidate list. The second-level search determines M second search ranges based on the M BVs recorded after the first-level search. A smaller search step size is used within the second search ranges, and the candidate list is then updated based on the differences in the reference block templates found. Hierarchical search is a precise and detailed search method that can quickly and accurately find reference block templates with high matching scores in the reconstruction area and complete the construction of the candidate list.

[0125] In an example of this embodiment, after updating the candidate list according to the difference, the method further includes:

[0126] Determine M' third search ranges based on the BVs corresponding to the M' reference block templates with the smallest differences found in the second search, and determine M' groups of BVs based on the third search step and the M' third search ranges; search for corresponding reference block templates based on the M' groups of BVs in the M" third search ranges, calculate differences between the searched reference block templates and the current block template, and update the candidate list based on the differences; wherein the third search step is smaller than the second search step, the third search ranges are smaller than the second search ranges and do not overlap with each other, and M' ≥ N;

[0127] Among them, a group of BVs determined for each third search range is BVs of integer pixels; or, a group of BVs determined for each third search range is BVs of sub-pixels, and the reconstructed values ​​of the reference block templates corresponding to the BVs of the sub-pixels are obtained by interpolation.

[0128] This embodiment employs a three-stage search method. A more refined third-stage search is performed based on the second-stage search. This method can search for reference block templates in more locations, thereby increasing the likelihood of finding a reference block template with a higher degree of actual matching. The reference block corresponding to this reference block template also has a higher probability of being closer to the current block. Therefore, this embodiment's method can improve coding efficiency.

[0129] In an exemplary embodiment of the present disclosure, updating the candidate list according to the difference includes:

[0130] Determine the smallest difference d1 among the differences of the reference block templates searched in the same local search range. <D N In the case of N The corresponding BV is deleted from the candidate list, and the BV corresponding to d1 is added to the candidate list;

[0131] Among them, D N It is the largest difference among the differences corresponding to the N BVs in the candidate list before updating, the difference corresponding to the BV refers to the difference of the reference block template corresponding to the BV, and the local search range is the second search range or the third search range.

[0132] As described above, a local search range is determined based on the BV corresponding to a reference block template previously searched, and the BV used to determine the local search range is also the BV in the local search range. The reference block template previously searched based on the BV also belongs to the reference block template searched in the local search range. In this article, the reference block templates searched in the same local search range include not only the reference block templates searched after determining the local search range, but also the reference blocks corresponding to the BV used to determine the local search range. Taking the second search range as an example, the reference block templates searched in the same second search range include the reference block templates searched in the second search range when performing the second-level search, and also include the reference block templates corresponding to the BV used to determine the second search range (searched during the first-level search). For the third search range, the reference block templates searched in the same third search range include the reference block templates searched in the third search range when performing the third-level search, and also include the reference block templates corresponding to the BV used to determine the third search range (searched during the first-level search or the second-level search).

[0133] The update process of the candidate list in this embodiment can be applied after the second-level search or after the third-level search. When updating, this embodiment limits the BV corresponding to the reference block template searched from the same group of BVs, that is, the BV of the reference block template searched from a local search range, and at most only one can be added to the candidate list. The local search range can be the second search range, the third search range, etc. If it is a second-level search, each group of BVs in the determined M groups of BVs is processed in this way. If it is a third-level search, each group of BVs in the determined M' groups of BVs is processed in this way. In addition, when each embodiment adds a new BV to the candidate list, the N BVs in the updated candidate list can be reordered in ascending order according to the corresponding differences.

[0134] This embodiment limits the maximum number of BVs corresponding to reference block templates searched from a local search range that are added to the candidate list to 1. This is because the differences between reference block templates that are close to each other are usually similar, so it is easy to fill the candidate list with BVs of multiple reference block templates that are close to each other. This makes the reference blocks in the candidate list too concentrated in a certain position. If a reference block at that position is not very similar to the current block, then there will be multiple reference blocks in the candidate list that are not very similar to the current block. This makes the candidate list less adaptable. By limiting the above number, the positions of the reference blocks added to the candidate list will not be too concentrated, and the texture features of these reference blocks are different. This, to a certain extent, avoids the situation where the reference block positions are too concentrated and no blocks with a high degree of match with the current block can be found in the candidate list.

[0135] In an exemplary embodiment of the present disclosure, updating the candidate list according to the difference includes:

[0136] Determine the smallest K differences among the differences of the reference block templates searched in the same local search range, where at least one of the K differences is smaller than D N In the case of , the candidate list is updated, and the N BVs in the candidate list after the update are the BVs corresponding to the K differences and the N BVs in the candidate list before the update with the smallest differences, K is a set threshold, K≥2;

[0137] Among them, D N It is the largest difference among the differences corresponding to the N BVs in the candidate list before updating, the difference corresponding to the BV refers to the difference of the reference block template corresponding to the BV, and the local search range is the second search range or the third search range.

[0138] The difference between this embodiment and the previous embodiment is that the maximum number of BVs corresponding to reference block templates found in a local search range that are added to the candidate list is limited to K, where K is an integer greater than or equal to 2. This threshold can be agreed upon, that is, a default value, or it can be set at the encoder and then transmitted from the encoder to the decoder.

[0139] In an exemplary embodiment of the present disclosure, updating the candidate list according to the difference includes:

[0140] For each reference block template found, the difference between the reference block template is less than D N In the case of N The corresponding BV is deleted from the candidate list, and the BV corresponding to the reference block template is added to the candidate list;

[0141] Among them, D NIt is the largest difference among the differences corresponding to the N BVs in the candidate list before updating. The difference corresponding to the BV refers to the difference of the reference block template corresponding to the BV.

[0142] This embodiment does not limit the number of BVs corresponding to reference block templates found from a local search range that can be added to the candidate list. Each BV of a reference block template found from the local search range can be added to the candidate list if the corresponding difference is small enough. In this embodiment, multiple BVs added to the candidate list may be concentrated in a local area. Although there are certain problems with adaptability, if the reference block at that location has a high degree of match with the current block, a reference block that is closer to the highest match may be found. This embodiment's update processing of the candidate list includes determining not to update the candidate list through difference comparison, and determining to update the candidate list through difference comparison.

[0143] This embodiment can perform a difference comparison and update process after calculating the difference of each reference block template; or perform a difference comparison and update process one by one after calculating the difference of the reference block templates searched within a local search range; or perform a difference comparison and update process one by one after calculating the difference of the reference block templates searched within all local search ranges. The first processing method requires fewer cache resources.

[0144] In an exemplary embodiment of the present disclosure, the size of the first search range is determined according to the size of the current block.

[0145] In an example of this embodiment, relative to a base point representing the current block position, a first search distance in the width direction and a second search distance in the height direction of the first search range are determined as follows:

[0146] Calculating the product of the current block width and a first scale factor, and using the larger value of the product and a set minimum search distance in the width direction as the first search distance; calculating the product of the current block height and a second scale factor, and using the larger value of the product and a set minimum search distance in the height direction as the second search distance, wherein the first scale factor and the second scale factor are equal or different; or

[0147] Determine a larger value between a current block width and a set minimum search distance in a width direction, and use the product of the larger value and a first scale factor as the first search distance; determine a larger value between a current block height and a set minimum search distance in a height direction, and use the product of the larger value and a second scale factor as the second search distance, where the first scale factor and the second scale factor are equal or different; or

[0148] The first search distance is obtained by multiplying the current block width by the corresponding first scale factor. There are multiple first scale factors. The larger the first scale factor, the larger the corresponding current block width. The second search distance is obtained by multiplying the current block height by the corresponding second scale factor. There are multiple second scale factors. The larger the second scale factor, the larger the corresponding current block height.

[0149] In this embodiment, the search range size is represented by the search distances in the width and height directions. As shown in FIG10 , the searchRangeWidth in the figure represents the first search distance in the width direction of the first search range relative to the base point (the upper left corner of the current block), and the searchRangeHeight represents the second search distance in the height direction of the first search range relative to the base point. Of course, the size of the search range in this embodiment can also be represented in different ways. For example, twice the searchRangeWidth in the figure can be defined as the search distance in the width direction, twice the searchRangeHeight can be defined as the search distance in the height direction, and so on.

[0150] This embodiment adopts different search ranges according to different current blocks. Since the reference block has the same size as the current block, this method ensures that the number of reference blocks searched does not vary greatly with the current block size, thereby ensuring that there are enough reference blocks available for matching, thereby ensuring the coding effect of the intraTMP mode.

[0151] In an exemplary embodiment of the present disclosure, determining a first search range for intraTMP on a current block includes determining the area covered by the first search range based on a base point representing the current block's position, a search distance relative to the base point, and available reconstruction areas during the search. In this embodiment, as shown in Figures 8 and 10 , determining the area actually covered by the first search range requires consideration of the base point, the search distance relative to the base point (including in both directions), and the available reconstruction areas during the search. The available search areas are dependent on the set search directions. For example, as shown in the example of Figure 10 , only the left, top, top-left, bottom-left, and top-right sides of the current block are searched (i.e., reference block templates at corresponding positions are found based on the BV), indicating that reconstruction areas in these directions are available. In other examples, the search directions can be limited to the left, top, and top-left sides, although this disclosure is not limited to this. The available reconstruction areas during the search can also be directly set. For example, as shown in the example of Figure 8 , the search allows the use of reconstruction areas in the CTU containing the current block, as well as in the CTUs above, left, and top-left of the current block. In addition to this, there may be some other restrictions, for example, stipulating that the area above and to the left of the current block in the CTU where the current block is located is unavailable, etc.

[0152] After determining a set of BVs based on the search range and search step, if the reference block sample searched based on the BV is not within the available reconstruction area, the reference block template can be discarded; or, after determining the BV, it is determined whether the reference block template searched based on a BV is within the available reconstruction area, and if not, the BV is discarded, so that the searched reference block template is within the available reconstruction area.

[0153] The present disclosure also provides a video decoding method, as shown in FIG11 , including:

[0154] Step S210, decoding the intra template matching prediction intraTMP mode usage flag of the current block;

[0155] Step S220: When it is determined according to the intraTMP mode use flag that the current block uses the intraTMP mode, continue decoding the intraTMP index of the current block, where the intraTMP index is used to indicate the position of the reference block used by the current block in the intraTMP candidate list;

[0156] Step S230 , constructing a candidate list, determining a reference block used by the current block according to the intraTMP index and the candidate list, and performing intra-frame prediction on the current block according to the reference block used by the current block.

[0157] The intraTMP mode of this embodiment is a multi-candidate intraTMP mode. During decoding, a candidate list is constructed. The reference block used by the current block is determined based on the intraTMP index and the candidate list. Intra-frame prediction is then performed on the current block based on the reference block used by the current block. Because the candidate list includes multiple reference blocks, the current block may find a reference block with a higher degree of matching during prediction, thereby improving coding efficiency.

[0158] In an exemplary embodiment of the present disclosure, the candidate list is constructed according to the candidate list construction method of ntraTMP in any embodiment of the present disclosure. It should be noted that when constructing a candidate list according to the candidate list construction method of ntraTMP in any embodiment of the present disclosure, it is not necessary to construct a candidate list with a length of N. A candidate list with a length less than N can be constructed to simplify the processing. For example, according to the intraTMP index, it is determined that the reference block used by the current block is in the third position in the candidate list. When N=5, the decoding end can construct a candidate list with a length of 3. The construction method can be the same, but the length is different.

[0159] In an exemplary embodiment of the present disclosure, after decoding the intraTMP index of the current block, the method further includes:

[0160] When the intraTMP index indicates the first position in the candidate list, no longer constructing the candidate list, and performing intra prediction on the current block according to the single-candidate intraTMP mode;

[0161] When the intraTMP index indicates a position other than the first position in the candidate list, the candidate list is reconstructed, and a reference block used by the current block is determined according to the intraTMP index and the candidate list.

[0162] This embodiment takes into account that when the ntraTMP index indicates the first position in the candidate list, the reference block used by the current block can be found by adopting the single-candidate intraTMP mode, so there is no need to construct a candidate list, which can reduce the complexity of decoding.

[0163] In an exemplary embodiment of the present disclosure, the method further includes: decoding an intraTMP multiple candidate flag, and determining whether to allow the use of a multiple-candidate intraTMP mode according to the intraTMP multiple candidate flag, wherein the intraTMP multiple candidate flag is a sequence-level, picture-level, or slice-level flag;

[0164] After determining that the current block uses the intraTMP mode according to the intraTMP mode use flag, the method further includes:

[0165] When it is determined according to the intraTMP multiple candidate flag that the intraTMP mode allowing the use of multiple candidates is allowed, continue decoding the intraTMP index of the current block;

[0166] When it is determined according to the intraTMP multiple candidate flag that the multiple candidate intraTMP mode is not allowed, decoding of the intraTMP index of the current block is skipped, and intra-frame prediction is performed on the current block according to the single candidate intraTMP mode.

[0167] This embodiment uses a high-level intraTMP multi-candidate flag to indicate whether the multi-candidate intraTMP mode is allowed. In this way, when the intraTMP multi-candidate flag indicates that the multi-candidate intraTMP mode is not allowed, when decoding the intraTMP mode using the flag to determine that the current block uses the intraTMP mode, there is no need to decode the intraTMP index, and the current block can be directly intra-frame predicted according to the single-candidate intraTMP mode, which can simplify the processing flow of the decoder.

[0168] In an exemplary embodiment of the present disclosure, decoding the intraTMP index of the current block includes:

[0169] Debinarization of the intraTMP index is achieved by using a parsing method corresponding to variable-length coding, fixed-length coding, truncated unary code, or truncated binary code; or

[0170] Parse the value of the first binary symbol in the intraTMP index. When the value indicates that the intraTMP index adopts variable-length coding or truncated unary code coding, use the parsing method corresponding to the variable-length coding or truncated unary code to achieve debinarization of the intraTMP index. When the value indicates that the intraTMP index adopts fixed-length coding or truncated binary code, use the parsing method corresponding to the fixed-length coding or truncated binary code to achieve debinarization of the intraTMP index.

[0171] This embodiment determines a parsing method for implementing debinarization of an intraTMP index by using the value of the first binary symbol in the intraTMP index. When the intraTMP index can adopt multiple encoding methods, decoding of the intraTMP index can be implemented simply and conveniently.

[0172] In an exemplary embodiment of the present disclosure, before constructing the candidate list, the method further includes: decoding a search step index of an intraTMP mode, the search step index being used to indicate an index of a used search step among a plurality of candidate search step indexes;

[0173] When constructing the candidate list, the reference block template is searched in the first search range according to the search step determined by the search step index.

[0174] This embodiment can use different step lengths to search the intraTMP mode for the current block, which has better adaptability to different images.

[0175] The present disclosure also provides a video encoding method, as shown in FIG12 , comprising:

[0176] Step S310 , when it is determined that the current block allows the use of a multi-candidate intraTMP mode, a candidate list of intraTMP is constructed according to the method described in any embodiment of the present disclosure;

[0177] Step S320, calculating the coding costs when predicting the current block based on the N reference blocks in the candidate list, and taking the smallest coding cost as the coding cost of the multi-candidate intraTMP mode to perform rate-distortion optimization on the current block;

[0178] Step S330 : When it is determined through rate-distortion optimization that the current block uses multiple candidate intraTMP modes for intra prediction, syntax elements related to the current block and the multiple candidate intraTMP modes are encoded.

[0179] The intraTMP mode of this embodiment is a multi-candidate intraTMP mode. During encoding, a candidate list is constructed. When the intraTMP mode is selected for rate-distortion optimization, syntax elements related to the current block and the multi-candidate intraTMP mode are encoded to indicate the reference block used by the current block in this multi-candidate intraTMP mode. Because the candidate list includes multiple reference blocks, the current block may find a reference block with a higher degree of matching during prediction, improving encoding efficiency.

[0180] In an exemplary embodiment of the present disclosure, the encoding of syntax elements related to the current block and multiple candidate intraTMP modes includes:

[0181] Encode the intraTMP mode usage flag of the current block to indicate that the current block uses the intraTMP mode; and

[0182] The intraTMP index of the current block is encoded to indicate the position of the reference block with the smallest encoding cost among the N reference blocks in the candidate list in the candidate list.

[0183] This embodiment encodes the intraTMP mode usage flag and the intraTMP index so that the decoder can determine the reference block used by the current block according to these two syntax elements, and then perform intra-frame prediction on the current block based on the reference block used by the current block.

[0184] In an exemplary embodiment of the present disclosure, encoding the intraTMP index of the current block includes:

[0185] Binarization of the intraTMP index is achieved by variable-length coding, fixed-length coding, truncated unary code, or truncated binary code; or

[0186] When the value of the intraTMP index is in the first value range, the intraTMP index is binarized by variable-length coding or truncated unary code; when the value of the intraTMP index is in the second value range, the intraTMP index is binarized by fixed-length coding or truncated binary code, and the value of the first value range is smaller than the value of the second value range.

[0187] In this embodiment, when encoding, variable-length encoding or truncated unary encoding can be used when the intraTMP index value is small, resulting in shorter encoded words, while fixed-length encoding or truncated binary encoding can be used when the intraTMP index value is large, thereby saving encoding overhead.

[0188] In an exemplary embodiment of the present disclosure, determining that the current block is allowed to use the intraTMP mode with multiple candidates includes: when all conditions for not allowing the use of the intraTMP mode with multiple candidates are not met, determining that the current block is allowed to use the intraTMP mode with multiple candidates, wherein the conditions for not allowing the use of the intraTMP mode with multiple candidates include: an intraTMP multiple candidate flag at the sequence level, image level or slice level indicating that the intraTMP mode with multiple candidates is not allowed.

[0189] In an example of this embodiment, when encoding screen content, the intraTMP multi-candidate flag is set to a value indicating that the intraTMP mode of multiple candidates is not allowed; when encoding video images captured by the camera, the intraTMP multi-candidate flag is set to a value indicating that the intraTMP mode of multiple candidates is allowed.

[0190] This embodiment can be used for different usage scenarios. When it is suitable to use the intraTMP mode with multiple candidates, the intraTMP multiple candidate flag can be encoded to indicate that the intraTMP mode with multiple candidates is allowed to be used, so as to improve the coding effect; when it is not suitable to use the intraTMP mode with multiple candidates, the intraTMP multiple candidate flag can be encoded to indicate that the intraTMP mode with multiple candidates is not allowed to be used, so as to avoid unnecessary coding complexity.

[0191] In an exemplary embodiment of the present disclosure, the method further includes: when performing intra-frame prediction encoding on the current block according to the single-candidate intraTMP mode, encoding an intraTMP mode usage flag for the current block to indicate that the current block uses the intraTMP mode; and encoding an intraTMP index for the current block to indicate that the reference block used by the current block is the first in the candidate list. This embodiment does not require an additional flag to indicate whether the current block uses the single-candidate intraTMP mode or the multiple-candidate intraTMP mode. Instead, the determination is made using the intraTMP index, which can simplify the complexity of the decoding end.

[0192] An embodiment of the present disclosure further provides a multi-candidate intra-frame template matching prediction (intraTMP) method. In this embodiment, N candidates are set for intraTMP, where N≥2, that is, a candidate list of intraTMP with a length of N is set, which is recorded as intraTMPCandList[N].

[0193] The encoding end finds multiple reference block templates within the set search range according to the set search rules, calculates the differences between the multiple reference block templates and the current block template based on the reconstructed pixel values ​​of the multiple reference block templates and the reconstructed pixel value of the current block template, and fills the position identifiers of the reference blocks corresponding to the N reference block templates into the candidate list of intraTMP in order of the differences from small to large.

[0194] The encoder calculates the differences between the N reference blocks in the candidate list and the original pixel values ​​of the current block, and determines the value of the intraTMP index based on the position of the reference block with the smallest difference in the candidate list. The reference block with the smallest difference is the reference block used by the current block in intraTMP mode, which is also the best matching block found. If the BV is used as the position identifier of the reference block in the candidate list, the BV at the position indicated by the intraTMP index in the candidate list can also be called the BV used by the current block in intraTMP mode.

[0195] If the encoding end has undergone rate-distortion optimization and selected the intraTMP mode for the current block among multiple intra-frame prediction modes (that is, it is determined that the current block uses the intraTMP mode), then after encoding the flag indicating that the current block uses the intraTMP, the intraTMP index is continued to be encoded to indicate the position of the reference block used by the current block in the candidate list.

[0196] Correspondingly, the decoding syntax is as follows,

[0197]

[0198] Among them, intraTMPFlag is a flag indicating whether the current block uses the intraTMP mode, and intraTMPIndex is an intraTMP index used to indicate the position of the reference block used by the current block in the candidate list.

[0199] During decoding, if intraTMPFlag is true (such as 1), continue to parse intraTMPIndex. The decoding end constructs the candidate list intraTMPCandList of intraTMP according to the same method, finds the position identifier at the position indicated by intraTMPIndex in intraTMPCandList, and finds the corresponding reference block based on the position identifier. The reconstructed value of the reference block can be used as the prediction value of the current block.

[0200] When constructing intraTMPCandList in this embodiment, each time a BV is searched within the search range, the difference between the reference block template corresponding to the BV and the current block template is calculated. The reference block template is a block of the same shape and size as the current block searched in the reconstruction area. The difference can be SAD, SATD, SSE, etc. When constructing intraTMPCandList, the BVs corresponding to the searched reference block templates can be filled into intraTMPCandList in order of difference from small to large. In other words, the searched reference block templates are sorted in order according to their corresponding differences from small to large, and the reference blocks corresponding to the first N reference block templates are used as the N reference blocks in intraTMPCandList. Alternatively, only the top N candidates with the smallest differences can be maintained, and reference block templates with a ranking exceeding N can be directly discarded, thereby saving computational effort.

[0201] Usually, the blocks corresponding to adjacent BVs are relatively close, especially when BV supports pixel-wise precision, such as 1 / 2, 1 / 4, 1 / 8, 1 / 16 precision, etc. The reference block template corresponding to the pixel-wise BV needs to be obtained by interpolation. When interpolating the intra-frame template (intraTmp), the same filter as the inter-frame interpolation filter can be used, which can reduce the storage of additional filters through multiplexing. A simpler interpolation method can also be used. Inter-frame interpolation uses a 12-tap filter. This embodiment can use fewer filters, such as 8-tap, 4-tap or even 2-tap filters, to reduce the amount of calculation.

[0202] When BV supports pixel-level accuracy, if there is no control and it is only sorted by the difference rows of the reference block template, it is easy to concentrate multiple candidates into a very small range. This embodiment provides the following method to make some control to avoid excessive concentration of candidate BVs in intraTMPCandList.

[0203] The first method is as follows:

[0204] During the search process, the search is not performed sequentially according to each possible BV. For example, the usual search order is from left to right and from top to bottom. Generally speaking, the BV of integer pixels can be searched sequentially. As shown in Figures 13A and 13B, assuming that the BV currently being searched is (x0, y0), the next one is (x0+1, y0), provided that the boundary of the search range has not been reached. The first method is to do a sparse search first. For example, for the BV of integer pixels, if the BV currently being searched is (x0, y0), the next one is (x0+4, y0), provided that the boundary of the search range has not been reached. Template matching is performed every certain number of pixels (i.e., searching for the reference block template and calculating the difference between the searched reference block template and the current block module), and template matching is performed according to the set search step size. The search step size can be a preset value, such as 2, 3, 4, 8, etc. The same process can be performed in the vertical direction.

[0205] First, find N BVs with the smallest corresponding differences. Then, based on the N BVs with the smallest corresponding differences (differences can also be called costs, or distortion costs), re-search within a small local search range for each BV to improve. For example, if the search interval in the x and y directions of the first-level search is 4 pixels, the local search range can be set to 4x4. If the difference between the reference block templates found in the local search range is small, the corresponding BV can replace the BV in the candidate list, and the N candidates in the candidate list are re-sorted. The BVs corresponding to the N reference blocks in the candidate list are thus spaced apart.

[0206] As shown in Figure 13A , a first-level search is performed using a preset step size. The top-left corner of the reference block being searched is indicated by the cross in the figure. After the search, three sorted BVs are found. The top-left corners of the reference blocks corresponding to these three BVs (i.e., the locations indicated by the BVs) are indicated by the cross in Figure 13B . In this example, the horizontal search step size is 4, and the vertical search step size is also 4. During the second-level search, a local search range is determined based on the three sorted BVs. In this example, the search range is a 4x4 rectangular area covering the locations indicated by the three sorted BVs (the cross-shaped cells in the figure) in Figure 13B . If the difference between the reference block templates found within each 4x4 local search range is less than the difference between the reference block templates found within the corresponding sorted BV, the BV corresponding to the newly found reference block template can replace the sorted BV in the candidate list and participate in the sorting of intraTMPCandList. Otherwise, the candidate list is not updated.

[0207] In this embodiment, the sizes of the local search range in the horizontal and vertical directions are exactly the same as the first step search step, which can avoid overlapping of the local search ranges.

[0208] If sub-pixel precision is supported, a third-level search can be performed after the second-level search. The BV used in the third-level search is the sub-pixel BV. For example, based on the position indicated by the integer pixel BV selected in the second-level search, a 1 / 2 pixel search is performed within a pixel range of one pixel above, below, and left. Based on the integer pixel BV selected in the second step, a 1 / 2 pixel search is performed within a pixel range of one pixel above, below, and left. Four BVs are set by offsetting the x coordinate of the integer pixel BV by ±1 / 2 pixels and the y coordinate of the integer pixel BV by ±1 / 2 pixels. In another example, four BVs can be obtained by offsetting both the x and y coordinates by ±1 / 2 pixels. Thus, four or eight BVs can be set to search within the local search range. In other embodiments, the sub-pixel BV can be used in the second-level search, or the sub-pixel BV can be used only in the fourth-level search.

[0209] A local search range is determined based on a BV. The position indicated by the BV can be used as the center point of the local search range, or a point near the center, but is not limited to this. As shown in Figure 13B, the position indicated by the BV used in the first-level search is located at a point near the center of the 4x4 local search range. The coordinates in the local search range can be recorded as (2, 2). However, the position indicated by the BV used in the first-level search can also be used as the point in the lower right corner of the local search range to determine the local search range.

[0210] Constructing a candidate list is a process that must be performed by both the encoding end and the decoding end to ensure that the candidate list obtained by the encoding end is consistent with the candidate list obtained by the decoding end.

[0211] In this embodiment, the number required by intraTMPCandList is N. After the first-level search, N BVs are recorded (written into the candidate list). During the second-level search, N local search ranges are determined based on these N BVs and the search continues. In another embodiment, more BVs can be recorded after the first-level search (in addition to the N BVs written into the candidate list, the remaining BVs can be saved separately), such as recording M BVs, where M>N. For example, M=2N. This allows for more opportunities to improve the search and reduces the areas suitable for the second-level search that are missed due to the first-level sparse search.

[0212] In this embodiment, a maximum number of BVs to be retained for each local search range can be set, that is, a threshold for the maximum number of BVs that can be populated into intraTMPCandList for each local search range. This threshold can be determined based on the length N of the candidate list and the size of the local search range. For example, when N is small, each local search range can be set to retain a larger number of BVs to avoid an excessive concentration of candidate reference blocks. If N is large, that is, the number of candidate reference blocks is large, each local search range can retain more BVs, thereby ensuring a certain degree of precision while ensuring coverage.

[0213] The number of BVs to be retained for each local search range can be determined using one of the following methods:

[0214] Method 1

[0215] Each local search range can only retain at most one BV in intraTMPCandList.

[0216] Method 2

[0217] Each improved region can have any number of BVs retained in intraTMPCandList, that is, there is no limit on the maximum number of BVs retained in each improved search region. If the number of candidates is large enough, this setting can be used to improve the precision.

[0218] Method 3

[0219] Set a number threshold K, and the number of BVs retained in each local search range in intraTMPCandList must be less than or equal to K. In each local search range, you can first determine the K BVs with the smallest differences by sorting, and then try to add these K BVs to intraTMPCandList.

[0220] The encoder and decoder need to perform the same search to ensure that the lists they construct are the same. Generally speaking, a larger search range can search more BVs and increase the probability, but the complexity is also higher. Therefore, a reasonable search range can balance the trade-off between performance and complexity.

[0221] IntraTMP itself is an intra-frame block copying technology. It copies a block of the same size as the current block. This means that the larger the current block, the larger the area to copy, and the smaller the current block, the smaller the area to copy. One method is to set the search range relative to the block size, such as setting the horizontal search range to searchRangeWidth = ratio * width and the vertical search range to searchRangeHeight = ratio * height. Ratio is a multiple, such as 4, 5, or 6. Width is the width of the current block, and height is the height of the current block. However, the search range cannot exceed the available reconstruction area. Considering that current codecs support a minimum 4x4 block, taking a 4x4 block as an example, ignoring the maximum available area, we set the ratio to 5, and its searchRangeWidth and searchRangeHeight to 20. This range is very small. The ideal situation for the intraTMP search is to find textures that duplicate the current block, so a threshold can be set to ensure that the minimum search range is not too small. Specifically, the size of the search range may be set in one of the following ways, where the size of the search range is expressed as a search distance relative to a base point representing the current block position:

[0222] Method 1

[0223] searchRangeWidth=max(ratio*width,thrLowerBoundary)

[0224] searchRangeHeight=max(ratio*height,thrLowerBoundary)

[0225] Where thrLowerBoundary is the lowest search range, such as 64, 128, etc.

[0226] Method 2

[0227] You can also use the following method

[0228] searchRangeWidth=ratio*max(width,thrLowerBoundary)

[0229] searchRangeHeight=ratio*max(height,thrLowerBoundary)

[0230] Where thrLowerBoundary is 16 or 32, etc.

[0231] Among them, width and height are the width and height of the current block respectively, ratio is the set proportional factor in the width direction and height direction; searchRangeWidth and searchRangeHeight are the search distances in the width direction and height direction respectively, and thrLowerBoundary is the minimum search distance in the width direction and height direction.

[0232] Method 3

[0233] This method sets a larger ratio for the small block. For example, if the width or height is less than 16, the corresponding ratio is 10, otherwise, the corresponding ratio is 5.

[0234] The setting of the search range does not depend on multiple candidates. For intraTmp with a single candidate, the search range can also be set using the above method.

[0235] In one embodiment, a high-level control syntax can be set to control the size of the first search step, such as the sps_intraTmp_search_step_idx control in an SPS (sequence parameter set). If sps_intraTmp_search_step_idx is 0, the search step is 3; if sps_intraTmp_search_step_idx is 1, the search step is 4. A larger search step can be set for high-resolution videos, and a smaller search step can be set for low-resolution videos.

[0236] In this embodiment, intraTMPCandList is sorted. According to statistical laws, the candidate at the front has a greater probability of being selected. Variable-length coding can be set for the binarization and debinarization of intraTMPIndex, as follows, or a truncated unary code (TU) can be used.

[0237]

[0238] If the probability of each candidate reference block being selected is similar, a fixed-length truncated binary code can be used to binarize intraTMPCandList. The Bin index in the table above is the index of the binary symbol. A Bin index of 0 indicates the first binary symbol, and a Bin index of 1 indicates the second binary symbol.

[0239] In scenarios where N is relatively large, the probability of the candidate in the front is high, the probability of the candidate in the back is small, and the probability of the candidate in the back is close, the codeword can be short when the intraTMPIndex value is small, and the codeword can be long when the intraTMPIndex value is large. Some candidate reference blocks close to each other in the candidate list can use the same code length. For example:

[0240]

[0241] In this example, N is 15, indexes 3 to 6 use codewords of the same length, and indexes 7 to 14 use codewords of the same length. The x in the above table can be obtained through truncated binary encoding.

[0242] This embodiment can use a high-level control syntax to control whether to use the multi-candidate method. If the multi-candidate method is not used, the existing single-candidate method can be used. An example is to use an SPS (sequence parameter set) flag, such as sps_intra_tmp_multi_cand_enabled_flag. If the value of sps_intra_tmp_multi_cand_enabled_flag is 1, the intraTMP multi-candidate method is used for the current sequence; otherwise, the intraTMP single-candidate method is used for the current sequence.

[0243] The corresponding syntax is as follows

[0244]

[0245]

[0246] One use case for the high-level control syntax is to set sps_intra_tmp_multi_cand_enabled_flag to 1 for camera capture sequences and to 0 for screen content sequences. Of course, you can also use flags in PPS (picture parameter set), picture header, slice header, etc. to achieve image-level or slice-level control.

[0247] The disclosed embodiments can set more candidates for intraTMP to reduce the situation where the best matching block found by template matching is not ideal, thereby improving compression performance.

[0248] An embodiment of the present disclosure also provides a candidate list construction device for intra-frame template matching prediction, as shown in Figure 14, including a processor 71 and a memory 73 storing a computer program, wherein when the processor 71 executes the computer program, it can implement the candidate list construction method for intra-frame template matching prediction as described in any embodiment of the present disclosure.

[0249] An embodiment of the present disclosure further provides a video decoding device, see FIG14 , comprising a processor and a memory storing a computer program, wherein the processor can implement the video decoding method as described in any embodiment of the present disclosure when executing the computer program.

[0250] An embodiment of the present disclosure further provides a video encoding device, see FIG17 , comprising a processor and a memory storing a computer program, wherein the processor can implement the video encoding method as described in any embodiment of the present disclosure when executing the computer program.

[0251] The processor of the above-mentioned embodiment of the present disclosure may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a microprocessor, etc., or other conventional processors; the processor may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), discrete logic or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or other equivalent integrated or discrete logic circuits, or a combination of the above devices. That is, the processor of the above-mentioned embodiment may be any processing device or device combination that implements the various methods, steps and logic block diagrams disclosed in the embodiments of the present disclosure. If the embodiments of the present disclosure are partially implemented in software, the instructions for the software may be stored in a suitable non-volatile computer-readable storage medium, and one or more processors may be used to execute the instructions in hardware to implement the methods of the embodiments of the present disclosure. The term "processor" used herein may refer to the above-mentioned structure or any other structure suitable for implementing the technology described herein.

[0252] An embodiment of the present disclosure further provides a video encoding and decoding system, which includes the video encoding device described in any embodiment of the present disclosure and the video decoding device described in any embodiment of the present disclosure.

[0253] An embodiment of the present disclosure further provides a non-transitory computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, can implement the method described in any embodiment of the present disclosure.

[0254] An embodiment of the present disclosure further provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, the method described in any embodiment of the present disclosure can be implemented.

[0255] In one or more exemplary embodiments above, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a computer-readable medium or transmitted via a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium that facilitates the transfer of a computer program from one place to another, such as according to a communication protocol. In this way, a computer-readable medium may generally correspond to a non-transitory tangible computer-readable storage medium or a communication medium such as a signal or carrier wave. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the technology described in this disclosure. A computer program product may include a computer-readable medium.

[0256] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Furthermore, any connection may also be referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient (transient) media, but rather refer to non-transient tangible storage media. As used herein, disk and optical disk include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, or Blu-ray disc, among others, where disks typically reproduce data magnetically, while optical discs use lasers to reproduce data optically. Combinations of the above should also be included within the scope of computer-readable media.

[0257] In some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Furthermore, the techniques may be fully implemented in one or more circuits or logic elements.

[0258] The technical solutions of the embodiments of the present disclosure can be implemented in a wide variety of devices or equipment, including wireless mobile phones, integrated circuits (ICs), or a group of ICs (e.g., a chipset). Various components, modules, or units are described in the embodiments of the present disclosure to emphasize the functional aspects of the devices configured to perform the described techniques, but they do not necessarily need to be implemented by different hardware units. Instead, as described above, the various units can be combined in the codec-side hardware unit or provided by a collection of interoperable hardware units (including one or more processors as described above) in combination with appropriate software and / or firmware.

Claims

1. A method for constructing a candidate list for intra-frame template matching prediction, comprising: Determine a first search range for intra template matching prediction intraTMP for the current block; Searching for a reference block template according to the first search range, and calculating a difference between the searched reference block template and the current block template, wherein the reference block template corresponds to the reference block one by one; A candidate list of intraTMP is constructed according to the difference, and N reference blocks in the candidate list and the order of the N reference blocks are determined, where N≥2.

2. The method according to claim 1, characterized in that: The first search range is located in the reconstruction area of ​​the current image; The difference of the reference block template relative to the current block template is determined based on the SAD, SATD or MSE between the reconstructed value of the reference block template and the reconstructed value of the current block template.

3. The method according to claim 1, characterized in that: The order of the N reference blocks in the candidate list is determined from small to large according to the difference between the corresponding reference block templates.

4. The method according to claim 1 or 3, characterized in that: The reference block in the candidate list is represented by a block vector BV of the reference block, and the BV of the reference block is used to indicate the position of the reference block relative to the current block. The BV corresponding to the reference block template is the BV of the reference block corresponding to the reference block template.

5. The method according to claim 4, characterized in that: Searching for a reference block template according to the first search range, calculating a difference between the searched reference block template and the current block template, and constructing the candidate list of intraTMP according to the difference, comprising: Determine a group of BVs according to the first search step and the first search range, where the positions indicated by the group of BVs are within the first search range; Searching for a corresponding reference block template according to the group of BVs; The difference between the searched reference block template and the current block template is calculated, and the BVs corresponding to the N reference block templates with the smallest differences are filled into the candidate list.

6. The method according to claim 4, characterized in that: Searching for a reference block template according to the first search range, calculating a difference between the searched reference block template and the current block template, and constructing the candidate list of intraTMP according to the difference, comprising: Determine a group of BVs according to the first search step and the first search range, perform a search according to the group of BVs, calculate the difference between the searched reference block template and the current block template, and fill the BVs corresponding to the N reference block templates with the smallest differences into the candidate list; Determine M second search ranges according to the BVs corresponding to the M reference block templates with the smallest differences found in the first search, determine M groups of BVs according to the second search step and the M second search ranges, search for corresponding reference block templates in the M second search ranges according to the M groups of BVs, calculate the difference between the searched reference block template and the current block template, and update the candidate list according to the difference; wherein, the second search step is smaller than the first search step, the second search range is smaller than the first search range, and the second search ranges do not overlap with each other, and M≥N.

7. The method according to claim 6, characterized in that: The BV is represented by a position offset relative to a base point, where the base point is a point in the current block; The M second search ranges respectively cover the positions indicated by the BVs corresponding to the M reference block templates, and the positions indicated by the BVs are determined according to the base point and the position offset.

8. The method according to claim 6, characterized in that: After updating the candidate list according to the difference, the method further includes: Determine M' third search ranges according to the BVs corresponding to the M' reference block templates with the smallest differences searched for the second time, determine M' groups of BVs according to the third search step and the M' third search ranges; search for corresponding reference block templates according to the M' groups of BVs in the M" third search ranges, calculate the difference between the searched reference block template and the current block template, and update the candidate list according to the difference; wherein the third search step is smaller than the second search step, the third search ranges are smaller than the second search ranges and do not overlap with each other, and M'≥N; Among them, a group of BVs determined for each third search range is BVs of integer pixels; or, a group of BVs determined for each third search range is BVs of sub-pixels, and the reconstruction value of the reference block template corresponding to the BV of the sub-pixel is obtained by interpolation.

9. The method according to claim 6 or 8, characterized in that: The updating process of the candidate list according to the difference includes: Determine the smallest difference d1 among the differences of the reference block templates searched in the same local search range. <D N In the case of , the candidate list is updated, the BV corresponding to DN is deleted from the candidate list, and the BV corresponding to d1 is added to the candidate list; Among them, D N It is the largest difference among the differences corresponding to the N BVs in the candidate list before updating, the difference corresponding to the BV refers to the difference of the reference block template corresponding to the BV, and the local search range is the second search range or the third search range.

10. The method according to claim 6 or 8, characterized in that: The updating process of the candidate list according to the difference includes: Determine the smallest K differences among the differences of the reference block templates searched in the same local search range, where at least one of the K differences is smaller than D N In the case of, updating the candidate list, the N BVs in the candidate list after the update are the BVs corresponding to the K differences and the N BVs with the smallest differences among the N BVs in the candidate list before the update, K is a set threshold, K≥2; Among them, D N It is the largest difference among the differences corresponding to the N BVs in the candidate list before updating, the difference corresponding to the BV refers to the difference of the reference block template corresponding to the BV, and the local search range is the second search range or the third search range.

11. The method according to claim 6 or 8, characterized in that: The updating process of the candidate list according to the difference includes: For each reference block template searched, the difference between the reference block template is less than D N In the case of N The corresponding BV is deleted from the candidate list, and the BV corresponding to the reference block template is added to the candidate list; Among them, D N It is the largest difference among the differences corresponding to the N BVs in the candidate list before updating, and the difference corresponding to the BV refers to the difference of the reference block template corresponding to the BV.

12. The method according to claim 10, characterized in that: K is determined according to any one or more of the following parameters: N, the size of the local search range.

13. The method of claim 1, wherein: The size of the first search range is determined according to the size of the current block.

14. The method according to claim 13, characterized in that: Relative to a base point representing the current block position, a first search distance in a width direction and a second search distance in a height direction of the first search range are determined as follows: Calculate the product of the current block width and the first scale factor, and use the larger value of the product and the set minimum search distance in the width direction as the first search distance; calculate the product of the current block height and the second scale factor, and use the larger value of the product and the set minimum search distance in the height direction as the second search distance, wherein the first scale factor and the second scale factor are equal or different; or Determine a larger value between the current block width and a set minimum search distance in the width direction, and use the product of the larger value and a first scale factor as the first search distance; determine a larger value between the current block height and a set minimum search distance in the height direction, and use the product of the larger value and a second scale factor as the second search distance, wherein the first scale factor and the second scale factor are equal or different; or The first search distance is obtained by multiplying the current block width by the corresponding first scale factor, wherein there are multiple first scale factors, and the larger the first scale factor is, the larger the corresponding current block width is; The second search distance is obtained by multiplying the current block height by the corresponding second scale factor. There are multiple second scale factors. The larger the second scale factor is, the larger the corresponding current block height is.

15. A video decoding method, comprising: Decode the intraTMP mode usage flag of the intra-frame template matching prediction of the current block; When determining that the current block uses the intraTMP mode according to the intraTMP mode use flag, continue decoding the intraTMP index of the current block, where the intraTMP index is used to indicate the position of the reference block used by the current block in the intraTMP candidate list; A candidate list is constructed, a reference block used by the current block is determined according to the intraTMP index and the candidate list, and intra-frame prediction is performed on the current block according to the reference block used by the current block.

16. The method of claim 15, wherein: The candidate list is constructed according to the candidate list construction method of intraTMP as described in any one of claims 1 to 14.

17. The method of claim 15, wherein: The method further comprises: decoding an intraTMP multiple candidate flag, and determining whether to allow the use of a multiple candidate intraTMP mode according to the intraTMP multiple candidate flag, wherein the intraTMP multiple candidate flag is a flag of a sequence level, a picture level or a slice level; After determining that the current block uses the intraTMP mode according to the intraTMP mode use flag, the method further includes: When determining that the intraTMP mode that allows the use of multiple candidates is allowed according to the intraTMP multiple candidate flag, continue decoding the intraTMP index of the current block; When it is determined according to the intraTMP multiple candidate flag that the multiple candidate intraTMP mode is not allowed, the decoding of the intraTMP index of the current block is skipped, and the current block is intra predicted according to the single candidate intraTMP mode.

18. The method of claim 15, wherein: After decoding the intraTMP index of the current block, the method further includes: When the intraTMP index indicates the first position in the candidate list, the candidate list is no longer constructed, and intra prediction is performed on the current block according to the single-candidate intraTMP mode; When the intraTMP index indicates a position other than the first position in the candidate list, the candidate list is reconstructed, and a reference block used by the current block is determined according to the intraTMP index and the candidate list.

19. The method of claim 15, wherein: The intraTMP index of the decoding current block includes: Debinarization of the intraTMP index is achieved by using a parsing method corresponding to variable-length coding, fixed-length coding, truncated unary code or truncated binary code; or Parse the value of the first binary symbol in the intraTMP index. When the value is one of 0 and 1, use variable-length coding or a parsing method corresponding to a truncated unary code to debinarize the intraTMP index. When the value is the other of 0 and 1, use fixed-length coding or a parsing method corresponding to a truncated binary code to debinarize the intraTMP index.

20. The method of claim 16, wherein: Before constructing the candidate list, the method further includes: decoding a search step index of the intraTMP mode, the search step index being used to indicate an index of the used search step in a plurality of candidate search step; When constructing the candidate list, the reference block template is searched in the first search range according to the search step determined by the search step index.

21. A video encoding method, comprising: When determining that the current block allows the use of a multi-candidate intra-frame template matching prediction intraTMP mode, constructing a candidate list of intraTMP according to the method of claims 1 to 14, the candidate list includes N reference blocks, N≥2; Calculate the coding cost when predicting the current block according to the N reference blocks in the candidate list, and use the smallest coding cost as the coding cost of the intraTMP mode of multiple candidates to perform rate-distortion optimization of the current block; When it is determined through rate-distortion optimization that the current block uses multiple candidate intraTMP modes for intra-frame prediction, syntax elements related to the current block and the multiple candidate intraTMP modes are encoded.

22. The method of claim 21, wherein: The coding of the syntax elements related to the intraTMP mode of the current block and multiple candidates includes: Encode the intraTMP mode usage flag of the current block to indicate that the current block uses the intraTMP mode; and The intraTMP index of the current block is encoded to indicate the position of the reference block with the smallest encoding cost among the N reference blocks in the candidate list in the candidate list.

23. The method of claim 22, wherein: The intraTMP index of the encoding current block includes: Binarization of the intraTMP index is achieved by variable-length coding, fixed-length coding, truncated unary code or truncated binary code; or When the value of the intraTMP index is in the first value range, the intraTMP index is binarized by variable-length coding or truncated unary code; when the value of the intraTMP index is in the second value range, the intraTMP index is binarized by fixed-length coding or truncated binary code, and the value of the first value range is smaller than the value of the second value range.

24. The method of claim 21, wherein: The determining that the current block is allowed to use the intraTMP mode with multiple candidates includes: when all conditions that do not allow the use of the intraTMP mode with multiple candidates are not met, determining that the current block is allowed to use the intraTMP mode with multiple candidates, wherein the conditions that do not allow the use of the intraTMP mode with multiple candidates include: an intraTMP multiple candidate flag at the sequence level, image level or slice level indicates that the intraTMP mode with multiple candidates is not allowed.

25. The method of claim 24, wherein: When encoding screen content, setting the intraTMP multiple candidate flag to a value indicating an intraTMP mode that does not allow the use of multiple candidates; When encoding a video image captured by a camera, the intraTMP multiple candidate flag is set to a value indicating that the intraTMP mode that allows the use of multiple candidates is allowed.

26. The method of claim 21, wherein: The method also includes: when intra-frame prediction encoding is performed on the current block according to the intraTMP mode of a single candidate, encoding the intraTMP mode usage flag of the current block to indicate that the current block uses the intraTMP mode; and encoding the intraTMP index of the current block to indicate that the reference block used by the current block is at the first position in the candidate list.

27. A code stream, characterized in that The code stream is generated based on the video encoding method according to any one of claims 21 to 26.

28. A candidate list construction device for intra-frame template matching prediction, comprising a processor and a memory storing a computer program, wherein: When the processor executes the computer program, it is capable of implementing the candidate list construction method as described in any one of claims 1 to 14.

29. A video decoding device, comprising a processor and a memory storing a computer program, wherein: When the processor executes the computer program, the video decoding method according to any one of claims 15 to 20 can be implemented.

30. A video encoding device, comprising a processor and a memory storing a computer program, wherein: When the processor executes the computer program, the video encoding method according to any one of claims 21 to 26 can be implemented.

31. A video encoding and decoding system, wherein: It comprises the video encoding device as claimed in claim 30 and the video decoding device as claimed in claim 29.

32. A non-transitory computer-readable storage medium storing a computer program, wherein: The computer program can implement the method according to any one of claims 1 to 26 when executed by a processor.

33. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, it can implement the method according to any one of claims 1 to 26.

Images