Method and apparatus for intra block copy merging and encoder / decoder including the same

Abstract

A method of predicting one or more blocks of a picture is described. The method comprises predicting the one or more blocks of the picture by applying fusion of at least two predictions. The at least two predictions comprise a prediction using Intra Block Copy (IBC) and / or a prediction using Intra Template Matching prediction (IntraTmp).

Description

Cross-references to related applications

[0001] This application is based on and claims priority to European Patent Application No. 24172741.1, filed on 26 April 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure generally relates to the field of encoding / decoding frames, images, or videos, and embodiments of this disclosure relate to improvements in prediction. More specific embodiments of this disclosure relate to an intra-frame prediction mode that applies the fusion of at least two predictions. Background Technology

[0003] The encoding and decoding of images, pictures or videos are performed according to specific standards, such as the Advanced Video Codec (AVC) standard (see reference [1]), the High Efficiency Video Codec (HEVC) standard (see reference [2]), or the Universal Video Codec (VVC) standard (see reference [3]).

[0004] Predictive patterns can be applied to encode and decode images, pictures, or videos. Various predictive patterns with different characteristics are known. For example, one or more predictive patterns can provide fusion that offers certain advantages while relying on stencil textures; while one or more other predictive patterns do not provide fusion but rely on block textures and require signaling.

[0005] Therefore, further improvements are needed for the predictive patterns that will be applied to encoding and decoding images, pictures, or videos. Summary of the Invention

[0006] This disclosure provides a method for predicting one or more blocks of a frame, the method comprising: The one or more blocks of the frame are predicted by applying the fusion of at least two predictions, wherein the at least two predictions include a prediction using Intra-Block Copy (IBC) and / or a prediction using IntraTmp (Intra-Template Matching).

[0007] Optionally, the prediction includes: - Obtain at least one explicit block vector EBV prediction signal. - Calculate at least one implicit block vector (IBV) prediction signal, and - The at least one EVB prediction signal is fused with the at least one IBV prediction signal to obtain at least one fused prediction signal.

[0008] Optionally, obtaining at least one EBV and calculating at least one IBV are performed sequentially or in parallel.

[0009] Optionally, the at least one EBV prediction signal is obtained using the IBC process.

[0010] Optionally, the at least one IBV prediction signal is calculated using the IntraTmp process.

[0011] Optionally, using the IntraTmp procedure to calculate the at least one IBV prediction signal includes: selecting an IntraTmp candidate with the highest similarity to the following as the IBV prediction signal: - The template surrounding the current block, or - The EBV prediction signal.

[0012] Optionally, selecting the IntraTmp candidate with the highest similarity to the templates surrounding the current block as the IBV prediction signal includes: - Select the IntraTmp candidate with the lowest template cost from the IntraTmp candidate set as the IBV prediction signal.

[0013] Optionally, the IntraTmp candidate set is provided by template search.

[0014] Optionally, the IntraTmp candidate set is sorted based on its template cost.

[0015] Optionally, selecting the IntraTmp candidate signal with the highest similarity to the EBV prediction signal as the IBV prediction signal includes: - Perform a search on the reconstructed area of ​​the image, and - For each potential IBV prediction signal, ○ Compare the similarity measure between the EBV prediction signal and the potential IBV prediction signal. ○ Select the IBV prediction signal with the highest similarity.

[0016] Optionally, the similarity measure is based on: - The distortion between the EBV prediction signal and the IBV prediction signal, or - Texture analysis of the EBV prediction signal and the IBV prediction signal.

[0017] Optionally, the distortion between the EBV predicted signal and the IBV predicted signal is determined using the sum of absolute differences (SAD), the sum of squared errors (SSE), or the sum of absolute transform differences (SATD).

[0018] Optionally, the texture analysis of the EBV prediction signal and the IBV prediction signal includes: - Apply horizontal and vertical Sobel filters to each predicted signal. - A virtual intra-prediction mode (VIPM) is derived from each predicted signal using an algorithm that uses decoder-side intra-mode derivation (DIMD). - Compare the VIPM to determine the similarity metric for the potential IBV prediction signal.

[0019] Optionally, using the IntraTmp procedure to calculate the at least one IBV prediction signal includes: - Select a subset of IntraTmp candidates whose template cost is below a certain threshold from the IntraTmp candidate set, and - Select the IntraTmp candidate that has the highest similarity metric to the EBV prediction signal from the IntraTmp candidate subset as the IBV prediction signal.

[0020] Optionally, in response to one or more constraints, the IBV prediction signal may not be used.

[0021] Optionally, the one or more constraints include: the Euclidean distance between the EBV prediction signal and the IBV prediction signal exceeds a threshold.

[0022] Optionally, applying the fusion of the at least two predictions includes: - The at least two predictions are linearly weighted at the block level to generate the at least one fused prediction signal.

[0023] Optionally, the at least two predictions include the EBV prediction signal and the IBV prediction signal, and each pixel value of the at least one fused prediction signal is calculated as follows: , in - The pixel value of the at least one fused prediction signal, -The fusion weights of the IBV prediction signal, -IBV prediction signal, -The fusion weights of the EBV prediction signal, -EBV prediction signal, -i and j are the horizontal and vertical positions of the pixels within the block that will be predicted through fusion.

[0024] Optionally, the fusion weights used for the EBV prediction signal and the fusion weights used for the IBV prediction signal are different, and are constant weights for all blocks.

[0025] Optionally, the fusion weights for the EBV prediction signal and the fusion weights for the IBV prediction signal are adaptively adjusted for each block.

[0026] Optionally, the fusion weights are adaptively adjusted either explicitly or implicitly.

[0027] Optionally, in order to explicitly adaptively adjust the fusion weights, the fusion weights are explicitly signaled.

[0028] Optionally, in order to implicitly and adaptively adjust the fusion weights, the fusion weights are derived using available information.

[0029] Optionally, the available information includes template cost, and the fusion weights are derived as follows: , , in It is based on the following template cost: the template surrounding the EBV prediction signal. The cost and the template around the current block The cost, and It is based on the following template cost: the IBV predicts the surrounding template. The cost and the template around the current block The cost.

[0030] Optionally, applying the fusion of the at least two predictions includes: - Use a linear model LM solver to generate the at least one fused prediction signal.

[0031] Alternatively, by using the EBV to predict the surrounding template and the template surrounding the IBV prediction As input, and using the template around the current block. The objective is to overfit the LM solver, and the LM solver outputs the at least one fused prediction signal. .

[0032] Optionally, the applicability of the fusion of the at least two predictions to a specific block of the image depends on one or more criteria.

[0033] Optionally, the one or more criteria include one or more of the following: - The block is IBC encoded / decoded. - The block has a specific block size. The applicability of IBC is limited by its combined use with other codec tools such as LIC, bidirectional prediction, or CIIP.

[0034] Optionally, the method includes: - Explicitly or implicitly signal the applicability of the fusion of the at least two predictions to a particular block.

[0035] Optionally, explicitly signaling the applicability includes: - For each eligible block, a signal is sent indicating whether the fusion of the at least two predictions is used.

[0036] Optionally, implicitly signaling the applicability includes: - Use the information available on the decoder side to determine whether a compliant block uses the fusion of the at least two predictions.

[0037] Optionally, the at least two predictions include the EBV prediction signal and the IBV prediction signal, and wherein determining the block that meets the conditions includes: - Calculate the cost of the template surrounding the IBV prediction signal. - When the cost of the template around the IBV prediction signal exceeds the cost of the template around the EBV prediction signal, determine to use the fusion of the at least two predictions, and - When the cost of the template around the IBV prediction signal is less than the cost of the template around the EBV prediction signal, it is determined that the fusion of the at least two predictions will not be used.

[0038] Optionally, predicting the one or more blocks of the image by applying the fusion of at least two predictions includes: - Combine primary predictions with secondary predictions, wherein the secondary predictions are implicitly determined.

[0039] Optionally, the primary prediction includes a prediction based on the explicit block vector EBV, and the secondary prediction includes a prediction based on the implicit block vector IBV or another prediction based on EBV.

[0040] Optionally, the master prediction is obtained by using a signaled BV indicator that predicts the EBV for the current block in the bitstream.

[0041] Optionally, the secondary prediction is another EBV-based prediction associated with explicit signaling for another block.

[0042] Optionally, the secondary prediction is selected from a candidate list that includes both EBV-based predictions and IBV-based predictions.

[0043] Optionally, the list includes: - Use for IBC Advanced motion vector prediction The AMVP and merge modes yield N EBV-based candidates; and - M IBV-based candidates obtained using the intraTMP search mechanism.

[0044] Optionally, the method includes: - The list is streamlined to avoid including duplicate predictions or predictions that are the same as the master prediction.

[0045] Optionally, the method includes: - Sort the list from best prediction to worst prediction, and - Use the best prediction from the sorted list as the secondary prediction to be merged with the main prediction.

[0046] Optionally, the list is sorted based on the following: - The template cost associated with the candidate prediction and relative to the template of the current block, or - The similarity between the candidate prediction and the master prediction. In this case, the master prediction is used as a reference, and the cost of the candidate combination list is calculated accordingly, or - The similarity between the virtual intra-frame prediction mode (VIPM) of each candidate prediction and the VIPM of the master prediction.

[0047] This disclosure provides a method for decoding an encoded data stream, the encoded data stream including data representing a picture, the method comprising: Decode the image from the encoded data stream. Decoding the frame includes: predicting one or more blocks of the frame by applying the fusion of at least two predictions, wherein the at least two predictions include a prediction using Intra-Block Copy (IBC) and / or a prediction using Intra-Tmp (Intra-Template Matching).

[0048] This disclosure provides a method for encoding a picture into an encoded data stream, the encoded data stream including data representing the picture, the method comprising: Receive the original image, and The image is encoded into the encoded data stream. Encoding the frame includes: predicting one or more blocks of the frame by applying the fusion of at least two predictions, wherein the at least two predictions include a prediction using Intra-Block Copy (IBC) and / or a prediction using IntraTmp (Intra-Template Matching).

[0049] Optionally, the method includes: - The data stream includes a flag that indicates for a block whether the fusion of the at least two predictions is used.

[0050] This disclosure provides a non-transitory computer-readable storage medium including instructions that, when executed by a computer, cause the computer to perform the method described in any one of the embodiments of this disclosure.

[0051] This disclosure provides an apparatus for predicting one or more blocks of a frame, the apparatus comprising: The prediction module is configured to predict one or more blocks of the frame by applying the fusion of at least two predictions, wherein the at least two predictions include a prediction using Intra-Block Copy (IBC) and / or a prediction using IntraTmp (Intra-Template Matching).

[0052] This disclosure provides an apparatus for decoding an encoded data stream, the encoded data stream including data representing a picture, the apparatus comprising: A decoder module is configured to decode the image from the encoded data stream; and A prediction module, the prediction module including the apparatus according to embodiments of the present disclosure.

[0053] This disclosure provides an apparatus for encoding a picture into an encoded data stream, the encoded data stream including data representing the picture, the apparatus comprising: An encoder module is configured to receive a raw image and encode the image into the encoded data stream; and A prediction module, the prediction module including the apparatus according to embodiments of the present disclosure.

[0054] This disclosure provides a data stream in which a picture is encoded, wherein the data stream includes: A flag indicating whether the one or more blocks of the frame are predicted on the decoder side by applying the fusion of at least two predictions, wherein the at least two predictions include a prediction using Intra-Block Copy (IBC) and / or a prediction using IntraTmp (Intra-Template Matching).

[0055] The technical solutions provided according to the embodiments of this disclosure have the following beneficial effects.

[0056] In existing techniques, multiple prediction modes can be used to predict frames, such as using prediction modules at the encoder and / or decoder. Each prediction mode offers its own advantages, which may come at the cost of certain disadvantages. For example, IntraTmp prediction provides fusion, which is advantageous because it can induce, for example, smoothing or denoising. However, IntraTmp is heavily dependent on the template texture, which may be unrelated to the texture of the currently processed block or CU. On the other hand, IntraBlockCopy (IBC) prediction does not provide fusion but relies on the block texture itself, at the cost of signaling for Block Vector Difference (BVD) and / or block vector identifiers. Therefore, each available prediction mode has its own advantages and disadvantages, and using only one prediction mode may not be suitable for obtaining the desired decoding results.

[0057] Embodiments of this disclosure address the aforementioned drawbacks by enabling an intra-prediction mode that applies the fusion of at least two predictions, wherein at least one prediction uses the Intra-Block Copy (IBC) method, and at least one prediction uses IntraTmp prediction with Intra-Stencil Matching (IntraTmp). According to an embodiment, this codec mode is referred to as IbcFusion.

[0058] It should be understood that the descriptions in this section are not intended to identify key or essential features of the embodiments of this disclosure, nor are they intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0059] The accompanying drawings are illustrative and intended to explain this disclosure, and should not be construed as limiting the disclosure to the illustrated embodiments. Figure 1 This is a block diagram of a conventional video encoder; Figure 2 This is a block diagram of a conventional video decoder; Figure 3 (A) to Figure 3 (D) The diagram illustrates the current CTU processing order for Intra-Block Copy (IBC) mode and the available reference samples in the current CTU and the left CTU; Figure 4 The diagram illustrates the intra-template matching search region for IntraTMP mode. Figure 5 The diagram illustrates the use of the IntraTMP block vector for IBC blocks; Figure 6 The illustration is a flowchart of a method for predicting one or more blocks of a screen according to an embodiment of the present disclosure; Figure 7The illustration shows an embodiment of the IBV convergence mode that sequentially applies the steps of the IBV convergence mode according to an embodiment of the present disclosure; Figure 8 The illustration shows an embodiment of the IBV convergence mode that applies some steps of the IBV convergence mode in parallel according to an embodiment of the present disclosure; Figure 9 The illustration shows an embodiment of the present disclosure for determining whether to use a potential IBV candidate for the current CU's IbcFusion; Figure 10 The diagram illustrates an adaptive fusion weighting of template costs according to an embodiment of this disclosure; Figure 11 The illustration shows adaptive fusion using an LM solver according to an embodiment of this disclosure; Figure 12 The illustration shows how template costs are compared to determine whether the IbcFusion mode should be used for eligible CUs, according to an embodiment of this disclosure. Figure 13 The figure shows a flowchart of a method for decoding an encoded data stream according to an embodiment of the present disclosure; Figure 14 The figure shows a flowchart of a method for encoding a screen into an encoded data stream according to an embodiment of the present disclosure; Figure 15 The illustration is a block diagram of an apparatus for displaying one or more blocks of a predicted screen according to an embodiment of the present disclosure; Figure 16 The figure shows a block diagram of a decoder for decoding an encoded data stream according to an embodiment of the present disclosure; Figure 17 The figure shows a block diagram of an encoder for encoding a picture into an encoded data stream according to an embodiment of the present disclosure; Figure 18 The diagram illustrates a data flow according to an embodiment of this disclosure; and Figure 19 The figure shows a block diagram of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0060] The illustrative embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding, and these details should be considered as exemplary only. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and constructions are omitted in the following description.

[0061] In this disclosure, the term “and / or” is intended to cover all possible combinations and subcombinations of the listed elements, including any single element, any subcombination, or all elements listed, and does not necessarily exclude additional elements.

[0062] In this disclosure, the phrase “...or at least one of ...” is intended to cover any one or more of the listed elements, including any single element, any sub-combination, or all of the listed elements, without necessarily excluding any additional elements, and without necessarily requiring all elements.

[0063] In this disclosure, the term "coding" refers to "encoding" or "decoding," as will become clear from the context of the described embodiments. Similarly, the term "coder" refers to "encoder" or "decoder."

[0064] Figure 1 The diagram illustrates a block diagram of a standard video compression system 100 operating according to the VVC standard. The standard video encoder 100 compresses and encodes frames 102 of a video sequence. The frames 102 to be encoded are divided into blocks 104, also referred to as codec units (CUs). The encoder 100 includes a precoding filter 106 and a prediction residual signal former 108, which generates a prediction residual signal 110 to measure the deviation of the prediction signal 112 relative to the signal 114 output by the filter 106. The encoder also includes a converter 116 and a quantizer 118, and uses an entropy encoder 122 to provide an output bitstream or data stream 120. In addition, encoder 100 includes prediction stage 124 for determining prediction signal 112, which includes dequantizer or inverse quantizer 126, inverse transformer 128, combiner 130, in-loop filter 134, frame buffer 136, motion estimator 138, intra-frame / inter-frame mode selector 140, inter-frame predictor 142 and intra-frame predictor 144.

[0065] Reference Figure 1The video encoder 100 described below compresses and encodes frames 102 of a video sequence as follows. The frames 102 to be encoded are divided into blocks or CUs 104. Each encoding / decoding unit 104 is encoded using either an intra-frame encoding / decoding mode or an inter-frame encoding / decoding mode. When a CU is encoded in intra-frame mode, intra-frame prediction is performed by an intra-frame predictor 144. The intra-frame prediction involves predicting the current CU 114 being encoded using encoded, decoded, and reconstructed frame samples located around the current CU (e.g., above and to the left of the current CU). The intra-frame prediction is performed in the spatial domain. In inter-frame mode, motion compensation and motion estimation are performed by a motion estimator 138, which searches for a CU that serves as a good predictor for the current CU among one or more reference frames provided by a frame buffer 136 and used for predictive encoding / decoding of the current frame. For example, a good predictor for the current CU is a predictor similar to the current CU, i.e., one with low distortion between the two CUs or below a certain threshold. The motion estimation may also consider signaling the rate cost to the predictor to optimize the rate distortion tradeoff. The output of the motion estimation step is one or more motion vectors and reference indices associated with the current CU. The motion compensation then predicts the current CU using the one or more motion vectors and reference frame indices determined by the motion estimator 138. Essentially, the block or CU contained in the selected reference frame, pointed to by the determined motion vector, is used as the prediction block for the current CU. The encoder 100 selects one of the intra-frame or inter-frame coding modes for encoding the CU using the selector 140, and indicates the intra / inter-frame decision, for example, using a prediction mode flag. The prediction residual 110 is then transformed and quantized by blocks 116 and 118, and the quantized transform coefficients, along with the motion vectors and other syntax elements, are entropy-encoded and written into the output bitstream 120. The encoder 100 may skip transform stage 116 and, in a so-called transform-skipped coding mode, apply the quantization directly to the untransformed residual signal 110. After a block or CU has been encoded, the encoder decodes and reconstructs it to obtain a reconstructed signal 110''', which can be used as reference data for predicting future CUs or blocks to be encoded. The quantized transform coefficients 110' are dequantized and inversely transformed to obtain the decoded predicted CU or block 110''. Then, the decoded prediction residual and the predicted block are combined (typically summed) at 130 to provide the reconstructed block or CU 110'''. An in-loop filter 134 is applied to the reconstructed frame to reduce compensation artifacts. For example, a deblocking filter, a Sample Adaptive Offset (SAO) filter, and an Adaptive Loop Filter (ALF) can be applied to reduce coding artifacts. The filtered frame is stored in a buffer 136 (also known as the Decoded Frame Buffer (DPB)) so that it can be used as a reference frame for encoding and decoding subsequent frames.

[0066] Figure 2 This is a block diagram of a video decoder 150, which predictively decodes frames or video from a data stream or bitstream 152, the frames or video being provided at the output 154 of the decoder 150. The decoder 150 includes an entropy decoder 156, a segmentation block 158, an inverse quantizer 160, an inverse transformer 162, a combiner 164, an in-loop filter 166, an optional post-decoding processor 168, and a prediction module 170. The prediction module 170 includes a decoded frame buffer 180, a motion compensator 182, and an intra-frame predictor 184.

[0067] The encoded frames of the video sequence are decompressed and decoded by decoder 150 as follows. The input bitstream 152 is entropy-decoded by decoder 156, which provides information such as block segmentation, the encoding / decoding mode of each encoding / decoding unit, the transform coefficients contained in each transform block, prediction information such as intra-frame prediction mode, motion vectors, reference frame index, and other encoding / decoding information. The block segmentation information indicates how the frame is segmented, and decoder 150 can divide the input frame into codec tree units (CTUs), typically 64×64 or 128×128 pixels in size, and further divide each CTU into rectangular or square codec units (CUs) based on the decoded segmentation information. The entropy-decoded quantization coefficients 172 are dequantized 160 and inversely transformed 162 to obtain the decoded residual frame or CU 174. The decoded prediction parameters are used to predict the current block or CU, i.e., whether the block is to be obtained through its intra-frame prediction or its motion-compensated temporal prediction. The prediction process performed on the decoder side is the same as the prediction process performed on the encoder side. The decoded residual block 174 is added to the prediction block 176 to produce a reconstructed current image block 1768. An in-loop filter 166 is applied to the reconstructed frame or image, which is also stored in the decoded frame buffer 180 to serve as a reference frame for future decoding. As described above, the decoded frame can also undergo post-decoding processing, such as performing inverse color transformations, for example, a conversion from YCbCr 4:2:0 to RGB 4:4:4.

[0068] In all the above processes, the entropy encoding and decoding (decoding) of syntax elements representing encoder decisions (such as block segmentation information, prediction modes / parameters, quantized transform coefficients, etc.) can be performed using a context-adaptive entropy codec (such as context-adaptive binary arithmetic codec (CABAC)). To use CABAC, each syntax element is first binarized to be represented by a series of binary bits (bins). Each binary bit is then associated with a CABAC context model that keeps track of past binary values ​​for that particular bit in order to more effectively model its probability distribution.

[0069] To encode and decode images, pictures, or videos in the manner described above, intra-frame prediction modes can be applied, such as the Intra-Block Copy (IBC) tool or the Intra-Template Match Prediction (IntraTMP) tool.

[0070] Intra-block copy (IBC) Intra-Block Copy (IBC) is a tool used in HEVC extensions related to SCC (Screen Content Codec Extension). IBC significantly improves the encoding and decoding efficiency of screen content footage. Since IBC mode is implemented as a block-level encoding and decoding mode, block matching (BM) is performed at the encoder to find the optimal block vector (or motion vector) for each CU. Here, the block vector is used to indicate the displacement from the current block to a reference block that has been reconstructed within the current frame. When combined with AMVR (Adaptive Motion Vector Resolution), IBC mode can switch between 1-pixel and 4-pixel motion vector precision. CUs encoded with IBC are treated as a third prediction mode in addition to intra-prediction mode or inter-prediction mode. IBC mode is suitable for CUs with a width and height of 64 luminance samples or less.

[0071] On the encoder side, hash-based motion estimation is performed for IBC. The encoder performs RD (rate-distortion) checks on blocks with a width or height no greater than 16 luminance samples. For non-merging modes, a block vector search is first performed using a hash-based search. If the hash search does not return valid candidates, a local search based on block matching is performed.

[0072] In hash-based search, hash key matching (32-bit CRC) between the current block and reference blocks is extended to all allowed block sizes. Hash key calculation for each position in the current frame is based on 4×4 sub-blocks. For larger current blocks, a hash key match with the reference block is determined when all hash keys of all 4×4 sub-blocks match the hash key at the corresponding reference position. If multiple reference blocks are found to match the current block's hash key, the block vector cost of each matching reference is calculated, and the one with the lowest cost is selected.

[0073] In block matching search, the search scope is set to cover both the previous CTU and the current CTU.

[0074] At the CU level, IBC mode is signaled using flags, and it can be signaled as either IBCAMVP (Advanced Motion Vector Prediction) mode or IBC skip / merge mode as follows: - IBC Skip / Merge Mode: The merge candidate index indicates which block vector from the list of neighboring candidate IBC codec blocks is used to predict the current block. The merge list consists of spatial candidates, HMVP candidates, and paired candidates; - IBC AMVP Mode: Block vector differences are encoded and decoded in the same way as motion vector differences. The block vector prediction method uses two candidates as predictors, one from the left neighbor and one from the top neighbor (if IBC encoding / decoding is used). When either neighbor is unavailable, a default block vector is used as the predictor. A signal flag is used to indicate the block vector predictor index.

[0075] To reduce memory consumption and decoder complexity, IBC in VVC only allows the reconstruction of predefined regions, which include the current CTU region and some regions of the left CTU. Figure 3 (A) to Figure 3 (D) The diagram illustrates the current CTU 300 processing order and its available reference samples in the current CTU 300 and the left CTU 302. More specifically, it illustrates the reference area of ​​the IBC mode, where each block represents a 64×64 lumen sample unit. Depending on the current codec CU's position within the current CTU 300, the following applies: - If the current block CU falls within the top-left 64×64 block of the current CTU 300 (see...) Figure 3 (A)) In addition to the reconstructed samples in the current CTU 300, it can also use CPR mode to reference the reference samples in the lower right 64×64 block of the left CTU 302. The current block CU can also use CPR mode to reference the reference samples in the lower left 64×64 block of the left CTU 302 and the reference samples in the upper right 64×64 block of the left CTU (302); - If the current block CU falls within the top right 64×64 block of the current CTU 300 (see...) Figure 3(B) If, in addition to the reconstructed samples in the current CTU 300, the brightness position (0, 64) relative to the current CTU (300) has not yet been reconstructed, the current block CU can also use the CPR mode to refer to the reference samples in the lower left 64×64 block and the lower right 64×64 block of the left CTU 302; otherwise, the current block CU can also refer to the reference samples in the lower right 64×64 block of the left CTU 302. - If the current block CU falls within the bottom left 64×64 block of the current CTU 300 (see...) Figure 3 (C) If, in addition to the reconstructed samples in the current CTU 300, the brightness position (64, 0) relative to the current CTU 300 has not yet been reconstructed, the current block CU can also use CPR mode to reference the reference samples in the upper right 64×64 block and the lower right 64×64 block of the left CTU 302. Otherwise, the current block CU can also use CPR mode to reference the reference samples in the lower right 64×64 block of the left CTU 300; - If the current block CU falls within the bottom right 64×64 block of the current CTU 300 (see...) Figure 3 (D)) can only use the CPR mode to refer to the reconstructed sample in the current CTU 300.

[0076] This restriction allows for the use of local on-chip memory to implement the IBC mode, depending on the hardware implementation.

[0077] Intra-template matching prediction (IntraTMP) Intra-Template Matching Prediction (IntraTMP) is a special intra-prediction mode that copies the best prediction block from the reconstructed portion of the current frame, whose L-shaped template matches the current template. For a predefined search range, the encoder searches the reconstructed portion of the current frame for the template most similar to the current template and uses the corresponding block as the prediction block. The encoder then signals the use of this mode and performs the same prediction operation on the decoder side.

[0078] By comparing the L-shape of the current block with, or only the top or only the left causal neighbors... Figure 4 The predicted signal is generated by matching another block within the predefined search region illustrated in the diagram. There are 6 predefined search regions, namely... Figure 4 R1 to R6 in the current block contain reconstruction samples from CTUs 304 to 308 above and CTU 302 to the left, as well as partial reconstruction samples located above, to the left, to the lower left, and to the upper right of the current block CU within the current CTU 300.

[0079] IntraTMP employs an implicit merging mode, where merge candidates are considered without signaling merge flags or indices. Specifically, the reference position pointed to by the block vectors of all adjacent and non-adjacent merge candidates (encoded in IntraTMP or IBC mode) is used as an additional candidate beyond the default search area. The same template matching cost is used to compare the merge position with the default position. For bidirectional IBC merge candidates, two candidates corresponding to each reference frame are retained. Similarly, for IntraTMP, two candidates are considered, corresponding to the best candidate obtained through template search and the already encoded candidate.

[0080] The sum of absolute differences (SAD) is used as the cost function.

[0081] Utilizing a given search order across the six regions, such as R4, R5, R6, R1, R2, and R3, the decoder constructs a candidate list of up to "19" template-matching block vectors within each region, ordered in ascending order based on template cost (SAD). The following modes are supported: 1 - Single predictor: Select a single predictor from the candidate list; 2 - Fusion of Multiple Predictors: Multiple predictors are combined to derive the final prediction block. The fusion weights are calculated based on the template matching cost of each predictor, or using a weight derivation method based on Wiener filters; 3 - Subpixel precision: When using a single predictor, subpixel precision can be used, namely 1 / 2 pixel precision, 1 / 4 pixel precision and 3 / 4 pixel precision, each with 8 possible directions; 4 - Linear Filter Model: A linear filter can be learned between the reference template and the current template, and this linear model is applied to the reference block. This mode can be used for a single predictor when sub-pixel precision is not used.

[0082] Additionally, IntraTMP with local illuminance compensation is allowed. Consider the following: 1 - For a given CU, the use of LIC is mutually exclusive with the use of FLM (CCCM-like filtering); 2 - Allows LIC to be used with Fusion in IntraTMP; 3 - For screen content encoding / decoding, the LIC model can be determined using either a top-only template or a left-only template. For camera capture encoding / decoding, only the top-left template is used. 4 - For screen content encoding and decoding, support the Multi-Mode Linear Model (MMLM) in a manner similar to IBC-LIC.

[0083] When LIC is used for a given CU, the IntraTMP search process uses the MRSAD distortion function instead of the SAD distortion function.

[0084] The dimensions of all regions (SearchRange_w, SearchRange_h) are set to be proportional to the block size (BlkW, BlkH) so that each pixel has a fixed number of SAD comparisons. That is: SearchRange_w = min(64,a BlkW), SearchRange_h = min(64,a BlkH), Here, 'a' is a constant that controls the trade-off between gain and complexity. In practice, 'a' can be equal to 5.

[0085] To accelerate the template matching process, the search range of all search regions is subsampled by a factor of 4. After finding the best match, an improvement process is performed. This improvement is accomplished by conducting a second template matching search around the best match with a reduced range.

[0086] For CUs with a width and height of 64 or less, enable the intra-frame template matching tool. This maximum CU size for intra-frame template matching is configurable.

[0087] When the decoder-side intra-mode derivation (DIMD) is not used in the current CU, a special flag is used at the CU level to signal the intra-template matching prediction mode.

[0088] Block vector candidates for IntraTMP derivation for IBC In this method, the block vector (BV) derived from IntraTMP (IntraTMP) prediction is used for IntraBlock Copy (IBC). The stored IntraTMP BVs of neighboring blocks, together with the IBC BVs, are used as spatial BV candidates in the IBC candidate list construction.

[0089] The IntraTMP block vector is stored in the IBC block vector buffer, and the current IBC block can use both the IBC BV and IntraTMP BV of neighboring blocks as BV candidates for the IBC BV candidate list, such as... Figure 5 As shown, Figure 5 The diagram illustrates the use of the IntraTMP block vector for IBC blocks.

[0090] IntraTMP block vectors are added as spatial candidates to the IBC block vector candidate list. For encoding and decoding the IBC block vectors and for history-based motion vector prediction (HMVP), the IntraTMP block vectors are stored at a quarter-pixel resolution.

[0091] Other tools This disclosure relates to the following encoding and decoding tools. For further information, a detailed description of these algorithms is provided in the algorithm description of ECM-12.0 (see reference [5]): • Combined Intra-Inter-Frame Prediction (CIIP) • Decoder-side intra-frame mode derivation (DIMD) • Linear Model (LM) Solver • Illuminance compensation (LIC).

[0092] The current design of the ECM has the following two aspects: 1. IntraTmp has fusion functionality, while IBC does not. Fusion offers advantages such as smoothing and noise reduction. 2. IntraTmp heavily relies on stencil textures, which may be unrelated to block textures (e.g., object boundaries match block boundaries). However, IBC relies on the block texture itself, at the cost of signaling for block vector differences (BVD) and / or block vector identifiers (from a list of block vector candidates).

[0093] The aforementioned drawbacks in the current design of the ECM can be addressed by enabling fusion (e.g., enabling fusion for IBC) in this disclosure. More specifically, this disclosure enables an intra-prediction mode that applies fusion of at least two predictions. The at least two predictions include a prediction using intra-block copying of IBC and a prediction using intra-template matching of IntraTmp.

[0094] Figure 6 The diagram illustrates a flowchart of a method for predicting one or more blocks of an image. The method includes the following: S100: Predict one or more blocks of the frame by applying the fusion of at least two predictions. The at least two predictions include a prediction using Intra-Block Copy (IBC) and a prediction using Intra-Tmp (Intra-Template Matching).

[0095] IbcFusion An embodiment of this disclosure enabling an intra-prediction mode that applies the fusion of at least two predictions will now be described. According to the embodiment, at least one prediction uses the current IBC method, and at least one prediction uses IntraTmp prediction. Hereinafter, this new codec mode is also referred to as IbcFusion. In this new codec mode, at least one BV is signaled, and at least one template matching search is applied.

[0096] On the decoder side, for a given CU using IbcFusion as the encoding / decoding tool, the final predicted signal (i.e., before applying residual encoding / decoding) is calculated by performing the following three steps: 1 - At least one first prediction signal is computed using a block vector (BV) that has already been explicitly signaled in the bitstream. Hereinafter, this prediction signal is also referred to as the explicit BV (EBV) prediction signal; 2 - At least one prediction signal is calculated using a method that does not require signaling any explicit BV in the bitstream. Instead, the prediction is calculated using an encoding / decoding tool that employs implicit BV. Hereinafter, this prediction signal is also referred to as the implicit BV (IBV) prediction signal; 3 - Apply the fusion of the EBV prediction signal and the IBV prediction signal. In the following text, the output of this fusion step is also referred to as the fused prediction signal.

[0097] Since both EVB and IBV may have more than one prediction signal, only one prediction signal is considered for each prediction signal in order to simplify the rest of this specification. However, all aspects can be generalized to more than one prediction signal.

[0098] Figure 7 The illustration shows an embodiment of the IBV convergence mode that sequentially applies the three main steps described above. The method includes the following steps: S200: Obtain the explicit BV (EBV) prediction signal from the bitstream of the EBV in IBC mode; S202: Calculate the implicit BV (IBV) prediction signal; S204: Obtain a fused prediction signal by applying a fusion method to the EBV prediction signal and the IBV prediction signal.

[0099] Figure 8 The illustration shows an embodiment of the IBV convergence mode that applies two of the main steps described above in parallel. The method includes the following steps: S300: Obtain the explicit BV (EBV) prediction signal from the bitstream of the EBV in IBC mode; S302a: Calculate the implicit BV (IBV) prediction signal; S302b: Calculate the explicit BV (EBV) prediction signal; S304: Obtain a fused prediction signal by applying a fusion method to the EBV prediction signal and the IBV prediction signal. As shown in the figure, steps S302a and S302b are executed in parallel.

[0100] Explicit BV Prediction Signal According to an embodiment, the calculation of the prediction signal based on BV follows the process of the IBC algorithm, wherein the signal to BV may be sent in different ways depending on the internal parameters of IBC.

[0101] In some cases where the IBC method explicitly signals more than one BV to a block, there may also be more than one BV-based prediction signal.

[0102] Implicit BV prediction signal According to other embodiments, to calculate the IBV predicted signal, all that is needed is to obtain the IBV and then apply the same process as IBC. That is, use the IBV and copy the region of the reconstructed signal pointed to by the IBV in the current frame.

[0103] According to an embodiment, a first method for calculating the IBV prediction signal can follow the IntraTmp process, in which template costs are calculated at different implicit BV locations. In IntraTmp, the set of potential IBV candidates is provided by template search, and these candidates are typically ranked based on their template costs. The best IntraTmp candidate with the lowest template cost can be used as the IBV.

[0104] According to an embodiment, a second method for obtaining IBV is to perform a search on the reconstructed region and, for each potential IBV, compare the following similarity metrics: 1) Explicit BV prediction signal (provided by IBC's EBV), and 2) The latent implicit BV prediction signal (provided by the latent IBV), Once all potential IBVs have been searched, the one with the highest similarity is selected. Different similarity metrics can be used in this method. One type of metric is based on the distortion between the two predicted signals (e.g., SAD, SSE, SATD, etc.). Another type of metric is based on texture analysis and feature extraction of the two predicted signals. For example, horizontal and vertical Sobel filters can be applied to each predicted signal, and then a so-called Virtual IPM (VIPM) can be derived from each predicted signal using DIMD's algorithm. The two VIPMs are then compared to determine the similarity metric for the potential IBV. Figure 9 This illustrates a schematic example of how to determine whether two potential IBV candidates are used in the IbcFusion algorithm.

[0105] The main difference between the first and second methods is that in the first method, the templates surrounding the current CU are used as a similarity reference, while in the second method, the prediction signal provided by EBV is used as the similarity reference.

[0106] According to an embodiment, a third method for obtaining IBV is to combine the methods described above. In one example, the first method is first used to select a subset of IntraTmp candidates. Then, the optimal IBV candidate is calculated based on the similarity metric of the second method.

[0107] Regardless of which method is used, according to the embodiments, further constraints can be applied to potential IBV candidates to exclude them from the process. Figure 9 The illustration shows an embodiment for determining whether to use potential IBV candidates IBV-1, IBV-2 for the current CU's IbcFusion based on a similarity metric to the predicted EBV signal EBV. For example, the Euclidean distance between EBV and each IBV (IBV-1, IBV-2) can be calculated and compared to thresholds THR1, THR2 (which may be the same or different). If it is greater than the threshold, it can be excluded from the rest of the process.

[0108] Fusion Once both the EBV prediction signal and the IBV prediction signal are calculated, a fusion step is applied to generate the final prediction.

[0109] In all the examples in this section, it is assumed that an EBV prediction signal (denoted as ) will be used. ) and an IBV prediction signal (denoted as ) Furthermore, the output of the fusion step (i.e., the final prediction) is called the fusion, denoted as . .

[0110] One approach to applying the predictive fusion step is to use linear weighting at the block level. In other words, the pixel value of the fused predictive signal is calculated as follows: , in - The pixel value of the at least one fused prediction signal, -The fusion weights of the IBV prediction signal, -IBV prediction signal, -The fusion weights of the EBV prediction signal, -EBV prediction signal, -i and j are the horizontal and vertical positions of the pixels within the block that will be predicted through fusion.

[0111] In the above method, there is a way to obtain the fusion weight. and Different methods. According to an embodiment, a constant weight is applied (e.g., ...). and These constant weights are used for all blocks using the proposed IBC fusion method. Alternatively, according to other embodiments, the fusion weights can be adaptively changed for each block. This adaptation can be explicit or implicit. In explicit adaptation, the fusion weights are explicitly signaled in the bitstream; while in implicit adaptation, the weights are derived using available information.

[0112] According to an embodiment, one method for implicit fusion weight adaptation is to use template cost. Figure 10 The diagram illustrates an adaptive fusion weighting method using template cost according to an embodiment of this disclosure. Let... , and These are templates surrounding the EBV prediction signal, the IBV prediction signal, and the current CU, respectively. Figure 10 As illustrated, the adaptive weighting is based on these template costs, where one EBV prediction signal is provided by the EBV of the IBC mode, and one IBV prediction signal is provided by the IBV of one of the candidates from the IntraTmp mode. Each of these BVs points to a region of the same size as the current CU, which allows their template costs to be calculated. The adaptive weights are calculated proportionally to the template costs, as shown in the diagram. Figure 10 The equation shown is illustrated in the figure.

[0113] According to other embodiments, the fusion step can be applied using a linear model (LM) solver. Figure 11 The illustration shows an adaptive fusion using an LM solver according to an embodiment of this disclosure. Instead of using weights, a predicted signal (i.e., and The LM solver is overfitted by taking the templates surrounding the current CU as input and using the templates surrounding the current CU as the target. Once overfitted (or optimized) on the templates, the solver is ready for fusion of the predicted signals (i.e., optimized). That is, it receives the internal pixel content of the predicted signal as input and outputs the fused prediction. . Figure 11 The left side (to the left of the vertical dashed line) illustrates the process of optimizing the LM solver on the template. Once optimized, Figure 11 The right side (to the right of the vertical dashed line) shows how the LM solver can be used to predict blocks, for example by receiving BV-based and non-BV-based prediction signals as input and providing fused predictions. As output.

[0114] Applicability and Signaling The applicability of IbcFusion to a given CU can depend on various factors. For example, in one embodiment, IbcFusion can be applied to all IBC-encoded (i.e., CUs with the IBC flag set to true) CUs. According to other embodiments, the use of IbcFusion can be limited to certain block sizes of IBC-encoded CUs. Furthermore, according to yet another embodiment, the applicability of the IbcFusion method can be limited by the combined use of IBC with other codec tools such as LIC, bidirectional prediction, or CIIP. For example, in one embodiment, only IBC CUs that do not use LIC are eligible. In other words, if the current block is an IBC block using LIC (in a combined manner), then the block is not eligible for the application of the fusion.

[0115] Given eligible CUs, the use of the IbcFusion method can be explicit or implicit. In explicit use of IbcFusion, a flag called IbcFusionFlag is signaled for each eligible CU. Alternatively, implicit use of the IbcFusion method determines whether to use IbcFusion for each eligible CU based on information available on the decoder side. For example, the cost of the template around the IBV prediction signal can be calculated, and it can be determined whether IBC prediction can provide good predictions. Figure 12 The illustration shows how template costs are compared according to an embodiment of this disclosure to determine whether the IbcFusion mode should be used for a qualifying CU. , and These are templates surrounding the EBV prediction signal, the IBV prediction signal, and the current CU, respectively. Figure 12 The diagram shows a comparison of template costs. , This can implicitly determine whether to use IbcFusion for eligible CUs. More specifically, if template cost... Less than template cost For example, less than the cost of the template If a specific threshold is set, IbcFusion can be used.

[0116] According to the embodiment, the SPS parameter " sps_ibcfusion_enabled_flag To signal each eligible CU to the explicit use of IbcFusion, as shown below:

[0117] According to the embodiment, the codec unit syntax can be modified as follows to use the SPS parameter " intra_ ibcitmpfusion_flag "to indicate the use of OBIC-IntraTMP fusion mode for each CU"

[0118] List of candidate combinations of EBV and IBV According to an embodiment, to obtain the final prediction for a block, prediction fusion can be performed by combining an EBV-based prediction (i.e., the primary prediction) with a secondary prediction, wherein the secondary prediction is implicitly determined and can be either an IBV-based prediction or another EBV-based prediction. In the case of the EBV-based prediction, the EBV is associated with explicit signaling of another block. Therefore, the current block can be easily derived without signaling costs (and thus implicitly for the current CU).

[0119] Therefore, the EBV-based master prediction is obtained by using a signaled BV indicator in the bitstream that predicts the EBV for the current block.

[0120] The secondary prediction can be selected from a candidate list that includes both EBV-based and IBV-based predictions. This list will be referred to as the combined list. For example, N EBV-based candidates can be added to the list using existing AMVP and merge patterns for IBC; and M IBV-based candidates can be added to the list using existing intraTMP search mechanisms. A simplification process can be applied during or after the creation of the combined list to avoid including duplicate predictions or predictions identical to the primary prediction. As previously stated, in this method, the N EBV-based candidates come from other blocks (e.g., neighbors), therefore, their explicit BV signaling does not impose additional rate costs on the current block.

[0121] The candidate combinations are then sorted, for example, from best to worst prediction. The best prediction from the sorted combination list can then be selected and used as the secondary prediction to be merged with the primary prediction.

[0122] The sorting mechanism of the combined list can be implemented in various ways.

[0123] According to an embodiment, the list is sorted based on its template cost relative to the template of the current block.

[0124] According to other embodiments, the candidate combinations can be ranked based on their similarity to the master prediction. In this case, the master prediction is used as a reference, and the cost of the candidate combinations is calculated accordingly.

[0125] In both cases, the cost metric can be any existing distortion metric, such as SAD, SSE, or SATD.

[0126] According to another embodiment, the sorting mechanism for the candidate combination list can be implemented in other ways. For example, a virtual IPM (VIPM) of the candidate can be calculated, and the candidates can be sorted based on the similarity between the VIPM of each candidate and the VIPM of the master prediction.

[0127] Therefore, as described above, embodiments of this disclosure provide or introduce an intra-frame coding / decoding mode called IbcFusion, in which at least one explicit BV (EBV) and at least one implicit BV (IBV) are obtained. Then, by applying fusion to the corresponding prediction signals of the EBV and the IBV, a fused prediction signal to be used for block prediction is obtained.

[0128] According to an embodiment, the EBV is obtained from the tool via explicit BV signaling of the intra-block copy (IBC) tool. In other words, the current block is encoded / decoded as IBC, which requires explicit signaling of the BV.

[0129] According to one embodiment, the IBV is calculated on the decoder side based on texture analysis of reconstructed textures in the same frame. In such an embodiment, a set of potential IBV candidates is analyzed. For each potential candidate, a metric is calculated to determine whether the candidate should be used as an IBV candidate.

[0130] According to an embodiment, the IBV is obtained by invoking the texture analysis algorithm of the IntraTmp tool to provide a set of IBV candidates by calculating the template cost around the current block and potential IBV candidates.

[0131] According to an embodiment, the IBV is obtained by calculating a virtual IPM (VIPM) for prediction signals associated with different potential BV candidates. For each potential BV candidate, the calculated VIPM is compared with the VIPM calculated for the IBV prediction signal.

[0132] According to an embodiment, the IBV is obtained by calculating a distortion measure between a prediction signal associated with different potential BV candidates on the one hand and the EBV prediction signal on the other hand.

[0133] According to an embodiment, the IBV is obtained by combining at least two of the methods described above for obtaining IBV. For example, a list of potential IBV candidates can first be calculated by invoking texture analysis using IntraTmp. Then, the VIPM of the predicted signals of those potential IBV candidates is compared with the VIPM of the predicted IBV signal, and candidates with significant differences are eliminated.

[0134] According to an embodiment, constraints are applied to the difference between the potential IBV and the EBV. For example, if the Euclidean distance between the potential IBV and the EBV is higher than a threshold, it is excluded from the remainder of the IbcFusion process.

[0135] According to an embodiment, the fusion step of the IbcFusion algorithm uses constant weights when fusing the EBV prediction signal and the IBV prediction signal. This means that the same weights are applied to all blocks using IbcFusion.

[0136] According to an embodiment, fusion weights are adaptively determined for each IbcFusion block. This adaptation can be explicit (e.g., signaling the weights) or implicit (e.g., deriving the weights from available information).

[0137] According to the embodiment, the block-level adaptive fusion weights are calculated based on template cost.

[0138] According to an embodiment, the block-level adaptive fusion weights are calculated based on a similarity metric between the IBV prediction signal and the EBV prediction signal.

[0139] According to an embodiment, the fusion step of the IbcFusion algorithm uses at least one LM model solver on the template to determine how the predicted signals are fused.

[0140] According to the implementation example, a subset of all codec blocks meets the criteria for using IbcFusion. For example, certain block sizes are considered to meet the criteria.

[0141] According to an embodiment, for each block that meets the IbcFusion condition, a flag is explicitly signaled in the bitstream to indicate whether IbcFusion is used for that block.

[0142] According to an embodiment, for each block that meets the IbcFusion condition, a flag is implicitly derived based on information available on the decoder side (e.g., the texture of previously encoded / decoded blocks) to indicate whether IbcFusion is used for that block.

[0143] According to an embodiment, the EBV prediction signal is first obtained. Then, to obtain the IBV prediction signal, a list of BV candidates, called a combination list, is calculated. This list may contain IBV candidates and / or EBV candidates whose explicit BV is associated with previously encoded / decoded blocks (therefore, no content needs to be parsed for the current block). This list may be sorted based on the similarity of the predictions to the EBV prediction signal of the current IbcFusion block.

[0144] Figure 13 The diagram illustrates a flowchart of a method for decoding an encoded data stream, the encoded data stream including data representing a frame. The method includes the following: S400: Decode the image from the encoded data stream. Decoding the frame includes: at S402, predicting one or more blocks of the frame by applying the fusion of at least two predictions. The at least two predictions include a prediction using Intra-Block Copy (IBC) and a prediction using Intra-Tmp (Intra-Tmp).

[0145] Figure 14 The diagram illustrates a flowchart of a method for encoding a scene into an encoded data stream. The encoded data stream includes data representing the scene, and the method includes the following: S500: Receives raw video; S502: Encode the image into the encoded data stream. Encoding the frame includes: at S504, predicting one or more blocks of the frame by applying the fusion of at least two predictions. The at least two predictions include a prediction using Intra-Block Copy (IBC) and a prediction using Intra-Tmp (Intra-Template Matching).

[0146] Optionally, the method includes: at S506 including a flag in the data stream, the flag indicating for a block whether the fusion of the at least two predictions is used.

[0147] Other embodiments The inventive concept has been described so far with reference to aspects and embodiments relating to methods for predicting one or more blocks of a frame, methods for decoding, and methods for encoding. According to another embodiment, this disclosure also provides an apparatus for predicting one or more blocks of a frame, and an encoder / decoder including such an apparatus.

[0148] Figure 15 The illustration is a block diagram of an apparatus for predicting one or more blocks of a screen according to an embodiment of the present disclosure. The apparatus 400 includes the following modules: 402: A prediction module configured to operate according to embodiments of the present disclosure to predict one or more blocks of the frame by applying a fusion of at least two predictions. The at least two predictions include a prediction using Intra-Block Copy (IBC) and / or a prediction using IntraTmp (Intra-Template Matching).

[0149] Figure 16 The figure shows a block diagram of a decoder for decoding an encoded data stream according to an embodiment of the present disclosure. The encoded data stream includes data representing a picture. The decoder 500 includes the following modules: 502: Decoder module, configured to decode the image from the encoded data stream; 504: Prediction module, for example Figure 2 The prediction module 170 is depicted in the diagram. Prediction module 502 is configured to operate according to embodiments of this disclosure to predict one or more blocks of the frame by applying a fusion of at least two predictions. The at least two predictions include a prediction using intra-block copy (IBC) and / or a prediction using intra-template matching (IntraTmp). For example, prediction module 502 includes a reference... Figure 15 The device described.

[0150] Figure 17 The figure shows a block diagram of an encoder for encoding a picture into an encoded data stream according to an embodiment of the present disclosure. The encoded data stream includes data representing the picture. The encoder 600 includes the following modules: 602: Encoder module, configured to receive raw images and encode the images into the encoded data stream; 604: Prediction module, for example Figure 1 The prediction module 124 is depicted in the diagram. Prediction module 602 is configured to operate according to embodiments of this disclosure to predict one or more blocks of the frame by applying a fusion of at least two predictions. The at least two predictions include a prediction using intra-block copy (IBC) and / or a prediction using intra-template matching (IntraTmp). For example, prediction module 502 includes a reference... Figure 15 The device described.

[0151] Figure 18 The illustration shows a data stream 700 according to an embodiment of the present disclosure, in which a picture and associated information data are encoded. For example, the data stream or bitstream 700 may be provided by an encoder 600 performing the method of the present invention for encoding the picture into the data stream 700. The data stream 700 is transmitted to a decoder 500 via a wired or wireless transmission medium 704, such as a cable or radio link, and the decoder 500 decodes the picture from the data stream 700. The data stream includes a flag 702 encoded into the data stream by the encoder 600. The flag indicates, for one or more blocks of the picture, whether the one or more blocks of the picture are predicted according to an embodiment of the present disclosure by applying the fusion of at least two predictions on the decoder side. The at least two predictions include a prediction using Intra-Block Copy (IBC) and / or a prediction using IntraTmp (Intra-Template Matching).

[0152] Although some aspects of the disclosed concept have been described in the context of the apparatus, it is clear that these aspects also represent descriptions of the corresponding methods, where a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step also represent descriptions of corresponding blocks, items, or features of the corresponding apparatus.

[0153] Figure 19This is a block diagram illustrating an electronic device 900 according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are described by way of example only and are not intended to limit the implementations of the present disclosure described herein and / or claimed. Device 900 includes a computing unit 901 that performs various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 902 or loaded from storage unit 908 into random access memory (RAM) 903. Various programs and data for storing the operation of device 900 may also be stored in RAM 903. The computing unit 901, ROM 902, and RAM 903 are connected to each other via bus 904. Input / output (I / O) interface 905 is also connected to bus 904.

[0154] Components in device 900 are connected to I / O interface 905, including: input unit 906, such as a keyboard and mouse; output unit 907, such as various types of displays and speakers; storage unit 908, such as a disk and optical disk; and communication unit 909, such as a network interface card (NIC), modem, wireless communication transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices via computer networks such as the Internet and / or various telecommunications networks. Computing unit 901 may consist of various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of computing unit 901 include, but are not limited to: central processing unit (CPU), graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Computing unit 901 performs various methods and processes described above, such as image processing methods. For example, in some embodiments, the image processing methods may be implemented as computer software programs tangibly embodied on a machine-readable medium (such as storage unit 908). In some embodiments, part or all of the computer program may be loaded and / or installed onto the device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by computing unit 901, one or more steps of the image processing method described above may be performed. In some embodiments, computing unit 901 may be configured to execute the image processing method by any other suitable means (e.g., by means of firmware).

[0155] Various implementations of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementation in one or more computer programs executable and / or interpretable on a programmable system comprising at least one programmable processor, wherein the programmable processor may be a dedicated or general-purpose programmable processor capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and capable of transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0156] Program code used to implement the methods of this disclosure can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions and / or operations specified in the flowcharts and / or block diagrams to be performed. The program code can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0157] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include: electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0158] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD)) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0159] The systems and technologies described herein can be implemented in computing systems including back-end components (e.g., as data servers), computing systems including middleware components (e.g., application servers), computing systems including front-end components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with the implementations of the systems and technologies described herein), or computer systems including such back-end components, middleware components, front-end components, or any combination thereof. The components of the system can be interconnected via digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0160] The computer system may include clients and servers. Clients and servers are generally geographically separated and typically interact via a communication network. The client-server relationship is established through computer programs running on the respective computers and having a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in a cloud computing service system, addressing the shortcomings of traditional physical hosts and Virtual Private Servers (VPS) in terms of management difficulty and weak business scalability. The server may also be a server in a distributed system or a server incorporating blockchain technology.

[0161] It should be understood that the steps can be rearranged, added, or deleted using the various forms of processes shown above. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution of this disclosure can be achieved, and this document does not impose any restrictions on this.

[0162] The specific embodiments described above do not limit the scope of protection of this disclosure. Those skilled in the art will understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure or this invention should be included within the scope of protection of this disclosure or this invention.

[0163] References [1] H.264: Advanced video coding for general audiovisual services, https: / / www.itu.int / rec / T-REC-H.264-202108-P / en [2] H.265: High-efficiency video coding, https: / / www.itu.int / rec / T-REC-H.265-202108-P / en [3] H.266: Universal Video Coding, https: / / www.itu.int / rec / T-REC-H.266-202008-I / en [4] AV1 bitstream and decoding process specification, http: / / aomedia.org / av1 / specification / [5] Algorithm description of Enhanced Compression Model 12 (ECM 12), document JVET-AG2025, https: / / jvet-experts.org / doc_end_user / documents / 33_Teleconference / wg11 / JVET-AG2025-v1.zip.

Claims

1. A method for predicting one or more blocks of a frame, the method comprising: The one or more blocks of the frame are predicted by applying the fusion of at least two predictions, wherein the at least two predictions include a prediction using Intra-Block Copy (IBC) and / or a prediction using IntraTmp (Intra-Template Matching).

2. The method of claim 1, wherein the prediction comprises: - Obtain at least one explicit block vector EBV prediction signal. - Calculate at least one implicit block vector (IBV) prediction signal, and - The at least one EVB prediction signal is fused with the at least one IBV prediction signal to obtain at least one fused prediction signal.

3. The method according to claim 1 or 2, The at least one IBV prediction signal is calculated using the IntraTmp process, and The use of the IntraTmp process to calculate the at least one IBV prediction signal includes: The IntraTmp candidate with the highest similarity to the following was selected as the IBV prediction signal: - The template surrounding the current block, or - The EBV prediction signal.

4. The method of claim 3, wherein selecting the IntraTmp candidate with the highest similarity to the EBV prediction signal as the IBV prediction signal comprises: - Perform a search on the reconstructed area of ​​the image, and - For each potential IBV prediction signal, ○ Compare the similarity measure between the EBV prediction signal and the potential IBV prediction signal. ○ Select the IBV prediction signal with the highest similarity.

5. The method according to claim 1 or 2, The at least one IBV prediction signal is calculated using the IntraTmp process, and The use of the IntraTmp process to calculate the at least one IBV prediction signal includes: - Select a subset of IntraTmp candidates whose template cost is below a certain threshold from the IntraTmp candidate set, and - Select the IntraTmp candidate that has the highest similarity metric to the EBV prediction signal from the IntraTmp candidate subset as the IBV prediction signal.

6. The method according to any one of the preceding claims, wherein applying the fusion of the at least two predictions comprises: - The at least two predictions are linearly weighted at the block level to generate the at least one fused prediction signal.

7. The method of claim 6, wherein the at least two predictions include the EBV prediction signal and the IBV prediction signal, and each pixel value of the at least one fused prediction signal is calculated as follows: in - The pixel value of the at least one fused prediction signal, -The fusion weights of the IBV prediction signal, -IBV prediction signal, -The fusion weights of the EBV prediction signal, -EBV prediction signal, -i and j are the horizontal and vertical positions of the pixels within the block that will be predicted through fusion.

8. The method according to claim 6 or 7, The fusion weights used for the EBV prediction signal and the fusion weights used for the IBV prediction signal are respectively... - These are different constant weights, which are applied to all blocks; or - Adaptively adjust for each block, and Optionally, the fusion weights are derived as follows: in -Based on the following template cost: the template surrounding the EBV prediction signal The cost and the template around the current block The cost, and -Based on the following template cost: the IBV predicts the template around it. The cost and the template around the current block The cost.

9. The method according to any one of claims 1 to 5, wherein applying the fusion of the at least two predictions comprises: - Use a linear model LM solver to generate the at least one fused prediction signal.

10. The method according to any one of the preceding claims, The applicability of the fusion of the at least two predictions to a specific block of the image depends on one or more criteria, and Optionally, the one or more criteria include one or more of the following: - The block is IBC encoded / decoded. - The block has a specific block size. The applicability of IBC is limited by its combined use with other codec tools such as LIC, bidirectional prediction, or CIIP.

11. The method according to any one of claims 1 to 10, The method of predicting one or more blocks of the image by applying the fusion of at least two predictions includes: - Combine the primary forecast with the secondary forecast, wherein the secondary forecast is implicitly determined, and The primary prediction includes a prediction based on the explicit block vector EBV, for example, which is obtained by using a signaled BV indicator that predicts the EBV for the current block in the bitstream, and the secondary prediction includes a prediction based on the implicit block vector IBV or another prediction based on EBV.

12. The method according to claim 11, The secondary forecasts are selected from a candidate list that includes both EBV-based forecasts and IBV-based forecasts. The list mentioned includes: - Use for IBC Advanced motion vector prediction The AMVP and merge modes yield N EBV-based candidates; and - M IBV-based candidates obtained using the intraTMP search mechanism.

13. A non-transitory computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform the method according to any one of the preceding claims.

14. An apparatus for predicting one or more blocks of a scene, the apparatus comprising: The prediction module is configured to predict one or more blocks of the frame by applying the fusion of at least two predictions, wherein the at least two predictions include a prediction using Intra-Block Copy (IBC) and / or a prediction using IntraTmp (Intra-Template Matching).

15. A data stream wherein a picture is encoded, wherein the data stream comprises: A flag indicating whether the one or more blocks of the frame are predicted on the decoder side by applying the fusion of at least two predictions, wherein the at least two predictions include a prediction using Intra-Block Copy (IBC) and / or a prediction using IntraTmp (Intra-Template Matching).

Images