Encoding / decoding video picture data

Abstract

The present disclosure relates to intra prediction of a target block of a video picture based on template matching that compares a target template block to a candidate reference template block. The intra-frame prediction comprises the following steps: reconstructing sample availability according to a target block neighborhood and determining a candidate reference block neighborhood according to a geometric transformation type; obtaining a candidate reference template block by applying a geometric transformation identified by a geometric transformation type to the candidate reference block neighborhood; selecting a candidate reference block having a minimum cost obtained by comparing the candidate reference template block with the target template block; and generating an intra prediction block for the target block by applying a geometric transform identified by a geometric transform type associated with the selected candidate reference block to the selected candidate reference block.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of European Patent Application No. 22306422.1 filed on September 27, 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present application generally relates to video picture encoding and decoding. In particular, but not limited thereto, the technical field of the present application relates to prediction of video picture blocks based on intra-frame template matching. Background Art

[0004] This section is intended to introduce the reader to various aspects of the art, which may be related to various aspects of at least one embodiment of the present application described and / or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present application. Therefore, it should be understood that these statements should be read in this light, and not as an admission of the prior art.

[0005] In state-of-the-art video compression systems, such as HEVC (ISO / IEC 23008-2 High Efficiency Video Coding, ITU-T Recommendation H.265, https: / / www.itu.int / rec / T-REC-H.265-202108-P / en) or VVC (ISO / IEC 23090-3 Versatile Video Coding, ITU-T Recommendation H.266, https: / / www.itu.int / rec / T-REC-H.266-202008-I / en), low-level and high-level picture partitioning are provided to divide the video picture into picture regions, so-called Codec Tree Units (CTUs), whose size may typically be between 16x16 and 64x64 pixels for HEVC and between 32x32, 64x64 or 128x128 pixels for VVC.

[0006] The CTU partitioning of a video picture forms a grid composed of CTUs of a fixed size, namely a CTU grid, whose upper and left boundaries spatially coincide with the top and left boundaries of the video picture. The CTU grid represents the spatial partitioning of the video picture.

[0007] In VVC and HEVC, the CTU sizes (CTU width and CTU height) of all CTUs of the CTU grid are equal to the same default CTU size (default CTU width CTU DW and default CTU height CTU DH). For example, the default CTU size (default CTU height, default CTU width) may be equal to 128 (CTU DW = CTU DH = 128). The default CTU size (height, width) is encoded into the bitstream, for example at the sequence level in a sequence parameter set (SPS).

[0008] The spatial position of a CTU in the CTU grid is determined by the CTU address ctuAddr, which defines the spatial position of the upper left corner of the CTU relative to the origin. Figure 1 As shown in , a CTU address may define a spatial position starting from the upper left corner of a higher level spatial structure S that contains the CTU.

[0009] Each CTU is associated with a codec tree to determine the tree partitioning of the CTU.

[0010] like Figure 1 As shown in , in HEVC, the codec tree is a quadtree partition of CTU, where each leaf is called a codec unit (CU). The spatial position of the CU in the video picture is defined by the CU index cuIdx, which indicates the spatial position starting from the upper left corner of the CTU. The CU is spatially partitioned into one or more prediction units (PUs). The spatial position of the PU in the video picture VP is defined by the PU index puIdx, which defines the spatial position starting from the upper left corner of the CTU, and the spatial position of the elements of the partitioned PU is defined by the PU partition index puPartIdx, which defines the spatial position starting from the upper left corner of the PU. Each PU is assigned some intra-frame or inter-frame prediction data.

[0011] The codec mode intra or inter is assigned at the CU level. This means that although the prediction parameters vary from PU to PU, the same intra / inter codec mode is assigned to each PU of a CU.

[0012] According to a quadtree called a transform tree, a CU can also be spatially partitioned into one or more transform units (TUs). A transform unit is a leaf of a transform tree. The spatial position of a TU in a video picture is defined by a TU index tuIdx, which defines the spatial position starting from the upper left corner of the CU. Each TU is assigned some transform parameters. The transform type is assigned at the TU level, and 2D individual transforms are performed at the TU level during encoding or decoding of a picture block.

[0013] Figure 2The existing PU partition types in HEVC are illustrated in Figure 1. They include square partitions (2N×2N and N×N), which are the only partitions used in both intra- and inter-predicted CUs, symmetric non-square partitions (2N×N, N×2N, used only in inter-predicted CUs), and asymmetric partitions (used only in inter-predicted CUs). For example, PU type 2NxnU represents an asymmetric horizontal partition of a PU, where the smaller partition is located at the top of the PU. According to another example, PU type 2NxnL represents an asymmetric horizontal partition of a PU, where the smaller partition is located at the top of the PU.

[0014] like Figure 3 As shown in , in VVC, the codec tree starts from the root node (i.e., CTU). Next, the quadtree (or quadtree) splits the root node into 4 nodes, corresponding to 4 equally sized sub-blocks (solid lines). Next, the quadtree (or quadtree) leaves can be further partitioned by the so-called multi-type tree, which involves Figure 4 The 4 splitting modes shown in Figure 1 are used for binary or ternary splitting. These splitting types are vertical and horizontal binary splitting modes (denoted as SBTV and SBTH) and vertical and horizontal ternary splitting modes SPTTV and STTH.

[0015] In the case of a joint codec tree shared by luma and chroma components, the leaves of the codec tree of a CTU are CUs.

[0016] In contrast to HEVC, in VVC, in most cases, CU, PU and TU have equal sizes, which means that the codec unit is generally not partitioned into PUs or TUs except in some specific codec modes.

[0017] Figure 5 and Figure 6 An overview of video encoding / decoding methods, such as used in current video standard compression systems (such as VVC) is provided.

[0018] Figure 5 A schematic block diagram showing the steps of a method 100 for encoding a video picture VP according to the prior art is shown.

[0019] In step 110, the video picture VP is partitioned into a CTU grid and the partition information data is signaled into the bitstream. Each CTU of the CTU grid is associated with a codec tree, and each CU of the codec tree associated with each CTU is a block of samples of the video picture VP. In short, a CU of a CTU is a block.

[0020] The CTUs of the CTU grid are considered along a scan order, typically a raster scan order of a video picture. Likewise, each block of the CTU is considered along a scan order, typically a raster scan order of blocks of the CTU.

[0021] Each block of each CTU is then encoded using intra or inter prediction codec mode.

[0022] Intra prediction (step 120) consists in predicting the current block with the help of predicted blocks based on coded, decoded and reconstructed samples, which are located around the current block, usually at the top and left of the current block. Intra prediction is performed in the spatial domain.

[0023] In inter prediction mode, motion estimation (step 130) and motion compensation (135) are performed. Motion estimation searches for candidate reference blocks that are good predictors of the current block in one or more reference video pictures used to predictively encode the current video picture. For example, a good predictor of the current block is a predictor that is similar to the current block. The output of the motion estimation step 130 is one or more motion vectors and reference picture indices associated with the current block. Next, motion compensation (step 135) obtains a prediction block with the help of the (one or more) motion vectors and (one or more) reference picture indices determined by the motion estimation step 130. Basically, the block belonging to the selected reference picture and pointed to by the motion vector can be used as the prediction block for the current block. In addition, since the motion vectors are expressed as fractions of integer pixel positions (this is called sub-pixel precision motion vector representation), motion compensation generally involves spatial interpolation of some reconstructed samples of the reference pictures to calculate the prediction block samples.

[0024] The prediction information data is signaled into the bitstream B. The prediction information may include prediction mode, prediction information codec mode, intra prediction mode or motion vector(s) and reference picture index, and any other information for obtaining the same prediction block at the decoding side.

[0025] The method 100 selects one of the intra-frame or inter-frame coding modes by optimizing the rate-distortion tradeoff taking into account the encoding of the calculated prediction residual block (e.g., by subtracting the candidate prediction block from the current block) and the signaling of the prediction information data required to determine the candidate prediction block at the decoding side.

[0026] Typically, the best prediction mode is given as the prediction mode of the best codec mode p* for the current block given by:

[0027]

[0028] Where P is the set of all candidate codec modes for the current block, p represents the candidate codec mode in the set, and RD cost (p) is the rate-distortion cost of candidate codec mode p, usually expressed as:

[0029] RD cost(p)=D(p)+λ.R(p)

[0030] D(p) is the distortion between the current block and the reconstructed block obtained after encoding / decoding the current block with the candidate codec mode p, R(p) is the rate cost associated with encoding and decoding the current block with the codec mode p, and λ is a Lagrangian parameter representing the rate constraint for encoding and decoding the current block and is usually calculated based on the quantization parameter used to encode the current block.

[0031] The current block is usually coded from a prediction residual block PR. More precisely, the prediction residual block PR is calculated, for example, by subtracting the best prediction block from the current block. The prediction residual block PR is then transformed (step 140) using, for example, a DCT (discrete cosine transform) or DST (discrete sine transform) type transform, and the transformed coefficient block obtained is quantized (step 150).

[0032] In a variant, the method 100 may also skip the transform step 140 and directly apply quantization to the prediction residual block PR according to the so-called transform-skip coding mode.

[0033] The block of quantized transform coefficients (or the block of quantized prediction residuals) is entropy encoded into the bitstream B (step 160).

[0034] Next, the quantized transform coefficient block (or quantized residual block) is dequantized (step 170) and inverse transformed (180) (or not) to obtain a decoded prediction residual block. The decoded prediction residual block and the prediction block are then combined (usually summed) to provide a reconstructed block.

[0035] Other information data may also be entropy encoded to encode the current block of the video picture VP.

[0036] A loop filter (step 190) may be applied to the reconstructed picture (including the reconstructed block) to reduce compression artifacts. The loop filter may be applied after reconstructing all picture blocks. For example, they include a deblocking filter, a sample adaptive offset (SAO) or an adaptive loop filter.

[0037] The reconstructed block or the filtered reconstructed block forms a reference picture, which can be stored in a decoded picture buffer (DPB) so that it can be used as a reference picture for encoding the next current block of the video picture VP or as a reference picture for encoding the next video picture to be encoded.

[0038] Figure 6 A schematic block diagram showing the steps of a method 200 for decoding a video picture VP according to the prior art is shown.

[0039] In step 210, partition information data, prediction information data and a quantized transform coefficient block (or a quantized residual block) are obtained by entropy decoding the bitstream B.

[0040] The partition information data defines a CTU grid (layout) on a video picture. The CTU grid splits a video picture VP into multiple CTUs. The CTUs of the CTU grid are considered along a scan order, typically a raster scan order of the video picture VP. Also along a scan order, typically a raster scan order of the blocks of CTUs, the blocks of the considered CTUs are considered.

[0041] Other information data may also be decoded from the bitstream B to decode a current block of a current CTU of the CTU grid.

[0042] In step 220, each current block of the current CTU is entropy decoded.

[0043] Each decoded current block may be a quantized transform coefficient block or a quantized prediction residual block.

[0044] In step 230, the current block (of the current CTU) is dequantized and possibly inverse transformed (step 240) to obtain a decoded prediction residual block.

[0045] On the other hand, the current block is predicted using the prediction information data. The predicted block is obtained by its intra-frame prediction (step 250) or its motion compensated temporal prediction (step 260). The prediction process performed on the decoding side is the same as the prediction process performed on the encoding side.

[0046] Next, the decoded prediction residual block and the prediction block are combined (usually summed), which provides the reconstructed block.

[0047] In step 270, the filter in the loop may be applied to the reconstructed picture (including the reconstructed block), and the reconstructed block or the filtered reconstructed block forms a reference picture, which may be stored in a decoded picture buffer (DPB), as discussed above ( Figure 5 ).

[0048] The intra-frame template matching prediction (ITMP) mode is a special intra-frame prediction mode of the target block (the current block of the video picture to be predicted), which copies the best block (becomes the prediction block) from the reconstructed block of the video picture, and its L-shaped template matches the L-shaped template of the target block. Basically, for a predefined search area (search range), the encoder searches for the L-shaped template of the reconstructed block of the video picture that is most similar to the L-shaped template of the target block, and uses the corresponding block as the best prediction block. The encoder then signals the use of the ITMP mode as prediction information, and performs the same prediction operation on the decoding side.

[0049] In ECM (Algorithm description of Enhanced Compression Model 5 (ECM 5), M. Coban et al., JVET-Z2025, April 2022), ITMP mode is enabled for blocks whose size is less than or equal to a maximum size of 64 in width and height. This maximum size is configurable.

[0050] Figure 7 A block diagram of a method 300 for performing intra-frame template matching prediction (ITMP) on a target block of a video picture according to the prior art is shown.

[0051] Method 300 is implemented in the same way by both method 100 (encoding) and method 200 (decoding), thereby saving signaled data with controlled additional complexity. Only the use of ITMP can be signaled with a single flag carried at the CU level via a dedicated syntax element intra_tmp_flag ( Fig. 20 ). On both the encoding side and the decoding side, the IMTP mode can be tested as an intra-frame predictor candidate for predicting the target block.

[0052] In step 310, the neighborhood of the target block is analyzed to determine the space of a search region R. The search region R generally includes a plurality of available CTUs, and a search region boundary is determined for each available CTU.

[0053] Possibly, the search region R may slightly exceed the CTU boundary if such a CTU is available. For implementation principles, there may be a slight overlap between two different search regions.

[0054] Figure 8 The search area defined according to the prior art for predicting a target block TB of a video picture is shown.

[0055] Typically, the search area includes four spatial regions R1, R2, R3 and R4. Region R1 is the current CTU, which includes the target block TB (block to be predicted of the video picture), region R2 is the left top CTU, R3 is the upper CTU, and R4 is the left CTU.

[0056] The width SearchRange_w and height SearchRange_h of the region R1-R3 are set to be proportional to the width BlkW and the height BlkH of the target block to have a fixed number of comparisons per pixel. That is:

[0057] SearchRange_w=a*BlkW

[0058] SearchRange_h = a * BlkH (2)

[0059] Where 'a' is a constant integer value that controls the gain / complexity tradeoff. The coefficient 'a' may depend on the width and height of the target block, the width of the target template (e.g. fixed at 4 samples), and / or the CTU size. In practice, 'a' is equal to 5.

[0060] In step 320, a target block neighborhood TBN is defined according to the availability of reconstructed samples in the current CTU, which includes the target block TB and surrounding CTUs. The target block neighborhood TBN defines the locations of available reconstructed samples for building a target template patch (TTP), as discussed later.

[0061] For example, Figure 8 As shown in FIG, when the target block TB is flipped vertically or horizontally, the adjacent samples located below or to the right of the target block are unavailable (have not been reconstructed) and cannot be used to generate Fig. 9 The target block neighborhood TBN used in the template matching candidate search.

[0062] If reconstructed samples in the target block's L-shaped neighborhood are available, the target block neighborhood TBN is of the "L-shaped template" type (e.g. Figure 8 As shown in the figure), if only the reconstructed samples in the left part of the target block TB are available (i.e., when the TB reaches the top boundary of the video picture or the search range R, the upper CTU samples are generally not available), the target block neighborhood TBN is the "left template" type, if only the reconstructed samples in the upper part of the target block TB are available (i.e., when the TB reaches the left boundary of the video picture or the search range R, the left CTU samples are generally not available), the target block neighborhood TBN is the "upper template" type, or if no reconstructed samples are available around the target block (usually, when the target block TB reaches the top left corner of the video picture or the search range R), the target block neighborhood TBN is "no template".

[0063] If the target block neighborhood TBN is “no template”, then in step 330 the prediction block used to predict the target block TB is set to a DC (direct coding) value, for example, 1<<(1–bitdepth Y ), where bitdepth Y Indicates the bit depth of luma samples in the video picture.

[0064] Otherwise, in step 340, a candidate reference block RB in the search range R is selected according to the template matching candidate search. The prediction block of the target block is the candidate reference block RB.

[0065] Fig. 9 A block diagram of the template matching candidate search (step 340 ) according to the prior art is schematically illustrated.

[0066] The target template block TTP is associated with the target block TB. The target template block TTP is constructed from the available reconstructed samples in the target block neighborhood TBN. The availability of reconstructed samples in the target block neighborhood TBN depends on the position of the target block TB relative to the video picture or the search area boundary.

[0067] For each current position in the search region R, in step 341, a candidate reference block RB located at the current position is considered and a candidate reference template block RTP is associated with the candidate reference block RB. The candidate reference template RTP is constructed from reconstructed samples in the candidate reference block neighborhood RBN.

[0068] The candidate reference block neighborhood RBN is associated with the candidate reference block RB. The candidate reference block neighborhood RBN defines the position of the reconstructed sample in the search region R in the neighborhood of the candidate reference block RB. The candidate reference block neighborhood RBN is defined based on the target block neighborhood TBN, i.e. the position defined by the candidate reference block neighborhood RBN is defined based on the position of the target block neighborhood TBN. The shape and layout of the candidate reference block neighborhood RBN and the target block neighborhood TBN are the same.

[0069] In step 342 , a cost is associated with the candidate reference block RB to evaluate the match between the reconstructed samples of the target template block TTP and the reconstructed samples of the candidate reference template block RTP.

[0070] If the current cost is not an improvement over the best cost, then another location in the search area is considered (step 341).

[0071] If the current cost improves the best cost, in step 343, the best cost is set equal to the current cost, and the candidate reference block RB is set as the best candidate reference block.

[0072] Step 343 is followed by step 341 until all positions in the search area have been considered.

[0073] For example, the cost is the result of the sum of absolute differences (SAD) between the reconstructed samples of the candidate reference block RTP and the reconstructed samples of the target template block TTP.

[0074] Template matching is an important area in image and video processing. However, due to its inherent complexity, it has been excluded from video standardization activities in the past. With the introduction of template matching into ECM by proposal JVET-U0048 (Evaluation of Template Matching Prediction for VVC, K. Naser et al., JVET-U0048, January 2021; and EE2: Intra Template Matching, K. Naser et al., JVET-V0130, April 2021), template matching has been applied to intra-frame prediction mode encoding and decoding. In fact, ECM has lower requirements in terms of affordability of complexity, because compression performance dominates in this exploration stage before the assumed standardization stage itself.

[0075] The target block neighborhood TBN (or candidate reference block neighborhood RBN) defined in the prior art limits the matching possibilities. It consists of a horizontal (when upper reconstructed samples are available) strip and / or a vertical (when left reconstructed samples are available) strip of 4 samples next to the target block, which is matched with the candidate reference template block (the strip of 4 samples next to the candidate reference block) defined in the search area.

[0076] The technical problem solved by the present disclosure is to improve the compression efficiency and usage of ITMP mode.

[0077] At least one embodiment of the present application is designed in consideration of the above situation. Summary of the invention

[0078] The following section briefly summarizes at least one embodiment to provide a basic understanding of some aspects of the present application. This overview is not an exhaustive overview of the embodiments. Its purpose is not to identify the key or core elements of the embodiments. The following overview only presents some aspects of at least one embodiment in a simplified form as a preface to the more detailed description provided elsewhere in this document.

[0079] According to a first aspect of the present application, a method for intra-frame prediction of a target block of a video picture based on template matching is provided, wherein the template matching compares a target template block including a reconstructed sample in a neighborhood of the target block with a candidate reference template block including a reconstructed sample in a neighborhood of a candidate reference block of a position of a search area, wherein the method comprises:

[0080] -Determine candidate reference block neighborhoods based on the availability of reconstructed samples in the target block neighborhood and based on the type of geometric transformation;

[0081] - obtaining a reconstructed sample of the candidate reference template block by applying a geometric transformation identified by the geometric transformation type to the reconstructed sample in the neighborhood of the candidate reference block;

[0082] - associating a cost with a candidate reference block obtained by comparing reconstructed samples of the candidate reference template block with reconstructed samples of the target template block;

[0083] - selecting a candidate reference block having a minimum cost, said selected candidate reference block being associated with a geometric transformation type identified with a geometric transformation; and

[0084] - generating an intra-prediction block for the target block by applying a geometric transformation identified by a geometric transformation type associated with the selected candidate reference block to reconstructed samples of the selected candidate reference block.

[0085] The present disclosure is advantageous because template matching candidate searches incur very low costs even in cases where the sample pattern represented in the candidate reference template block is symmetric to the target template block. Sample pattern symmetry is likely to occur in computer graphics or screen content. In other words, the present disclosure captures the similarity between the visual content in the neighborhood of the target block and the visual content in the neighborhood of the candidate reference block, even if one of these visual contents is a geometric transformation of the other. The present disclosure can be applicable to both IMTP applied to luma samples of a target block TB or IMTP applied to chroma samples of the target block.

[0086] In one embodiment, the method further comprises adjusting the search area according to the type of geometric transformation and the position of the target block in the video picture.

[0087] In one embodiment, for each geometric transformation type indicating a set of geometric transformations or indicating a combination of at least two of the geometric transformations,

[0088] -Determine the candidate reference block neighborhood according to the availability of reconstruction samples in the target block neighborhood and the current geometric transformation type;

[0089] - for each current position in the search area, obtaining reconstructed samples of the candidate reference template block of the candidate reference block located at the current position in the search area, and associating a cost with the candidate reference block located at the current position in the search area;

[0090] - When all positions in the search area have been considered, consider another type of geometric transformation,

[0091] - When all geometric transformation types have been considered, the selected candidate reference block is the candidate reference block with the minimum cost (450).

[0092] In one embodiment, for each current position in the search area, a current geometric transformation type indicating a geometric transformation of a set of geometric transformations or a combination of at least two of the geometric transformations is considered,

[0093] -Determine the candidate reference block neighborhood according to the availability of reconstruction samples in the target block neighborhood and the current geometric transformation type;

[0094] - obtain reconstructed samples of candidate reference template blocks of candidate reference blocks located at the current position of the search area;

[0095] - associate a cost with a candidate reference block at the current position;

[0096] - When all geometry change types have been considered, consider another current position in the search area;

[0097] - When all positions in the search area have been considered, the selected candidate reference block is the one with the minimum cost.

[0098] In one embodiment, for each current geometric transformation type indicating a set of geometric transformations or indicating a combination of at least two such geometric transformations, considering the current position in the search area,

[0099] -Determine the candidate reference block neighborhood according to the reconstructed sample availability and geometric transformation type of the target block neighborhood;

[0100] - for each current position in the search area, obtaining reconstructed samples of the candidate reference template block of the candidate reference block located at the current position in the search area, and associating a cost with the candidate reference block located at the current position in the search area;

[0101] - When all positions in the search area have been considered, the selected candidate reference block is the one with the minimum cost.

[0102] In one embodiment, the cost associated with the candidate reference block is evaluated by the sum of absolute differences between the transformed and reordered samples of the reference template block and the reconstructed samples of the target template block.

[0103] In one embodiment, the geometric transformation type indicates at least one of the following geometric transformations or a combination of at least two of the following geometric transformations:

[0104] -Flip horizontally;

[0105] -Flip vertically;

[0106] - Rotation;

[0107] -Constant.

[0108] In one embodiment, the rotation angle of the rotation is a multiple of 90°.

[0109] In one embodiment, the method further comprises signaling a geometric transformation type associated with the selected candidate reference block.

[0110] According to a second aspect of the present application, a method for encoding a block of a video picture based on a prediction block of the block is provided, wherein the prediction block is obtained according to any method of the first aspect.

[0111] According to a third aspect of the present application, a method for decoding a block of a video picture based on a prediction block of the block is provided, wherein the prediction block is obtained according to any method of the first aspect.

[0112] According to a fourth aspect of the present application, there is provided a bit stream formatted to include encoded video picture data obtained by the method according to the first aspect.

[0113] According to a fifth aspect of the present application, there is provided an apparatus comprising means for executing one of the methods according to the first, second or third aspect.

[0114] According to a sixth aspect of the present application, there is provided a computer program product comprising instructions, which, when executed by one or more processors, causes the one or more processors to execute the method according to the first, second or third aspect.

[0115] According to a seventh aspect of the present application, there is provided a non-transitory storage medium carrying instructions of a program code for executing a method according to the first, second or third aspect.

[0116] According to an eighth aspect of the present application, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor, wherein the processor is configured to execute one of the methods according to the first, second or third aspect.

[0117] The specific nature of at least one embodiment and other objects, advantages, features and uses of at least one embodiment will become apparent from the following description of examples taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0118] Reference will now be made, by way of example, to the accompanying drawings which illustrate embodiments of the present application, in which:

[0119] Figure 1 An example of a codec tree unit according to HEVC is shown;

[0120] Figure 2 An example of partitioning a codec unit into prediction units according to HEVC is shown;

[0121] Figure 3 An example of CTU partitioning according to VVC is shown;

[0122] Figure 4 An example of splitting modes supported in multi-type tree partitioning according to VVC is shown;

[0123] Figure 5 A schematic block diagram showing the steps of a method 100 for encoding a video picture VP according to the prior art;

[0124] Figure 6 A schematic block diagram showing the steps of a method 200 for decoding a video picture VP according to the prior art;

[0125] Figure 7 A block diagram of a method 300 for performing intra-frame template matching prediction on a target block of a video picture according to the prior art is illustrated;

[0126] Figure 8 shows the search area used in ITMP according to the prior art;

[0127] Fig. 9 Schematically illustrates a block diagram of template matching candidate search (step 340) according to the prior art;

[0128] Fig.10 A block diagram schematically illustrates a method 400 for performing intra-frame prediction on a target block of a video picture based on template matching according to an embodiment;

[0129] Fig.11 and Fig.12 An example of adjusting a search area at a video frame or a search area boundary according to an embodiment is illustrated;

[0130] Fig.13 FIG. 1 illustrates a case where the geometric transformation is a horizontal flip according to an embodiment. Fig.10 Method 400;

[0131] Fig.14 illustrates some examples of neighborhood template determination according to flipping (geometric transformation type) according to an embodiment;

[0132] Fig.15 illustrates some examples of neighborhood template determination according to rotation (geometric transformation type) according to an embodiment;

[0133] Fig.16 Schematically illustrates a block diagram of a method 400 according to an embodiment;

[0134] Fig.17 Schematically illustrates a block diagram of a method 400 according to an embodiment;

[0135] Fig.18 Schematically illustrates a block diagram of a method 400 according to an embodiment;

[0136] Fig.19 illustrates a syntax element structure for signaling a geometry transform type according to an embodiment;

[0137] Fig. 20 A schematic block diagram of an example of a system in which various aspects and embodiments are implemented is illustrated.

[0138] Similar or identical elements are referenced with the same reference numerals. DETAILED DESCRIPTION

[0139] At least one of the embodiments will be described more fully below with reference to the accompanying drawings, wherein an example of at least one of the embodiments is depicted. However, the embodiments may be implemented in a variety of alternative forms and should not be construed as being limited to the examples set forth herein. Thus, it should be understood that the present invention is not intended to limit the embodiments to the disclosed specific forms. On the contrary, the application is intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the application.

[0140] At least one of the aspects generally relates to video picture encoding and decoding, another aspect generally relates to transmitting provided or encoded bitstreams, and another aspect relates to receiving / accessing decoded bitstreams.

[0141] At least one of the embodiments is described for encoding / decoding one video picture, but extends to encoding / decoding multiple video pictures (sequences of pictures) in that each video picture is encoded / decoded in turn as described below.

[0142] In addition, at least one embodiment is not limited to the current version of VVC. At least one embodiment can be applied to VVC and recommended existing or future developments and extensions. Unless otherwise specified or technically excluded, the various aspects described in this application can be used alone or in combination.

[0143] A pixel corresponds to the smallest display unit on the screen, which can be composed of one or more light sources (1 for a monochrome screen and 3 or more for a color screen).

[0144] A video picture, also called a frame or picture frame, comprises at least one component (also called a picture component or channel) determined by a specific picture / video format, which specifies all information related to pixel values ​​and all information that can be used by a display unit and / or any other device to display and / or decode video picture data related to the video picture.

[0145] A video picture comprises at least one component which is usually represented in the form of an array of samples.

[0146] A monochrome video picture includes a single component, while a color video picture may include three components.

[0147] For example, a color video picture may include one luminance (or brightness) component and two chrominance components when the picture / video format is the well-known (Y,Cb,Cr) format, or may include three color components (one for red, one for green, and one for blue) when the picture / video format is the well-known (R,G,B) format.

[0148] Each component of a video picture may comprise a number of samples relative to the number of pixels of the screen on which the video picture is to be displayed. In a variant, the number of samples comprised in a component may be a multiple (or fraction) of the number of samples comprised in another component of the same video picture.

[0149] For example, in case the video format includes one luma component and two chroma components (such as (Y, Cb, Cr) format), the chroma components may contain half the number of samples in width and / or height relative to the luma components, depending on the color format considered.

[0150] A sample is the smallest visual information unit of a component that makes up a video picture. A sample value can be, for example, a brightness or chrominance value, or a color value in (R, G, B) format.

[0151] Reconstructed samples are samples that have been encoded according to method 100 and decoded according to method 200 .

[0152] A pixel value is the value of a pixel of the screen. For a monochrome video picture, a pixel value may be represented by one sample, while for a color video picture, a pixel value may be represented by multiple co-located samples. The co-located samples associated with a pixel refer to samples corresponding to the position of the pixel in the screen.

[0153] A video picture is usually viewed as a set of pixel values, with each pixel represented by at least one sample.

[0154] A block of a video picture is a group of samples of a component of a video picture. When the picture / video format is the well-known (Y, Cb, Cr) format, a block of at least one luma sample or a block of at least one chroma sample may be considered, or when the picture / video format is the well-known (R, G, B) format, a block of at least one color sample may be considered.

[0155] At least one embodiment is not limited to a particular picture / video format.

[0156] In general, the present disclosure relates to a method for intra-frame prediction of a target block of a video picture based on template matching, which template matching compares a target template block including reconstructed samples in a neighborhood of the target block with a candidate reference template block including reconstructed samples in a neighborhood of a candidate reference block of a position located in a search area. The method includes: determining a neighborhood of a candidate reference block according to the availability of reconstructed samples in the neighborhood of the target block and according to a geometric transformation type; obtaining a reconstructed sample of the candidate reference template block by applying a geometric transformation identified by the geometric transformation type to the reconstructed sample in the neighborhood of the candidate reference block; associating a cost with a candidate reference block, the candidate reference block being obtained by comparing the reconstructed sample of the candidate reference template block with the reconstructed sample of the target template block; selecting a candidate reference block with a minimum cost, the selected candidate reference block being associated with a geometric transformation type identified with a geometric transformation; and generating an intra-frame prediction block for the target block by applying a geometric transformation identified by the geometric transformation type associated with the selected candidate reference block to the reconstructed sample of the selected candidate reference block.

[0157] Fig.10 A block diagram of a method 400 for performing intra-frame prediction on a target block TB of a video picture based on template matching according to an embodiment is schematically illustrated.

[0158] The method 400 is based on a template matching candidate search that compares a target template block TTP associated with a target block TB with a candidate reference template block RTP associated with a candidate reference block RB located at a position in a search region R.

[0159] In step 410, the sample availability and the geometric transformation type GTT (see example) of the target block neighborhood reconstruction are determined. Fig.14 and Fig.15 )Determine the candidate reference block neighborhood RBN.

[0160] The geometry transformation type GTT identifies the geometry transformation GT.

[0161] In step 430, a candidate reference template block RTP associated with a candidate reference block RB located at a position in the search area is obtained. The candidate reference template block RTP includes reconstructed samples. The reconstructed samples are obtained by applying a geometric transformation identified by a geometric transformation type GTT on the reconstructed samples in the candidate reference block neighborhood RBN.

[0162] For example, if the target block neighborhood TBN indicates that the upper and left adjacent reconstructed samples of the target block TB are available, the target template block TTP includes the upper and left adjacent reconstructed samples of the target block TB. If the geometric transformation is a horizontal flip, the transformed reconstructed sample positions correspond to the upper and right reconstructed samples in the candidate reference block neighborhood RTP.

[0163] In another example, if the target block neighborhood TBN indicates that the upper and left neighboring reconstructed samples of the target block TB are available, the target template block TTP includes the upper and left neighboring reconstructed samples of the target block TB. If the geometric transformation is a vertical flip, the transformed reconstructed sample positions correspond to the bottom and left reconstructed samples in the candidate reference block neighborhood RTP.

[0164] The reconstructed samples of the target block do not directly correspond to the reconstructed samples of the candidate reference template block RTP (ie are not co-located with respect to the reference and target block neighborhood) because a geometric transformation has been applied on the reconstructed sample positions.

[0165] When the geometric transformation GT is horizontal flip, Fig.13 An example of reordered transformed samples is given in step 3 of .

[0166] In step 440 , a cost is associated with the candidate reference block RB by comparing the reconstructed samples of the candidate reference template block RTP with the reconstructed samples of the target template block TTP.

[0167] In one embodiment, the cost is evaluated by the sum of absolute differences (SAD) between the reconstructed samples of the reference template patch RTP and the reconstructed samples of the target template patch TTP.

[0168] In step 450, a candidate reference block is selected. It corresponds to the candidate reference template block RTP providing the lowest cost. The selected candidate reference block RB is associated with a geometric transformation type GTT, which identifies a geometric transformation GT applied on the reconstructed sample positions.

[0169] In step 460, an intra prediction block for the target block TB is generated by applying a geometric transformation GT identified by a geometric transformation type GTT associated with the selected candidate reference block RB to the reconstructed samples of the selected candidate reference block RB.

[0170] In one embodiment, the method 400 further includes step 420, in which the search area is adjusted according to the geometric transformation type GTT and the position of the target block TB in the video picture.

[0171] Compared with method 300 (ITMP method as defined in the prior art), the boundaries of the search area are adjusted. Therefore, the search area must contain the candidate reference and target template block reconstruction samples. In fact, in the case of vertical and / or horizontal flipping, the candidate reference template block reconstruction samples at the bottom edge and / or right edge of the candidate reference block RB should be available.

[0172] Therefore, in the vertical flip case, the search area must take into account the template strips (eg 4) added to the reference block height when close to the bottom video picture border or close to the target block.

[0173] In case of horizontal flipping, the search area must take into account the strip size added to the right of the reference block width when close to the right video picture boundary or close to the target block.

[0174] Corresponding to the flipping, the candidate reference template region located on the right and / or bottom of the candidate reference block RB should be included in the search region.

[0175] On the contrary, when the flipping is horizontal and / or vertical, since the left and / or top reconstructed samples are not used for the candidate reference block RB, respectively, the reference block position in the search area can move to the left and / or top of the video picture or the boundary of the search area.

[0176] Therefore, the search area boundary is determined according to the target block neighborhood TBN and the type of geometric transformation (especially when flipping), so that the candidate reference template reconstructed samples can be used for comparison with the target template block reconstructed samples.

[0177] Fig.11 and Fig.12 An example of search area adjustment (and the result on possible candidate reference block RB positions) at a video picture or search area border according to an embodiment is illustrated.

[0178] For example, when the target block TB is located at the bottom boundary of the video picture ( Fig.11 ) and the geometric transformation type GTT is vertical and horizontal flip, the search area boundary is adjusted so that the candidate reference block neighborhood RBN includes the lower boundary of the video screen and the left side sample of the candidate reference block RB coincides with the left edge of the video screen. Fig.11 ) and the geometric transformation type GTT is vertical flip, the search area boundary is adjusted so that the candidate reference block neighborhood RBN includes the target block neighborhood TBN row. When the target block TB is located at the top boundary of the video screen ( Fig.12 ) and the geometry type transform (GTT) is horizontally flipped, the search area boundary is adjusted so that the candidate reference block RB can move to the top left edge of the video screen. Fig.12 ) and the geometric transformation type (GTT) is horizontal and vertical flip, the search area boundary is adjusted so that the candidate reference block neighborhood RBN includes at least one sample (e.g., 4 samples) located along the right boundary of the video picture.

[0179] In one embodiment, the geometric transformation type indicates at least one of the following geometric transformations or a combination of at least two of the following geometric transformations:

[0180] -Flip horizontally;

[0181] -Flip vertically;

[0182] - Rotation;

[0183] -Constant.

[0184] In one embodiment, the rotation angle of the rotation is a multiple of 90°.

[0185] In some implementations (with high-end processing capabilities), combining rotation and flip transformations is advantageous because it produces geometric transformations that cannot be generated from pure combinations of flips or rotations, thereby increasing the number of candidates, ie improving the adaptability of signaling.

[0186] In another implementation (with limited memory, such as in an embedded system), combining only flips or only rotations may avoid executing the rotation or flip routines, respectively (e.g. a 180° rotation may be implemented as a combination of a horizontal flip followed by a vertical flip), thereby reducing the implementation footprint of the geometric transformations applied to the reconstructed sample.

[0187] In one embodiment of the method 400 , in step 470 , a geometry transform type GTT associated with the selected candidate reference block RB is signaled into the bitstream B.

[0188] This embodiment increases the bandwidth for transmitting the bit stream B but limits the complexity of the decoder.

[0189] In one embodiment, the geometric transform type GTT associated with the selected candidate reference block RB is not signaled into the bitstream B and the method 400 considers, at both the encoder and the decoder, each of a set of geometric transforms of a set of candidate reference blocks RB located at positions in the search area.

[0190] In one embodiment, the geometry transform type GTT is inherited from the reconstructed block. As an example, if one of the neighboring blocks of the target block TB is coded in ITMP mode, the geometry transform type GTT2 associated with this neighboring block is used as the predictor of the geometry transform type GTT.

[0191] Advantageously, if the geometry transform type GTT2 is not the optimal geometry transform type (determined at the encoding side), the geometry transform GTT is explicitly signaled into the bitstream.

[0192] In one variant, if there are more than one neighboring blocks of the target block TB coded in ITMP mode and their respective geometric transform types are different, the geometric transform type that occurs most often is selected as the geometric transform type GTT predictor.

[0193] In another variant, the index signals the location of a neighboring block coded with ITMP mode and selects the associated geometric transform type.

[0194] These last embodiments limit the bandwidth required to transmit the bit stream B, but increase the complexity of the decoder.

[0195] Fig.13 The method 400 is illustrated when the geometric transformation is a horizontal flip according to an embodiment.

[0196] In step 1, the target block neighborhood is analyzed to determine the space of the search region R, and the available reconstructed samples in the target block neighborhood TBN are identified.

[0197] In the example, only reconstructed samples located in the left neighbourhood of the target block TB are available. The candidate reference block neighbourhood RBN is of the "left template" type (a strip of 4 samples next to the target block TB).

[0198] In step 2, the candidate reference block neighborhood RBN is determined according to the reconstructed sample availability of the target block neighborhood TBN and the geometric transformation type GTT (see Fig.14 ), so that in this example considering the horizontal flip GTT, the candidate reference block neighborhood RBN is located on the right side of the candidate reference block RB.

[0199] In step 3, a geometric transformation GT (here horizontal flip) associated with the geometric transformation type GTT is applied to the reconstructed samples of the candidate reference block neighborhood RBN. Here, this includes a horizontal reordering of the positions of these samples. The candidate reference template block RTP includes the reconstructed samples located at the positions of the reordered reconstructed samples. The output of step 3 produces the samples that constitute the reference template block RTP.

[0200] When horizontal flipping is applied to the samples of the candidate reference block RB, as Fig.13 As shown, the transformed reconstructed samples are reordered according to their distance from the center of the candidate reference block. Fig.13 As shown, 4 reconstructed sample positions with indices 1, 2, 3 and 4 are flipped horizontally, and correspond to transformed reconstructed sample position indices 4, 3, 2 and 1, respectively. When the geometric transformation is a rotation according to a direction (clockwise or counterclockwise), the sample positions in the candidate reference block neighborhood RBN of the candidate reference block RB samples are reordered according to the rotation direction.

[0201] In step 4, for a given target block TB, a cost is associated with each candidate reference block RB by comparing the reconstructed samples of the candidate reference template block RTP with the reconstructed samples of the target template block TTP.

[0202] In step 5, a candidate reference block RB associated with a geometric transformation type GTT is selected. The selected candidate reference block corresponds to the candidate reference block RB associated with the minimum cost. The geometric transformation identified by the geometric transformation type GTT associated with the selected candidate reference block RB is applied to the reconstructed samples of the selected candidate reference block RB. Here, the selected candidate reference block RB is horizontally flipped, and the intra-frame prediction block for the target block TB is the horizontally flipped selected candidate reference block RB.

[0203] Fig. 9 The common template matching candidate search does not check the match between the target template block and the candidate reference template block whose reconstructed samples are geometrically transformed. For example, if the target block neighborhood TBN indicates "above template", only the available reconstructed samples located above the target block TB (i.e., the target template block TTP) are compared with the reconstructed samples located above the candidate reference block RB (i.e., the reference template block RTP). Instead, according to the method 400, the reconstructed samples located above the candidate reference RB (reference template block RPT) are geometrically transformed (i.e., reordered) to check whether the transformed versions of these reconstructed samples match the reconstructed samples of the target template block.

[0204] Fig.14 Some examples of neighborhood template determination according to flipping (geometric transformation type) according to embodiments are illustrated.

[0205] When the upper and left neighbors of the target block TB are available and the geometric transformation GT is the identity function, horizontal flipping, vertical flipping, or vertical and horizontal flipping, the candidate reference block neighborhood RBN indicates an "L-shaped template" whose layout for the candidate reference block RB depends on the flipping mode.

[0206] When only upper neighbor samples of the target block TB are available and the geometric transformation is an identity function or a horizontal flip, the candidate reference block neighborhood RBN indicates an “upper template” whose layout for the candidate reference block RB depends on the flip mode.

[0207] When only upper neighbor samples of the target block TB are available and the geometric transformation is vertical flipping or horizontal and vertical flipping, the candidate reference block neighborhood RBN indicates a "bottom template" whose layout for the candidate reference block RB depends on the flipping mode.

[0208] When only left neighbor samples of the target block TB are available and the geometric transformation is the identity function or vertical flipping, the candidate reference block neighborhood RBN indicates a "left template" whose layout for the candidate reference block RB depends on the flipping mode.

[0209] When only left neighbor samples of the target block TB are available and the geometric transformation is horizontal flipping or horizontal and vertical flipping, the candidate reference block neighborhood RBN indicates a "right template" whose layout for the candidate reference block RB depends on the flipping mode.

[0210] Fig.15 Some examples of neighborhood template determination according to rotation (geometric transformation type) according to embodiments are illustrated.

[0211] When the upper and left neighboring samples of the target block TB are available, the candidate reference block neighborhood RBN indicates an “L-shaped template”. The “L-shaped template” layout also depends on the degree of rotation.

[0212] When only the upper neighborhood samples of the target block TB are available, if the rotation degree is equal to 0, the candidate reference block neighborhood RBN is the "upper template"; if the rotation degree is equal to 90, the candidate reference block neighborhood RBN is the "right template"; if the rotation degree is equal to 180, the candidate reference block neighborhood RBN is the "bottom template"; if the rotation degree is equal to 270, the candidate reference block neighborhood RBN is the "left template".

[0213] When only the left neighborhood samples of the target block TB are available, if the rotation degree is equal to 0, the candidate reference block neighborhood RBN is the "left template"; if the rotation degree is equal to 90, the candidate reference block neighborhood RBN is the "upper template"; if the rotation degree is equal to 180; the candidate reference block neighborhood RBN is the "right template"; if the rotation degree is equal to 270, the candidate reference block neighborhood RBN is the "bottom template".

[0214] Fig.16 A block diagram of a method 400 according to an embodiment is schematically illustrated.

[0215] This embodiment applies only to method 100 (encoding).

[0216] For each geometry transform type GTT indicating a geometric transform of a set of geometric transforms or a combination of at least two such geometric transforms, step 410 is performed. Optionally, step 420 is also performed. For each position P in the determined search area, steps 430 and 440 are performed. When all positions in the determined search area have been considered, another geometry transform type (new GTT) is considered.

[0217] When all geometric transformation types have been considered, the candidate reference block selected is the candidate reference block with the minimum cost (450).

[0218] Fig.17 A block diagram of a method 400 according to an embodiment is schematically illustrated.

[0219] This embodiment applies only to method 100 (encoding).

[0220] For each new position (new P) in the initial search area, each newly considered geometric transformation type (new GTT) indicating a geometric transformation of a set of geometric transformations or a combination of at least two such geometric transformations is considered.

[0221] For each newly considered geometry transform type (new GTT), steps 410, 430 and 440 are performed, and

[0222] Optionally, the initial search area is adjusted according to step 420 (between steps 410 and 430).

[0223] When all newly considered geometric transformation types (new GTTs) have been considered, another position in the initial search area is considered.

[0224] When all positions in the initial search area have been considered, the candidate reference block selected is the candidate reference block with the minimum cost (450).

[0225] Fig.18 A block diagram of a method 400 according to an embodiment is schematically illustrated.

[0226] This embodiment applies only to method 200 (decoding).

[0227] A geometry transform type GTT indicating a set of geometry transforms or a combination of at least two of the geometry transforms is obtained by an instance of parsing a bitstream.

[0228] Optionally, step 420 is also performed.

[0229] For each position P in the determined search area, steps 430 and 440 are performed. When all positions in the determined search area have been considered, the selected candidate reference block is the candidate reference block with the minimum cost (450).

[0230] In one embodiment of step 470, Fig.19 As shown, when the geometric transform type indicates a flip mode (ie, horizontal flip or vertical flip or horizontal and vertical flip), the geometric transform type is signaled in the bitstream B as an integer value of the codec unit syntax element intra_tmp_flip_type defined in VVC.

[0231] For example, when intra_tmp_flip_type is equal to 0, the geometric transform type is disabled (i.e., identity). When intra_tmp_flip_type is equal to 1, the geometric transform type is a horizontal flip for intra-prediction of a target block of a video picture according to method 400. When intra_tmp_flip_type is equal to 2, the geometric transform type is a vertical flip mode for intra-prediction of a block of a video picture according to method 400. When intra_tmp_flip_type is equal to 3, the geometric transform type is a combination of vertical and horizontal flips for intra-prediction of a target block of a video picture according to method 400.

[0232] As another example, intra_tmp_flip_flag is equal to 1 to set intra_tmp_flip_type. If intra_tmp_flip_flag is equal to 0, intra_tmp_flip_type is not signaled and its value is inferred to be 0.

[0233] The present disclosure is not limited to a specific structure of the syntax element used to carry the geometric transformation type in the bitstream. Fig.19 The example of does not limit the meaning of the geometric transformation type, but can be expanded to indicate other geometric transformations, such as rotation or a combination of rotations or a combination of at least one flip and at least one rotation.

[0234] Fig. 20 A schematic block diagram illustrating an example of a system 500 in which various aspects and embodiments are implemented is shown.

[0235] System 500 may be embedded as one or more devices, including various components described below. In various embodiments, system 500 may be configured to implement one or more aspects described in this application.

[0236] Examples of equipment that may constitute all or part of system 500 include personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected household appliances, connected vehicles and their associated processing systems, head-mounted display devices (HMD, perspective glasses), projectors (projectors), "caves" (systems including multiple displays), servers, video encoders, video decoders, post-processors for processing outputs from video decoders, pre-processors for providing inputs to video encoders, web servers, video servers (e.g., broadcast servers, video-on-demand servers or web servers), static or video cameras, encoding or decoding chips or any other communication devices. The elements of system 500 may be implemented individually or in combination in a single integrated circuit (IC), multiple ICs and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 500 may be distributed across multiple ICs and / or discrete components. In various embodiments, system 500 may be connected to other similar systems or other electronic devices in a communication manner via, for example, a communication bus or by dedicated input and / or output ports.

[0237] The system 500 may include at least one processor 510 configured to execute instructions loaded therein for implementing, for example, various aspects described in the present application. The processor 510 may include embedded memory, input-output interfaces, and various other circuits known in the art. The system 500 may include at least one memory 520 (e.g., a volatile memory device and / or a non-volatile memory device). The system 500 may include a storage device 540, which may include a non-volatile memory and / or a volatile memory, including but not limited to an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a programmable read-only memory (PROM), a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a disk drive, and / or an optical drive. As a non-limiting example, the storage device 540 may include an internal storage device, an attached storage device, and / or a network accessible storage device.

[0238] The system 500 may include an encoder / decoder module 530, which is configured to, for example, process data to provide encoded / decoded video picture data, and the encoder / decoder module 530 may include its own processor and memory. The encoder / decoder module 530 may represent a module (or modules) that may be included in a device to perform encoding and / or decoding functions. As known, a device may include one or both of the encoding and decoding modules. In addition, the encoder / decoder module 530 may be implemented as a separate element of the system 500, or may be incorporated into the processor 510 as a combination of hardware and software known to those skilled in the art.

[0239] Program code to be loaded into the processor 510 or the encoder / decoder 530 to perform various aspects described in the present application may be stored in the storage device 540 and subsequently loaded into the memory 520 to be executed by the processor 510. According to various embodiments, during the execution of the processes described in the present application, one or more of the processor 510, the memory 520, the storage device 540, and the encoder / decoder module 530 may store one or more of various items. Such stored items may include, but are not limited to, video picture data, information data for encoding / decoding video picture data, bit streams, matrices, variables, and intermediate or final results of equations, formulas, operations, and operation logic processing.

[0240] In several embodiments, memory internal to the processor 510 and / or encoder / decoder module 530 may be used to store instructions and provide working memory for processes that may be performed during encoding or decoding.

[0241] However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 510 or the encoder / decoder module 530) is used for one or more of these functions. The external memory may be a memory 520 and / or a storage device 540, such as a dynamic volatile memory and / or a non-volatile flash memory. In several embodiments, the external non-volatile flash memory is used to store the operating system of the television. In at least one embodiment, a fast external dynamic volatile memory such as RAM may be used as working memory for video encoding and decoding operations, such as for MPEG-2 Part 2 (also known as ITU-T Recommendation H.262 and ISO / IEC 13818-2, also known as MPEG-2 Video), AVC, HEVC, EVC, VVC, AV1, etc.

[0242] As indicated in block 590, input to the elements of system 500 may be provided through various input devices. Such input devices include, but are not limited to, (i) an RF section that may receive an RF signal transmitted over the air, for example, by a broadcast device, (ii) a composite input terminal, (iii) a USB input terminal, (iv) an HDMI input terminal, (v) a bus such as a CAN (Controller Area Network), CAN FD (Controller Area Network Flexible Data Rate), FlexRay (ISO 17458), or Ethernet (ISO / IEC 802-3) bus when the present disclosure is implemented in the automotive field.

[0243] In various embodiments, the input device of block 590 has associated corresponding input processing elements, as known in the art. For example, the RF portion may be associated with elements necessary for: (i) selecting a desired frequency (also referred to as selecting a signal, or limiting the signal band to within a band), (ii) down-converting the selected signal, (iii) limiting the band again to a narrower band to select a signal band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired packet stream. The RF portion of various embodiments may include one or more elements that perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF portion may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or a baseband.

[0244] In one set-top box embodiment, the RF section and its associated input processing elements can receive RF signals transmitted over a wired (e.g., cable) medium. The RF section can then perform frequency selection by filtering, down-converting, and filtering again to a desired frequency band.

[0245] Various embodiments rearrange the order of the above-described (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions.

[0246] Adding an element may include inserting an element between existing elements, such as, for example, inserting an amplifier and an analog-to-digital converter.In various embodiments, the RF portion may include an antenna.

[0247] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 500 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, in a separate input processing IC or in the processor 510 when necessary. Similarly, various aspects of USB or HDMI interface processing may be implemented in a separate interface IC or in the processor 510 when necessary. The demodulated, error-corrected, and demultiplexed streams may be provided to various processing elements, including, for example, the processor 510 and the encoder / decoder 530, which operate in conjunction with memory and storage elements to process the data streams for presentation on output devices when necessary.

[0248] The various elements of system 500 may be provided within an integrated housing. Within the integrated housing, the various elements may be interconnected and data transferred between them using a suitable connection arrangement 590, such as an internal bus (including an I2C bus), wiring, and printed circuit boards known in the art.

[0249] System 500 may include a communication interface 550 that enables communication with other devices via a communication channel 551. Communication interface 550 may include, but is not limited to, a transceiver configured to send and receive data over communication channel 551. Communication interface 550 may include, but is not limited to, a modem or a network card, and communication channel 551 may be implemented, for example, within a wired and / or wireless medium.

[0250] In various embodiments, a Wi-Fi network such as IEEE 802.11 may be used to stream data to the system 500. The Wi-Fi signals of these embodiments may be received via a communication channel 551 suitable for Wi-Fi communications and a communication interface 550. The communication channel 551 of these embodiments may typically connect to an access point or router that provides access to external networks including the Internet to allow streaming applications and other over-the-top communications.

[0251] Other embodiments may provide streaming data to system 500 using a set top box that delivers the data through an HDMI connection to input block 590 .

[0252] Still other embodiments may use the RF connection of input block 590 to provide streaming data to system 500 .

[0253] Streamed data may be used as a means of signaling information used by the system 500. The signaled information may include the bitstream B and / or information such as the number of pixels of a video picture and / or any encoding / decoding setting parameters.

[0254] It should be appreciated that signaling can be implemented in a variety of ways. For example, in various embodiments, one or more syntax elements, flags, etc. may be used to signal information to a corresponding decoder.

[0255] The system 500 may provide output signals to various output devices, including a display 561, speakers 571, and other peripherals 581. In various examples of embodiments, the other peripherals 581 may include one or more of a stand-alone DVR, a disc player, a stereo system, a lighting system, and other devices that provide functionality based on the output of the system 500.

[0256] In various embodiments, control signals may be communicated between the system 500 and the display 561, speakers 571, or other peripherals 581 using signaling such as AV.Link (Audio / Video Link), CEC (Consumer Electronics Control), or other communication protocols that enable device-to-device control with or without user intervention.

[0257] Output devices may be communicatively coupled to system 500 through respective interfaces 560 , 570 , and 580 via dedicated connections.

[0258] Alternatively, the output device may be connected to the system 500 via the communication interface 550 using the communication channel 551. The display 561 and the speaker 571 may be integrated with the other components of the system 500 in a single unit in an electronic device such as, for example, a television.

[0259] In various embodiments, the display interface 560 may include a display driver, such as, for example, a timing controller (TCon) chip.

[0260] For example, if the RF portion of input 590 is part of a separate set-top box, then display 561 and speaker 571 may optionally be separate from one or more of the other components. In various embodiments where display 561 and speaker 571 may be external components, output signals may be provided via dedicated output connections including, for example, an HDMI port, a USB port, or a COMP output.

[0261] exist Figure 1-20 In the present invention, various methods are described herein, and each method includes one or more steps or actions to implement the described method. Unless a specific order of steps or actions is required for the correct operation of the method, the order and / or use of specific steps and / or actions can be modified or combined.

[0262] Some examples are described about block diagrams and / or operational flow charts. Each square block represents a portion of a circuit element, module or code, which includes one or more executable instructions for implementing (one or more) specified logical functions. It should also be noted that, in other embodiments, the (one or more) functions marked in the square block may not occur in the order indicated. For example, depending on the functions involved, two square blocks shown in succession can actually be executed substantially concurrently, or sometimes these square blocks can be executed in reverse order.

[0263] The embodiments and aspects described herein may be implemented in, for example, a method or process, an apparatus, a computer program, a data stream, a bit stream, or a signal. Even if only discussed in the context of a single form of embodiment (e.g., discussed only as a method), the embodiments of the features discussed may also be implemented in other forms (e.g., an apparatus or a computer program).

[0264] The method may be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. A processor also includes a communication device.

[0265] In addition, the method can be implemented by instructions executed by a processor, and such instructions (and / or data values ​​generated by the implementation) can be stored on a computer-readable storage medium. The computer-readable storage medium can take the form of a computer-readable program product implemented in one or more computer-readable media and having a computer-readable program code implemented thereon that can be executed by a computer. Considering the inherent ability to store information therein and the inherent ability to provide information retrieval therefrom, the computer-readable storage medium used herein can be considered as a non-transient storage medium. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. It should be appreciated that although more specific examples of computer-readable storage media to which the present embodiment can be applied are provided below, as those of ordinary skill in the art will readily appreciate, it is merely illustrative and not exhaustive: portable computer floppy disk; hard disk; read-only memory (ROM); erasable programmable read-only memory (EPROM or flash memory); portable compact disk read-only memory (CD-ROM); optical storage device; magnetic storage device; or any suitable combination of the foregoing.

[0266] The instructions may form an application program tangibly embodied on a processor-readable medium.

[0267] For example, instructions may be in hardware, firmware, software, or a combination. For example, instructions may be found in an operating system, a separate application, or a combination of both. Thus, a processor may be characterized as, for example, a device configured to perform a process and a device including a processor-readable medium (such as a storage device) having instructions for performing the process. Additionally, in addition to or in lieu of instructions, a processor-readable medium may store data values ​​generated by an embodiment.

[0268] Device can be realized in suitable hardware, software and firmware for example.The example of this device comprises personal computer, laptop computer, smart phone, tablet computer, digital multimedia set-top box, digital television receiver, personal video recording system, connected household appliances, head-mounted display device (HMD, perspective glasses), projector (projector), "cave" (system including multiple displays), server, video encoder, video decoder, post-processor for processing the output from video decoder, pre-processor for providing input to video encoder, web server, set-top box, and any other equipment for processing video picture, or other communication equipment.It should be clear that equipment can be mobile and even installed in a mobile vehicle.

[0269] Computer software may be implemented by processor 510 or by hardware, or by a combination of hardware and software. As a non-limiting example, embodiments may also be implemented by one or more integrated circuits. Memory 520 may be of any type suitable for the technical environment and may be implemented using any appropriate data storage technology (such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples). Processor 510 may be of any type suitable for the technical environment and may encompass one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture, as non-limiting examples.

[0270] As will be apparent to one of ordinary skill in the art, embodiments may generate various signals formatted to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for executing a method or data generated by one of the described embodiments. For example, a signal may be formatted to carry a bit stream of the described embodiments. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

[0271] The terms used herein are only used to describe the purpose of a particular embodiment and are not intended to be limited. As used herein, the singular forms "one", "a kind of" and "the" may also be intended to include plural forms unless the context clearly indicates otherwise. It will be further understood that, when used in this specification, the terms "include / comprise" and / or "including / comprising" may specify the existence of stated, for example, features, integers, steps, operations, elements and / or components, but do not exclude the existence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. Moreover, when an element is referred to as "in response to" or "connected to" another element or "associated with another element", it may directly respond or be connected to or be associated with another element, or there may be an intermediate element. In contrast, when an element is referred to as "directly responding to" or "directly connected to" another element or "directly associated with another element", there is no intermediate element.

[0272] It should be appreciated that use of any of the symbols / terms " / ", "and / or", and "at least one of" may be intended to encompass selection of only the first listed option (A), or only the second listed option (B), or both options (A and B), such as in the case of "A / B", "A and / or B", and "at least one of A and B". As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C", such wording is intended to encompass selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). This may be extended to as many items as listed, as will be apparent to one of ordinary skill in this and related arts.

[0273] Various numerical values ​​may be used in the present application. Specific values ​​may be used for example purposes and the described aspects are not limited to these specific values.

[0274] It will be understood that although the terms first, second, etc. can be used to describe various elements in this article, these elements are not limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the teaching of this application, the first element can be referred to as the second element, and similarly, the second element can be referred to as the first element. There is no suggestion of sorting between the first element and the second element.

[0275] References to "one embodiment" or "an embodiment" or "one implementation" or "an implementation" and other variations thereof are frequently used to convey that a particular feature, structure, characteristic, etc. (described in conjunction with the embodiment / implementation) is included in at least one embodiment / implementation. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in an implementation" or "in an implementation" and any other variations appearing in various places in this application do not necessarily all refer to the same embodiment.

[0276] Similarly, references herein to "according to an embodiment / example / implementation" or "in an embodiment / example / implementation" and other variations thereof are frequently used to convey that a particular feature, structure, or characteristic (described in conjunction with an embodiment / example / implementation) may be included in at least one embodiment / example / implementation. Therefore, the expressions "according to an embodiment / example / implementation" or "in an embodiment / example / implementation" appearing in various places in this application do not necessarily all refer to the same embodiment / example / implementation, nor do separate or alternative embodiments / examples / implementations necessarily exclude other embodiments / examples / implementations.

[0277] Reference numerals appearing in the claims are for illustration purposes only and have no limiting effect on the scope of the claims.The present embodiments / examples and variations may be employed in any combination or sub-combination although not explicitly described.

[0278] When a figure is presented as a flow chart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a figure is presented as a block diagram, it should be understood that it also provides a flow chart of the corresponding method / process.

[0279] While some diagrams include arrows on communication paths to illustrate a primary direction of communication, it should be understood that communication can occur in the opposite direction to the depicted arrows.

[0280] Various embodiments relate to decoding. As used in the present application, "decoding" may encompass, for example, all or part of a process performed on a received video picture (which may include a received bitstream encoding one or more video pictures) to produce a final output suitable for display or further processing in a reconstructed video domain. In various embodiments, such a process includes one or more of the processes typically performed by a decoder. In various embodiments, for example, such a process also includes or optionally includes a process performed by a decoder of the various embodiments described in the present application.

[0281] As a further example, in one embodiment "decoding" may refer only to dequantization, in one embodiment "decoding" may refer to entropy decoding, in another embodiment "decoding" may refer only to differential decoding, and in another embodiment "decoding" may refer to a combination of dequantization, entropy decoding, and differential decoding. Based on the context of the specific description, whether the phrase "decoding process" may be intended to specifically refer to a subset of operations, or generally refer to a broader decoding process will be clear and is believed to be well understood by those skilled in the art.

[0282] Various embodiments all relate to encoding. In a manner similar to the above discussion about "decoding", "encoding" as used in this application may encompass, for example, all or part of a process performed on an input video picture to produce an output bitstream. In various embodiments, such a process includes one or more of the processes typically performed by an encoder. In various embodiments, such a process also includes or optionally includes a process performed by an encoder of the various embodiments described in this application.

[0283] As a further example, in one embodiment "encoding" may refer only to quantization, in one embodiment "encoding" may refer only to entropy coding, in another embodiment "encoding" may refer only to differential coding, and in another embodiment "encoding" may refer to a combination of quantization, differential coding, and entropy coding. Based on the context of the particular description, whether the phrase "encoding process" may be intended to specifically refer to a subset of operations, or to generally refer to a broader encoding process will be clear and is believed to be well understood by those skilled in the art.

[0284] Additionally, the present application may refer to "obtaining" various information. Obtaining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from a memory, processing information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0285] Furthermore, this application may refer to "receiving" various pieces of information. Receiving information may include, for example, one or more of: accessing the information or receiving the information from a communication network.

[0286] Moreover, as used herein, the word "signaling" is especially directed to the corresponding decoder to indicate something, etc. For example, in some embodiments, the encoder signals specific information, such as codec parameters or encoded video picture data. In this way, in an embodiment, the same parameter can be used on the encoder side and the decoder side. Therefore, for example, the encoder can transmit (explicitly signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. On the contrary, if the decoder already has specific parameters and other parameters, signaling can be used without transmission (implicit signaling) to simply allow the decoder to know and select specific parameters. By avoiding the transmission of any actual function, bit saving is achieved in various embodiments. It should be appreciated that signaling can be accomplished in a variety of ways. For example, in various embodiments, one or more syntax elements, flags, etc. are used to signal information to the corresponding decoder. Although the verb form of the word "signaling" is mentioned above, the word "signaling" can also be used as a noun in this article.

[0287] A number of embodiments have been described. However, it should be understood that various modifications may be made. For example, elements of different embodiments may be combined, supplemented, modified, or removed to produce other embodiments. In addition, it will be understood by those of ordinary skill that other structures and processes may replace the disclosed structures and processes, and that the resulting embodiments will perform at least substantially the same (one or more) functions in at least substantially the same (one or more) manners to achieve at least substantially the same (one or more) results as the disclosed embodiments. Thus, the present application contemplates these and other embodiments.

Claims

1. A method for intra-frame prediction of a target block (TB) of a video picture based on template matching, wherein the template matching compares a target template block (TTP) including reconstructed samples in a target block neighborhood (TBN) with a candidate reference template block (RTP) including reconstructed samples in a candidate reference block neighborhood (RBN) of a candidate reference block (RB) located at a position in a search area, wherein the method comprises: - determining (410) the candidate reference block neighborhood (RBN) according to the availability of reconstruction samples of the target block neighborhood (TBN) and according to the geometric transform type (GTT); - obtaining (430) the reconstructed samples of the candidate reference template block (RTP) by applying a geometric transformation identified by the geometric transformation type (GTT) to the reconstructed samples in the candidate reference block neighborhood (RBN); - associating (440) a cost with the candidate reference block (RB), the candidate reference block (RB) being obtained by comparing the reconstructed samples of the candidate reference template block (RTP) with the reconstructed samples of the target template block (TTP); - selecting (450) said candidate reference block having the smallest cost, said selected candidate reference block being associated with a geometric transform type (GTT) identified with a geometric transform (GT); and - generating (460) an intra-prediction block for the target block (TB) by applying the geometric transformation identified by the geometric transformation type (GTT) associated with the selected candidate reference block to reconstructed samples of the selected candidate reference block.

2. The method according to claim 1, wherein the method further comprises: The search area is adjusted (420) according to the geometric transform type (GTT) and the position of the target block (TB) in the video picture.

3. The method according to claim 1 or 2, wherein for each geometric transformation indicating a set of geometric transformations or a current geometric transformation type (GTT) indicating a combination of at least two of the geometric transformations, - determining (410) a candidate reference block neighborhood (RBN) based on the availability of reconstructed samples of the target block neighborhood (TBN) and the current geometric transform type (GTT); - for each current position in the search area, obtaining (430) reconstructed samples of a candidate reference template block of a candidate reference block (RB) located at the current position in the search area, and associating (440) a cost with the candidate reference block (RB) located at the current position in the search area; - when all said positions in said search area have been considered, another type of geometric transformation is considered, - When all of the geometric transformation types have been considered, the selected candidate reference block is the candidate reference block with the minimum cost (450).

4. The method according to claim 1 or 2, wherein for each current position in the search area, a current geometric transformation type indicating a geometric transformation of a set of geometric transformations or a combination of at least two of the geometric transformations is considered, - determining (410) a candidate reference block neighborhood (RBN) based on the availability of reconstructed samples of the target block neighborhood (TBN) and the current geometric transform type (GTT); - obtaining (430) reconstructed samples of candidate reference template blocks of candidate reference blocks (RB) located at said current position in the search area; - associating a cost with the candidate reference block (RB) located at the current position (440); - when all of the geometric change types have been considered, consider another current position in the search area; - When all the positions in the search area have been considered, the selected candidate reference block is the candidate reference block with the minimum cost (450).

5. The method according to claim 1 or 2, wherein for each current geometric transformation type indicating a set of geometric transformations or indicating a combination of at least two of the geometric transformations, the current position in the search area is considered, - determining (410) a candidate reference block neighborhood (RBN) based on the reconstructed sample availability and the geometric transformation type (GTT) of the target block neighborhood (TBN); - for each current position in the search area, obtaining (430) reconstructed samples of a candidate reference template block of a candidate reference block (RB) located at the current position in the search area, and associating (440) a cost with the candidate reference block (RB) located at the current position in the search area; - When all the positions in the search area have been considered, the selected candidate reference block is the candidate reference block with the minimum cost (450).

6. A method according to any one of claims 1 to 5, wherein the cost associated with a candidate reference block (RB) is evaluated by the sum of absolute differences (SAD) between the transformed and reordered samples of a reference template block (RTP) and the reconstructed samples of the target template block (TTP).

7. The method according to any one of claims 1 to 6, wherein the geometric transformation type indicates at least one of the following geometric transformations or a combination of at least two of the following geometric transformations: -Flip horizontally; -Flip vertically; - Rotation; -Constant. The method according to claim 7 , wherein the rotation angle of the rotation is a multiple of 90°.

9. The method according to any one of claims 1 to 8, wherein the method further comprises signaling (470) the geometric transformation type associated with the selected candidate reference block.

10. A method for encoding a block of a video picture based on a prediction block of the block, wherein the prediction block is obtained according to the method of any one of claims 1 to 4 or 6 to 9.

11. A method for decoding a block of a video picture based on a prediction block of the block, wherein the prediction block is obtained according to the method of any one of claims 1 to 9.

12. A bit stream formatted to include encoded video picture data obtained by the method according to claim 10.

13. An apparatus comprising means for carrying out the method according to any one of claims 1 to 11.

14. A computer program product comprising instructions, which, when executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 1 to 11.

15. A non-transitory storage medium carrying instructions of a program code for executing the method according to any one of claims 1 to 11.

16. An electronic device comprising: processor; and a memory for storing instructions executable by the processor, wherein the processor is configured to perform the method according to any one of claims 1 to 11.

Images