Perspective intra prediction signaling with decoder-side intra mode derivation and template-based intra mode derivation

Abstract

A method for decoding, comprising: applying (1307) a backward intra prediction process until at least one angular mode of a target block is obtained, determining at least one perspective point in a neighborhood of the target block from the at least one angular mode; applying (1307) perspective intra prediction (PIP) from samples in the neighborhood of the target block to generate a PIP predictor using at least one of the determined at least one perspective point; predicting (1307) the target block using the PIP predictor.

Description

[0001] 1. Cross-reference of related applications This application claims priority to European application number 24305494.7, filed on 29 March 2024, the entire contents of which are incorporated herein by reference. 2. Technical Field At least one embodiment in this example relates generally to a method and apparatus for applying intra-frame prediction in an image. 3. Background Technology To achieve high compression efficiency, video decoding schemes typically employ prediction and transform to utilize spatial and temporal redundancy in the video content. During encoding, images of the video content are divided into blocks of samples (i.e., pixels), which are then further subdivided into one or more sub-blocks, hereinafter referred to as original sub-blocks. Intra-frame or inter-frame prediction is then applied to each sub-block to utilize intra-frame or inter-frame image correlations. Regardless of the prediction method used (intra-frame or inter-frame), a predictor sub-block is determined for each original sub-block. The sub-block representing the difference between the original sub-block and the predictor sub-block, typically referred to as the prediction error sub-block, or prediction residual sub-block, or simply residual sub-block, is then transformed, quantized, and entropy-encoded to generate the encoded video stream. To reconstruct the video, the compressed data is decoded through the inverse process corresponding to the transform, quantization, and entropy decoding.

[0004] Many decoding tools for intra-frame prediction have been proposed in the past. Many of these intra-frame prediction methods are forward prediction tools, meaning the intra-frame prediction method used at the encoder is explicitly signaled to the decoder. Recently, tools employing backward estimation have been proposed (i.e., estimating the prediction mode using reconstructed samples already decoded in the image). Decoder-side intra-frame mode derivation (DIMD) and template-based intra-frame mode derivation employ backward estimation methods to estimate multiple prediction modes for a block. The encoder uses a minimal amount of signaling to indicate which modes to employ and, if necessary, performs a blending of multiple predictions corresponding to multiple prediction modes. Another approach for multiple prediction modes based on a block is through the use of perspective intra-frame prediction modes. In current implementations, perspective intra-frame prediction modes may require signaling perspective points predicted from their derivation.

[0005] It is desirable to propose solutions that can overcome the above-mentioned problems. In particular, it is desirable to propose methods that allow for a reduction in the overhead required to transmit perspective intra-frame prediction modes via signals. 4. Summary of the Invention In a first aspect, one or more of the embodiments in this paper provide a method for encoding, comprising: Apply a backward intra-frame prediction process until at least one angular mode of the target block is obtained, and determine at least one perspective point in the neighborhood of the target block from at least one angular mode; Perspective intra-frame prediction (PIP) of reconstructed samples from the neighborhood of the target block is applied to generate a PIP predictor using at least one of the determined at least one perspective point. Use the PIP predictor to predict the target block.

[0007] In a second aspect, one or more of the embodiments in this paper provide a method for decoding, comprising: Apply a backward intra-frame prediction process until at least one angular pattern of the target block is obtained, and determine at least one perspective point in the neighborhood of the target block from the at least one angular pattern. Perspective intra-frame prediction (PIP) of samples from the neighborhood of the target block is applied to generate a PIP predictor using at least one of the determined at least one perspective point. Use the PIP predictor to predict the target block.

[0008] In the embodiments of the first or second aspect, the backward intra-prediction process is decoder-side intra-mode derivation (DIMD).

[0009] In embodiments of the first or second aspect, the method includes transmitting in video data or decoding from video data information indicating that DIMD is enabled for a target block and information indicating that a PIP predictor is applied to predict the target block.

[0010] In the embodiments of the first or second aspect, the backward intra-prediction process is based on template-based intra-mode derivation (TIMD).

[0011] In an embodiment of the first or second aspect, the method includes transmitting in video data or decoding from video data information indicating that TIMD is enabled for a target block and information indicating that a PIP predictor is applied to predict the target block.

[0012] In an embodiment of the first or second aspect, the neighborhood of the target block is divided into a plurality of reference arrays, and the method includes signaling information representing one of the plurality of reference arrays to be used for applying perspective intra-frame prediction.

[0013] In embodiments of the first or second aspect, at least one value representing information of one of a plurality of reference arrays indicates that the PIP predictor is applied to predict the target block.

[0014] In an embodiment of the first or second aspect, the method includes, in response to generating a plurality of perspective points, generating at least one additional PIP predictor, each additional PIP predictor being generated from a perspective point among the plurality of perspective points, and identifying a final PIP predictor that provides the best rate-distortion performance among the PIP predictors, the target block being predicted from the final PIP predictor.

[0015] In embodiments of the first or second aspect, the method includes signaling information or decoding information representing the final PIP predictor.

[0016] In embodiments of the first or second aspect, information representing one of a plurality of reference arrays allows for the derivation of information representing the final PIP predictor.

[0017] In a third aspect, one or more of the embodiments in this example provide an encoding device, including electronic circuitry configured to: Apply a backward intra-frame prediction process until at least one angle pattern of the target block is obtained, and determine at least one perspective point in the neighborhood of the target block from the angle pattern. Perspective intra-frame prediction (PIP) of samples from the neighborhood of the target block is applied to generate a PIP predictor using at least one of the determined perspective points. Use the PIP predictor to predict the target block.

[0018] In a fourth aspect, one or more of the embodiments in this example provide a device for decoding, including electronic circuitry configured to: Apply a backward intra-frame prediction process until at least one angle pattern of the target block is obtained, and determine at least one perspective point in the neighborhood of the target block from the angle pattern. Perspective intra-frame prediction (PIP) of samples from the neighborhood of the target block is applied to generate a PIP predictor using at least one of the determined perspective points. Use the PIP predictor to predict the target block.

[0019] In the fifth aspect, one or more of the present embodiments provide signals representing video data, the video data including information indicating that decoder-side intra-frame mode derivation (DIMD) or template-based intra-frame mode derivation (TIMD) is enabled for a target block to determine at least one angular mode, determining at least one perspective point in the neighborhood of the target block from the at least one angular mode, and information indicating that perspective intra-frame prediction (PIP) using samples from the neighborhood of the target block of at least one of the determined perspective points is applied to determine a predictor to be used to predict the target block.

[0020] In a sixth aspect, one or more of the embodiments in this embodiment provide a non-transitory information storage medium that stores program code instructions for implementing the method according to the first or second aspect.

[0021] In a seventh aspect, one or more of the present embodiments provide a computer program including program code instructions for implementing the method according to the first or second aspect. 5. Description of the attached drawings Figure 1 The context of the implementation embodiment is illustrated schematically; Figure 2 An example of segmentation that a pixel image of the original video undergoes is illustrated schematically; Figure 3 A method for encoding video streams is illustrated schematically; Figure 4 A method for decoding encoded video streams is illustrated schematically; Figure 5A An example of a hardware architecture for a processing module that implements various aspects and embodiments and is capable of implementing an encoding module or a decoding module is illustrated schematically. Figure 5B A block diagram illustrating an example of a first system implementing various aspects and embodiments is shown; Figure 5C A block diagram illustrating an example of a second system in which various aspects and embodiments are implemented is shown; Figure 6A and Figure 6B This demonstrates the application of a perspective intra-frame prediction mode to blocks; Figure 7 The derivation of the angular parameters of the target pixel at a given pixel location in perspective intra-frame prediction mode is shown. Figure 8 The generation of a histogram of gradients for DIMD modes is shown; Figure 9 This describes the process of deriving the DIMD predictor. Figure 10 A variation of the DIMD mode is shown; Figure 11 The TIMD mode is shown; Figure 12 The encoding process according to the first embodiment is illustrated schematically; Figure 13 The decoding process according to the first embodiment is illustrated schematically; Figure 14 The encoding process according to the second embodiment is illustrated schematically; Figure 15The decoding process according to the second embodiment is illustrated schematically; Figure 16 The PDPC mode is shown; Figure 17 The process for encoding a target block according to the sixth embodiment is illustrated schematically; Figure 18 The decoding process according to the sixth embodiment is illustrated schematically; Figure 19 The process for encoding a target block according to the seventh embodiment is illustrated schematically; and Figure 20 The process for decoding a target block according to the seventh embodiment is illustrated schematically. 6. Detailed Implementation Methods The following examples of embodiments are described in the context of video formats similar to ECM, which are hereinafter simply referred to as ECM. JVET-AF2025 literature JVET-AF2025: Algorithm description of Enhanced Compression Model 11 (ECM 11) / Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29, 32nd Meeting, Hannover, DE, October 13–20, 2023 The embodiments described herein are not limited to video decoding / decoding methods corresponding to ECM. In particular, these embodiments are applicable to a variety of video formats, including (and derived from) such as VVC (Universal Video Decoding (VVC), ITU-T H.266), HEVC (ISO / IEC 23008-2-MPEG-H Part 2, High Efficiency Video Decoding / ITU-T H.265), AVC (ISO / CEI 14496-10), EVC (Basic Video Decoding / MPEG-5), AV1, AV2, and VP9. Additionally, these embodiments are also applicable to various still image formats, such as JPEG.

[0024] Figure 1 The context of the implementation embodiment is illustrated schematically.

[0025] exist Figure 1 In the system 11, which may be a camera, storage device, computer, server, or any device capable of delivering video streams (i.e., video data), the system 11 uses communication channel 12 to transmit the video stream to system 13. The video stream is encoded and transmitted by system 11, or received and / or stored by system 11 and then transmitted. Communication channel 12 is a wired (e.g., Internet or Ethernet) or wireless (e.g., WiFi, 3G, 4G, or 5G) network link.

[0026] System 13, such as a set-top box, receives and decodes the video stream to generate a sequence of decoded images. Post-processing can then be applied to the decoded images.

[0027] The sequence of decoded images is then transmitted to the display system 15 using communication channel 14, which can be a wired or wireless network. The display system 15 then displays the images.

[0028] In this embodiment, system 13 is included in display system 15. In this case, system 13 and display system 15 are included in a TV, computer, tablet computer, smartphone, head-mounted display, etc.

[0029] Figure 2 , 3 Sections 4 and 5 provide examples of video formats.

[0030] Figure 2 An example of the segmentation that image 21, representing the pixels of the original video sequence 20, undergoes is shown. Here, a pixel is considered to consist of three components: one luminance component and two chrominance components. However, it is possible to include other types of pixels, such as only a luminance component or additional depth or transparency components, with fewer or more components.

[0031] Divide the image into multiple decoded entities. First, such as... Figure 2 The figure in the attached drawing, indicated by reference numeral 23, divides the image into sections called... Decoding Tree Unit A grid of blocks (CTU). CTU is composed of brightness samples. It consists of a block and two corresponding blocks of the chromaticity sample. N Typically, it's a power of two, such as having a maximum value of "128". Secondly, divide the image into one or more CTU groups. For example, it can be divided into one or more tile rows and tile columns. tile It is a sequence of CTUs covering a rectangular area of ​​the image. In some cases, the tile can be divided into one or more... Brick Each brick consists of at least one row of CTUs within the tile. Beyond the concepts of tiles and bricks, there exists a concept called... slice Another decoded entity may contain at least one tile of the image or at least one brick of the tile.

[0032] exist Figure 2 In the example shown, as indicated by reference numeral 22 in the figure, image 21 is divided into three slices S1, S2 and S3 in raster scan slice mode, each slice including multiple tiles (not shown), and each tile including only one brick.

[0033] As by Figure 2 As indicated by reference numeral 24 in the attached figure, the CTU can be divided into what is called Decoding unitA CTU is a hierarchical tree of one or more sub-blocks of a CU. A CTU is the root (i.e., the parent node) of the hierarchical tree and can be divided into multiple CUs (i.e., child nodes). If each CU is not further divided into smaller CUs, it becomes a leaf of the hierarchical tree, or if it is further divided, it becomes the parent node of the smaller CUs (i.e., child nodes).

[0034] exist Figure 2 In the example, CTU 24 is first divided into 4 square CUs using a quadtree-type partitioning method. The top-left CU is a leaf of the hierarchy because it is not further partitioned; that is, it is not the parent node of any other CU. The top-right CU is further divided into 4 smaller square CUs using a quadtree-type partitioning method again. The bottom-right CU is vertically divided into 2 rectangular CUs using a binary tree-type partitioning method. The bottom-left CU is vertically divided into 3 rectangular CUs using a ternary tree-type partitioning method.

[0035] During image decoding, segmentation is adaptive; each CTU is segmented to optimize the compression efficiency of the CTU standard.

[0036] HEVC also introduces the concepts of prediction units (PU) and transform units (TU). In practice, in HEVC, the decoding entities used for prediction (PU) and transformation (TU) can be different subdivisions of the CU. For example, such as... Figure 2 The size represented in the text The CU can be divided into sizes or size PU 2411. Additionally, the CU can be divided into sizes. The “4” TU 2412 may be divided into sizes There are 16 TUs. TUs are always square in shape.

[0037] It can be noted that in VVC, except in some specific cases, the boundaries of TU and PU are aligned with the boundary of CU. Therefore, CU typically includes one TU and one PU.

[0038] In this application, the term "block" or "picture block" may be used to refer to any one of CTU, CU, PU, ​​and TU. Additionally, the term "block" or "picture block" may be used to refer to macroblocks, partitions, and subblocks as specified in H.264 / AVC or other video decoding standards, and more generally, to an array of samples of numerous sizes.

[0039] In this application, the terms "reconstructed" and "decoded" are used interchangeably, as are the terms "pixel" and "sample," and the terms "image," "picture," "subpicture," "slice," and "frame." Generally, but not necessarily, the term "reconstructed" is used on the encoder side, while "decoded" is used on the decoder side.

[0040] Figure 3 The method for encoding a video stream, performed by the encoding module, is illustrated schematically. For example, it is performed by system 11. Figure 3 The method used for encoding is described below. Variations of this method for encoding are envisioned, but for clarity, they are described below. Figure 3 The method used for encoding, without describing all expected variations.

[0041] Before being encoded, the current raw image of the original video sequence may undergo preprocessing. For example, in step 301, a color transformation is applied to the current raw image (e.g., a conversion from RGB 4:4:4 to YCbCr 4:2:0), or a remapping is applied to the components of the current raw image to obtain a signal distribution that is more resilient to compression (e.g., histogram equalization using one of the color components). The image obtained through preprocessing is referred to below as the preprocessed image.

[0042] The encoding of the preprocessed image begins during the segmentation of the preprocessed image in step 302, as per [reference to...]. Figure 2 As described, the preprocessed image is therefore segmented into CTU, CU, PU, ​​TU, etc.

[0043] For each block, the encoding module then determines the decoding mode between the intra-frame prediction mode and the inter-frame prediction mode.

[0044] Intra-frame prediction aims to utilize spatial redundancy in the image. In step 303, spatial prediction is performed on the current block (i.e., PU or CU) from samples of causal (reconstructed) neighboring blocks in the same image, namely, the top and upper right blocks, the left and lower left blocks, and the upper left block. The encoder constructs multiple predictor blocks for the current block from these samples and selects the predictor block that results in the best rate-distortion (RD) performance. In recent video compression methods, up to 67 intra-frame prediction modes are constructed, resulting in 67 predictor blocks. The multiple predictor blocks include planar modes (indexed as mode 0), DC modes (indexed as mode 1), and the remaining 65 are angular modes. Planar and DC modes are designed to model regions of slowly changing intensity, while angular modes are designed to model directional structures present in the image. The best prediction mode is encoded and transmitted to the decoder, causing the decoder to construct the same intra-frame prediction block for the current block.

[0045] To keep the decoder complexity low, intra-frame prediction typically uses only the top row of reference pixels and the left column of reference pixels of the current block. Although the encoder has [a feature called...] Multiple reference lines In the (MRL) mode, the option to select from three sets of reference lines is available, but the prediction is always based on a single reference line consisting of the top row and the left column. Due to this limitation, the amount of information contained in the reference pixels is very limited, and therefore the efficiency of intra-frame prediction based on the correlation between samples in the current block and reference samples is also limited. Recently, proposals have been made to use up to two adjacent reference lines. In this case, the predictor is constructed as a weighted average of two predictors, each based on a single reference line.

[0046] Recently, the Perspective Intra-Prediction (PIP) mode was introduced. In PIP, perspective points are used to predict target blocks. A perspective point is selected on a straight line passing through the center of the block, and this line indicates the prediction direction associated with the intra-frame mode.

[0047] Figure 6A and 6B The illustration shows an example of applying PIP mode on a block. In PIP mode, the corresponding... Figure 6B Point P in m The perspective point is used to predict the target block. The perspective point is positioned at a certain offset distance from the target block along the prediction direction. For vertical prediction mode, the vertical offset (y-offset) distance can be determined by... dy This indicates that, for the horizontal prediction mode, the horizontal offset (x-offset) distance can be determined by... dx This indicates. In the following text, we will... dx and dy These are referred to as horizontal and vertical perspective distances, respectively. In the following text, we consider deriving the horizontal and vertical perspective distances from the dimensions (height and width) of the target block. For the vertical angle mode (an intra-frame prediction mode that is neither planar nor DC), we will refer to the corresponding perspective prediction as... vertical Perspective Prediction or Vertical PIP And similarly, for the horizontal angle pattern, we call the corresponding perspective prediction... Horizontal perspective predict or Horizontal PIP .

[0048] The main difference between standard angle intra-frame prediction and PIP mode is that, in PIP mode, the prediction direction along the row or column of the target block is variable. Angle parameters predIntraAngle A defined function that is the position of the target pixel.

[0049] Figure 7 The angle parameters of the target pixel at pixel position (x, y) are shown. predIntraAngleThe derivation is as follows: The texture pixels at the top and left of the target block are reference samples used for prediction. The predictor for the target pixel is located at the perspective point P. m A reference sample in the direction of joining. For illustration purposes, we have chosen perspective point P. m The vertical perspective prediction mode is above the target block. The situation is similar for the horizontal perspective mode when the perspective point is to the left of the target block, as the prediction can be obtained in the same way by simply swapping the top and left reference arrays of the target block and transposing the resulting prediction at the end. Let... W and H Let represent the width and height of the target block, respectively. d This represents the perspective height, that is, the vertical distance from the perspective point to the top edge of the object block. To avoid division, we assume... d It is a power of 2. Specifically, for vertical perspective mode, we use... d=k×H ,in k =2 n n=0,1,... Kmax ,in Kmax It is a preset integer. For horizontal perspective mode, we use... d=k×W ,in k =2 n n=0,1,... Kmax .make A(x, y) Represents the angle parameter at (x, y). predIntraAngle And make An angle parameter associated with the prediction mode of the target block's parent block. predIntraAngle (That is, the target block is generated by the splitting of the parent block).

[0050] Angular parameters at target pixel (x, y) predIntraAngle A(x, y) is calculated as The angle parameter A(x, 0), which is located in the first row but in the same column as the target pixel, is calculated as follows: in s The values ​​are either "5" or "6", depending on whether the sample resolution for the predicted direction is (1 / 32) or (1 / 64) pixels. (Angle parameter) predIntraAngle A(x, y) can also be calculated directly as: Once the angle parameters have been calculated predIntraAngle A(x, y) still identifies and interpolates predictor samples on the reference array. For example, the identification and interpolation process defined in ECM is applied. Let... deltaPos(x, y) This represents the x-offset (horizontal offset) of the predictor from the target pixel at (x, y). Let... deltaInt(x, y) and deltaFrac(x, y) The x-offsets calculated below are respectively represented as follows. deltaPos(x, y) Integral and fractional parts: .

[0051] Now, the predictor samples can be interpolated using six nearest reference samples, three on each side of the predictor sample, as denoted in ECM. in refMain It is a reference sample array pointing to the sample in the top left corner. According to deltaFrac(x, y) The value of determines the interpolation filter. If f (n) represents the interpolation filter coefficients, and the predictor samples are obtained as follows: In another example, a different number of reference samples could be used. Alternatively, only the two most recent reference samples could be used, such as, for example... P 2 and P 3. Perform linear interpolation on the predictor samples.

[0052] Finally, for some eligible prediction modes, the initial prediction can be followed by a position-dependent intra-prediction combination (PDPC).

[0053] In intra-frame prediction, the purpose of PDPC is to eliminate brightness discontinuities near the reference array that may occur after the initial prediction. These discontinuities can occur in planar mode, DC mode, and all angular prediction modes, which are associated with prediction directions below and including purely horizontal prediction modes, and prediction directions to the right and including purely vertical prediction modes. A PIP is eligible for PDPC if the prediction direction at the left reference array (for vertical PIP) or the top reference (for horizontal PIP) is non-negative. This means that PDPC can be applied after the initial prediction if the perspective point is at the top of the block and on the right and includes the left reference line (for vertical PIP), or on the left of the block and below and includes the top reference line (for horizontal perspective prediction). This condition can be mathematically expressed as: or, .

[0054] in A O It is an angle parameter. predIntraAngle ,ands It is equal to 5 or 6 respectively, depending on whether the sample resolution in the angular direction is (1 / 32) or (1 / 64) pixels. Therefore, if the angular parameter predIntraAngle If the above conditions are met, PDPC can be applied after the initial PIP mode.

[0055] PDPC applied in the case of vertical PIP Figure 16 As shown in the image. The initial predicted values ​​are modified as follows: in P(x,y) is the final perspective prediction value at pixel (x,y) after applying PDPC. P_0(x,y) is the initial PIP prediction at pixel (x,y), which is equal to the main reference sample R(x',-1). R(-1, y') is a secondary reference sample obtained by expanding the predicted direction at pixel (x, y). wL The weight function is calculated as follows. wL =32>>(2*x>>scale) Where the calculation parameters scale This allows secondary reference samples to be used for all target pixels in the first (3<< scale) column of the target block. Here, we use a weighting function similar to that used in VVC and ECM to more easily adapt to existing PDPCs with PIP. In general, it can be any other positively decreasing function.

[0056] The position of the secondary reference sample R(-1, y') is calculated as follows: For a (1 / 32) sample resolution, For a (1 / 64) sample resolution, When applying PDPC in the case of a vertical PIP, a similar operation is applied.

[0057] For more details on the application of PDPC with PIP mode, please refer to EP23306661.2, which is incorporated by reference.

[0058] As can be seen from the above, in order for the decoder to reconstruct blocks encoded using PIP mode, the encoder needs to signal information representing the perspective point. When using a single perspective point, the encoder signals the angle pattern as in regular intra-frame prediction mode, knowing that the decoder can receive the angle pattern and the perspective distances from the horizontal and vertical perspective. dxand dy (Decoder known) dx and dy Deriving the perspective point. Compared to the regular intra-frame prediction mode, no additional signaling is required in the PIP mode with a single perspective point. When multiple perspective points are possible for each angle mode, additional signaling is needed for the perspective point selected from the multiple perspective points.

[0059] The aforementioned intra-prediction modes are forward prediction tools, meaning that the intra-prediction method used at the encoder is explicitly signaled to the decoder. Recently, many tools employing backward estimation (i.e., using already decoded causal samples in the image to estimate the prediction modes) have been proposed. Decoder-side Intra-Mode Derivation (DIMD), Template-based Intra-Mode Derivation and Fusion (TIMD), and Spatial Geometric Segmentation Mode (SGPM) all employ backward estimation methods to estimate multiple prediction modes for a block. The encoder uses a minimal amount of signaling to indicate which modes to employ and, if necessary, performs a mixture of multiple predictions corresponding to multiple prediction modes. The fundamental assumption behind these methods is that not all parts of a block have the same orientation, and therefore using multiple predictions and then mixing those predictions can lead to better predictions. Estimating these modes in a backward manner eliminates the need for multiple prediction modes that can require significant overhead during transmission. This assumption seems reasonable for large blocks, as natural images can contain different orientations of object structures.

[0060] Decoder-side Intra-Modal Derivation (DIMD) is a novel tool for deriving intra-prediction patterns used to decode the current block. DIMD relies on the assumption that reconstructed samples surrounding the current block carry information inferring the texture orientation within that block. When DIMD is applied, up to five of the 65 angle patterns currently considered in intra-prediction that are likely the optimal angle patterns for predicting the current block are derived from the Histogram of Oriented Gradients (HoG) calculated based on the neighboring samples of the current block. The HoG is calculated with respect to a three-sample width / height L-shaped template formed by the already reconstructed samples, such as... Figure 8 As shown in the diagram. Predictions with these up to five angular patterns are combined (hybridized) with planar pattern predictors having weights derived from HoG.

[0061] Figure 9 This describes the process of deriving the DIMD predictor.

[0062] Figure 9 The process is applied by a processing module, which may be described later. Figure 5A The processing module 500.

[0063] In step 90, the processing module obtains the target block.

[0064] In step 91, the processing module obtains a three-sample width / height L-shaped template formed by the reconstructed samples adjacent to the target block. It can be noted that the shape of the template can be adaptive, depending on sample availability.

[0065] In step 92, the processing module derives the Oriented Gradient Histogram (HoG) from the samples of the template. For Figure 8 The HoG is obtained by summing the magnitudes of all gradients in a given direction for all samples within the dashed region and applying a Sobel filter. The bins of the HoG are then mapped to the “65” angle patterns used in intra-frame prediction.

[0066] In step 93, the processing module selects up to five angle modes with the highest cumulative amplitude in the HoG to obtain the DIMD mode.

[0067] In step 94, the processing module determines the weight of each selected DIMD mode. Each weight depends on the relative magnitude of the DIMD mode in the HoG.

[0068] In step 95, the processing module uses determined weights to calculate a weighted sum of the predictors obtained from each DIMD mode and for the planar mode to obtain the final predictor for the target block. All predictions are made using a first reference line, i.e., the reference array immediately adjacent to the target block.

[0069] Because DIMD modes are derived using information from neighboring reconstructed samples rather than the target block, the decoder can derive up to five optimal angle modes identical to the encoder's and can compute DIMD predictions as a weighted sum utilizing a planar mode predictor. For each intra-frame decoded block, a flag indicating whether a DIMD mode will be applied is used. DimdFlag Transmitted via signal.

[0070] Several variations of the DIMD pattern were proposed. For example, two additional DIMD patterns were proposed that allow the selection of different neighboring samples. DIMD_T and DIMD_L In DIMD_T mode, the top-left, top-right, and top-left adjacent samples are used as the reference region. In DIMD_L mode, the top-left, left, and bottom-left adjacent samples are used as the reference region. The number of rows and columns of samples in the reference region in both DIMD_T and DIMD_L modes is "4". The reference region used in these two modes is... Figure 10 As shown in the diagram. In each CU, the DIMD enable flag ( DimdFlag Then transmit the signal named cu_dimd_mode A new flag is used to determine which DIMD mode is being used.

[0071] Another variation is DIMD merging. When using DIMD merging, DIMD information extracted from neighboring blocks is used to compute the DIMD prediction for the current block. Specifically, a new merge gradient histogram (MHoG) is computed for the target block based on the HoG of neighboring blocks.

[0072] In TIMD, instead of explicitly transmitting the intra-prediction mode of the target block using a Most Probable Mode (MPM) list, the information is inferred from neighboring reconstructed samples of the target block at both the encoder and decoder. The template (made by...) Figure 11 The texture region indicator (in the template) specifies the set of reconstructed samples used to derive the intra-frame prediction mode. The template size is expressed as the number of samples within the template extending above and to the left of the current block. For example, a template size of "2" (i.e., ...) tW =2 or tH =2) is used for 4x4 and 8x8 blocks, and the template size is "4" (i.e., tW =4 or tH =4) Used for 16x16 and larger blocks. In some cases, when the left (top) neighboring blocks of the current block are not available, the stencil consists only of the textured area of ​​the top (left) of the current block.

[0073] For each intra-prediction mode in the MPM list, the decoder, based on the template reference (in... Figure 11 (Instructions) Calculate predictions. For each prediction, calculate the sum of absolute transform differences (SATD) between the prediction and the original reconstructed sample of the template. In the first step, select the two initial intra-prediction modes corresponding to the minimum SATD. The first initial intra-prediction mode with the minimum SATD is denoted as... And the initial intra-prediction mode corresponding to the second lowest SATD is represented as In the second step, the two initial intra-prediction modes are refined. For each of the two intra-prediction modes, if the initial intra-prediction mode is neither planar nor DC, SATD is calculated for the two closest extended angle modes (i.e., the two closest angle modes at (1 / 64) of the sample resolution of the initial angle mode). This results in two sets, each containing the initial angle mode (generated in the first step) and the two refined angle modes generated by the refinement. Within each set, the angle mode that minimizes SATD is selected. The derived angular pattern is represented as And from the pattern The derived angular pattern is represented as Therefore, for TIMD, by increasing the angular resolution to (1 / 64), the set of intra-prediction modes is expanded from "65" to "129". This is if the SATD of the two selected angular modes satisfies the condition. The predictions of the target block obtained using these two angle modes are then combined with weights to obtain the final TIMD prediction of the target block. (Using the mode...) The first prediction was obtained using the first reference line (the top and left reference arrays closest to the target block), while using the pattern... The second prediction is obtained using the second reference line (the closest top and left reference array). Weights are determined based on the relative magnitudes of the associated SATDs. Otherwise ( SATD IPM_timd_2nd ≥ 2∗ SATD IPM_timd_1st First Angle Mode Only Used to predict the target block as a TIMD pattern. In this case, using the pattern... Weighted MRL prediction is performed using two predictions based on the first and second reference lines, with weights respectively. and An encoder transmits signals and is called an encoder. timd_flag A special flag is used to indicate whether the TIMD model was used to obtain the best prediction. timd_ flag In the DIMD logo DimdFlag MIP logo and IntraTmp The flags are then CABAC encoded. Therefore, if the decoder decodes these flags as all zeros, it decodes... timd_flag If the flag is "1", it derives the two TIMD patterns exactly in the same way as the encoder. and Then, using the same SATD conditions described above, it determines whether to mix the two predictions. If mixing is performed, the mixing weights are the same as those used by the encoder. timd_ flag If the value is decoded as "0", the decoder will not deduce the TIMD mode.

[0074] Inter-frame prediction involves predicting the pixels of the current block from a block of pixels in an image preceding or following the current image, referred to as a reference block; this image is called the reference image. During decoding of the current block according to the inter-frame prediction method, a motion estimation step 304 determines the block in the reference image closest to the current block according to a similarity criterion. During step 304, a motion vector indicating the position of the reference block in the reference image is determined. This motion vector is used during a motion compensation step 305, during which a residual block is calculated as the difference between the current block and the reference block. In the first video compression standard, the above-described unidirectional inter-frame prediction mode was the only available inter-frame mode. As video compression standards have evolved, the family of inter-frame modes has grown significantly and now includes many different inter-frame modes.

[0075] During the selection step 306, the encoding module selects a prediction mode that optimizes compression performance in the tested prediction modes (intra-frame prediction mode, inter-frame prediction mode) according to the rate / distortion optimization standard (i.e., the RDO standard).

[0076] When a prediction mode is selected, the residual block is transformed during step 307. The transformed block is then quantized during step 309.

[0077] Note that the encoding module can skip the transform and apply quantization directly to the untransformed residual signal. When decoding the current block according to the intra-prediction mode, the entropy encoder encodes information indicating the selected intra-prediction mode during step 310. When encoding the current block according to inter-prediction, the block's motion vector is predicted, where appropriate, based on prediction vectors selected from a set of motion vector predictors derived from reconstructed blocks located spatially and temporally near the block to be encoded. The motion information is then encoded by the entropy encoder in the form of motion residuals and indices used to identify the prediction vectors during step 310. During step 310, the transformed and quantized residual block is encoded by the entropy encoder.

[0078] Note that the encoding module can bypass both transformation and quantization; that is, entropy coding can be applied to the residual without applying either transformation or quantization. The result of entropy coding is inserted into the encoded video stream 311 (i.e., the video data).

[0079] Metadata such as SEI (Supplemental Enhancement Information) messages can be attached to the encoded video stream.311 For example, an SEI message, as defined in standards such as AVC, HEVC, or VVC (or in the Standard General Supplemental Enhancement Information (VSEI) message used for decoding video bitstreams—H.274), is a data container or syntax structure associated with the video stream and containing metadata that provides information related to the video stream.

[0080] After quantization step 309, the current block is reconstructed so that the pixels corresponding to that block can be used for future predictions. This reconstruction stage is also known as... Prediction loop Therefore, inverse quantization is applied to the transformed and quantized residual block during step 312, and inverse transform is applied during step 313. The predictor block of the block is reconstructed based on the prediction mode used for the block obtained during step 314. If the current block is encoded according to an inter-frame prediction mode, the encoding module applies motion compensation using the motion vector of the current block as appropriate during step 316 to identify the reference block of the current block. If the current block is encoded according to an intra-frame prediction mode, the intra-frame prediction mode selected for the current block is used to reconstruct the predictor block of the current block during step 315. The predictor block and the reconstructed residual block are added together to obtain the reconstructed current block.

[0081] Following reconstruction, during step 317, an in-loop filter designed to reduce coding artifacts is applied to the reconstructed block. This filter is referred to as... In-loop filtering This filtering occurs within the prediction loop to obtain the same reference image as the encoder at the decoder, thus avoiding drift between the encoding and decoding processes. In-loop filtering tools include deblocking filtering, SAO (Sample Adaptive Shift), and ALF (Adaptive Loop Filtering).

[0082] When a block is reconstructed, it is inserted into a reconstructed image stored in memory 319, typically referred to as a decoded image buffer (DPB), during step 318. Thus, the stored reconstructed image can then be used as a reference image for other images to be encoded.

[0083] Figure 4 The schematic depiction illustrates the process performed by the decoding module for processing data based on... Figure 3 The described method is a method for decoding an encoded video stream (i.e., video data) 311. For example, Figure 4 The decoding method is executed by system 13. Variations of this method for decoding have been considered, but they are described below for clarity. Figure 4 The method used for decoding does not describe all expected variations.

[0084] Decoding is performed block by block. For the current block, it begins with entropy decoding of the current block during step 410. Entropy decoding allows at least the predicted pattern of the block to be obtained.

[0085] If the block has already been encoded according to the inter-frame prediction mode, entropy decoding allows the motion vector predictor index, motion residual, and prediction residual block to be obtained when appropriate. During step 408, the motion vector is reconstructed for the current block using the prediction vector index and motion residual.

[0086] If the block has already been encoded according to an intra-prediction mode (e.g., DIMD, TIMP, PIP, etc.), entropy decoding allows the prediction mode and prediction residual block to be obtained when appropriate. Steps 412, 413, 414, 415, 416, and 417 implemented by the decoding module are identical in all respects to steps 312, 313, 314, 315, 316, and 317 implemented by the encoding module.

[0087] In step 418, the decoded block is saved in the decoded image, and the decoded image is stored in DPB419. When the decoding module decodes a given image, the image stored in DPB419 is the same as the image stored in DPB319 by the encoding module during the encoding of the given image. The decoded image can also be sent by the decoding module as output, for example, for display.

[0088] After loop filtering (i.e., after generating the decoded image), post-processing step 421 can be applied.

[0089] Figure 5A , 5B 5C describes examples of devices, apparatuses, and / or systems that allow for the implementation of various embodiments.

[0090] Figure 5A An example of a hardware architecture for a processing module 500 capable of implementing an encoding or decoding module is schematically shown. This encoding or decoding module can be modified according to different aspects and embodiments. Figure 3 Methods for encoding and Figure 4 The method used for decoding. When the system is responsible for encoding the video stream, the encoding module is included in system 11, for example. The decoding module is included in system 13, for example.

[0091] The processing module 500 includes, via a communication bus 5005, a processor or CPU (Central Processing Unit) 5000, which includes one or more microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures, as non-limiting examples; random access memory (RAM) 5001; read-only memory (ROM) 5002; a storage unit 5003, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives and / or optical disk drives, or storage media readers such as SD (Secure Digital) card readers and / or hard disk drives (HDDs) and / or network-accessible storage devices; and at least one communication interface 5004 for exchanging data with other modules, devices, or systems. The communication interface 5004 may include, but is not limited to, a transceiver configured to transmit and receive data via a communication channel. The communication interface 5004 may include, but is not limited to, a modem or a network card.

[0092] If the processing module 500 implements a decoding module, then the communication interface 5004 enables, for example, the processing module 500 to receive an encoded video stream and provide a sequence of decoded images. If the processing module 500 implements an encoding module, then the communication interface 5004 enables, for example, the processing module 500 to receive a sequence of raw image data to be encoded and provide an encoded video stream.

[0093] Processor 5000 is capable of executing instructions loaded into RAM 5001 from ROM 5002, external memory (not shown), storage media, or a communication network. When processing module 500 is powered on, processor 5000 is capable of reading instructions from RAM 5001 and executing them. These instructions form a computer program that causes, for example, processor 5000 to implement... Figure 4 The described decoding method and / or about Figure 3 The described encoding method and about Figure 6A , Figure 6B and Figures 7 to 20 The methods shown include various aspects and embodiments described below in this document.

[0094] Figure 3 , Figure 4 , Figure 6A , Figure 6B and Figures 7 to 20 All or some of the algorithms and steps of the method can be implemented in software by executing the instruction set by a programmable machine such as a DSP (Digital Signal Processor) or a microcontroller, or in hardware by a machine or special component such as a FPGA (Field Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit).

[0095] As can be seen, microprocessors, general-purpose computers, special-purpose computers, processors based on or not based on multi-core architectures, DSPs, microcontrollers, FPGAs, and ASICs are suitable for or configured to at least partially implement Figure 3 , Figure 4 , Figure 6A , Figure 6B and Figures 7 to 20 Electronic circuits using this method.

[0096] Figure 5CA block diagram illustrating an example of a system 13 implementing various aspects and embodiments is shown. System 13 may be embodied as a device including the various components described below and configured to perform one or more aspects and embodiments described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and head-mounted displays. Elements of system 13 may be embodied individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, system 13 includes a processing module 500 implementing a decoding module. In various embodiments, system 13 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, system 13 is configured to implement one or more aspects described in this document.

[0097] As shown in box 531, input to the processing module 500 can be provided through various input modules. Such input modules include, but are not limited to, (i) a radio frequency (RF) module that receives RF signals transmitted over the air, for example by a broadcaster, (ii) a component (COMP) input module (or a collection of COMP input modules), (iii) a universal serial bus (USB) input module, and / or (iv) a high-definition multimedia interface (HDMI) input module. Figure 5C Other examples not shown include composite video.

[0098] In various embodiments, the input module of block 531 has associated corresponding input processing elements known in the art. For example, the RF module may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or band-limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) band-limiting it again to a narrower frequency band to select, for example, a signal band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF module in various embodiments includes one or more elements for performing these functions, such as a frequency selector, signal selector, band limiter, channel selector, filter, downconverter, demodulator, error corrector, and demultiplexer. The RF section may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband. In one set-top box embodiment, the RF module and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to a desired frequency band. Various embodiments rearrange the order of the aforementioned (and other) components, remove some of these components, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as, for example, inserting amplifiers and analog-to-digital converters. In various embodiments, the RF module includes an antenna.

[0099] Additionally, the USB and / or HDMI modules may include corresponding interface processors for connecting system 13 to other electronic devices across USB and / or HDMI connections. It should be understood that various aspects of input processing, such as Reed-Solomon error correction, may be implemented as needed, for example, within a separate input processing IC or within processing module 500. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed, either within a separate interface IC or within processing module 500. Demodulation, error correction, and demultiplexing streams are provided to processing module 500.

[0100] Various components of system 13 can be provided within an integrated housing. Within the integrated housing, the various components can be interconnected and transmit data between them using suitable connection arrangements, such as internal buses known in the art, including inter-IC (I2C) buses, wiring, and printed circuit boards. For example, in system 13, processing module 500 is interconnected to other components of system 13 via bus 5005.

[0101] The communication interface 5004 of the processing module 500 allows the system 13 to communicate on the communication channel 12. As described above, the communication channel 12 can be implemented, for example, in a wired and / or wireless medium.

[0102] In various embodiments, data is streamed or otherwise provided to system 13 using a wireless network such as a Wi-Fi network, for example, IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). Wi-Fi signals from these embodiments are received via a communication channel 12 and communication interface 5004 suitable for Wi-Fi communication. The communication channel 12 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top communications. Other embodiments use the RF connection of input block 531 to provide streaming data to system 13. As described above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0103] System 13 can provide output signals to various output devices, including display system 15, speakers 535, and other peripheral devices 536. Display system 15 in various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a flexible display, and / or a foldable display. Display 15 can be used in televisions, tablet computers, laptop computers, cellular phones (mobile phones), head-mounted displays, or other devices. Display system 15 can also be integrated with other components (e.g., as in a smartphone) or separate (e.g., an external monitor for a laptop computer). In various examples of embodiments, other peripheral devices 536 include one or more of a standalone digital video disc (or digital multifunction disc) (DVR, for both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 536 that provide functionality based on the output of system 13. For example, a disc player performs the function of playing the output of system 13.

[0104] In various embodiments, signaling is used to transmit control signals between system 13 and display system 15, speaker 535, or other peripheral devices 536. This signaling may be AV.Link, Consumer Electronics Control (CEC), or other communication protocols enabling device-to-device control with or without user intervention. Output devices can be communicatively coupled to system 13 via dedicated connections through corresponding interfaces 532, 533, and 534. Alternatively, output devices can use communication channel 12 via communication interface 5004 or via communication interface 5004 to... Figure 5CA dedicated communication channel corresponding to communication channel 12 is connected to system 13. Display system 15 and speaker 535 can be integrated into a single unit with other components of system 13 in an electronic device such as a television. In various embodiments, display interface 532 includes a display driver, such as, for example, a timing controller (TCon) chip.

[0105] Display system 15 and speaker 535 may alternatively be separated from one or more other components. In various examples in which display system 15 and speaker 535 are external components, output signals may be provided via dedicated output connections, including, for example, HDMI ports, USB ports, or COMP outputs.

[0106] Figure 5B A block diagram illustrating an example of system 11 implementing various aspects and embodiments is shown. System 11 is very similar to system 13. System 11 can be embodied as a device including the various components described below and configured to perform one or more aspects and embodiments described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, cameras, and servers. Elements of system 11 can be embodied individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, system 11 includes a processing module 500 implementing an encoding module. In various embodiments, system 11 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, system 11 is configured to implement one or more aspects described in this document.

[0107] Input to the processing module 500 can be provided through various input modules, such as those already specified. Figure 5C As indicated in box 531 of the description.

[0108] Various components of system 11 can be provided within an integrated housing, within which the various components can be interconnected and transmit data between them using a suitable connection arrangement, such as an internal bus known in the art, including inter-IC (I2C) buses, wiring, and printed circuit boards. For example, in system 11, processing module 500 is interconnected to other components of system 11 via bus 5005.

[0109] The communication interface 5004 of the processing module 500 allows the system 11 to communicate on the communication channel 12.

[0110] In various embodiments, a wireless network, such as a Wi-Fi network, is used to stream or otherwise provide data to system 11. The Wi-Fi network is, for example, IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). Wi-Fi signals from these embodiments are received via a communication channel 12 and a communication interface 5004 suitable for Wi-Fi communication. The communication channel 12 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top communications. Other embodiments use an RF connection of input block 531 to provide streaming data to system 11.

[0111] As described above, various embodiments provide data in a non-streaming manner. Furthermore, various embodiments utilize wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0112] The data provided to system 11 may be provided in different formats. In various embodiments, this data is encoded and conforms to known video compression formats such as AV1, VP9, ​​VVC, HEVC, AVC, etc. In various embodiments, this data is raw data provided, for example, by an image and / or audio acquisition module connected to or included in system 11. In this case, processing module 500 is responsible for encoding this data.

[0113] System 11 can provide output signals to various output devices such as System 13 that are capable of storing and / or decoding output signals.

[0114] Various implementations involve decoding. As used herein, "decoding" can encompass all or part of a process performed, for example, on a received encoded video stream to produce a final output suitable for display. In various examples, such a process includes one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and prediction. In various embodiments, such a process also includes, or alternatively includes, processes performed by a decoder of the various implementations described herein, such as applying DIMD or a combination of TIMD and PIP according to embodiments of this application.

[0115] Based on the specific context of the description, it will be clear whether the phrase "decoding process" is intended to specifically refer to a subset of the operations or generally to the broader decoding process, and is considered well understood by those skilled in the art.

[0116] Various implementations involve encoding. In a manner similar to the above discussion of “decoding,” the term “encoding” as used herein can encompass all or part of the process performed on an input video sequence to produce an encoded video stream. In various embodiments, such a process includes one or more processes typically performed by an encoder, such as segmentation, prediction, transform, quantization, and entropy coding. In various embodiments, such a process also includes, or alternatively includes, processes performed by an encoder of the various implementations described herein, such as applying DIMD or a combination of TIMD and PIP according to embodiments of this application.

[0117] Based on the specific context of the description, it will be clear whether the phrase "encoding process" is intended to specifically refer to a subset of the operations or generally to the broader encoding process, and is considered well understood by those skilled in the art.

[0118] Note that the grammatical element names used in this article are descriptive terms. Therefore, they do not preclude the use of other grammatical element names.

[0119] When the accompanying drawings are presented as flowcharts, it should be understood that they also provide block diagrams of the corresponding apparatus. Similarly, when the accompanying drawings are presented as block diagrams, it should be understood that they also provide flowcharts of the corresponding methods / processes.

[0120] Various embodiments involve rate distortion optimization. Specifically, a balance or trade-off between rate and distortion is typically considered during the encoding process. Rate distortion optimization is generally formulated as minimizing a rate distortion function, which is a weighted sum of rate and distortion. Different methods exist for solving the rate distortion optimization problem. For example, these methods can be based on extensive testing of all encoding options, including all considered mode or decoding parameter values, to fully evaluate their decoding costs and the associated distortion of the reconstructed signal after decoding and decoding. Faster methods can also be used to reduce encoding complexity, particularly by utilizing the computation of approximate distortion based on the predicted or predicted residual signal rather than the reconstructed signal. A hybrid of these two methods can also be used, such as by using approximate distortion only for some possible encoding options and full distortion for others. Other methods evaluate only a subset of possible encoding options. More generally, many methods employ any of a variety of techniques to perform optimization, but the optimization is not necessarily a complete evaluation of decoding costs and associated distortion.

[0121] The implementations and aspects described herein can be implemented, for example, in a method or process, apparatus, software program, data stream, or signal. Even if discussed only in the context of a single implementation form (e.g., discussed only as a method), the implementation of the features in question can also be implemented in other forms (e.g., apparatus or program). An apparatus can be implemented, for example, in appropriate hardware, software, and firmware. A method can be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. A processor also includes communication devices, such as, for example, a computer, a cellular phone, a portable / personal digital assistant (“PDA”), and other devices that facilitate information communication between end users.

[0122] References to “an embodiment” or “an embodiment” or “an implementation” or “an implementation” and their variations mean that a particular feature, structure, characteristic, and such is included in at least one embodiment. Therefore, the phrases “in an embodiment” or “in an embodiment” or “in an implementation” or “in an implementation” appearing throughout this application and any other variations do not necessarily refer to the same embodiment.

[0123] Additionally, this application may involve "determining" various types of information. Determining information may include, for example, estimation information, calculation information, prediction information, information retrieved from memory, or information obtained, for example, from another device, module, or user.

[0124] Furthermore, this application may involve "accessing" various types of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information, or one or more of these.

[0125] Additionally, this application may involve "receiving" various types of information. Like "access," receiving is intended to be a broad term. Receiving information may include, for example, accessing information or retrieving information (e.g., from memory) or one or more of it. Furthermore, "receiving" is generally involved in one or more ways during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0126] It should be understood that, for example, the use of any of the above " / ", "and / or", and "at least one of ...", "one or more of ..." in the cases of "A / B", "A and / or B", "at least one of A and B", and "one or more of ..." is intended to cover selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As another example, in the cases of "A, B and / or C", "at least one of A, B and C", and "one or more of A, B and C", this wording is intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A, B, and C). As will be apparent to those skilled in the art and related fields, this can be extended to as many items as listed.

[0127] Furthermore, as used herein, the term "signal" refers, among other things, to instructing a corresponding decoder to do something. For example, in some embodiments, an encoder signals the use of certain decoding tools. In this way, in embodiments, the same parameters can be used on both the encoder and decoder sides. Thus, for example, the encoder can transmit (explicit signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. Conversely, if the decoder already has specific parameters as well as other parameters, signaling can be used without transmission (implicit signaling) to simply allow the decoder to know and select specific parameters. Bit savings are achieved in various embodiments by avoiding the transmission of any actual functionality. It should be understood that signaling can be implemented in a variety of ways. For example, in various embodiments, one or more syntax elements, flags, and the like are used to signal information to the corresponding decoder. While the foregoing refers to the verb form of the term "signal," the term "signal" can also be used as a noun herein.

[0128] It will be apparent to those skilled in the art that implementations can generate various signals that are formatted to carry information, for example, that can be stored or transmitted. This information may include, for example, instructions for performing a method or data generated by one of the described implementations. For example, a signal may include a signal indicating a selected intra-frame prediction mode. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. Formatting may include, for example, encoding a coded video stream and modulating a carrier wave with the coded video stream. The information carried by the signal may be, for example, analog or digital information. As is known, signals can be transmitted via a variety of different wired or wireless links. Signals may be stored on a processor-readable medium.

[0129] In the following, various embodiments propose using the procedures applied in DIMD and TIMD modes to reduce the overhead required to signal perspective points in PIP mode.

[0130] Figure 12 The process for encoding a target block according to the first embodiment is illustrated schematically.

[0131] When system 11 implements the encoder Figure 12 The process is executed by the processing module 500 of system 11. The encoder, for example, has step 303, which implements the modifications of the first embodiment. Figure 3 The encoder.

[0132] In step 1200, the processing module 500 obtains the target block to be encoded.

[0133] In steps 1201, 1202, and 1203, processing module 500 determines the DIMD mode without PIP (hereinafter referred to as...). generally DIMD mode The combination of DIMD mode and PIP, hereinafter referred to as DIMD-based PIP And when these patterns are applied to the target block, by Figure 3 The encoder implements RD performance for other prediction modes. In the example of the first embodiment, when DIMD is applied (with a typical DIMD mode or a DIMD-based PIP mode), only the first reference line is used to obtain the prediction.

[0134] Here, step 1202, the DIMD-based PIP pattern, includes a process of at least partially using DIMD patterns until at least one angular pattern is obtained. At least one perspective point is then determined from the at least one angular pattern. Each determined perspective point is then used to determine a PIP predictor (the predictor obtained by applying the PIP pattern). In this embodiment, a single angular pattern is determined. This single angular pattern corresponds to the direction with the highest cumulative amplitude in the HoG obtained through the DIMD application process.

[0135] In the first embodiment, in the DIMD-based PIP pattern, there is no mixing of the PIP predictor with other predictions obtained, for example, by using other patterns selected using the usual DIMD process. The PIP predictor is the final PIP predictor of the DIMD-based PIP pattern.

[0136] In this embodiment, PDPC is applied after DIMD-based PIP prediction.

[0137] In step 1201, the usual DIMD process is applied to obtain the DIMD predictor for the target block.

[0138] In step 1204, processing module 500 determines whether the DIMD-based PIP mode provides optimal RD performance. If so, in step 1205, a DIMD-based PIP predictor is used to predict the target block. Furthermore, in step 1205, processing module 500 transmits a flag in the bitstream (in the video data) using a signal. DimdFlag Equal to "1", enable DIMD for the target block using semaphore, and use semaphore flags. perpectiveFlag The value is "1", which signals the final DIMD-based PIP predictor applied to predict the target block.

[0139] If the DIMD-based PIP mode does not provide optimal RD performance, processing module 500 continues to step 1206. During step 1206, processing module 500 determines whether the normal DIMD mode provides optimal RD performance. If so, processing module 500 uses a DIMD predictor (obtained using the normal DIMD process) to predict the target block in step 1207. Additionally, in step 1207, processing module 500 sends a signal to a flag. DimdFlag The value is "1", which enables DIMD for the target block using semaphore and uses a semaphore flag. perspectiveFlag The value is equal to "0", which is the signal typically used by DIMD predictors to predict target blocks.

[0140] If the standard DIMD mode does not provide optimal RD performance, processing module 500 proceeds to step 1208. During step 1208, processing module 500 applies another intra-frame or inter-frame prediction mode to predict the target block and transmits a flag. DimdFlag Equal to "0" (no signal transmission flag) perspectiveFlag ).

[0141] Figure 13 The process for decoding a target block according to the first embodiment is illustrated schematically.

[0142] When system 13 implements the decoder Figure 13 The process is executed by the processing module 500 of system 13. The decoder, for example, is an implementation of various embodiments with modified step 415. Figure 4 The decoder.

[0143] In step 1300, processing module 500 decodes the flag. DimdFlag .

[0144] In step 1301, the processing module 500 determines the flag. DimdFlag Is it equal to "1"?

[0145] Response to Flags DimdFlagEqual to "1", processing module 500 decoding flag perspectiveFlag .

[0146] In step 1304, the processing module 500 determines the flag. perspectiveFlag Is it equal to "1"?

[0147] If the logo perspectiveFlag If the value is "1", then in step 1307, processing module 500 applies the DIMD process to determine a single angular pattern corresponding to the direction with the highest cumulative amplitude in the HoG. Based on the determined angular pattern, processing module 500 determines the perspective point and applies the PIP pattern to obtain a DIMD-based PIP predictor. The DIMD-based PIP predictor is then used to predict the target block.

[0148] In this embodiment, PDPC is applied after DIMD-based PIP prediction.

[0149] If the logo perspectiveFlag If the value is "0", then in step 1306, the processing module 500 applies the normal DIMD procedure to determine the DIMD predictor. The DIMD predictor is then used to predict the target block.

[0150] Response to Flags DimdFlag If the expression is "0", the processing module 500 decodes other syntax elements in step 1302 and determines a predictor based on the decoded syntax elements. The obtained predictor is used to predict the target block.

[0151] In the above embodiments, both the standard DIMD mode and the DIMD-based PIP mode apply a first reference line, i.e., using a reference array immediately above and to the left of the target block. In the second embodiment, the constraints on the reference line are relaxed. Predictions in both the DIMD mode and the DIMD-based PIP mode are applied using multiple reference lines, for example, reference lines with indices "0", "1", and "3". Here, the reference line with index N consists of a reference array offset by "N" pixels above and to the left of the target block. The second embodiment includes, in addition to the flag... dimdFlag and symbols perspectiveFlag Signaling of the reference line index outside of the signaling.

[0152] Figure 14 The process for encoding a target block according to the second embodiment is illustrated schematically.

[0153] When system 11 implements the encoder Figure 14 The process is executed by the processing module 500 of system 11. The encoder, for example, implements step 303 of the second embodiment. Figure 3 The encoder.

[0154] Figure 14 The process and Figure 12 The process is the same, except that steps 1202, 1205 and 1207 are replaced by steps 1402, 1405 and 1407 respectively.

[0155] In step 1402, the processing module 500 determines the reference line index. multiRefIdx_PIP This allows for optimal RD performance in the DIMD-based PIP mode.

[0156] In step 1405, a DIMD-based PIP predictor is used to predict the target block. Additionally, in step 1405, the processing module 500 transmits a flag in the bitstream (in the video data) using a signal. DimdFlag Equal to "1", this uses signal transmission to enable DIMD for the target block, syntax element multiRefIdx equal multirefIdx_PIP This uses signals to transmit the optimal reference line and markers. perspectiveFlag Equal to "1", this is used to signal the application of the DIMD-based PIP predictor to predict the target block.

[0157] In step 1407, the target block is predicted using a DIMD predictor obtained by applying a typical DIMD mode. Additionally, in step 1407, the processing module 500 transmits a flag in the bitstream (in the video data) using a signal. DimdFlag Equal to "1", this uses signal transmission to enable DIMD for the target block, syntax element multiRefIdx The value is "0", which should be transmitted using the first reference line and the flag. perspectiveFlag The value is "0", which means that the target block should be predicted using a DIMD predictor when the signal is transmitted.

[0158] Figure 15 The process for decoding a target block according to the second embodiment is illustrated schematically.

[0159] When system 13 implements the decoder Figure 15 The process is executed by the processing module 500 of system 13. The decoder, for example, implements the second embodiment with a modified step 415. Figure 4 The decoder.

[0160] about Figure 13 All steps described remain in Figure 15 And keep it the same during the process.

[0161] Step 1300 is followed by step 1500. During step 1500, processing module 500 processes syntax elements. multiRefIdx Decode it.

[0162] Step 1500 is followed by step 1301. If the flag... dimdFlag If the value equals 1, then processing module 500 continues to step 1501. During step 1501, processing module 500 determines the syntax element. multiRefIdx Is it equal to "0"?

[0163] Responding to syntax elements multiRefIdx Unlike "0", processing module 500 directly knows that a DIMD-based PIP mode will be applied. Therefore, step 1307 follows step 1501.

[0164] Responding to syntax elements multiRefIdx If the value is "0", the processing module 500 will continue as follows. Figure 13 Steps 1304, 1305, and 1306 or 1307 in the process.

[0165] So far, we have considered that in DIMD-based PIP patterns, the DIMD process is used to determine a single angular pattern corresponding to the direction with the highest cumulative magnitude in the HoG. In the third embodiment, during steps 1202 or 1402, processing module 500 selects up to five directions with the highest cumulative magnitude in the HoG. Each selected direction allows for the determination of an angular pattern. The determined angular patterns allow for the determination of a set of perspective points. Considering each perspective point of the set to which PIP patterns are applied, and the perspective point that provides the best DIMD-based PIP predictor (the DIMD-based PIP predictor provides the best RD performance among all DIMD-based PIP predictors) is the final DIMD-based PIP predictor. If this final DIMD-based PIP predictor provides the best predictor for the target block, then in step 1205 or 1405, processing module 500 performs a process on the syntax elements. modeFlag Encode the syntax element modeFlag This indicates which of the five directions in the HoG has been selected that has the highest cumulative magnitude. Due to the syntax element... modeFlag The decoder knows which direction to consider when applying a DIMD-based PIP mode to determine the perspective point to use. In a variant, the number of directions that can be selected by processing module 500 in steps 1202 or 1402 is limited to two (these two directions correspond to the two highest cumulative magnitudes in the HoG). In this case, the syntax element... modeFlag The first value indicates that the direction corresponding to the highest amplitude in the HoG is selected, and the second value indicates that the second highest amplitude in the HoG is selected.

[0166] In a fourth embodiment, which can be considered a variation of the second and third embodiments, the direction corresponding to the highest amplitude in the HoG is associated with the first reference line, and the direction corresponding to the second highest amplitude in the HoG is associated with the second reference line. Therefore, by knowing the syntax elements... multiRefIdx The value of is sufficient to know which direction the encoder has selected. Therefore, there is no need to send additional syntax elements indicating the selected direction using signals. modeFlag ).

[0167] Typical DIMD prediction can be restricted to using only the first reference line, or it can be extended to include both the first and second reference lines. In the case where typical DIMD prediction is limited to the first reference line, when the flag... dimdFlag Equal to "1" and syntax element multiRefIdx When it is greater than "0", no flag is needed. perspectiveFlag Encoding is performed (similar to the second embodiment where the processing module 500 directly applies step 1307 after step 1501).

[0168] In the fifth embodiment, a variation of the fourth embodiment, five directions corresponding to the five highest amplitudes in the HoG are considered. Signaling depends on the number of reference lines used and the direction-to-reference line assignment. If the assignment is one-to-one, i.e., each direction is used with a different reference line, then no additional syntax element is needed to transmit the direction with the signal; the syntax element... multiRefIdx This is used to transmit reference lines and selected directions using signals (similar to the fourth embodiment). However, if multiple directions are using the same reference line, or if multiple reference lines are possible for each direction, a suitable signaling method is needed to distinguish between directions. For example, reference line "0" can be used with the direction corresponding to the highest amplitude in the HoG, reference line "1" can be used with the direction corresponding to the second and third highest amplitudes in the HoG, and reference line "2" can be used with the direction corresponding to the fourth and fifth highest amplitudes in the HoG. Therefore, in response to reference lines "1" and "2", the syntax element... multiRefIdx Use signals to transmit syntax elements. multiRefIdx Then a 1-bit flag is transmitted via signal ( modeFlag () to indicate the prediction mode used by the encoder.

[0169] In the sixth embodiment, at least a portion of the process of applying the TIMD pattern is used to determine the perspective point of the PIP pattern.

[0170] Figure 17 The process for encoding a target block according to the sixth embodiment is illustrated schematically.

[0171] When system 11 implements the encoder Figure 17 The process is executed by the processing module 500 of system 11. The encoder, for example, implements step 303 of the first embodiment with modifications. Figure 3 The encoder.

[0172] In step 1700, the processing module 500 obtains the target block to be encoded.

[0173] In steps 1701, 1702, and 1703, processing module 500 determines the TIMD pattern as defined in the ECM (hereinafter referred to as...). Conventional TIMD The combination of TIMD mode and PIP, referred to below as TIMD mode, is a pattern. PIP mode based on TIMD The RD performance, wherein at least a portion of the TIMD mode process allows for the determination of the perspective points used by the PIP mode and the performance of the target block when these modes are applied. Figure 3 The encoder implements perspective points for other prediction modes. In the example of the sixth embodiment, when applying the usual TIMD mode or the TIMD-based PIP mode, only the first reference line is used to obtain the prediction.

[0174] During step 1702, the processing module 500 determines the first TIMD mode, namely Is it an angle mode (neither planar nor DC)? If it's the first TIMD mode... If it is angle mode, then the processing module 500 has the first TIMD mode in the application. The PIP pattern is used to determine RD performance when obtaining a TIMD-based PIP predictor. If the first TIMD pattern... If it is not directional mode but planar or DC mode, then processing module 500 (optionally) checks the secondary TIMD mode. If the secondary TIMD flag If it is angle mode, then the processing module 500 has a second TIMD flag in the application. The PIP pattern is used to determine RD performance when obtaining a TIMD-based PIP predictor. If the secondary pattern... If it is not an angle mode, then processing module 500 (optionally) applies the PIP mode using a strict vertical mode represented as EXT_VER_IDX. Then, processing module 500 determines RD performance when applying the PIP mode with mode EXT_VER_IDX to determine the TIMD-based PIP predictor.

[0175] In the variant, if the first TIMD mode If it is not an angle mode, then processing module 500 decides not to apply the TIMD-based PIP mode. In the second variation, if the first TIMD mode... Second TIMD mode If it is not angle mode, then the processing module 500 decides not to apply the TIMD-based PIP mode.

[0176] In step 1701, processing module 500 applies the TIMD mode defined in the ECM. If the mixing conditions are met, it uses the first reference line to... Perform the first forecast and use the second reference line to Perform a second prediction and combine the two predictions.

[0177] In step 1704, processing module 500 determines whether the TIMD-based PIP mode provides optimal RD performance. If so, in step 1705, a TIMD-based PIP predictor is used to predict the target block. Furthermore, in step 1705, processing module 500 transmits a flag in the bitstream (in the video data) using a signal. timd_flag The value is "1", which enables TIMD for the target block using semaphore and uses a semaphore flag. perspectiveFlag Equal to "1", this is used by the TIMD-based PIP predictor to predict the target block.

[0178] If the TIMD-based PIP pattern does not provide optimal RD performance, processing module 500 continues to step 1706. During step 1706, processing module 500 determines whether the normal TIMD pattern provides optimal RD performance. If so, processing module 500 uses a TIMD predictor (obtained using the normal TIMD process) to predict the target block in step 1707. Additionally, in step 1707, processing module 500 sends a signal to a flag. timd_flag The value is "1", which enables TIMD for the target block using semaphore and uses a semaphore flag. perspectiveFlag The value is "0", which is the signal typically used by TIMD predictors to predict target blocks.

[0179] If the TIMD mode does not provide optimal RD performance, processing module 500 continues to step 1708. During step 1708, processing module 500 applies another intra-frame or inter-frame prediction mode to predict the target block and transmits a flag using a signal. timd_ flag Equals "0" (no signal transmission flag) perspectiveFlag ).

[0180] Figure 18 The process for decoding a target block according to the sixth embodiment is illustrated schematically.

[0181] When system 13 implements the decoder Figure 18 The process is executed by the processing module 500 of system 13. The decoder is, for example, an implementation of various embodiments with modified step 415. Figure 4 The decoder.

[0182] In step 1800, processing module 500 decodes the flag. timd_flag .

[0183] In step 1801, the processing module 500 determines the flag. timd_flag Is it equal to "1"?

[0184] Response to Flags timd_flag Equal to "1", processing module 500 decoding flag perspectiveFlag .

[0185] In step 1804, the processing module 500 determines the flag. perspectiveFlag Is it equal to "1"? If the flag... perspectiveFlag If the value is "1", then in step 1807, the processing module 500 applies the TIMD procedure to determine the angle mode (e.g., in the first TIMD mode). Second TIMD mode (Between the EXT_VER_IDX and the mode). Based on the determined angle mode, the processing module 500 determines the perspective point and applies the PIP mode to obtain a TIMD-based PIP predictor. The TIMD-based PIP predictor is then used to predict the target block.

[0186] If the logo perspectiveFlag If the value is "0", then in step 1806, processing module 500 applies the normal TIMD procedure to determine the TIMD predictor. The TIMD predictor is then used to predict the target block.

[0187] Response to Flags timd_flag If the value is "0", the processing module 500 decodes other syntax elements in step 1802 and determines a predictor based on the decoded syntax elements. The obtained predictor is used to predict the target block.

[0188] In the sixth embodiment described above, the TIMD-based PIP pattern is applied with a first reference line, i.e., using a reference array immediately above and to the left of the target block. In the seventh embodiment, the constraints on the reference lines are relaxed. The TIMD-based PIP pattern is applied with multiple reference lines, for example, with reference lines having indices "0", "1", and "3". The number of reference lines to be checked is variable and depends on their availability. The seventh embodiment includes, in addition to flags... timdFlag and symbols perspectiveFlag Signaling of the reference line index outside of the signaling.

[0189] Figure 19 The process for encoding a target block according to the seventh embodiment is illustrated schematically.

[0190] When system 11 implements the encoder Figure 19 The process is executed by the processing module 500 of system 11. The encoder is, for example, an implementation of the second embodiment with modified step 303. Figure 3 The encoder.

[0191] Figure 19 The process and Figure 17 The process is the same, except that steps 1702 and 1705 are replaced by steps 1902 and 1905, respectively.

[0192] In step 1902, processing module 500 determines a reference line index that allows for optimal RD performance of the TIMD-based PIP mode. multiRefIdx_PIP .

[0193] In step 1905, a TIMD-based PIP predictor is used to predict the target block. Additionally, in step 1905, the processing module 500 transmits a flag in the bitstream (in the video data) using a signal. timd_flag Equal to "1", this uses signal transmission to enable TIMD for the target block, flag. perspectiveFlag The value is "1", which is used by the TIMD-based PIP predictor to predict the target block and syntax elements. multiRefIdx equal multirefIdx_PIP This uses a signal to transmit the optimal reference line.

[0194] Figure 20 The process for decoding a target block according to the seventh embodiment is illustrated schematically.

[0195] When system 13 implements the decoder Figure 20 The process is executed by the processing module 500 of system 13. The decoder is, for example, an implementation of the second embodiment with a modified step 415. Figure 4 The decoder.

[0196] Retain information Figure 18 All the steps are described, but step 2000 is added.

[0197] Step 1805 is followed by step 2000, during which the processing module 500, when applying the TIMD-based PIP pattern, modifies the syntax elements indicating the reference lines to be used. multiRefIdx Decode it.

[0198] Step 2000 is followed by step 1807, during which processing module 500 applies the TIMD process to determine the angle mode (e.g., in the first TIMD mode). Second TIMD mode (between the EXT_VER_IDX and the mode). Based on the determined angle mode, the processing module 500 determines the perspective point and applies the PIP mode to use the syntax elements. multiRefIdx The reference line is used to obtain a TIMD-based PIP predictor. The TIMD-based PIP predictor is then used to predict the target block.

[0199] In a variant of the seventh embodiment, the processing module 500 (implementing the encoder) of system 11 applies the usual TIMD pattern only with the first reference line; that is, it uses the first reference line to determine the TIMD pattern based on the first reference line. TIMD predictor and second TIMD pattern based Both TIMD predictors. In this variant, only when the flag... timd_flag Equal to "1" and syntax element multiRefIdx A signal is used to transmit a flag only when the value is "0". perspectiveFlag If the sign timd_flag It equals "1", but the syntax element multiRefIdx If the value is greater than "0", then the processing module 500 of system 11 does not need to transmit the signal flag. perspectiveFlag In fact, if the mark timd_flag Equal to "1" and syntax element multiRefIdx If the value is greater than "0", then the processing module 500 (which implements the decoder) of system 13 knows the flag. perspectiveFlag It equals "1".

[0200] In another variation of the seventh embodiment, the processing module 500 (implementing the encoder) of system 11 uses the first reference line in a first TIMD mode. Apply PIP prediction and use a second reference line in a secondary TIMD mode. PIP prediction is applied. If either of them is a planar or DC mode, the processing module 500 of system 11 uses the strictly vertical mode EXT_VER_IDX instead of the first reference line, and if it replaces the TIMD mode... If so, and instead of the second reference line, if it replaces the secondary TIMD mode. If using the TIMD predictor obtained using the usual TIMD pattern or the TIMD-based PIP predictor results in the best RD performance for the target block, then the processing module 500 of system 11 will flag... timd_flag It is transmitted as "1" using a signal. It also indexes the reference line. multiRefIdxEncode regardless of whether the usual TIMD pattern is applied with multiple reference lines. If the flag timd_flag Equal to "1" and syntax element multiRefIdx If it equals "0", then its encoding flag is... perspectiveFlag If the sign timdFlag Equal to "1" and syntax element multiRefIdx If the value is "1", then in the normal TIMD mode, the second reference line is also used. multiRefIdx =1) Signal transmission of flags when permitted. perspectiveFlag Otherwise, there is no need to use signals to transmit flags. perspectiveFlag If the sign timd_flag The decoded value is equal to "1" and if the syntax element multiRefIdx If the value is "0", then the processing module 500 (which implements the decoder) of system 13 will check the flag. perspectiveFlag Decode it. Conversely, if the flag timd_flag and syntax elements multiRefIdx If both are equal to "1", then the processing module 500 of system 13 will only decode the flag if the normal TIMD mode is also allowed at the second reference line. perspectiveFlag Otherwise, it will mark perspectiveFlag The value is inferred to be "1". If the flag timd_flag If the decoded value is "0", then the processing module 500 of system 13 will mark it. perspectiveFlag The value is inferred to be "0", which is the default value.

[0201] In the eighth embodiment, similar to the fifth DIMD-based embodiment, a TIMD process is used to determine more than two angle patterns, while maintaining the angle patterns that provide the best TIMD-based PIP predictor in terms of RD performance. For example, from the angle patterns with the lowest SATD... Determine the first TIMD mode From the perspective of the second lowest SATD mode Determine the second TIMD mode And from the perspective of the third lowest SATD mode Determine the third TIMD mode If the maintained TIMD-based PIP predictor provides optimal RD performance for the target block, then the processing module 500 of system 11 uses a signal to indicate that the timd_flag is equal to "1" and the flag is... perspectiveFlag This equals "1" and includes information indicating the selected angle mode. Different reference lines are associated with each TIMD mode (e.g., the first reference line is associated with the first angle mode). Related, the second reference line and the second angle pattern Related, and the third reference line is consistent with the third angle pattern. (related), then syntax elements multiRefIdx It can be used to transmit the selected angle mode using a signal to apply a TIMD-based PIP mode.

[0202] If any reference line can be associated with multiple angle patterns, then in addition to the syntax element indicating the reference line to be used... multiRefIdx In addition, signals are sent to the target block indicating which mode should be used for the syntax elements of the TIMD-based PIP mode. modeIdx .

[0203] In the above embodiments, a single perspective point is used to apply the TIMD-based PIP pattern. In another embodiment, multiple perspective points are used to apply the TIMD-based PIP pattern. In this case, the encoder transmits the index of the perspective point used in the PIP prediction via a signal.

[0204] We have described several embodiments above. The features of these embodiments may be provided individually or in any combination. Furthermore, embodiments may include one or more of the following features, devices, or aspects, individually or in any combination, across various claim classes and types: • Includes a bitstream or signal of one or more of the described syntax elements or their variants. • Create and / or transmit and / or receive and / or decode bit streams or signals including one or more of the described syntax elements or their variants. • A TV, set-top box, cellular phone, tablet computer or other electronic device that performs at least one of the described embodiments. • A TV, set-top box, cellular phone, tablet computer or other electronic device that performs at least one of the described embodiments and displays (e.g., using a monitor, screen or other type of display) the resulting image. • Tuning (e.g., using a tuner) a channel to receive signals including encoded video streams, and performing at least one of the described embodiments of a TV, set-top box, cellular phone, tablet computer, or other electronic device. • A TV, set-top box, cellular phone, tablet computer or other electronic device that receives signals including encoded video streams over the air (e.g., using an antenna) and performs at least one of the described embodiments. • A server, camera, cellular phone, tablet computer or other electronic device that transmits signals including encoded video streams over the air (e.g., using an antenna) and performs at least one of the described embodiments. • A server, camera, cellular phone, tablet computer or other electronic device that tunes (e.g., uses a tuner) to transmit signals including encoded video streams and performs at least one of the described embodiments.

Claims

1. A method for encoding, comprising: The backward intra-frame prediction process (1202) is applied until at least one angle pattern for the target block is obtained. Each backward intra-frame prediction process allows samples independent of the target block to estimate the corresponding angle pattern for the target block using already reconstructed samples in the causal neighborhood of the target block. At least one perspective point in the neighborhood of the target block is determined from at least one angle pattern. Each perspective point is selected on a straight line, its direction is indicated by one of the angle patterns and passes through the center of the target block. Each perspective point indicates the prediction direction applied to each sample of the target block. The perspective intra-frame prediction (PIP) of the reconstructed samples from the neighborhood of the target block is applied (1202) to generate a PIP predictor using at least one of the determined perspective points. Predict the target block (1205) using the PIP predictor.

2. A method for decoding, comprising: The backward intra-frame prediction process (1307) is applied until at least one angular pattern for the target block is obtained. Each backward intra-frame prediction process allows samples independent of the target block to estimate the corresponding angular pattern for the target block using already reconstructed samples in the causal neighborhood of the target block. At least one perspective point in the neighborhood of the target block is determined from at least one angular pattern. Each perspective point is selected on a straight line whose direction is indicated by one of the angular patterns and passes through the center of the target block. Each perspective point indicates the prediction direction applied to each sample of the target block. The perspective intra-frame prediction (PIP) of samples from the neighborhood of the target block is applied (1307) to generate a PIP predictor using at least one of the determined perspective points. Predict the target block (1307) using the PIP predictor.

3. The method according to claim 1 or 2, wherein, The backward intra-prediction process is the decoder-side intra-mode derivation (DIMD).

4. The method of claim 3, further comprising transmitting in the video data or decoding from the video data information indicating that DIMD is enabled for the target block and information indicating that the PIP predictor is applied to predict the target block.

5. The method according to claim 1 or 2, wherein, The backward intra-prediction process is based on template-based intra-mode derivation (TIMD).

6. The method of claim 5, further comprising transmitting in the video data or decoding from the video data information indicating that TIMD is enabled for the target block and information indicating that the PIP predictor is applied to predict the target block.

7. The method according to claim 1 or 3 to 6, or the method according to claims 2 to 6, wherein, The neighborhood of the target block is divided into multiple reference arrays, and the method includes signaling information representing one of the multiple reference arrays to be used for applying perspective intra-frame prediction.

8. The method according to claim 7 when subordinate to claim 1, 2, 3 or 5, wherein, At least one value representing information about one of the multiple reference arrays indicates that the PIP predictor is applied to predict the target block.

9. The method according to any of the preceding claims, comprising, in response to generating a plurality of perspective points, generating at least one additional PIP predictor, each additional PIP predictor being generated from a perspective point among the plurality of perspective points, and identifying a final PIP predictor that provides the best rate-distortion performance among the PIP predictors, the target block being predicted from the final PIP predictor.

10. The method of claim 9, comprising signaling information representing the final PIP predictor or decoding information representing the final PIP predictor.

11. The method according to claim 9 when subordinate to claim 7 or 8, wherein, Information representing one of multiple reference arrays allows for the derivation of information representing the final PIP predictor.

12. A device for encoding, comprising electronic circuitry configured to: The backward intra-frame prediction process (1202) is applied until at least one angle pattern for the target block is obtained. Each backward intra-frame prediction process allows samples independent of the target block to estimate the corresponding angle pattern for the target block using already reconstructed samples in the causal neighborhood of the target block. At least one perspective point in the neighborhood of the target block is determined from the angle pattern. Each perspective point is selected on a straight line, its direction is indicated by one of the angle patterns and passes through the center of the target block. Each perspective point indicates the prediction direction applied to each sample of the target block. The perspective intra-frame prediction (PIP) of samples from the neighborhood of the target block is applied (1202) to generate a PIP predictor using at least one of the determined perspective points. Predict the target block (1205) using the PIP predictor.

13. A device for decoding, comprising electronic circuitry configured to: The backward intra-frame prediction process (1307) is applied until at least one angle pattern for the target block is obtained. Each backward intra-frame prediction process allows samples independent of the target block to estimate the corresponding angle pattern for the target block using already reconstructed samples in the causal neighborhood of the target block. At least one perspective point in the neighborhood of the target block is determined from the angle pattern. Each perspective point is selected on a straight line, its direction is indicated by one of the angle patterns and passes through the center of the target block. Each perspective point indicates the prediction direction applied to each sample of the target block. The perspective intra-frame prediction (PIP) of samples from the neighborhood of the target block is applied (1307) to generate a PIP predictor using at least one of the determined perspective points. Predict the target block (1307) using the PIP predictor.

14. A signal representing video data, including information instructing decoder-side intra-frame mode derivation (DIMD) or template-based intra-frame mode derivation (TIMD) to be enabled for a target block to determine at least one angular mode, determining at least one perspective point in the neighborhood of the target block based on the at least one angular mode, and information instructing perspective intra-frame prediction (PIP) using samples from the neighborhood of the target block of at least one of the determined at least one perspective point to be used to predict the target block to be determined.

15. A non-transitory information storage medium storing program code instructions for implementing the method according to any one of the preceding claims 1 to 11.

Images