Image decoding apparatus, image decoding method, and program

Abstract

The image decoding device (200) according to the present application includes an inter prediction section (241) that derives motion information with respect to a geometric partition mode (GPM) and generates a motion compensated pixel, an intra prediction section (242) that derives an intra prediction mode with respect to the geometric partition mode (GPM) and generates an intra prediction pixel, and a prediction information buffer (244) that stores or outputs the motion information of a decoding target block to which the geometric partition mode (GPM) is applied or prediction information including the intra prediction mode and a prediction type capable of determining which of inter prediction and intra prediction is applied.

Description

Technical Field

[0001] This invention relates to an image decoding device, an image decoding method, and a program. Background Technology

[0002] Non-patent literature 1 discloses a GPM (Geometric Partitioning Mode).

[0003] GPM divides the rectangular block into two at an angle and performs motion compensation on each. Specifically, in GPM, the two segmented regions are motion compensated using motion vectors from a fusion pattern and then synthesized through a weighted average. Sixty-four patterns are prepared as the angled segmentation patterns, depending on the angle and position.

[0004] Existing technical documents

[0005] Patent documents

[0006] Non-patent literature 1: ITU-T H.266 / VVC

[0007] Non-patent literature 2: JVET-U0100, "Compression efficiency methods beyond VVC"

[0008] Non-patent literature 3: JVET-W0097, "EE2-related Combination of EE2-3.3, EE2-3.4 and EE2-3.5" Summary of the Invention

[0009] The problem that the invention aims to solve

[0010] However, the GPM disclosed in Non-Patent Document 1 is limited to the fusion mode, and therefore has the problem that there is room for improvement in coding performance.

[0011] Therefore, the present invention was made in view of the above-mentioned problems, and its object is to provide an image decoding apparatus, image decoding method and program that can improve coding performance by specifying prediction information and a method for storing prediction information when an intra-prediction mode is added to the GPM.

[0012] Methods for solving problems

[0013] The first technical solution of the present invention is an image decoding apparatus, the main feature of which is: an inter-frame prediction unit configured to derive motion information relative to a geometric segmentation pattern and generate motion-compensated pixels; an intra-frame prediction unit configured to derive an intra-frame prediction pattern relative to the geometric segmentation pattern and generate intra-frame prediction pixels; and a prediction information buffer configured to store or output motion information of a decoding target block to which the geometric segmentation pattern is applied, or prediction information including an intra-frame prediction pattern and a prediction type that can determine whether inter-frame prediction or intra-frame prediction is applied.

[0014] The second technical solution of the present invention is an image decoding apparatus, the main feature of which includes: an inter-frame prediction unit configured to derive motion information relative to a geometric segmentation pattern and generate motion-compensated pixels; an intra-frame prediction unit configured to derive an intra-frame prediction pattern relative to the geometric segmentation pattern and generate intra-frame prediction pixels; a prediction information buffer configured to store or output motion information of a decoding target block to which the geometric segmentation pattern is applied, or prediction information including the intra-frame prediction pattern and a prediction type that can determine whether inter-frame prediction or intra-frame prediction is applied; and a synthesis unit configured to synthesize new prediction pixels by performing a weighted average of the motion-compensated pixels and the intra-frame prediction pixels with predetermined weights.

[0015] The third technical solution of the present invention is an image decoding apparatus, the main feature of which includes: an inter-frame prediction unit configured to derive motion information relative to a geometric segmentation pattern and generate motion-compensated pixels; an intra-frame prediction unit configured to derive an intra-frame prediction pattern relative to the geometric segmentation pattern and generate intra-frame prediction pixels; a prediction information buffer configured to store or output motion information of a decoding target block to which the geometric segmentation pattern is applied, or prediction information including an intra-frame prediction pattern and a prediction type that can determine whether inter-frame prediction or intra-frame prediction is applied; and a loop filtering processing unit configured to determine the boundary strength value when a deblocking filter is applied to the block boundary and apply the deblocking filter.

[0016] The fourth technical solution of the present invention is an image decoding method, the main idea of ​​which is to include: a step of deriving motion information relative to a geometric segmentation pattern and generating motion-compensated pixels; a step of deriving an intra-frame prediction pattern relative to the geometric segmentation pattern and generating intra-frame prediction pixels; and a step of saving or outputting motion information of a decoding target block applied with the geometric segmentation pattern or prediction information containing an intra-frame prediction pattern and a prediction type that can determine whether inter-frame prediction or intra-frame prediction is applied.

[0017] The fifth feature of the present invention is a technical solution whose main purpose is to enable a computer to function as an image decoding device, wherein the image decoding device comprises: an inter-frame prediction unit configured to derive motion information relative to a geometric segmentation pattern and generate motion-compensated pixels; an intra-frame prediction unit configured to derive an intra-frame prediction pattern relative to the geometric segmentation pattern and generate intra-frame prediction pixels; and a prediction information buffer configured to store or output motion information of a decoding target block to which the geometric segmentation pattern is applied, or prediction information containing an intra-frame prediction pattern and a prediction type capable of determining whether inter-frame prediction or intra-frame prediction is applied.

[0018] Invention Effects

[0019] According to the present invention, an image decoding apparatus, image decoding method, and program are provided that can improve coding performance by specifying prediction information and a method for storing prediction information when an intra-frame prediction mode is added to the GPM. Attached Figure Description

[0020] Figure 1 This is a diagram illustrating an example of the structure of an image processing system 1 according to one embodiment.

[0021] Figure 2 This is a diagram illustrating an example of the functional blocks of an image encoding apparatus 100 according to one embodiment.

[0022] Figure 3 This is a diagram illustrating an example of the functional blocks of an image decoding apparatus 200 according to one embodiment.

[0023] Figure 4 This is a diagram illustrating an example of a rectangular decoding target block being divided into two geometrically shaped segmented regions A and B by the segmentation lines of the geometric segmentation pattern disclosed in Non-Patent Document 1.

[0024] Figure 5 This is a diagram illustrating the method for constructing the fusion candidate list disclosed in Non-Patent Document 1.

[0025] Figure 6 This represents an example of the intra-frame prediction mode for GPM applications as described in this embodiment.

[0026] Figure 7 This represents an example of the intra-frame prediction mode for GPM applications as described in this embodiment.

[0027] Figure 8 This is a diagram illustrating an example of the weighting coefficient w of the predicted pixels relative to each segmented region A / B in Non-Patent Document 1 and the present embodiment of GPM.

[0028] Figure 9 This is a diagram showing an example of angleIdx (angle index) representing the angle of the dividing line of the GPM.

[0029] Figure 10 This is a diagram representing an example of disLut.

[0030] Figure 11 This is a diagram illustrating examples of the types of saved prediction information disclosed in Non-Patent Document 1 for 4×4 pixel sub-blocks and the types of saved prediction information involved in this embodiment.

[0031] Figure 12 This is a diagram showing a list of motion information disclosed in Non-Patent Document 1 and prediction information involved in this embodiment, stored according to the sType values ​​of the sub-blocks constituting the GPM application block.

[0032] Figure 13 It means relative to Figure 4 A diagram illustrating an example of the prediction information stored in a GPM consisting of two different inter-frame predictions.

[0033] Figure 14 It means relative to Figure 6 A diagram illustrating an example of the prediction information stored in a GPM consisting of intra-frame prediction and inter-frame prediction.

[0034] Figure 15 It means relative to Figure 7 A diagram illustrating an example of prediction information stored in a GPM consisting of two different intra-frame predictions.

[0035] Figure 16 This diagram illustrates an example of how the synthesis unit 243 in this embodiment uses prediction information stored in the prediction information buffer 244 to control the weight determination of the inter-frame and intra-frame joint prediction of the decoded target block when applying a geometric segmentation mode to neighboring blocks adjacent to the decoded target block.

[0036] Figure 17 This is a diagram illustrating an example of a deblocking filter applied by the loop filtering processing unit 250 to adjacent block boundaries.

[0037] Symbol Explanation

[0038] 10: Image processing system; 100: Image encoding device; 111, 241: Inter-frame prediction unit; 112, 242: Intra-frame prediction unit; 113, 243: Synthesis unit; 114, 224: Prediction information buffer; 121: Subtractor; 122, 230: Adder; 131: Transform / quantization unit; 132, 220: Inverse transform / inverse quantization unit; 140: Encoding unit; 150, 250: Loop filtering processing unit; 160, 260: Frame buffer; 200: Image decoding device; 210: Decoding unit. Detailed Implementation

[0039] The following is for reference only. Figure 1 The embodiments of the present invention will be described below. Furthermore, the constituent elements in the following embodiments can be appropriately replaced with existing constituent elements, and various modifications, including combinations with other existing constituent elements, are possible. Therefore, the scope of the technical solutions described in the claims is not limited based on the following description of the embodiments.

[0040] <First Implementation Method>

[0041] The following is for reference. Figures 1 to 15 The image processing system 10 according to the first embodiment of the present invention will be described. Figure 1 This is a diagram illustrating the image processing system 10 according to this embodiment.

[0042] (Image Processing System 10)

[0043] like Figure 1 As shown, the image processing system 10 according to this embodiment includes an image encoding device 100 and an image decoding device 200.

[0044] The image encoding device 100 is configured to generate encoded data by encoding an input image signal (image). The image decoding device 200 is configured to generate an output image signal by decoding the encoded data.

[0045] Here, the encoded data can be transmitted from the image encoding device 100 to the image decoding device 200 via a transmission path. Alternatively, the encoded data can also be provided from the image encoding device 100 to the image decoding device 200 after being stored in a storage medium.

[0046] (Image encoding device 100)

[0047] The following is for reference. Figure 2 The image encoding apparatus 100 involved in this embodiment will be described. Figure 2 This diagram illustrates an example of the functional blocks of the image encoding apparatus 100 according to this embodiment.

[0048] like Figure 2 As shown, the image encoding apparatus 100 includes an inter-frame prediction unit 111, an intra-frame prediction unit 112, a synthesis unit 113, a prediction information buffer 114, a subtractor 121, an adder 122, a transform / quantization unit 131, an inverse transform / inverse quantization unit 132, an encoding unit 140, a loop filtering processing unit 150, and a frame buffer 160.

[0049] The inter-frame prediction unit 111 is configured to generate an inter-frame prediction signal through inter-frame prediction.

[0050] Specifically, the inter-frame prediction unit 111 is configured to identify the reference block contained in the reference frame by comparing the encoded target frame (target frame) with the reference frame stored in the frame buffer 160, and to determine the motion vector (MV) for the specific reference block. Here, the reference frame is a frame different from the target frame.

[0051] Furthermore, the inter-frame prediction unit 111 is configured to generate an inter-frame prediction signal contained in a coded target block (hereinafter, target block) based on a reference block and a motion vector.

[0052] Furthermore, the inter-frame prediction unit 111 is configured to output the inter-frame prediction signal to the synthesis unit 113.

[0053] In addition, although Figure 2 Although not shown in the figure, the inter-frame prediction unit 111 is configured to output information related to the control of inter-frame prediction (specifically, information such as inter-frame prediction mode, motion vector, reference frame list, and reference frame number) to the encoding unit 140.

[0054] The intra-frame prediction unit 112 is configured to generate an intra-frame prediction signal through intra-frame prediction.

[0055] Specifically, the intra-frame prediction unit 112 is configured as a reference block contained in a specific target frame, and generates an intra-frame prediction signal based on the specific reference block and the target block. Here, the reference block is a block that references the target block. For example, the reference block is a block adjacent to the target block.

[0056] In addition, the intra-frame prediction unit 112 is configured to output the intra-frame prediction signal to the synthesis unit 113.

[0057] In addition, although Figure 2 Although not shown in the figure, the intra-prediction unit 112 is configured to output information related to the control of intra-prediction (specifically, information such as intra-prediction mode) to the coding unit 140.

[0058] The synthesis unit 113 is configured to synthesize the inter-frame prediction signal input from the inter-frame prediction unit 111 and / or the intra-frame prediction signal input from the intra-frame prediction unit 112 using preset weighting coefficients, and output the synthesized prediction signal (hereinafter collectively referred to as the prediction signal) to the subtractor 121 and the adder 122.

[0059] The prediction information buffer 114 is configured to store prediction information input from the inter-frame prediction unit 111 or the intra-frame prediction unit 112, or to output the stored prediction information to the inter-frame prediction unit 111, the intra-frame prediction unit 112, the synthesis unit 113, or the loop filtering processing unit 150. Details of the prediction information will be explained later.

[0060] Here, the synthesis processing of inter-frame prediction signals and / or intra-frame prediction signals of the synthesis unit 113 can also be performed using the same structure as in Non-Patent Document 1 in this embodiment, so the description is omitted.

[0061] Subtractor 121 is configured to subtract the prediction signal from the input image signal and output the prediction residual signal to the transform / quantization unit 131. Here, subtractor 121 is configured to generate the prediction residual signal, which is the difference between the prediction signal generated by intra-frame prediction or inter-frame prediction and the input image signal.

[0062] Adder 122 is configured to add the prediction signal output from synthesis unit 113 to the prediction residual signal output from inverse transform / inverse quantization unit 132 to generate a pre-filtering decoded signal, and output the pre-filtering decoded signal to intra-frame prediction unit 112 and loop filtering unit 150.

[0063] Here, the decoded signal before filtering constitutes a reference block used by the intra-frame prediction unit 112.

[0064] The transform / quantization unit 131 is configured to perform transform processing on the prediction residual signal and obtain coefficient level values. Furthermore, the transform / quantization unit 131 can also be configured to perform quantization on the coefficient level values.

[0065] Here, the transformation process is the process of transforming the predicted residual signal into a frequency component signal. As the transformation process, a basis pattern (transformation matrix) corresponding to the Discrete Cosine Transform (DCT) or a basis pattern (transformation matrix) corresponding to the Discrete Sine Transform (DST) can be used.

[0066] Alternatively, as a transformation process, the deviations of coefficients suitable for predicting residual signals disclosed in Non-Patent Document 1, which can be selected from multiple transformation bases in a horizontal / vertical direction, can also be used, such as MTS (Multiple Transform Selection), which can select coefficients suitable for predicting residual signals in a horizontal / vertical direction, and LFNST (Low Frequency Non-Separable Transform), which can improve coding performance by concentrating the transformation coefficients after a single transformation in a lower frequency region.

[0067] The inverse transform / inverse quantization unit 132 is configured to perform inverse transform processing on the coefficient level values ​​output from the transform / quantization unit 131. Alternatively, the inverse transform / inverse quantization unit 132 may be configured to perform inverse quantization of the coefficient level values ​​before the inverse transform processing.

[0068] Here, the inverse transformation processing and inverse quantization are performed in the opposite manner to the transformation processing and quantization performed by the transformation / quantization unit 131.

[0069] The encoding unit 140 is configured to encode the coefficient level values ​​output from the transform / quantize unit 131 and output the encoded data.

[0070] Here, for example, the encoding is entropy encoding, which assigns codes of different lengths based on the probability of occurrence of coefficient hierarchy values.

[0071] In addition, the encoding unit 140 is configured to encode control data used by the decoding process, in addition to coefficient level values.

[0072] Here, the control data may include block size-related information (flags or indexes) such as coded block size, predicted block size, and transformed block size.

[0073] In addition, the control data may also include information (flags or indexes) required for control, such as the inverse transform / inverse quantization processing of the inverse transform / inverse quantization unit 220, the inter-frame prediction signal generation processing of the inter-frame prediction unit 241, the intra-frame prediction signal generation processing of the intra-frame prediction unit 242, the inter-frame prediction signal and / or intra-frame prediction signal synthesis processing of the synthesis unit 243, and the filtering processing of the loop filtering unit 250 in the image decoding apparatus 200 described later.

[0074] Furthermore, in Non-Patent Document 1, these control data are referred to as syntax, and their definition is referred to as semantics.

[0075] In addition, control data may also include header information such as Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Picture Header (PH), and Slice Header (SH), which will be described later.

[0076] The loop filtering processing unit 150 is configured to filter the decoded signal before filtering output from the adder 122 and output the decoded signal after filtering to the frame buffer 160.

[0077] Here, for example, the filtering process is a deblocking filter process that reduces distortion generated at the boundary portions of blocks (coding blocks, prediction blocks, or transform blocks) or an adaptive cyclic filtering process that switches filters based on filter coefficients, filter selection information, local properties of the image's graphics, etc., transmitted from the image coding device 100.

[0078] The frame buffer 160 is configured to accumulate reference frames used by the inter-frame prediction unit 111.

[0079] Here, the filtered decoded signal constitutes a reference frame used by the inter-frame prediction unit 111.

[0080] (Image decoding device 200)

[0081] The following is for reference. Figure 3 The image decoding apparatus 200 according to this embodiment will be described. Figure 3 This diagram illustrates an example of the functional blocks of the image decoding apparatus 200 according to this embodiment.

[0082] like Figure 3 As shown, the image decoding apparatus 200 includes a decoding unit 210, an inverse transform / inverse quantization unit 220, an adder 230, an inter-frame prediction unit 241, an intra-frame prediction unit 242, a synthesis unit 243, a prediction information buffer 224, a loop filtering processing unit 250, and a frame buffer 260.

[0083] The decoding unit 210 is configured to decode the encoded data generated by the image encoding device 100 and to decode the coefficient level values.

[0084] Here, decoding is, for example, entropy decoding, which is the reverse of the entropy encoding steps performed by the encoding unit 140.

[0085] Alternatively, the decoding unit 210 may be configured to obtain control data through decoding of encoded data.

[0086] Here, the control data may include information related to the block size of the aforementioned decoding block (which has the same meaning as the encoding target block in the image encoding device 100 described above. Hereinafter, they are collectively referred to as target blocks).

[0087] In addition, the control data may also include information (flags or indexes) required for control, such as the inverse transformation / inverse quantization processing of the inverse transformation / inverse quantization unit 220, the prediction pixel generation processing of the inter-frame prediction unit 241 or the intra-frame prediction unit 242, and the filtering processing of the loop filtering unit 250.

[0088] In addition, control data may also include header information such as the sequence parameter set (SPS), picture parameter set (PPS), picture header (PH), and slice header (SH).

[0089] The inverse transform / inverse quantization unit 220 is configured to perform inverse transform processing on the coefficient level values ​​output from the decoding unit 210. Alternatively, the inverse transform / inverse quantization unit 220 may be configured to perform inverse quantization of the coefficient level values ​​before the inverse transform processing.

[0090] Here, the inverse transformation processing and inverse quantization are performed in the opposite manner to the transformation processing and quantization performed by the transformation / quantization unit 131.

[0091] Similar to the inter-frame prediction unit 111, the inter-frame prediction unit 241 is configured to generate an inter-frame prediction signal through inter-frame prediction.

[0092] Specifically, the inter-frame prediction unit 241 is configured to generate an inter-frame prediction signal based on the motion vector decoded from the encoded data and the reference signal contained in the reference frame. The inter-frame prediction unit 241 is configured to output the inter-frame prediction signal to the synthesis unit 243.

[0093] Similar to the intra-frame prediction unit 112, the intra-frame prediction unit 242 is configured to generate an intra-frame prediction signal through intra-frame prediction.

[0094] Specifically, the intra-prediction unit 242 is configured to be a reference block contained in a specific target frame, and generates an intra-prediction signal according to the specific reference block. The intra-prediction unit 242 is configured to output the intra-prediction signal to the synthesis unit 243.

[0095] Similar to the synthesis unit 113, the synthesis unit 243 is configured to synthesize the inter-frame prediction signal input from the inter-frame prediction unit 241 and / or the intra-frame prediction signal input from the intra-frame prediction unit 242 using preset weighting coefficients, and output the synthesized prediction signal (hereinafter collectively referred to as the prediction signal) to the adder 230.

[0096] Similar to prediction information buffer 114, prediction information buffer 244 is configured to store prediction information input from inter-frame prediction unit 221 or intra-frame prediction unit 222, or output the stored prediction information to inter-frame prediction unit 241, intra-frame prediction unit 242, synthesis unit 243, or loop filtering processing unit 250. Details of the prediction information will be explained later.

[0097] The adder 230 is configured to add the prediction signal output from the synthesis unit 243 to the prediction residual signal output from the inverse transform / inverse quantization unit 220 to generate a pre-filtering decoded signal, and output the pre-filtering decoded signal to the loop filtering unit 250.

[0098] Here, the decoded signal before filtering constitutes a reference block used by the intra-frame prediction unit 242.

[0099] Similar to the loop filtering processing unit 150, the loop filtering processing unit 250 is configured to filter the decoded signal before filtering output from the adder 230, and output the decoded signal after filtering to the frame buffer 260.

[0100] Here, for example, the filtering process is a deblocking filter process that reduces distortion at the boundary portions of blocks (coding blocks, prediction blocks, transform blocks, or sub-blocks into which these are divided) or an adaptive cyclic filtering process that switches filters based on filter coefficients, filter selection information, local properties of the image graphics, etc., transmitted from the image coding device 100.

[0101] Similar to frame buffer 160, frame buffer 260 is configured to accumulate reference frames used by inter-frame prediction unit 241.

[0102] Here, the filtered decoded signal constitutes a reference frame used by the inter-frame prediction unit 241.

[0103] (Geometric segmentation mode)

[0104] The following uses Figures 4-7 The application of the geometric segmentation mode (GPM) disclosed in Non-Patent Document 1 to the decoding unit 210, inter-frame prediction unit 241 and intra-frame prediction unit 242 and the intra-frame prediction mode relative to the first geometric segmentation mode (GPM) to which this embodiment is involved will be described.

[0105] Figure 4This represents an example of a rectangular decoding target block being divided into two geometrically shaped segmented regions A and B by the segmentation line L of the geometric segmentation pattern disclosed in Non-Patent Document 1.

[0106] Here, the dividing line L of the geometric division pattern disclosed in Non-Patent Document 1 is prepared with 64 patterns according to the angle and position.

[0107] In addition, the GPM described in Non-Patent Document 1 applies a common fusion mode as a type of inter-frame prediction to segmented regions A and B respectively, and generates inter-frame prediction (motion compensation) pixels.

[0108] Specifically, in the GPM, a fusion candidate list disclosed in Non-Patent Document 1 is constructed. Based on the fusion candidate list and two fusion indices (merge_gpm_idx0, merge_gpm_idx1) relative to each segmented region A / B transmitted from the image encoding device 100, motion vectors (mvA, mvB) and reference frames of each segmented region A / B are derived, and reference blocks, i.e., inter-frame prediction (or motion compensation) blocks, are generated. Finally, the inter-frame prediction pixels of each segmented region A / B are synthesized by weighted averaging through pre-set weights.

[0109] Figure 5 This is a diagram illustrating the method for constructing the fusion candidate list disclosed in Non-Patent Document 1.

[0110] like Figure 5 As shown, the fusion candidate list consists of list 0 (L0) and list 1 (L1) representing two prediction directions, and multiple candidates for motion information, which will be described later, are constructed in the list relative to each list.

[0111] The motion information of the segmented regions A / B of the GPM disclosed in Non-Patent Document 1 consists of merge_gpm_idx0 / merge_gpm_idx1 and relative to the segmented regions A / B. Figure 5 The GPM-oriented fusion candidate list (MergeCandList[m, n]) is exported as shown.

[0112] Here, to minimize the duplication of motion information derived from merge_gpm_idx0 and merge_gpm_idx1, such as... Figure 5 As shown in X, the list numbers of the motion information selected in merge_gpm_idx0 and merge_gpm_idx1 as export targets are nested according to the even and odd numbers of MergeCandList.

[0113] Specifically, based on merge_gpm_idx0 and merge_gpm_idx1, calculate the following m and n.

[0114] m = merge_gpm_idx0[xCb][yCb]

[0115] n=merge_gpm_idx1[xCb][yCb]+((merge_gpm_idx1[xCb][yCb]>=m)?1:0)

[0116] Based on the calculated value of m, the motion vector, reference image index, and prediction list flags constituting the motion information of segmented region A are derived as follows.

[0117] First, calculate the value of X based on m&0x01 (determining whether the value of m is even) and n&0x01 (determining whether the value of n is even). Here, if the calculated X is 0, set the value of X to (1-X).

[0118] Finally, derive the motion vector mvA, reference image index refIdxA, prediction list flag preListFlagA for segmented region A, and the motion vector mvB, reference image index refIdxB, and prediction list flag preListFlagB for segmented region B, as follows.

[0119] mvA = mvLXM

[0120] refIdxA=refIdxLXM

[0121] preListFlagA = X

[0122] mvB = mvLXN

[0123] refIdxB=refIdxLXN

[0124] preListFlagB = X

[0125] Here, M and N are the fusion candidate numbers shown as m and n in the fusion candidate list, respectively.

[0126] M = MergeCandList[m]

[0127] N = MergeCandList[n].

[0128] Figure 6 and Figure 7 This represents an example of the intra-frame prediction mode for GPM applications as described in this embodiment.

[0129] Specifically, Figure 6 This illustrates a structural example of GPM in this embodiment where intra-frame prediction (modeX) and inter-frame prediction are applied relative to each segmented region A / B. Figure 7 This describes a structural example of GPM in this embodiment where different intra-frame predictions (modeX, modeY) are applied to each segmented region A / B.

[0130] In the first GPM involved in this embodiment, either a general fusion mode or an intra-prediction mode can be applied to each segmented region A / B, and the type of intra-prediction mode is limited according to the segmentation shape (segmentation line) of the target block.

[0131] Furthermore, in the second GPM involved in this embodiment, a specific method is specified for whether a GPM can be applied to the intra-prediction mode of the decoded target block and for the type of prediction mode of each segmented region A / B when the GPM is applied.

[0132] Therefore, by appropriately applying the GPM with the intra-prediction mode to the decoded target block, and by specifying the optimal prediction mode, further improvements in coding performance can be achieved.

[0133] (Weighting factor of GPM)

[0134] The following uses Figures 8-10 The weighting coefficient w of GPM in Non-Patent Literature 1 and the present embodiment will be explained for the decoding unit 210, inter-frame prediction unit 241, intra-frame prediction unit 242 and synthesis unit 243.

[0135] Figure 8 This is a diagram illustrating an example of the weighting coefficient w of the predicted pixels relative to each segmented region A / B in Non-Patent Document 1 and the present embodiment of GPM.

[0136] The predicted pixels of each segmented region A / B generated by the inter-frame prediction unit 241 or the intra-frame prediction unit 242 are synthesized (weighted average) in the synthesis unit 243 by weighting coefficient w.

[0137] In Non-Patent Document 1, the weighting coefficient w is used with values ​​from 0 to 8. In this embodiment, the weighting coefficient w can also be used. Here, weighting coefficient w values ​​of 0 and 8 represent non-blending regions, and weighting coefficient w values ​​of 1 to 7 represent blending regions.

[0138] Furthermore, in this embodiment, the weighting coefficient w can be calculated using the same method as in Non-Patent Document 1, which is based on offset values ​​(offsetX, offsetY) calculated from pixel positions (xL, yL) and target block size. Figure 9 The displacements (diplacementX, displacementY) shown are calculated from the angleIdx of the dividing lines of the specified geometric division pattern (GPM). Figure 10 The table value disLut, calculated from displacementX and displacementY, is calculated in the following manner.

[0139] weightIdx=(((xL+offsetX)<<1)+1)×disLut[diplacementX]+(((yL+offsetY)<<1)+1)×disLut[diplacementY]

[0140] weightIdxL=partFlip? 32+weightIdx:32-weightIdx

[0141] w=Clip3(0, 8, (weightIdxL+4)>>3)

[0142] (Types of motion information saved and types of prediction information saved)

[0143] The following uses Figure 11 The types of motion information to be saved disclosed in Non-Patent Document 1 regarding the decoding unit 210, inter-frame prediction unit 241, intra-frame prediction unit 242, and prediction information buffer 244, as well as the types of prediction information to be saved in this embodiment, will be explained.

[0144] Figure 11 This is a diagram illustrating examples of the types of saved prediction information disclosed in Non-Patent Document 1 for 4×4 pixel sub-blocks and the types of saved prediction information involved in this embodiment.

[0145] The calculation methods for both are the same as shown below, but the difference lies in the following: Figure 8 As shown, the information stored in Non-Patent Document 1 is motion information, while in this embodiment it is prediction information.

[0146] First, similar to Non-Patent Document 1, the values ​​of the saved motion information type and the saved prediction information type (since the calculation method is the same, all values ​​will be defined as sType for convenience) are calculated in the following manner based on the index (xSbIdx, ySbIdx) of the 4×4 pixel sub-block unit, the offset value (offsetX, offsetY), the displacement (diplacementX, displacementY), and the table (disLut) calculated in the same way as the weight coefficient w mentioned above.

[0147] motionIdx=(((4×xSbIdx+offsetX)<<1)+5)×disLut[diplacementX]+(((4×ySbIdx+offsetY)<<1)+5)×disLut[diplacementY]

[0148] sType=Abs(motionIdx)<32?2:(motionIdx<=0?(1-isFlip):isFlip)

[0149] Here, as Figure 11 As shown, the value of sType consists of three values: 0, 1, and 2. Based on each value, the motion information and prediction information stored in the prediction information buffer 244 are controlled by the inter-frame prediction unit 241 and the intra-frame prediction unit 242 as follows.

[0150] When the value of sType is 0, the motion information of segmented region A is stored in Non-Patent Document 1, and the prediction information of segmented region A is stored in this embodiment.

[0151] When the value of sType is 1, the motion information of the segmented region B is stored in Non-Patent Document 1, and the prediction information of the segmented region B is stored in this embodiment.

[0152] When the value of sType is 2, the motion information of segmented region A and segmented region B or only the motion information of segmented region B is stored in Non-Patent Document 1. In this embodiment, the prediction information of segmented region A and segmented region B or only the prediction information of segmented region B is stored.

[0153] The saved motion information and prediction information will be explained later.

[0154] Furthermore, the 4×4 pixel sub-block unit described above can be modified according to the designer's ideas to change the sType calculation unit and the motion information or prediction information storage unit described later.

[0155] Specifically, in order to reduce the amount of information to be stored, the calculation unit of the sType mentioned above and the storage unit of motion information or prediction information described later can be increased to 8×8 pixels, 16×16 pixels, etc.

[0156] Alternatively, although the amount of information stored can be increased, in order to improve the accuracy of motion information or prediction information when referencing other blocks or frames, the calculation unit of the sType mentioned above and the storage unit of motion information or prediction information described later can be reduced to 2×2 pixels, etc.

[0157] (Motion information and prediction information stored in the prediction information buffer)

[0158] The following uses Figure 12 and Figure 13 The motion information disclosed in Non-Patent Document 1 and the prediction information involved in this embodiment, which are stored in the prediction information buffer 244 from the inter-frame prediction unit 241, will be described respectively.

[0159] Figure 12 This is a diagram showing a list of motion information disclosed in Non-Patent Document 1 and prediction information involved in this embodiment, stored according to the sType values ​​of the sub-blocks constituting the GPM application block.

[0160] First, the motion information ultimately saved in the GPM disclosed in Non-Patent Document 1 consists of the following parameters.

[0161] • Prediction direction (predFlagL0, predFlagL1)

[0162] • Motion vectors of L0 and L1 (mvL0, mvL1)

[0163] • Reference image indices for L0 and L1 (refIdxL0, refIdxL1)

[0164] ·BcwIdx

[0165] In addition, the prediction direction (predFlagL0, predFlagL1) is a parameter that represents the prediction direction of the sub-block stored according to sType (described later). Based on the values ​​of predFlagL0 and predFlagL1, it is classified into three types: L0 single prediction, L1 single prediction, and L0 / L1 double prediction.

[0166] Here, L0 single prediction refers to the inter-frame prediction of a motion vector derived from the L0 list, where the cases where predFlagL0 is 1 and predFlagL1 is 0 are respectively saved as values ​​representing the conditions.

[0167] Additionally, L1 single prediction refers to the inter-frame prediction of a motion vector derived from the L1 list, where the cases where predFlagL0 is 0 and predFlagL1 is 1 are respectively saved as values ​​representing that condition.

[0168] In addition, L0 / L1 dual prediction refers to the inter-frame prediction of two motion vectors derived from the L0 list and the L1 list respectively. The case where predFlagL0 is 1 and predFlagL1 is 1 is saved as the value representing the condition.

[0169] In addition, the motion vectors (mvL0, mvL1) of L0 and L1 are motion vectors relative to the list numbers L0 and L1 mentioned above.

[0170] In addition, the reference image indices (refIdxL0, refIdxL1) of L0 and L1 respectively represent the reference frames referenced by mvL0 and mvL1.

[0171] Additionally, BcwIdx is an index of the weight coefficient values ​​of BCW (Bi-prediction with CU-level weights) disclosed in specific non-patent document 1.

[0172] In contrast to the motion information stored in the prediction information buffer 244 disclosed in Non-Patent Document 1, the prediction information stored in the prediction information buffer 244 in this embodiment, such as Figure 12 As shown, parameters are added to store the prediction type and intra-frame prediction mode.

[0173] Here, as Figure 12 As shown, the prediction type is an internal parameter representing either inter-frame prediction (Inter) or intra-frame prediction (Intra).

[0174] In addition, as the prediction information involved in this embodiment, such as Figure 12 As shown, hpeIfIdx, IBCFlag, and LIC Flag can also be added.

[0175] Here, hpeIfIdx and IBC Flag are respectively the SIF (Switchable Interpolation Filter) and IBC (Intra Block Copy) disclosed in Non-Patent Document 1 and the LIC (Local Illumination Compensation) disclosed in Non-Patent Document 2, which are specific to the application.

[0176] (Details of the prediction information saved based on the type of prediction information being saved)

[0177] The following uses Figures 13-15 The details of the prediction information stored in the prediction information buffer 244 by the inter-frame prediction unit 241 or the intra-frame prediction unit 242 according to the type of prediction information sType to be stored in this embodiment will be described.

[0178] Figure 13 It means relative to Figure 4 This is an example of the prediction information stored in the GPM, which consists of two different inter-frame predictions. The details of each prediction information stored according to the sType value are explained below.

[0179] First, in all sType regions, the prediction type is saved as inter-frame prediction (Inter).

[0180] Second, predFlagL0, predFlagL1, mvL0, mvL1, refIdxL0, and refIdxL1 are stored in the same manner as disclosed in Non-Patent Document 1, based on the value of sType and the values ​​of predListFlagA and predListFlagB, which represent the list numbers of the fusion candidate list, wherein the list number of the fusion candidate list represents the destination of the motion vector of the aforementioned segmented region A / B.

[0181] First, when sType=0, it is calculated as follows.

[0182] predFlagL0=(predListFlagA==0)? 1:0

[0183] predFlagL1=(predListFlagA==0)? 0:1

[0184] refIdxL0=(predListFlagA==0)? refIdxA:-1

[0185] refIdxL1=(predListFlagA==0)? -1:refIdxA

[0186] mvL0=(predListFlagA==0)? mvA:0

[0187] mvL1=(predListFlagA==0)? 0:mvA

[0188] Next, in the case where sType = 1 or sType = 2 and predListFlagA + predListFlagB ≠ 1, the calculation is performed as follows.

[0189] predFlagL0=(predListFlagB==0)? 1:0

[0190] predFlagL1=(predListFlagB==0)? 0:1

[0191] refIdxL0=(predListFlagB==0)? refIdxB:-1

[0192] refIdxL1=(predListFlagB==0)? -1:refIdxB

[0193] mvL0=(predListFlagB==0)? mvB:0

[0194] mvL1=(predListFlagB==0)? 0:mvB

[0195] Here, `predListFlagA + predListFlagB ≠ 1` indicates the case where the list numbers for segmented regions A and B are the same. In this case, to avoid duplication of motion vectors, even when `sType = 2`, it is configured to only store the motion vector of segmented region B.

[0196] Next, with sType = 2 and predListFlagA + predListFlagB = 1, the calculation is performed as follows.

[0197] predFlagL0 = 1

[0198] predFlagL1=1

[0199] refIdxL0=(predListFlagA==0)? refIdxA:refIdxB

[0200] refIdxL1=(predListFlagA==0)? refIdxB:refIdxA

[0201] mvL0=(predListFlagA==0)? mvA:mvB

[0202] mvL1=(predListFlagA==0)? mvB:mvA

[0203] Here, predListFlagA + predListFlagB = 1 indicates a situation where the list numbers of the segmented regions A and B are inconsistent. In this case, the configuration directly stores two motion vectors of the segmented regions A and B.

[0204] In addition, although Figure 13 Not illustrated, but the above-mentioned mvL0 and mvL1 can preserve the motion vectors relative to the GPM disclosed in Non-Patent Document 2 before correction by MMVD (Merge with Motion Vector Difference) or InterTM (Template Matching).

[0205] Alternatively, mvL0 and mvL1 mentioned above can also be motion vectors modified by MMVD and InterTM relative to the GPM disclosed in Non-Patent Document 2.

[0206] By preserving the corrected motion vectors, the prediction accuracy of the prediction block that obtains motion vectors from the GPM application block and generates prediction pixels is improved.

[0207] On the other hand, while preserving the motion vector before correction, it is not expected to improve the prediction accuracy of the prediction block from the GPM reference motion vector. However, since the derivation process of the motion vector of the reference base block of the GPM application block can be started without waiting for the completion of the MMVD and Inter TM processes performed relative to the GPM block, it is expected to reduce the decoding processing time.

[0208] Furthermore, regarding the following... Figures 13-15 Similarly, it is possible to choose which motion vector to save, before or after the correction.

[0209] Next, the intra-prediction mode can also be stored in none of the sType regions. Alternatively, a value indicating invalid intra-prediction can be stored in all sType regions. This is because in Figure 13 In the structure shown, since all regions are inter-frame predictions, there is no intra-frame prediction mode applied to the target block.

[0210] Next, BcwIdx, hpeIfIdx, IBC Flag, and LIC Flag can also store values ​​representing invalid values ​​in all sType areas. This is because BCW, SIF, IBC, and LIC are all encoding tools that XOR with GPM, so these encoding tools are obviously invalid in target blocks where GPM is applied.

[0211] Relatedly, although in Figure 12 and Figure 13 Not illustrated, but the motion vectors used in IBC can be omitted, while the zero vector can be saved. These parameters are explained later. Figure 14 and Figure 15 The same structure is used in the middle, therefore it is omitted thereafter. Figure 14 and Figure 15 Detailed explanation of these parameters.

[0212] Figure 14 It means relative to Figure 6 This is an example of the prediction information stored in the GPM, which consists of intra-frame prediction and inter-frame prediction. The details of each prediction information stored based on the sType value are explained below.

[0213] First, when sTyper=1, the prediction information stored in the segmented region (Inter segmented region) where inter-frame prediction is applied can be used in the same way as in the previous frame prediction. Figure 13 The prediction information for this region has the same structure as described in the previous section, so the description is omitted.

[0214] Second, when sTyper=0, the prediction information stored in the segmented region (Intra's segmented region) where intra-prediction is applied is as follows: Figure 14 As shown, it is saved in the following manner.

[0215] Prediction type = Intra

[0216] predFlag0 = 0

[0217] predFlag1 = 0

[0218] mvL0=0

[0219] mvL1=0

[0220] refIdxL0 = -1

[0221] refIdxL1=-1

[0222] Intra-frame prediction mode = modeX

[0223] Here, in the segmented region, since intra-frame prediction is applied, Intra is saved as the prediction type and modeX is saved as the intra-frame prediction mode, as described above.

[0224] Furthermore, as a variation, for example, as in Non-Patent Document 1, if an adjacent sub-block unit contains only a coding tool that references only the intra-prediction type and not the intra-prediction mode, a structure that saves Intra as the prediction type and not the intra-prediction mode can also be adopted.

[0225] On the other hand, since there is no motion information, as described above, 0 can be saved as predFlag0 and predFlagL1, 0 (representing zero vector) can be saved as mvL0 and mvL1, and -1 (representing no reference frame) can be saved as refIdxL0 and refIdxL1.

[0226] Alternatively, as a variation, to avoid exhausting the capacity of the buffer area of ​​the prediction information buffer 244, a structure that does not store this motion information may be adopted.

[0227] Third, when sTyper=2, the prediction information stored in the segmented region (Intra+Inter segmented region) where intra-frame prediction and inter-frame prediction are applied is as follows: Figure 14 As shown, it is saved in the following manner.

[0228] Prediction type = Inter

[0229] predFlag0=(predListFlagB==0)? 1:0

[0230] predFlag1=(predListFlagB==0)? 0:1

[0231] refIdxL0=(predListFlagB==0)? refIdxB:-1

[0232] refIdxL1=(predListFlagB==0)? -1:refIdxB

[0233] mvL0=(predListFlagB==0)? mvB:0

[0234] mvL1=(predListFlagB==0)? 0:mvB

[0235] Intra-frame prediction mode = modeX

[0236] Here, in the segmented region, since intra-prediction with sType=1 is applied, the parameters saved as the prediction type and intra-prediction mode in the prediction information are the same as those saved in sType=1, as described above.

[0237] Furthermore, in the segmented region, since inter-frame prediction with sType=2 is applied, the parameters involved in the motion state information in the prediction information are stored as described above, and are the same as those stored in sType=2.

[0238] Figure 15 It means relative to Figure 7 This is an example of the prediction information stored in a GPM consisting of two different intra-frame predictions. The details of each prediction information stored according to the sType value are explained below.

[0239] First, the parameters other than the intra-frame prediction mode in the prediction information stored in all sType regions can be used in conjunction with... Figure 14 The structure described above, which stores the same parameters in the Intra segmentation region when sType=0, is omitted from the description.

[0240] Second, the intra-frame prediction modes for regions with sType=0 and sType=1, such as Figure 15 As shown, two different intra-prediction modes, modeX and modeY, are stored in each region.

[0241] Third, in the region where sType=2, such as Figure 15 As shown, it can save both the intra-prediction modes of the region with sType=0 and the region with sType=1, or it can save any intra-prediction mode.

[0242] For the former, for example, if the two intra-frame prediction modes can be used in the image encoding device 100 and the image decoding device 200, a structure that stores the two intra-frame prediction modes can be adopted.

[0243] For the latter, for example, the intra prediction mode with sType=0 can be selected as the intra prediction mode with sType=2 in the hard-coded form.

[0244] Alternatively, for example, by further subdividing the 4×4 pixel sub-block into 2×2 pixel sub-block units, one can calculate, for example, which intra-prediction mode the 4×4 pixel sub-block was generated from, and select the intra-prediction mode of the region with sType=2 as the dominant intra-prediction mode.

[0245] Alternatively, the intra prediction mode with sType=2 can be selected as the intra prediction mode where the distance between the adjacent reference pixels in the direction shown by the two intra prediction modes of the sub-block is smaller.

[0246] By using the prediction information and the method for storing prediction information described above, it is possible to appropriately reference the prediction information when the intra-frame prediction is appended to the GPM, either intra-frame or extra-frame, and thus an improvement in coding performance can be expected.

[0247] Furthermore, the parameters other than the intra-frame prediction mode in the prediction information stored in the prediction information buffer 244 described above can be deleted from the prediction information buffer 244 if they are no longer referenced from the intra-frame or extra-frame.

[0248] Additionally, initialization can be performed if the storage area for this parameter is secured in the prediction information buffer 244. Here, the time when the frame is no longer referenced from outside the frame is the same as the time when the frame containing the GPM application block is deleted from the frame buffer 260 (frame buffer 160).

[0249] Furthermore, the intra-prediction mode stored in the prediction information buffer 244 can be deleted from the prediction information buffer 244 if it is no longer referenced from the intra frame. Additionally, initialization can be performed if only the storage area for the intra-prediction mode quantity is ensured in the prediction information buffer 244.

[0250] (Weight determination and control method for inter-frame and intra-frame joint prediction when applying geometric segmentation mode to adjacent blocks)

[0251] The following uses Figure 16 The method for determining the weights of inter-frame and intra-frame joint prediction of the decoding target block when the synthesis unit 243 of this embodiment uses the prediction information stored in the prediction information buffer 244 to control the application of the geometric segmentation mode to the adjacent blocks adjacent to the decoding target block is described.

[0252] Figure 16 This diagram illustrates an example of how the synthesis unit 243 in this embodiment uses prediction information stored in the prediction information buffer 244 to control the weight determination of the inter-frame and intra-frame joint prediction of the decoded target block when applying a geometric segmentation mode to the adjacent block adjacent to the decoded target block.

[0253] Here, the Inter-and Intra Prediction (CIIP) disclosed in Non-Patent Document 1 is a technique that synthesizes a new prediction pixel PCIIP by weighting the inter-prediction pixel (motion compensation pixel) Pinter and the intra-prediction pixel PIntra of the decoded target block with a specified weight wt.

[0254] The specified weight wt is calculated based on the prediction types of the two adjacent blocks to the left and top of the target block being decoded, in the following manner.

[0255] First, if the adjacent block above is an intra-predicted block, the internal parameter isIntraTop is set to 1. Otherwise, isIntraTop is set to 0.

[0256] Next, if the adjacent block to the left is an intra-predicted block, the intrinsic parameter isIntraLeft is set to 1. Otherwise, isIntraLeft is set to 0.

[0257] Next, based on isIntraTop and isIntraLeft, determine the weight wt according to the following three patterns.

[0258] First, when isIntraTop+isIntraLeft=2 (i.e., when both adjacent blocks are intra-predicted blocks), wt=3 is set.

[0259] Second, when isIntraTop+isIntraLeft=1 (i.e., when one of the two adjacent blocks is an intra-predicted block and the other is an inter-predicted block), wt=2 is set.

[0260] Third, in other cases (i.e., when both adjacent blocks are inter-predictive blocks), wt is set to 1.

[0261] Finally, the synthesis unit 243 uses the weight wt to calculate PCIIP as follows.

[0262] PCIIP=((4-wt)×Pinter+wt×Pintra+2)>>2

[0263] In non-patent literature 1, since the determination of the weight wt of the aforementioned CIIP uses the prediction type of the adjacent blocks, for example, as Figure 16 As shown, the configuration is such that when GPM is applied to adjacent blocks and the GPM of adjacent blocks is composed of inter-frame prediction and intra-frame prediction, the prediction type of adjacent blocks is determined to be inter-frame prediction.

[0264] In contrast, in this embodiment, the determination of the CIIP weight wt does not use the prediction type of adjacent blocks, but uses the prediction type stored by the 4×4 pixel sub-block unit adjacent to the decoding target block.

[0265] Furthermore, the determination of the prediction type of the adjacent 4×4 pixel sub-blocks can use the same determination formula as the adjacent blocks of isIntraTop and isIntraLeft mentioned above.

[0266] Therefore, in Figure 16 In the case shown, if CIIP is applied to the target block and GPM consisting of inter-frame prediction and intra-frame prediction is applied to the adjacent blocks adjacent to the target block, the prediction information stored by the sub-block units of the segmented region considering GPM can be appropriately used to determine the prescribed weight wt of CIIP of the target block, and thus an improvement in prediction performance can be expected.

[0267] (Boundary strength value determination and control of the deblocking filter in the case of applying geometric segmentation mode)

[0268] The following uses Figure 17 The method by which the loop filtering processing unit 250 of this embodiment uses the prediction information stored in the prediction information buffer 244 to control the determination of the boundary strength value of the deblocking filter applied to the adjacent block boundaries will be described.

[0269] Figure 17 This is a diagram illustrating an example of a deblocking filter applied by the loop filtering processing unit 250 to adjacent block boundaries.

[0270] Here, the deblocking filter disclosed in Non-Patent Document 1 is a technique for smoothing block boundary distortion caused by adjacent block boundaries.

[0271] The target of the deblocking filter is both the luminance block and the chrominance block. Based on the value of the boundary strength (BS) described later, it is determined whether a deblocking filter or filter strength is applied to the luminance block, and whether a deblocking filter or filter strength is applied to the chrominance block.

[0272] In the determination criteria for the BS value disclosed in Non-Patent Document 1, the loop filter unit 250 determines the luminance signal for each of the following six conditions. Figure 16 Y) and color difference signal ( Figure 16 The values ​​of U and V.

[0273] Condition 1: If at least one of the two blocks contained in the target pixel has a prediction type of intra-frame prediction or a valid CIIP flag, the BS values ​​of the Y component, U component and V component of the target pixel are set to 2, 2 and 2 respectively.

[0274] Condition 2: If at least one of the two blocks contained in the target pixel has a non-zero coefficient, the BS values ​​of the Y component, U component and V component of the target pixel are set to 1, 1 and 1 respectively.

[0275] Condition 3: If one of the two blocks contained in the target pixel has an IBC flag indicating a valid value, and the other block has a prediction type of inter-frame prediction, then the BS values ​​of the Y component, U component, and V component of the target pixel are set to 1, 1, and 1, respectively.

[0276] Condition 4: If the two blocks contained in the target pixel are determined to be of the inter-frame prediction type, and the absolute difference of the MV of each block is greater than 0.5 luminance pixels, the BS values ​​of the Y component, U component and V component of the target pixel are set to 1, 0 and 0 respectively.

[0277] Condition 5: When the two blocks contained in the target pixel have the prediction type of inter-frame prediction and the reference frames (i.e., refIdx) of each block are different, the BS values ​​of the Y component, U component and V component of the target pixel are set to 1, 0 and 0 respectively.

[0278] Condition 6: If conditions 1 to 5 are not met, set the BS values ​​of the Y, U, and V components of the target pixel to 0, 0, and 0, respectively.

[0279] Here, in Non-Patent Document 1, as described above, the determination of the prediction type of the two blocks contained in the target pixel, the determination of whether it is a CIIP application block, the determination of whether it is an IBC application block, and the determination of the absolute value difference of the motion vector are made by the loop filter unit 250 through the prediction type of the block unit and the flag indicating whether there is a block unit applying CIIP or IBC.

[0280] In contrast, in this embodiment, the loop filter unit 250 uses the prediction information stored in the prediction information buffer 244 (4×4 pixel sub-blocks) to make the determination. That is, conditions one through six are modified as follows.

[0281] Condition 1: If at least one of the two sub-blocks contained in the target pixel has a prediction type of intra-frame prediction or a valid CIIP flag, the BS values ​​of the Y component, U component and V component of the target pixel are set to 2, 2 and 2 respectively.

[0282] Condition 2: If at least one of the two sub-blocks contained in the target pixel has a non-zero coefficient, the BS values ​​of the Y component, U component and V component of the target pixel are set to 1, 1 and 1 respectively.

[0283] Condition 3: If one of the two sub-blocks contained in the target pixel has an IBC flag indicating a valid value, and the other block has a prediction type of inter-frame prediction, then the BS values ​​of the Y component, U component and V component of the target pixel are set to 1, 1 and 1 respectively.

[0284] Condition 4: If the two sub-blocks contained in the target pixel have the prediction type of inter-frame prediction and the absolute difference of the MV of each block is greater than 0.5 luminance pixels, the BS values ​​of the Y component, U component and V component of the target pixel are set to 1, 0 and 0 respectively.

[0285] Condition 5: When the two sub-blocks contained in the target pixel have inter-frame prediction as their prediction type and the reference frames (i.e., refIdx) of each block are different, the BS values ​​of the Y component, U component and V component of the target pixel are set to 1, 0 and 0 respectively.

[0286] Condition 6: If conditions 1 to 5 are not met, set the BS values ​​of the Y, U, and V components of the target pixel to 0, 0, and 0, respectively.

[0287] Furthermore, as described above, the BS value of this embodiment can be the same as the value of each condition in Non-Patent Document 1 determined based on the prediction information of the block unit.

[0288] Therefore, in Figure 17 In the BS value determination of the deblocking filter relative to the applied block of the GPM with additional intra-frame prediction as shown, compared with the structural example disclosed in Non-Patent Document 1, the BS value relative to the target pixel can be determined more accurately, thus optimizing the smoothing of block boundary distortion and improving coding performance.

[0289] The image encoding device 100 and image decoding device 200 described above can be implemented by a program that causes a computer to execute each function (each process).

[0290] Furthermore, in the above embodiments, the present invention has been described as being applied to an image encoding device 100 and an image decoding device 200, but the present invention is not limited thereto, and can also be applied to image encoding systems and image decoding systems having the functions of an image encoding device 100 and an image decoding device 200.

[0291] Industrial availability

[0292] Furthermore, according to this embodiment, for example, since it is possible to improve the overall quality of service in dynamic image communication, it can contribute to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs): "Building resilient infrastructure, promoting sustainable industrialization and enabling expanded innovation."

Claims

1. An image decoding device, characterized in that, have: The inter-frame prediction unit is configured to derive motion information relative to the geometric segmentation pattern and generate motion-compensated pixels. The intra-prediction unit is configured to derive an intra-prediction mode relative to the geometric segmentation mode and generate intra-prediction pixels. as well as A prediction information buffer is configured to store or output motion information of the decoded target block with the applied geometric segmentation mode, or prediction information containing intra-frame prediction mode and prediction type that can determine whether inter-frame prediction or intra-frame prediction is applied. The inter-frame prediction unit and the intra-frame prediction unit are configured to control the content of the prediction information applied to the sub-block unit and the method of saving the prediction information based on the type of prediction information calculated by the sub-block unit constituting the decoding target block.

2. The image decoding device according to claim 1, characterized in that, The inter-frame prediction unit is configured such that, when the type of the stored prediction information in the sub-block is represented as 0 or 1, and the prediction type of the sub-block is inter-frame prediction, in the prediction information buffer, Save inter-frame predictions as the prediction type. The motion information applied to the sub-blocks whose saved prediction information type is 0 or 1 is saved as the motion information. As the intra-frame prediction mode, it is configured not to store any data.

3. The image decoding apparatus according to claim 1, characterized in that, The inter-frame prediction unit is configured such that, when the sub-block has a stored prediction information type representation of 2 and the prediction type of the sub-block is composed of two different inter-frame predictions applied to the sub-blocks with stored prediction information types of 0 and 1, in the prediction information buffer, Save inter-frame predictions as the prediction type. The motion information that is not a zero vector applied to the sub-blocks whose saved prediction information is of type 0 and 1 is saved as the motion information. As the intra-frame prediction mode, it is configured not to store any data.

4. The image decoding apparatus according to claim 1, characterized in that, The inter-frame prediction unit and the intra-frame prediction unit are configured such that, when the sub-block has a stored prediction information type representation of 2 and the prediction type of the sub-block is constituted by inter-frame prediction or intra-frame prediction applied to the sub-block with a stored prediction information type of 0 or 1, in the prediction information buffer, Save inter-frame predictions as the prediction type. The motion information of the applied inter-frame prediction side in the sub-block where the saved prediction information type is 0 or 1 is saved as the motion information. The intra-prediction mode of the sub-block in which the type of the saved prediction information is 0 or 1 is saved as the intra-prediction mode.

5. The image decoding apparatus according to claim 1, characterized in that, The intra-frame prediction unit is configured such that, when the type of the stored prediction information in the sub-block is represented as 0 or 1, and the prediction type of the sub-block is intra-frame prediction, in the prediction information buffer, Save the intra-frame prediction as the prediction type. The motion information consisting of zero vectors is saved as the motion information. The intra-prediction mode applied to the sub-block whose type of the saved prediction information is 0 or 1 is saved as the intra-prediction mode.

6. The image decoding apparatus according to claim 1, characterized in that, The intra-frame prediction unit is configured such that, when the sub-block has the stored prediction information type representation 2 and the prediction type of the sub-block is intra-frame prediction, in the prediction information buffer, Save the intra-frame prediction as the prediction type. The motion information consisting of zero vectors is saved as the motion information. Both of the intra prediction modes applied to the sub-blocks whose saved prediction information types are 0 and 1 are saved as the intra prediction modes.

7. The image decoding apparatus according to claim 1, characterized in that, The intra-frame prediction unit is configured such that, when the sub-block has the stored prediction information type representation 2 and the prediction type of the sub-block is intra-frame prediction, in the prediction information buffer, Save the intra-frame prediction as the prediction type. The motion information consisting of zero vectors is saved as the motion information. The intra-prediction mode is selected and saved by a prescribed method, which is either the intra-prediction mode applied to the sub-block whose type of the saved prediction information is 0 or 1.

8. The image decoding apparatus according to claim 1, characterized in that, The inter-frame prediction unit and the intra-frame prediction unit are configured such that, while storing the motion information, they also store the motion vector corrected by fused motion vector difference or inter-frame template matching as a parameter constituting the motion information.

9. The image decoding apparatus according to any one of claims 1 to 8, characterized in that, The inter-frame prediction unit is configured such that, while storing the motion information, each flag stores an invalid value in the prediction information buffer as a parameter constituting the motion information. Each flag indicates whether a switchable interpolation filter, intra-frame block copying, or local illumination compensation has been applied.

10. The image decoding apparatus according to claim 1, characterized in that, The prediction information buffer is configured to control the deletion method of the prediction information based on the prediction information stored in the decoded target block applying the geometric segmentation mode.

11. The image decoding apparatus according to claim 10, characterized in that, The prediction information buffer is configured as follows: If the prediction information stored in the decoded target block using the applied geometric segmentation mode is the motion information, the prediction information is deleted when the motion information is no longer referenced from within or outside the frame. When the prediction information stored in the decoded target block applying the geometric segmentation mode is the intra-frame prediction mode, the prediction information is deleted when the motion information is no longer referenced from within the frame.

12. An image decoding device, characterized in that, have: The inter-frame prediction unit is configured to derive motion information relative to the geometric segmentation pattern and generate motion-compensated pixels. The intra-prediction unit is configured to derive an intra-prediction mode relative to the geometric segmentation mode and generate intra-prediction pixels. A prediction information buffer is configured to store or output motion information of a decoded target block with the applied geometric segmentation mode or prediction information containing an intra-frame prediction mode and a prediction type that can determine whether inter-frame prediction or intra-frame prediction is applied. as well as The synthesis unit is configured to synthesize a new prediction pixel by performing a weighted average of the motion-compensated pixels and the intra-frame prediction pixels with predetermined weights. The inter-frame prediction unit and the intra-frame prediction unit are configured to control the content of the prediction information applied to the sub-block unit and the method of saving the prediction information based on the type of prediction information calculated by the sub-block unit constituting the decoding target block.

13. The image decoding apparatus according to claim 12, characterized in that, The synthesis unit determines, based on the prediction information stored in the sub-block units of the prediction information buffer, the predetermined weights for synthesizing new prediction pixels by weighting the motion-compensated pixels and the intra-frame prediction pixels with predetermined weights.

14. The image decoding apparatus according to claim 13, characterized in that, The prediction information stored in the sub-block unit of the prediction information buffer consists of prediction type, motion information, and intra-frame prediction mode.

15. An image decoding device, characterized in that, have: The inter-frame prediction unit is configured to derive motion information relative to the geometric segmentation pattern and generate motion-compensated pixels. The intra-prediction unit is configured to derive an intra-prediction mode relative to the geometric segmentation mode and generate intra-prediction pixels. A prediction information buffer is configured to store or output motion information of a decoded target block with the applied geometric segmentation mode or prediction information containing an intra-frame prediction mode and a prediction type that can determine whether inter-frame prediction or intra-frame prediction is applied. as well as The loop filtering processing unit is configured to determine the boundary strength value when applying a deblocking filter to the block boundary and then apply the deblocking filter. The inter-frame prediction unit and the intra-frame prediction unit are configured to control the content of the prediction information applied to the sub-block unit and the method of saving the prediction information based on the type of prediction information calculated by the sub-block unit constituting the decoding target block.

16. The image decoding apparatus according to claim 15, characterized in that, The loop filtering processing unit determines the boundary strength value based on the prediction information stored in the sub-block units of the prediction information buffer.

17. The image decoding apparatus according to claim 16, characterized in that, The prediction information stored in the sub-block unit of the prediction information buffer consists of prediction type, motion information, and intra-frame prediction mode.

18. An image decoding method, characterized in that, have: The process of exporting motion information relative to the geometric segmentation pattern and generating motion-compensated pixels; The process of deriving the intra-prediction mode relative to the geometric segmentation mode and generating intra-prediction pixels. as well as The process of saving or outputting motion information of the decoded target block with the applied geometric segmentation mode, or prediction information including intra-frame prediction mode and prediction type that can determine whether inter-frame prediction or intra-frame prediction is applied. The process of controlling the content of the prediction information applied to the sub-block unit and the method of saving the prediction information based on the type of prediction information calculated by the sub-block units constituting the decoding target block.

Images