Video decoding device and video encoding device

Abstract

To dispense with a search for converting the gradient of a pixel value to an intra direction prediction mode, the search having interfered with concurrent processing in conventional decoder-side intra-mode derivation (DIMD) prediction.SOLUTION: A moving image decoding device comprises: a gradient derivation unit which derives the gradient of a pixel value in a block adjacent to the top and the left of an object block; an angle mode derivation unit which derives an angle mode for deriving an angle mode corresponding to the gradient; an angle mode selection unit which selects the derived angle mode, and a temporal prediction image derivation unit which derives a temporal prediction image based on the angle mode. The device refers to a table including the angle mode corresponding to the gradient and thereby derives the angle mode.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] Embodiments of the present invention relate to a moving image decoding device, a moving image encoding device, and an angle mode derivation device.

Background Art

[0002] In order to efficiently transmit or record a moving image, a moving image encoding device that generates encoded data by encoding the moving image, and a moving image decoding device that generates a decoded image by decoding the encoded data are used.

[0003] Specific moving image encoding methods include, for example, methods proposed in H.264 / AVC and HEVC (High-Efficiency Video Coding).

[0004] In such a moving image encoding method, an image (picture) constituting the moving image is managed by a hierarchical structure composed of a slice obtained by dividing the image, a coding tree unit (CTU: Coding Tree Unit) obtained by dividing the slice, a coding unit (sometimes called a Coding Unit: CU) obtained by dividing the coding tree unit, and a transform unit (TU: Transform Unit) obtained by dividing the coding unit, and is encoded / decoded for each CU. and a prediction image is usually generated based on a local decoded image obtained by encoding / decoding an input image, and a prediction error (sometimes called a "difference image" or a "residual image") obtained by subtracting the prediction image from the input image (original image) is encoded. Examples of the method for generating the prediction image include inter-picture prediction (inter prediction) and intra-picture prediction (intra prediction).

[0005] Also, in such a moving image encoding method, usually, a prediction image is generated based on a local decoded image obtained by encoding / decoding an input image, and a prediction error (sometimes called a "difference image" or a "residual image") obtained by subtracting the prediction image from the input image (original image) is encoded. Examples of the method for generating the prediction image include inter-picture prediction (inter prediction) and intra-picture prediction (intra prediction).

[0006] Furthermore, Non-Patent Document 1 can be cited as a recent technique for video encoding and decoding. Non-Patent Document 1 discloses decoder-side intra-mode derivation (DIMD) prediction, in which the decoder derives a predicted image by deriving an intra-directional predicted mode number using pixels in adjacent regions. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] M. Abdoli, T. Guionnet, E. Mora, et. al, “Non-CE3: Decoder-side Intra Mode Derivation with Prediction Fusion Using Planar”, JVET-O0449, Gothenburg, July 2019. [Overview of the Initiative] [Problems that the invention aims to solve]

[0008] In decoder-side intra-mode derivation methods such as Non-Patent Document 1, the intra-direction prediction mode in the target block is estimated using the gradient of the pixel values ​​in the reference region. However, a problem arises in that the processing load increases because array searching is performed to convert the direction of the gradient into the intra-direction prediction mode.

[0009] The present invention aims to derive a suitable decoder-side intra-mode without increasing the processing load required to derive the intra-mode from the gradient in the decoder-side intra-mode derivation process. [Means for solving the problem]

[0010] To solve the above problems, a video decoding device according to one aspect of the present invention is provided. A gradient derivation unit that derives the gradient of pixel values ​​in blocks adjacent to the target block above and to the left; an angle mode derivation unit that derives an angle mode corresponding to the gradient; an angle mode selection unit that selects the derived angle mode; and a provisional prediction image derivation unit that derives a provisional prediction image based on the angle mode. A motion image decoding device and an angle mode derivation device comprising, which derives an angle mode corresponding to a gradient based on a table reference. It is characterized by the following: [Effects of the Invention]

[0011] According to one aspect of the present invention, suitable intra-mode prediction can be performed without increasing the computational complexity of the decoder-side intra-mode derivation. [Brief explanation of the drawing]

[0012] [Figure 1] This is a schematic diagram showing the configuration of the image transmission system according to this embodiment. [Figure 2] This diagram shows the hierarchical structure of the encoded stream data. [Figure 3] This is a schematic diagram showing the types (mode numbers) of intra-prediction modes. [Figure 4] This is a schematic diagram showing the configuration of a video decoding device. [Figure 5] This diagram shows the configuration of the intra-predictive image generation unit. [Figure 6] This diagram shows the details of the DIMD prediction unit. [Figure 7] This is an example of DIMD syntax. [Figure 8] This is an example of a spatial filter. [Figure 9] This figure shows an example of a pixel for which gradient derivation is performed. [Figure 10] This diagram shows the relationship between the gradient and the region. [Figure 11] This is a block diagram showing the configuration of a video encoding device. [Modes for carrying out the invention]

[0013] (First Embodiment) Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0014] FIG. 1 is a schematic diagram showing the configuration of an image transmission system 1 according to this embodiment.

[0015] The image transmission system 1 is a system that transmits an encoded stream obtained by encoding an image to be encoded and decodes the transmitted encoded stream to display an image. The image transmission system 1 includes a moving image encoding device (image encoding device) 11, a network 21, a moving image decoding device (image decoding device) 31, and a moving image display device (image display device) 41.

[0016] An image T is input to the moving image encoding device 11.

[0017] The network 21 transmits the encoded stream Te generated by the moving image encoding device 11 to the moving image decoding device 31. The network 21 is the Internet, a wide area network (WAN), a local area network (LAN), or a combination thereof. The network 21 is not necessarily limited to a two-way communication network and may be a one-way communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. Also, the network 21 may be replaced by a storage medium that records the encoded stream Te such as a DVD (Digital Versatile Disc: registered trademark) or a BD (Blu-ray Disc: registered trademark).

[0018] The moving image decoding device 31 decodes each of the encoded streams Te transmitted by the network 21 and generates one or more decoded images Td.

[0019] The moving image display device 41 displays all or part of one or more decoded images Td generated by the moving image decoding device 31. The moving image display device 41 includes, for example, a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. Examples of the form of the display include stationary, mobile, and HMD. Also, when the moving image decoding device 31 has high processing power, an image with high image quality is displayed, and when it has only lower processing power, an image that does not require high processing power or display ability is displayed.

[0020] <Operator> The operators used in this specification are described below.

[0021] >> is a right bit shift, << is a left bit shift, & is a bitwise AND, | is a bitwise OR , ^ is a bitwise XOR, |= is an OR assignment operator,! is a logical negation (NOT), && is a logical AND , || indicates a logical OR.

[0022] x?y:z is a ternary operator that takes y when x is true (non-zero) and z when x is false (0).

[0023] Clip3(a, b, c) is a function that clips c to a value between a and b. When c < a, it returns a; when c > b, it returns b; and in other cases, it returns c (where a <= b).

[0024] Clip1Y(c) is an operator where in Clip3(a, b, c), a = 0 and b = (1 << BitDepthY) - 1 are set. BitDepthY is the bit depth of luminance.

[0025] abs(a) is a function that returns the absolute value of a.

[0026] Int(a) is a function that returns the integer value of a.

[0027] Floor(a) is a function that returns the largest integer less than or equal to a.

[0028] Ceil(a) is a function that returns the smallest integer greater than or equal to a.

[0029] a / d represents division of a by d (rounding down to the nearest whole number).

[0030] Min(a,b) is a function that returns the smaller of a and b.

[0031] <Structure of the coded stream Te> Prior to a detailed description of the video encoding device 11 and video decoding device 31 according to this embodiment, the data structure of the encoded stream Te generated by the video encoding device 11 and decoded by the video decoding device 31 will be described.

[0032] Figure 2 shows the hierarchical structure of data in the encoded stream Te. The encoded stream Te includes, exemplarily, a sequence and multiple pictures that make up the sequence. Figure 2 shows, respectively, the encoded video sequence that defines the sequence SEQ, the encoded picture that defines the picture PICT, the encoded slice that defines the slice S, the encoded slice data that defines the slice data, the encoded tree unit contained in the encoded slice data, and the encoded unit contained in the encoded tree unit.

[0033] (Encoded video sequence) In encoded video sequences, video decoding is performed to decode the sequence (SEQ) to be processed. The set of data that device 31 references is defined. Sequence SEQ is the encoded video in Figure 2. As shown in the Osequence, it includes a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), a Picture (PICT), and Supplemental Enhancement Information (SEI).

[0034] The video parameter set VPS is used in video footage composed of multiple layers. A set of encoding parameters common to a number of moving images, as well as sets of encoding parameters associated with multiple layers contained in a moving image and each individual layer, are defined.

[0035] The Sequence Parameter Set (SPS) uses video decoding to decode the target sequence. The set of encoding parameters referenced by position 31 is defined. For example, the width and height of the picture are defined. Note that there may be multiple SPS. In that case, one of the multiple SPS is selected from the PPS. Select this option.

[0036] In the Picture Parameter Set (PPS), to decode each picture in the target sequence... A set of encoding parameters referenced by the video decoding device 31 is defined. For example, it includes a reference value for the quantization width used for decoding the picture (pic_init_qp_minus26) and a flag indicating the application of weighted prediction (weighted_pred_flag). Note that multiple PPSs may exist. In that case, one of the multiple PPSs is selected from each picture in the target sequence.

[0037] (Encoded picture) The encoded picture defines the set of data that the video decoding device 31 references to decode the picture PICT to be processed. As shown in the encoded picture in Figure 2, the picture PICT includes slices 0 to NS-1 (NS being the total number of slices included in the picture PICT). .

[0038] Furthermore, in cases where it is not necessary to distinguish between slices 0 to NS-1, the sign is The subscript may be omitted in the description. The same applies to other data included in the encoded stream Te, which is described below, that has subscripts.

[0039] (Encoded slice) In the encoded slice, the video decoding device 31 references the slice S to be processed in order to decode it. A set of data to be used is defined. A slice includes a slice header and slice data, as shown in the encoded slice in Figure 2.

[0040] The slice header contains a set of encoding parameters that the video decoding device 31 references to determine the decoding method for the target slice. The slice type specification information (slice_type), which specifies the slice type, is an example of the encoding parameters included in the slice header.

[0041] The slice types that can be specified by the slice type specification information include (1) I slices that use only intra prediction during encoding, (2) P slices that use unidirectional prediction or intra prediction during encoding, and (3) B slices that use unidirectional prediction, bidirectional prediction or intra prediction during encoding. Note that interpretation is not limited to single or bidirectional prediction, and prediction images may be generated using more reference pictures. Hereinafter, P When referred to as a B-slice, it is a slice containing blocks where interpretation can be used. It refers to S.

[0042] The slice header may also include a reference to the picture parameter set PPS (pic_parameter_set_id).

[0043] (Encoded slice data) The encoded slice data defines the set of data that the video decoding device 31 references to decode the slice data to be processed. The slice data includes a CTU, as shown in the encoded slice header of Figure 2. A CTU is a fixed-size (e.g., 64x64) block that makes up a slice, and is sometimes called a Largest Coding Unit (LCU).

[0044] (Code tree unit) The coding tree unit in Figure 2 includes a video decoding device 31 for decoding the CTU to be processed. The set of data that it references is defined. The CTU is divided into coding units CU, which are the basic units of coding, by recursive quad tree partitioning (QT), binary tree partitioning (BT), or ternary tree partitioning (TT). BT and TT partitioning together are called multi-tree partitioning (MT). The nodes of the tree structure obtained by recursive quad tree partitioning are called coding nodes. The intermediate nodes of quad trees, binary trees, and ternary trees are coding nodes, and the CTU itself is defined as the top-level coding node.

[0045] (Encoding Unit) As shown in the coding unit of Figure 2, a set of data that the video decoding device 31 references to decode the coding unit to be processed is defined. Specifically, the CU consists of a CU header CUH, prediction parameters, transformation parameters, quantization transformation coefficients, etc. The prediction mode, etc., is defined in the CU header.

[0046] Prediction processing can be performed at the CU (Unit) level or at the subCU level, which is a further division of the CU. If the size of the CU and the subCU are equal, there is one subCU within the CU. If the CU is larger than the size of the subCU, the CU is divided into subCUs. For example, if the CU is 8x8 and the subCU is 4x4, the CU will be divided into four subCUs, each consisting of two horizontal and two vertical divisions.

[0047] There are two types of predictions (prediction modes): intra-prediction and inter-prediction. Intra-prediction is prediction within the same picture, while inter-prediction refers to prediction processing performed between different pictures (for example, between display times or between layer images).

[0048] The conversion and quantization processes are performed in units of CUs, but the quantization conversion coefficients are in sub-block units such as 4x4. Entropy coding can also be used.

[0049] (Prediction parameters) The predicted image is derived from the prediction parameters associated with the block. These prediction parameters include intra-prediction and inter-prediction parameters.

[0050] The following describes the prediction parameters for intra-prediction. The intra-prediction parameters consist of the luminance prediction mode IntraPredModeY and the color difference prediction mode IntraPredModeC. Figure 3 is a schematic diagram showing the types (mode numbers) of intra-prediction modes. As shown in the figure, there are, for example, 67 types (0 to 66) of intra-prediction modes. For example, planar prediction (0), DC prediction (1), and angular prediction (2 to 66). In addition, linear model (LM) predictions such as cross-component linear model (CCLM) prediction and multi-mode linear model (MMLM) prediction may be used. Furthermore, an LM mode may be added for color difference.

[0051] (Configuration of the video decoding device) The configuration of the video decoding device 31 (Figure 4) according to this embodiment will be described below.

[0052] The video decoding device 31 includes an entropy decoding unit 301 and a parameter decoding unit (predictive video decoding device). The system includes a 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a prediction image generation unit (prediction image generation device) 308, an inverse quantization / inverse transform unit 311, and an adder 312. This is accomplished. Furthermore, in accordance with the video encoding device 11 described later, there is also a configuration in which the video decoding device 31 does not include a loop filter 305.

[0053] Furthermore, the parameter decoding unit 302 is configured to include an inter-prediction parameter decoding unit 303 and an intra-prediction parameter decoding unit 304 (not shown). The prediction image generation unit 308 is configured to include an inter-prediction image generation unit 309 and an intra-prediction image generation unit 310.

[0054] Furthermore, the following examples will use CTU and CU as processing units, but this is not limited to these examples. Alternatively, you can process on a sub-CU basis. You can also interpret this as "ku" and process it as a block or subblock unit.

[0055] The entropy decoding unit 301 applies the entropy to the encoded stream Te input from the outside. P-decoding is performed to separate and decode individual codes (syntax elements). Entropy coding has two methods: one that uses a context (probability model) adaptively selected according to the type of syntax element and the surrounding circumstances to encode syntax elements of variable length, and another that uses a predetermined table or calculation formula to encode syntax elements of variable length. The former, CABAC (Context Adaptive Binary Arithmetic Coding), stores an updated probability model in memory for each encoded or decoded picture (slice). Then, P-picture, Alternatively, the initial state of the B-picture context could be a probabilistic model stored in memory. From this, a probabilistic model of the picture is set using the same slice type and quantization parameters at the same slice level. This initial state is used for encoding and decoding. The separated codes include prediction information for generating the predicted image and prediction errors for generating the difference image.

[0056] The entropy decoding unit 301 outputs the separated codes to the parameter decoding unit 302. The control of which codes to decode is performed based on the instructions of the parameter decoding unit 302.

[0057] (Configuration of the intra-prediction parameter decoding unit 304) The intra-prediction parameter decoding unit 304 uses the code input from the entropy decoding unit 301 to predict the intra-prediction parameter stored in the prediction parameter memory 307. Decode the parameters, for example, IntraPredMode. The meter decoding unit 304 outputs the decoded intra-prediction parameters to the prediction image generation unit 308 and also stores them in the prediction parameter memory 307. The intra-prediction parameter decoding unit 304 may derive different intra-prediction modes based on luminance and color difference.

[0058] The intra-prediction parameter decoding unit 304 decodes the syntax elements related to intra-prediction as shown in Figure 7.

[0059] Loop filter 305 is a filter placed within the encoding loop to correct block distortion and ringing. This filter removes distortion and improves image quality. The loop filter 305 applies a deblocking filter, sample adaptive offset (SAO), and other filters to the decoded CU image generated by the summing unit 312. Apply filters such as an automatic loop filter (ALF).

[0060] The reference picture memory 306 stores the decoded image of the CU generated by the adder 312 at a predetermined location for each target picture and target CU.

[0061] The prediction parameter memory 307 stores prediction parameters at predetermined locations for each CTU or CU to be decoded. Specifically, the prediction parameter memory 307 stores the parameters decoded by the parameter decoding unit 302 and the prediction mode predMode separated by the entropy decoding unit 301. ru.

[0062] The prediction image generation unit 308 receives input such as the prediction mode (predMode) and prediction parameters. The predictive image generation unit 308 reads a reference picture from the reference picture memory 306. The predictive image generation unit 308 uses the prediction mode indicated by the prediction mode predMode to read the prediction parameters and reference picture. A block or subblock of predicted image is generated using a reference picture (reference picture block). Here, a reference picture block is a set of pixels on the reference picture (usually rectangular, hence called a block), and is the region referenced to generate the predicted image.

[0063] (Intra predictive image generation unit 310) When the prediction mode predMode indicates intra-prediction mode, the intra-prediction image generation unit 310 The intra-prediction parameters input from the intra-prediction parameter decoding unit 304 and the reference pixel Intra prediction is performed using the reference pixels read from memory 306.

[0064] Specifically, the intra predictive image generation unit 310 predicts from the target block on the target picture. Reads adjacent blocks within a predetermined range from reference picture memory 306. The "enclosure" refers to the adjacent blocks to the left, upper left, top, and upper right of the target block, and the area referenced differs depending on the intra-prediction mode.

[0065] The intra-predictive image generation unit 310 generates a predicted image of the target block by referring to the read decoded pixel value and the prediction mode indicated by IntraPredMode. The predicted image of the block is output to the summing unit 312.

[0066] The generation of predicted images based on the intra prediction mode is described below. In planar prediction, DC prediction, and Angular prediction, the decoded surrounding region adjacent to (nearby) the block to be predicted is used. This is set as the illumination region R. Then, the predicted image is generated by extrapolating pixels on the reference region R in a specific direction. For example, the reference region R may be set as an L-shaped region that includes the left and top (or even the upper left, upper right, and lower left) of the block to be predicted.

[0067] (Details of the predictive image generation unit) Next, Figure 5 will be used to explain the details of the configuration of the intra predictive image generation unit 310. The intra predictive image generation unit 310 consists of a reference sample filter unit 3103 (second reference image setting unit) and a prediction unit 3104 The system also includes a predictive image correction unit 3105 (predictive image correction unit, filter switching unit, and weight coefficient changing unit).

[0068] Based on each reference pixel (reference image) on the reference region R, the filtered reference image generated by applying a reference pixel filter (first filter), and the intra-prediction mode, the prediction unit 3104 generates a provisional prediction image (pre-correction prediction image) of the block to be predicted and outputs it to the prediction image correction unit 3105. The prediction image correction unit 3105 corrects the provisional prediction image according to the intra-prediction mode, generates a prediction image (corrected prediction image), and outputs it.

[0069] The following describes the various components of the intra-predictive image generation unit 310.

[0070] (Reference sample filter section 3103) The reference sample filter unit 3103 refers to the reference image and selects the reference values ​​for each position (x,y) on the reference region R. The illuminated sample s[x][y] is derived. Furthermore, the reference sample filter unit 3103 is an intra-prediction model. Depending on the code, apply the reference pixel filter (first filter) to the reference sample s[x][y]. The reference samples s[x][y] for each position (x,y) on the reference region R are updated (a filtered reference image s[x][y] is derived). Specifically, a low-pass filter is applied to the reference image at position (x,y) and its surroundings to derive a filtered reference image. Note that it is not necessary to apply a low-pass filter to all intra-prediction modes; a low-pass filter may be applied to some intra-prediction modes. The filter applied to the reference image on the reference region R in the reference sample filter unit 3103 is called the "reference pixel filter (first filter)," while the filter used to correct the provisional predicted image in the predicted image correction unit 3105, described later, is called the "position-dependent filter (second filter)."

[0071] (Configuration of the intra-prediction unit 3104) The intra-prediction unit 3104 generates a provisional prediction image (provisional prediction pixel values, pre-correction prediction image) of the block to be predicted based on the intra-prediction mode, the reference image, and the filtered reference pixel values, and outputs it to the prediction image correction unit 3105. The prediction unit 3104 internally includes a Planar prediction unit 31041, a DC prediction unit 31042, an Angular prediction unit 31043, an LM prediction unit 31044, an MIP (Matrix-based Intra Prediction) unit 31045, and a DIMD prediction unit 31046. The prediction unit 3104 performs a specific prediction according to the intra-prediction mode. Select the section and input the reference image and filtered reference image. The relationship between the intra prediction mode and the corresponding prediction section is as follows: • Planar prediction • Planar prediction unit 31041 • DC prediction • DC prediction unit 31042 ·Angular prediction ····Angular prediction unit 31043 • LM prediction • LM prediction unit 31044 • Matrix Intranet Prediction • MIP Department 31045 ·DIMD prediction······DIMD prediction unit 31046 (Planar prediction) The Planar prediction unit 31041 uses a reference sample based on the distance between the pixel position to be predicted and the reference pixel position. A provisional prediction image is generated by linearly adding s[x][y] and output to the prediction image correction unit 3105.

[0072] (DC prediction) The DC prediction unit 31042 derives a DC prediction value corresponding to the average value of the reference sample s[x][y], and outputs a provisional prediction image q[x][y] with the DC prediction value as the pixel value.

[0073] (Angular prediction) The Angular prediction unit 31043 generates a provisional prediction image q[x][y] using the reference sample s[x][y] in the prediction direction (reference direction) indicated by the intra prediction mode, and outputs it to the prediction image correction unit 3105.

[0074] (LM prediction) The LM prediction unit 31044 predicts the pixel value of color difference based on the pixel value of brightness. Specifically, This method generates predicted color difference images (Cb, Cr) using a linear model based on the generated luminance image. One type of LM prediction is CCLM (Cross-Component Linear Model prediction). There is a prediction method called CCLM prediction. CCLM prediction is a prediction method that uses a linear model to predict color difference from luminance for each block.

[0075] (Matrix intra prediction) The MIP unit 31045 performs a sum-of-products operation on the reference sample s[x][y] derived from the adjacent region and the weight matrix. A provisional predicted image q[x][y] is generated and output to the predicted image correction unit 3105.

[0076] (DIMD prediction) The DIMD prediction unit 31046 predicts the image using an intra-prediction mode that is not explicitly signaled. This is a prediction method that generates the parameters. The intra-prediction parameter decoding unit 304 uses information from adjacent regions. The DIMD prediction unit 31046 then derives an intra prediction mode suitable for the target block, and this intra... Predictive images are generated using the prediction mode. Details will be described later.

[0077] (Configuration of the predictive image correction unit 3105) The prediction image correction unit 3105 corrects the provisional prediction image output from the prediction unit 3104 according to the intra prediction mode. Specifically, for each pixel of the provisional prediction image, the prediction image correction unit 3105 derives a position-dependent weight coefficient according to the reference region R and the position of the target prediction pixel. Then, by weighted addition (weighted average) of the reference sample s[][] and the provisional prediction image q[x][y], it derives a corrected prediction image (corrected prediction image) Pred[][]. Note that in some intra prediction modes, the prediction image correction unit 3105 does not correct the provisional prediction image q[x][y] and the prediction image is... You can set it up like this.

[0078] (Example 1) Figure 6 shows the configuration of the DIMD prediction unit 31046 in this embodiment. The DIMD prediction unit 31046 is It consists of a gradient derivation unit 310461, an angle mode derivation unit 310462, an angle mode selection unit 310463, and a provisional prediction image generation unit 310464.

[0079] Figure 7 shows an example of the syntax for encoded data related to DIMD. Intra prediction parameter The meter decoding unit 304 decodes a flag dimd_flag from the encoded data for each block, indicating whether DIMD is used. If dimd_flag is 1, the subsequent luminance intra prediction mode and MPM are used. It is not necessary to decode the related syntax elements.

[0080] If dimd_flag is 1, the DIMD prediction unit 31046 indicates the corner of the texture direction in the adjacent region. The degree is derived using the pixel value. Then, a provisional prediction image is generated using the intra prediction mode corresponding to that angle. For example, (1) for a pixel at a predetermined position in an adjacent region, the pixel value (2) Derive the gradient direction. (3) Use the derived gradient direction to determine the corresponding direction prediction mode (Angular prediction). (3) Convert to mode. (4) Create a histogram of the obtained prediction direction for each predetermined pixel in the adjacent region. (5) From the histogram, include the prediction mode of the most frequent value or the most frequent value. Multiple prediction modes are selected, and a provisional prediction image is generated using the selected prediction mode. The processing in each part of the DIMD prediction unit 31046 shown in Figure 6 will be explained in more detail below.

[0081] (1) Gradient derivation unit (image-based angle derivation unit) The gradient derivation unit 310461 calculates the texture based on the image data of the adjacent region of the target block. The angle indicating the direction (angle information) is derived. The angle information may be a value representing an angle with 1 / 36 accuracy, or it may be any other value. The gradient derivation unit 310461 derives gradients (e.g., Dx, Dy) in two or more specific directions, and derives the direction of the gradient (angle information) from the relationship between gradients Dx and Dy.

[0082] A spatial filter may be used to derive the gradient. For example, a 3x3 pixel Sobel filter corresponding to the horizontal and vertical directions, as shown in Figures 8(a) and (b), can be used as the spatial filter. This is also acceptable. For N points Pn[0][0] (n=0..N-1) in the adjacent region that are the target locations for gradient derivation, the gradient The following is derived. Figure 9(a) shows an example of the position of the pixels for gradient derivation in an 8x8 pixel target block. This is shown. Also, Figure 9(b) shows the target pixels for gradient derivation in a 4x4 pixel block. An example of positioning is shown. Shaded pixels in the adjacent region of the target block are the pixels for gradient derivation. In this way, the number of pixels for gradient derivation, the positional pattern, and the reference range of the spatial filter may be changed depending on information such as the size of the target block and the prediction mode of the blocks included in the adjacent region.

[0083] Specifically, the gradient derivation unit 310461 derives the horizontal and vertical gradients Dx and Dy for each point Pn as shown in the following equation. Dx = Pn[-1][-1] + 2*Pn[-1][0] + Pn[-1][1] - Pn[1][-1] - 2*Pn[1][0] - Pn[1][1] Dy = - Pn[-1][-1] - 2*Pn[0][-1] - Pn[1][-1] + Pn[-1][1] + 2*Pn[0][1] + Pn[1][1] Here, Pn[x][y] is the pixel value at the pixel expressed in relative coordinates from point Pn([0][0]) at the position for gradient derivation. Figure 8(c) is obtained by flipping the filters in Figures 8(a) and (b) horizontally or vertically. You may also use the filter (d). In that case, derive Dx and Dy using the following equation. Dx = - Pn[-1][-1] - 2*Pn[-1][0] - Pn[-1][1] + Pn[1][-1] + 2*Pn[1][0] + Pn[1][1] Dy = Pn[-1][-1] + 2*Pn[0][-1] + Pn[1][-1] - Pn[-1][1] - 2*Pn[0][1] - Pn[1][1] The method for deriving the gradient is not limited to this; other methods (filters, formulas, tables, etc.) may be used. For example, a Prewitt filter or Scherr filter may be used instead of a Sobel filter, and the filter size may be set to 2x2 or 5x5. The gradient derivation unit 310461 is a Prewitt filter Using Ruta, we derive Dx and Dy as follows. Dx = Pn[-1][-1] + Pn[-1][0] + Pn[-1][1] - Pn[1][-1] - Pn[1][0] - Pn[1][1] Dy = - Pn[-1][-1] - Pn[0][-1] - Pn[1][-1] + Pn[-1][1] + Pn[0][1] + Pn[1][1] The following equation is an example of deriving Dx and Dy using a Scharr filter. Dx = 3*Pn[-1][-1]+10*Pn[-1][0]+3*Pn[-1][1] -3*Pn[1][-1]-10*Pn[1][0]-3*Pn[1][1] Dy = -3*Pn[-1][-1]-10*Pn[0][-1]-3*Pn[1][-1] +3*Pn[-1][1]+10*Pn[0][1]+3*Pn[1][1] The method for deriving the gradient may be changed for each block. For example, for a 4x4 pixel target block... For smaller blocks, we use the Sobel filter, and for blocks larger than 4x4, we use the Scherr filter. By using filters with simpler operations for smaller blocks in this way, we can suppress the increase in computational complexity for smaller blocks.

[0084] The method for deriving the gradient may be changed for each position of the gradient-deriving symmetric pixel. For example, for gradient-derivation target pixels within the upper or left adjacent region, a Sobel filter may be used, while for the upper left adjacent region... For gradient-derived symmetric pixels within the structure, use a Scharr filter.

[0085] The gradient derivation unit 310461 calculates the texture of the target block based on the signs and magnitude relationship of Dx and Dy. The quadrant of an angle (hereinafter referred to as a region) and the angle within that quadrant. Degree information is derived. Depending on the region, it becomes possible to standardize the processing of directions that are rotationally symmetric or lineally symmetric. However, the angle information is not limited to angles within the region and quadrant. For example, only angle information may be used, and the region may be derived as needed. Also, in this embodiment, the intra-direction prediction mode derived below is limited to the direction from the bottom left to the top right (2 to 66 in Figure 3), and the intra-direction prediction mode for directions with 180-degree rotational symmetry is treated the same as this.

[0086] Figure 10(a) is a table showing the signs (signx, signy), magnitude relationship (xgty) of Dx and Dy, and the relationship between the regions (Ra~Rd are constants representing the region numbers). Figure 10(b) shows the quadrants represented by the regions Ra~Rd. The gradient derivation unit 310461 derives signx, signy, and xgty as follows. absx = abs(Dx) absy = abs(Dy) signx = Dx < 0 ? 1 : 0 signy = Dy < 0 ? 1 : 0 xgty = absx > absy ? 1 : 0 Here, inequality signs (>, <) can also be equality signs (>=, <=). The region represents a rough angle and can be derived only from the signs signx, signy of Dx and Dy and the inequality xgty.

[0087] The gradient derivation unit 310461 derives region from the signs signx, signy and the magnitude relationship xgty using operations or table lookups. The gradient derivation unit may derive the corresponding region by referring to the table of Fig. 10(a). It may be derived.

[0088] The gradient derivation unit 310461 may also derive region using the following logical formula. region = xgty? ( (signx^signy)? 1 : 0 ) : ( (signx^signy)? 2 : 3) Here, ^ represents XOR (exclusive OR). region is represented by a value from 0 to 3. {Ra, Rb, Rc, Rd} = {0, 1, 2, 3}. Note that the assignment method of the region values is not limited to the above.

[0089] The gradient derivation unit 310461 may also derive region using another logical formula and addition / multiplication as follows. region = 2*(!xgty) + (signx^signy^!xgty) Here, the symbol! means logical negation.

[0090] (2) Angle mode derivation unit The angle mode derivation unit 310462 derives an angle mode (a prediction mode corresponding to the gradient, for example, an intra prediction mode) based on the gradient information of point Pn.

[0091] The angle mode derivation unit 310462 derives a slope iRatio (= absy ÷ absx) based on the absolute values of the gradient absx, absy. Here, as iRatio, an integer representing ratio in steps of 1 / R_UNIT is used. iRatio = int(R_UNIT*absy / absx) ≒ ratio*R_UNIT R_UNIT uses a power of 2 (1<<shiftR), for example, 65536 (shiftR = 16), etc.

[0092] The angle mode derivation unit 310462 derives idx from iRatio, uses idx to refer to a lookup table (LUT), and derives the angle mode difference value mode_delta, which is difference information from the reference direction. idx = iRatio >> 10 mode_delta = LUT[idx] Here, LUT[N_LUT] = {0,1,1,2,2,3,3,4, 4,4,5,5,5,5,6,6, 6,6,7,7,7,7,8,8, 8,8,9,9,9,9,10,10, 10,10,11,11,11,11,12,12, 12,12,12,13,13,13,13,13, 13,14,14,14,14,14,14,15, 15,15,15,15,15,15,16,16,16,16}; Here, N_LUT indicates the number of elements in the LUT. N_LUT = (1 << (shiftR - shiftI)) + 1 Furthermore, when iRatio is approximately 0, i.e., idx == 0, the gradient is either horizontal or vertical. Yes, and it can be easily determined that the difference from the reference direction is 0. Therefore, the first element 0 of the LUT is omitted, and the LUT is constructed with N_LUT' = (1 << (shiftR - shiftI)) powers of 2 elements. This is also acceptable. In this case, the lookup table LUT' would be as follows: LUT'[N_LUT'] = {1,1,2,2,3,3,4,4, 4,5,5,5,5,6,6,6, 6,7,7,7,7,8,8,8, 8,9,9,9,9,10,10,10, 10,11,11,11,11,12,12,12, 12,12,13,13,13,13,13,13, 14,14,14,14,14,14,15,15, 15,15,15,15,16,16,16,16}; At this point, the angle mode derivation unit derives mode_delta using the following equation. mode_delta = idx ? LUT'[idx-1] : 0 In this way, since the number of elements N_LUT' is a power of 2, memory utilization efficiency can be improved.

[0093] The angle mode derivation unit 310462 adds mode_delta to the reference direction base_mode[region]. Alternatively, the angle mode modeVal can be derived by subtraction. base_mode[4] = {18, 18, 50, 50} direction[4] = {-1, 1, -1, 1} modeVal = base_mode[region] + direction[region] * mode_delta A value of 18 for base_mode represents the horizontal intra-direction prediction (HOR), and 50 represents the vertical intra-direction prediction (VER). It means...

[0094] Note that this is not the only way to derive modeVal. The angle mode derivation unit 310462 may use rounding operations as follows when deriving idx. stomach. idx = (iRatio + roundI) >> shiftI Here, roundI = 1 << (shiftI - 1), where shiftI is a predetermined constant such as 10.

[0095] In the angle mode derivation unit 310462, the number of elements in the LUT N_LUT can be 17 (shiftR-shiftI=4), 33 (shiftR-shiftI=5), 129 (shiftR-shiftI=7), in addition to 65 (shiftR-shiftI=6) in the above example. Other numbers such as 257 (shiftR-shiftI=8) are also acceptable.

[0096] (3) Gradient mode selection section The angle mode selection unit 310463 uses the modeVal(modeValPn) values ​​at multiple points Pn included in the reference region corresponding to the target block to derive representative values ​​dimdModeVal(dimdModeVal0, dimdModeVal1, ...) for one or more angle modes. In this embodiment, the representative values ​​of the angle modes are estimated values ​​of the directionality of the texture pattern of the target block. Here, the representative value dimdModeVal is derived from the mode derived using a histogram. The angle mode values ​​modeValPn obtained for each point Pn are histogrammed, and the first mode dimdModeVal0 and the second mode dimdModeVal1 are derived by selecting the mode with the highest frequency and the next highest frequency in the histogram, respectively.

[0097] Note that the method for deriving dimdModeVal0 or dimdModeVal1 is not limited to a histogram. For example, the angle mode selection unit 310463 may use the average value of modeValPn as dimdModeVal0 or dimdModelVal1.

[0098] The angle mode selection unit 310463 sets a predetermined intra-prediction mode in dimdModeVal2 as a third mode. Here, dimdModeVal2=0 (Planar) is used, but this is not the only option. Other modes may be set adaptively, or the third mode may not be used at all.

[0099] The angle mode selection unit 310463 also derives weights corresponding to representative values ​​for each angle mode. For example, the weight for the third mode is set to w2 = 21, and the remainder to the first and second modes in the histogram. The weights w0 and w1 are allocated according to the ratio of their frequencies. The total weight is 64. The derivation of the weights is not limited to this, and the weights w0, w1, and w2 of the first, second, and third modes may be changed adaptively. For example, w2 may be increased or decreased according to the number of the first or second mode, or their frequency or ratio. The angle mode selection unit sets the corresponding weight value to 0 for any of the first to third modes if that mode is not used.

[0100] As described above, the angle mode selection unit 310463 selects an intra-prediction mode (a representative value of the angle mode) estimated from the gradient and outputs it along with the weight corresponding to each intra-prediction mode.

[0101] Furthermore, the configuration of the gradient derivation unit, angle mode derivation unit, and angle mode selection unit can also be used to derive information other than the intra-prediction mode. For example, if the angle mode selection unit selects a residual transformation matrix using the directionality of the texture pattern derived by the gradient derivation unit and angle mode derivation unit, it is possible to select a matrix suitable for the residual transformation in the region with that directionality, and an improvement in coding efficiency can be expected.

[0102] (4) Predictive image generation unit The provisional prediction image generation unit 310464 generates a provisional prediction image using one or more input intra-prediction modes. If there is one intra-prediction mode, it generates an intra-prediction image using that prediction mode and outputs it as a provisional prediction image q[x][y]. If there are multiple intra-prediction modes... Next, predictive images (pred0, pred1, pred2) are generated for each prediction mode. Multiple predictive images are combined using the corresponding weights (w0, w1, w2) and output as a provisional predictive image q[x][y]. The provisional predictive image q[x][y] is derived as follows. q[x][y] = (w0 * pred0[x][y] + w1 * pred1[x][y] + w2 * pred2[x][y]) >> 6 However, the frequency of the second mode is 0, or it is not a direction prediction mode (such as DC mode). In this case, the predicted image pred0 from the first prediction mode is used as the provisional predicted image.

[0103] The inverse quantization / inverse transformation unit 311 inversely quantizes the quantization transformation coefficients input from the entropy decoding unit 301 to obtain the transformation coefficients. These quantization transformation coefficients are obtained by quantizing the prediction error in the encoding process by performing frequency transformations such as DCT (Discrete Cosine Transform) and DST (Discrete Sine Transform). The inverse quantization / inverse transformation unit 311 performs inverse frequency transformations such as inverse DCT and inverse DST on the transformation coefficients to calculate the prediction error. The inverse quantization / inverse transformation unit 311 outputs the prediction error to the summing unit 312.

[0104] The addition unit 312 adds the predicted image of the block input from the prediction image generation unit 308 and the prediction error input from the inverse quantization / inverse transform unit 311 pixel by pixel to generate a decoded image of the block. The summing unit 312 stores the decoded image of the block in the reference picture memory 306 and also outputs it to the loop filter 305.

[0105] (Configuration of the video encoding device) Next, the configuration of the video encoding device 11 according to this embodiment will be described. Figure 11 is a block diagram showing the configuration of the video encoding device 11 according to this embodiment. The video encoding device 11 includes a prediction image generation unit 101, a subtraction unit 102, a transformation / quantization unit 103, an inverse quantization / inverse transformation unit 105, an addition unit 106, a loop filter 107, a prediction parameter memory (prediction parameter storage unit, frame memory) 108, a reference picture memory (reference image storage unit, frame memory) 109, an encoding parameter determination unit 110, a parameter encoding unit 111, and an entropy encoding unit 104.

[0106] The predictive image generation unit 101 generates a predictive image for each CU, which is a region obtained by dividing each picture of image T. The predictive image generation unit 101 operates in the same way as the predictive image generation unit 308, which has already been described, so its explanation will be omitted.

[0107] The subtraction unit 102 subtracts the pixel values ​​of the predicted image of the block input from the predicted image generation unit 101 from the pixel values ​​of image T to generate a prediction error. The subtraction unit 102 outputs the prediction error to the conversion / quantization unit 103.

[0108] The conversion / quantization unit 103 calculates conversion coefficients by frequency conversion for the prediction error input from the subtraction unit 102, and derives quantized conversion coefficients by quantization. The quantization conversion coefficients are output to the entropy coding unit 104 and the inverse quantization / inverse conversion unit 105.

[0109] The inverse quantization / inverse transformation unit 105 is the inverse quantization / inverse transformation unit 311 (Figure 4) in the video decoding device 31. This is the same as before, so the explanation is omitted. The calculated prediction error is output to the summing unit 106.

[0110] The entropy coding unit 104 receives quantization conversion coefficients from the conversion / quantization unit 103 and coding parameters from the parameter coding unit 111. The entropy coding unit 104 entropy codes the division information, prediction parameters, quantization conversion coefficients, etc., to generate and output a coded stream Te.

[0111] The parameter coding unit 111 includes a header coding unit 1110 (not shown), a CT information coding unit 1111, and a CU The coding unit 1112 (prediction mode coding unit), and the interprediction parameter coding unit 112 and It includes an interpretation parameter coding unit 113. The CU coding unit 1112 further includes a TU coding unit 1114.

[0112] (Configuration of the intra-predictive parameter coding unit 113) The intra-prediction parameter coding unit 113 encodes IntraPredMode and other parameters input from the coding parameter determination unit 110. The intra-prediction parameter coding unit 113 includes a configuration that is partially identical to the configuration in which the intra-prediction parameter decoding unit 304 derives the intra-prediction parameters.

[0113] The addition unit 106 generates a decoded image by adding the pixel values ​​of the predicted image of the block input from the predicted image generation unit 101 and the prediction error input from the inverse quantization / inverse transform unit 105, pixel by pixel. Unit 106 stores the generated decoded image in the reference picture memory 109.

[0114] The loop filter 107 applies a deblocking filter, SAO, and ALF to the decoded image generated by the summing unit 106. Note that the loop filter 107 does not necessarily include the three types of filters mentioned above. It is not necessary; for example, a configuration consisting only of a deblocking filter is acceptable.

[0115] The prediction parameter memory 108 stores the prediction parameters generated by the coding parameter determination unit 110 in predetermined locations for each target picture and CU.

[0116] The reference picture memory 109 stores the decoded images generated by the loop filter 107 at predetermined locations for each target picture and CU.

[0117] The coding parameter determination unit 110 selects one set from among multiple sets of coding parameters. Select the appropriate option. The coding parameters are the QT, BT, or TT segmentation information, prediction parameters, or parameters that are generated in relation to these and are subject to coding. The prediction image generation unit 101 generates a prediction image using these coding parameters.

[0118] The coding parameter determination unit 110 determines the amount of information and coding for each of the multiple sets. The RD cost value, which indicates the error, is calculated. The coding parameter determination unit 110 determines the calculated cost value The minimum set of coding parameters is selected. The entropy coding unit 104 then outputs the selected set of coding parameters as a coded stream Te. The coding parameter determination unit 110 stores the determined coding parameters in the prediction parameter memory 108.

[0119] Furthermore, some parts of the video encoding device 11 and video decoding device 31 in the above-described embodiment, such as the entropy decoding unit 301, parameter decoding unit 302, loop filter 305, and predictive image generation, are included. Unit 308, inverse quantization / inverse transformation unit 311, addition unit 312, prediction image generation unit 101, subtraction unit 102, transformation / quantization The sub-quantization unit 103, entropy coding unit 104, inverse quantization / inverse transform unit 105, loop filter 107, coding parameter determination unit 110, and parameter coding unit 111 may be implemented by a computer. In that case, the program for implementing this control function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be loaded into a computer system and executed. The term "computer system" here refers to a computer system built into either the video encoding device 11 or the video decoding device 31, and includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and storage devices such as hard disks built into a computer system. Furthermore, the term "computer-readable recording medium" may also include those that dynamically hold programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or communication lines such as telephone lines, and those that hold programs for a certain period of time, such as volatile memory inside a computer system that acts as a server or client in such a case. Furthermore, the above program may be intended to implement only a part of the functions described above, and may also be able to implement the above functions in combination with programs already recorded in the computer system.

[0120] Furthermore, some or all of the video encoding device 11 and video decoding device 31 in the above-described embodiment may be implemented as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the encoding device 11 and the video decoding device 31 may be implemented as a separate processor, or some or all of them may be integrated into a single processor. Furthermore, the method of implementing the integrated circuit is not limited to LSIs; it may also be implemented using dedicated circuits or general-purpose processors. In addition, if advances in semiconductor technology lead to the emergence of integrated circuit implementation technologies that can replace LSIs, integrated circuits using such technologies may be used.

[0121] Although one embodiment of this invention has been described in detail above with reference to the drawings, the specific configuration is not limited to that described above, and various design changes can be made without departing from the spirit of this invention. [Industrial applicability]

[0122] Embodiments of the present invention can be suitably applied to a video decoding device that decodes encoded data from image data, and a video encoding device that generates encoded data from image data. Furthermore, they can be suitably applied to the data structure of encoded data generated by the video encoding device and referenced by the video decoding device. [Explanation of Symbols]

[0123] 31 Image Decoder 301 Entropy Decoder 302 Parameter Decoding Unit 303 Interpretation parameter decoding unit 304 Intra Prediction Parameter Decoding Unit 308 Predictive Image Generation Unit 309 Interpretation Image Generation Unit 310 Intra Predictive Image Generation Unit 31046 DIMD Prediction Unit 311 Inverse Quantization / Inverse Transformation Section 312 Addition section 11 Image encoding device 101 Predictive Image Generation Unit 102 Subtraction Unit 103 Conversion / Quantization Section 104 Entropy coding unit 105 Inverse Quantization / Inverse Transformation Section 107 Loop Filter 110 Encoding parameter determination unit 111 Parameter coding section 112 Interpretation Parameter Coding Unit 113 Intra Prediction Parameter Coding Unit 1110 Header Encoding Section 1111 CT information encoder 1112 CU coding unit (predictive mode coding unit) 1114 TU encoder

Claims

1. A gradient derivation unit that derives the gradient of each pixel value in the blocks adjacent to the target block above and to the left, An angle mode derivation unit that derives angle modes corresponding to each of the above gradients, An angle mode selection unit that selects a representative value for each of the angle modes described above, A motion image decoding device comprising: a provisional prediction image generation unit that generates a provisional prediction image based on a representative value of the above angle mode, The above angle mode is derived from the index of the angle mode table derived using the above gradient and from the said angle mode table. A video decoding device characterized by generating a predicted image by correcting the above-mentioned provisional predicted image according to the intra-prediction mode.

2. The motion image decoding device according to claim 1, characterized in that the angle mode described above is difference information from the reference direction of the intra-direction prediction mode corresponding to the gradient.

3. The motion image decoding device according to claim 1, characterized in that the above index is derived using the slope of the absolute value of the above gradient.

4. The motion image decoding device according to claim 1, characterized in that the angle mode selection unit sets the average value of the angle mode as a representative value of the angle mode.

5. A gradient derivation unit that derives the gradient of each pixel value in the blocks adjacent to the target block above and to the left, An angle mode derivation unit that derives angle modes corresponding to each of the above gradients, An angle mode selection unit that selects a representative value for each of the angle modes described above, A video encoding device comprising: a provisional prediction image generation unit that generates a provisional prediction image based on a representative value of the above angle mode; The above angle mode is derived from the index of the angle mode table derived using the above gradient and from the said angle mode table. A motion image encoding device characterized by generating a predicted image by correcting the above-mentioned provisional predicted image according to the intra-prediction mode.

Images